Been racking my brain for quite a while trying to devise a scripted solution to my problem. We've got a particular application that runs over about 10 servers, and each server has a plethora of services running on them that may or may not have issues starting every night. Each night, we have a job that runs that restarts all the services, but once in a while a service on one of the servers will get hung in a "Starting" state. I'd like to script a solution that checks a list of services running on a list of servers, and if one or more are found to be not started, then I'd like to kill that process, start the service again, and send an email to myself.

I have gotten bits and pieces of the script to work, but not all together as a cohesive unit. So I'd rather just start over from a more basic script and see if I can learn how to engineer this thing the right way (with the help of the community). 

I'm starting with a Get-Content command that gets my list of servers via txt file into a $Servers array variable. 

The next leg is part of what I'm not understanding. Rather than creating another array variable of a static list of service names from a text file, I'd like to just say "any service that starts with this string". Since all of our application's services start with the same two letters, I'd like to only focus on those services, and not the rest of the services on the machines (as there are several other services unrelated to our application that aren't running, and don't need to be). Is something like this possible? I'm thinking along the lines of:

$Services = Get-Service -ComputerName $Servers -Name "AB*"

But I don't think it's working in practice how I'm picturing it in my head. Any scripting experts out there that can help me out? I just want a simple solution that will:

1) Loop through an array of services within an array of servers

2) If a service is found to be in a status other than "running", kill the process and start the service

3) E-mail myself an alert telling me which service on which server was restarted

I'm pretty confident I've already got pieces for 2 and 3, it's the looping logic that is confusing me. Thanks in advance for taking the time to read this and possibly assist me.

-Chris 

The "simple" solution you seek probably involves hiring a consultant to get eyes on the problem. (It's going to be difficult to design a solution from afar because we don't have access to your network or see your screen.)

Of course, the greater question is why you're having to restart services all the time. Seems it would be better to fix the underlying problem rather than band-aid with service restarts.

Thanks for the response Bill. Unfortunately, the application is not ours (developed by a 3rd party), but I manage the Windows boxes that it runs on, as well as making sure everything is kosher. This 3rd party recommends a restart of their services every night (I know, I know..) to keep it "running smoothly". Once in a blue moon, one of the 100-some services will get hung in a Starting state, and I won't know it until a station is screaming bloody murder. I suppose the "simple" solution here is to just log into each server every morning and see if any services are "starting", but that wouldn't be very SysAdmin-y of me. I suppose this may be even better:

get-service -ComputerName server1, server2, server3, etc -Name "WO*" | Select Name, MachineName, Status

And just look for any status of "Starting" in the Status column.

Not very elegant, I know, but it seems that scripting something like this to happen automatically each morning is quite the undertaking. I already have a script that will monitor ONE particular service, and if it's found to be hung, it will kill the process, start the service, and then email me. Seems to be trickier than I thought to apply that to multiple servers and services. :(

Thanks

If you have a working solution for one service, then run separate instances of the script for each service and schedule each one.

It's beyond the scope of this forum to design customized scripting solutions for specific scenarios; sorry.

Hello,

See http://blogs.technet.com/b/heyscriptingguy/archive/2011/12/12/use-a-powershell-script-to-start-non-running-automatic-services.aspx for a start.

Karl

Dear 

This  can be achieved by adding a small line of script in the recovery tab of the service properties of all the servers services which you wanted.

First Failure -> Run the Script

taskkill <servicename>

net start servicename.

Thanks

Syed Abdul Kadar M.

Hello,

See http://blogs.technet.com/b/heyscriptingguy/archive/2011/12/12/use-a-powershell-script-to-start-non-running-automatic-services.aspx for a start.

Dear 

This  can be achieved by adding a small line of script in the recovery tab of the service properties of all the servers services which you wanted.

First Failure -> Run the Script

taskkill <servicename>

net start servicename.

Thanks

Syed Abdul Kadar M.

Hello,

See http://blogs.technet.com/b/heyscriptingguy/archive/2011/12/12/use-a-powershell-script-to-start-non-running-automatic-services.aspx for a start.

Alright so I think I'm very close. I think the final thing that's hanging me up is a purely cosmetic issue, regarding how the service names are being written in the e-mail message. Here's my script so far (with actual names replaced):

-------------------------------------------------------------

----------------------------------------------------------

My issue is that when I get the emails, the $Svc variable in the Subject and Body does not appear as I had expected. The result is something like \\srv1\root\cimv2:Win32_Service.Name="ABCservice". I have tried $Svc.DisplayName and $Svc.Name with essentially the same result. I don't think I'm understanding how the WMIobject variable is being stored, and I'm not calling the correct field to pass through in the email. Other than that, I think my script is doing what I need it to do, anyone mind taking a look? Although, it will be difficult to test for real until I get it scheduled as a task into production to run every morning, and will need to wait until a service actually gets hung in Starting.

Thanks all

You should be able to use

$($Svc.Name)

in your string. See help about_Operators (subexpression operator).

Is this what you are trying to do?

$Servers='srv1','srv2','srv3','srv4'
$WmiObject=@{
	Class='Win32_Service'
	Filter='Name LIKE "ABC%" and State!="Running"'
}
$mailprops=@{
	To = 'Chris@abc.com'
	From = ''
	SmtpServer = 'mail.abc.com'
}

foreach ($Server in $Servers) {
	foreach ($Svc in (Get-WmiObject @WmiObject -ComputerName $Server)) {
                $svcname=$svc.Name
		if($Svc.State='Stopped'){
			Start-Service -Name $svcame
			Send-MailMessage  @mailprops -Subject "$svcname is Stopped on $Server" -Body "The $svcname service was found Stopped and has been started on $Server"
		}else{
			Stop-Process -Id $Svc.processID -Force
			Start-Service -Name $Svcname
			Send-MailMessage  @mailprops -Subject "$svcname is Hung on $Server" -Body "The $svcname service was found in the Starting state and has been killed and started on $Server"
		}
	}
}

If you format it so it is readable you will get more resp

Now to reduce complexity you can do the following.

1. Set service to auto-restart if stopped. 
2. Scrip now only looks for services that are stuck in "starting" mode.

$Servers = 'srv1','srv2','srv3','srv4'
$WmiObject=@{
	Class='Win32_Service'
	Filter='Name LIKE "ABC%" and State="Starting"'
}
$mailprops=@{
	To = 'Chris@abc.com'
	From = ''
	SmtpServer = 'mail.abc.com'
}

foreach ($Server in $Servers) {
	foreach ($Svc in (Get-WmiObject @WmiObject -ComputerName $Server)){
		Stop-Process -Id $Svc.processID -Force
		Start-Service -Name $Svc.Name
		Send-MailMessage  @mailprops -Subject "$Svc is Hung on $Server" -Body "The $Svc service was found in the Starting state and has been killed and started on $Server"
	}
}

Much simpler.