SCOM email notifications based on a value in alert description

We have the below requirement. We are at SCOM 2012 R2 UR6.  

.      System Center will check twice daily and be configured to notify indicated staff regarding servers that have not been rebooted as such:

  1.       At 20 days, notify ITS to reboot (manually run the script)
  2.       At 23 days, notify ITS and RITS to reboot
  3.       At 25 days, notify ITS, RITS and DMO to reboot
  4.       At 28 days, notify ITS, RITS, and DMO to reboot
  5.       At 30 days, force reboot for the following Wednesday, permitting at least 2 days prior notice.  Send notice/countdown every day til reboot occurs. [Final notice]

I am in process of determining whats the best solution for above. I created a unit performance monitor (double threshold) that reads the perf counter "System Up time". I set this to turn in warning state when the value is 20 or more. When value is 23 it turns critical.  Since there are only two values thresholds I can specify in dbl threshold monitor, I am thinking I will need to create a copy of the above monitor - again double threshold and then make this turn warning when it is more than 25, and critical when 28 or more.  I can use the monitor and state to setup subscription, but there will need to be different subscriptions sicne subscribers are different. Also there are 60 different countries for which subscribers are different. I

Can you suggest whats the best possible design for email notification for above. Is it possible to use the perf counter value which is alert description - to set notification subscription. Do I need 2/3 different monitor.  Please suggest how you would design this. Thanks a lot in advance !

September 1st, 2015 12:33pm

Personally, I'd use System Center Orchestrator for this. You can build in a lot more logic than within SCOM. If that isn't an option, I'd probably do 5 separate rules with the logic in each rule.

1. Greater than or equal to 20 days but less than 23 days --> Alert 1 --> Subscription 1

2. Greater than or equal to 23 days but less than 25 days --> Alert 2 --> Subscription 2

Etc.

If you have some naming convention or identifier for each country then you can use dynamic groups to group the servers together into groups and leverage this with the rules above for notifications to specific teams.

Regards

Graham

Free Windows Admin Tool Kit Click here and download it now
September 1st, 2015 4:06pm

 Yes we have dynamically populated groups for each country, but that would still mean creating one subscription per country. But thats how we have setup our notification for the countries.

Regarding the logic - I do have option of using SCORCH but have never used it before. What level of complexity would it be for a SCORCH newbie. I will have to choose either SCORCH or rules as you suggested above.     An additional question - the notification is showing the System up time perf counter value in seconds in alert description, is there some way to convert that to days ? Also is it possible to display this value in email subject line ?  Thanks a lot.

September 1st, 2015 5:02pm

The advantage of Orchestrator (at least in my opinion - others may disagree) is that you can script the logic as you want it to work rather than having to fit in with the way SCOM rules or monitors are defined. If you don't need a SCOM alert but really just want the email (and possibly a log file as an audit trail) then it makes this option even more appealing. 

So you could have a single PowerShell script that pulls back the value you want, converts seconds to days and then runs through your business logic. If you wanted to make it a little more sophisticated you could even create a simple database that acts as a look up table for server naming convention \ country \ who to email. Then one function that just pulls back the required data rather than 60 subscriptions.

One challenge with Orchestrator is if the script was to run locally on each box then you'd need to give the correct rights to the Orchestrator service account. However, an option would be to:

- have a SCOM Collection rule that runs once per day to pull back this value in seconds.

- have the Orchestrator script pull back that value from SCOM. E.g. most recent data point for each server. This way the Orchestrator service account just needs to be able to read data from SCOM.

It would take a little effort but generally I feel the Orchestrator option would be easier to implement and manage.

Cheers

Graham

Free Windows Admin Tool Kit Click here and download it now
September 1st, 2015 5:20pm

Graham, I am using creating the individual rules approach. I had a query - this rule will collect perf counter System Up Time but a perf counter collection rule cannot gerenate an alert. We require SCOM alert to be generated and email notification sent. Can I make a rule to generate alert based on perf. counter values thresholds.  Thanks a lot once again !
September 2nd, 2015 12:27pm

I am using unit performance monitor (double threshold) for the monitoring. I have 2 monitos - first remains healthy below 20 days for perf counter system up time, turns into warning for above 20 and critical above 23. The second monitor remains health below 25, warning above 25 and critical above 28. I am using monitor name and severity state to email notify, This probably isnt the best solution and is confusing. I am still implementing it. ANyone have better ideas ? SCORCH will take some time, so I am not using that.  Also is there a way to setup notification subscription based on a value in alert description.

Thanks much

Free Windows Admin Tool Kit Click here and download it now
September 4th, 2015 3:35pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics