SCOM Command Notification - Powershell - Polling Interval Minutes
Hi, We are using command notification to execute a powershell script to upate the resolution state,owner and a custom field property of specific alerts. The Alert then basically shows up in specific views when the corresponding team can see and act on it. I see that there is a considerable delay during this process (almost upto 2-3 mins). The part of the script that retireves and updates the alert takes only about 3 secs. But the delay seems to be contributed due to the following factors. 1. Loading the Snap-in (30-40 secs) http://www.jamssupport.com/KnowledgebaseArticle50030.aspx I am trying to see, if we can get some GPO set up to do this. I can see it works locally for my ID..But when I try to set it up for the ID that runs the subscription (MSAA account) it works only when I am logged in to the server as MSAA and doesnot work when it is auto kicked off. How can I set this by default for all users,always? 2. Subscription Polling Interval. While it works okay for single alerts (max delay of 1 min if the alert arrived just a second after the poll, would be better if that can be avoided too) , it is extremely delayed when multiple alerts come in at the same time because the subscription seems to process each alert one by one per minute. so if there are 100 alerts, all coming in at the same time or couple of minutes apart, the 100th alert gets processed at 100th minute. which means we report the error after almost 2 hours. Since we want to process the alert in a case-by-case basis, we handle the alert update based on the alert-id in the script. I understand, we can process this by alert name instead of ID so it can be done in bulk. But I really want to know, if there is any work around to this like multiple threads running parallely or by modifying the polling interval (in secs) from the default 1 min. Appreciate your time. Thanks Karpakam Karpakam Balasubramanian
February 10th, 2012 2:22pm

A couple ideas. 1. If you do not like the time it takes to load the POSH snapin, you could incorporate your script into a schedule rule that runs on the RMS (note: this may not work as designed in OM12 since RMS is removed). If a workflow runs a POSH script, and given you are using the correct provider, you do not need to load the snapin, thus reducing runtime duration. 2. If you want to process more than one alert in each polling interval, do not pass AlertId into your script. Rather, run the script with specific criteria, like getting all new alerts, and then process accordingly. HTH, Jonathan Almquist - MSFT
Free Windows Admin Tool Kit Click here and download it now
February 12th, 2012 10:54am

if your still on powershell v1, then there is a way to increase the load time of powershell, take a look at the following, and run update-gac.ps1 on the machine loading the powershell snapin. http://blogs.msdn.com/b/powershell/archive/2008/09/02/speeding-up-powershell-startup-updating-update-gac-ps1.aspxScott Moss MVP (Operations Manager) President - System Center Virtual Users Group |Vice President - Atlanta Southeast Management Users Group (ATL SMUG) Please remember to click Mark as Answer on the post that helps you! my new blog om2012.wordpress.com
February 12th, 2012 3:05pm

Jon, Thanks for your response. 1. Do you mean use scheduled rules, instead of subscriptions?? We have over 30 subscriptions with different criteria. So we have to handle all the criteria in the script and just schedule it from a rule? Also we will be moving to OM12 anyways. Thanks! 2.Unfortunately, I cant do get all new alerts or any such generic cirteria because, the alerts go to different queues based on specific criteria. So I have to process them on an alert by alert basis. I do agree , I can use Alert Name instead ..that way when a bulk of exact same alerts alerts pop up at the same time(like IISapp pool recyles on several servers), I can processs them in bulk rather than individually by AlertID. So for 100 such alerts,.I dont have to wait 100 minutes. I can get them all assigned in 2/3 polls.But that is just one case. Most of our subscription criteria are based on "Generated by specific Rule/Monitor". Example, certain alerts from SQL rules/monitors are subscribed under a SQL subscription(command channel), that gets assigned to the "SQLDBAs " as the owner. So my question is, If two different alerts (meaning.. generated by two different rules/monitors) that are subscribed under the same subscription are raised at the exact same time, do they get processed only in a FIFO basis?? ( 1 minute apart) And, If two different alerts that belong to two different subscriptions are raised at the exact same time, will OpsMgr still process them minute by minute?? Thanks in Advance. Scott, Thank u. But we are using powershell v2. Karpakam Balasubramanian
Free Windows Admin Tool Kit Click here and download it now
February 13th, 2012 10:12am

Reguarding the conosle start up time: where is the powershell console being kicked off in relation to the RMS and db and AD boxes? are they over a 56k line or a highly saturated T1 line? The snap-in loads relatively fast, however it has to make a connection over what ever lan/wan connection to your RMS, then validate your credentials (requires a connection from rms to AD DC), then return back to you. So if there is a hokey connection between your posh console or RMS or RMS to AD and back, any one of those could cause latency. I've never seen it take more than a few to5 seconds to load the posh console even on my overloaded lab system. The only time it took longer was due to horrible SAN problem that had I/O jacked way high, which eventually brought opsmgr to its knees till the SAN problem was fixed. As far as your script, sounds like you should consider other options like the one's Jonathan suggested. I have not seen your script, so its really hard to fill in the blanks on your what if scenarios. Scott Moss MVP (Operations Manager) President - System Center Virtual Users Group |Vice President - Atlanta Southeast Management Users Group (ATL SMUG) Please remember to click Mark as Answer on the post that helps you! my new blog om2012.wordpress.com
February 13th, 2012 3:47pm

Hi Scott, Thanks! The powershell console is kicked off on the RMS. I believe the delay in loading the snap-in is due to the fact that PowerShell is verifying the digital signatures on the snap-in that is being loaded. As per this article (http://www.jamssupport.com/KnowledgebaseArticle50030.aspx). I actually verified that when I uncheck these check boxes (control panel->Internet Options->Advanced->Security) and load the snap-in (logged in to the RMS as me) , it takes just 2 secs (even less). But I am not sure, how to set this for all users all the time.Because, the Powershell will be launched by the MSAA account when the subscription kicks off. Since it is a user based setting, It is not taking effect when the powershell is launched programtically by SCOM (under MSAA account). It only takes effect when I am physically logged into the box as the MSAA account and uncheck the security settings. But when I logoff and the subscription runs again automatically, it once again gets back to the original setting of checking the certificates/signatures and takes around 30-40 secs.I think that is where I am stuck. The second point for the delay is ofcourse, how scom susbcription polls only once a minute and processes the alert one by one per minute.Even If I decide to use the Alert-Name in the script to process alerts in bulk (instead of the ID), it will only get around the issue when there is a bunch of exact same alerts (from the same rule) coming in at the same time (which is good). But , it is not going to work when different alerts that belong to a same subscription, come in at the exact same time. I cannot use a generic "Select All New Alerts" condition in the script, because they will have to be processed and assigned to different owners on a case-by-case basis. The script is a pretty simple one...101 (our custom resolution state - "Assigned") param([System.string]$AlertID,[System.string]$Name) add-pssnapin "Microsoft.EnterpriseManagement.OperationsManager.Client"; set-location "OperationsManagerMonitoring::"; $server = "localhost" # Connect to SCOM new-managementGroupConnection -ConnectionString:$server; set-location $server; $alert=Get-Alert -Criteria "Id = '$AlertID' and ResolutionState = '0'" $alert.set_Owner($Name) $alert.ResolutionState=101 if ($Name -eq "XXXX") { $alert.CustomField2 = "Some Value" } $alert.update("Owner Resolution State and CustomField2 auto assigned by SCOM notification engine") Thanks for your help. Karpakam Balasubramanian
Free Windows Admin Tool Kit Click here and download it now
February 14th, 2012 11:48am

Yes, I did mean scheduled rule(s) instead of subscription(s). It might be difficult to write this if you are not familiar with authoring your own modules in SCOM, but it is very possible to write something sophisticated enough to re-use as a template, and just populate your criteria on a rule-by-rule basis. As far as using alert id as criteria, I haven't run into a situation where this is required. The same criteria can be put into the script that processes the alerts without having some adverse behavior in your process. Passing alert id only means that you will only process one alert at a time - there is no added benefit from a logic or processing perspective by doing this, in my opinion. By using subscriptions and command channel, I believe it follows the FIFO rules - and the command channel only processes one alert at a time if you pass in alert id.HTH, Jonathan Almquist - MSFT
February 28th, 2012 12:14pm

Jon , Thanks! My Subscriptions are all based on "Generated by specific Rule/Monitor". When an alert that belongs to a specific subscription is generated, it follows a particular command channel , that basically modifies the properties of that alert so that it gets assigned to a specifc owner and shows up in a specific view for action. Sorry...But I dont see how I can modify the properties of such specific alerts without passing AlertID/AlertName to the script. While I do agree that passing alertID strictly restrics the script to process only one alert at a time, I sure can pass Alert Name that could help me when there are tons of exact same alerts (generated by same rule from different source) to be processed in bulk. But I still have the same problem when multiple alerts from different rules rush in at the same time that has to follow the same command channel. It goes one by one and the 10th alert is picked up at the 10th minute. Thanks for ur time. I think once I fix the CRL issue, the snap in load time which is the major contributor for the delay (30-35 secs)should disappear and passing the Alert name instead of ID can help reduce time delay on certain cases if not all.Karpakam Balasubramanian
Free Windows Admin Tool Kit Click here and download it now
March 2nd, 2012 10:35am

Hi, as for delays of loading snap-ins.....Yes it's very long-run process...But you can avoid loading every time these snap-ins...like this.. you have to create application that can be run only in single instance. When app runs first time it loads necessary Snap-ins and then when new alert arise ..using a command channel you pass necessary params to your app...and any Powershell scripts run inside app). Using this method you can process far more alerts without delay. In my environment more difficult powershell script, than your example, process 100 alerts during some seconds (in my script I load SCOM and SQL snapins, and if I load these snapins each time it will take about 1 minut).
August 23rd, 2012 7:27am

Hi, as for delays of loading snap-ins.....Yes it's very long-run process...But you can avoid loading every time these snap-ins...like this.. you have to create application that can be run only in single instance. When app runs first time it loads necessary Snap-ins and then when new alert arise ..using a command channel you pass necessary params to your app...and any Powershell scripts run inside app). Using this method you can process far more alerts without delay. In my environment more difficult powershell script, than your example, process 100 alerts during some seconds (in my script I load SCOM and SQL snapins, and if I load these snapins each time it will take about 1 minut). As for using schedule ...it depends on how often you have to run script..if you need to run it every five minute or more often you will overload you management servers..it's not a good choice...specially when you have to load many heavy snapins in script.
Free Windows Admin Tool Kit Click here and download it now
August 23rd, 2012 7:30am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics