Hello,
I've got 2 Management Groups (PPD & PRD). In PPD, whenever I put something into Maintenance Mode (MM), the object never comes out of maintenance mode, even 5 minutes after the ScheduledEndTime! The only way to bring it back out of MM is by manually stopping it via the GUI!
In PRD, it works fine! Considering this, my research lead me to the 'Maintenance Mode' rule, whose associated stored procs (p_ScheduledJobsEveryFiveMinutes & p_MaintenanceModeJob) appear to do the required cleanup in the tables. Their inspection however, has turned up nothing. The only thing I could imagine happening is that the rule is not running, despite the fact that there are no overrides associated with it. Is there an easy way to determine whether a rule is firing or not?
Any ideas are welcome,
Thanks,
Larry
Larry
Hi Larry, you are in the right area,
The "dbo.p_MaintenanceModeJob" stored procedure is called on by the stored procedure called "dbo.p_ScheduledJobsEveryFiveMinutue", as this states it runs every 5 minutes, so firstly, sometimes it can take upto 5 minutes to bring the object out of maintenenance mode
In case your interested the SQL query executed by the "dbo.p_ScheduledJobsEveryFiveMinutues" Stored proc is:
USE [OperationsManager]
GO
/****** Object: StoredProcedure [dbo].[p_ScheduledJobsEveryFiveMinutes] Script Date: *****/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[p_ScheduledJobsEveryFiveMinutes]
AS
BEGIN
SET NOCOUNT ON
EXEC dbo.p_MaintenanceModeJob
EXEC dbo.p_JobStatusTimeout
RETURN 0
END
Some of the common issues for objects not making it out of maintenance mode are:
1. Agent will never come out of scheduled maintenance mode if above stored procedure is not executing. Execution of above SP can be confirmed in SQL profiler.
2. Time differences between Domain conntroller and root management server (RMS).
3. There is a rule in SCOM called "Maintenance Mode" in "System Center Internal Library" Management Pack, which is targeted to "Root Management Server". This rule calls the stored procedure "p_ScheduledJobsEveryFiveMinutues" in OperationsManager DB. If this rule is not running then Agents will never come out of Maintenance mode after the scheduled time.
Let me know if this helps,
Gordon
- Proposed as answer by Vaman Salkar Saturday, March 03, 2012 10:28 AM
Hi Larry, you are in the right area,
The "dbo.p_MaintenanceModeJob" stored procedure is called on by the stored procedure called "dbo.p_ScheduledJobsEveryFiveMinutue", as this states it runs every 5 minutes, so firstly, sometimes it can take upto 5 minutes to bring the object out of maintenenance mode
In case your interested the SQL query executed by the "dbo.p_ScheduledJobsEveryFiveMinutues" Stored proc is:
USE [OperationsManager]
GO
/****** Object: StoredProcedure [dbo].[p_ScheduledJobsEveryFiveMinutes] Script Date: *****/
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE PROCEDURE [dbo].[p_ScheduledJobsEveryFiveMinutes]
AS
BEGIN
SET NOCOUNT ON
EXEC dbo.p_MaintenanceModeJob
EXEC dbo.p_JobStatusTimeout
RETURN 0
END
Some of the common issues for objects not making it out of maintenance mode are:
1. Agent will never come out of scheduled maintenance mode if above stored procedure is not executing. Execution of above SP can be confirmed in SQL profiler.
2. Time differences between Domain conntroller and root management server (RMS).
3. There is a rule in SCOM called "Maintenance Mode" in "System Center Internal Library" Management Pack, which is targeted to "Root Management Server". This rule calls the stored procedure "p_ScheduledJobsEveryFiveMinutues" in OperationsManager DB. If this rule is not running then Agents will never come out of Maintenance mode after the scheduled time.
Let me know if this helps,
Gordon
- Proposed as answer by Vaman Salkar Saturday, March 03, 2012 10:28 AM
1. I am unfortunately unable to determine if the stored proc is running or not, despite tracing for Stored Procedures\RPC:Completed, and filtering on TextData with "LIKE p_ScheduledJobsEveryFiveMinutues" doesn't turn up anything in PPD & PRD!?!? I must be doing something wrong with the filter.
2. There is no significant time difference between the domain controller and the RMS. They are in the same domain and in the same server room.
Looking forward to your response,
Larry
Is this still a problem?
Cheers
Graham
I attempted to re-run the trace this morning, this time filtering on TextData with "LIKE % p_ScheduledJobsEveryFiveMinutues% ". Happily, I am now able to see the stored procedure firing, as expected (in both environments). It was likely the missing '%' that solved the issue of not seeing the stored proc firing.
However, I am no longer able to reproduce the original problem (objects not coming out of Maintenance Mode) in PPD, as objects are now coming out of MM as expected. :P
Now that I have a better understanding of this aspect of OpsMgr, I'll be quicker on my feet next time and run the trace immediately.
Thanks,
Larry
- Marked as answer by larry.leblanc Tuesday, March 02, 2010 3:23 PM
I attempted to re-run the trace this morning, this time filtering on TextData with "LIKE % p_ScheduledJobsEveryFiveMinutues% ". Happily, I am now able to see the stored procedure firing, as expected (in both environments). It was likely the missing '%' that solved the issue of not seeing the stored proc firing.
However, I am no longer able to reproduce the original problem (objects not coming out of Maintenance Mode) in PPD, as objects are now coming out of MM as expected. :P
Now that I have a better understanding of this aspect of OpsMgr, I'll be quicker on my feet next time and run the trace immediately.
Thanks,
Larry
- Marked as answer by larry.leblanc Tuesday, March 02, 2010 3:23 PM
Hi Gordon,
Thanks for your suggestion, I have the same kind of issue. DB confirmed that dbo.p_MaintenanceModeJob is not executing on DB server., also they have informed that, "Usually these type of store procedures called by the application and execute, please check application configuration. "
Could you help here ?. Thanks for your valuable input
System Center 2012 Version 7.0.8560
Regards,
Karthik.