Event 31555 and 33333 on RMS
Hi We are getting event 31555 failed to generate data to be forwarded to the data warehouse. Workflow name : microsoft.systemcenter.datawarehouse.synchronization.managedentity and event 33333 with requestdataWarehousesynchmanagedentitychangelist -- batchsize =100 When We open health explorer of RMS, it is critical for 1)data warehouse managed object synchronization data reader state 2) data warehouse relationship synchronization data reader state. Steps tried: 1)In all 3 DW Run as profiles, i can see all 3 SCOM servers. 2)I changed both run as accounts DW SQL server auth account and Reporting SDK Sql server auth with blank logins n password. 3)Deleted health store files on RMS but no success ## We have 5 run as accout for each DW sql server auth and reporting sdk sql server. But we have only 3 SCOM servers. Is that normal? ~Cheers, Rohit Kochher
May 15th, 2012 5:14pm

The "SQL Auth" ones are typically not used... only the "Windows Auth" ones really are. How many do you have... it depends on your configuration; really hard to tell. I only have one, for example... since I use the same user account for both reader and writer to the DW, and I use "SYSTEM" as the default action account on the management servers, otherwise. The main thing to understand is that runas ACCOUNTS are really just the username and password. 1) You define that windows user only once. 2) you determine where those credentials are DISTRIBUTED (Distribution tab of the runas account) it should of course be distributed to all servers that will have to make use of it. 3) you ASSOCIATE the runas ACCOUNT with a runas PROFILE for a given HEALTHSERVICE and/or - even more granularly - for specific classes. Now, some of the confusion comes from the fact that some of the PROFILES have display names which say "account"... In a typical environment, you would have NOTHING associated with the following two PROFILES (remember, they are PROFILES< despite their name!): Data Warehouse SQL Server Authentication AccountReporting SDK SQL Server Authentication Account and you would have the Windows Runas Accounts that have rights to the DB, DW and SSRS associated with the following two profiles: Data Warehouse AccountData Warehouse Report Deployment Account (again they are PROFILES - despite of their names...)
Free Windows Admin Tool Kit Click here and download it now
May 15th, 2012 7:37pm

Hi Daniel, Thank you for all information. It is really helpful. I will verify our configuration, according to what you mentioned. So now i am concerned about 33333 and 31555. So how can we get them fixed?~Cheers, Rohit Kochher
May 15th, 2012 9:41pm

31555 can have various reasons, and so does 33333. One possible cause of 31555 could be security (as you were hinting, and that's why I explained the accounts/profiles stuff) as the account might not be able to READ data from OpsDB in order to then try to Synchronize it to the Datawarehouse. But there could be other reasons why this is failing (i.e. SQL issues, out of memory, all sort of things - see for example http://social.technet.microsoft.com/Forums/en-US/systemcentermonitoring/thread/9d166f9c-cc57-454c-80a6-80eef7d0945f ). It would be useful if you could post the WHOLE 31555 message. Same for 33333 - actually worse - 33333 can happen in a gazillion of situations.... some of which are not even errors, but just a "healthy" defense of the system to prevent "bad" data from corrupting the DB, i.e. http://blogs.msdn.com/b/boris_yanushpolsky/archive/2008/01/24/troubleshooting-event-id-33333-logged-by-the-data-access-layer.aspx as well as many others... just search for http://www.bing.com/search?q=opsmgr+event+id+33333 Here, again, posting the FULL text of the message would add more context.
Free Windows Admin Tool Kit Click here and download it now
May 15th, 2012 9:56pm

Hi, Would you please let us know if there were any changes before the issue occurred? On the RMS, please try restarting the following services: System Center Management service System Center Management Configuration service System Center Data Access service Regarding the Event 3333, please also check it referring to the following post: OpsMgr 2007: DataAccessLayer Event ID 33333 with "should not generate data about this managed object" http://blogs.technet.com/b/smsandmom/archive/2008/03/13/opsmgr-2007-dataaccesslayer-event-id-33333-with-should-not-generate-data-about-this-managed-object.aspx Hope this helps. Thanks. Nicholas Li TechNet Community Support
May 17th, 2012 2:04am

Hi Daniel/Nicholas Thanks for your response. I imported latest cluster and Sharepoint MP's. But SP servers have no SCOM agent yet. And no clusters in current SCOM agents. I checked the accounts permissions following Kevin's spreadsheet and looks good. I already restarted services and even clear health on RMS. ~Cheers, Rohit Kochher
Free Windows Admin Tool Kit Click here and download it now
May 17th, 2012 10:42am

And this is how Health explorer from RMS looks like ~Cheers, Rohit Kochher
May 17th, 2012 1:49pm

The error is very ugly - seems something wrong with SQL, see for example this article http://support.microsoft.com/kb/2515286 Can you try if restarting SQL (and then SCOM, after SQL is back online) solves it? Also check if you are running up-to-date and supported SQL - AND SCOM - Service packs / Cumulative Updates / Patches...
Free Windows Admin Tool Kit Click here and download it now
May 17th, 2012 5:56pm

The error is very ugly - seems something wrong with SQL, see for example this article http://support.microsoft.com/kb/2515286 Can you try if restarting SQL (and then SCOM, after SQL is back online) solves it? Also check if you are running up-to-date and supported SQL - AND SCOM - Service packs / Cumulative Updates / Patches...
May 17th, 2012 5:56pm

The error is very ugly - seems something wrong with SQL, see for example this article http://support.microsoft.com/kb/2515286 Can you try if restarting SQL (and then SCOM, after SQL is back online) solves it? Also check if you are running up-to-date and supported SQL - AND SCOM - Service packs / Cumulative Updates / Patches...
Free Windows Admin Tool Kit Click here and download it now
May 17th, 2012 6:03pm

Hi Daniel, Thanks for the response. Restarting the servers can be done on weekend :) From health explorer of RMS, every rule other then relationship sync and managed object sync state seems to work. What can be the impact for these two rules failing? also these are failing only for data reader state. Same rules are working for data writer and state reader. Again, Thanks a Ton!!~Cheers, Rohit Kochher
May 17th, 2012 6:09pm

It's VERY bad. Things will keep working for a while (hours? days?) but eventually the queues on the RMS will fill up and everything will start degrading. We can resist without syncing data for a while.... but not forever :-) The fact that some work and some fail, is because some queries work, and some get back that weird error from SQL! --> it depends on the query, most likely... doesn't seem like a permission issue, as much as SQL saying that it doesn't have some function defined, which it should have. The KB article seems to suggest this can happen and is a bug in SQL which seem to have been fixed with the latest Cumulative Update.... workaround before that was to restart SQL. Now if neither of those actions resolve... then I would probably open a support incident, as something might be severely broken in the OM DBs... Let us know how it goes after you rebooted during the weekend.
Free Windows Admin Tool Kit Click here and download it now
May 17th, 2012 6:18pm

Hi Daniel Its working now :) Thanks a Ton! Before rebooting SQL server, I bounced all SQL services on OM DB server and that fixed the issue. Its a weird issue and none of the searches on internet takes you to the KB. It would be great if someone can blog on that :) Have a great weekend. ~Cheers, Rohit Kochher
May 18th, 2012 10:06am

Hi Daniel Its working now :) Thanks a Ton! Before rebooting SQL server, I bounced all SQL services on OM DB server and that fixed the issue. Its a weird issue and none of the searches on internet takes you to the KB. It would be great if someone can blog on that :) Have a great weekend. ~Cheers, Rohit Kochher
Free Windows Admin Tool Kit Click here and download it now
May 18th, 2012 10:06am

Glad to hear it was that. FYI, the query I used to find the KB was this one http://www.bing.com/search?q=%22there+is+no+function%22+%22local-name%22 As I wrote before, those events in SCOM can have multiple causes... it is the actual inner exception that comes from SQL which is the real issue - I had never seen this specific cause of the error myself happening on a SCOM machine before yesterday, as a matter of fact... again, because it is a SQL issue. Anyway, again, glad it worked.
May 18th, 2012 11:01am

Glad to hear it was that. FYI, the query I used to find the KB was this one http://www.bing.com/search?q=%22there+is+no+function%22+%22local-name%22 As I wrote before, those events in SCOM can have multiple causes... it is the actual inner exception that comes from SQL which is the real issue - I had never seen this specific cause of the error myself happening on a SCOM machine before yesterday, as a matter of fact... again, because it is a SQL issue. Anyway, again, glad it worked.
Free Windows Admin Tool Kit Click here and download it now
May 18th, 2012 11:01am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics