Data Access Service failure - Production Data migration to Test Environment (Network Steve Forum)

Data Access Service failure - Production Data migration to Test Environment

In testing for upgrading my SCSM environment from 2012 SP1 to 2012 R2, I've hit a roadblock migrating production data to the test instance. The Data Access Service tries to launch and fails withing seconds.

What I've done so far:

I built a separate SCSM instance for a Dry Run upgrade. Two server configuration with primary management server hosting its own database (SQL Server 2008 R2 Developer Edition) and the same for the data warehouse management server.

Installation and patching to CU6 went smoothly. Made some test tickets and connectors. The in-place upgrade when smoothly for the most part. I only ran into trouble when trying to restart the data warehouse jobs but that was relatively straight forward to fix once I found the right article here.

With the process and documentation in place, my next phase is to restore the production environment into the test environment and perform an dress rehearsal. I followed the steps listed in this article Setting Up a Service Manager 2010 Lab Environment with Production Data.

I only deviated from the instructions in one place. Instead of disabling the connectors before backing up the system, I exported the ServiceManager.LinkingFramework.Configuration management pack, edited all the Exchange Connectors to Enabled = false, and used T-SQL to override the management pack XML in the database directly. Both the MPXML and MPRunTimeXML fields were updated.

The trouble that I've run into is that I cannot get the test instance's Data Access Service to start.These errors are recurring:

The encryption keys in the registry were either not valid or not present.
The System Center Data Access service will not start.

Followed by:

An exception was thrown while initializing the service container.
Exception message: Initialize
Full exception: Feature of type 'Microsoft.EnterpriseManagement.ServiceDataLayer.IManagementGroupPropertiesFeature, Microsoft.EnterpriseManagement.DataAccessService.Core, Version=7.0.5000.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' cannot be added to the container.

Lastly:

The System Center Data Access service failed due to an unhandled exception.
The service will attempt to restart.
Exception:

Microsoft.EnterpriseManagement.ConfigurationReaderException: Feature of type 'Microsoft.EnterpriseManagement.ServiceDataLayer.IManagementGroupPropertiesFeature, Microsoft.EnterpriseManagement.DataAccessService.Core, Version=7.0.5000.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35' cannot be added to the container. ---> Microsoft.EnterpriseManagement.Common.SdkServiceNotInitializedException: The Data Access service has not yet initialized. Please try again.
   at Microsoft.EnterpriseManagement.ServiceDataLayer.ManagementGroupPropertiesFeatureImplementation.InitializeManagementGroupProperties()
   at Microsoft.EnterpriseManagement.SingletonLifetimeManager`1.GetComponent[K]()
   at Microsoft.EnterpriseManagement.LifetimeManagerWrapper`2.GetComponent[K]()
   at Microsoft.EnterpriseManagement.FeatureContainer.GetFeatureInternal[T](Type type, String featureName)
   at Microsoft.EnterpriseManagement.FeatureContainer.AddFeatureInternal[T,V](ActivationContext`1 context, String featureName)
   --- End of inner exception stack trace ---
   at Microsoft.EnterpriseManagement.ConfigurationReaderHelper.ReadFeatures(XPathNavigator navi, IContainer container)
   at Microsoft.EnterpriseManagement.ConfigurationReaderHelper.Process()
   at Microsoft.EnterpriseManagement.ServiceDataLayer.DispatcherService.Initialize(InProcEnterpriseManagementConnectionSettings configuration)
   at Microsoft.EnterpriseManagement.ServiceDataLayer.DispatcherService.InitializeRunner(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart(Object obj)

As I examine this problem, I am inclined to believe that it is the management pack overrides that are the issue. I don't really want to disable all the connectors in the production environment as there are over 50.

Any ideas would be helpful.

March 27th, 2015 4:26pm

Hi,

Here is a similar thread for your reference:

https://social.technet.microsoft.com/Forums/systemcenter/en-US/445ef4bc-d9ff-4d98-b30f-d6ca2d05a789/scsm-2012-data-access-service-dw-wont-start?forum=dwreportingdashboards

The suggestion is to uninstall the Data Warehouse from the DW managment server and reinstall it.

You may go through the thread and try all suggestions in it.

Regards,

Yan Li

Free Windows Admin Tool Kit Click here and download it now

March 30th, 2015 2:52am

Okay, I've come at this from several different angles and I'm still getting nowhere fast. I looked through the thread suggested by Yan and I didn't see much pertaining to this particular problem. 1.) This wasn't a Data Warehouse failure 2.) This is a production restore into a (previously) functioning Service Manager instance.

What I've done to this point:

First time, I created a test installation and performed brought up the test environment up to the same version as my production environment (SP1 CU6). Grabbed a snapshot of the VM. Performed the steps to upgrade the test environment to R2. Upgrade pretty much when off without a hitch. For the restore from production, i followed instructions but deviated and tried to short-cut disabling the connectors. The data access service refused to launch.

This time around, I followed the instructions exactly. Disabled all the connectors. Disabled the services on all the management servers. Then had our DBA do a full back up of our production database. Brought up everything. Re-enabled the connectors. No issues reported from the end users.

Now, to the test environment, which at this point should be identical to prod, but with disable connectors. I tried to restart the data access service and it crashed again with the same errors. I don't understand how it could fail on encryption keys not existing since it was working before the production restore. I restored the keys anyways. Nothing changed. Still fails.

The next step was uninstalling Service Manager from the test management server and doing a full re-install and pointing to the test environment database. The install fails and this is what I find in the Setup Log:

Calling custom action CAManaged!Microsoft.MOMv3.Setup.MOMv3ManagedCAs.WaitForSDKServiceStart
WaitForSDKServiceStart:Entering
WaitForSDKServiceStart:Sleeping for SDK to start 0 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 10 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 20 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 30 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 40 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 50 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 60 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 70 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 80 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 90 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 100 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 110 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 120 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 130 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 140 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 150 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 160 seconds
WaitForSDKServiceStart:Sleeping for SDK to start 170 seconds
WaitForSDKServiceStart: SDK Service connect error, after 3 mins, fail this function
MSI (s) (3C:34) [14:47:20:391]: NOTE: custom action _WaitForSDKServiceStart unexpectedly closed the hInstall handle (type MSIHANDLE) provided to it. The custom action should be fixed to not close that handle.
CustomAction _WaitForSDKServiceStart returned actual error code 1603 (note this may not be 100% accurate if translation happened inside sandbox)

The Application log has these errors:

Product: Microsoft System Center 2012 - Service Manager -- The installer has encountered an unexpected error installing this package. This may indicate a problem with this package. The error code is 25211. The arguments are: -2147024809, The parameter is incorrect.,

The Open Procedure for service "MOMConnector" in DLL "C:\Program Files\Microsoft System Center 2012\Service Manager\MOMConnectorPerformance.dll" failed. Performance data for this service will not be available. The first four bytes (DWORD) of the Data section contains the error code.

The Open Procedure for service "HealthService" in DLL "C:\Program Files\Microsoft System Center 2012\Service Manager\HealthServicePerformance.dll" failed. Performance data for this service will not be available. The first four bytes (DWORD) of the Data section contains the error code.

I've been running the install with an admin account and, for good measure, I had someone with an Enterprise Admin (God) account run the install and I still get failures. I've had the service try to run as both Local Service and the production service account.

Last ditch effort was to restore the production encryption keys to the test server and rerunning the install. No luck there either, same result.

Edited by Foothill1 10 hours 42 minutes ago

March 31st, 2015 4:32pm

What I've done to this point:

The Application log has these errors:

Last ditch effort was to restore the production encryption keys to the test server and rerunning the install. No luck there either, same result.

Edited by Foothill1 Tuesday, March 31, 2015 8:32 PM

Free Windows Admin Tool Kit Click here and download it now

March 31st, 2015 8:30pm

What I've done to this point:

The Application log has these errors:

Last ditch effort was to restore the production encryption keys to the test server and rerunning the install. No luck there either, same result.

Edited by Foothill1 Tuesday, March 31, 2015 8:32 PM

March 31st, 2015 8:30pm

What I've done to this point:

The Application log has these errors:

Last ditch effort was to restore the production encryption keys to the test server and rerunning the install. No luck there either, same result.

Edited by Foothill1 Tuesday, March 31, 2015 8:32 PM

Free Windows Admin Tool Kit Click here and download it now

March 31st, 2015 8:30pm

What I've done to this point:

The Application log has these errors:

Last ditch effort was to restore the production encryption keys to the test server and rerunning the install. No luck there either, same result.

Edited by Foothill1 Tuesday, March 31, 2015 8:32 PM

March 31st, 2015 8:30pm

This topic is archived. No further replies will be accepted.