Connector Status not updating

As a result of a dramatic failure of communication the SCSM workflow AD account was deleted and recreated, after some muching about everything in SCSM is now working normally again except the config manager and AD connectors are not updating their status.

I initially thought they were not running at all but I have confirmed that they are in fact synchronizing data from their respective sources and its just the display of the status that is not updating.  I tried deleting them and recreating them and now they display no status i.e. the start time, finish time, status, and Percentage columns are all blank.

I have done a bit of research and believe that the problem is that the relationships between the Connectors and their respective SyncStatus objects is missing/broken, and for some reason creating new Connectors either does not establish a corresponding SyncStatus object or does but does not relate it to the connector.

I have a couple of reasons for this hypothesis first is when I run this powershell*:

$connectorDispName="Your Connector Name Here"
#Following will return in instance of Microsoft.EnterpriseManagement.ServiceManager.Sdk.Connectors.ADConnector
$myConnector=Get-SCSMConnector -DisplayName $connectorDispName
#Get the EMO object for relationship lookup
$emoConnObj=$myConnector.ConnectorObject
#Relationship id can be retrieved with following:
#Get-SCRelationship -Name 'Microsoft.SystemCenter.LinkingFramework.DataSourceHostSyncStatus'|select id
$relationshipId='1548950d-6cea-d9c1-11ec-53701fbcbbec'
#Find the matching relationship object (with the above relationshipId)
$relationshipObject=Get-SCRelationshipInstance -sourceInstance $emoConnObj|?{$_.RelationshipId -eq $relationshipId}
#$relationshipObject.TargetObject contains an EMO.
#Grab the id and use Get-SCClassInstance to return a type of Microsoft.SystemCenter.LinkingFramework.SyncStatus
$syncStatusObj=Get-SCClassInstance -Id $relationshipObject.TargetObject.Id

The following error is returned:

Get-SCClassInstance : Cannot validate argument on Parameter 'Id'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again. At D:\tools\scripts\ConnectorSyncStatus.ps1:14 char:40 + $syncStatusObj=Get-SCClassInstance -Id $relationshipObject.TargetObject.Id + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : InvalidData: (:) [Get-SCClassInstance], ParameterBindingValidationException + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.SystemCenter.Core.Commands.GetSCClassInstanceCommand

Possibly the Powershell is incorrect but this result implies to me that the Get-SCRelationshipInstance call is returning a null value for the syncstatus object which is being fed into the Get-SCClassinstance call.

My second reason for this hypothesis is the following SQL results:

select RelationshipTypeId from dbo.RelationshipType
	where dbo.relationshiptype.RelationshipTypeName = 'Microsoft.SystemCenter.LinkingFramework.DataSourceHostSyncStatus';

1548950D-6CEA-D9C1-11EC-53701FBCBBEC

select * from dbo.relationship
	where RelationshipTypeId = '1548950D-6CEA-D9C1-11EC-53701FBCBBEC';

Returns no results.

This implies to me that there are no instances of the relationship linking a Connector to its corresponding SyncStatus object.

I am not an SQL or Powershell expert but I know enough to be dangerous, so does my reasoning seem correct? And if so how could I fix this?

Thanks In advance

* this Powershell is not of my own devising, I cribbed it from Steve IAnson in this question: Query Connector Status

November 25th, 2014 2:03am

you should have specific accounts for each connector. Try recreating both connectors with each their own account and see if the issue persist.
Free Windows Admin Tool Kit Click here and download it now
November 25th, 2014 7:18am

Thanks for the reply,

I disabled the existing AD and ConfigMgr connectors and I have created two new AD accounts and used them to provide the credentials for newly created AD and CM connectors.  This has made no difference.  The new connectors reported that they were successfully created but they have no status, i.e. start time, finish time, status, and percentage are all blank.  When first created I'm pretty sure a connector should at least have a status of 'Never Run'.

If I select either connector and click synchronize now in the tasks panel I get the 'synchronization request submitted box pop up, but the all four fields of the status remain stubbornly blank.  However if I make a change in either my AD or Config Manager, I can see that change reflected in the corresponding CI in Service Manager so I know the synchronization is actually occurring, it just the reporting of the SyncStatus that is not.  

Previously, before it was deleted and recreated, the workflow account was used for both connectors and the status reported fine and the connectors both worked.  So having a specific account for each connector may be good practice but it seems it is not a technical requirement.

November 25th, 2014 10:18pm

Is there any entries in the event log? It is called Operations Manager (on the primary management server), source is Data Connectors.
Free Windows Admin Tool Kit Click here and download it now
November 26th, 2014 8:57am

Yes there is every 1 minute there is a squence of entires as follows:

Source: Lfx Service

Event: ID 3334

Data:

Error setting status for property &. Error:The relationship source specified in the discovery data item is not valid. 
Relationship source ID: ae5465d3-2565-c7d3-0de2-acc994c1e928 
Rule ID: eb496966-4fb7-45bd-bdda-b689af383733 
Instance: 

<?xml version="1.0" encoding="utf-16"?>
<RelationshipInstance TypeId="{1548950d-6cea-d9c1-11ec-53701fbcbbec}" SourceTypeId="{71f6cfcd-99b3-3a07-471d-bb9c4bf5ba76}" TargetTypeId="{2d4afd51-d2ff-92c6-266f-2b6060000dae}">
<Settings />
 <SourceRole>
  <Settings>
   <Setting>
    <Name>29906075-e9fc-7bc4-487b-f964f91d6532</Name>
    <Value>1e6592a2-96c3-4b99-a50a-c04ee2a793e6</Value>
   </Setting>
  </Settings>
 </SourceRole>
 <TargetRole>
  <Settings>
   <Setting>
    <Name>29906075-e9fc-7bc4-487b-f964f91d6532</Name>
    <Value>1e6592a2-96c3-4b99-a50a-c04ee2a793e6</Value>
   </Setting>
  </Settings>
 </TargetRole>
</RelationshipInstance>

Status data:ConnectorId:ec4a037e-e0f0-47a2-b7e7-7932eba2a0ac;
BaseManagementEntityId:e22fffb2-9b85-de47-fe3d-875914b21797;
LastRunStartTime:25/11/2014 2:00:44 AM;
LastRunFinishTime:25/11/2014 2:02:57 AM;
NextSyncTime:27/11/2014 2:00:00 AM;
Status:FinishedSuccess;
SyncPercent:100;
MaxValue:100;
MinValue:0;.

Source:OpsMgr SDK Service

Event ID: 26319

Data:

An exception was thrown while processing ProcessDiscoveryData for session ID uuid:e47c3453-2524-44af-a9d9-2475e3a204a6;id=3.
Exception message: The relationship source specified in the discovery data item is not valid.
Relationship source ID: ae5465d3-2565-c7d3-0de2-acc994c1e928
Rule ID: eb496966-4fb7-45bd-bdda-b689af383733
Instance:
<?xml version="1.0" encoding="utf-16"?>
<RelationshipInstance TypeId="{1548950d-6cea-d9c1-11ec-53701fbcbbec}" SourceTypeId="{71f6cfcd-99b3-3a07-471d-bb9c4bf5ba76}" TargetTypeId="{2d4afd51-d2ff-92c6-266f-2b6060000dae}">
 <Settings />
 <SourceRole>
  <Settings>
   <Setting>
    <Name>29906075-e9fc-7bc4-487b-f964f91d6532</Name>
    <Value>1e6592a2-96c3-4b99-a50a-c04ee2a793e6</Value>
   </Setting>
  </Settings>
 </SourceRole>
 <TargetRole>
  <Settings>
   <Setting>
    <Name>29906075-e9fc-7bc4-487b-f964f91d6532</Name>
    <Value>1e6592a2-96c3-4b99-a50a-c04ee2a793e6</Value>
   </Setting>
  </Settings>
 </TargetRole>
 </RelationshipInstance>

Full Exception: Microsoft.EnterpriseManagement.Common.DiscoveryDataInvalidRelationshipSourceException: The relationship source specified in the discovery data item is not valid.
Relationship source ID: ae5465d3-2565-c7d3-0de2-acc994c1e928
Rule ID: eb496966-4fb7-45bd-bdda-b689af383733
Instance:
<?xml version="1.0" encoding="utf-16"?><RelationshipInstance TypeId="{1548950d-6cea-d9c1-11ec-53701fbcbbec}" SourceTypeId="{71f6cfcd-99b3-3a07-471d-bb9c4bf5ba76}" TargetTypeId="{2d4afd51-d2ff-92c6-266f-2b6060000dae}">
 <Settings />
 <SourceRole>
  <Settings>
   <Setting>
    <Name>29906075-e9fc-7bc4-487b-f964f91d6532</Name>
    <Value>1e6592a2-96c3-4b99-a50a-c04ee2a793e6</Value>
   </Setting>
  </Settings>
 </SourceRole>
 <TargetRole>
  <Settings>
   <Setting>
    <Name>29906075-e9fc-7bc4-487b-f964f91d6532</Name>
    <Value>1e6592a2-96c3-4b99-a50a-c04ee2a793e6</Value>
   </Setting>
  </Settings>
 </TargetRole>
</RelationshipInstance>
   at Microsoft.EnterpriseManagement.Mom.DiscoveryDatabaseAccess.ManagementStoreWriter.BulkUpsertRelationshipInstances(DiscoveryDatabaseApi dbApi, DiscoveryDataInstance discoveryDataInstance, Boolean isCalledByWorkflow, TypeSpaceData typeSpaceData)
   at Microsoft.EnterpriseManagement.Mom.DiscoveryDatabaseAccess.ManagementStoreWriter.BulkAddUpdate(DiscoveryDataInstance discoveryDataInstance, Boolean isCalledByWorkflow, TypeSpaceData typeSpaceData, DiscoveryDatabaseApi dbApi)
   at Microsoft.EnterpriseManagement.Mom.DiscoveryDatabaseAccess.ManagementStoreWriter.Write(DiscoveryDataInstance discoveryDataInstance, Boolean isCalledByWorkflow, TypeSpaceData typeSpaceData, DiscoveryDatabaseApi dbApi)
   at Microsoft.EnterpriseManagement.Mom.DiscoveryDatabaseAccess.DiscoveryPackageProcessor.ProcessWithNoRetryUnauthorized(DiscoveryDatabaseApi dbApi, Boolean useProcessContext)
   at Microsoft.EnterpriseManagement.ServiceDataLayer.DiscoveryDataManager.DiscoveryPackageIncrementalProcessingHandler.ProcessIncrementalDiscoveryData(DatabaseConnection databaseConnection)
   at Microsoft.EnterpriseManagement.ServiceDataLayer.DiscoveryDataManager.DiscoveryPackageIncrementalProcessingHandler.Process()
   at Microsoft.EnterpriseManagement.Mom.DiscoveryDatabaseAccess.DiscoveryPackageProcessor.ProcessWithRetry(HandleProcessing handleProcessing, RetryPolicy retryPolicy)
   at Microsoft.EnterpriseManagement.ServiceDataLayer.DiscoveryDataManager.ProcessDiscoveryDataWithRetry(DatabaseConnection dbconnection, Guid discoverySourceId, IList`1 sdkEntityInstances, IDictionary`2 streams, IContext context)
   at Microsoft.EnterpriseManagement.ServiceDataLayer.ConnectorFrameworkConfigurationService.ProcessDiscoveryData(Guid discoverySourceId, IList`1 entityInstances, IDictionary`2 streams, ObjectChangelist`1 extensions)

Source:DataAccessLayer

Event ID: 33333

Data:

Data Access Layer rejected retry on SqlError:
 Request: ProcessRelationshipStagingForUpsert -- (TransactionId=732419), (DiscoverySourceId=eb496966-4fb7-45bd-bdda-b689af383733), (TimeGenerated=26/11/2014 9:31:53 PM), (PerformHealthServiceCheck=False), (HealthServiceId=), (RETURN_VALUE=1)
 Class: 16
 Number: 777980002
 Message: The specified relationship doesn't have a valid source.

November 26th, 2014 10:09pm

Guess that doesn't make it any clearer compared to what you have already hypothesized.

Can you provide some information on how you have configured the AD-connector?

Free Windows Admin Tool Kit Click here and download it now
November 27th, 2014 8:10am

AD Connector is pretty vanilla:

Its not just the AD connector though.  The Config Manager connector has the same fault.

December 1st, 2014 4:20am

You shouldn't sync the entire AD, but just what you need. Ex. OU=Users. Also take a look at this.

I doubt that will actually help with the problem at hand, but might aswell do it right.

Free Windows Admin Tool Kit Click here and download it now
December 1st, 2014 8:20am

Just for clarification, there is nothing wrong with syncing the whole AD. you'll end up with some users you don't need, and the connector will run longer, but more data isn't really a problem here, especially since they changed the connector to run daily in 2012. unless you have lots of objects (i.e. 100,000+) there are much bigger performance hurdles to worry about. 

The specific error you are running into is in regards to a relationship definition being invalid, so the obvious question is: have you deleted or added any relationship definitions? is the ServiceManager.LinkingFramework.Library MP that defines this relationship still in the management group? 

December 1st, 2014 4:28pm

I haven't knowingly deleted any relationship definitions.  I did however have some orphaned connectors that I couldn't remove through the console and I forcefully deleted them using this Powershell command:

Get-SCSMObject Class (Get-SCSMClass Name Microsoft.SystemCenter.Connector) | ?{$_.DisplayName eq Insert Connector Name Here} | Remove-SCSMObject -Force -Confirm
I can confirm that the servicemanager.linkingframework.library MP is still listed in the console, and as its sealed I don't believe I could have changed anything in there.  I note that there a related unsealed MP, servicemanager.linkingframework.configuration, and when I've exported it out and taken a look at the XML I've found that it seems to still contain connector definitions for the connectors that I have deleted either through the console or via powershell.  I'm tempted to replace the servicemanager.linkingframework.configuration I have with an empty one to see what happens, would that be an avenue worth investigating further?

Free Windows Admin Tool Kit Click here and download it now
December 2nd, 2014 1:04am

did you get any where with this because i have the same issue. all my Cireson add in connectors show the status but non of my out of the box connectoers show any status.

i'm thinking of going to R2 to try and fix but going to R2 for us is a huge step because of how custom our datawarehouse is.

January 6th, 2015 8:57pm

Did you try going with the empty MP?  I have the same exact issue going on, and i also had to delete some Connectors with PowerShell.  Now my AD and SCCM Connectors are in the same faulted state.  Im just wondering if the empty MP fixes the issue, i have no issue recreating the connectors, thanks.
Free Windows Admin Tool Kit Click here and download it now
February 27th, 2015 7:39pm

I eventually logged a premium support call with Microsoft and the cause was identified as a number of orphaned connectors that had been configured and then later deleted, they were not visible in the Connectors view but their remnants were in the database and in the "Service Manager Linking Framework Configuration" management pack.  I was supplied some custom SQL to remove them from the database and instructions for removing them from the management pack.

Here is a link to the instructions I was given which resolved the issue of connector status's not updating:

https://onedrive.live.com/redir?resid=BA4F0DDA721038D8!107&authkey=!AFGxi0P3fzkX2g8&ithint=file%2cdocx

March 11th, 2015 9:39pm

I eventually logged a premium support call with Microsoft and the cause was identified as a number of orphaned connectors that had been configured and then later deleted, they were not visible in the Connectors view but their remnants were in the database and in the "Service Manager Linking Framework Configuration" management pack.  I was supplied some custom SQL to remove them from the database and instructions for removing them from the management pack.

Here is a link to the instructions I was given which resolved the issue of connector status's not updating:

https://onedrive.live.com/redir?resid=BA4F0DDA721038D8!107&authkey=!AFGxi0P3fzkX2g8&ithint=file%2cdocx

Free Windows Admin Tool Kit Click here and download it now
March 12th, 2015 1:39am

I eventually logged a premium support call with Microsoft and the cause was identified as a number of orphaned connectors that had been configured and then later deleted, they were not visible in the Connectors view but their remnants were in the database and in the "Service Manager Linking Framework Configuration" management pack.  I was supplied some custom SQL to remove them from the database and instructions for removing them from the management pack.

Here is a link to the instructions I was given which resolved the issue of connector status's not updating:

https://onedrive.live.com/redir?resid=BA4F0DDA721038D8!107&authkey=!AFGxi0P3fzkX2g8&ithint=file%2cdocx

March 12th, 2015 1:39am

I eventually logged a premium support call with Microsoft and the cause was identified as a number of orphaned connectors that had been configured and then later deleted, they were not visible in the Connectors view but their remnants were in the database and in the "Service Manager Linking Framework Configuration" management pack.  I was supplied some custom SQL to remove them from the database and instructions for removing them from the management pack.

Here is a link to the instructions I was given which resolved the issue of connector status's not updating:

https://onedrive.live.com/redir?resid=BA4F0DDA721038D8!107&authkey=!AFGxi0P3fzkX2g8&ithint=file%2cdocx

Free Windows Admin Tool Kit Click here and download it now
March 12th, 2015 1:39am

I eventually logged a premium support call with Microsoft and the cause was identified as a number of orphaned connectors that had been configured and then later deleted, they were not visible in the Connectors view but their remnants were in the database and in the "Service Manager Linking Framework Configuration" management pack.  I was supplied some custom SQL to remove them from the database and instructions for removing them from the management pack.

Here is a link to the instructions I was given which resolved the issue of connector status's not updating:

https://onedrive.live.com/redir?resid=BA4F0DDA721038D8!107&authkey=!AFGxi0P3fzkX2g8&ithint=file%2cdocx

March 12th, 2015 1:39am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics