SQL Server deadlocks when distribution manager is updating content

Hi

I have a ConfigMgr 2012 environment which consists of a single Primary Site running on SQL Server 2008 R2 SP2 CU8. The environment has been upgraded to SP1 CU3 and I'm noticing that the following deadlock error is occasionally being logged in distmgr.log when a package or application is distributed or updated on all distribution points. After the deadlock ends the content successfully distributes.

An extract of the error from distmgr.log is below

user(NT AUTHORITY\SYSTEM) runing application(SMS_DISTRIBUTION_MANAGER) from machine (LABCM01.lab.local) is submitting SDK changes from site(LAB)	SMS_DISTRIBUTION_MANAGER	3/10/2013 2:08:46 PM	11952 (0x2EB0)
*** IF NOT EXISTS (select 1 from vSMS_SC_SysResUse_Properties where ID = 72057594037927944 and Name = N'AvailableContentLibDrivesList' ~) insert into vSMS_SC_SysResUse_Properties (ID, Name, Value1, Value2, Value3) values (72057594037927944, N'AvailableContentLibDrivesList', N'FFEDC', N'', 0)~ ELSE update vSMS_SC_SysResUse_Properties set ID = 72057594037927944, Name = N'AvailableContentLibDrivesList', Value1 = N'FFEDC', Value2 = N'', Value3 = 0  where ID = 72057594037927944 and Name = N'AvailableContentLibDrivesList'	SMS_DISTRIBUTION_MANAGER	3/10/2013 2:08:47 PM	1432 (0x0598)
*** [40001][1205][Microsoft][SQL Server Native Client 11.0][SQL Server]Transaction (Process ID 115) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction. : spUpdateSiteControl	SMS_DISTRIBUTION_MANAGER	3/10/2013 2:08:47 PM	1432 (0x0598)
CSCItem_Base_Source::UpdateItem: Failed to save item (["Display=\\LABCM01.lab.local\"]MSWNET:["SMS_SITE=LAB"]\\LABCM01.lab.local\,SMS Distribution Point) properties	SMS_DISTRIBUTION_MANAGER	3/10/2013 2:08:47 PM	1432 (0x0598)
CSiteSettings::SubmitChanges: Failed to update item(["Display=\\LABCM01.lab.local\"]MSWNET:["SMS_SITE=LAB"]\\LABCM01.lab.local\,SMS Distribution Point)	SMS_DISTRIBUTION_MANAGER	3/10/2013 2:08:47 PM	1432 (0x0598)
CSiteSettings::SubmitDeltaSCFToDatabase:Failed to submit changes made from (LAB)	SMS_DISTRIBUTION_MANAGER	3/10/2013 2:08:47 PM	1432 (0x0598)
CSiteControlEx::SubmitDeltaSCFFromServerComponent:Failed to submit site control file changes made by componenet SMS_DISTRIBUTION_MANAGER	SMS_DISTRIBUTION_MANAGER	3/10/2013 2:08:47 PM	1432 (0x0598)
user(NT AUTHORITY\SYSTEM) runing application(SMS_DISTRIBUTION_MANAGER) from machine (LABCM01.lab.local) is submitting SDK changes from site(LAB)	SMS_DISTRIBUTION_MANAGER	3/10/2013 2:08:47 PM	1432 (0x0598)
Any ideas as to why the occasional deadlocks are occurring? 

Cheers

Sam

October 3rd, 2013 5:23am

Hi,

Deadlock is a mechanism of SQL server to handle a same object being accessed by two process at a same time.

Maybe you may review the SQL server's log to find out the other process's name and kill it.

Free Windows Admin Tool Kit Click here and download it now
October 5th, 2013 3:01pm

Hi Juke


Thanks for the reply, ConfigMgr is the only application running on each server so I'm not sure what other process would be causing deadlocks? Deadlocks didn't occur prior to the SP1 upgrade

October 6th, 2013 11:57pm

Hi,

Because this is a SQL server mechanism. The only way is to anayze the SQL server's log to find why there is a deadlock.

Free Windows Admin Tool Kit Click here and download it now
October 8th, 2013 5:35am

Try to analyse the deadlocks using SQL Profiler

http://msdn.microsoft.com/en-us/library/ms188246.aspx

October 8th, 2013 7:13am

Thanks guys, I've managed to capture a deadlock using SQL Profiler and the graph is below.

Additionally the deadlock XML output is below

<deadlock-list>
 <deadlock victim="process769288">
  <process-list>
   <process id="process769288" taskpriority="0" logused="1476" waitresource="KEY: 8:72057594111983616 (47c4361ec2ad)" waittime="484" ownerId="273355729" transactionname="user_transaction" lasttranstarted="2013-10-08T21:03:59.523" XDES="0x10b357270" lockMode="U" schedulerid="2" kpid="14220" status="suspended" spid="147" sbid="0" ecid="0" priority="0" trancount="2" lastbatchstarted="2013-10-08T21:03:59.527" lastbatchcompleted="2013-10-08T21:03:59.527" clientapp="SMS_DISTRIBUTION_MANAGER" hostname="SHDEPLOY2012" hostpid="4004" loginname="NT AUTHORITY\SYSTEM" isolationlevel="read committed (2)" xactid="273355729" currentdb="8" lockTimeout="4294967295" clientoption1="671287392" clientoption2="128056">
    <executionStack>
     <frame procname="CM_LAB.dbo.spUpdateSiteControl" line="12" stmtstart="622" stmtend="1400" sqlhandle="0x0300080041e141055d1bc40049a200000100000000000000">
MERGE INTO SiteControl AS T 
    USING ( 
        SELECT @SiteCode AS SiteCode, GETUTCDATE() AS DateLastModified 
    ) AS S(SiteCode, DateLastModified) 
    ON S.SiteCode = T.SiteCode 
    WHEN MATCHED THEN 
        UPDATE SET T.DateLastModified = S.DateLastModified 
    WHEN NOT MATCHED THEN 
        INSERT (SiteCode, DateLastModified) 
        VALUES(S.SiteCode, S.DateLastModified);     </frame>
     <frame procname="CM_LAB.dbo.tr_SC_SysResUse_Property_ins_upd_del" line="22" stmtstart="1594" stmtend="1672" sqlhandle="0x03000800a1e81749c924c40049a200000000000000000000">
EXEC spUpdateSiteControl @SiteCode     </frame>
     <frame procname="CM_LAB.dbo.tr_vSMS_SysResUse_Properties_upd" line="12" stmtstart="728" stmtend="1374" sqlhandle="0x03000800c63fcf7aa431c40049a200000000000000000000">
update rel 
        set rel.Value1=ins.Value1, 
            rel.Value2=ins.Value2, 
            rel.Value3=ins.Value3 
    from SC_SysResUse_Property as rel 
    inner join inserted as ins on rel.Name=ins.Name and rel.SysResUseID=ins.ID 
    where rel.Value1!=ins.Value1 or rel.Value2!=ins.Value2 or rel.Value3!=ins.Value3     </frame>
     <frame procname="adhoc" line="3" stmtstart="228" sqlhandle="0x02000000a88e68210a5e17dbd208c883c5925dc10fd661eb">
UPDATE [vSMS_SC_SysResUse_Properties] set [ID] = @1,[Name] = @2,[Value1] = @3,[Value2] = @4,[Value3] = @5  WHERE [ID]=@6 AND [Name]=@7     </frame>
     <frame procname="adhoc" line="3" stmtstart="586" stmtend="1026" sqlhandle="0x0200000089cc65023756c15d75189d92931f875bffb8980b">
update vSMS_SC_SysResUse_Properties set ID = 72057594037927953, Name = N&apos;AvailableContentLibDrivesList&apos;, Value1 = N&apos;DDC&apos;, Value2 = N&apos;&apos;, Value3 = 0  where ID = 72057594037927953 and Name = N&apos;AvailableContentLibDrivesList&apos;     </frame>
    </executionStack>
    <inputbuf>
IF NOT EXISTS (select 1 from vSMS_SC_SysResUse_Properties where ID = 72057594037927953 and Name = N&apos;AvailableContentLibDrivesList&apos; 
) insert into vSMS_SC_SysResUse_Properties (ID, Name, Value1, Value2, Value3) values (72057594037927953, N&apos;AvailableContentLibDrivesList&apos;, N&apos;DDC&apos;, N&apos;&apos;, 0)
 ELSE update vSMS_SC_SysResUse_Properties set ID = 72057594037927953, Name = N&apos;AvailableContentLibDrivesList&apos;, Value1 = N&apos;DDC&apos;, Value2 = N&apos;&apos;, Value3 = 0  where ID = 72057594037927953 and Name = N&apos;AvailableContentLibDrivesList&apos;
    </inputbuf>
   </process>
   <process id="process8fa5a748" taskpriority="0" logused="3160" waitresource="KEY: 8:72057595436728320 (d51fef0ca10c)" waittime="206" ownerId="273355664" transactionname="user_transaction" lasttranstarted="2013-10-08T21:03:59.493" XDES="0x12b176e90" lockMode="U" schedulerid="4" kpid="3812" status="suspended" spid="102" sbid="0" ecid="0" priority="0" trancount="2" lastbatchstarted="2013-10-08T21:03:59.800" lastbatchcompleted="2013-10-08T21:03:59.800" clientapp="SMS_DISTRIBUTION_MANAGER" hostname="SHDEPLOY2012" hostpid="4004" loginname="NT AUTHORITY\SYSTEM" isolationlevel="read committed (2)" xactid="273355664" currentdb="8" lockTimeout="4294967295" clientoption1="671156320" clientoption2="128056">
    <executionStack>
     <frame procname="adhoc" line="1" sqlhandle="0x02000000dd107007806e2488472b44e8e2cb235ae39d3ffb">
delete [vSMS_SC_SysResUse_Properties] where ID=72057594037927955 and Name not in (N&apos;ADSiteName&apos;, N&apos;AllowInternetClients&apos;, N&apos;AvailableContentLibDrivesList&apos;, N&apos;AvailablePkgShareDrivesList&apos;, N&apos;BindPolicy&apos;, N&apos;BITS download&apos;, N&apos;CertificateContextData&apos;, N&apos;CertificateExpirationDate&apos;, N&apos;CertificateFile&apos;, N&apos;CertificatePFXData&apos;, N&apos;CertificateType&apos;, N&apos;DPDrive&apos;, N&apos;DPMonEnabled&apos;, N&apos;DPMonPriority&apos;, N&apos;DPMonSchedule&apos;, N&apos;DPShareDrive&apos;, N&apos;IdentityGUID&apos;, N&apos;IISConfigCheckTimeInterval&apos;, N&apos;InstallInternetServer&apos;, N&apos;IPSubnets&apos;, N&apos;IPv6Prefixes&apos;, N&apos;IsActive&apos;, N&apos;IsAnonymousEnabled&apos;, N&apos;IsMulticast&apos;, N&apos;IsPXE&apos;, N&apos;LastIISConfigCheckTime&apos;, N&apos;MinFreeSpace&apos;, N&apos;PreStagingAllowed&apos;, N&apos;PXEPassword&apos;, N&apos;RemoveWDS&apos;, N&apos;ResponseDelay&apos;, N&apos;Server Remote Name&apos;, N&apos;Server Remote Public Name&apos;, N&apos;SslState&apos;, N&apos;SupportUnknownMachines&apos;, N&apos;UdaSetting&apos;)     </frame>
    </executionStack>
    <inputbuf>
delete [vSMS_SC_SysResUse_Properties] where ID=72057594037927955 and Name not in (N&apos;ADSiteName&apos;, N&apos;AllowInternetClients&apos;, N&apos;AvailableContentLibDrivesList&apos;, N&apos;AvailablePkgShareDrivesList&apos;, N&apos;BindPolicy&apos;, N&apos;BITS download&apos;, N&apos;CertificateContextData&apos;, N&apos;CertificateExpirationDate&apos;, N&apos;CertificateFile&apos;, N&apos;CertificatePFXData&apos;, N&apos;CertificateType&apos;, N&apos;DPDrive&apos;, N&apos;DPMonEnabled&apos;, N&apos;DPMonPriority&apos;, N&apos;DPMonSchedule&apos;, N&apos;DPShareDrive&apos;, N&apos;IdentityGUID&apos;, N&apos;IISConfigCheckTimeInterval&apos;, N&apos;InstallInternetServer&apos;, N&apos;IPSubnets&apos;, N&apos;IPv6Prefixes&apos;, N&apos;IsActive&apos;, N&apos;IsAnonymousEnabled&apos;, N&apos;IsMulticast&apos;, N&apos;IsPXE&apos;, N&apos;LastIISConfigCheckTime&apos;, N&apos;MinFreeSpace&apos;, N&apos;PreStagingAllowed&apos;, N&apos;PXEPassword&apos;, N&apos;RemoveWDS&apos;, N&apos;ResponseDelay&apos;, N&apos;Server Remote Name&apos;, N&apos;Server Remote Public Name&apos;, N&apos;SslState&apos;, N&apos;SupportUnknownMachines&apos;, N&apos;UdaSetting&apos;)    </inputbuf>
   </process>
  </process-list>
  <resource-list>
   <keylock hobtid="72057594111983616" dbid="8" objectname="CM_LAB.dbo.SiteControl" indexname="SiteControl_PK" id="lockf8de5880" mode="X" associatedObjectId="72057594111983616">
    <owner-list>
     <owner id="process8fa5a748" mode="X"/>
    </owner-list>
    <waiter-list>
     <waiter id="process769288" mode="U" requestType="wait"/>
    </waiter-list>
   </keylock>
   <keylock hobtid="72057595436728320" dbid="8" objectname="CM_LAB.dbo.SC_SysResUse_Property" indexname="SC_SysResUse_Property_AK" id="lock187f4f680" mode="U" associatedObjectId="72057595436728320">
    <owner-list>
     <owner id="process769288" mode="U"/>
    </owner-list>
    <waiter-list>
     <waiter id="process8fa5a748" mode="U" requestType="wait"/>
    </waiter-list>
   </keylock>
  </resource-list>
 </deadlock>
</deadlock-list>

Any ideas? Again this only occasionally happens when content is updated on the distribution points. 

Thanks

Sam



  • Edited by slewis1310 Tuesday, October 08, 2013 10:31 PM
Free Windows Admin Tool Kit Click here and download it now
October 8th, 2013 10:29pm

Do these occasional deadlocks cause a performance hit or loss of service?
October 9th, 2013 1:03pm

They don't appear to, I guess I'm interested as to why it's occurring as I've never seen it before in any ConfigMgr implementation I've done. Thanks for your help!
Free Windows Admin Tool Kit Click here and download it now
October 10th, 2013 6:49am

I have a similar issue after upgrading Configmgr 2012 to SP2.  I can update distribution points as long as the package does not have "Copy the content in this package to a package share on distribution points".  This means that our OSD Task sequence cannot be run from distribution points and packages must be copied (slower overall).

If I remove the package to package share option the content will distribute properly, adding this back gives the same deadlock issue as above.    I have tried creating a new package - same issue with deadlock.

November 15th, 2013 6:16pm

You should call Microsoft support if you can reproduce the issue. There's nothing we can do here in the forums.
Free Windows Admin Tool Kit Click here and download it now
November 15th, 2013 6:56pm

Issue has been resolved.  Ended up being interference on Distribution  points from Antivirus software.  The errors being presented in the Configmgr server logs were not the real cause.  Errors on the distribution points logs led to the resolution (update of AV exclusion lists).
May 13th, 2014 9:49pm

Which exclusions did you have to make for this to be resolved?
Free Windows Admin Tool Kit Click here and download it now
June 6th, 2015 1:50pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics