PKI Client starts to intialize, then 7 minutes later client agents go back to disabled for upwards of an hour

Starting a new thread on this as I have done much digging and no longer believe its a conflicting record issue.

I am SCCM 2012 SP1 on Server 2008 R2/SQL 2012 CU2.  My "Lan" based management point is HTTP and I also have an Internet Management point that is HTTPS.  All scenerios discussed in this thread are installing the 2012 SP1 client while connected to the Lan.  We have a fully functional PKI infrastructure with auto enrol enabled.

The issue at hand:

Wether I install the client during OSD, or manually install the client I get the same result.  This is that the client agent upon successful install begins communicating with the HTTP management point and starts retrieving policy.  If I open the ConfigMgr applet in Control Panel I see the client shows "Client Certificate: PKI", "Connection Type: Currently Intranet".  If I view actions I see all actions with the exception of Discovery Data Collection Cycle and Hardware Inventory Cycle.

I have watched the client logs the only thing that seems to stick out is in the CcmNotificationAgent.log which shows the bgb client agent actions.  it repeats the following aprox every 5 minutes:

bgb client agent is starting...
bgb client agent is disabled
TCP Listener is disabled
bgbController main thread us started with settings: [bgb enable = 0], {tcp enable = 0} and {http enable = 0}.
Wait 3600 seconds for even notification

The ClientIDManagerStartup.log files shows:

PopulateRegistrationHint: Using the Certificate selected by the current version of SCCM to set the hint. ClientIDManagerStartup 1/23/2013 5:00:51 PM 2100 (0x0834)
CCMCreateAuthHeadersEx failed (0x80004005). ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
PopulateRegistrationHint failed (0x80004005), expected upon first start of non-upgrade client. ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Begin searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Finding certificate by issuer chain returned error 80092004 ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Completed searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Unable to find any Certificate based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Raising event:

instance of CCM_ServiceHost_CertRetrieval_Status
{
 ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
 DateTime = "20130123230052.204000+000";
 HRESULT = "0x87d00215";
 ProcessID = 2352;
 ThreadID = 2100;
};
 ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Failed to submit event to the Status Agent. Attempting to create pending event. ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Raising pending event:

instance of CCM_ServiceHost_CertRetrieval_Status
{
 ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
 DateTime = "20130123230052.204000+000";
 HRESULT = "0x87d00215";
 ProcessID = 2352;
 ThreadID = 2100;
};
 ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
PKI Client Certificate matching SCCM certificate selection criteria is not available. ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Generated a new Signing certificate ClientIDManagerStartup 1/23/2013 5:00:54 PM 2100 (0x0834)
Generated a new Encryption certificate ClientIDManagerStartup 1/23/2013 5:00:54 PM 2100 (0x0834)
Initializing registration renewal for potential PKI issued certificate changes. ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
Succesfully intialized registration renewal. ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
[RegTask] - Executing registration task synchronously. ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
Read SMBIOS (encoded): 32003100350039002D0039003900320037002D0032003500360036002D0038003300320035002D0037003500340032002D0035003200370031002D0033003900 ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
Evaluated SMBIOS (encoded): 32003100350039002D0039003900320037002D0032003500360036002D0038003300320035002D0037003500340032002D0035003200370031002D0033003900 ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
No SMBIOS Changed ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
SMBIOS unchanged ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
SID unchanged ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
HWID unchanged ClientIDManagerStartup 1/23/2013 5:00:57 PM 2348 (0x092C)
Windows To Go requires a minimum operating system of Windows 8 ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
GetSystemEnclosureChassisInfo: IsFixed=TRUE, IsLaptop=FALSE ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
Windows To Go requires a minimum operating system of Windows 8 ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
Computed HardwareID=2:9C8C08C4B3E16249A2F1457998D16528B656DE30
 Win32_SystemEnclosure.SerialNumber=2159-9927-2566-8325-7542-5271-39
 Win32_SystemEnclosure.SMBIOSAssetTag=9344-3677-7824-5579-3797-0729-37
 Win32_BaseBoard.SerialNumber=2159-9927-2566-8325-7542-5271-39
 Win32_BIOS.SerialNumber=2159-9927-2566-8325-7542-5271-39
 Win32_NetworkAdapterConfiguration.MACAddress=00:15:5D:0B:78:20 ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
[RegTask] - Client is not registered. Sending registration request for GUID:b4aacc70-6de3-4829-88e0-498777c49379 ... ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
[RegTask] - Client registration is pending. Server assigned ClientID is GUID:b4aacc70-6de3-4829-88e0-498777c49379 ClientIDManagerStartup 1/23/2013 5:01:00 PM 2348 (0x092C)
[RegTask] - Sleeping for 60 seconds ... ClientIDManagerStartup 1/23/2013 5:01:00 PM 2348 (0x092C)
RenewalTask: Executing renewal task. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Begin searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Based on Certificate Issuer 'MyrootCA' found Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Completed searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Begin to select client certificate ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
>>> Client selected the PKI Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Raising event:

instance of CCM_ServiceHost_CertRetrieval_Status
{
 ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
 DateTime = "20130123230143.106000+000";
 HRESULT = "0x00000000";
 ProcessID = 2352;
 ThreadID = 2620;
};
 ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Client PKI cert is available. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
RenewalTask: Certificate has changed, initiating a renewal. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Aborting any pending registration. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Re-registration/renewal initiated. Restarting the service. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
[----- SHUTDOWN -----] ClientIDManagerStartup 1/23/2013 5:01:44 PM 2100 (0x0834)
[----- STARTUP -----] ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Machine: W21599927256 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
OS Version: 6.1 Service Pack 1 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
SCCM Client Version: 5.00.7804.1000 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Client is set to use HTTPS when available. The current state is 448. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Based on Certificate Issuer 'MyrootCA' found Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Completed searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin to select client certificate ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
>>> Client selected the PKI Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Raising event:

instance of CCM_ServiceHost_CertRetrieval_Status
{
 ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
 DateTime = "20130123230145.722000+000";
 HRESULT = "0x00000000";
 ProcessID = 3612;
 ThreadID = 3888;
};
 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Failed to submit event to the Status Agent. Attempting to create pending event. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Raising pending event:

instance of CCM_ServiceHost_CertRetrieval_Status
{
 ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
 DateTime = "20130123230145.722000+000";
 HRESULT = "0x00000000";
 ProcessID = 3612;
 ThreadID = 3888;
};
 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
'RDV' Identity store does not support backup. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
CCM Identity is in sync with Identity stores ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Based on Certificate Issuer 'MyrootCA' found Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Completed searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin to select client certificate ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
>>> Client selected the PKI Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Raising event:

instance of CCM_ServiceHost_CertRetrieval_Status
{
 ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
 DateTime = "20130123230145.752000+000";
 HRESULT = "0x00000000";
 ProcessID = 3612;
 ThreadID = 3888;
};
 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Failed to submit event to the Status Agent. Attempting to create pending event. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Raising pending event:

instance of CCM_ServiceHost_CertRetrieval_Status
{
 ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
 DateTime = "20130123230145.752000+000";
 HRESULT = "0x00000000";
 ProcessID = 3612;
 ThreadID = 3888;
};
 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Client PKI cert is available. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Initializing registration renewal for potential PKI issued certificate changes. ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
Succesfully intialized registration renewal. ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
[RegTask] - Executing registration task synchronously. ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
Read SMBIOS (encoded): 32003100350039002D0039003900320037002D0032003500360036002D0038003300320035002D0037003500340032002D0035003200370031002D0033003900 ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
Evaluated SMBIOS (encoded): 32003100350039002D0039003900320037002D0032003500360036002D0038003300320035002D0037003500340032002D0035003200370031002D0033003900 ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
No SMBIOS Changed ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
SMBIOS unchanged ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
SID unchanged ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
HWID unchanged ClientIDManagerStartup 1/23/2013 5:01:49 PM 3928 (0x0F58)
Windows To Go requires a minimum operating system of Windows 8 ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
GetSystemEnclosureChassisInfo: IsFixed=TRUE, IsLaptop=FALSE ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
Windows To Go requires a minimum operating system of Windows 8 ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
Computed HardwareID=2:9C8C08C4B3E16249A2F1457998D16528B656DE30
 Win32_SystemEnclosure.SerialNumber=2159-9927-2566-8325-7542-5271-39
 Win32_SystemEnclosure.SMBIOSAssetTag=9344-3677-7824-5579-3797-0729-37
 Win32_BaseBoard.SerialNumber=2159-9927-2566-8325-7542-5271-39
 Win32_BIOS.SerialNumber=2159-9927-2566-8325-7542-5271-39
 Win32_NetworkAdapterConfiguration.MACAddress=00:15:5D:0B:78:20 ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
[RegTask] - Client is not registered. Sending registration request for GUID:b4aacc70-6de3-4829-88e0-498777c49379 ... ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
[RegTask] - Client registration is pending. Server assigned ClientID is GUID:b4aacc70-6de3-4829-88e0-498777c49379 ClientIDManagerStartup 1/23/2013 5:01:54 PM 3928 (0x0F58)
[RegTask] - Sleeping for 60 seconds ... ClientIDManagerStartup 1/23/2013 5:01:54 PM 3928 (0x0F58)
[RegTask] - Client registration is pending. Sending confirmation request for GUID:b4aacc70-6de3-4829-88e0-498777c49379 ... ClientIDManagerStartup 1/23/2013 5:02:54 PM 3928 (0x0F58)
[RegTask] - Client is registered. Server assigned ClientID is GUID:b4aacc70-6de3-4829-88e0-498777c49379. Approval status 2 ClientIDManagerStartup 1/23/2013 5:02:54 PM 3928 (0x0F58)

After almost 7 minutes exactly, the all the client agent actions besides Machine Policy Retrieval and UserPolicy Retrieval disappear (as if client policy was Reset)

If I let the client sit for 45-minutes to an hour, everything starts working again and works fine from there on out. (cycling the SMS Agent service nor rebooting the machine makes it recover until the 45-min to an hour pass)

My command line I am using to install the client is:

ccmsetup.exe /mp:MYLANBASEDMP /UsePKICert /NOCRLCheck CCMHOSTNAME="myinternetmp.mydomain.com" SMSSITECODE=P10 SMSCACHESIZE=7000 FSP=MYLANBASEDMP CCMLOGMAXSIZE=1000000

If I take out the /usePKICert, /NOCRLCheck and CCMHOSTNAME Entries, the client install and continues to function without issue.

Anyone have any others ideas on where to troubleshoot this issue?  It would make more sense if the client NEVER worked after install.  Tearing my hair out trying to figure out why it starts to intialize, then reverts, then comes back online and works fine.  This happens at both my primary site MP as well as my secondary site/mp.  It happens on my standard Win7 image as well as Windows 8 test machines so I dont believe its a client OS issue.


January 24th, 2013 2:48am

More information:

I noticed that Windows Management Framwork 3.0 (beta) was installed on the primary site server.  I uninstalled this version and installed the latest released version.  I was then able to reinstall the client on the machine above and it did not go dormant after 7 minutes and continued to work without issue (using the PKI stuff).  I then rebuilt this machine completely and the client once again did not exhibit the behavior and continued work successfully. 

I then pushed the client install to a Windows Server 2012 box and once again the client came online and stayed online.

Both of these clients are located at my secondary site server subnet using a proxy management point.

I also have a VMware client thats in the boundaries of the Primary Site server.  I built the machine a few days ago to see if the same thing would happen in a different location and it did indeed go dormant after 7 minutes and after about an hour came back online.

After my success against the secondary I rebuilt the VM in the Primary boundary and it still goes dormant after 7 minutes.  :-/

I noticed on the 2 clients here that worked that the CcmNotificationAgent.log shows "bgbController main thread is started with settings: {bgb enabled = 1}, {tcp enabled = 1}, {tcp port = 10123} and {http enabled = 1}.

On the client that exhibits the behavior (the vm in the primary site boundaries) the CcmNotificationAgent.log shows "bgbController main thread is started with settings: {bgb enabled = 0}, {tcp enabled = 0}, {tcp port = 0} and {http enabled = 0}.

Can't say for sure that this is whats causing the behavior but it does appear to be consistent for client that "fail" and then recover.

Anyone have any suggestions on where to look next?



Free Windows Admin Tool Kit Click here and download it now
January 24th, 2013 8:14pm

Just got off the phone with MS Support and as expected, the gentleman had never seen this behavior.  I captured 2 sets of logs from different machines and uploaded to them.  He is going to escelate the issue with the logs for analysis.  Likely wont hear back until Monday.
January 25th, 2013 9:50pm

William, thanks for posting this!  I am having the exact same issue!  It has been driving me nuts trying to figure it out.  I was just able to validate, as a workaround, I can delete the record from database and reinstall client and it does appear to resolve issue as best I can tell.  I've had tons of clients go "Inactive" (Disabled) just like yours.  I haven't been able to find a pattern to why some clients work and some don't, besides deleting records from database.  You found any other info yet?
  • Edited by bcehr Tuesday, January 29, 2013 2:59 PM
Free Windows Admin Tool Kit Click here and download it now
January 29th, 2013 5:52pm

hello William,

i found your thread because i have a similar problem (maybe even the exact same), with the error message "Finding certificate by issuer chain returned error 80092004". i have a similar setup (lan MP with http and another internet MP with https).

when i do a client push install i see this in ClientIDManagerStartup.log:

i tried already a lot to fix this thing, and i'm short before reinstalling the entire PKI/cert things from scratch. so far i have found one workaround:

http://austrianalex.com/2012/10/system-center-configuration-manager-sccm-2012-client-pki-and-subordinate-ca-woes/

this actually works for me - we have our root CA certificate distributed through active directory to our clients, so when i remove the cert from SCCM primary site properties / client computer communication / trusted root certification authorities, installation of the clients works again, like described in the link above.

however, i am not sure if this is a correct "solution" because i now got errors on other parts of the infrastructure (e.g. pxe driver installation), which may be because i do not have a trusted root cert set in SCCM....

i thought i share this here so maybe you want to try this on your end too.

EDIT:

after a lot of testing, this "solution" does not help me, as PXE no longer works without the trusted root CA cert set in site properties.

February 12th, 2013 3:16pm

Also seeing this same issue. Happens very randomly and as far as I can tell with no environment changes, or warnings for changes.

2012 Configuration Manager only environment, client has a the new SP1 agent. Was working just fine and even though there was no changes all of the sudden says "Client is set to use HTTPS when available. The current state is 448" and PKI none.


Free Windows Admin Tool Kit Click here and download it now
April 29th, 2013 8:57pm

I see this behavior on almost every install, whether its a new install or a reinstall.  In ALL cases however if I just let the machine sit, the client will come back online (typically about 45 minutes) and then work just fine from there on out.  Do your clients eventually recover bcehr or once they are disabled they never "self-correct"?
May 6th, 2013 4:32pm

Is there still no hotfix out for this?
Free Windows Admin Tool Kit Click here and download it now
August 5th, 2013 4:26pm

Still nothing yet bcehr.  From the latest info I have its supposed to be included in CU3.  No eta that I am aware on a release date.
August 5th, 2013 7:21pm

Just want to add I am having a remarkably similar issue (with the exception being that I am not running native mode).

Currently, working with Microsoft on it and I even referenced this thread and your issue.

He hasn't given me any additional info on CU3 either.

Free Windows Admin Tool Kit Click here and download it now
August 28th, 2013 4:09pm

So CU3 has been released: http://support.microsoft.com/kb/2882125/en-us

I don't see anything in the noted fixes that translates directly to this issue however we are going to proceed with it and keep our fingers crossed.  I'll post back when testing is completed.

September 23rd, 2013 4:41pm

I think this fix may be related (since I did observe this on my systems when the issue occurred):

  • The reimaging of an existing client computer may result in a policy being invalidated. This issue can occur if the SMS Agent Host service restarts before all Data Transfer Service (DTS) jobs are completely processed. Additionally, the PolicyAgent.log file on the reimaged client will contain entries that resemble the following: [CCM_Policy_Policy5.PolicyID="ScopeId_<var>GUID</var>/Application_<var>GUID</var>/CA",PolicySource="SMS:PRI",PolicyVersion="1.00"] is pointing to invalid DTS job [{<var>DTS_JOB_GUID</var>}]. Will attempt to re-download.

Free Windows Admin Tool Kit Click here and download it now
September 23rd, 2013 5:54pm

Yeah, that's the one item that gives me a glimmer of hope!
September 23rd, 2013 6:44pm

Unfortunately, from my initial testing, I am still seeing the issue.


  • Edited by WillMF Monday, September 23, 2013 9:20 PM
Free Windows Admin Tool Kit Click here and download it now
September 24th, 2013 12:19am

That's discouraging.. I will be testing tomorrow and will report back my findings.
September 24th, 2013 12:39am

I can confirm that CU3 DOES NOT fix this issue. Disappointing to say the least.  :-/  It has now been almost 9 months and still no fix. :(
Free Windows Admin Tool Kit Click here and download it now
September 24th, 2013 11:34pm

Just to double-check and to avoid potential confusion: have you installed CU3 during the task sequence?
September 24th, 2013 11:38pm

Hey Torsten,My testing came from uninstalling the client/reinstalling the client (with the CU3 msp using the PATCH parameter).  But to answer your question yes, I have updated my Task Sequences to point to CU3 during client install (replacing CU1 as we skipped CU2).

I have been able to reproduce this issue regardless of how the client gets installed so its a safe assumption at this point that when I do test my updated TS that I will get the same result.

Free Windows Admin Tool Kit Click here and download it now
September 25th, 2013 12:18am

Just an FYI for anyone following this thread.  I have also update the escalation engineer at Microsoft with this information.  I'll post back if/when I get any additional info...
September 25th, 2013 12:20am

I am still testing to confirm, but I think I may have found the trigger for this issue (at least in my case).

When testing, I have always immediately logged into the re-imaged system as soon as imaging completed so I could look for the symptoms of the issue (client works initially, then stops working, then starts working again after 45 minutes - hours later). In that case, there was a high likelyhood that I would experience the issue.

I decided to re-image the systems and not login immediately after and see if anything changed. When I did this, I noticed (by monitoring the client logs) that (in four tests so far), I did not experience the issue.

I have been trying to find a sweet spot as far as how long I should wait from the first client initialization after imaging before logging in (10 minutes, so far) without triggering the issue.

Figured I would post to see if anyone could replicate the results and possibly pass this on to Microsoft.

Free Windows Admin Tool Kit Click here and download it now
September 26th, 2013 11:10pm

Hmm, I haven't tested this since rolling out CU3 however I am "almost" positive I did this same test several months ago and got the same result. Can't say for certain though so I'll give it another try and see. I'll actually test with uninstalling and reinstalling the client remotely while no one is logged in. I'll report back my findings.

Thanks WillMF

September 27th, 2013 4:18pm

WillMF, curious if you have a root cert configured on your primary site server?
Free Windows Admin Tool Kit Click here and download it now
September 27th, 2013 4:34pm

No root cert.
September 27th, 2013 4:39pm

Thanks for the quick reply. Well that blows my theory that it was only related to infrastructures that are using PKI. :-/ Don't understand why some (shall I say most) are not having (or at least not reporting) this issue. I was able to reproduce the issue in my lab as well. What the heck are we doing so different??!!
Free Windows Admin Tool Kit Click here and download it now
September 27th, 2013 5:15pm

So my first test of reinstalling the client without a user being logged in, waiting for 15 minutes, then logging in did NOT reproduce the issue. Going to try it a few more times to ensure it wasn't an anomaly. Doesn't really fix anything obviously but good info to know..
September 27th, 2013 5:56pm

Yeah, my lab setup as it is now is pretty much a standard OOB SCCM 2012 setup with a few desktops, a laptop, and a VM.

Like you, I was surprised more people haven't seen/noticed this.

So far, when not logging in immediately after re-image, the issue does not appear to be coming back.

I am running through a few more tests on my "not logging in immediately" theory. Then I am going to run another test logging on immediately after re-image again and see if the issue returns.

Free Windows Admin Tool Kit Click here and download it now
September 27th, 2013 6:25pm

I can confirm that when I install the client without a user being logged on and wait 10-15 minutes before logging on after the client is installed the issue does not appear. 

For sanity check I logged on to the machine and then installed the client (same way as previous, using psexec remotely) the issue came right back.

Definitely related to having a logged on user.  :-/

September 27th, 2013 7:15pm

Hi,

We rolled out CU3 because of the problems you talk about. We also have the same problems with approx. 10~15 clients a week.

Waiting for a solution. Disappointed.

-Dietmar-

Free Windows Admin Tool Kit Click here and download it now
September 30th, 2013 7:08pm

Just checking in to see if you have received any update/s from the Microsoft escalation engineer you were working with.
October 14th, 2013 8:04pm

Nope, honestly been busy and forgot until this post. I just sent another email. So far no response. :-/
Free Windows Admin Tool Kit Click here and download it now
October 14th, 2013 8:52pm

Just got the following response:

Hope you are doing well. I am really sorry for the late reply since I was out of office for last two weeks. However, I just check few of the documents and found that the fix is not available in CU3. Since it was not a business critical issue so it was not fixed in CU3 it will have a fix soon.

Have a nice day.

October 14th, 2013 9:03pm

Just a general FYI for the thread.

I have updated my lab setup to SCCM 2012 R2 and this issue still occurs under the same scenarios previously discussed.

Free Windows Admin Tool Kit Click here and download it now
October 22nd, 2013 9:50pm

True. Just noticed that in my lab, too :-(
October 22nd, 2013 11:07pm

Thanks for the follow up gentlemen.  Still waiting for a fix :-|
Free Windows Admin Tool Kit Click here and download it now
November 6th, 2013 7:27pm

This is my unhappy face -----> :-[

I have this issue with a client and read this thread (thank you Torsten) with sad eyes.

Non-native, 45 minutes +, Clients come back into shape and work thereafter ok.

We are going to try and build and not login, thanks for the workaround ... MS fix it now this is annoying the business so it must be critical!

November 28th, 2013 6:59pm

Yeah, its been a LONG time since this issue was identified (as evidence of my original post date).  Can't believe its still not resolved.  There has been a SP, 3 CU's, and a new version.  :-/

I contest its not business critical.  When I rebuild an existing machine the users may get notified of Available Software but by the time they go to install it, the client likely has gone quiet and presents them with a nice error message.

Although I am certainly not happy others are having the same issue, maybe if enough people cry out about it they will get around to correcting it.

Free Windows Admin Tool Kit Click here and download it now
December 2nd, 2013 5:42pm

Definitely critical for me since I cannot proceed with anything but a lab deployment until this is resolved.

I would open another case with Microsoft, but the last time I did that for this issue they were too busy focusing on trying to find things I was doing wrong and wasting my time.

December 6th, 2013 3:54am

So I received a call from Microsoft Support yesterday saying asking if this issue had been resolved stating that it was supposed to be fixed in CU3 (contrary to what my previous support tech told me).  As you might expect I informed him that the issue had not been resolved in CU3 (and that I didn't see anything in the "resolved" list from the CU3 KB article that stated this issue had been fixed).

I was told I need to open a new case as this issue is now considered NEW due the fact that it should have been resolved in CU3.  :-/ 

Unfortunately it appears I get to now rerun diagnostics and verbose debug logs again.  Good times.  I'll keep this thread posted with any progress.

Free Windows Admin Tool Kit Click here and download it now
December 6th, 2013 7:18pm

Good news! (and potentially bad news)

Good news, I received and email from the new support tech stating that indeed this issue was NOT fixed in CU3, however there is a workaround:

At present we have a workaround for the issue by setting  the following registry

HKLM\Software\Microsoft\CCM\UserPolicyReRequestDelay (REG_DWORD) value: 6,000,000 (decimal).

Please add a step in your Task Sequence to add this registry value.

If you are also facing the issue while trying to install the client manually then please follow these steps

1. Install the client manually

2. Immediately disable and stop the CCMExec service (SMS Agent Host)

3. Set the following registry HKLM\Software\Microsoft\CCM\UserPolicyReRequestDelay (REG_DWORD) value: 6,000,000 (decimal)

4. Enable and set the CCMExec service to automatic

5. Start the CCMExec service

I have tested when doing a manual client install and it works perfectly..IF you make sure to stop CCMEXEC directly after the client finishes installing as denoted in step 2 above.  I have every reason to believe that will will also work during a Task Sequence base don these results.  I'll be testing that soon.

The bad news: If you are using Client Push, this workaround would not work since there isn't a way to wrap a script around the installer to perform the steps above.  Maybe you could add this value to all machines prior to client push using GPP's or a startup script?

I don't currently use Client Push so its not a huge issue for me, I will just need to adjust my machine startup script to perform the steps but can see how this will still be an issue for others.

Either way, its a step in the right direction.  Certainly tells me they have identified the issue and will hopefully be including a true fix in an upcoming hotfix or update.

January 2nd, 2014 7:47pm

Just another quick update...adding the reg key prior to client push install doesn't work as ccmsetup.exe removes any existing keys prior to installing the client.  :-/ Not sure how to get around that one.. 
Free Windows Admin Tool Kit Click here and download it now
January 2nd, 2014 10:27pm

Many thanks for this post William.

I will be very much interested in your procedure for adding this via a task seqeunce. Im currently also doing some testing and will report my results.

cheers

January 3rd, 2014 4:45pm

I added the following Run Command Line task as the first step in the State Restore phase. 

reg add HKLM\Software\Microsoft\CCM /v UserPolicyReRequestDelay /t REG_DWORD /d 6000000

 Initial testing looks good.

Free Windows Admin Tool Kit Click here and download it now
January 3rd, 2014 11:40pm

Ok, indeed all does look well...

I havent had a single instance of a faulty sccm cleint.

One thing i have noticed is when i try to install software from the Application Catalog the install gets stuck at "Evaluting Requirements ...This may take a few minutes" However, if i reboot the machine or restart the SMS Agent Host Service, then  try again the install seems to go through fine..

Which then of course make me ask the question, what exactly is this workaround doing...  obviously delaying 'User Policy ReRequest Delay'  but exactly what effect is this having and is this whats causing the issue i am seeing above..???

But overall., the workaround is great, lets just hopes it sticks until an official fix is released.

Edit.....

Seems that if i leave the computer alone for a while everything works first time.. could just be a case of me not allowing the SCCM Client agent time to settle down.

  • Edited by The Overfiend Monday, January 06, 2014 3:49 PM update..
January 6th, 2014 4:26pm

Glad to see they have at least ID'd the issue this time.

I am curious to know did they give you any indication as far as what exactly this setting is doing?

For example, would it delay the ability for the system to detect user-targeted deployments? And if so, for how long?

In any case, happy to see that this may finally be resolved in upcoming hotfix/CU :-)

Free Windows Admin Tool Kit Click here and download it now
January 6th, 2014 9:41pm

Adding the line you noted does appear to prevent the client from experiencing the previously noted issue.

Unfortunately, so far, it appears to cause user-targeted deployments to not be evaluated.

Previously, before applying the noted fix, the client would start deploying the user targeted deployments once the client issue resolved itself.

With the fix in play and over an hour since the image completed, no user targeted deployments have run.

I may play with lowering the DWORD value (assuming it equates to a timeframe) to see if that resolves this new issue.

Has anyone else who has tested this seen this?


  • Edited by WillMF Tuesday, January 07, 2014 8:50 PM grammar
January 7th, 2014 11:40pm

Just want to update on my additional testing and tweaking. Please see results below:

 

DWORD Value: 6000000

Initial Client Policy Rec'd: 2:21PM

Start of User Targeted App Deployments: 4:07PM

Duration: 1hr 46 minutes

 

DWORD Value: 1500000

Initial Client Policy Rec'd: 6:50PM

Start of User Targeted App Deployments: 7:16PM

Duration: 26 minutes

 

DWORD Value: 1000000

Initial Client Policy Rec'd: 12:58PM

Start of User Targeted App Deployments: 1:16PM

Duration: 18 minutes

 

I am going to continue testing to see if it is consistent.


  • Edited by WillMF Wednesday, January 08, 2014 6:31 PM
Free Windows Admin Tool Kit Click here and download it now
January 8th, 2014 9:30pm

Just wanted to give my final update from my testing.

I settled on a DWORD value of 500000 (i.e. reg add HKLM\Software\Microsoft\CCM /v UserPolicyReRequestDelay /t REG_DWORD /d 500000)

With that setting added during the task sequence as per previous guidance, I am seeing the application deployments starting between 9 - 13 minutes after initial client policy is received and no recurrence of the original client policy loss issue.

Thanks again William for seeing this through to what appears to be a positive conclusion.

January 9th, 2014 10:28pm

Thanks for the additional testing and feedback WillMF.  Glad to see some movement on this issue finally.  Quite a nasty bug.  I did reach back out to the support tech asking for clarification on the key and got the following response:

As per my understanding this registry key adds a delay to the user policy request after the SMS Agent Host Service starts thus preventing a conflict condition with the sms agent. However give me some time to consult my Technical support leads who provided the workaround to get more information.

So our suspicions and your testing are indeed validated.

Thanks!

Free Windows Admin Tool Kit Click here and download it now
January 10th, 2014 5:07pm

Just a general FYI for the thread.

I installed CU1 for SCCM 2012 R2 and then tested imaging without the above noted tweak in play and the issue returned.

So, it appears that without the tweak in play, the issue still persists in CU1 for R2.

March 31st, 2014 8:54pm

That's disappointing to say the least.  This issue must really have the devs stumped.
Free Windows Admin Tool Kit Click here and download it now
April 1st, 2014 5:09pm

Have you any news about a fix?
July 23rd, 2014 10:04am

Hi TWEESTY,

I have unfortunately moved on from that position and into a consulting role.  I can say though that with the 3 SCCM projects I have done with SCCM 2012 R2 (up to CU1) I have not seen this issue at other clients so honestly forgot about it.  :-/

Free Windows Admin Tool Kit Click here and download it now
July 25th, 2014 4:43pm

After encountering something similar, even if without PKI, I ended up here and here is my explanation for what happened.

- I created an image by a task sequence which during the image creation installed the SCCM client and talked with the SCCM server.

- After this "conversation", a new device appeared in my system which showed simply as "unknown", and which is separate from the other default unknown devices used for targeting systems not existing in the database (these are named x86 Unknown Computer and x64 Unknown Computer).

- after sysprepping the image during the task sequence I went ahead with introducing the computer into my SCCM environment (included in the domain, triggered discovery, waited to appear).

- initially it appeared in the list as a separate device from the "unknown" one, but after a while, after in started to receive policies and tried to apply an update deployment, suddenly went quiet and actions previously available in the SCCM Client interface dissapeared.

- after this behavior happened, the "unknown" device disappeared from my list (my presumption is that it had the same GUIDs or hardware IDs and got merged)

- Because of this merge, the assigned policies must have been reevaluated on the server and the ones already assigned to the client temporarily revoked as the SCCM client appeared to be the same as the "unknown" one.

- after around one hour, the client started to work again.

My conclusion is that maybe this happens because of this image that when created in the SCCM environment, it also creates this kind of shadow client, waiting to create a conflict with new computers imaged with the same GUIDs.

Maybe deleting the SCCM cert from the image (ccmdelcert.exe) before doing the sysprep step in the task sequence might help.

December 12th, 2014 1:16am

Hey Narcis, not to rain on your theory but the environment I was in, we created all images in standalone MDT so the SCCM Client never touched the core image.  :-/
Free Windows Admin Tool Kit Click here and download it now
December 12th, 2014 1:36am

Just wanted to say thanks for this thread, i thought I was going crazy when I would see the client become active, and then revert to a newly deployed state where it would not function and re-download client policy. Implementing this reg hack now and testing...
January 10th, 2015 12:30am

I have this problem as well and will be trying the DWORD reg key hack shortly.
Free Windows Admin Tool Kit Click here and download it now
January 26th, 2015 6:58pm

This reg key helped out a bunch.  I used a value of 500000.  Thank you.
January 27th, 2015 12:17am

Good deal.  Thanks for the reply Enten
Free Windows Admin Tool Kit Click here and download it now
January 27th, 2015 5:43am

I just wanted to add that I had a case open with MS concerning a massive failure when attempting to upgrade from SP1 to R2 and after finally upgrading my client was still showing the symptoms described in this thread.

From another thread I posted I was directed to this one and when I brought it up to MS support they were pleased I found this solution on my own and were going to suggest the reg key to me, but I found the "resolution" beforehand.

Very...sketchy...support...

Either way it sounds like its a known bug, but they do not know of any official patch in development.  I asked for a communication if support finds out if a patch will be released, but I'm not expecting a call/email ever.

January 28th, 2015 6:17pm

I am getting ready to build a new server at a new site. I am surprised they still haven't fixed this.
Free Windows Admin Tool Kit Click here and download it now
January 29th, 2015 10:27pm

The only good news WillMF is that it doesn't affect all installs of SCCM.  I have built, and/or worked with several infrastructures since I had this issue at a particular place and have (luckily maybe) not seen it again myself. The configurations all very different.

If its a separate infrastructure, maybe it will work?  Or does the issue follow the network...  Hmm, has anyone built a 'different' infrastructure on the same network or vlan and had the issue disappear? 

Also for good measure,

2012 R2 CU4 was recently released.  The release notes do not list any fixes that appear to address the issue, however I would love to hear feedback from anyone having this issue who updates to CU4.  :-)


February 3rd, 2015 9:34pm

Bracken, still seems to be happening to us in 2012 R2 CU4 as well.
Free Windows Admin Tool Kit Click here and download it now
April 9th, 2015 2:51pm

Thanks for the feedback Mike.  Mind blowing this is still an issue!
April 17th, 2015 10:17am

Premier Support advised me to include this in my task sequence:

An enterprise hotfix rollup is available for Windows 7 SP1 and Windows Server 2008 R2 SP1

https://support.microsoft.com/en-us/kb/2775511/en-us

I included it in my reference machine, captured it and added the Operating system Image to my task sequence.

The issue persists :(

I have noticed that the issue is not consistent. e.g. On one machine today I ran the t/s and it finished in the estimated 60 minutes. I deleted the machine from AD and SCCM and ran the task sequence again. The issue then occurred again in this second task sequence.

Free Windows Admin Tool Kit Click here and download it now
June 11th, 2015 5:21am

Just wanted to let you know I have this issue on 2012 R2 CU4 as well.

My task sequence steps:

1. Copy CU4 patches to local drive

2. Install Config Mgr client with PATCH command (PATCH="C:\windows\CCMHotfixes\configmgr2012ac-r2-kb3026739-i386.msp")

3. Exit provisioning mode (REG ADD HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\CCM\CcmExec /v ProvisioningMode /t REG_SZ /d false /f)

4. Clear system excludes (part of provisioning mode) REG ADD HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\CCM\CcmExec /v SystemTaskExcludes /t REG_SZ /d /f

After this I have a step to install one piece of software and then reboot the machine. After the reboot the client takes 30 minutes to initialize. Once it initializes it installs the next software. Any subsequents reboots see the client taking 30 minutes to initialize. Strangely after 120 minutes the client wakes up and all is well.

This issue only started occurring this past Sunday. It worked fine last week and no change was made to anything on the server on Friday and Saturday. I have a case open with Premier Support!

June 11th, 2015 7:29am

Thanks clintcdsss, let us know if you get any good feedback from Premier.
Free Windows Admin Tool Kit Click here and download it now
June 11th, 2015 12:22pm

This must be a severe stumper for dev team.  Its been what, 2 and a half years?  For those having the issue, have you tried the workaround posted above (user policy reg key)?  It did work for "better" for me and some others as well.
June 12th, 2015 5:30pm

OMG known issue. Can't wait to try the fix tomorrow!

1. Disable both tasks with the command lines to exit the provisioning mode

2. Add a run command line task immediately after Setup Windows and ConfigMgr task

3. Set the following command line : sc.exe config ccmexec start= delayed-auto

4. Following that, create a Restart Computer task.

Free Windows Admin Tool Kit Click here and download it now
June 17th, 2015 11:03am

Seems I might have found another nasty side effect of this bug.  Windows Embedded 8.1 Industry Pro...  When the client flushes, it triggers a servicing mode restart.  The client is in the dormant state but the restart command persists.  The results in the machine sitting in servicing mode indefinitely.  Absolutely frustrating.
September 3rd, 2015 10:15am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics