PKI Client starts to intialize, then 7 minutes later client agents go back to disabled for upwards of an hour
Starting a new thread on this as I have done much digging and no longer believe its a conflicting record issue.
I am SCCM 2012 SP1 on Server 2008 R2/SQL 2012 CU2. My "Lan" based management point is HTTP and I also have an Internet Management point that is HTTPS. All scenerios discussed in this thread are installing the 2012 SP1 client while connected
to the Lan. We have a fully functional PKI infrastructure with auto enrol enabled.
The issue at hand:
Wether I install the client during OSD, or manually install the client I get the same result. This is that the client agent upon successful install begins communicating with the HTTP management point and starts retrieving policy. If I open the
ConfigMgr applet in Control Panel I see the client shows "Client Certificate: PKI", "Connection Type: Currently Intranet". If I view actions I see all actions with the exception of Discovery Data Collection Cycle and Hardware Inventory
Cycle.
I have watched the client logs the only thing that seems to stick out is in the CcmNotificationAgent.log which shows the bgb client agent actions. it repeats the following aprox every 5 minutes:
bgb client agent is starting...
bgb client agent is disabled
TCP Listener is disabled
bgbController main thread us started with settings: [bgb enable = 0], {tcp enable = 0} and {http enable = 0}.
Wait 3600 seconds for even notification
The ClientIDManagerStartup.log files shows:
PopulateRegistrationHint: Using the Certificate selected by the current version of SCCM to set the hint. ClientIDManagerStartup 1/23/2013 5:00:51 PM 2100 (0x0834)
CCMCreateAuthHeadersEx failed (0x80004005). ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
PopulateRegistrationHint failed (0x80004005), expected upon first start of non-upgrade client. ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Begin searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Finding certificate by issuer chain returned error 80092004 ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Completed searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Unable to find any Certificate based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Raising event:
instance of CCM_ServiceHost_CertRetrieval_Status
{
ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
DateTime = "20130123230052.204000+000";
HRESULT = "0x87d00215";
ProcessID = 2352;
ThreadID = 2100;
};
ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Failed to submit event to the Status Agent. Attempting to create pending event. ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Raising pending event:
instance of CCM_ServiceHost_CertRetrieval_Status
{
ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
DateTime = "20130123230052.204000+000";
HRESULT = "0x87d00215";
ProcessID = 2352;
ThreadID = 2100;
};
ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
PKI Client Certificate matching SCCM certificate selection criteria is not available. ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Generated a new Signing certificate ClientIDManagerStartup 1/23/2013 5:00:54 PM 2100 (0x0834)
Generated a new Encryption certificate ClientIDManagerStartup 1/23/2013 5:00:54 PM 2100 (0x0834)
Initializing registration renewal for potential PKI issued certificate changes. ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
Succesfully intialized registration renewal. ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
[RegTask] - Executing registration task synchronously. ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
Read SMBIOS (encoded): 32003100350039002D0039003900320037002D0032003500360036002D0038003300320035002D0037003500340032002D0035003200370031002D0033003900 ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
Evaluated SMBIOS (encoded): 32003100350039002D0039003900320037002D0032003500360036002D0038003300320035002D0037003500340032002D0035003200370031002D0033003900 ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
No SMBIOS Changed ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
SMBIOS unchanged ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
SID unchanged ClientIDManagerStartup 1/23/2013 5:00:56 PM 2348 (0x092C)
HWID unchanged ClientIDManagerStartup 1/23/2013 5:00:57 PM 2348 (0x092C)
Windows To Go requires a minimum operating system of Windows 8 ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
GetSystemEnclosureChassisInfo: IsFixed=TRUE, IsLaptop=FALSE ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
Windows To Go requires a minimum operating system of Windows 8 ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
Computed HardwareID=2:9C8C08C4B3E16249A2F1457998D16528B656DE30
Win32_SystemEnclosure.SerialNumber=2159-9927-2566-8325-7542-5271-39
Win32_SystemEnclosure.SMBIOSAssetTag=9344-3677-7824-5579-3797-0729-37
Win32_BaseBoard.SerialNumber=2159-9927-2566-8325-7542-5271-39
Win32_BIOS.SerialNumber=2159-9927-2566-8325-7542-5271-39
Win32_NetworkAdapterConfiguration.MACAddress=00:15:5D:0B:78:20 ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
[RegTask] - Client is not registered. Sending registration request for GUID:b4aacc70-6de3-4829-88e0-498777c49379 ... ClientIDManagerStartup 1/23/2013 5:00:59 PM 2348 (0x092C)
[RegTask] - Client registration is pending. Server assigned ClientID is GUID:b4aacc70-6de3-4829-88e0-498777c49379 ClientIDManagerStartup 1/23/2013 5:01:00 PM 2348 (0x092C)
[RegTask] - Sleeping for 60 seconds ... ClientIDManagerStartup 1/23/2013 5:01:00 PM 2348 (0x092C)
RenewalTask: Executing renewal task. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Begin searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Based on Certificate Issuer 'MyrootCA' found Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Completed searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Begin to select client certificate ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
>>> Client selected the PKI Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Raising event:
instance of CCM_ServiceHost_CertRetrieval_Status
{
ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
DateTime = "20130123230143.106000+000";
HRESULT = "0x00000000";
ProcessID = 2352;
ThreadID = 2620;
};
ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Client PKI cert is available. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
RenewalTask: Certificate has changed, initiating a renewal. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Aborting any pending registration. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
Re-registration/renewal initiated. Restarting the service. ClientIDManagerStartup 1/23/2013 5:01:43 PM 2620 (0x0A3C)
[----- SHUTDOWN -----] ClientIDManagerStartup 1/23/2013 5:01:44 PM 2100 (0x0834)
[----- STARTUP -----] ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Machine: W21599927256 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
OS Version: 6.1 Service Pack 1 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
SCCM Client Version: 5.00.7804.1000 ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Client is set to use HTTPS when available. The current state is 448. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Based on Certificate Issuer 'MyrootCA' found Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Completed searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin to select client certificate ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
>>> Client selected the PKI Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Raising event:
instance of CCM_ServiceHost_CertRetrieval_Status
{
ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
DateTime = "20130123230145.722000+000";
HRESULT = "0x00000000";
ProcessID = 3612;
ThreadID = 3888;
};
ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Failed to submit event to the Status Agent. Attempting to create pending event. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Raising pending event:
instance of CCM_ServiceHost_CertRetrieval_Status
{
ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
DateTime = "20130123230145.722000+000";
HRESULT = "0x00000000";
ProcessID = 3612;
ThreadID = 3888;
};
ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
'RDV' Identity store does not support backup. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
CCM Identity is in sync with Identity stores ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Based on Certificate Issuer 'MyrootCA' found Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Completed searching client certificates based on Certificate Issuers ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin to select client certificate ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Begin validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Completed validation of Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
>>> Client selected the PKI Certificate [Thumbprint 52BFCF407B7071512CE65F8868D66578244ABDD9] issued to 'myclient.mydomain.com' ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Raising event:
instance of CCM_ServiceHost_CertRetrieval_Status
{
ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
DateTime = "20130123230145.752000+000";
HRESULT = "0x00000000";
ProcessID = 3612;
ThreadID = 3888;
};
ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Failed to submit event to the Status Agent. Attempting to create pending event. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Raising pending event:
instance of CCM_ServiceHost_CertRetrieval_Status
{
ClientID = "GUID:b4aacc70-6de3-4829-88e0-498777c49379";
DateTime = "20130123230145.752000+000";
HRESULT = "0x00000000";
ProcessID = 3612;
ThreadID = 3888;
};
ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Client PKI cert is available. ClientIDManagerStartup 1/23/2013 5:01:45 PM 3888 (0x0F30)
Initializing registration renewal for potential PKI issued certificate changes. ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
Succesfully intialized registration renewal. ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
[RegTask] - Executing registration task synchronously. ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
Read SMBIOS (encoded): 32003100350039002D0039003900320037002D0032003500360036002D0038003300320035002D0037003500340032002D0035003200370031002D0033003900 ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
Evaluated SMBIOS (encoded): 32003100350039002D0039003900320037002D0032003500360036002D0038003300320035002D0037003500340032002D0035003200370031002D0033003900 ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
No SMBIOS Changed ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
SMBIOS unchanged ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
SID unchanged ClientIDManagerStartup 1/23/2013 5:01:48 PM 3928 (0x0F58)
HWID unchanged ClientIDManagerStartup 1/23/2013 5:01:49 PM 3928 (0x0F58)
Windows To Go requires a minimum operating system of Windows 8 ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
GetSystemEnclosureChassisInfo: IsFixed=TRUE, IsLaptop=FALSE ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
Windows To Go requires a minimum operating system of Windows 8 ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
Computed HardwareID=2:9C8C08C4B3E16249A2F1457998D16528B656DE30
Win32_SystemEnclosure.SerialNumber=2159-9927-2566-8325-7542-5271-39
Win32_SystemEnclosure.SMBIOSAssetTag=9344-3677-7824-5579-3797-0729-37
Win32_BaseBoard.SerialNumber=2159-9927-2566-8325-7542-5271-39
Win32_BIOS.SerialNumber=2159-9927-2566-8325-7542-5271-39
Win32_NetworkAdapterConfiguration.MACAddress=00:15:5D:0B:78:20 ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
[RegTask] - Client is not registered. Sending registration request for GUID:b4aacc70-6de3-4829-88e0-498777c49379 ... ClientIDManagerStartup 1/23/2013 5:01:53 PM 3928 (0x0F58)
[RegTask] - Client registration is pending. Server assigned ClientID is GUID:b4aacc70-6de3-4829-88e0-498777c49379 ClientIDManagerStartup 1/23/2013 5:01:54 PM 3928 (0x0F58)
[RegTask] - Sleeping for 60 seconds ... ClientIDManagerStartup 1/23/2013 5:01:54 PM 3928 (0x0F58)
[RegTask] - Client registration is pending. Sending confirmation request for GUID:b4aacc70-6de3-4829-88e0-498777c49379 ... ClientIDManagerStartup 1/23/2013 5:02:54 PM 3928 (0x0F58)
[RegTask] - Client is registered. Server assigned ClientID is GUID:b4aacc70-6de3-4829-88e0-498777c49379. Approval status 2 ClientIDManagerStartup 1/23/2013 5:02:54 PM 3928 (0x0F58)
After almost 7 minutes exactly, the all the client agent actions besides Machine Policy Retrieval and UserPolicy Retrieval disappear (as if client policy was Reset)
If I let the client sit for 45-minutes to an hour, everything starts working again and works fine from there on out. (cycling the SMS Agent service nor rebooting the machine makes it recover until the 45-min to an hour pass)
My command line I am using to install the client is:
ccmsetup.exe /mp:MYLANBASEDMP /UsePKICert /NOCRLCheck CCMHOSTNAME="myinternetmp.mydomain.com" SMSSITECODE=P10 SMSCACHESIZE=7000 FSP=MYLANBASEDMP CCMLOGMAXSIZE=1000000
If I take out the /usePKICert, /NOCRLCheck and CCMHOSTNAME Entries, the client install and continues to function without issue.
Anyone have any others ideas on where to troubleshoot this issue? It would make more sense if the client NEVER worked after install. Tearing my hair out trying to figure out why it starts to intialize, then reverts, then comes back online and
works fine. This happens at both my primary site MP as well as my secondary site/mp. It happens on my standard Win7 image as well as Windows 8 test machines so I dont believe its a client OS issue.
January 24th, 2013 2:48am
I should add that this apears to only affect client installs where an object already exists in the SCCM database. On the machine in reference above, I can uninstall and reinstall the client over and over and witness the same behavior, if i
delete the object from the SCCM Database, then install the client the problem does not happen. the client comes online and stays online. :-/
January 24th, 2013 4:38am
More information:
I noticed that Windows Management Framwork 3.0 (beta) was installed on the primary site server. I uninstalled this version and installed the latest released version. I was then able to reinstall the client on the machine above and it did not
go dormant after 7 minutes and continued to work without issue (using the PKI stuff). I then rebuilt this machine completely and the client once again did not exhibit the behavior and continued work successfully.
I then pushed the client install to a Windows Server 2012 box and once again the client came online and stayed online.
Both of these clients are located at my secondary site server subnet using a proxy management point.
I also have a VMware client thats in the boundaries of the Primary Site server. I built the machine a few days ago to see if the same thing would happen in a different location and it did indeed go dormant after 7 minutes and after about an hour came
back online.
After my success against the secondary I rebuilt the VM in the Primary boundary and it still goes dormant after 7 minutes. :-/
I noticed on the 2 clients here that worked that the CcmNotificationAgent.log shows "bgbController main thread is started with settings: {bgb enabled = 1}, {tcp enabled = 1}, {tcp port = 10123} and {http enabled = 1}.
On the client that exhibits the behavior (the vm in the primary site boundaries) the CcmNotificationAgent.log shows "bgbController main thread is started with settings: {bgb enabled = 0}, {tcp enabled = 0}, {tcp port = 0} and {http enabled = 0}.
Can't say for sure that this is whats causing the behavior but it does appear to be consistent for client that "fail" and then recover.
Anyone have any suggestions on where to look next?
January 24th, 2013 8:14pm
Just a bit more info...just reinstalled the client again on one of the secondary machines where the behavior didnt reappear earlier and its back to going dormant after 7 minutes.
I have opened a case with Microsoft since I am getting nowhere fast. if I find a resolution I will post back to the forums.
I would however still LOVE to hear from anyone out there who has SCCM 2012 SP1 on wether they see this behavior or not..
January 24th, 2013 10:49pm
I've done some quick tests in my lab (http, not https) and do see the same behavior. I already contacted the product group but did not get any reply yet. Please update this thread if you got feedback from MS.
January 25th, 2013 10:57am
Thank you Torsten! Very glad to hear its not just me (and that i have not gone mad). I should be hearing from the CSS guy today so I'll certainly post back if/when we find the root cause. Not sure about you but it "feels" like it has something
to do with the new "Client Notification" system in SP1 (BGB Agent).
Thanks again for the reply and testing.
January 25th, 2013 5:35pm
Just got off the phone with MS Support and as expected, the gentleman had never seen this behavior. I captured 2 sets of logs from different machines and uploaded to them. He is going to escelate the issue with the logs for analysis.
Likely wont hear back until Monday.
January 25th, 2013 9:50pm
Just out of curiosity, if you check ccmeval.log is there any sort of client remediation taking place during this time?
January 26th, 2013 3:33am
Hi Adam, CcmEval.log is not yet created at the time the issue appears.
January 27th, 2013 11:29pm
William, thanks for posting this! I am having the exact same issue! It has been driving me nuts trying to figure it out. I was just able to validate, as a workaround, I can delete the record from database and reinstall client and it does
appear to resolve issue as best I can tell. I've had tons of clients go "Inactive" (Disabled) just like yours. I haven't been able to find a pattern to why some clients work and some don't, besides deleting records from database.
You found any other info yet?
January 29th, 2013 5:52pm
Hey bcehr, I am still waiting to hear back from microsoft support. I sent them client logs as well as ran a diagnostic util on the primary site server and uploaded the results. I did this on Friday. I sent an email yesterday asking for
a status and was told they are still working on it. Hoping I hear back today. :-/
January 29th, 2013 6:31pm
Hey, ok sounds good. Let us know what you find out. I'm also working on issue from my side, if I can find anything else, I'll post back an update.
January 29th, 2013 6:35pm
Just an update for anyone watching this thread..
I sent an email yesterday and asked for an update on the issue from the support person and never got a response. :-/ Just sent another email requesting an update. So at its stands, day 5 with CSS and still nothing to go on yet. I
will traveling the rest of the week so hoping to have something by the time I return on Monday.
January 30th, 2013 5:56pm
Just another update, finally got a response from the support tech who unfortunately is requesting more time. So no closer to a resolution at this time. I'll update again when i get more info.
February 5th, 2013 1:08am
William, thanks for keeping us in the loop. I also wanted to provide a little more info. I found that my secondary management servers, also have the same issue, but the workaround for them does not work, best I can tell, no way to fix those
machines. Are you seeing this too?
February 8th, 2013 4:21pm
I do see it at both my primary site and at my secondary site. I "thought" at one point that deleting the record from the database was the definitive workaround however have since then experienced the issue against a machine that was not already in
the DB there really doesnt seem to be any consistency with why/when it happens. I pushed 3 clients yesterday (upgrade to 2012 SP1 pushed via SCCM 2007) and 2 of 3 of then installed and stayed online, while the 3rd went dormat for about an hour and a
half before coming back online.
I have still yet to hear back from MS on my support case. If I inquire I am unfortunately told "we here at PRO and PRE are affected by an increased influx of case volume
so I will ask for some more time on this" so have no clue if/when we will actually work towards resolving this issue. :-/
February 8th, 2013 5:51pm
hello William,
i found your thread because i have a similar problem (maybe even the exact same), with the error message "Finding certificate by issuer chain returned error 80092004". i have a similar setup (lan MP with http and another internet MP with https).
when i do a client push install i see this in ClientIDManagerStartup.log:
i tried already a lot to fix this thing, and i'm short before reinstalling the entire PKI/cert things from scratch. so far i have found one workaround:
http://austrianalex.com/2012/10/system-center-configuration-manager-sccm-2012-client-pki-and-subordinate-ca-woes/
this actually works for me - we have our root CA certificate distributed through active directory to our clients, so when i remove the cert from SCCM primary site properties / client computer communication / trusted root certification authorities,
installation of the clients works again, like described in the link above.
however, i am not sure if this is a correct "solution" because i now got errors on other parts of the infrastructure (e.g. pxe driver installation), which may be because i do not have a trusted root cert set in SCCM....
i thought i share this here so maybe you want to try this on your end too.
EDIT:
after a lot of testing, this "solution" does not help me, as PXE no longer works without the trusted root CA cert set in site properties.
February 12th, 2013 3:16pm
hello again, i have done a lot of analysis in the last couple of days and i am now confident that i have the exact same issue as the OP. i opened a microsoft ticket too.
February 14th, 2013 2:40pm
Thanks for the post back guys. My case has been opened with MS Support for approaching a month now and unfortunately all I get back from the assigned engineer is that they are super busy and need more time. I have escelated to our AM who I
hope can spark a flame with the support team. The only other hope I have is that they are delaying response because the product team is working on a hotfix. Might be wishful thinking..
This is a show stopper issue for us so we cannot move our migration forward until this is resolved.
I'll post back if/when we ever start troubleshooting this issue with MS Support.
February 18th, 2013 6:36pm
A couple of quick questions:
Do you have a trusted root certification authority set up in the admin console?
If you do have this set up, are your client certificates issued by the specified trusted root CA?
February 18th, 2013 11:42pm
Hey Adam, yes we do have our Root CA configured and its the same CA that issues client certs. Once the client comes back "online" it works perfectly fine using PKI. We have succesfully tested the devices rolling over to the Internet MP when connected
a network other than our domain network. The problem only happens on intial client install (or reinstall). Once its dies (for a lack of a better term) it lays dormant for awhile but then eventually comes online and everything works as expected
and never goes dormant again.
Thanks!
February 19th, 2013 12:14am
Any news on this bug?
February 26th, 2013 2:01am
Not yet. I sent them a bunch of information on Client Settings last week. Still waiting on MS Support who have escelated the issue internally.
February 26th, 2013 2:20am
same here. no news from MS except that they are in delay.
February 26th, 2013 10:37am
update, a microsoft tech has been in touch with me yesterday, and we looked at the issue on our server for roughly 2 hours. no solution was found but they will try to replicate the issue internally and get back to me asap.
February 27th, 2013 12:46pm
Thanks for the update Robert..
February 27th, 2013 5:34pm
For those of you that are having this issue, what's the issuer name of the trusted root CA?
March 20th, 2013 3:16am
Adam, for me uses the default CA naming convention (domain-servername-CA); ie. CONTOSO-PKISRV-CA
March 20th, 2013 4:28am
CN = ABCD Root CA
DC = domain
DC = eu
(domain.eu is our internal DNS domain, ABCD is the company name, no special characters)
fyi, yesterday, microsoft tech has been on our server again, still no solution. i have requested this to be escalated to the dev team, to me it looks like a bug in sccm, especially after all that the MS tech has tried on our server.
March 20th, 2013 1:21pm
Robert, if you have configured a trusted root CA in the admin console, the client will only use a certificate for client authentication that was issued by that CA. This may explain what you were observing.
March 20th, 2013 9:16pm
On my side, I can confirm all my clients have a certificate from the CA that SCCM trusts. Also, I can confirm its not a certificate issue, because if I uninstall the SCCM agent, delete the record from SCCM database, then reinstall client, it works.
However, this is very inconvenient because this scenario can happen to any client.
March 20th, 2013 9:28pm
I can also confirm its not a trust issue with the cert. The Root CA I have configured in SCCM is the same CA that hands out certs to clients. As stated above, once the client "comes back" after letting it sit for a while it works fine from then
on out. This means it properly talks over HTTP to the Intranet MP/DP when on LAN and successfully flips over to HTTPS and talks to the Internet MP/DP when off LAN.
March 20th, 2013 9:50pm
Thanks to both of you for your detailed reports on the issues you are experiencing. Unfortunately, I'm out of ideas at this time as to what may be happening and I hope that you'll get the assistance that need from CSS to get to the bottom of what's
happening here.
March 21st, 2013 9:53pm
Thanks Adam. Appreciate the effort. :-)
March 21st, 2013 10:10pm
Any update on this guys?
April 8th, 2013 11:48pm
Had a PFE remoted in last week digging through more server logs and have sent additional verbose client logs. Still waiting :-/
April 9th, 2013 4:48pm
nothing new.
MS tech insisted that the problem must be our root certificate, so i should recreate it.
i contacted our hosting partner, who created the certificate and is using it for MS lync, and we decided not to go ahead with this as lync is working fine and the cert is showing no sign of problems. ticket back to MS...
April 9th, 2013 5:00pm
Our Root Cert comes from our internal CA which works fine for every other application we use it for. Given the fact that this issue also happen if I do not provide the IBCM settings during install and install only as Intranet sure doesn't sound like
a certificate issue to me, unless there is still cert checking stuff happening even when not using PKI for client communication?
April 9th, 2013 5:56pm
Not to mention that when the client does finally come back online, it works fine from there on out, including via PKI across the internet..Seems that if the cert itself was the issue then the client would never work properly..
April 9th, 2013 5:58pm
last update, microsoft asked me for a fresh set of logs, which i provided (http working state and https failed state). after checking through them, they again told me that it must be a problem with the certs, so we should recreate them. i still cannot
do this (root certs are in use by our lync infrastructure, all other certs i already recreated three times without effect), so finally management cancelled this project and i closed the ms ticket. bottom line, we can't use sccm 2012 for https/IBCM.
not to hijack this thread for the wrong reasons, i posted here because i get the same error messages as the thread OP in the client logs:
Certificate Issuer 1 [CN=myrootca; DC=mydomain; DC=com] ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834)
Finding certificate by issuer chain returned error 80092004 ClientIDManagerStartup 1/23/2013 5:00:52 PM 2100 (0x0834
so our issue may have been for the same reasons, maybe not.
i hope you find a solution for this, william, good luck!
April 23rd, 2013 4:33pm
Robert, in this specific case do you have a trusted root issuer configured in the admin console? If you do, the logging indicates that the trusted root issuer configured in the admin console hasn't issued a usable certificate on the client machine. See the
earlier post I made on that piece of the issue and see if that helps you any.
April 24th, 2013 12:12am
Hello Adam, yes we do (trusted root CA cert in console, as in your screenshot, and the client certs are issued by the same root CA through active directory enrollment)
this root CA cert is the cause of our issue. once we remove the root CA cert from sccm console, clients immediately detect their machine certificate as valid and start communicating with the MP without the slightest delay. sadly we can't leave it with
this, as pxe/osd now no longer works correctly. not going into too much detail here, we would need to fix the failure of sccm client to detect the machine cert as beeing valid one way or another, and MS tech's approach was to recreate the certs.
we recreated all the client and IIS certs many times without success, strictly following the MS docs. i cannot recreate the root CA cert because too much production systems are depending on it (lync 2010) and apart from that, the certificate chain validates
correctly from client cert to root CA cert so i don't see the issue here.
bottom line, we were running in circles with MS tech support for over a month, and have now cancelled this project to move on with other things.
April 24th, 2013 9:33am
Hi
Just want to inform you, and Microsoft if they are listening, this issue is not isolated to PKI environment. I havent got a PKI setup and i am experiencing the exact same issues. I have been battling with it for approx a month now and can find very little
info out there about, unitl that was unitil i found this post.
It is serously hindering my project, i am expecting to go 'LIVE' with SCCM based OS Deployment and then this starts happening, at first i thought it was becuase i was using ZTIWindowsUpdate.wsf as part of my task sequence so told everyone we cant use windows
update, but i can see now that windowd update it not the cause.
Could someone tell me how to log a call about this so i can also try to expediate things to get this resolved.
Cheers..
April 25th, 2013 7:25pm
Hi Torsten,
Im just wondering if you heard anything else regarding this ?
Cheers
April 26th, 2013 2:03pm
Hey Overfiend, I can tell you that my support engineer is still looking into it and have no additional updates at this time. Going on 3 months now with having this case open with Microsoft with no movement. :-/
April 26th, 2013 5:54pm
Hi William,
im sitting at a clinet right now and seeing this over and over again. The annoying thing is that its so random aswell, sometimes it 'appears' to be ok, then bang, it fails.
This is not good, it renders the computer unusable for my purpose.
I can't beleive its taken this long to fix..
Have you applied CU1 yet ?
April 26th, 2013 5:57pm
Yeah, we upgraded to CU1 this week. No change for this issue..
April 26th, 2013 6:45pm
Also seeing this same issue. Happens very randomly and as far as I can tell with no environment changes, or warnings for changes.
2012 Configuration Manager only environment, client has a the new SP1 agent. Was working just fine and even though there was no changes all of the sudden says "Client is set to use HTTPS when available. The current state is 448" and PKI none.
April 29th, 2013 8:57pm
Just another update to state we still have not received any movement from Microsoft on this issue. Its been over 3 months now and I am rapidly approaching the point where I just punt SCCM 2012 and any potential Windows 8 tablet projects we have in
the pipeline sadly and maybe try to revisit next year (assuming the issue has been identified and corrected by then of course). This sucks. :-) Anyone else hear anything by chance from MS on this?
May 1st, 2013 5:02pm
Hi William,
I had some difficulty logging a case with Microsoft so am looking here for answers and responses.
For the past two days i havent had any failures, all seems to be well. I had made one slight change to my setup, i was getting a lot of errors in my SMS_INVENTORY_DATA_LOADER log, stating that mesages or inventory data couldnt be processed due to the MAX
MIF Size being to small. The recommendation was to increase it larger than its default of 5MB (5000000), so i increased it to 20MB (20000000) just to get things working. Since then i havent had any errors. This could be just coincidence or good indeed be bad
data getting stored in the database for the client machines expereinceing the problems..
The reg key in question was ..HKLM\Software\Microsoft\SMS\Components\SMS_INVENTORY_DATA_LOADER, the key your looking for is Max MIF Size
Let me know if this works for you, or anyone else.
I only found this by looking at the errors being flagged up in my Sites Components Status, worked through trying to eliminate some of the errors that were appearing.
Cheers.
May 1st, 2013 6:31pm
Overfiend, are you saying you were having the issue of client installs coming online, then go offline for awhile and then come back on and start working (As is the basis for this thread) and when you increased your MIF size the client install issue disappeared?
Just want to make sure we are still talking about the same (original) issue.
If its any consolation, I am not getting inventory processing errors in SMS_INVENTORY_DATA_LOADER so assuming this is not the same issue (for me at least).
May 2nd, 2013 8:40pm
On my side, I still have the issue as well. I noticed today, the issue typically comes back when I either update for SCCM Agent, or reinstall Agent. If I ever do that, the agent gets Disabled like you have posted, and I have to uninstall, then delete
the record from the Database, then reinstall from scratch. Until someone finds a solution on this, I'm terrified to update to any CUs. You see this same behavior?
May 3rd, 2013 9:17pm
I see this behavior on almost every install, whether its a new install or a reinstall. In ALL cases however if I just let the machine sit, the client will come back online (typically about 45 minutes) and then work just fine from there on out.
Do your clients eventually recover bcehr or once they are disabled they never "self-correct"?
May 6th, 2013 4:32pm
Best I can tell, they don't recover, they stay stuck disabled forever. I have to delete them from database and reinstall to get them to wake up. The one way that agents do work correctly are ones that are installed via OSD, best I can tell,
those agents work immediately and continue to work if untouched. I don't do many "new" installs outside of OSD, so not sure if those might break too.
May 6th, 2013 4:35pm
Hmm, well ok, that different then from what I am seeing. :-/ On your side have you checked the conflicting records node to ensure there isn't an issue there? They are supposed to auto merge for any domain joined clients who's hardware
hasn't changed but might be worth a look just in case...
May 7th, 2013 12:36am
Hi.
Just got word from Microsoft support that they have raised a hotfix for this problem. It's beeing tested out at some other customer now. They should get back to me in a couple of weeks.
I'll keep up updated.
Bjrn Erik
May 22nd, 2013 10:50pm
Seeing exactly the same issue here in a PKI only environment. Root CA is configured in site properties and is also present on clients under Trust Root Certification Authorities. Certificates issued to the SCCM environment, including server and
client certificates are issues by subordinates. Be keen to see a fix for this...as the OP said the client does come back to life within around an hour but this means that any machine rebuilt for a customer is missing key software for quite some time
after build.
May 23rd, 2013 12:09pm
I can also confirm that this has now been submitted to the Product Team as a bug and will be corrected with a hotfix or CU. Now we just wait. :-/
May 29th, 2013 12:47am
That's good news. We are stuck until we have a resolution. Any idea if this will be just a client hotfix?
May 29th, 2013 11:06am
Nothing official on what systems the hotfix will apply to. If I were a betting man, I would bet its a server hotfix but only time will tell.
May 29th, 2013 4:13pm
Unfortunately no. This appears to fix an issue with having a 3rd party cert with the & symbol in it causing clients to not ever become assigned.
May 30th, 2013 9:33pm
This fix makes some changes to code responsible for site assignment and HTTPS certificate publishing into AD. I would recommend trying it out to see if it manages to resolve any of your issues.
It contains both client and server-side fixes.
May 31st, 2013 10:52pm
If anyone else tries this please post back your results. I am going to hold out for the "official" hot fix for my particular issue from MS. ;-)
June 3rd, 2013 9:30pm
Did MS give you any sort of ETA for hotfix or CU that will contain fix?
June 3rd, 2013 9:34pm
Unfortunately no. I was simply told that the issue has been identified as a bug and that the product team was working on a fix that would be released sometime in the future as a hotfix or as part of a CU. :-/
June 4th, 2013 4:44pm
Any hotfix ? I've spent days and days on this and finally found this thread.
MS support person working on this hasn't got anywhere yet.
Either client registration works (clear the trust root cert in site client communication tab) and pxe fails, or the other way around.
:(
June 27th, 2013 5:44pm
The last update I got was that this was identified as a bug and will be released with CU3. No ETA on when CU3 will be released. CU2 was just released a few days ago so I would think several weeks before CU3. :-/
June 27th, 2013 5:55pm
William, I want to follow up internally to make sure this issue is being tracked properly. Could you let me know what the ticket number is for your incident?
July 2nd, 2013 1:46am
Hey Adam, the Incident number is "113012410165068". The incident was "closed" several weeks ago however and I was told the resolution is that it's been identified and to wait for a fix.
The information I gave above was from another MS employee who I met during a sessions at the local MS office here in STL who has been kind enough to dig around for information for me.
July 2nd, 2013 12:14pm