System Center 2012 R2 Endpoint Protection Hanging on full scans

Since Monday we multiple machines trying to run a full scan. The scan will get so far and hang up on random files. I have watched the scans and when it hangs it shows the file and the file count stops. When this happens we get the following every 30 seconds to a minute in the system log "A timeout (30000 milliseconds) was reached while waiting for a transaction response from the MsMpSvc service." If you try to do anything with Endpoint Protection after this point it freezes up" You end up having to reboot the computer. Since it never completes the full scan it starts over and does the same thing after logging in.

Our computer turn on early in the morning so some of the computers start running this scan. By the time the users come to work and tries to log in the login just sets and spins. You have to physically turn the computer off and back on in order to log in.

Is any one else having these problems?

August 25th, 2015 11:17am

We are also having the exact same issue describe above plus some other issues. Some machines will sit at the starting windows screen forever, others will sit at the welcome screen.
Free Windows Admin Tool Kit Click here and download it now
August 25th, 2015 3:43pm

As an update in event viewer there are a ton of errors of the following:

"A timeout (30000 millieseconds) was reached while waiting for a transaction response from the MsMpSvc Service." Event 7011.

This starts on August 23rd at 2:10am. It grabbed the following:

"Current Signature Version: 1.205.188.0"

"Current Engine Version "1.1.1200.2"

This appears to be 100% bad definition update.

August 25th, 2015 4:21pm

Here is the message about AV updating:

Free Windows Admin Tool Kit Click here and download it now
August 25th, 2015 5:08pm

And it breaking:

August 25th, 2015 5:09pm

We are also having the exact same issue by one of our customers since Monday.

AV-Def: 1.205.320.0
Spy-Def: 1.205.320.0

Also, latest MS-Updates (last week) are installed.

Free Windows Admin Tool Kit Click here and download it now
August 26th, 2015 5:05am

I have opened a ticket with Microsoft. I have also turned off all scheduled scans until everything gets back to normal. I will post what Microsoft says as soon as I know something but I have yet to here from them.
August 26th, 2015 7:14am

many thanks! I'm curious about your findings.
Free Windows Admin Tool Kit Click here and download it now
August 26th, 2015 7:17am

I have reported the same issue to Premier support. Awaiting their response.
August 26th, 2015 9:28am

Thanks for sharing. We are experiencing the same symptoms as well. Please share your findings after working with Microsoft Support.
Free Windows Admin Tool Kit Click here and download it now
August 26th, 2015 12:26pm

We are having the same issue with one of our file servers, we have had to restart the server once a day to keep it going. I have changed the real-time settings to only Monitor incoming files rather than all files and the CPU usage has stayed at a constant low now, I will also be logging a call with MS Premier support

August 26th, 2015 7:19pm

Same problem for us, started appearing last Saturday (22/08/2015) morning, we run a full scan every Saturday at 02:00.

Agent details of 1 of our affected machines:-

Antimalware Client Version: 4.5.216.0
Engine Version: 1.1.12002.0
Antivirus definition: 1.205.659.0
Antispyware definition: 1.205.659.0
Network Inspection System Engine Version: 2.1.11804.0
Network Inspection System Definition Version: 115.3.0.0

Free Windows Admin Tool Kit Click here and download it now
August 27th, 2015 1:43am

We also are having the error: "A timeout (30000 millieseconds) was reached while waiting for a transaction response from the MsMpSvc Service." It happened since the beginning of this week.

I thought reinstalling the sccm client was the solution but it helps for only 1 day. The next day the problem is back. We don't do scheduled full scans so I have no clue what triggers the error. I hope there will be a fix very soon.

August 27th, 2015 4:37am

My company is also experiencing issues, I've had to fix three PCs already today. Users get stuck at the 'Welcome' logging on screen for at least 15-20 minutes. Any access to network resources appears extremely delayed, non responsive even.

I'm having to use a third party tool called RunAsTI which runs as command prompt as TrustedInstaller, from there I run Process Explorer, from the services tab of the MsMpEng process I can stop the process from running (eventually!). After this I can uninstall the Endpoint Protection and the System Center client. Reboot then reinstall System Center client, which in turn installs the AV.

I've also noticed when I manage to forcibly close the MsMpEng process I get a failed definition update event log:

Installation Failure: Windows failed to install the following update with error 0x80246007: Definition Update for Microsoft Endpoint Protection - KB2461484 (Definition 1.205.681.0).

After reinstalling System Center this update installs with no problems and the PC itself seems happy again. I don't plan on doing this for 500 PCs so it would be good if Microsoft could fix this issue.

Free Windows Admin Tool Kit Click here and download it now
August 27th, 2015 7:46am

I'm so glad I'm not the only one having this problem.

We are experiencing this problem as well and have no solutions up until now.

Waiting for the response of microsoft support as well.

August 27th, 2015 8:35am

It appears that KB3087985 is the culprit.  I removed it from 3 machines and the MsMpSvc Service timeout errors no longer appear.  Scans are running normally.  My short term plan is to change scan policies and turn off scans entirely and hope for a quick resolution from Microsoft, rather than remove the security update.

Did you make sure your AV signatures did not update on reboot? We had that update pushed out on the 20th and no issues until the 23rd right after a AV update.  

Free Windows Admin Tool Kit Click here and download it now
August 27th, 2015 9:23am

It appears that KB3087985 is the culprit.  I removed it from 3 machines and the MsMpSvc Service timeout errors no longer appear.  Scans are running normally.  My short term plan is to change scan policies and turn off scans entirely and hope for a quick resolution from Microsoft, rather than remove the security update.


well, in my case I don't have this update installed on the affected server, I also rather think it has something to do with the SCEP version, at first my version was the buggy one (4.8.204.0) but in this thread there seems to be another versions as well...
August 27th, 2015 9:29am

We are using anti-malware version 4.7.209.0.

So far the following 3 versions have been listed here as having issues: 

-4.7.209.0

-4.8.204.0

-4.5.216.0

Free Windows Admin Tool Kit Click here and download it now
August 27th, 2015 9:54am

interessting, that makes me think that it couldn't be the client version rather than other components?

One server who had this problem has these SCEP Definition

maybe someone else can recognize some similarities of any versions?


  • Edited by DTICT 17 hours 18 minutes ago
August 27th, 2015 10:02am

nevermind, I just now realized there are information already posted in this topic. I think it has to be the engine version 1.1.12002.0

I also restored from last week and it was on 1.1.11903.0

now the question of how to downgrade the engine version....

Free Windows Admin Tool Kit Click here and download it now
August 27th, 2015 10:36am

I spoke too soon on this one.  Knowing that KB3087985 was the last patch that was deployed to our environment I looked at it as a possibility, but this morning the SCEP problem is back. 

A scan kicked off on my workstation and about an hour later I was getting timeout errors again.  This was preceded by a lot of dynamic signature update retrievals, which were preceded by dynamic signature update deletions, all of which have the same odd timestamp of 1/1/1601 12:02:24 AM. 

August 27th, 2015 10:41am

We have switched to quick scan as a workaround. This seems to work well while we investigate.

I am not sure if this is a coincidence, but the scan stops in \windows\winsxs\backup on several systems i have checked.

Edit: it stops on files in other folders as well, so it must be a coinci

Free Windows Admin Tool Kit Click here and download it now
August 27th, 2015 12:12pm

interessting, that makes me think that it couldn't be the client version rather than other components?

One server who had this problem has these SCEP Definition

maybe someone else can recognize some similarities of any versions?


  • Edited by DTICT Thursday, August 27, 2015 2:02 PM
August 27th, 2015 2:01pm

interessting, that makes me think that it couldn't be the client version rather than other components?

One server who had this problem has these SCEP Definition

maybe someone else can recognize some similarities of any versions?


  • Edited by DTICT Thursday, August 27, 2015 2:02 PM
Free Windows Admin Tool Kit Click here and download it now
August 27th, 2015 2:01pm

interessting, that makes me think that it couldn't be the client version rather than other components?

One server who had this problem has these SCEP Definition

maybe someone else can recognize some similarities of any versions?


  • Edited by DTICT Thursday, August 27, 2015 2:02 PM
August 27th, 2015 2:01pm

interessting, that makes me think that it couldn't be the client version rather than other components?

One server who had this problem has these SCEP Definition

maybe someone else can recognize some similarities of any versions?


  • Edited by DTICT Thursday, August 27, 2015 2:02 PM
Free Windows Admin Tool Kit Click here and download it now
August 27th, 2015 2:01pm

Has anyone heard back from MS Support?

Also, what OSes are involved?  Anecdotally, I can't replicate the issue on my Windows 8.1 workstation but have had the issue reported to me on a couple of Windows 7 workstations.  Is it just Windows 7 and Server 2012?

August 27th, 2015 4:38pm

Had the same issue today looking forward to hearing what Microsoft says
Free Windows Admin Tool Kit Click here and download it now
August 27th, 2015 5:21pm

And to add to that...

I have the scan freezing w/ various definitions...

1.201.681

1.201.538

1.201.415

This problem is causing major productivity issues in my office.  When the SCEP hangs...  the client can no longer save work properly either locally or over the network.  It's as if their entire workstation comes to a stand still.  Items that take seconds, take minutes.  Software that ties to the network, software that teams rely on to work together, lock up, bringing them all to a halt, until the offender reboots his workstation. 

Something changed over the weekend and we need a resolution ASAP.

August 27th, 2015 5:59pm

Hi all, same issue in our company.

Several people having trouble saving/opening files on the network and outlook is not working good.

Scep is locking up, when you try to do a reboot the computer is hanging, only the trick with the button is working to get it restarted.

Now disabled all full and fast scan on computers.

Thx for the help

Free Windows Admin Tool Kit Click here and download it now
August 28th, 2015 5:11am

Has anyone heard back from MS Support?

Also, what OSes are involved?  Anecdotally, I can't replicate the issue on my Windows 8.1 workstation but have had the issue reported to me on a couple of Windows 7 workstations.  Is it just Windows 7 and Server 2012?

still waiting for the answer of microsoft, they said their "escalation manager" is working on it...

windows 2008 r2 is involved on my side

August 28th, 2015 5:11am

Microsoft support is suspecting this update KB3076895 is causing a deadlock when scanning.

I have initiated tests after removing this patch.

Free Windows Admin Tool Kit Click here and download it now
August 28th, 2015 6:40am

Microsoft support is suspecting this update KB3076895 is causing a deadlock when scanning.

I have initiated tests after removing this patch.

August 28th, 2015 7:40am

Excellent.  I'm going to look into whether affected workstations have that patch, espensh.  Then maybe try some experimentation.

Thanks!

Free Windows Admin Tool Kit Click here and download it now
August 28th, 2015 8:45am

The first test had a positive result. The full scan complete without the service freezing. It failed last evening before removing the patch.
August 28th, 2015 9:09am

Frist off, thanks for everyone's valuable input.  I have a case with Microsoft, but haven't received much assistance yet.  

I removed KB3076895 from my test machine and the re-ran scan that froze yesterday, it passed today since removal of that patch. 

I want to run a few more tests, but then the next tasks would be to figure a way to automate the removal of this KB3076895 from 275+ workstations. 

And also remove it from SCCM from re-distributing it. 

We've always had our SCCM in set-it and forget-it mode for the last year when it came to Endpoint and Window Updates. 

At least we are now making some progress it seems. 

Free Windows Admin Tool Kit Click here and download it now
August 28th, 2015 10:21am

This is the case on several computers is my company as well. Not sure why it doesnt apply to all of them, but at least 4-500 computers are affected. It seems like uninstall of update KB3076895 (and reboot) resolves the case. Thanks for sharing!
August 28th, 2015 11:48am

Does this look like the same issue with Symantec AV?  http://answers.microsoft.com/en-us/windows/forum/windows_7-update/issues-with-update-kb3076895-released-on-13082015/f887adce-9183-4d3d-86fb-2d01a9305981?page=1
Free Windows Admin Tool Kit Click here and download it now
August 28th, 2015 4:24pm

https://isc.sans.edu/forums/diary/Microsoft+patch+tuesday+problem+with+Symantec+Cloud+Endpoint+protection/20037/ ?
August 28th, 2015 4:25pm

Looks like we have a HotFix from Microsoft for KB3076895 on Windows 7 and Windows Server 2008R2.

https://support.microsoft.com/en-us/kb/3090303

I haven't had a chance to test it yet though.  Comes with the usual caveats from Microsoft, "proceed with caution and only deploy to machines actively running into the problem and whatnot."

Free Windows Admin Tool Kit Click here and download it now
August 29th, 2015 9:41am

Looks like we have a HotFix from Microsoft for KB3076895 on Windows 7 and Windows Server 2008R2.

https://support.microsoft.com/en-us/kb/3090303

I haven't had a chance to test it yet though.  Comes with the usual caveats from Microsoft, "proceed with caution and only deploy to machines actively running into the problem and whatnot."

  • Proposed as answer by superbug73 15 hours 15 minutes ago
August 29th, 2015 1:41pm

Looks like we have a HotFix from Microsoft for KB3076895 on Windows 7 and Windows Server 2008R2.

https://support.microsoft.com/en-us/kb/3090303

I haven't had a chance to test it yet though.  Comes with the usual caveats from Microsoft, "proceed with caution and only deploy to machines actively running into the problem and whatnot."

  • Proposed as answer by superbug73 Monday, August 31, 2015 4:04 PM
Free Windows Admin Tool Kit Click here and download it now
August 29th, 2015 1:41pm

Looks like we have a HotFix from Microsoft for KB3076895 on Windows 7 and Windows Server 2008R2.

https://support.microsoft.com/en-us/kb/3090303

I haven't had a chance to test it yet though.  Comes with the usual caveats from Microsoft, "proceed with caution and only deploy to machines actively running into the problem and whatnot."

  • Proposed as answer by superbug73 Monday, August 31, 2015 4:04 PM
August 29th, 2015 1:41pm

Looks like we have a HotFix from Microsoft for KB3076895 on Windows 7 and Windows Server 2008R2.

https://support.microsoft.com/en-us/kb/3090303

I haven't had a chance to test it yet though.  Comes with the usual caveats from Microsoft, "proceed with caution and only deploy to machines actively running into the problem and whatnot."

  • Proposed as answer by superbug73 Monday, August 31, 2015 4:04 PM
Free Windows Admin Tool Kit Click here and download it now
August 29th, 2015 1:41pm

Looks like we have a HotFix from Microsoft for KB3076895 on Windows 7 and Windows Server 2008R2.

https://support.microsoft.com/en-us/kb/3090303

I haven't had a chance to test it yet though.  Comes with the usual caveats from Microsoft, "proceed with caution and only deploy to machines actively running into the problem and whatnot."

Yes, i just got the information from Microsoft :)
August 31st, 2015 4:17am

Looks like we have a HotFix from Microsoft for KB3076895 on Windows 7 and Windows Server 2008R2.

https://support.microsoft.com/en-us/kb/3090303

I haven't had a chance to test it yet though.  Comes with the usual caveats from Microsoft, "proceed with caution and only deploy to machines actively running into the problem and whatnot."

did anybody have success with this hotfix and didnt remove KB3076895?
Free Windows Admin Tool Kit Click here and download it now
August 31st, 2015 5:25am

We do not install the HotFix KB3090303 or uninstall KB3076895.

We just downgrade SCEP to version 4.3.215.0. It seems that this works for us, until Microsoft has a better solution for it.

Update:

Answer from Microsoft today ... to install the Update above... we will do it now.

Based on memory dump analysis (dump provided by another customer), the hang issue is likely caused by the following security update :
fix KB3076895/ MS15-084  (https://www.microsoft.com/en-us/download/details.aspx?id=48320).
The above XML security fix contains code that causes deadlock related to loader lock and produce applications hang.

There is a hotfix available for Windows 7 :
https://support.microsoft.com/en-us/kb/3090303

[abstract]
Windows freezes or applications freeze after you install security update 3076895

<...>
Microsoft Common Antimalware Platform (CAMP) Server stops responding.
Microsoft Forefront Endpoint Protection (FEP) Application stops responding.
<...>

A GDR (Genral Distribution Release) of the fix will be available later this week.

  • Edited by bkliever 17 hours 53 minutes ago
August 31st, 2015 7:42am

We applied the hotfix mentioned above and it appears to have fixed our problems.
Free Windows Admin Tool Kit Click here and download it now
August 31st, 2015 8:36am

We do not install the HotFix KB3090303 or uninstall KB3076895.

We just downgrade SCEP to version 4.3.215.0. It seems that this works for us, until Microsoft has a better solution for it.

Update:

Answer from Microsoft today ... to install the Update above... we will do it now.

Based on memory dump analysis (dump provided by another customer), the hang issue is likely caused by the following security update :
fix KB3076895/ MS15-084  (https://www.microsoft.com/en-us/download/details.aspx?id=48320).
The above XML security fix contains code that causes deadlock related to loader lock and produce applications hang.

There is a hotfix available for Windows 7 :
https://support.microsoft.com/en-us/kb/3090303

[abstract]
Windows freezes or applications freeze after you install security update 3076895

<...>
Microsoft Common Antimalware Platform (CAMP) Server stops responding.
Microsoft Forefront Endpoint Protection (FEP) Application stops responding.
<...>

A GDR (Genral Distribution Release) of the fix will be available later this week.

  • Edited by bkliever Monday, August 31, 2015 1:26 PM
August 31st, 2015 11:41am

We do not install the HotFix KB3090303 or uninstall KB3076895.

We just downgrade SCEP to version 4.3.215.0. It seems that this works for us, until Microsoft has a better solution for it.

Update:

Answer from Microsoft today ... to install the Update above... we will do it now.

Based on memory dump analysis (dump provided by another customer), the hang issue is likely caused by the following security update :
fix KB3076895/ MS15-084  (https://www.microsoft.com/en-us/download/details.aspx?id=48320).
The above XML security fix contains code that causes deadlock related to loader lock and produce applications hang.

There is a hotfix available for Windows 7 :
https://support.microsoft.com/en-us/kb/3090303

[abstract]
Windows freezes or applications freeze after you install security update 3076895

<...>
Microsoft Common Antimalware Platform (CAMP) Server stops responding.
Microsoft Forefront Endpoint Protection (FEP) Application stops responding.
<...>

A GDR (Genral Distribution Release) of the fix will be available later this week.

  • Edited by bkliever Monday, August 31, 2015 1:26 PM
Free Windows Admin Tool Kit Click here and download it now
August 31st, 2015 11:41am

We do not install the HotFix KB3090303 or uninstall KB3076895.

We just downgrade SCEP to version 4.3.215.0. It seems that this works for us, until Microsoft has a better solution for it.

Update:

Answer from Microsoft today ... to install the Update above... we will do it now.

Based on memory dump analysis (dump provided by another customer), the hang issue is likely caused by the following security update :
fix KB3076895/ MS15-084  (https://www.microsoft.com/en-us/download/details.aspx?id=48320).
The above XML security fix contains code that causes deadlock related to loader lock and produce applications hang.

There is a hotfix available for Windows 7 :
https://support.microsoft.com/en-us/kb/3090303

[abstract]
Windows freezes or applications freeze after you install security update 3076895

<...>
Microsoft Common Antimalware Platform (CAMP) Server stops responding.
Microsoft Forefront Endpoint Protection (FEP) Application stops responding.
<...>

A GDR (Genral Distribution Release) of the fix will be available later this week.

  • Edited by bkliever Monday, August 31, 2015 1:26 PM
August 31st, 2015 11:41am

We just tested the hotfix on 2 PCs and it has resolved the issues for us as well.

Now to roll out to all the other affected PC's...

Thanks everyone!

Free Windows Admin Tool Kit Click here and download it now
August 31st, 2015 12:03pm

We just tested the hotfix on 2 PCs and it has resolved the issues for us as well.

Now to roll out to all the other affected PC's...

Thanks everyone!

  • Edited by superbug73 Monday, August 31, 2015 4:04 PM
August 31st, 2015 4:02pm

We just tested the hotfix on 2 PCs and it has resolved the issues for us as well.

Now to roll out to all the other affected PC's...

Thanks everyone!

  • Edited by superbug73 Monday, August 31, 2015 4:04 PM
Free Windows Admin Tool Kit Click here and download it now
August 31st, 2015 4:02pm

We just tested the hotfix on 2 PCs and it has resolved the issues for us as well.

Now to roll out to all the other affected PC's...

Thanks everyone!

  • Edited by superbug73 Monday, August 31, 2015 4:04 PM
August 31st, 2015 4:02pm

This should be fixed in KB3092627 (which is also distributed via WSUS)

https://support.microsoft.com/de-de/kb/3092627

  • Proposed as answer by Rotronic 21 minutes ago
Free Windows Admin Tool Kit Click here and download it now
September 2nd, 2015 3:02am

This should be fixed in KB3092627 (which is also distributed via WSUS)

https://support.microsoft.com/de-de/kb/3092627

  • Proposed as answer by Rotronic Wednesday, September 02, 2015 7:00 AM
September 2nd, 2015 7:00am

This should be fixed in KB3092627 (which is also distributed via WSUS)

https://support.microsoft.com/de-de/kb/3092627

  • Proposed as answer by Rotronic Wednesday, September 02, 2015 7:00 AM
Free Windows Admin Tool Kit Click here and download it now
September 2nd, 2015 7:00am

This should be fixed in KB3092627 (which is also distributed via WSUS)

https://support.microsoft.com/de-de/kb/3092627


This has worked for us so far.
September 2nd, 2015 9:02am

The above hotfix (KB3090303) took care of our earlier problem but now we are seeing something new 

I am now noticing on some machines when they run a full scan and hit a recovery partition on the machine called (Q:\FactoryRecovery\cdrivebackup.wim) it is crashing the Antimalware service and not starting back up. This recovery partition is common to Lenovo products and every machine with this partition is crashing the Antimalware service. It never did this in the past. Anyone else see this problem with recovery partitions, drives or directories.

Free Windows Admin Tool Kit Click here and download it now
September 3rd, 2015 7:54am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics