Hey Guys, I have a 3 server Exchange 2010 SP2 RU2 DAG. All three servers yesterday had store.exe crash on them repeatedly. Yesterday was a holiday here, so there wasn't much going on so I'm trying to figure out what could've triggered this. The only entries in the System logs are "Exchange Information service terminated uenxpectedly" entries that happened every few minutes for 2 hours with a 30 minute gap at one point. The application logs show that store.exe crashed: Faulting application name: store.exe, version: 14.2.283.3, time stamp: 0x4f14dc0c Faulting module name: VSAPIScanSourceX64.dll, version: 7.0.1425.0, time stamp: 0x4ee7f77f There's not much to go on that was happening prior to these issues. The first time it happened, there were a couple of warnings 3 minutes prior to the crash: "folder is being scoped by a search which no longer exists" "An exception occurred and was handled by Exchange ActiveSync. This may have been caused by an outdated or corrupted Exchange ActiveSync device partnership. " Otherwise not much going on, DB health checks all passed 5 minutes prior to the crash. The only thing I can assume is that the faulting module (FSAPIScanSourceX64.dll) suggests there's yet another issue with McAfee GroupShield for Exchange. I'm mostly posting this in hopes of finding someone else with the same issue since maybe this is caused by some new bug. Still, if you have any suggestions on how to further track down the cause, please let me know. I found one post with similar errors in the migration forum, except in our case this had nothing to do with migrations. http://social.technet.microsoft.com/Forums/hi-IN/exchangesvrmigration/thread/95ab6f87-6766-4791-a30f-10cf38f3b37c. I will try the suggestion in there of running Test-MRSHealth to see if it reports back similar errors. Thanks, Matt

Please find below article for your reference https://kc.mcafee.com/corporate/index?page=content&id=KB52898 As per above article VSAPIScanSourceX64.dll pointing to Mcafee GroupShield 7.0.2 Stop all GroupShield services and check the status of "Exchange Information service also suggest you to consult with Mcafee support team for above issue Run chkdsk read only mode on all drives to check any disk problems on server. Exclude exchange database from file level scanning in antivirus Girishp

Hey, No, sadly our solution to this problem was to uninstall McAfee because after my post it randomly happened again a few weeks later. Store.exe kept crashing every few minutes just as you had experienced. No idea why, seems completely unprovoked. We also didn't have any McAfee updates or Windows updates to link this problem to. We're going to try reinstalling it again but this time disable the VSAPI scanning and only leave transport scanning enabled. We didn't have these problems until we switched to having all roles installed on each of our 3 servers in the DAG. Prior to that our Transport servers were separate. What's your current setup? Matt

We've only got the 1 server so like you all roles are on 1 machine. It seems pretty sporadic, it seemed to happen approx every 10 minutes for 3 hours on Friday but today for example it's done it twice, once at 6am and then again around 9:34. There goes my idea that it might have been load related.... Let me know how you get on and I'll do likewise. For now I've just disabled VSAPI Scanning but with it being so sporadic it's going to be a while before we can say it's "fixed". Strange how there's hardly anyone else out there posting this message. What OS are you running on the 3 boxes? Ours is 2008R2 Standard, SP1. Josh

Hey Josh, 2008 R2 SP1 Enterprise. And yea I agree, I was expecting to see more of these posts.. glad to have some company though since at least now we both know it's not an isolated issue. Our issues were also completely random, not load related or backup or anything. One happened middle of the day on a long weekend with practically 0 load. What kind of server is yours on? Ours are all IBM x3650 M3's. Maybe we can narrow this down to something unique to us. Matt

Hey Matt, Yeah good to have some company on this! Our server is a HP DL380 G5. The McAfee setup is, GroupShield for Exchange v. 7.0.1406.110, SP2, Hotfixes: HF648829;GSE7SP2Rollup2;Patch1;HF726815 & Virus Scan Console: 8.8. Since I disabled VSAPI scanning yesterday it hasn't happened, but it's gone away for at least a day before so I'm going to leave it disabled over the weekend see how that goes, re-enable it on Monday see if it comes back. Josh

Hey Josh, Looks like we have something in common... the versions match exactly. GroupShield Version 7.0.1406.110 SP2, Hotfixes HF648829;GSE7SP2Rollup2;Patch1;HF726815. Virus Scan Console 8.8. I guess we'll wait and see if disabling VSAPI scanning works. Trouble with this issue is that we had it happen and then suddenly stop for weeks before randomly happening again. So it'll be a while before we know for certain if that's doing it. Matt

Hey Matt, Interesting! I've seen they had acknowledged a problem with the store.exe crashing but in prior versions so the hotfix available isn't applicable. Our store hasn't crashed since I disabled VSAPI but like you say it's still too early to say. I'll log a case with McAfee if it comes back after enabling VSAPI scanning again. Will keep you updated. Josh

Still no crash. I've decided I'm going to re-enable scanning on Friday and leave it over the weekend seeing as it wasn't load causing the crash.

Sounds good, thanks for the updates. We're going ahead and reinstalling McAfee tonight but will leave VSAPI disabled for a few weeks.

Hey Josh, How's it going after enabling the VSAPI scanning again? We've had ours disabled for the last few weeks and so far no issues. Matt

Hey Matt, We've been really busy so haven't had chance to turn it back on yet! It's not happened since turning it off though, pretty much spot on a month since I disabled it so pretty sure the fault lies in there! When things quieten down a bit I'll re-enable it and let you know how I get on. Did you disable it straight from the reinstall? Josh

Good to hear it hasn't happened again, it's definitely the VSAPI scanning. I don't recall there being an option to disable it during the install. We disabled it afterwards by accessing GroupShield for Exchange --> Settings & Diagnostics --> Uncheck "Enabled" under Microsoft Virus Scanning API (VSAPI).

Hey Matt, Turned it back on, hasn't happened yet but then it's only been on since late Friday afternoon. Will keep you posted over the week.

Still no sign of the problem Matt, if you haven't already I'd probably say you should be ok to re-enable VSAPI.

Hey Josh, thanks for the update. Given that we went a month with it enabled before the problem came back again, I'm still a bit hesitant to turn it on. I take it everything has still been running well for you?

I've had this same issue this morning totally out of the blue. On all 3 of my Exchange 2010 servers store.exe is crashing due to GroupShield. Faulting application name: store.exe, version: 14.2.298.4, time stamp: 0x4f7f074d Faulting module name: VSAPIScanSourceX64.dll, version: 7.0.1425.0, time stamp: 0x4ee7f77f Exception code: 0xc000000d Fault offset: 0x000000000000b24c Faulting process id: 0x2c14 Faulting application start time: 0x01cd8112b16dd0cb Faulting application path: E:\Exchange\bin\store.exe Faulting module path: C:\Program Files (x86)\McAfee\GroupShield for Exchange\bin\VSAPIScanSourceX64.dll Report Id: 6970ede7-ed06-11e1-b626-622fdb358573

Hey Matt & Seb, Ours has been absolutely fine since re-enabling it. Here's our latest: Product Name McAfee GroupShield for Exchange Product Version 7.0.1406.110 Service Pack SP2 Hotfixes HF648829;GSE7SP2Rollup2;Patch1;HF726815 Anti-Virus Engine | DAT Version | Extra Drivers 5400.1158 | 6812.0000 | 0

Seb, Not sure if this could be related but we've just had a call from McAfee regarding a DAT they released on Sunday being the wrong version. Check your DAT is after Sunday and I'd say you can rule the DAT being the cause out.