Exchange 2007 store.exe / rpc hung issue?
I had a strange problem happen today, this morning many of our users started reporting that their outlook clients were no longer connecting to the server. At first it was only a handful of users, all located on the same server, but located on seperate stores.Their local outlook clients showed an established connection to exchange, but were all waiting on RPC requests to be processed, causing outlook to hang.The first thing I checked was that event viewer was clean, which it was. No errors or warnings reported. All of the databases were mounted, and almost all the users (including myself) seemed to have to have no problems using the server. I'm actually located on the same store as several of the users who were affected.The queues on exchange were all at normal levels. CPU, memory all seemed in line as well. I checked my disk IO traffic, which all reported to have a disk queue length of 0, and minimal (10-20iops) were kicking off per store.At this point I was at a total loss, but as the minutes ticked on, more and more people started having issues. At this point I decided to reset the information store. When it attempted to stop, it hung. I left it in a stopping state for about 15 minutes before I finally killed store.exe.On restart, all the databases mounted fine, and everything appears to be running smooth. I'm glad restarting the services fixed the issue, but having no logs or error messages to go off of is leaving me unsure if this will happen again.I'm currently running the troubleshooter over the next 4 hours to collect logs, but it even then, I'm not sure if this was a client driven issue.I'm also upset that it seems exmon.exe is not updated for 2007? Or isn't publicly available?Has anyone else ever seen issues like this? Is there any extra troubleshooting that I might of missed?Thanks or your help.
October 2nd, 2008 7:08pm

Hi, From the symptoms looks like server is affected from Scalable Networking Pack (SNP) feature and TCP/IP Offload-enabled on network adapter so first thing I would suggest you to Check RSS and TCP chimney Offload settings which are by default enabled in Windows 2003 SP2. Further information and procedure is explained in these resources. Windows 2003 Scalable Networking pack and its possible effects on Exchange http://msexchangeteam.com/archive/2007/07/18/446400.aspx You may experience network-related problems after you install Windows Server 2003 SP2 or the Scalable Networking Pack on a Windows Server 2003-based computer http://support.microsoft.com/kb/936594 Additionally, run ExBPA to find out any non-standard settings or errors on your Exchange server. http://exbpa.com/ ExMon is not available for Exchange 2007 or you may need to contact PSS to get/run it. Alternatively you can take a look at below utillity which is similar. ExInsight: http://www.bitrunes.com/Download
Free Windows Admin Tool Kit Click here and download it now
October 3rd, 2008 2:28am

Find serveral similar thread for your reference: Store.exe service stopped, no error is logged in Event viewer http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=3763326&SiteID=17 Exchange 2007 STORE.exe dropping too much RAM! https://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=3433289&SiteID=17 Memory Utilization on Exch 2007 with W2K8 https://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=3436916&SiteID=17 Event 9665 and 12800 http://forums.microsoft.com/TechNet/ShowPost.aspx?PostID=3694010&SiteID=17&pageid=0#3694010 Hope it helps. Xiu
October 6th, 2008 2:11pm

This is precisely the problem that I am experiencing. I have been working with technet for close to a month now with no resolution. Were you able to find a fix? Our setup is 1 exchange server (CAS, HUB, MB), 16GB RAM, 2 quad core processors.
Free Windows Admin Tool Kit Click here and download it now
October 9th, 2008 5:43pm

No, we have yet to find a solution, other then pray it doesn't happen again. The exchange server that had the problem is just a HUB/MB server and runs on a Dell Poweredge 2950 with 16GB of ram and 2 quad core processors.We've run the exchange troubleshooter for 24 hours collecting RPC and Permon counters/logs, it found 0 problems, not even one warning. EXBPA also returns perfect results.At this point we are at a total loss, except for updating to the new Exchange security update. Please let me know if you get anywhere with Microsoft support, is this a recurring issue for you guys?Thanks.
October 9th, 2008 6:26pm

Interesting...ours is a poweredge 2900. I just downloaded the most recent netextremeII drivers from support.dell.com. I haven't had a chance to install though (probably will tonight though). we have installed update rollup 3 for Exchange 07 SP1...no improvement and also have run the setup and monitoring tools that you have run...no findings. Our server hangs fairly often, so I am catching a lot of heat for it. The problem first appeared about a month ago when DPM 2007 stopped communicating with the server hosting exchange and reported the following error: RPC Server is unavailable. This occurred in close proximity to moving some mailboxes to some new databases. All 5 storage groups and databases reside on the same server.
Free Windows Admin Tool Kit Click here and download it now
October 9th, 2008 8:24pm

Was paroosing the net trying to find a resolution for the same problem and found you guys. My email was running fine until our "IT" guy decided he wanted to beef up the security on our server. Every since then I cannot get my email. It keeps telling me that Outlook is not connected. Now I can't even get it to verify my mailbox while trying to re-setup the email account. It keeps telling me that "RPC Server is unavailable. For what it's worth, when I saw our IT guy connect to the server, I noticed that it was a DELL also. Don't know if this problem is exclusive to DELL or not.
October 21st, 2008 4:14pm

Hi, It is absolutely normal for Store.exe to grab as much RAM on the server as it can possibly get - as long as Store thinks it needs it to optimize performance. Store was written to do so. It does this as it wants to do as much stuff in memory as possible, without having to go to the slow page file. It is a common misconception - that Store.exe's increasing memory consumption is a "memory leak". Besides, other application such as SQL also will use store.exe. Please use Process Explorer to check it. You can download the Process Explorer from the following link: http://www.microsoft.com/technet/sysinternals/ProcessesAndThreads/ProcessExplorer.mspx We also recommend you to update your network driver. BTW,is there any error event in event log ?Please post the detail error event here. Related information share with you: Why is Exchange Store.exe so RAM hungry? http://msexchangeteam.com/archive/2004/08/02/206012.aspx The Microsoft Exchange Information Store service stops responding on a computer that is running Windows Server 2003 and Exchange Server 2007 http://support.microsoft.com/kb/928368/en-us Troubleshooting Slow RPC Request Processing Issueshttp://technet.microsoft.com/en-us/library/cc540464(EXCHG.80).aspx
Free Windows Admin Tool Kit Click here and download it now
October 22nd, 2008 11:49am

Xiu ~Your reply seems to be a copy and paste from an article many years ago from the exchange team blog. http://msexchangeteam.com/archive/2004/08/02/206012.aspxI would recommend that you read a thread before replying to it. No one here is complaining about exchange eating up too much RAM. In fact you were the first person to even mention RAM in this thread.As for the links you posted, the update for the information store applies to a kernel version from 2 years ago. IT DOES NOT APPLY TO A PATCHED SYSTEM. Linking old kernel patches is a good way to completely destroy an admins machine.Since you are a moderator, I'd ask that you remove you're previous post.Thanks.
October 23rd, 2008 5:08pm

Here's the deal with our system. I finally decided to just switch out the NIC. Everything was solved until I tried to team the ports on the new Intel Pro Quad VT adapter. The same problem came up, so apparently it has something to do with the way the x64 drivers handle the teaming of adapers. I haven't had time to research this issue, but, as long as my x64 machines are using only 1 port, they work.
Free Windows Admin Tool Kit Click here and download it now
October 28th, 2008 9:12pm

I had a similar problem yesterday with Exchange 2007 sp1 UP6 on windows 2008. The server just hung no events were written and I can't seem to figure out what caused it. It is a cluster setup and did not even failover. We are using HP dl380's. This is the strangest case I have ever seen.... please post if there are any solutions
March 16th, 2009 8:38pm

I know this thread is old but I was wondering if anyone found a solution. I seem to be having the same issue Windows 2003/Exchnage 2007 SP1 RU9 in a SCC using HP blades.
Free Windows Admin Tool Kit Click here and download it now
November 20th, 2009 1:59pm

Mojea, In our case there was a bug in store.exe and VSS enginethat caused the users to disconnect. If there is a databases maintenence scheduleand DPM backup overlab this issue appears. We have applied a hotfix to both Exchange and update our DPM environment. This hotfix is built into UP9 so I am not why it is happening to your setup? It took us a while resolve this issue.What type of backup solutions are you using?Bindazz
November 24th, 2009 12:14am

We have about 10 different Exchange mailbox servers and it seems to only happen on one particular CMSapprox every 3-4 days. Our backup solution is HP dataprotector.
Free Windows Admin Tool Kit Click here and download it now
November 24th, 2009 2:59pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics