I have Exchange 2007 deployed in ccr cluster. The server has 16GB of memory for about 1500 light/average users. memory/pagefile usage starts of low and steadily climbs until there is none left and store.exe crashes. Has anyone seen this? How can it be prevented. Thanks.

Hiya chap, The memory and indeed page file utilisation that you are seeing is normal - however not to the point where your server crashes. Are there any significant entries in the system or application event log - what actually happens on a crash? Additionally what SP is Exchange 2007 running at (RTM or SP1). The following article is a very good overview of the memory consumption - however in order to help you further would you be ok with supplying some more data about the state of the server when it crashes? http://msexchangeteam.com/archive/2008/08/06/449484.aspx Cheers A

Exchange is running SP1 completely patched. The application log has: at 9:05 source MSExchange Cluster catagory: ExRes event id: 1014 Clustered Mailbox Server: Physical Server: Exchange Information Store Instance (Exchange2k7): IsAlive checking for this resource failed due to a timeout. For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp. source: Application Error catagory: (100) eventid: 1000 Faulting application store.exe, version 8.1.291.1, faulting module ntdll.dll, version 5.2.3790.3959, fault address 0x000000000001b787. at 9:07 source MSExchange Cluster catagory: ExRes event id: 1005 Clustered Mailbox Server: Physical Server: The IsAlive check for the resource Exchange Information Store Instance (Exchange2k7) failed. MSExchangeRepl event id 2137 Log truncation open RPC context to the Information Store failed for storage group 'Exchange2k7\Public Folder Storage Group'. Error code: 3355444403. I have had the same issue on both nodes of the cluster.

Ok, is there anything else on each node of the cluster - for example Group Shield or other AV software? Cheers A

Groupshield 7 and McAfee AV 8,5 with the appropriate exclusions listed in the mcafee kb. Other then that they have exchange only.

store.exe application error would be because of memory corrupted by some application and sotre.exe is trying to access it and crashing but for exact cause you may need to contact Microsoft PSS. They may ask you to attach adplus to store.exe to generate crash dump for further diagnostics to find exact problem. How to use ADPlus to troubleshoot "hangs" and "crashes" Agree with Andy,you may check new updates/patches released for the antivirus system running on Exchange servers,itfixesproblem sometimes...

Thanks for the tip on adplus. I have it running and hopefully it will tell me something. I also approached pss, but I work for government, and getting to pss is difficult because of city policy. Thanks.

Clarify: store.exe use too much memory and finally crashed Troubleshooting: 1. Update Exchange server to latest Rollup Notes: Please review KB 945938, theres a known resolution to the cause which similar to yours 2. Also check your windows servers version and make sure that it has been updated to latest 3. Please try to disable AV temporarily and check issue for testing 4. Please try to modify database cache. [Operation Steps] Notes: As in 64-bit Exchange 2007, the Max cache depends on the amount of available memory and database I/O 5. Check NIC on your nodes; disable the TCP Chimney if it belongs to Boardcom or HP vendors for testing. It has potential effects on exchange http://blogs.technet.com/brad_rutkowski/archive/2007/08/10/how-to-know-if-tcp-offload-is-working.aspx Collect info: 1. Can you perform movement between nodes? Hows the issue behave on the different nodes? Related Resources: The Store.exe process uses almost 100 percent of CPU resources, and the size of the public folder store increases quickly in Exchange Server 2007 Set create dump file if Information Store crashed Ensure we have proper file-level exclusions if we are running File-Level AV on the Exchange sever PS: Other application may use store.exe. Please use Process Explorer to check it

p.MsoNormal, li.MsoNormal, div.MsoNormal {margin-top:0cm;margin-right:0cm;margin-bottom:10.0pt;margin-left:0cm;line-height:115%;font-size:11.0pt;font-family:'Calibri','sans-serif';} a:link, span.MsoHyperlink {color:blue;text-decoration:underline;text-underline:single;} a:visited, span.MsoHyperlinkFollowed {color:purple;text-decoration:underline;text-underline:single;} p.MsoListParagraph, li.MsoListParagraph, div.MsoListParagraph {margin-top:0cm;margin-right:0cm;margin-bottom:10.0pt;margin-left:36.0pt;line-height:115%;font-size:11.0pt;font-family:'Calibri','sans-serif';} p.MsoListParagraphCxSpFirst, li.MsoListParagraphCxSpFirst, div.MsoListParagraphCxSpFirst {margin-top:0cm;margin-right:0cm;margin-bottom:0cm;margin-left:36.0pt;margin-bottom:.0001pt;line-height:115%;font-size:11.0pt;font-family:'Calibri','sans-serif';} p.MsoListParagraphCxSpMiddle, li.MsoListParagraphCxSpMiddle, div.MsoListParagraphCxSpMiddle {margin-top:0cm;margin-right:0cm;margin-bottom:0cm;margin-left:36.0pt;margin-bottom:.0001pt;line-height:115%;font-size:11.0pt;font-family:'Calibri','sans-serif';} p.MsoListParagraphCxSpLast, li.MsoListParagraphCxSpLast, div.MsoListParagraphCxSpLast {margin-top:0cm;margin-right:0cm;margin-bottom:10.0pt;margin-left:36.0pt;line-height:115%;font-size:11.0pt;font-family:'Calibri','sans-serif';} .MsoChpDefault {;} .MsoPapDefault {margin-bottom:10.0pt;line-height:115%;} @page Section1 {size:595.3pt 841.9pt;margin:72.0pt 72.0pt 72.0pt 72.0pt;} div.Section1 {page:Section1;} ol {margin-bottom:0cm;} ul {margin-bottom:0cm;} Hi mate,Just reading your thread...I had this warning message being generated from every 3 minutes on my active and passive node (CCR cluster, SP1, Rollup 2): ---------------------------------- MSExchangeRepl with EventID 2137 Log truncation request to the Information Store using RPC has failed for storage group 'MARS\Sixth Storage Group'. Error code: 4294966264. -------------------------------- My backup software (Data Protector) was also reporting error messages for the storage group mentioned in the event above event log.The problem seemed to be caused by my AV software (Trend 8.0). I did not have real-time scanning exclusions set for my transaction log directories. Resolution: <!--[if !supportLists]-->1. <!--[endif]-->Unload Trend on both nodes (Stop AV service). <!--[if !supportLists]-->2. <!--[endif]-->Ran a full backup Backup Succeeded without errors. <!--[if !supportLists]-->3. <!--[endif]-->Still got the error message above in the event viewer. <!--[if !supportLists]-->4. <!--[endif]-->Restarted the Microsoft Exchange Replication Service on the passive node. Dont worry your cluster wont fail-over, its not a cluster resource. <!--[if !supportLists]-->5. <!--[endif]-->Wait for the service to come back up. <!--[if !supportLists]-->6. <!--[endif]-->Check the event logs on the active node you should see transaction replication start for each storage group. <!--[if !supportLists]-->7. <!--[endif]-->Give it 5 mins to settle in. <!--[if !supportLists]-->8. <!--[endif]-->Bingo! <!--[if !supportLists]-->9. <!--[endif]-->Replication is restarted and transaction log files are committed to the database. <!--[if !supportLists]-->10. <!--[endif]-->Error messages gone. Looks slightly different to your event log but still may be the AV software Gotta set those exclusions correctly!!! This article tells you what to exclude http://technet.microsoft.com/en-us/library/bb332342.aspx Good luck! Sherwinator

This issue is the victim of a memory manager trimming operation bug. I got the same problem. And it's fixed after applying http://support.microsoft.com/kb/938486. Good luck! Regards, Jiang

Install MSKB 954337 hotfix as a replacement for 938486 to get the latest kernel OS fixes. Hal.dll 5.2.3790.4313 Ntkrnlmp.exe 5.2.3790.4313 Ntoskrnl.exe 5.2.3790.4313 BTW, this issue may be due to store.exe utilizing all available RAM as there is no limit on database cache size for E2K7. However incase the Exchange 2007 server is sluggish and unresponsive, we can configure a maximum size for the database cache by configuring the msExchESEParamCacheSizeMax parameter. (Though not recommended by Microsoft). The msExchESEParamCacheSizeMax parameter specifies the size of the database cache in 8-kilobyte pages. http://technet.microsoft.com/en-us/library/bb691304(EXCHG.80).aspx