CAS Server (E2k7 SP1) Reports W2SVC-WP Warning ID 2262 - Deadlock
Hi,Today, my Outlook Anywhere users lost their connections to the Exchange server and I had to rebootthe CAS server After rebooting, I found the followingwarnings that were logged on the CAS server.Event Type:WarningEvent Source:W3SVC-WPEvent Category:NoneEvent ID:2262Date:5/11/2009Time:12:28:18 PMUser:N/AComputer:EXCAS01Description:ISAPI 'c:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\aspnet_isapi.dll' reported itself as unhealthy for the following reason: 'Deadlock detected'. I found this article (http://support.microsoft.com/kb/821268) but don't knowwhat to do about it on my CAS server (Exchange 2007 SP1).Could someone please help me?Thanks in advance,Eric
May 12th, 2009 12:28am

Hello Eric,I would recommend you to apply the below mentioned KB's. This problem occurs because,Microsoft .NET Framework CLR memory management inefficiencies. FIX: You may notice that the memory load is very high when you run an application that is built on the .NET Framework 2.0http://support.microsoft.com/kb/942027You may experience network-related problems after you install Windows Server 2003 SP2 or the Scalable Networking Pack on a Windows Server 2003-based computerhttp://support.microsoft.com/kb/936594And check the below KB to dump the stack information of the Worker process crashing.How to generate a dump file when ASP.NET deadlocks in IIS 6.0http://support.microsoft.com/kb/828222/en-usArun Kumar | MCSE - 2K3 + Messaging | ITIL-F V3
Free Windows Admin Tool Kit Click here and download it now
May 12th, 2009 2:21am

Hi Arun, Thank you very much for your post. I read the articles you listed but cannot figure out how they can help me. 1) For KB942027, my files are newer. 2) For KB936594, the SNP is not installed on this server and W2k3 SP2 was installed several months ago. 3) For KB828222, I noticed the risk of running the Orphan Worker Process and don't know what to do even I get a dump file because there is no other application, other than Exchange 2007 (CAS role), installed. Here I put more entries of the logs of the server, which I believe have already told us what caused and how it caused Deadlock: (System Log)Event Type:InformationEvent Source:W3SVCEvent Category:NoneEvent ID:1074Date:5/11/2009Time:12:23:45 PMUser:N/AComputer:EXCAS01Description:A worker process with process id of '9352' serving application pool 'DefaultAppPool' has requested a recycle because the worker process reached its allowed processing time limit. (Application Log)Event Type:InformationEvent Source:RPC ProxyEvent Category:Startup Event ID:3Date:5/11/2009Time:12:23:46 PMUser:N/AComputer:EXCAS01Description:RPC Proxy successfully loaded in Internet Information Services (IIS) mode 6.0. (System Log)Event Type:WarningEvent Source:W3SVCEvent Category:NoneEvent ID:1013Date:5/11/2009Time:12:25:15 PMUser:N/AComputer:EXCAS01Description:A process serving application pool 'DefaultAppPool' exceeded time limits during shut down. The process id was '9352'. (Application Log)Event Type:WarningEvent Source:W3SVC-WPEvent Category:NoneEvent ID:2262Date:5/11/2009Time:12:28:18 PMUser:N/AComputer:EXCAS01Description:ISAPI 'c:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\aspnet_isapi.dll' reported itself as unhealthy for the following reason: 'Deadlock detected'. Event Type:WarningEvent Source:W3SVC-WPEvent Category:NoneEvent ID:2262Date:5/11/2009Time:12:28:19 PMUser:N/AComputer:EXCAS01Description:ISAPI 'c:\WINDOWS\Microsoft.NET\Framework64\v2.0.50727\aspnet_isapi.dll' reported itself as unhealthy for the following reason: 'Deadlock detected'. I believe two worker processes, both serving 'DefaultAppPool' (one is stuck during shutdown and another one is started to replace it), caused the Deadlock.If this is true,will it be an Exchange 2007 bug or there is a wayto prevent this happen? Please help.Thanks,Eric
May 12th, 2009 6:50pm

Hi, 1. Please ensure that we have proper file-level exclusions if we are running File-Level Antivirus on the Exchange sever:http://technet.microsoft.com/en-us/library/bb332342.aspx 2. Please enable logging on Outlook Anywhere and then send log to me. What is the Enable logging (troubleshooting) option? http://office.microsoft.com/en-us/outlook/HA012304211033.aspx 3. Please try to generate MPS(Mpsrpt_exchange.exe) report and then send to me. Besides,Id like to know when did the issue occur,when serveral Outlook anywhere clients try to connect to Exchange Server or just lost connect suddenly? What did the user do before the issue occur? How many user will be affected by this issue? Regards, Xiu
Free Windows Admin Tool Kit Click here and download it now
May 13th, 2009 11:20am

Any update after following Xiu's plan?Arun Kumar | MCSE - 2K3 + Messaging | ITIL-F V3
May 16th, 2009 6:35pm

It happened again yesterday.Looked into the logs and suspected it could be related to someissues happened on theMailbox server.Going to open a ticket with Microsoft for further investigation.Will update.Thanks,Eric
Free Windows Admin Tool Kit Click here and download it now
May 21st, 2009 6:03pm

Eric,Lets understand the deadlocks first. Basically, develepors write the locks on the applications to prevent any corruptions that may occur due to simultenous access of services. Deadlock is the next stage of it when these locks start preventing acess to resources. This may happen to due to number of reasons like heavy access of resources at the same time, automatic hits on resources by applications and others. Probably a database administrator can explain it better than me of course :)As we are discussing about an IIS deadlocksI would suggest you looking at the entire configuration around EWS, OWA, CAS, EAS and other web services. I presume you do not have any other website running on the CAS that may cause interference. Additionally, look at your IIS logs, they are the best place to isolate these issues. May be you can get some more information it before you open a case with EMS. All over that IIS diagnostics tool kit can also point you to correct direction.MMilind Naphade | MCTS:M | http://www.msexchangegeek.com
May 21st, 2009 10:28pm

I think "RPC over HTTP runs in the Default Application Pool (DefaultAppPool) in IIS. By default, this application pool is configured to recycle worker processes every 1,740 minutes (29 hours). During the recycling process, IIS allows active worker threads an additional 90 seconds to finish servicing requests before IIS terminates the active threads. "Because RPC over HTTP uses long-running connections, the connections may not finish within the additional 90 seconds that were given to the worker threads. In this scenario, the connections are terminated. Therefore, Outlook loses connectivity with IIS. When this action occurs, Outlook immediately tries to reconnect. If many Outlook clients are disconnected at the same time, the large number of concurrent reconnections may overwhelm the server."The aboveis from http://technet.microsoft.com/en-us/library/dd421855.aspx(Many Outlook Anywhere Users Intermittently Lose Connectivity with Their Exchange Mailboxes at the Same Time)I also think Microsoft should better document the sympton of this problem in this article(to include 'Deadlock detected') so that users like me (andMVPs)as well as Microsoftpeople (tech & moderator)can easily find this article when doing a search based on the server log events.Eric
Free Windows Admin Tool Kit Click here and download it now
August 19th, 2009 11:22pm

Hi Eric,Did you find any solution to this? What did MS say when you opened a ticket?We have the exact same problem, and are getting the Event ID 2262 about every other hour. Some times more often.We are running one server, with all roles (CAS, HUB, Mailbox). Everything worked fine, until we enabled Outlook Anywhere. Then we are getting Event ID 2262.We have recently migrated from a physical server to Vmware. There is a slightly difference in Exchange version and hardware. Both servers running Windows 2003 SP2Old Server:Exchange 2007 SP1 Rollup 8.16 GB RAMNew Server:Exchange 2007 SP24 GB RAMI suspect that the new server should have more memory, even thoug we only have about 60 mailboxes.I have tried the following without any luck:1. Disable Recycle Worker Processes on DefaulAppPool2. Incrase Limit the kernel request queue to 10.000 on DefaultAppPool3. Tried to create a new AppPool for Rpc4. Verified that antivirus realtime scanning has excluded all folders and extensions according to MS best practise for Exchange5. Ran Exchange Best Practise Analyser and did several suggested changes to registry, including installing ASP. Our old server never had ASP nor the 2262 error.6. Looked into the hotfix KB942027, and all my files are also newer (from 2008).I would really appreciate all the help I can get to sort this out.Jostein
September 27th, 2009 2:51pm

Hello, Facing exactly same problem ! Started 14 APRIL 2010 - 20 min outage Again, Yesterday 19 APRIL 2010 How do we solve this problem !!!! no updates sinc September for this thread? Thanks a lot! Details Event Type: Warning Event Source: W3SVC-WP Event Category: None Event ID: 2262 Date: 4/19/2010 Time: 4:14:47 PM User: N/A Computer: HT-CAS1 Description: ISAPI 'C:\SIOUXSIE\Microsoft.NET\Framework64\v2.0.50727\aspnet_isapi.dll' reported itself as unhealthy for the following reason: 'Deadlock detected'. Event Type: Warning Event Source: W3SVC-WP Event Category: None Event ID: 2262 Date: 4/19/2010 Time: 4:19:03 PM User: N/A Computer: HT-CAS1 Description: ISAPI 'C:\SIOUXSIE\Microsoft.NET\Framework64\v2.0.50727\aspnet_isapi.dll' reported itself as unhealthy for the following reason: 'Deadlock detected'. AUTO RESOLUTION Event Type: Information Event Source: MSExchange Web Services Event Category: Core Event ID: 2 Date: 4/19/2010 Time: 4:20:03 PM User: N/A Computer: HT-CAS1 Description: The Exchange Web Services started successfully. Event Type: Information Event Source: MSExchange Availability Event Category: Availability Service Event ID: 4007 Date: 4/19/2010 Time: 4:20:07 PM User: N/A Computer: HT-CAS1 Description: Process 4928[w3wp.exe:/LM/W3SVC/1/ROOT/EWS-1-129161387718704321]: Successfully
Free Windows Admin Tool Kit Click here and download it now
April 20th, 2010 10:49am

I have the same Issue. if anyone figured out the problem please post a solution. jeff.ferguson at gmail.com
December 8th, 2010 2:16pm

OK here is a update. I rebooted my Mailbox Server and it worked. Solved! *shaking head* This is probably why there was no solution..to simple
Free Windows Admin Tool Kit Click here and download it now
December 9th, 2010 8:58am

I'm seeing users disconnect and I'm using Outlook Anywhere (RPC over HTTP). I'm seeing event 2262 too and I'm unable to fix it. What is the solution to this?
December 20th, 2010 3:28pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics