Exchange 2013 - Event ID 4999 MS Exchange Common - Watson report about to be sent for process id: 5816

Hi,

We have a single Exchange 2013 server and are getting this error many times in a day.  I haven't been able to find a solution and was wondering if anyone came across this error or have any suggestions.

Edition             : Standard
AdminDisplayVersion : Version 15.0 (Build 913.22)


Watson report about to be sent for process id: 5816, with parameters: E12IIS, c-RTL-AMD64, 15.00.0913.022, M.Exchange.Imap4, M.Exchange.Net, M.E.N.NetworkConnection.BeginNegotiateTlsAsClient, System.InvalidOperationException, 99a4, 15.00.0913.007.
ErrorReportingEnabled: True 

Thanks

Amit

September 30th, 2014 9:43pm

Hi

I see you running an older version of exchange 2013. can you update your server and then report your findings?

Free Windows Admin Tool Kit Click here and download it now
September 30th, 2014 10:06pm

Hi,

We are on Exchange 2013 CU5.  The only release after that is CU6.  I don't believe CU6 addresses this.  This issue has been occurring for quite some time also in Exchange 2010.


September 30th, 2014 10:08pm

Hi,

Please verify service health with the following command.

Test-ServiceHealth

Use the Test-ServiceHealth cmdlet to test whether all the Microsoft Windows services that Exchange requires on a server have started.

The Test-ServiceHealth cmdlet returns an error for any service required by a configured role when the service is set to start automatically and isn't currently running.

If that service isnt currently running, please try to leave the service startup type set to Manual or change it to Disabled to prevent the system from trying to automatically start the service.

Best Regards.

Free Windows Admin Tool Kit Click here and download it now
October 1st, 2014 10:31am

Any updates on this?  We are getting the same issue on a single multi-role Exchange 2013 CU4 on Windows 2008 R2 Enterprise.

I ran the Test-ServiceHealth and it came up with passing marks on all required service running.

The actual event ID: 4999 description is:

Watson report about to be sent for process id: 3268, with parameters: E12IIS, c-RTL-AMD64, 15.00.0847.032, M.E.RpcClientAccess.Service, unknown, M.E.R.S.U.DatabaseInfo..ctor, System.NullReferenceException, aba8, unknown.

ErrorReportingEnabled: True

We get these in spurts, where they will disappear for hours and then suddenly start re-appearing in floods.  Right now, they are occurring every 3-5 seconds.

If I was confident that updating to a newer CU was beneficial, then I would, however I didn't see any mention of clearing 4999 events or rpcclientaccess.service errors so I don't want to open up another can of worms that a new version may bring.

October 3rd, 2014 9:58pm

No updates.  We are on CU5 and again like you said if you saw that doing the upgrade would resolve these issues it would make sense.  

Test-ServiceHealth also came back with a pass so not sure what the next steps are.  Hoping someone else can chime in.

thx

Free Windows Admin Tool Kit Click here and download it now
October 3rd, 2014 10:01pm

I can confirm this is still happening with CU6 installed also.  This and the perflib errors are all over the event logs.
October 7th, 2014 4:23pm

We have exchange 2013 cu6 mailbox server and we keep getting the below event error message

Log Name:      Application
Source:        MSExchange Common
Date:          10/2/2014 1:06:25 PM
Event ID:      4999
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:     
Description:
Watson report about to be sent for process id: 30632, with parameters: E12, c-RTL-AMD64, 15.00.0995.029, M.E.RpcClientAccess.Service, M.E.Data.ApplicationLogic, M.E.D.A.U.HttpPhotoRequestBuilder.Build, System.NotSupportedException, 4e29, 15.00.0995.027.
ErrorReportingEnabled: True

Event Xml:
< Event xmlns="">
  <System>
    <Provider Name="MSExchange Common" />
    <EventID Qualifiers="16388">4999</EventID>
    <Level>2</Level>
    <Task>1</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2014-10-02T20:06:25.000000000Z" />
    <EventRecordID>7826297</EventRecordID>
    <Channel>Application</Channel>
    <Computer></Computer>
    <Security />
  </System>
  <EventData>
    <Data>30632</Data>
    <Data>E12</Data>
    <Data>c-RTL-AMD64</Data>
    <Data>15.00.0995.029</Data>
    <Data>M.E.RpcClientAccess.Service</Data>
    <Data>M.E.Data.ApplicationLogic</Data>
    <Data>M.E.D.A.U.HttpPhotoRequestBuilder.Build</Data>
    <Data>System.NotSupportedException</Data>
    <Data>4e29</Data>
    <Data>15.00.0995.027</Data>
    <Data>True</Data>
    <Data>False</Data>
    <Data>Microsoft.Exchange.RpcClientAccess.Service</Data>
    <Data>
    </Data>
  </EventData>
< /Event>

Services seem to be fine (Via test-servicehealth), but i do have a group of users that complain that their outlook 2013 email clients will go non-responsive a few times a days. Wondering if this is related. Also this error is appearing every few minutes. Anyone else experiencing this?

Free Windows Admin Tool Kit Click here and download it now
October 7th, 2014 9:02pm

[PS] C:\Windows\system32>Test-ServiceHealth


Role                    : Mailbox Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeDelivery, MSExchangeIS,
                          MSExchangeMailboxAssistants, MSExchangeRepl, MSExchangeRPC, MSExchangeServiceHost,
                          MSExchangeSubmission, MSExchangeThrottling, MSExchangeTransportLogSearch, W3Svc, WinRM}
ServicesNotRunning      : {}

Role                    : Client Access Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeIMAP4, MSExchangeMailboxReplication,
                          MSExchangeRPC, MSExchangeServiceHost, W3Svc, WinRM}
ServicesNotRunning      : {}

Role                    : Unified Messaging Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeServiceHost, MSExchangeUM, W3Svc, WinRM}
ServicesNotRunning      : {}

Role                    : Hub Transport Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeEdgeSync, MSExchangeServiceHost,
                          MSExchangeTransport, MSExchangeTransportLogSearch, W3Svc, WinRM}
ServicesNotRunning      : {}
October 16th, 2014 8:24pm

We are also running into this issue with Exchange 2013 CU6. We're running two servers in a DAG and a hardware load balancer for client connections. Whenever this error appears the RPC service crashes and causes about 50% of our client connections to be lost and starts prompting user's to log back into Outlook (Those internal and using Outlook on our Terminal servers).

Also this error seems to happen on both servers with no real indication of why it's happening and it happens at random times of the day.

Event Errors associated with this one are:

Source: .NET Runtime, Event ID: 1026

Application: Microsoft.Exchange.RpcClientAccess.Service.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: Microsoft.Exchange.Diagnostics.ExWatson+CrashNowException
Stack:
   at Microsoft.Exchange.Diagnostics.ExWatson.CrashNow()
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.ThreadHelper.ThreadStart()

Source: Application Error, Event ID: 1000

Faulting application name: Microsoft.Exchange.RpcClientAccess.Service.exe, version: 15.0.995.28, time stamp: 0x53eadfe9
Faulting module name: KERNELBASE.dll, version: 6.3.9600.17278, time stamp: 0x53eebf2e
Exception code: 0xe0434352
Fault offset: 0x000000000000606c
Faulting process id: 0x16468
Faulting application start time: 0x01cfeed2863c9c76
Faulting application path: C:\Program Files\Microsoft\Exchange Server\V15\bin\Microsoft.Exchange.RpcClientAccess.Service.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: c4ee1d6f-5f7e-11e4-80e8-00155d3a5401
Faulting package full name: 
Faulting package-relative application ID: 

Free Windows Admin Tool Kit Click here and download it now
October 29th, 2014 4:46pm

We are also running into this issue with Exchange 2013 CU6. We're running two servers in a DAG and a hardware load balancer for client connections. Whenever this error appears the RPC service crashes and causes about 50% of our client connections to be lost and starts prompting user's to log back into Outlook (Those internal and using Outlook on our Terminal servers).

Also this error seems to happen on both servers with no real indication of why it's happening and it happens at random times of the day.

Event Errors associated with this one are:

Source: .NET Runtime, Event ID: 1026

Application: Microsoft.Exchange.RpcClientAccess.Service.exe
Framework Version: v4.0.30319
Description: The process was terminated due to an unhandled exception.
Exception Info: Microsoft.Exchange.Diagnostics.ExWatson+CrashNowException
Stack:
   at Microsoft.Exchange.Diagnostics.ExWatson.CrashNow()
   at System.Threading.ExecutionContext.RunInternal(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object, Boolean)
   at System.Threading.ExecutionContext.Run(System.Threading.ExecutionContext, System.Threading.ContextCallback, System.Object)
   at System.Threading.ThreadHelper.ThreadStart()

Source: Application Error, Event ID: 1000

Faulting application name: Microsoft.Exchange.RpcClientAccess.Service.exe, version: 15.0.995.28, time stamp: 0x53eadfe9
Faulting module name: KERNELBASE.dll, version: 6.3.9600.17278, time stamp: 0x53eebf2e
Exception code: 0xe0434352
Fault offset: 0x000000000000606c
Faulting process id: 0x16468
Faulting application start time: 0x01cfeed2863c9c76
Faulting application path: C:\Program Files\Microsoft\Exchange Server\V15\bin\Microsoft.Exchange.RpcClientAccess.Service.exe
Faulting module path: C:\Windows\system32\KERNELBASE.dll
Report Id: c4ee1d6f-5f7e-11e4-80e8-00155d3a5401
Faulting package full name: 
Faulting package-relative application ID: 

October 29th, 2014 7:46pm

Is Microsoft not going to address this issue.  We've been experiencing this issue since CU5 and we are now on CU6 and they are still occurring.  The only difference is that so far I don't have any user population complaining of disconnects during the middle of the day as was happening on CU5.  There is for sure an issue for constant Watson errors on Windows.  Microsoft please look into this and issue a fix asap.  Thank you.
Free Windows Admin Tool Kit Click here and download it now
November 13th, 2014 11:57pm

Were you ever able to resolve this issue?  I"m getting the same error message and we're on CU7.
January 14th, 2015 1:50am

I have this issue as well.

Server 2012 R2

Exchange 2013 Ent CU5

[PS] C:\Windows\system32>Test-ServiceHealth


Role                    : Mailbox Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeDelivery, MSExchangeIS,
                          MSExchangeMailboxAssistants, MSExchangeRepl, MSExchangeRPC, MSExchangeServiceHost,
                          MSExchangeSubmission, MSExchangeThrottling, MSExchangeTransportLogSearch, W3Svc, WinRM}
ServicesNotRunning      : {}

Role                    : Client Access Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeIMAP4, MSExchangeMailboxReplication,
                          MSExchangePOP3, MSExchangeRPC, MSExchangeServiceHost, W3Svc, WinRM}
ServicesNotRunning      : {}

Role                    : Unified Messaging Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeServiceHost, MSExchangeUM, W3Svc, WinRM}
ServicesNotRunning      : {}

Role                    : Hub Transport Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeEdgeSync, MSExchangeServiceHost,
                          MSExchangeTransport, MSExchangeTransportLogSearch, W3Svc, WinRM}
ServicesNotRunning      : {}


  • Edited by Alex_0s Friday, January 30, 2015 6:32 PM
Free Windows Admin Tool Kit Click here and download it now
January 30th, 2015 6:31pm

I have this issue as well.

Server 2012 R2

Exchange 2013 Ent CU5

[PS] C:\Windows\system32>Test-ServiceHealth


Role                    : Mailbox Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeDelivery, MSExchangeIS,
                          MSExchangeMailboxAssistants, MSExchangeRepl, MSExchangeRPC, MSExchangeServiceHost,
                          MSExchangeSubmission, MSExchangeThrottling, MSExchangeTransportLogSearch, W3Svc, WinRM}
ServicesNotRunning      : {}

Role                    : Client Access Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeIMAP4, MSExchangeMailboxReplication,
                          MSExchangePOP3, MSExchangeRPC, MSExchangeServiceHost, W3Svc, WinRM}
ServicesNotRunning      : {}

Role                    : Unified Messaging Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeServiceHost, MSExchangeUM, W3Svc, WinRM}
ServicesNotRunning      : {}

Role                    : Hub Transport Server Role
RequiredServicesRunning : True
ServicesRunning         : {IISAdmin, MSExchangeADTopology, MSExchangeEdgeSync, MSExchangeServiceHost,
                          MSExchangeTransport, MSExchangeTransportLogSearch, W3Svc, WinRM}
ServicesNotRunning      : {}


  • Edited by Alex_0s Friday, January 30, 2015 6:32 PM
January 30th, 2015 9:31pm

Do you guys have any update on this? I am being flooded with this alerts and its making server manager much more difficult. I have a 6 member DAG setup running Windows Server 2012 R2 and Exchange 2013 SP1
Free Windows Admin Tool Kit Click here and download it now
February 3rd, 2015 4:16am

We are still also having this issue even after applying CU7 and the issue doesn't seem to have an ending. We have opened up a MS support case and they had us remove AV, Diskeeper, CU7 applied and several other small changes that have not resolved the issue.

There are other Event ID's that are associated to this issue as listed below:

Event ID, Source

4999,MSExchange Common

4009,MSExchange Availability

1026,.NET Runtime

1000,Application Error (Faulting application name: Microsoft.Exchange.RpcClientAccess.Service.exe)

We are running on CLR.DLL version 4.0.30319.34209 which some of the .NET runtime errors lead us to thinking we needed to apply .NET 4.5.2 patch from KB2934520. Still no resolution of the issue.

I'm not sure what other peoples environments are like currently but we are using a Load Balancer, 2 Exchange servers each performing the CA and MB roles. The exchange servers are VM's on 2012 R2 Hyper-V.

The next steps MS support has requested is some logging of when the issues happen. Short of this we are talking about scrapping Hyper-V and just making Exchange 2013 a physical server instead.

February 4th, 2015 5:58pm

So I had a case open with MS for similar issue with event logging entries 4999 errors constantly.  In my case, the 4999 errors where complaining about Diagnostic monitors...so my events where more noise than anything else.  Exchange was/is running fine.  After working this case for about 2 months with Exchange Engineer, then a Escalation Engineer we could not find a resolution.  Support wanted to begin doing traces and this thing was taking to much time away from projects so I just closed the case.  We are moving to Office365/Exchange Online, already have Hybrid setup, DirSync and ADFS rolled out, although I wanted to get to the bottom of this issue, i couldnt continue to waste time on it.  It may take a while before they find the root cause for these...MS is certainly investigating.
Free Windows Admin Tool Kit Click here and download it now
February 4th, 2015 6:26pm

Any updates on this from MS? One of our clients has recently experienced this issue on 2013 Exchange.
February 13th, 2015 2:32am

Hi guys,

We have had the same issue with Exchange 2013 CU6 every time a user opened a mailbox.

We enabled OWA and logged into the accounts with OWA and then logged out removed the OWA access and the errors stopped! No idea why?

We were getting some problems logging into exchange at the same time with new accounts.

Let us know if using OWA helps with any of yours please....

Version 15.0 (Build 995.29) Standard  CU6

OS Name Microsoft Windows Server 2012 Standard
Version 6.2.9200 Build 9200
Processor Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz, 2494 Mhz, 4 Core(s)
Hardware Abstraction Layer Version = "6.2.9200.16442"

Thank you

  • Proposed as answer by Rampling 16 hours 46 minutes ago
Free Windows Admin Tool Kit Click here and download it now
March 16th, 2015 10:37am

Hi guys,

We have had the same issue with Exchange 2013 CU6 every time a user opened a mailbox.

We enabled OWA and logged into the accounts with OWA and then logged out removed the OWA access and the errors stopped! No idea why?

We were getting some problems logging into exchange at the same time with new accounts.

Let us know if using OWA helps with any of yours please....

Version 15.0 (Build 995.29) Standard  CU6

OS Name Microsoft Windows Server 2012 Standard
Version 6.2.9200 Build 9200
Processor Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz, 2494 Mhz, 4 Core(s)
Hardware Abstraction Layer Version = "6.2.9200.16442"

Thank you

  • Proposed as answer by Rampling Monday, March 16, 2015 2:37 PM
March 16th, 2015 2:35pm

Hi guys,

We have had the same issue with Exchange 2013 CU6 every time a user opened a mailbox.

We enabled OWA and logged into the accounts with OWA and then logged out removed the OWA access and the errors stopped! No idea why?

We were getting some problems logging into exchange at the same time with new accounts.

Let us know if using OWA helps with any of yours please....

Version 15.0 (Build 995.29) Standard  CU6

OS Name Microsoft Windows Server 2012 Standard
Version 6.2.9200 Build 9200
Processor Intel(R) Xeon(R) CPU E5-2640 0 @ 2.50GHz, 2494 Mhz, 4 Core(s)
Hardware Abstraction Layer Version = "6.2.9200.16442"

Thank you

  • Proposed as answer by Rampling Monday, March 16, 2015 2:37 PM
Free Windows Admin Tool Kit Click here and download it now
March 16th, 2015 2:35pm

Rampling,

Were you only getting this 4999 error when people we logging in?

We seem to have it on one of out two servers continually.

Did you disable OWA access for all users or individaul users that had the error? If individual, how did you find out the user causing the fault.

Regards

April 29th, 2015 4:23am

I know this thread is old but I kept coming back to it due to having the exact same problem.

It was on a Exchange 2013 infrastructure with 4 separate CAS servers and 4 MB servers in a DAG.

I isolated the problem to a single Mailbox database by activating them one at a time across the DAG.

Once I found the MB DB that was causing the problem it would without fail follow the MB DB to whatever server it was active on.

4999 errors and continual crashing of the MSExchangeAssistant service.

I then ran a mailbox repair -detectonly across the entire suspect DB.

No corruption.

I then started investigating the individual mailboxes.

One user had 91 search folders and was recently migrated from a 2007 environment.

Once the mailbox was repaired by running outlook.exe /cleanfinders (could have been done with MPCMAPI tool as well), all errors disappeared.

I wish there was a replacement for ExMon to identify which mailbox was causing the issue for Exchange 2013, but as it was it was inspection of each mailbox for search folders.

Free Windows Admin Tool Kit Click here and download it now
June 11th, 2015 5:50pm

I'm on CU8 and am getting 4-5 of these a day.

Eventsource: Windows Exchange Events

Windows Event ID: 4999

Message: Watson report about to be sent for process id: 5048, with parameters: E12IIS, c-RTL-AMD64, 15.00.1076.009, M.Exchange.Imap4, M.E.PopImap.Core, M.E.P.C.StringResponseItemProcessor.GetNextChunk, System.InvalidOperationException, daf0, 15.00.1076.000.

Detected on: 2015-07-06 15:17:15 EST


July 6th, 2015 4:31pm

How does this particular parameter for imapsetting relate to the above thread 

EnableGSSAPIAndNTLMAuth

GSSAPI-based Kerberos authentication

Free Windows Admin Tool Kit Click here and download it now
July 21st, 2015 4:47am

Our problem with the Source: .NET Runtime, Event ID: 1026 Application: Microsoft.Exchange.RpcClientAccess.Service.exe errors ended up being a long phone call over several weeks with Microsoft (114110311984190) that ended in that they recognize the issue and can reproduce the problem on their systems but they were not going to fix it. I think this is because we are not a company that has enterprise agreement with them (we pay per case).

In order to resolve the issue we moved all of our clients to using MAPI over HTTP vs RPC over HTTP; which by default is not enabled. Just know that if you take this course of action that as soon as Outlook see's it can use MAPI over HTTP it will start switching the client connections over; so do this during off hour times.

If you're not sure if you're using MAPI over HTTP just view your Outlook connections (both inside and outside the network) and verify the Protocol column says "HTTP." If it says "RPC" then moving over to MAPI over HTTP could potentially help.

After moving to MAPI over HTTP the issue has all but went away. Outlook 2010 (via a patch) and higher supports using MAPI over HTTP.

Good luck to the rest of you having this issue; it has been a big pain and sadly based on the response from MS support it will never be fixed. With that said I did stumble onto some references that MS is trying to age out RPC so this may be why they will not look into it any further.

If you would like to enable MAPI over HTTP here is a good starting point (EX2013): https://technet.microsoft.com/en-us/library/Dn635177(v=EXCHG.150).aspx


  • Edited by CraigI 20 hours 16 minutes ago gramar fixes
July 21st, 2015 7:12am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics