Exchange Server 2013 CU7 Back Pressure Version Buckets

Hello,

Since upgrading from SP1 to CU7 I'm noticing periodic crashes when users send large files. This latest time, two 25 to 31 MB messages were sent within 10 minutes. The second 31 mb file caused the server to crash (or in microsofts words delay, in my book no mail flow equals crash.)

Each time I've seen 15004 events stating the following.

The resource pressure increased from Normal to Medium.

The following resources are under pressure:

Version buckets = 132 [Medium] [Normal=80 Medium=120 High=200]

The following components are disabled due to back pressure:

Inbound mail submission from the Internet

Mail submission from Pickup directory

Mail submission from Replay directory

Mail delivery to remote domains

Content aggregation

Mail resubmission from the Message Resubmission component.

Mail resubmission from the Shadow Redundancy Component

The following resources are in normal state:

Queue database and disk space ("C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\data\Queue\mail.que") = 32% [Normal] [Normal=95% Medium=97% High=99%]

Queue database logging disk space ("C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\data\Queue\") = 32% [Normal] [Normal=95% Medium=97% High=99%]

Private bytes = 4% [Normal] [Normal=71% Medium=73% High=75%]

Physical memory load = 82% [limit is 94% to start dehydrating messages.]

Submission Queue = 0 [Normal] [Normal=2000 Medium=4000 High=10000]

Temporary Storage disk space ("C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\data\Temp") = 32% [Normal] [Normal=95% Medium=97% High=99%]

We have had user's size limits set to 30MB. I've re-examined the size settings throughout exchange and made sure they match.

I've read in some places just to increase the threshold, but microsoft says to leave the defaults if you can and find the underlying issue. I've also read on another blog someone was able to fix similar issues by replacing the edgetransport.exe.config.

Here's the blog: https://social.technet.microsoft.com/Forums/en-US/c3d300c5-473f-4ab3-8df2-e85c0ea2ed99/backpressure-activated-after-exchange-cu5-installversion-buckets?forum=exchangesvrsecuremessaging

I'm rather skeptical of replacing the config file.... Anyone have experience changing the version bucket thresholds? 

March 19th, 2015 6:02pm

Hi 

Please check the disk space where the transport queue resides

Basically high level of hard drive space utilization is calculated by using the following formula in Exchange 2013:
100 * (hard disk size fixed constant) / hard drive size
The value of fixed constant is 500 megabytes (MB)

You can also refer my blog to understand more about back pressure - 

http://exchangequery.com/category/back-pressure/

Free Windows Admin Tool Kit Click here and download it now
March 20th, 2015 3:44am

Transport Queue resides on a 600 GB VM drive with 391 GBs free. Should this be enough?

The message says nothing about hard drive space... only version buckets.

  • Edited by BJonWayne 17 hours 20 minutes ago
March 23rd, 2015 10:04am

Here is the rest of the event error which states hard drive and memory levels are fine:

he following resources are in normal state:

Queue database and disk space ("C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\data\Queue\mail.que") = 32% [Normal] [Normal=95% Medium=97% High=99%]

Queue database logging disk space ("C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\data\Queue\") = 32% [Normal] [Normal=95% Medium=97% High=99%]

Private bytes = 4% [Normal] [Normal=71% Medium=73% High=75%]

Physical memory load = 82% [limit is 94% to start dehydrating messages.]

Submission Queue = 0 [Normal] [Normal=2000 Medium=4000 High=10000]

Temporary Storage disk space ("C:\Program Files\Microsoft\Exchange Server\V15\TransportRoles\data\Temp") = 32% [Normal] [Normal=95% Medium=97% High=99%]

Free Windows Admin Tool Kit Click here and download it now
March 23rd, 2015 10:07am

Saw another question, the same or similar to yours, last week.  The question was answered and the answer appeared to be related to a large message causing the issues.  Here is the link to the thread itself:

https://social.technet.microsoft.com/Forums/en-US/463794eb-1f84-4950-8be4-dc4054662f73/exchange-2013-back-pressure-event?forum=exchangesvradmin

And here's the TechNet article referenced for the resolution:

https://technet.microsoft.com/en-us/library/bb124375(v=exchg.150).aspx

March 23rd, 2015 10:27am

Transport Queue resides on a 600 GB VM drive with 391 GBs free. Should this be enough?

The message says nothing about hard drive space... only version buckets.

  • Edited by BJonWayne Monday, March 23, 2015 2:03 PM
Free Windows Admin Tool Kit Click here and download it now
March 23rd, 2015 2:02pm

Transport Queue resides on a 600 GB VM drive with 391 GBs free. Should this be enough?

The message says nothing about hard drive space... only version buckets.

  • Edited by BJonWayne Monday, March 23, 2015 2:03 PM
March 23rd, 2015 2:02pm

Saw another question, the same or similar to yours, last week.  The question was answered and the answer appeared to be related to a large message causing the issues.  Here is the link to the thread itself:

https://social.technet.microsoft.com/Forums/en-US/463794eb-1f84-4950-8be4-dc4054662f73/exchange-2013-back-pressure-event?forum=exchangesvradmin

And here's the TechNet article referenced for the resolution:

https://technet.microsoft.com/en-us/library/bb124375(v=exchg.150).aspx

  • Marked as answer by BJonWayne 10 hours 22 minutes ago
Free Windows Admin Tool Kit Click here and download it now
March 23rd, 2015 2:26pm

Saw another question, the same or similar to yours, last week.  The question was answered and the answer appeared to be related to a large message causing the issues.  Here is the link to the thread itself:

https://social.technet.microsoft.com/Forums/en-US/463794eb-1f84-4950-8be4-dc4054662f73/exchange-2013-back-pressure-event?forum=exchangesvradmin

And here's the TechNet article referenced for the resolution:

https://technet.microsoft.com/en-us/library/bb124375(v=exchg.150).aspx

  • Marked as answer by BJonWayne Tuesday, March 24, 2015 9:00 PM
March 23rd, 2015 2:26pm

Saw another question, the same or similar to yours, last week.  The question was answered and the answer appeared to be related to a large message causing the issues.  Here is the link to the thread itself:

https://social.technet.microsoft.com/Forums/en-US/463794eb-1f84-4950-8be4-dc4054662f73/exchange-2013-back-pressure-event?forum=exchangesvradmin

And here's the TechNet article referenced for the resolution:

https://technet.microsoft.com/en-us/library/bb124375(v=exchg.150).aspx

  • Marked as answer by BJonWayne Tuesday, March 24, 2015 9:00 PM
Free Windows Admin Tool Kit Click here and download it now
March 23rd, 2015 2:26pm

You are spot on. B0ndoo7. There were 2 emails with multiple attachments sent at the time of failure; One 28 mb message that got through OK and one 31mb message that went through immediately after we rebooted the server to get it going again. The 31 mb email was originally time stamped by our barracuda spam device at the exact time our mail flow stopped. When I look it up in message tracking it says successfully sent and the time stamp is directly after the reboot. Should I be looking at the message tracking info from the shell, because the ECP delivery report screen doesn't show anything other than a successful report that the email was delivered.

Assuming this was a problem with a 31 mb file, it shouldn't be. If this size is allowed, my email server should be able to handle it, if it's not allowed then it should just be rejected and no problems.

After realizing this had to be a problem with message size, I checked all our settings. They are as follows:

Organization config send/receive 30 MB

Front end send/receive connectors 36 MB Default

Receive and send connectors 35 MB Default

User Mailbox size restriction: 30 MB

Could it be that these mismatched settings are what caused the problem?

I've now changed the settings as follows. Am I missing anything?

Organization config send/receive 35 MB

Front end send/receive connectors 36 MB Default

Receive and send connectors 35 MB Default

User Mailbox size restriction: none Default

Thanks!

March 23rd, 2015 4:15pm

Yes, you have way more options looking through the logs using the shell. Do you have any size limitations on your Barracuda to modify?  Has pressure (version buckets) number decreased now, or are you still having the issue (medium or high levels)?
Free Windows Admin Tool Kit Click here and download it now
March 24th, 2015 10:11am

Yes, the default was set to 100mb, I've changed it to 35 MB to match now.

Pressure was down right after I rebooted, but I'm worried that it will crash every time someone sends a message larger than our limit.

March 24th, 2015 10:21am

You can always test it yourself, right?  Send a large attachment from an external mail account to your internal mailbox.  Of course you should do this after hours. :)

Is the reply from Sathish Veerapandian really the answer?  This issue has nothing to due with a shortage of disk space; this is a different issue relating to large inbound emails.

Free Windows Admin Tool Kit Click here and download it now
March 24th, 2015 2:01pm

Yes, I plan on testing it. I've done some more message tracking with the shell and found the following event id for the large message that caused the problem. I've looked this up and it seems that this is caused by shadow redundancy, which is turned on by default. Apparently it should be disabled for environments with 1 mailbox server or one server in general for that matter. I'm going to see if I can reproduce the issue afterhours and if so, I will disable this and test again.

SourceContext           : No suitable shadow servers
ConnectorId             :
Source                  : SMTP
EventId                 : HAREDIRECTFAIL

March 24th, 2015 5:06pm

Hello,  We are seeing this same issue.  Do you have any updates?   We are taking this error several times a day, and it is causing mail delays until the back pressure is resolved.   We have a single Exchange 2013 environment with CU7. 

Thanks
Matthew

Free Windows Admin Tool Kit Click here and download it now
April 21st, 2015 11:20pm

Update on this issue:

I have disabled shadow redundancy to no avail. A few days later we had another server crash/delay caused by a large email received from an external address. Right around the time of the crash I am seeing event ID 5011. Articles on this event advise to troubleshoot exchange server health and to use IIS debug to troubleshoot app crashes. I ran the shell command "get-serverhealth servername" and found out that the alert values for "ActiveSync," "ActiveSync.protocol," and "FrontendTransport" are all "unhealthy."

Some more digging to find out what monitors caused the alerts found are the following:

OnPremissesInboundProxyMonitor

OnPremisesSmtpClientSubmissionMonitor

ActiveSyncCTPMonitor

ActiveSyncDeepTestMonitor.

Troubleshooting steps here recommend Recycling app pools, IIS and server power. https://technet.microsoft.com/en-us/library/ms.exch.scom.activesync(v=exchg.150).aspx#EXP

None of these has returned the sets to "Healthy."

Their last step is to contact Microsoft. My plan is to run updates this weekend and contact Microsoft to troubleshoot after as I'm sure this will be their first recommendation when I open the case.

I'm also going to install IIS debug diagnostic tool to see if I can get more information from crash dumps if they continue.

April 28th, 2015 1:00pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics