Hey Guys,

since we upgraded from Exchange 2010 to Exchange 2013 (SP1) any SMTP Receive Connector we create (including the default one) show the same strange behavior. When you send an email (no matter if internal or external) it sometimes takes up to 30 seconds for the exchange server to acknowledge the message. Using the SMTP log I see the following entries:

2014-08-08T08:58:29.053Z,                       MAIL FROM:<test@test.de>,

2014-08-08T08:58:29.053Z,                       SMTPSubmit SMTPAcceptAnySender SMTPAcceptAuthoritativeDomainSender AcceptRoutingHeaders,Set Session Permissions

2014-08-08T08:58:29.053Z,                       receiving message

2014-08-08T08:58:29.053Z,                       250 2.1.0 Sender OK,

2014-08-08T08:58:29.053Z,                       RCPT TO:<fake@mail.de>,

2014-08-08T08:58:29.053Z,                       250 2.1.5 Recipient OK,

2014-08-08T08:58:29.053Z,                       DATA,

2014-08-08T08:58:29.053Z,                       354 Start mail input; end with <CRLF>.<CRLF>,

2014-08-08T08:58:29.068Z,                       Proxy destination(s) obtained from OnProxyInboundMessage event

2014-08-08T08:58:40.960Z,                       "250 2.6.0 <5160cd2a-9160-4a35-9007-1f9c17761bc0@--------> [InternalId=17776869639170, Hostname=-----------] Queued mail for delivery",

As you can see theres a delay between 08:58:29 and 08:58:40 where nothing happens. The sending smtp service waits for the server to acknowledge the message at least thats what I guess. Since we use this kind of connector a lot for internal mail traffic with non-Outlook clients it is essential to get rid of this issue. For example using Trac Ticket system or Subversion also leads to slow or delayed responses. Creating a ticket in Trac sometimes takes up to a Minute since several emails are created and sent in background processes. We didnt have this issue with Exchange 2010 and I actually couldnt find much using google. 

Since there's no error showing it's quite difficult to track down the issue. After I did some researching I tried configuring the receive connector to change this behavior but nothing helped. I tested the following options:

-          MaxAcknowledgementDelay 00:00:00

-          TarpitInterval 00:00:00

-          MessageRateSource None

-          MessageRateLimit unlimited

-          MaxInboundConnectionPercentagePerSource: 20

-          MaxInboundConnectionPerSource 100

 I Also took a look at the Throttling Policies with no luck either. 

Calling the Microsoft support simply led to the typical we do not support 3^rd party issues. In fact this is NOT a third party issue but arguing didn't help. I tested every possible solution I found and after weeks I simply dont' have the slightest idea any more, how to solve the problem.

Some information about our Exchange environment:

-          Exchange DAG with 2 servers

-          Windows Server 2012 (latest updates)

-          Exchange 2013 SP1 installed

Disabling DAG members also didnt help and all the members show the same behavior no matter which connector I choose to use.  

Any hint or idea would be very much appreciated.

Thanks,

Christoph

Hi Christoph,

Thank you for your question.

I am trying to involve someone familiar with this topic to further look at this issue.

Best regards,

If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com

Hi Christoph,

From the SMTP session, the delay happened when processing data. This can be caused by several factors, such as the length of message content.

In Exchange 2013, we have the function called shadow redundancy, the transport server makes a redundant copy of any messages it receives before it acknowledges successfully receiving the message back to the sending server. This helps to ensure that all messages in the Exchange 2013 transport pipeline are made redundant while they're in transit. If Exchange 2013 determines the original message was lost in transit, the redundant copy of the message is redelivered.

For more information about shadow redundancy, we can refer to the article:

Shadow redundancy

http://technet.microsoft.com/en-us/library/dd351027(v=exchg.150).aspx

By default, it is enabled, and if we want to disable it, we can use the command:

Set-TransportConfig ShadowRedundancyEnabled $false

Best Regards,

Tracy Liang

Hi,

Is there any update on this issue?

Best Regards,

Tracy Liang

Hi Tracy, 

Thank's a lot for your answer and sorry for the delayed reply. I had to focus on a different topic for a few days so I wasn't able to test the feature you mentioned until today. 

I disabled shadow redundancy and ran my tests again but I had the same behavior showing. Do I need to restart any services bevor the changes take effect. 

Thanks, 

Christoph

Hi Christoph,

Please restart the Microsoft Exchange Transport service to take effective.

Also, from the original SMTP log, the delay happened when transferring data, it can be caused if the message body size is large and it takes a long time to complete the data transfer. You can try to telnet the Exchange Mailbox server  to test if the delay also happens.

http://technet.microsoft.com/en-us/library/aa995718(v=exchg.65).aspx

Best Regards,

Tracy Liang

Hi Tracy, 

since the problem doesn't appear every time we send an email using SMTP I already wrote a PowerShell script that sends SMTP messages using "net.mail.smtpclient" and counts the errors. Therefore the message body size should be at a minimum to avoid size issues. 

I restarted the Transport Service on both servers and ran the script but again I had several delayed connections. I first thought that the delay is always 11 seconds but I had some outliers with 2, 12 and 21 seconds. Still the majority shows pretty much exactly 11 seconds (plus a few milliseconds probably depending on the network and service latency). Does this help in any way?

Best Regards, 

Christoph

Hi Christoph,

Do you have any firewall/antivirus software deployed? If so, please try to disable them for the test.

Frankly, I think the 11 seconds is acceptable as it can be caused by the network.

Best Regards,

Tracy Liang

Hi Tracy, 

thanks for your reply. If it would only be 11 Seconds sometimes, I might be able to live with it. But the issue stacks up when several emails are send within a short time frame. For example our ticket system. It happens quite often that creating a ticket takes up to a Minute because emails are sent in the background to all persons involved. If you sum this up quite some time gets wasted while our employees wait for a ticket to be created. And this is not the only system affected.

I also don't believe this is a network related issue since I experience the same behavior when run my tests directly on our Exchange server. Also the problem didn't exist with Exchange 2010. 

I disabled firewall and anti virus on all Exchange servers, still no luck. 

Best Regards, 

Christoph

Hi Christoph,

When the issue happened, especially delayed for 1 minute, was there any high message traffic happened? How many messages would be sent from your ticket system simultaneously?

You can check the Application Log to see if there is any back pressure issue happened on your Mailbox server. You may refer to this article:

http://technet.microsoft.com/en-us/library/bb201658(v=exchg.150).aspx

If the issue still occurs, we may need to analyze the SMTP log, Application Log and transport pipeline in detail. So, it is best if you can open up a ticket to CTS support team for further assistance.

Best Regards,

Tracy Liang

• Marked as answer by Da XuMicrosoft employee, Moderator Friday, August 29, 2014 1:22 AM

Hi Tracy, 

I checked the logs but didn't find a single issue related to back pressure.

I opened up a support case again and I'll post a solution here if anything helpful comes out. 

I also set up a test environment with a 2012 R2 Domain Controller and a Exchange 2013 CU5 running on Server 2012. Even in a completely new environment the same behavior shows up. Seems a little odd to me since I thought it might be a configuration problem. 

Well we see how it works out with the support call. Thank's for your help though. 

Regards, 

Christoph

Hi Christoph,

did you already set the receive connectors loglevel to "high" ?

Strange thing: sometimes I noticed the same symptoms as on your initial post. Setting the loglevel to "high", and restarting the Transport / Frontend Transport Service solved my issue. Setting it back to low leads to the Problem then, again. So I was able to reproduce a behaviour like that.

Regards,
Martin 

• Proposed as answer by leppard47 Thursday, February 26, 2015 6:17 PM

Hi Christoph

you've got a solution for this issue.

Thanks.

Carlos

This link proved successful in resolving my issue.

https://social.technet.microsoft.com/Forums/en-US/66f0629f-21fb-444b-b3f1-99ed8a4f52b2/slow-mail-flow?forum=exchangesvrsecuremessaging