I am in the process of testing a migration from Exchange 2007 to Exchange 2013. There is a single multi-role 2007 server (running on a W2008 DC) to migrate to a single 2013 server running on a W2012 domain member.

2007 is at SP3 rollup 10
2013 was installed at CU2

The problem is that mail fails to be routed from 2007-2013, and builds up in a queue with the 5.7.3 error.

Routing from 2013-2007 works fine, as does outbound from either.

Having read other reports of this symptom and resolutions, I created a dedicated hub transport role receive connector on 2013, scoped to the IP of the 2007 server, configured for TLS and Exchange Server Authentication, with Exchange Server permission groups, and enabled verbose logging on it and the intra-organisation send connector on the 2007 system.

I include the log for a session from both the send/receive below.

My observation, comparing with a (working) log in the opposite direction, is that Exchange 2013 is only advertising X-EXPS GSSAPI NTLM - whereas 2007 adds X-EXPS ExchangeAuth.

I also note that the send connector only logs the inside of the STARTTLS.

The receive connector logs the following:

2013-07-19T15:01:49.946Z,EXCH1\Exchange Hub,08D0520BC10FD38B,0,172.16.11.42:25,172.16.11.10:53143,+,,
2013-07-19T15:01:49.946Z,EXCH1\Exchange Hub,08D0520BC10FD38B,1,172.16.11.42:25,172.16.11.10:53143,*,None,Set Session Permissions
2013-07-19T15:01:49.946Z,EXCH1\Exchange Hub,08D0520BC10FD38B,2,172.16.11.42:25,172.16.11.10:53143,>,"220 EXCH1.CUS.local Microsoft ESMTP MAIL Service ready at Fri, 19 Jul 2013 16:01:49 +0100",
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,3,172.16.11.42:25,172.16.11.10:53143,<,EHLO CUSDC1,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,4,172.16.11.42:25,172.16.11.10:53143,>,250-EXCH1.CUS.local Hello [172.16.11.10],
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,5,172.16.11.42:25,172.16.11.10:53143,>,250-SIZE,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,6,172.16.11.42:25,172.16.11.10:53143,>,250-PIPELINING,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,7,172.16.11.42:25,172.16.11.10:53143,>,250-DSN,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,8,172.16.11.42:25,172.16.11.10:53143,>,250-ENHANCEDSTATUSCODES,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,9,172.16.11.42:25,172.16.11.10:53143,>,250-STARTTLS,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,10,172.16.11.42:25,172.16.11.10:53143,>,250-X-ANONYMOUSTLS,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,11,172.16.11.42:25,172.16.11.10:53143,>,250-AUTH,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,12,172.16.11.42:25,172.16.11.10:53143,>,250-X-EXPS GSSAPI NTLM,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,13,172.16.11.42:25,172.16.11.10:53143,>,250-8BITMIME,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,14,172.16.11.42:25,172.16.11.10:53143,>,250-BINARYMIME,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,15,172.16.11.42:25,172.16.11.10:53143,>,250-CHUNKING,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,16,172.16.11.42:25,172.16.11.10:53143,>,250-XEXCH50,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,17,172.16.11.42:25,172.16.11.10:53143,>,250-XRDST,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,18,172.16.11.42:25,172.16.11.10:53143,>,250 XSHADOWREQUEST,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,19,172.16.11.42:25,172.16.11.10:53143,<,STARTTLS,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,20,172.16.11.42:25,172.16.11.10:53143,>,220 2.0.0 SMTP server ready,
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,21,172.16.11.42:25,172.16.11.10:53143,*,,Sending certificate
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,22,172.16.11.42:25,172.16.11.10:53143,*,"CN=exch1.cus.org, OU=Computing Service, O=Cambridge Union Society, L=Cambridge, S=Cambridgeshire, C=GB",Certificate subject
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,23,172.16.11.42:25,172.16.11.10:53143,*,"CN=DC0-ROOT-CA, DC=CUS, DC=local",Certificate issuer name
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,24,172.16.11.42:25,172.16.11.10:53143,*,1400000006BC292F9EFCF82130000000000006,Certificate serial number
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,25,172.16.11.42:25,172.16.11.10:53143,*,4298EAB6876106F8F8FFCD38012A7A901EC1091A,Certificate thumbprint
2013-07-19T15:01:49.962Z,EXCH1\Exchange Hub,08D0520BC10FD38B,26,172.16.11.42:25,172.16.11.10:53143,*,exch1.cus.org;exch1.cus.local;AutoDiscover.cambridge-union.org;AutoDiscover.CUS.local;AutoDiscover.cus.org;cambridge-union.org;CUS.local;cus.org;exch1,Certificate alternate names
2013-07-19T15:01:50.024Z,EXCH1\Exchange Hub,08D0520BC10FD38B,27,172.16.11.42:25,172.16.11.10:53143,*,,TLS negotiation succeeded
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,28,172.16.11.42:25,172.16.11.10:53143,<,EHLO CUS-DC1.CUS.local,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,29,172.16.11.42:25,172.16.11.10:53143,*,,TlsDomainCapabilities='None'; Status='NoRemoteCertificate'
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,30,172.16.11.42:25,172.16.11.10:53143,>,250-EXCH1.CUS.local Hello [172.16.11.10],
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,31,172.16.11.42:25,172.16.11.10:53143,>,250-SIZE,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,32,172.16.11.42:25,172.16.11.10:53143,>,250-PIPELINING,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,33,172.16.11.42:25,172.16.11.10:53143,>,250-DSN,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,34,172.16.11.42:25,172.16.11.10:53143,>,250-ENHANCEDSTATUSCODES,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,35,172.16.11.42:25,172.16.11.10:53143,>,250-AUTH,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,36,172.16.11.42:25,172.16.11.10:53143,>,250-X-EXPS GSSAPI NTLM,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,37,172.16.11.42:25,172.16.11.10:53143,>,250-8BITMIME,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,38,172.16.11.42:25,172.16.11.10:53143,>,250-BINARYMIME,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,39,172.16.11.42:25,172.16.11.10:53143,>,250-CHUNKING,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,40,172.16.11.42:25,172.16.11.10:53143,>,250-XEXCH50,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,41,172.16.11.42:25,172.16.11.10:53143,>,250-XRDST,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,42,172.16.11.42:25,172.16.11.10:53143,>,250 XSHADOWREQUEST,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,43,172.16.11.42:25,172.16.11.10:53143,<,QUIT,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,44,172.16.11.42:25,172.16.11.10:53143,>,221 2.0.0 Service closing transmission channel,
2013-07-19T15:01:50.040Z,EXCH1\Exchange Hub,08D0520BC10FD38B,45,172.16.11.42:25,172.16.11.10:53143,-,,Local

The Exchange 2007 Send connector logs the following:

2013-07-19T15:02:01.500Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,0,,172.16.11.42:25,*,,attempting to connect
2013-07-19T15:02:01.500Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,1,127.0.0.1:53142,172.16.11.42:25,+,,
2013-07-19T15:02:01.749Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,2,127.0.0.1:53142,172.16.11.42:25,<,"220 EXCH1.CUS.local Microsoft ESMTP MAIL Service ready at Fri, 19 Jul 2013 16:01:49 +0100",
2013-07-19T15:02:01.749Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,3,127.0.0.1:53142,172.16.11.42:25,>,EHLO CUS-DC1.CUS.local,
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,4,127.0.0.1:53142,172.16.11.42:25,<,250-EXCH1.CUS.local Hello [172.16.11.10],
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,5,127.0.0.1:53142,172.16.11.42:25,<,250-SIZE,
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,6,127.0.0.1:53142,172.16.11.42:25,<,250-PIPELINING,
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,7,127.0.0.1:53142,172.16.11.42:25,<,250-DSN,
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,8,127.0.0.1:53142,172.16.11.42:25,<,250-ENHANCEDSTATUSCODES,
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,9,127.0.0.1:53142,172.16.11.42:25,<,250-AUTH,
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,10,127.0.0.1:53142,172.16.11.42:25,<,250-X-EXPS GSSAPI NTLM,
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,11,127.0.0.1:53142,172.16.11.42:25,<,250-8BITMIME,
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,12,127.0.0.1:53142,172.16.11.42:25,<,250-XEXCH50,
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,13,127.0.0.1:53142,172.16.11.42:25,<,250-XRDST,
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,14,127.0.0.1:53142,172.16.11.42:25,<,250-XSHADOWREQUEST,
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,15,127.0.0.1:53142,172.16.11.42:25,<,250 STARTTLS,
2013-07-19T15:02:01.751Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,16,127.0.0.1:53142,172.16.11.42:25,>,QUIT,
2013-07-19T15:02:01.752Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,17,127.0.0.1:53142,172.16.11.42:25,<,221 2.0.0 Service closing transmission channel,
2013-07-19T15:02:01.752Z,Intra-Organization SMTP Send Connector,08D05205F7A57124,18,127.0.0.1:53142,172.16.11.42:25,-,,Local

Hi

Please check this KB

http://support.microsoft.com/kb/979175/en-us

To resolve this issue, all receive connectors that receive internal e-mail messages should have Exchange Authentication enabled. 

Note If there is a firewall located between the two servers, the Extended SMTP verbs X-ANONYMOUSTLS, X-EXPS, and GSSAPI must be able to pass.

Cheers

Sadly that doesn't help: Exchange Authentication is enabled on the receive connector, and the log indicates that that receive connector is the one being used.

X-ANONYMOUSTLS, X-EXPS and GSSAPI are all advertised (see the log above). The two servers are on the same network segment [so no firewall between], and both sides logs match.

The only difference I can see between this and a working session in the other direction is that 2007 is advertising X-EXPS ExchangeAuth

I found the problem here: it transpires that security software on the 2007 server (Avast) was transparently intercepting the outbound connection, and issuing a STARTTLS before handing back. As a result, the 2007 server didn't see the X-ANONYMOUSTLS - since that is not advertised once the connection is already secured - and ExchangeAuth is only advertised if X-ANONYMOUSTLS was used.

The key pointers in the logs were:

• The SmtpSend protocol log only showed the post-tls connection
• The SmtpSend protocol log showed STARTTLS appended to the advertised verbs
• The SmtpReceive protocol log showed that STARTTLS and X-ANONYMOUSTLS were advertised, but STARTTLS was issued

This is notably different from the Cisco appliance problems noted in the KB - in particular because verb filtering is not involved; instead the connection is caused to be in different states on the two servers.