Help - MessageTrackingLog data - differences

Hi,

I manage two exchange servers in two separate domains \ forests. Domain A works great. Domain B sends and receives fine but OOF messages are not being delivered either internally nor externally.

I have confirmed OOF is correctly enabled on both servers.

[PS] C:\>Get-RemoteDomain | fl AllowedOOFType, AutoReplyEnabled, AutoForwardEnabled

AllowedOOFType     : ExternalLegacy
AutoReplyEnabled   : True
AutoForwardEnabled : True

When I run a MessgaeTrackingLog query on both servers specifically looking at a senders message log and isolating the message subject which is prefixed with "Automatic Reply:" the logs are vastly different.

The working server in Domain A has usually between 10 and 11 steps for processing messages prefixed with "Automatic Reply" but the dysfunctional domain B only ever has 2 steps for messages prefixed with "Automatic Reply"!?!?

Below is the working Exchange server in Domain A - DomainA-EX01 log data...

Timestamp	    ClientHostname	                EventId	         Source	        TotalBytes   Sender	                   Recipients
2015/08/14 16:44		                        RECEIVE	         MAILBOXRULE	7096	          user@domainA.com	   abc@xyz.com
2015/08/14 16:44		                        HAREDIRECTFAIL   SMTP	        9000	          user@domainA.com	   abc@xyz.com
2015/08/14 16:44    DomainA-EX01.domain.local           RECEIVE	         SMTP	        9000	          user@domainA.com	   abc@xyz.com
2015/08/14 16:44		                        RECEIVE	         MAILBOXRULE	7112	          user@domainA.com	   abc@xyz.com
2015/08/14 16:44    DomainA-EX01	                DROP	         ROUTING	8996	          user@domainA.com	   abc@xyz.com
2015/08/14 16:44    DomainA-EX01	                AGENTINFO	 AGENT	        8996	          user@domainA.com	   abc@xyz.com
2015/08/14 16:44		                        HAREDIRECTFAIL   SMTP	        9016	          user@domainA.com	   abc@xyz.com
2015/08/14 16:44    DomainA-EX01.domain.local	        RECEIVE	         SMTP	        9016	          user@domainA.com	   abc@xyz.com
2015/08/14 16:44		                        TRANSFER	 ROUTING	8189	          user@domainA.com	   abc@xyz.com
2015/08/14 16:44    DomainA-EX01	                AGENTINFO	 AGENT	        11937	          user@domainA.com	   abc@xyz.com
2015/08/14 16:44    DomainA-EX01	                SEND	         SMTP	        8500	          user@domainA.com	   abc@xyz.com


Below is the server not sending OOF messages in Domain B - DomainB-EX01 log data...

Timestamp	    ClientHostname	        EventId	         Source	        TotalBytes   Sender	                Recipients
2015/08/14 13:26		                RECEIVE	         MAILBOXRULE	5042	          user@domainB.com qre@lfd.com
2015/08/14 13:26		                RECEIVE	         MAILBOXRULE	5054	          user@domainB.com qre@lfd.com

Note the vast difference is the EventId and Source transactions of log data. Can somebody please talk me through these steps and explain why they are missing on the dysfunctional exchange server?

Both servers also sit behind the same firewall and no MTA is working on the firewall \ proxy so I am ruling that out.

Thanks in advance...

durrie




  • Edited by durrie Monday, August 17, 2015 10:18 AM
August 17th, 2015 8:50am

Hi Durrie,

Please post the Search cmdlet used along with the filter, it might be limiting the result.

If you are not running it against single mesage, do a end to end search on a single message instead of all.

Just to confirm, there are only 2 multi-role Exchange servers in total across both environments.

How about changing -AutoReplyState Scheduled toEnabled and

Free Windows Admin Tool Kit Click here and download it now
August 18th, 2015 4:53am

Hi Satyajit,

Thanks for the suggestions...

I switched my OOF off then closed my outlook.  I run a 'Set-MailboxAutoReplyConfiguration' command on my own account. Then sent a mail from my hotmail account into the organisation and still no OOF reply over 10mins later!

[PS] C:\>Get-MailboxAutoReplyConfiguration -identity me@mydomain.com


RunspaceId       : a0c2ad94-6290-4f52-b2a9-8f8050f23f4a
AutoReplyState   : Enabled
EndTime          : 2015-08-19 11:00:00 AM
ExternalAudience : All

Below is the message tracking command I've been using, I ran it identically on both exchanges and the working one always returns MUCH more data and steps to the mail process flow.

I only posted the evntid and source info originally as I believe this is most pertinent. There is more info I am pulling in the log...see piped objects in command below.

Get-MessageTrackingLog -MessageSubject "Automatic Reply" -Sender me@mydomain.com | Select-Object Timestamp,Clienthostname,eventid,source,totalbytes,sender,@{Name="Recipients";Expression={$_.recipients}},Recipientcount,MessageSubject,serverhostname,clientip,connectorid,messageid,InternalMessageId | Sort-object -Property timestamp | Export-Csv OutFile.csv -NoTypeInformation -delimiter ';'

August 18th, 2015 5:46am

Hi Durrie,

That's really a refined cmdlet usage. I'll keep it noted :)

The short version I use gives enough data to start with, even though it doesn't expand the arrays.

Modify the filters before | select * and use as is:

get-mailboxserver | %{Get-MessageTrackingLog -Server $_ -Sender satyatest2@contoso.com | select *} | sort timestamp | Export-Csv C:\Tracking.csv

You have yet not answered how many servers you have, why I doubt is because you are getting HAREDIRECTFAIL indicating its trying to create a shadow copy for high availability. Running without -Server parameter will run it only for the local exchange server and not for all servers where the message might have moved.

Explaination on your list:

Source

EventId

Eventid-Desc

MAILBOXRULE

RECEIVE

A message was received by the SMTP receive component

SMTP

HAREDIRECTFAIL

A shadow message failed to be created. The details are stored in   the source-context field.

SMTP

RECEIVE

A message was received by the SMTP receive component

MAILBOXRULE

RECEIVE

A message was received by the SMTP receive component

ROUTING

DROP

The details are stored in the source-context field.

AGENT

AGENTINFO

This event is used by transport agents to log custom data.

SMTP

HAREDIRECTFAIL

A shadow message failed to be created. The details are stored in   the source-context field.

SMTP

RECEIVE

A message was received by the SMTP receive component

ROUTING

TRANSFER

Recipients were moved to a forked message because of content   conversion, message recipient limits, or agents.

AGENT

AGENTINFO

This event is used by transport agents to log custom data.

SMTP

SEND

A message was sent by SMTP between transport services.

'SMTP SEND' indicates message has done out of a service

No 'STOREDRIVER DELIVER' indicates, it was not delivered to mailbox and is lost.

Once you run my cmdlet, check for details listed in the 'source-context' field.


server-hostname: The host name or FQDN of the destination server. Is also a important field telling you where the target for this evetnid is.

Referneces:

Event types in the message tracking log:

https://technet.microsoft.com/en-us/library/bb124375(v=exchg.150).aspx#EventTyp

August 18th, 2015 6:39am

Hi,

Have you checked if anything pending in the Queue. Sometimes queue database corruption arises such issues.

Get-Queue

August 18th, 2015 6:57am

Here are the Remote Domain settings on the dysfunctional box...checked and they are identical to the working box...

[PS] C:\>Get-RemoteDomain | fl


RunspaceId                           : e0efee3d-89e4-4ec0-939a-c8a49ffcb61c
DomainName                         : *
IsInternal                              : False
TargetDeliveryDomain           : False
ByteEncoderTypeFor7BitCharsets       : Undefined
CharacterSet                         : iso-8859-1
NonMimeCharacterSet                  : iso-8859-1
AllowedOOFType                          : ExternalLegacy
AutoReplyEnabled                         : True
AutoForwardEnabled                     : True
DeliveryReportEnabled                 : True
NDREnabled                                   : True
MeetingForwardNotificationEnabled    : False
ContentType                                   : MimeHtmlText
DisplaySenderName                        : True
PreferredInternetCodePageForShiftJis : Undefined
RequiredCharsetCoverage              :
TNEFEnabled                          :
LineWrapSize                                   : Unlimited
TrustedMailOutboundEnabled           : False
TrustedMailInboundEnabled             : False
UseSimpleDisplayName                    : False
NDRDiagnosticInfoEnabled              : True
MessageCountThreshold                 : 2147483647
AdminDisplayName                     :
ExchangeVersion                             : 0.1 (8.0.535.0)
Name                                               : Default
DistinguishedName                          : CN=Default,CN=Internet Message Formats,CN=Global Settings,CN=Mydomain JHB,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=Mydomain,DC=local
Identity                                           : Default
Guid                                                : da078cad-10e6-485c-b895-b5a400e9709b
ObjectCategory                              : Mydomain.local/Configuration/Schema/ms-Exch-Domain-Content-Config
ObjectClass                                    : {top, msExchDomainContentConfig}
WhenChanged                               : 2015-08-17 11:46:22 PM
WhenCreated                                : 2005-07-06 10:38:25 AM
WhenChangedUTC                        : 2015-08-17 09:46:22 PM
WhenCreatedUTC                         : 2005-07-06 08:38:25 AM
OrganizationId                                 :
Id                                                  : Default
OriginatingServer                          : AMAJHBDC02.Mydomain.local
IsValid                                           : True
ObjectState                                   : Unchanged

RunspaceId                                    : e0efee3d-89e4-4ec0-939a-c8a49ffcb61c
DomainName                                  : Mydomain.com
IsInternal                                       : False
TargetDeliveryDomain                    : False
ByteEncoderTypeFor7BitCharsets       : Undefined
CharacterSet                                    :
NonMimeCharacterSet                  :
AllowedOOFType                       : ExternalLegacy
AutoReplyEnabled                     : True
AutoForwardEnabled                   : True
DeliveryReportEnabled                : True
NDREnabled                                  : True
MeetingForwardNotificationEnabled    : False
ContentType                                     : MimeHtmlText
DisplaySenderName                           : True
PreferredInternetCodePageForShiftJis : Undefined
RequiredCharsetCoverage              :
TNEFEnabled                          :
LineWrapSize                                    : Unlimited
TrustedMailOutboundEnabled           : False
TrustedMailInboundEnabled            : False
UseSimpleDisplayName                    : False
NDRDiagnosticInfoEnabled             : True
MessageCountThreshold                : 2147483647
AdminDisplayName                         :
ExchangeVersion                            : 0.1 (8.0.535.0)
Name                                              : Mydomain
DistinguishedName                         : CN=Mydomain,CN=Internet Message Formats,CN=Global Settings,CN=MydomainJHB,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=Mydomain,DC=local
Identity                                           : Mydomain
Guid                                                : d749ce4b-3ba0-4615-b3bc-06e5c170bccc
ObjectCategory                            : Mydomain.local/Configuration/Schema/ms-Exch-Domain-Content-Config
ObjectClass                                   : {top, msExchDomainContentConfig}
WhenChanged                              : 2015-08-17 11:48:08 PM
WhenCreated                               : 2015-08-17 02:36:50 PM
WhenChangedUTC                       : 2015-08-17 09:48:08 PM
WhenCreatedUTC                        : 2015-08-17 12:36:50 PM
OrganizationId                             :
Id                                                 : Mydomain
OriginatingServer                         : AMAJHBDC02.Mydomain.local
IsValid                                          : True
ObjectState                                  : Unchanged

Free Windows Admin Tool Kit Click here and download it now
August 18th, 2015 7:30am

Hi Satyajit,

I checked the queues and nothing...

Get-Queue -Exclude Internal

Identity      DeliveryType Status MessageCount Velocity RiskLevel OutboundIPPool NextHopDomain
--------      ------------ ------ ------------ -------- --------- -------------- -------------
amajhbex04\4  SmartHost... Ready  0            0        Normal    0              smtp.pinpointsecuremai...

I ran your cmdlet, and the results are now the same on both servers...only two entries per message with subject 'Automatic Reply:'

Working server....

ServerHostname    SourceContext   ConnectorId     Source             EventId
AISJHBEX02             1.56433E+19                               MAILBOXRULE   RECEIVE
AISJHBEX02             1.57153E+19                               MAILBOXRULE   RECEIVE

Dysfunctional server...

ServerHostname    SourceContext    ConnectorId   Source                EventId
amajhbex04            1.66469E+19                              MAILBOXRULE      RECEIVE
amajhbex04            1.6719E+19                                MAILBOXRULE      RECEIVE

Looking up the 'SourceContext' now to see if the missing digit is proof of a problem...

August 18th, 2015 8:45am

Hi Satyajit,

Both boxes are V15...not sure what you mean by 'see more'...?


I mean if you don't restrict using the subject line, do you see more logs in relation to the issue, try filtering using messageid instead of subjectname.
Free Windows Admin Tool Kit Click here and download it now
August 20th, 2015 6:00am

Hi Lynn-Li,

Thanks for the reply, to answer your queries no OOF works on serverB at all, not internal nor external. I can also confirm that exchange log tracking is enabled.

I am loathe to uninstall the patching as this issue started way before the patching of ServerB. I think I may have been able to identify a start date for this issue back in May so am going to run logs for may and then for this month and compare.

Thanks,

Heath.

August 21st, 2015 2:53am

Hey Satyajit,

I think we may have identified the issue as starting back in May so going to pull Monthly logs for all months back to March and compare.

Will be sure to check messageid for anomalies also...

Thanks,

Free Windows Admin Tool Kit Click here and download it now
August 21st, 2015 2:57am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics