I realise this might be slightly off topic, as I'm asking for advice on the OS configuration specific to a Domain Controller, rather than the Directory Service itself, but I thought this was still the best place for my question

I have a 2008 forest with a root domain and 4 child domains, 53 DC's in total over multiple sites.

I ran a DCDIAG /c /e from a DC in the root domain, and +40 of the DC's returned the following errors:

Starting test: Advertising
Fatal Error:DsGetDcName (DCName) call failed, error 1722 The Locator could not find the server.
DCName failed test Advertising

Starting test: MachineAccount
Could not open pipe with [DCName]:failed with 53
The network path was not found.
Could not get NetBIOSDomainName
Failed can not test for HOST SPN
Failed can not test for HOST SPN
DCName passed test MachineAccount

Starting test: NCSecDesc
Ldap search capability attribute search failed on server DCName, return value = 81
DCName failed test NCSecDesc

Starting test: NetLogons
[DCName] An net use or LsaPolicy operation failed with error 53, The network path was not found..:
DCName failed test NetLogons"

I did some testing and worked out DC's that passed these tests were either in the same domain or the same site as the source DC. Conclusion; it was resolving the DNS name in the former and doing a broadcast lookup for DC's in the same site for the latter. Therefore all DC's in child domains and not in the same site failed the tests listed above.

Therefore my question is; is this normal, should I just expect a dcdiag /e to fail in a multi-domain forest?  How would you normally configure DNS or the OS of the DC to resolve this?  Do people normally change the network properties to append DNS suffixes for all the domains/is it okay to append the DNS suffixes, or will this cause issues on the DC? 

Hi,

Have you done any migration of DC's also Can you provide the output of following command.

Repadmin /replsum /errorsonly

Hi

 First;error 1722 seems like there is connectivity issue,

- Have you checked the Firewall ports&Connectivity?

- Do you install AV on dc's,if yes check the AV logs.Also disbale AV and test.

check these articles also;

https://technet.microsoft.com/en-us/library/replication-error-1722-the-rpc-server-is-unavaible%28v=ws.10%29.aspx?f=255&MSPPError=-2147217396

http://blogs.technet.com/askds/archive/2011/03/22/what-does-dcdiag-actually-do.aspx

• Proposed as answer by Alex LvMicrosoft contingent staff, Moderator 32 minutes ago

Hi

 First;error 1722 seems like there is connectivity issue,

- Have you checked the Firewall ports&Connectivity?

- Do you install AV on dc's,if yes check the AV logs.Also disbale AV and test.

check these articles also;

https://technet.microsoft.com/en-us/library/replication-error-1722-the-rpc-server-is-unavaible%28v=ws.10%29.aspx?f=255&MSPPError=-2147217396

http://blogs.technet.com/askds/archive/2011/03/22/what-does-dcdiag-actually-do.aspx

• Proposed as answer by Alex LvMicrosoft contingent staff, Moderator Monday, May 18, 2015 6:28 AM

Re DC migration, not that I know off recently.  However I've only been in the company a few weeks.

DC names changed below.

C:\Windows\system32>Repadmin /replsum /errorsonly
Replication Summary Start Time: 2015-05-14 10:24:04

Beginning data collection for replication summary, this may take awhile:
  ..................................................
  ......


Source DSA          largest delta    fails/total %%   error
 A-DC01                  10m:54s    0 /  18    0
 A-DC02                  10m:36s    0 /  16    0
 B-DC01                  36m:54s    0 /  36    0
 B-DC02                  27m:12s    0 /  36    0
 B-DC03                  22m:11s    0 /  23    0
 B-DC04                  10m:34s    0 /  18    0
 B-DC05                  24m:23s    0 /  25    0
 B-DC08                  36m:54s    0 /  25    0
 B-DC10                  34m:36s    0 /  27    0
 B-DC11                       0s    0 /   9    0
 B-DC12                  09m:13s    0 /   9    0
 B-DC13                  34m:36s    0 /  27    0
 C-DC30                  09m:18s    0 /  53    0
 D-DC01                  04m:47s    0 /  10    0
 D-DC02                  04m:32s    0 /   5    0
 E-DC02                       0s    0 /   9    0
 ROOTDC01                36m:54s    0 /  23    0
 F-DC11                  22m:50s    0 /  34    0
 F-DC12                  22m:32s    0 /  25    0
 F-DC13                  22m:31s    0 /  32    0
 F-DC15                  03m:10s    0 /   9    0
 F-DC16                  22m:38s    0 /  48    0
 ROOT03                  21m:26s    0 /  32    0
 G-DC09                  17m:59s    0 /  23    0
 H-DC01                  16m:51s    0 /  97    0
 H-DC02                  17m:19s    0 /  77    0
 H-DC03                  17m:24s    0 /  79    0
 H-DC04                  35m:28s    0 /  30    0
 J-DC01                  17m:44s    0 /  23    0
 K-DC01                  03m:22s    0 /   9    0
 L-DC01          26d.23h:39m:53s    9 /   9  100  (8446) The replication opera
tion failed to allocate memory.
 M-DC01                  06m:43s    0 /  27    0
 N-DC01                  08m:12s    0 /   9    0
 ROOT02                  17m:41s    0 /  23    0
 P-DC01                  36m:16s    0 /  18    0
 Q-DC03                  16m:58s    0 /  23    0


Destination DSA     largest delta    fails/total %%   error
 A-DC01                  13m:47s    0 /  16    0
 A-DC02                  12m:12s    0 /  18    0
 B-DC01                  27m:36s    0 /  27    0
 B-DC02                  34m:56s    0 /  27    0
 B-DC03                  25m:50s    0 /  32    0
 B-DC05                  38m:00s    0 /  25    0
 B-DC06                  05m:58s    0 /   9    0
 B-DC08                  25m:27s    0 /  34    0
 B-DC10                  27m:38s    0 /  27    0
 B-DC11                  01m:23s    0 /   9    0
 B-DC12                  02m:49s    0 /   9    0
 B-DC13                  30m:30s    0 /  36    0
 C-DC30                  02m:04s    0 /  39    0
 D-DC01                  06m:06s    0 /  10    0
 D-DC02                  08m:18s    0 /  14    0
 E-DC02                  10m:38s    0 /   9    0
 R-DC01                  13m:21s    0 /   9    0
 S-DC01                  07m:19s    0 /   5    0
 ROOT01                  27m:38s    0 /  23    0
 F-DC11                  22m:44s    0 /  25    0
 F-DC12                  23m:03s    0 /  25    0
 F-DC13                  35m:41s    0 /  39    0
 F-DC15                  09m:02s    0 /   9    0
 F-DC16                  25m:14s    0 /  50    0
 ROOT03                  21m:52s    0 /  32    0
 G-DC09                  19m:24s    0 /  23    0
 S-DC01                  10m:42s    0 /   9    0
 T-DC01                  06m:46s    0 /   9    0
 U-DC01                  07m:41s    0 /   9    0
 H-DC01          26d.23h:40m:06s    9 /  43   20  (8446) The replication opera
tion failed to allocate memory.
 H-DC02                  17m:53s    0 /  41    0
 H-DC03                  18m:01s    0 /  43    0
 H-DC04                  23m:58s    0 /  30    0
 J-DC01                  19m:44s    0 /  23    0
 K-DC01                  11m:38s    0 /   9    0
 V-DC01                     :54s    0 /   9    0
 W-DC02                  05m:18s    0 /   9    0
 X-DC01                  06m:09s    0 /   9    0
 Y-DC02                  12m:44s    0 /   9    0
 M-DC01                  09m:58s    0 /  18    0
 N-DC01                  06m:59s    0 /   9    0
 Z-DC01                  14m:36s    0 /   9    0
 ROOT02                  18m:07s    0 /  23    0
 P-DC01                  06m:45s    0 /  18    0
 AA-DC01                 11m:23s    0 /  27    0
 Q-DC03                  18m:59s    0 /  23    0
 AB-DC02                 09m:55s    0 /   9    0
 AC-DC01                 39m:26s    0 /  27    0


Experienced the following operational errors trying to retrieve replication info
rmation:
          58 - B-DC04.APAC.domain.dom
          58 - AD-DC04.emea.domain.dom
          58 - AE-DC01.NA.domain.dom
          58 - AF-DC02.NA.domain.dom
          58 - L-DC01.NA.domain.dom

Hi,

L-DC01          26d.23h:39m:53s    9 /   9  100  (8446) The replication operation failed to allocate memory.

H-DC01          26d.23h:40m:06s    9 /  43   20  (8446) The replication operation failed to allocate memory.

From above It seems L-DC01 is down since last 26 days.

          58 - B-DC04.APAC.domain.dom
          58 - AD-DC04.emea.domain.dom
          58 - AE-DC01.NA.domain.dom
          58 - AF-DC02.NA.domain.dom
          58 - L-DC01.NA.domain.dom

for above Check the DNS and Firewall etc. is configured correctly. 

Thanks Purvesh,

I'm aware that some DC's are down, I have a lot to fix!

Does that account for the DNS 'issue' I discussed with the 40+ DC's that are up and working?

Hi,

can you provide the following command output.

dcdiag /v /c /d /e /s:FQDN

Also possible verify the DNS settings and subnets & Site configuration.

For me, it seems there are connectivity problems between the DCs so assure that no firewall is blocking ports http://technet.microsoft.com/en-us/library/dd772723(WS.10).aspx

Also, You have to check your security software like antivirus or any other tools, its not blocking the communication b/w the DC's.

Devaraj, I suspected that too, so I used the PortQry tool to check that.  When I attempted to connect to a DC that I have an issue with using the either the IP address or FQDN I had no issues.  Using the name only, it couldn't resolve the name.  We don't have any internal network firewalls, or ACL between VLAN (so I'm told).  I will check the AV software, but I need to raise a change to do that.

thanks

• Edited by idarryl Thursday, May 14, 2015 12:59 PM typo

Devaraj, I suspected that too, so I used the PortQry tool to check that.  When I attempted to connect to a DC that I have an issue with using the either the IP address or FQDN I had no issues.  Using the name only, it couldn't resolve the name.  We don't have any internal network firewalls, or ACL between VLAN (so I'm told).  I will check the AV software, but I need to raise a change to do that.

thanks

• Edited by idarryl Thursday, May 14, 2015 12:59 PM typo

Hi,

make sure you have check the port for following.

portqry -n X.X.X.X -e 3269 -p both

portqry -n X.X.X.X -e 3268 -p both

portqry -n X.X.X.X -e 389 -p both

portqry -n X.X.X.X -e 53 -p both

portqry -n X.X.X.X -e 123 -p both

portqry -n X.X.X.X -e 135 -p both

portqry -n X.X.X.X -e 137 -p both

portqry -n X.X.X.X -e 88 -p both

• Proposed as answer by Alex LvMicrosoft contingent staff, Moderator 32 minutes ago

Hi,

make sure you have check the port for following.

portqry -n X.X.X.X -e 3269 -p both

portqry -n X.X.X.X -e 3268 -p both

portqry -n X.X.X.X -e 389 -p both

portqry -n X.X.X.X -e 53 -p both

portqry -n X.X.X.X -e 123 -p both

portqry -n X.X.X.X -e 135 -p both

portqry -n X.X.X.X -e 137 -p both

portqry -n X.X.X.X -e 88 -p both

• Proposed as answer by Alex LvMicrosoft contingent staff, Moderator Monday, May 18, 2015 6:28 AM

I had check all of those except 123.  This is the FQDN output, followed by the dombios (single name) output:
The checks though FQDN seem to all pass for me, except 137, which i would expect, as it's not on the same subnet

PS that dcdiag /v /c /d /e /s:FQDN that you asked for is still running

=============================================
 Starting portqry.exe -n A-DC01.uk.domain.dom -e 135 -p TCP ...
Querying target system called:
 A-DC01.uk.domain.dom
Attempting to resolve name to IP address...
Name resolved to 10.11.4.10
querying...
TCP port 135 (epmap service): LISTENING
Using ephemeral source port
Querying Endpoint Mapper Database...
Server's response:
UUID: d95afe70-a6d5-4259-822e-2c84da1ddb0d 
ncacn_ip_tcp:A-DC01.uk.domain.dom[49152]
UUID: 367abb81-9844-35f1-ad32-98f038001003 
ncacn_ip_tcp:A-DC01.uk.domain.dom[57795]
UUID: 50abc2a4-574d-40b3-9d66-ee4fd5fba076 
ncacn_ip_tcp:A-DC01.uk.domain.dom[49353]
UUID: 3d267954-eeb7-11d1-b94e-00c04fa3080d 
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\HydraLsPipe]
UUID: 3d267954-eeb7-11d1-b94e-00c04fa3080d 
ncacn_ip_tcp:A-DC01.uk.domain.dom[49192]
UUID: 12d4b7c8-77d5-11d1-8c24-00c04fa3080d 
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\HydraLsPipe]
UUID: 12d4b7c8-77d5-11d1-8c24-00c04fa3080d 
ncacn_ip_tcp:A-DC01.uk.domain.dom[49192]
UUID: f5cc59b4-4264-101a-8c59-08002b2f8426 NtFrs Service
ncacn_ip_tcp:A-DC01.uk.domain.dom[49180]
UUID: d049b186-814f-11d1-9a3c-00c04fc9b232 NtFrs API
ncacn_ip_tcp:A-DC01.uk.domain.dom[49180]
UUID: a00c021c-2be2-11d2-b678-0000f87a8f8e PERFMON SERVICE
ncacn_ip_tcp:A-DC01.uk.domain.dom[49180]
UUID: e3514235-4b06-11d1-ab04-00c04fc2dcd2 MS NT Directory DRS Interface
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\lsass]
UUID: e3514235-4b06-11d1-ab04-00c04fc2dcd2 MS NT Directory DRS Interface
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\protected_storage]
UUID: e3514235-4b06-11d1-ab04-00c04fc2dcd2 MS NT Directory DRS Interface
ncacn_ip_tcp:A-DC01.uk.domain.dom[49155]
UUID: e3514235-4b06-11d1-ab04-00c04fc2dcd2 MS NT Directory DRS Interface
ncacn_http:A-DC01.uk.domain.dom[49158]
UUID: f5cc5a18-4264-101a-8c59-08002b2f8426 MS NT Directory NSP Interface
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\lsass]
UUID: f5cc5a18-4264-101a-8c59-08002b2f8426 MS NT Directory NSP Interface
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\protected_storage]
UUID: f5cc5a18-4264-101a-8c59-08002b2f8426 MS NT Directory NSP Interface
ncacn_ip_tcp:A-DC01.uk.domain.dom[49155]
UUID: f5cc5a18-4264-101a-8c59-08002b2f8426 MS NT Directory NSP Interface
ncacn_http:A-DC01.uk.domain.dom[49158]
UUID: 12345778-1234-abcd-ef00-0123456789ab 
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\lsass]
UUID: 12345778-1234-abcd-ef00-0123456789ab 
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\protected_storage]
UUID: 12345778-1234-abcd-ef00-0123456789ab 
ncacn_ip_tcp:A-DC01.uk.domain.dom[49155]
UUID: 12345778-1234-abcd-ef00-0123456789ab 
ncacn_http:A-DC01.uk.domain.dom[49158]
UUID: 12345778-1234-abcd-ef00-0123456789ac 
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\lsass]
UUID: 12345778-1234-abcd-ef00-0123456789ac 
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\protected_storage]
UUID: 12345778-1234-abcd-ef00-0123456789ac 
ncacn_ip_tcp:A-DC01.uk.domain.dom[49155]
UUID: 12345778-1234-abcd-ef00-0123456789ac 
ncacn_http:A-DC01.uk.domain.dom[49158]
UUID: 12345778-1234-abcd-ef00-0123456789ac 
ncacn_ip_tcp:A-DC01.uk.domain.dom[49159]
UUID: 12345678-1234-abcd-ef00-01234567cffb 
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\lsass]
UUID: 12345678-1234-abcd-ef00-01234567cffb 
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\protected_storage]
UUID: 12345678-1234-abcd-ef00-01234567cffb 
ncacn_ip_tcp:A-DC01.uk.domain.dom[49155]
UUID: 12345678-1234-abcd-ef00-01234567cffb 
ncacn_http:A-DC01.uk.domain.dom[49158]
UUID: 12345678-1234-abcd-ef00-01234567cffb 
ncacn_ip_tcp:A-DC01.uk.domain.dom[49159]
UUID: b25a52bf-e5dd-4f4a-aea6-8ca7272a0e86 KeyIso
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\lsass]
UUID: b25a52bf-e5dd-4f4a-aea6-8ca7272a0e86 KeyIso
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\protected_storage]
UUID: b25a52bf-e5dd-4f4a-aea6-8ca7272a0e86 KeyIso
ncacn_ip_tcp:A-DC01.uk.domain.dom[49155]
UUID: b25a52bf-e5dd-4f4a-aea6-8ca7272a0e86 KeyIso
ncacn_http:A-DC01.uk.domain.dom[49158]
UUID: b25a52bf-e5dd-4f4a-aea6-8ca7272a0e86 KeyIso
ncacn_ip_tcp:A-DC01.uk.domain.dom[49159]
UUID: b25a52bf-e5dd-4f4a-aea6-8ca7272a0e86 KeyIso
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\efsrpc]
UUID: 3473dd4d-2e88-4006-9cba-22570909dd10 WinHttp Auto-Proxy Service
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\W32TIME_ALT]
UUID: 1ff70682-0a51-30e8-076d-740be8cee98b 
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\atsvc]
UUID: 378e52b0-c0a9-11cf-822d-00aa0051e40f 
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\atsvc]
UUID: 86d35949-83c9-4044-b424-db363231fd0c 
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\atsvc]
UUID: 86d35949-83c9-4044-b424-db363231fd0c 
ncacn_ip_tcp:A-DC01.uk.domain.dom[49154]
UUID: 98716d03-89ac-44c7-bb8c-285824e51c4a XactSrv service
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\atsvc]
UUID: 98716d03-89ac-44c7-bb8c-285824e51c4a XactSrv service
ncacn_ip_tcp:A-DC01.uk.domain.dom[49154]
UUID: 552d076a-cb29-4e44-8b6a-d15e59e2c0af IP Transition Configuration endpoint
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\atsvc]
UUID: 552d076a-cb29-4e44-8b6a-d15e59e2c0af IP Transition Configuration endpoint
ncacn_ip_tcp:A-DC01.uk.domain.dom[49154]
UUID: 552d076a-cb29-4e44-8b6a-d15e59e2c0af IP Transition Configuration endpoint
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\srvsvc]
UUID: c9ac6db5-82b7-4e55-ae8a-e464ed7b4277 Impl friendly name
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\atsvc]
UUID: c9ac6db5-82b7-4e55-ae8a-e464ed7b4277 Impl friendly name
ncacn_ip_tcp:A-DC01.uk.domain.dom[49154]
UUID: c9ac6db5-82b7-4e55-ae8a-e464ed7b4277 Impl friendly name
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\srvsvc]
UUID: 30b044a5-a225-43f0-b3a4-e060df91f9c1 
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\atsvc]
UUID: 30b044a5-a225-43f0-b3a4-e060df91f9c1 
ncacn_ip_tcp:A-DC01.uk.domain.dom[49154]
UUID: 30b044a5-a225-43f0-b3a4-e060df91f9c1 
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\srvsvc]
UUID: 7d814569-35b3-4850-bb32-83035fcebf6e IAS RPC server
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\atsvc]
UUID: 7d814569-35b3-4850-bb32-83035fcebf6e IAS RPC server
ncacn_ip_tcp:A-DC01.uk.domain.dom[49154]
UUID: 7d814569-35b3-4850-bb32-83035fcebf6e IAS RPC server
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\srvsvc]
UUID: 201ef99a-7fa0-444c-9399-19ba84f12a1a AppInfo
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\atsvc]
UUID: 201ef99a-7fa0-444c-9399-19ba84f12a1a AppInfo
ncacn_ip_tcp:A-DC01.uk.domain.dom[49154]
UUID: 201ef99a-7fa0-444c-9399-19ba84f12a1a AppInfo
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\srvsvc]
UUID: 5f54ce7d-5b79-4175-8584-cb65313a0e98 AppInfo
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\atsvc]
UUID: 5f54ce7d-5b79-4175-8584-cb65313a0e98 AppInfo
ncacn_ip_tcp:A-DC01.uk.domain.dom[49154]
UUID: 5f54ce7d-5b79-4175-8584-cb65313a0e98 AppInfo
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\srvsvc]
UUID: fd7a0523-dc70-43dd-9b2e-9c5ed48225b1 AppInfo
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\atsvc]
UUID: fd7a0523-dc70-43dd-9b2e-9c5ed48225b1 AppInfo
ncacn_ip_tcp:A-DC01.uk.domain.dom[49154]
UUID: fd7a0523-dc70-43dd-9b2e-9c5ed48225b1 AppInfo
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\srvsvc]
UUID: 58e604e8-9adb-4d2e-a464-3b0683fb1480 AppInfo
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\atsvc]
UUID: 58e604e8-9adb-4d2e-a464-3b0683fb1480 AppInfo
ncacn_ip_tcp:A-DC01.uk.domain.dom[49154]
UUID: 58e604e8-9adb-4d2e-a464-3b0683fb1480 AppInfo
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\srvsvc]
UUID: f6beaff7-1e19-4fbb-9f8f-b89e2018337c Event log TCPIP
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\eventlog]
UUID: f6beaff7-1e19-4fbb-9f8f-b89e2018337c Event log TCPIP
ncacn_ip_tcp:A-DC01.uk.domain.dom[49153]
UUID: 30adc50c-5cbc-46ce-9a0e-91914789e23c NRP server endpoint
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\eventlog]
UUID: 30adc50c-5cbc-46ce-9a0e-91914789e23c NRP server endpoint
ncacn_ip_tcp:A-DC01.uk.domain.dom[49153]
UUID: 3c4728c5-f0ab-448b-bda1-6ce01eb0a6d5 DHCP Client LRPC Endpoint
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\eventlog]
UUID: 3c4728c5-f0ab-448b-bda1-6ce01eb0a6d5 DHCP Client LRPC Endpoint
ncacn_ip_tcp:A-DC01.uk.domain.dom[49153]
UUID: 3c4728c5-f0ab-448b-bda1-6ce01eb0a6d6 DHCPv6 Client LRPC Endpoint
ncacn_np:A-DC01.uk.domain.dom[\\pipe\\eventlog]
UUID: 3c4728c5-f0ab-448b-bda1-6ce01eb0a6d6 DHCPv6 Client LRPC Endpoint
ncacn_ip_tcp:A-DC01.uk.domain.dom[49153]
UUID: 76f226c3-ec14-4325-8a99-6a46348418af 
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\InitShutdown]
UUID: d95afe70-a6d5-4259-822e-2c84da1ddb0d 
ncacn_np:A-DC01.uk.domain.dom[\\PIPE\\InitShutdown]
Total endpoints found: 79
==== End of RPC Endpoint Mapper query response ====
portqry.exe -n A-DC01.uk.domain.dom -e 135 -p TCP exits with return code 0x00000000.
=============================================
 Starting portqry.exe -n A-DC01.uk.domain.dom -e 389 -p BOTH ...
Querying target system called:
 A-DC01.uk.domain.dom
Attempting to resolve name to IP address...
Name resolved to 10.11.4.10
querying...
TCP port 389 (ldap service): LISTENING
Using ephemeral source port
Sending LDAP query to TCP port 389...
LDAP query response:
currentdate: 05/14/2015 13:16:38 (unadjusted GMT)
subschemaSubentry: CN=Aggregate,CN=Schema,CN=Configuration,DC=domain,DC=dom
dsServiceName: CN=NTDS Settings,CN=A-DC01,CN=Servers,CN=UK-London,CN=Sites,CN=Configuration,DC=domain,DC=dom
namingContexts: CN=Configuration,DC=domain,DC=dom
defaultNamingContext: DC=UK,DC=domain,DC=dom
schemaNamingContext: CN=Schema,CN=Configuration,DC=domain,DC=dom
configurationNamingContext: CN=Configuration,DC=domain,DC=dom
rootDomainNamingContext: DC=domain,DC=dom
supportedControl: 1.2.840.113556.1.4.319
supportedLDAPVersion: 3
supportedLDAPPolicies: MaxPoolThreads
highestCommittedUSN: 343266713
supportedSASLMechanisms: GSSAPI
dnsHostName: A-DC01.UK.domain.dom
ldapServiceName: domain.dom:A-DC01$@UK.domain.dom
serverName: CN=A-DC01,CN=Servers,CN=UK-London,CN=Sites,CN=Configuration,DC=domain,DC=dom
supportedCapabilities: 1.2.840.113556.1.4.800
isSynchronized: TRUE
isGlobalCatalogReady: TRUE
domainFunctionality: 4
forestFunctionality: 2
domainControllerFunctionality: 4
======== End of LDAP query response ========
UDP port 389 (unknown service): LISTENING or FILTERED
Using ephemeral source port
Sending LDAP query to UDP port 389...
LDAP query response:
currentdate: 05/14/2015 13:16:42 (unadjusted GMT)
subschemaSubentry: CN=Aggregate,CN=Schema,CN=Configuration,DC=domain,DC=dom
dsServiceName: CN=NTDS Settings,CN=A-DC01,CN=Servers,CN=UK-London,CN=Sites,CN=Configuration,DC=domain,DC=dom
namingContexts: CN=Configuration,DC=domain,DC=dom
defaultNamingContext: DC=UK,DC=domain,DC=dom
schemaNamingContext: CN=Schema,CN=Configuration,DC=domain,DC=dom
configurationNamingContext: CN=Configuration,DC=domain,DC=dom
rootDomainNamingContext: DC=domain,DC=dom
supportedControl: 1.2.840.113556.1.4.319
supportedLDAPVersion: 3
supportedLDAPPolicies: MaxPoolThreads
highestCommittedUSN: 343266723
supportedSASLMechanisms: GSSAPI
dnsHostName: A-DC01.UK.domain.dom
ldapServiceName: domain.dom:A-DC01$@UK.domain.dom
serverName: CN=A-DC01,CN=Servers,CN=UK-London,CN=Sites,CN=Configuration,DC=domain,DC=dom
supportedCapabilities: 1.2.840.113556.1.4.800
isSynchronized: TRUE
isGlobalCatalogReady: TRUE
domainFunctionality: 4
forestFunctionality: 2
domainControllerFunctionality: 4
======== End of LDAP query response ========
UDP port 389 is LISTENING
portqry.exe -n A-DC01.uk.domain.dom -e 389 -p BOTH exits with return code 0x00000000.
=============================================
 Starting portqry.exe -n A-DC01.uk.domain.dom -e 636 -p TCP ...
Querying target system called:
 A-DC01.uk.domain.dom
Attempting to resolve name to IP address...
Name resolved to 10.11.4.10
querying...
TCP port 636 (ldaps service): LISTENING
portqry.exe -n A-DC01.uk.domain.dom -e 636 -p TCP exits with return code 0x00000000.
=============================================
 Starting portqry.exe -n A-DC01.uk.domain.dom -e 3268 -p TCP ...
Querying target system called:
 A-DC01.uk.domain.dom
Attempting to resolve name to IP address...
Name resolved to 10.11.4.10
querying...
TCP port 3268 (msft-gc service): LISTENING
Using ephemeral source port
Sending LDAP query to TCP port 3268...
LDAP query response:
currentdate: 05/14/2015 13:16:43 (unadjusted GMT)
subschemaSubentry: CN=Aggregate,CN=Schema,CN=Configuration,DC=domain,DC=dom
dsServiceName: CN=NTDS Settings,CN=A-DC01,CN=Servers,CN=UK-London,CN=Sites,CN=Configuration,DC=domain,DC=dom
namingContexts: CN=Configuration,DC=domain,DC=dom
defaultNamingContext: DC=UK,DC=domain,DC=dom
schemaNamingContext: CN=Schema,CN=Configuration,DC=domain,DC=dom
configurationNamingContext: CN=Configuration,DC=domain,DC=dom
rootDomainNamingContext: DC=domain,DC=dom
supportedControl: 1.2.840.113556.1.4.319
supportedLDAPVersion: 3
supportedLDAPPolicies: MaxPoolThreads
highestCommittedUSN: 343266723
supportedSASLMechanisms: GSSAPI
dnsHostName: A-DC01.UK.domain.dom
ldapServiceName: domain.dom:A-DC01$@UK.domain.dom
serverName: CN=A-DC01,CN=Servers,CN=UK-London,CN=Sites,CN=Configuration,DC=domain,DC=dom
supportedCapabilities: 1.2.840.113556.1.4.800
isSynchronized: TRUE
isGlobalCatalogReady: TRUE
domainFunctionality: 4
forestFunctionality: 2
domainControllerFunctionality: 4
======== End of LDAP query response ========
portqry.exe -n A-DC01.uk.domain.dom -e 3268 -p TCP exits with return code 0x00000000.
=============================================
 Starting portqry.exe -n A-DC01.uk.domain.dom -e 3269 -p TCP ...
Querying target system called:
 A-DC01.uk.domain.dom
Attempting to resolve name to IP address...
Name resolved to 10.11.4.10
querying...
TCP port 3269 (msft-gc-ssl service): LISTENING
portqry.exe -n A-DC01.uk.domain.dom -e 3269 -p TCP exits with return code 0x00000000.
=============================================
 Starting portqry.exe -n A-DC01.uk.domain.dom -e 53 -p BOTH ...
Querying target system called:
 A-DC01.uk.domain.dom
Attempting to resolve name to IP address...
Name resolved to 10.11.4.10
querying...
TCP port 53 (domain service): LISTENING
UDP port 53 (domain service): LISTENING
portqry.exe -n A-DC01.uk.domain.dom -e 53 -p BOTH exits with return code 0x00000000.
=============================================
 Starting portqry.exe -n A-DC01.uk.domain.dom -e 88 -p BOTH ...
Querying target system called:
 A-DC01.uk.domain.dom
Attempting to resolve name to IP address...
Name resolved to 10.11.4.10
querying...
TCP port 88 (kerberos service): LISTENING
UDP port 88 (kerberos service): LISTENING or FILTERED
portqry.exe -n A-DC01.uk.domain.dom -e 88 -p BOTH exits with return code 0x00000002.
=============================================
 Starting portqry.exe -n A-DC01.uk.domain.dom -e 445 -p TCP ...
Querying target system called:
 A-DC01.uk.domain.dom
Attempting to resolve name to IP address...
Name resolved to 10.11.4.10
querying...
TCP port 445 (microsoft-ds service): LISTENING
portqry.exe -n A-DC01.uk.domain.dom -e 445 -p TCP exits with return code 0x00000000.
=============================================
 Starting portqry.exe -n A-DC01.uk.domain.dom -e 137 -p UDP ...
Querying target system called:
 A-DC01.uk.domain.dom
Attempting to resolve name to IP address...
Name resolved to 10.11.4.10
querying...
UDP port 137 (dombios-ns service): LISTENING or FILTERED
Using ephemeral source port
Attempting domBIOS adapter status query to UDP port 137...
domBIOS name for 10.11.4.10 not found (timeout)
Adapter status query failed.
UDP port: FILTERED
portqry.exe -n A-DC01.uk.domain.dom -e 137 -p UDP exits with return code 0x00000001.
=============================================
 Starting portqry.exe -n A-DC01.uk.domain.dom -e 138 -p UDP ...
Querying target system called:
 A-DC01.uk.domain.dom
Attempting to resolve name to IP address...
Name resolved to 10.11.4.10
querying...
UDP port 138 (dombios-dgm service): LISTENING or FILTERED
portqry.exe -n A-DC01.uk.domain.dom -e 138 -p UDP exits with return code 0x00000002.
=============================================
 Starting portqry.exe -n A-DC01.uk.domain.dom -e 139 -p TCP ...
Querying target system called:
 A-DC01.uk.domain.dom
Attempting to resolve name to IP address...
Name resolved to 10.11.4.10
querying...
TCP port 139 (dombios-ssn service): FILTERED
portqry.exe -n A-DC01.uk.domain.dom -e 139 -p TCP exits with return code 0x00000002.
=============================================
 Starting portqry.exe -n A-DC01.uk.domain.dom -e 42 -p TCP ...
Querying target system called:
 A-DC01.uk.domain.dom
Attempting to resolve name to IP address...
Name resolved to 10.11.4.10
querying...
TCP port 42 (nameserver service): FILTERED
portqry.exe -n A-DC01.uk.domain.dom -e 42 -p TCP exits with return code 0x00000002.

------------------------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------

=============================================
 Starting portqry.exe -n A-DC01 -e 135 -p TCP ...
Querying target system called:
 A-DC01
Attempting to resolve name to IP address...
Failed to resolve name to IP address
portqry.exe -n A-DC01 -e 135 -p TCP exits with return code 0x00000063.
=============================================
 Starting portqry.exe -n A-DC01 -e 389 -p BOTH ...
Querying target system called:
 A-DC01
Attempting to resolve name to IP address...
Failed to resolve name to IP address
portqry.exe -n A-DC01 -e 389 -p BOTH exits with return code 0x00000063.
=============================================
 Starting portqry.exe -n A-DC01 -e 636 -p TCP ...
Querying target system called:
 A-DC01
Attempting to resolve name to IP address...
Failed to resolve name to IP address
portqry.exe -n A-DC01 -e 636 -p TCP exits with return code 0x00000063.
=============================================
 Starting portqry.exe -n A-DC01 -e 3268 -p TCP ...
Querying target system called:
 A-DC01
Attempting to resolve name to IP address...
Failed to resolve name to IP address
portqry.exe -n A-DC01 -e 3268 -p TCP exits with return code 0x00000063.
=============================================
 Starting portqry.exe -n A-DC01 -e 3269 -p TCP ...
Querying target system called:
 A-DC01
Attempting to resolve name to IP address...
Failed to resolve name to IP address
portqry.exe -n A-DC01 -e 3269 -p TCP exits with return code 0x00000063.
=============================================
 Starting portqry.exe -n A-DC01 -e 53 -p BOTH ...
Querying target system called:
 A-DC01
Attempting to resolve name to IP address...
Failed to resolve name to IP address
portqry.exe -n A-DC01 -e 53 -p BOTH exits with return code 0x00000063.
=============================================
 Starting portqry.exe -n A-DC01 -e 88 -p BOTH ...
Querying target system called:
 A-DC01
Attempting to resolve name to IP address...
Failed to resolve name to IP address
portqry.exe -n A-DC01 -e 88 -p BOTH exits with return code 0x00000063.
=============================================
 Starting portqry.exe -n A-DC01 -e 445 -p TCP ...
Querying target system called:
 A-DC01
Attempting to resolve name to IP address...
Failed to resolve name to IP address
portqry.exe -n A-DC01 -e 445 -p TCP exits with return code 0x00000063.
=============================================
 Starting portqry.exe -n A-DC01 -e 137 -p UDP ...
Querying target system called:
 A-DC01
Attempting to resolve name to IP address...
Failed to resolve name to IP address
portqry.exe -n A-DC01 -e 137 -p UDP exits with return code 0x00000063.
=============================================
 Starting portqry.exe -n A-DC01 -e 138 -p UDP ...
Querying target system called:
 A-DC01
Attempting to resolve name to IP address...
Failed to resolve name to IP address
portqry.exe -n A-DC01 -e 138 -p UDP exits with return code 0x00000063.
=============================================
 Starting portqry.exe -n A-DC01 -e 139 -p TCP ...
Querying target system called:
 A-DC01
Attempting to resolve name to IP address...
Failed to resolve name to IP address
portqry.exe -n A-DC01 -e 139 -p TCP exits with return code 0x00000063.
=============================================
 Starting portqry.exe -n A-DC01 -e 42 -p TCP ...
Querying target system called:
 A-DC01
Attempting to resolve name to IP address...
Failed to resolve name to IP address
portqry.exe -n A-DC01 -e 42 -p TCP exits with return code 0x00000063.

Hi,

By any chance do you have AV :Symantec :SEP v11.0.6005.562 installed on DC's?

From above PortQuey output everything is LISTENING. DCDIAG output will give more details.

Purvesh, thank you for all your help so far, I really, really appreciate it.

Unfortunately not on the AV front.  We use McAfee VirusScan Enterprise 8.8 on both the target and source server that I'm using for diagnoses.  I can turn it off on both servers the second the DCDIAG you asked for finishs, I didn't want to do it before hand for fear of changing the results.

Ok. Send the DCDIAG report here.

I advise you to turn off the security software on DCs post your change approval and share the results.

Hello,

please do NOT post complete output here, this make the thread unreadable.

Please UPLOAD the following files for starting:

ipconfig /all >c:\ipconfig.log [from each DC/DNS Server]
dcdiag /v /c /d /e /s:dcname >c:\dcdiag.log
repadmin /showrepl dc* /verbose /all /intersite >c:\repl.log  ["dc* is a place holder for the starting name of the DCs if they all begin the same (if more then one DC exists)]
dnslint /ad /s "DCipaddress" (http://support.microsoft.com/kb/321045)
ADREPLSTATUS http://www.microsoft.com/en-us/download/details.aspx?id=30005 can also be exported to file.

As the output will become large, DON'T post them into the thread, please use Windows OneDrive (https://onedrive.live.com) [with open access!] and add the link from it here. Also the /e in dcdiag scans the complete forest, so better run it on COB.

Hi idarryl,

From the current information it seems it is caused by blocked ports, please refer the following KB to confirm your firewall settings is correct.

The related KB:

How to configure a firewall for domains and trusts

http://support.microsoft.com/default.aspx?scid=kb;EN-US;179442

All the below error may also occur when your issue appered, please refer the following KB to fix them first.

Troubleshooting AD Replication error 8446: The replication operation failed to allocate memory

https://support.microsoft.com/en-us/kb/2693500

Event ID 1699 Replication Change List Creation

https://technet.microsoft.com/en-us/library/cc733224(v=ws.10).aspx

Event ID 1699 is logged many times and fills the Directory Service event log of a Windows Server 2008-based writable domain controller

https://support.microsoft.com/en-us/kb/953392

Im glad to be of help to you!

Hi,

Any updates on above?

Once again a big, big thank you to all that are helping me.  Sorry for no response on Friday.

Disabling AV didn't seem to help.  I ran the diags, with AV off on both servers, that Purvesh and Meinolf requested yesterday, start 10:00 local time, so all time stamps should be around them, the info is here: http://1drv.ms/1dejWud

I haven't had a opportunity to review the information myself yet, but will do now, and also go though Alex's suggestions.

To repacp, UKROOT03 dcdiag reported 1722 and 53 issues to 40+ DC's.  One of those DC's is UKDC13, so testing has been based around those two DC's only.  I can test against others if required.

FYI, I am aware of other issues in this forest, ISTG mis-configs, DC's down etc.  I will get to them, I just think this is the current most pressing issue.

• Edited by idarryl 20 hours 46 minutes ago additional info

Hello,

I have check the logs and found that you have many DNS & 1722 error and also ISTG mis-config issues.

I found few of DNS issues also. I would suggest you start with basic checks and fix one issue at a time.

1. Fix DNS configuration across sites.

2. check subnets and Site configuration Which includes IsTG & bridgehead servers.

3. Demote or remove DC's which are no longer required in estate using the metadata clen-up process

• Edited by Purvesh Adua 19 hours 48 minutes ago

Hello,

as there are lot's of "Access denied" errors assure that you use an elevated command prompt to run all the commands!!!

- "Doing initial required tests" some errors occur and must be checked.

Testing server: AU-Sydney\APACDC02

      Starting test: Connectivity

         * Active Directory LDAP Services Check
         Server APACDC02 resolved to these IP addresses: 10.196.1.21, but none

         of the addresses could be reached (pinged). Please check the network.

         Error: 0x2b02 "Error due to lack of resources."

         This error more often means that the targeted server is shutdown or

         disconnected from the network.

         Got error while checking LDAP and RPC connectivity. Please check your

         firewall settings.

         ......................... APACDC02 failed test Connectivity

Testing server: SG-Singapore\APACDC05

      Starting test: Connectivity

         * Active Directory LDAP Services Check
         Server APACDC05 resolved to these IP addresses: 10.220.1.5, but none

         of the addresses could be reached (pinged). Please check the network.

         Error: 0x2b02 "Error due to lack of resources."

         This error more often means that the targeted server is shutdown or

         disconnected from the network.

         Got error while checking LDAP and RPC connectivity. Please check your

         firewall settings.

         ......................... APACDC05 failed test Connectivity

Testing server: DK-CPH-IBC\DKDC04

      Starting test: Connectivity

         * Active Directory LDAP Services Check
         The host 9e48975d-f4e6-413c-a292-dfbccbf27d4f._msdcs.domain.net could

         not be resolved to an IP address. Check the DNS server, DHCP, server

         name, etc.

         Got error while checking LDAP and RPC connectivity. Please check your

         firewall settings.

         ......................... DKDC04 failed test Connectivity

Testing server: US-FL-SARASOTA\AWSUSRODC01

      Starting test: Connectivity

         * Active Directory LDAP Services Check
         Server AWSUSRODC01 resolved to these IP addresses: 10.190.32.4, but

         none of the addresses could be reached (pinged). Please check the

         network.

         Error: 0x2b02 "Error due to lack of resources."

         This error more often means that the targeted server is shutdown or

         disconnected from the network.

         Got error while checking LDAP and RPC connectivity. Please check your

         firewall settings.

         ......................... AWSUSRODC01 failed test Connectivity

Testing server: US-NY-LEXINGTON\USNYLA-DC02

      Starting test: Connectivity

         * Active Directory LDAP Services Check
         Got error while checking LDAP and RPC connectivity. Please check your

         firewall settings.

         ......................... USNYLA-DC02 failed test Connectivity

Testing server: US-NY-MADISON\USNYM-DC01

      Starting test: Connectivity

         * Active Directory LDAP Services Check
         Got error while checking LDAP and RPC connectivity. Please check your

         firewall settings.

         ......................... USNYM-DC01 failed test Connectivity

Testing server: US-AZ-Phoenix\USPCA-DC01

      Starting test: Connectivity

         * Active Directory LDAP Services Check
         Determining IP4 connectivity
         Failure Analysis: USPCA-DC01 ... OK.
         * Active Directory RPC Services Check
         The clock difference between the home server UKDC13 and target server

         USPCA-DC01 is greater than one minute. This may cause Kerberos

         authentication failures. Please check that the time service is working

         properly. You may need to resynchonize the time between these servers.

         ......................... USPCA-DC01 passed test Connectivity

------------------------------------------------------------------------------------------------------

The FSMO holders in the root domain and the child domains should be different in the 3 domain wide FSMOs BUT equal for the 2 root FSMOs which will be the same on all domains in the forest.

Means you should have always the following 2:

Starting test: KnowsOfRoleHolders

         Role Schema Owner = CN=NTDS Settings,CN=UKROOT03,CN=Servers,CN=UK-London,CN=Sites,CN=Configuration,DC=domain,DC=net
         Role Domain Owner = CN=NTDS Settings,CN=UKROOT03,CN=Servers,CN=UK-London,CN=Sites,CN=Configuration,DC=domain,DC=net

And then for each domain the following 3:

Role PDC Owner = CN=NTDS Settings,CN=xxxxx,CN=Servers,CN=xxxxx,CN=Sites,CN=Configuration,DC=domain,DC=net
         Role Rid Owner = CN=NTDS Settings,CN=xxxxx,CN=Servers,CN=xxxxx,CN=Sites,CN=Configuration,DC=domain,DC=net
         Role Infrastructure Update Owner = CN=NTDS Settings,CN=xxxxx,CN=Servers,CN=xxxxx,CN=Sites,CN=Configuration,DC=domain,DC=net

------------------------------------------------------------------------------------------------------

For the following please check with http://technet.microsoft.com/en-us/library/cc794759(WS.10).aspx:

[1] Problem: Missing Expected Value

             Base Object:

            CN=AUROOT01,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=domain,DC=net

             Base Object Description: "SYSVOL FRS Member Object"

             Value Object Attribute Name: serverReference

             Value Object Description: "DC Account Object"

             Recommended Action: Check if this server is deleted, and if so

            clean up this DCs SYSVOL FRS Member Object.

[1] Problem: Missing Expected Value

             Base Object:

            CN=UKDC11,OU=Domain Controllers,DC=UK,DC=domain,DC=net

             Base Object Description: "DC Account Object"

             Value Object Attribute Name: msDFSR-ComputerReferenceBL

             Value Object Description: "SYSVOL FRS Member Object"

             Recommended Action: See Knowledge Base Article: Q312862

            
            [2] Problem: Missing Expected Value

             Base Object:

            CN=UKDC13,OU=Domain Controllers,DC=UK,DC=domain,DC=net

             Base Object Description: "DC Account Object"

             Value Object Attribute Name: msDFSR-ComputerReferenceBL

             Value Object Description: "SYSVOL FRS Member Object"

             Recommended Action: See Knowledge Base Article: Q312862

            
            [3] Problem: Missing Expected Value

             Base Object:

            CN=UKDC12,OU=Domain Controllers,DC=UK,DC=domain,DC=net

             Base Object Description: "DC Account Object"

             Value Object Attribute Name: msDFSR-ComputerReferenceBL

             Value Object Description: "SYSVOL FRS Member Object"

             Recommended Action: See Knowledge Base Article: Q312862

            
            [4] Problem: Missing Expected Value

             Base Object:

            CN=USUKDC03,OU=Domain Controllers,DC=UK,DC=domain,DC=net

             Base Object Description: "DC Account Object"

             Value Object Attribute Name: msDFSR-ComputerReferenceBL

             Value Object Description: "SYSVOL FRS Member Object"

             Recommended Action: See Knowledge Base Article: Q312862

            
            [5] Problem: Missing Expected Value

             Base Object:

            CN=UKDC15,OU=Domain Controllers,DC=UK,DC=domain,DC=net

             Base Object Description: "DC Account Object"

             Value Object Attribute Name: msDFSR-ComputerReferenceBL

             Value Object Description: "SYSVOL FRS Member Object"

             Recommended Action: See Knowledge Base Article: Q312862

            
            [6] Problem: Missing Expected Value

             Base Object:

            CN=UKDC16,OU=Domain Controllers,DC=UK,DC=domain,DC=net

             Base Object Description: "DC Account Object"

             Value Object Attribute Name: msDFSR-ComputerReferenceBL

             Value Object Description: "SYSVOL FRS Member Object"

----------------------------------------------------------------------------------------------

If any of the listed DCs in the output files don't exist anymore please run metadata cleanup as described within http://blogs.msmvps.com/mweber/2010/05/16/active-directory-metadata-cleanup/ and assure all references to them is also removed from the DNS zones and DNS Server properties and AD sites and services.

-----------------------------------------------------------------------------------------------

Assure that firewalls are configured according to https://technet.microsoft.com/en-us/library/dd772723%28WS.10%29.aspx?f=255&MSPPError=-2147217396 so all DCs can replicate.

------------------------------------------------------------------------------------------------

Often there are FORWARDERS listed that are domain internal DNS servers. Assure that you don't create some kind of loop and they are not able to resolve correctly. If the Forwarders are used to connect the domain with the internet you should use external one instead.

------------------------------------------------------------------------------------------------

Replication errors shown with the following:

Doing intersite inbound replication test on site US-NY-WHITE-PLAINS:
            Locating & Contacting Intersite Topology Generator (ISTG) ...
               *** ERROR: The home server UKDC13 is not in sync with

               CN=NTDS Settings\0ADEL:9a4d72ec-2b85-4d39-ba1d-ae2d1c89da4f,CN=USWP-DC02\0ADEL:ecd9e147-8215-481d-ad5d-176cb755477c,CN=Servers,CN=US-NY-WHITE-PLAINS,CN=Sites,CN=Configuration,DC=domain,DC=net,

                unable to proceed. Suggest you run:

               dcdiag

               /s:CN=NTDS Settings\0ADEL:9a4d72ec-2b85-4d39-ba1d-ae2d1c89da4f,CN=USWP-DC02\0ADEL:ecd9e147-8215-481d-ad5d-176cb755477c,CN=Servers,CN=US-NY-WHITE-PLAINS,CN=Sites,CN=Configuration,DC=domain,DC=net

               <options>

Must be checked and the mentioned DCs must be removed from the domain if they no longer exist. The highlighted 0ADEL: belong normally to not longer existing objects in the domain.

Please assure that AD sites and services reflects your required environment and that all subnets belong to the correct site containing the correct DCs.

The 8446 (operation failed to allocate memory. This operation will not continue) status can occur when the Active Directory replication engine cannot allocate memory to perform Active directory replication.

These events can occur due to the following conditions:

• Low available physical memory 
• Low
  available paging file size versus physical memory (Wrong configuration
  of paging file) : Paging file should be 1.5 times the size of physical
  memory
• Paged Pool or Non-Paged pool exhaustion in the kernel
  
• LSASS
  Virtual memory depletion on 32 bit domain controllers. This is where
  the Virtual Memory of LSASS reaches the 2 GB limit of virtual memory
  available for a process running in user mode.


• The
  Virtual Memory depletion could be a leak inside the LSASS User mode
  Process, or the Database Cache (ESE Cache) may be consuming all the
  available memory.
  

The following information is important to understand:

Lsass.exe memory usage on domain controllers has two major components: one fixed and one variable.                                         

The
fixed component is made up of the code, the stacks, the heaps, and
various fixed size data structures (for example, the schema cache). The
amount of memory that LSASS uses may vary, depending on the load on the
computer. As the number of running threads increases, so does the number
of memory stacks. Lsass.exe usually uses 100 MB to 300 MB of memory.
Lsass.exe uses the same amount of memory no matter how much RAM is
installed in the computer.  

The
variable component is the database buffer cache. The size of the cache
can range from less than 1 MB to the size of the entire database.
Because a larger cache improves performance, the database
engine for AD (ESENT) attempts to keep the cache as large as possible.
While the size of the cache varies with memory pressure in the computer,
the maximum size of the cache is limited by both the amount of physical
RAM installed in the computer and by the amount of available virtual
address space (VA). AD uses only a portion of total VA space for the
cache.

https://support.microsoft.com/en-us/kb/2693500?wa=wsignin1.0

Thank you all for your recommendations, specifically Purvesh for your continued efforts and Meinolf for your extensive write up. As I've mentioned previously, I am aware of additional errors with my environment, and I am/will attend to those. The 1722 and 53 errors are still puzzling and so if anyone can offer me specific advice on either of those errors I would be extremely grateful. Thanks Darryl

Testing server: US-MA-SUNGUARD\USDC01

      Starting test: Advertising

         Fatal Error:DsGetDcName (USDC01) call failed, error 1722

         The Locator could not find the server.

         Printing RPC Extended Error Info:

         Error Record 1, ProcessID is 3020
          (DcDiag)
        
            System Time is: 5/17/2015 12:25:21:735

            Generating component is 2 (RPC runtime)
           
            Status is 1722 The RPC server is unavailable.

Testing server: SG-Sing-IGM\APACDC04

      Starting test: Advertising

         Fatal Error:DsGetDcName (APACDC04) call failed, error 1722

         The Locator could not find the server.

         Printing RPC Extended Error Info:

         Error Record 1, ProcessID is 3020
          (DcDiag)
        
            System Time is: 5/17/2015 12:50:19:685

            Generating component is 2 (RPC runtime)
           
            Status is 1722 The RPC server is unavailable.

Testing server: JP-Tok-IGM\APACDC06

      Starting test: Advertising

         Fatal Error:DsGetDcName (APACDC06) call failed, error 1722

         The Locator could not find the server.

         Printing RPC Extended Error Info:

         Error Record 1, ProcessID is 3020
          (DcDiag)
        
            System Time is: 5/17/2015 13:23:29:75

            Generating component is 2 (RPC runtime)
           
            Status is 1722 The RPC server is unavailable.

Testing server: SG-Singapore\APACDC08

      Starting test: Advertising

         Fatal Error:DsGetDcName (APACDC08) call failed, error 1722

         The Locator could not find the server.

         Printing RPC Extended Error Info:

         Error Record 1, ProcessID is 3020
          (DcDiag)

Source DC APACDC05 has possible security error (1722).  Diagnosing...

               Found KDC APACDC01 for domain APAC.domain.net in site SG-Singapore
               Checking time skew between servers:
                APACDC05
                APACDC08
                APACDC01
               Getting time for \\APACDC05.APAC.domain.net
               Error 2184 querying time on DC APACDC05.  Ignoring this DC and

               continuing...

From APACDC05 to APACDC08

            Naming Context: DC=ForestDnsZones,DC=domain,DC=net

            The replication generated an error (1722):

            The RPC server is unavailable.

            The failure occurred at 2015-05-17 14:49:04.

            The last success occurred at 2015-05-17 10:31:45.

            7 failures have occurred since the last success.

            The source remains down. Please check the machine.

Naming Context: CN=Configuration,DC=domain,DC=net

            The replication generated an error (1722):

            The RPC server is unavailable.

            The failure occurred at 2015-05-17 14:52:58.

            The last success occurred at 2015-05-17 10:15:46.

            7 failures have occurred since the last success.

            The source remains down. Please check the machine.

         DC=domain,DC=net has 99 cursors.

*****************************************************************************************

Hi,

Thanks for replying based on Error related to 1722 & RPC is basically you have to check the site-Link & Subnets from AD site. Also possible check the Router or firewall for ports are not blocked on sites. Make sure Time synch across site is perfect.