Windows 2008 (Not R2) Domain controller has stopped replicating to the other 2 Domain controllers

We had something (unknown) happen last week that stopped successful sysvol replication

I've been going through loads of articles looking for clues after running dcdiag on all 3 servers

This is from the Master Domain controller

https://support.microsoft.com/en-us/kb/840674/
Domain Controller Diagnosis

Performing initial setup:
   * Verifying that the local machine ch-dc1-2k8, is a DC.
   * Connecting to directory service on server ch-dc1-2k8.
   * Collecting site info.
   * Identifying all servers.
   * Identifying all NC cross-refs.
   * Found 3 DC(s). Testing 1 of them.
   Done gathering initial info.

Doing initial required tests
   
   Testing server: Cardiff\CH-DC1-2K8
      Starting test: Connectivity
         * Active Directory LDAP Services Check
         * Active Directory RPC Services Check
         ......................... CH-DC1-2K8 passed test Connectivity

Doing primary tests
   
   Testing server: Cardiff\CH-DC1-2K8
      Starting test: Replications
         * Replications Check
         * Replication Latency Check
            CN=Schema,CN=Configuration,DC=companyname,DC=local
               Latency information for 4 entries in the vector were ignored.
                  4 were retired Invocations.  0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc.  0 had no latency information (Win2K DC).  
            CN=Configuration,DC=companyname,DC=local
               Latency information for 4 entries in the vector were ignored.
                  4 were retired Invocations.  0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc.  0 had no latency information (Win2K DC).  
            DC=companyname,DC=local
               Latency information for 4 entries in the vector were ignored.
                  4 were retired Invocations.  0 were either: read-only replicas and are not verifiably latent, or dc's no longer replicating this nc.  0 had no latency information (Win2K DC).  
         * Replication Site Latency Check
         Site

         CN=NTDS Site Settings,CN=Edinburgh,CN=Sites,CN=Configuration,DC=companyname,DC=local

         was skipped because it never had an ISTG running in it.
         Site

         CN=NTDS Site Settings,CN=London,CN=Sites,CN=Configuration,DC=companyname,DC=local

         was skipped because it never had an ISTG running in it.
         Site

         CN=NTDS Site Settings,CN=Belfast,CN=Sites,CN=Configuration,DC=companyname,DC=local

         was skipped because it never had an ISTG running in it.
         ......................... CH-DC1-2K8 passed test Replications
      Test omitted by user request: Topology
      Test omitted by user request: CutoffServers
      Starting test: NCSecDesc
         * Security Permissions check for all NC's on DC CH-DC1-2K8.
         * Security Permissions Check for
           CN=Schema,CN=Configuration,DC=companyname,DC=local
            (Schema,Version 2)
         * Security Permissions Check for
           CN=Configuration,DC=companyname,DC=local
            (Configuration,Version 2)
         * Security Permissions Check for
           DC=companyname,DC=local
            (Domain,Version 2)
         ......................... CH-DC1-2K8 passed test NCSecDesc
      Starting test: NetLogons
         * Network Logons Privileges Check
         Unable to connect to the NETLOGON share! (\\CH-DC1-2K8\netlogon)
         [CH-DC1-2K8] An net use or LsaPolicy operation failed with error 67, Win32 Error 67.
         ......................... CH-DC1-2K8 failed test NetLogons
      Starting test: Advertising
         The DC CH-DC1-2K8 is advertising itself as a DC and having a DS.
         The DC CH-DC1-2K8 is advertising as an LDAP server
         The DC CH-DC1-2K8 is advertising as having a writeable directory
         The DC CH-DC1-2K8 is advertising as a Key Distribution Center
         Warning: CH-DC1-2K8 is not advertising as a time server.
         The DS CH-DC1-2K8 is advertising as a GC.
         ......................... CH-DC1-2K8 failed test Advertising
      Starting test: KnowsOfRoleHolders
         Role Schema Owner = CN=NTDS Settings,CN=CH-DC1-2K8,CN=Servers,CN=Cardiff,CN=Sites,CN=Configuration,DC=companyname,DC=local
         Role Domain Owner = CN=NTDS Settings,CN=CH-DC1-2K8,CN=Servers,CN=Cardiff,CN=Sites,CN=Configuration,DC=companyname,DC=local
         Role PDC Owner = CN=NTDS Settings,CN=CH-DC1-2K8,CN=Servers,CN=Cardiff,CN=Sites,CN=Configuration,DC=companyname,DC=local
         Role Rid Owner = CN=NTDS Settings,CN=CH-DC1-2K8,CN=Servers,CN=Cardiff,CN=Sites,CN=Configuration,DC=companyname,DC=local
         Role Infrastructure Update Owner = CN=NTDS Settings,CN=CH-DC1-2K8,CN=Servers,CN=Cardiff,CN=Sites,CN=Configuration,DC=companyname,DC=local
         ......................... CH-DC1-2K8 passed test KnowsOfRoleHolders
      Starting test: RidManager
         * Available RID Pool for the Domain is 12100 to 1073741823
         * ch-dc1-2k8.companyname.local is the RID Master
         * DsBind with RID Master was successful
         * rIDAllocationPool is 10600 to 11099
         * rIDPreviousAllocationPool is 10600 to 11099
         * rIDNextRID: 10613
         ......................... CH-DC1-2K8 passed test RidManager
      Starting test: MachineAccount
         Checking machine account for DC CH-DC1-2K8 on DC CH-DC1-2K8.
         * SPN found :LDAP/ch-dc1-2k8.companyname.local/companyname.local
         * SPN found :LDAP/ch-dc1-2k8.companyname.local
         * SPN found :LDAP/CH-DC1-2K8
         * SPN found :LDAP/ch-dc1-2k8.companyname.local/companyname
         * SPN found :LDAP/bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
         * SPN found :E3514235-4B06-11D1-AB04-00C04FC2DCD2/bfe39346-13d8-455a-a97a-2a33f9e779f5/companyname.local
         * SPN found :HOST/ch-dc1-2k8.companyname.local/companyname.local
         * SPN found :HOST/ch-dc1-2k8.companyname.local
         * SPN found :HOST/CH-DC1-2K8
         * SPN found :HOST/ch-dc1-2k8.companyname.local/companyname
         * SPN found :GC/ch-dc1-2k8.companyname.local/companyname.local
         ......................... CH-DC1-2K8 passed test MachineAccount
      Starting test: Services
         * Checking Service: Dnscache
         * Checking Service: NtFrs
         * Checking Service: IsmServ
         * Checking Service: kdc
         * Checking Service: SamSs
         * Checking Service: LanmanServer
         * Checking Service: LanmanWorkstation
         * Checking Service: RpcSs
         * Checking Service: w32time
         * Checking Service: NETLOGON
         ......................... CH-DC1-2K8 passed test Services
      Test omitted by user request: OutboundSecureChannels
      Starting test: ObjectsReplicated
         CH-DC1-2K8 is in domain DC=companyname,DC=local
         Checking for CN=CH-DC1-2K8,OU=Domain Controllers,DC=companyname,DC=local in domain DC=companyname,DC=local on 1 servers
            Object is up-to-date on all servers.
         Checking for CN=NTDS Settings,CN=CH-DC1-2K8,CN=Servers,CN=Cardiff,CN=Sites,CN=Configuration,DC=companyname,DC=local in domain CN=Configuration,DC=companyname,DC=local on 1 servers
            Object is up-to-date on all servers.
         ......................... CH-DC1-2K8 passed test ObjectsReplicated
      Starting test: frssysvol
         * The File Replication Service SYSVOL ready test
         File Replication Service's SYSVOL is ready
         ......................... CH-DC1-2K8 passed test frssysvol
      Starting test: frsevent
         * The File Replication Service Event log test
         There are warning or error events within the last 24 hours after the

         SYSVOL has been shared.  Failing SYSVOL replication problems may cause

         Group Policy problems.
         An Warning Event occured.  EventID: 0x800034C4
            Time Generated: 04/21/2015   21:42:20
            Event String: The File Replication Service is having trouble

enabling replication from NA-DC1-2K8 to

CH-DC1-2K8 for c:\windows\sysvol\domain using the

DNS name na-dc1-2k8.companyname.local. FRS

will keep retrying.

 Following are some of the reasons you would see

this warning.

 

 [1] FRS can not correctly resolve the DNS name

na-dc1-2k8.companyname.local from this

computer.

 [2] FRS is not running on

na-dc1-2k8.companyname.local.

 [3] The topology information in the Active

Directory Domain Services for this replica has

not yet replicated to all the Domain Controllers.



 

 This event log message will appear once per

connection, After the problem is fixed you will

see another event log message indicating that the

connection has been established.
         An Warning Event occured.  EventID: 0x800034C4
            Time Generated: 04/22/2015   01:54:49
            Event String: The File Replication Service is having trouble

enabling replication from CH-DC2-2K8 to

CH-DC1-2K8 for c:\windows\sysvol\domain using the

DNS name ch-dc2-2k8.companyname.local. FRS

will keep retrying.

 Following are some of the reasons you would see

this warning.

 

 [1] FRS can not correctly resolve the DNS name

ch-dc2-2k8.companyname.local from this

computer.

 [2] FRS is not running on

ch-dc2-2k8.companyname.local.

 [3] The topology information in the Active

Directory Domain Services for this replica has

not yet replicated to all the Domain Controllers.



 

 This event log message will appear once per

connection, After the problem is fixed you will

see another event log message indicating that the

connection has been established.
         ......................... CH-DC1-2K8 failed test frsevent
      Starting test: kccevent
         * The KCC Event log test
         Found no KCC errors in Directory Service Event log in the last 15 minutes.
         ......................... CH-DC1-2K8 passed test kccevent
      Starting test: systemlog
         * The System Event log test
         An Error Event occured.  EventID: 0x40000004
            Time Generated: 04/22/2015   07:16:20
            Event String: The Kerberos client received a

KRB_AP_ERR_MODIFIED error from the server

Administrator. The target name used was

companyname\CH-DC2-2K8$. This indicates that

the target server failed to decrypt the ticket

provided by the client. This can occur when the

target server principal name (SPN) is registered

on an account other than the account the target

service is using. Please ensure that the target

SPN is registered on, and only registered on, the

account used by the server. This error can also

happen when the target service is using a

different password for the target service account

than what the Kerberos Key Distribution Center

(KDC) has for the target service account. Please

ensure that the service on the server and the KDC

are both updated to use the current password. If

the server name is not fully qualified, and the

target domain (companyname.LOCAL) is different

from the client domain (companyname.LOCAL),

check if there are identically named server

accounts in these two domains, or use the

fully-qualified name to identify the server.
         An Error Event occured.  EventID: 0x40000004
            Time Generated: 04/22/2015   07:16:20
            Event String: The Kerberos client received a

KRB_AP_ERR_MODIFIED error from the server

administrator. The target name used was

companyname\NA-DC1-2K8$. This indicates that

the target server failed to decrypt the ticket

provided by the client. This can occur when the

target server principal name (SPN) is registered

on an account other than the account the target

service is using. Please ensure that the target

SPN is registered on, and only registered on, the

account used by the server. This error can also

happen when the target service is using a

different password for the target service account

than what the Kerberos Key Distribution Center

(KDC) has for the target service account. Please

ensure that the service on the server and the KDC

are both updated to use the current password. If

the server name is not fully qualified, and the

target domain (companyname.LOCAL) is different

from the client domain (companyname.LOCAL),

check if there are identically named server

accounts in these two domains, or use the

fully-qualified name to identify the server.
         ......................... CH-DC1-2K8 failed test systemlog
      Test omitted by user request: VerifyReplicas
      Starting test: VerifyReferences
         The system object reference (serverReference)

         CN=CH-DC1-2K8,OU=Domain Controllers,DC=companyname,DC=local and

         backlink on

         CN=CH-DC1-2K8,CN=Servers,CN=Cardiff,CN=Sites,CN=Configuration,DC=companyname,DC=local

         are correct.
         The system object reference (frsComputerReferenceBL)

         CN=CH-DC1-2K8,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=companyname,DC=local

         and backlink on

         CN=CH-DC1-2K8,OU=Domain Controllers,DC=companyname,DC=local are

         correct.
         The system object reference (serverReferenceBL)

         CN=CH-DC1-2K8,CN=Domain System Volume (SYSVOL share),CN=File Replication Service,CN=System,DC=companyname,DC=local

         and backlink on

         CN=NTDS Settings,CN=CH-DC1-2K8,CN=Servers,CN=Cardiff,CN=Sites,CN=Configuration,DC=companyname,DC=local

         are correct.
         ......................... CH-DC1-2K8 passed test VerifyReferences
      Test omitted by user request: VerifyEnterpriseReferences
      Test omitted by user request: CheckSecurityError
   
   Running partition tests on : Schema
      Starting test: CrossRefValidation
         ......................... Schema passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... Schema passed test CheckSDRefDom
   
   Running partition tests on : Configuration
      Starting test: CrossRefValidation
         ......................... Configuration passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... Configuration passed test CheckSDRefDom
   
   Running partition tests on : companyname
      Starting test: CrossRefValidation
         ......................... companyname passed test CrossRefValidation
      Starting test: CheckSDRefDom
         ......................... companyname passed test CheckSDRefDom
   
   Running enterprise tests on : companyname.local
      Starting test: Intersite
         Skipping site Cardiff, this site is outside the scope provided by the

         command line arguments provided.
         Skipping site Edinburgh, this site is outside the scope provided by

         the command line arguments provided.
         Skipping site London, this site is outside the scope provided by the

         command line arguments provided.
         Skipping site Belfast, this site is outside the scope provided by the

         command line arguments provided.
         ......................... companyname.local passed test Intersite
      Starting test: FsmoCheck
         GC Name: \\ch-dc1-2k8.companyname.local
         Locator Flags: 0xe00011bd
         PDC Name: \\ch-dc1-2k8.companyname.local
         Locator Flags: 0xe00011bd
         Warning: DcGetDcName(TIME_SERVER) call failed, error 1355
         A Time Server could not be located.
         The server holding the PDC role is down.
         Warning: DcGetDcName(GOOD_TIME_SERVER_PREFERRED) call failed, error 1355
         A Good Time Server could not be located.
         KDC Name: \\ch-dc1-2k8.companyname.local
         Locator Flags: 0xe00011bd
         ......................... companyname.local failed test FsmoCheck
      Test omitted by user request: DNS
      Test omitted by user request: DNS


  • Edited by aideyuk Wednesday, April 22, 2015 8:56 AM
April 22nd, 2015 8:55am

Hello,

Assuming you ran dcdiag with /e, I see complaining on the PDC emulator being off or not reachable and also time sync issues.

What is your ipconfig /all configuration on each DC?

Regards,

Calin

Free Windows Admin Tool Kit Click here and download it now
April 22nd, 2015 9:06am

Cheers Calin ....

Master is

Windows IP Configuration

   Host Name . . . . . . . . . . . . : ch-dc1-2k8
   Primary Dns Suffix  . . . . . . . : companyname.local
   Node Type . . . . . . . . . . . . : Hybrid
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No
   DNS Suffix Search List. . . . . . : companyname.local
                                       internal.ch

Ethernet adapter Production Lan:

   Connection-specific DNS Suffix  . : internal.ch
   Description . . . . . . . . . . . : Broadcom BCM5708C NetXtreme II GigE (NDIS VBD Client)
   Physical Address. . . . . . . . . : 00-1E-C9-B8-2E-18
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::79b0:af7a:5ccc:f613%12(Preferred)
   IPv4 Address. . . . . . . . . . . : 172.16.200.43(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Default Gateway . . . . . . . . . : 172.16.11.1
   DHCPv6 IAID . . . . . . . . . . . : 301997769
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-13-20-69-01-00-15-17-91-27-70
   DNS Servers . . . . . . . . . . . : 172.16.7.7
                                       172.16.7.17
   NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter Backup Lan:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Intel(R) PRO/1000 PT Dual Port Server Adapter
   Physical Address. . . . . . . . . : 00-15-17-91-27-70
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   IPv4 Address. . . . . . . . . . . : 10.10.10.164(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.252.0
   Default Gateway . . . . . . . . . :
   NetBIOS over Tcpip. . . . . . . . : Enabled

Tunnel adapter Local Area Connection* 8:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface
   Physical Address. . . . . . . . . : 02-00-54-55-4E-01
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 9:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : internal.ch
   Description . . . . . . . . . . . : isatap.internal.ch
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 11:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Microsoft ISATAP Adapter
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

DC2 is

DNS Suffix Search List. . . . . . : companyname.local
                                       internal.ch

Ethernet adapter Production Lan:

   Connection-specific DNS Suffix  . : internal.ch
   Description . . . . . . . . . . . : Intel(R) PRO/1000 PT Dual Port Server Adapter #2
   Physical Address. . . . . . . . . : 00-15-17-91-27-81
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::2546:8631:ee98:ee91%11(Preferred)
   IPv4 Address. . . . . . . . . . . : 172.16.200.46(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Default Gateway . . . . . . . . . : 172.16.11.1
   DHCPv6 IAID . . . . . . . . . . . : 268440855
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-13-20-18-C9-00-15-17-91-27-80
   DNS Servers . . . . . . . . . . . : 172.16.7.7
                                       172.16.7.17
   NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter Backup Lan:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Intel(R) PRO/1000 PT Dual Port Server Adapter
   Physical Address. . . . . . . . . : 00-15-17-91-27-80
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   IPv4 Address. . . . . . . . . . . : 10.10.10.165(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.252.0
   Default Gateway . . . . . . . . . :
   NetBIOS over Tcpip. . . . . . . . : Enabled

Tunnel adapter Local Area Connection* 8:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : internal.ch
   Description . . . . . . . . . . . : isatap.internal.ch
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 9:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : isatap.{F3C3588F-92A3-4610-92EF-9037AFA7F0A2}
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 11:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface
   Physical Address. . . . . . . . . : 02-00-54-55-4E-01
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

DC3 is

Windows IP Configuration

   Host Name . . . . . . . . . . . . : na-dc1-2k8
   Primary Dns Suffix  . . . . . . . : companyname.local
   Node Type . . . . . . . . . . . . : Hybrid
   IP Routing Enabled. . . . . . . . : No
   WINS Proxy Enabled. . . . . . . . : No
   DNS Suffix Search List. . . . . . : companyname.local
                                       internal.ch

Ethernet adapter Production Lan:

   Connection-specific DNS Suffix  . : internal.ch
   Description . . . . . . . . . . . : Broadcom BCM5708C NetXtreme II GigE (NDIS VBD Client)
   Physical Address. . . . . . . . . : 00-1E-C9-B9-01-32
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   Link-local IPv6 Address . . . . . : fe80::4ceb:7364:8d29:596f%12(Preferred)
   IPv4 Address. . . . . . . . . . . : 172.16.101.49(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.0.0
   Default Gateway . . . . . . . . . : 172.16.11.101
   DHCPv6 IAID . . . . . . . . . . . : 301997769
   DHCPv6 Client DUID. . . . . . . . : 00-01-00-01-13-22-80-47-00-15-17-91-D5-0C
   DNS Servers . . . . . . . . . . . : 172.16.101.7
                                       172.16.7.7
   NetBIOS over Tcpip. . . . . . . . : Enabled

Ethernet adapter Backup Lan:

   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Broadcom BCM5708C NetXtreme II GigE (NDIS VBD Client) #2
   Physical Address. . . . . . . . . : 00-1E-C9-B9-01-34
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes
   IPv4 Address. . . . . . . . . . . : 10.10.12.27(Preferred)
   Subnet Mask . . . . . . . . . . . : 255.255.248.0
   Default Gateway . . . . . . . . . :
   NetBIOS over Tcpip. . . . . . . . : Enabled

Tunnel adapter Local Area Connection* 8:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : isatap.{9C0D1E02-D02A-4CF3-A745-CC2156A09B1E}
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 11:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . : internal.ch
   Description . . . . . . . . . . . : isatap.internal.ch
   Physical Address. . . . . . . . . : 00-00-00-00-00-00-00-E0
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

Tunnel adapter Local Area Connection* 13:

   Media State . . . . . . . . . . . : Media disconnected
   Connection-specific DNS Suffix  . :
   Description . . . . . . . . . . . : Teredo Tunneling Pseudo-Interface
   Physical Address. . . . . . . . . : 02-00-54-55-4E-01
   DHCP Enabled. . . . . . . . . . . : No
   Autoconfiguration Enabled . . . . : Yes

April 22nd, 2015 9:22am

Hi

 Your log mentions that you have some old server which has not been removed compltely from active directory(seem 4)

resolve the issue;

 Use ADSIEdt to delete the FRS member objects.To do this,follow these steps

- Click Start,click Run,type adsiedit.msc in the Open box.and then click OK

- Expand the Domain Nc container.

- Expand DC=<var>Your DOmain</var>,DC=com,pri,local,net

- Expand CN=System

- Expand CN=File Replication Service

- Expand CN=DOmain System VOlume (SYSVOL share)

- Right click the domain controller you wer removed,and click delete

Finaly run repadmin /syncall /force

run "net share" on each dc

check replication again..


  • Edited by Burak Uur Wednesday, April 22, 2015 9:51 AM
Free Windows Admin Tool Kit Click here and download it now
April 22nd, 2015 9:50am

Cheers Burak

No server has been removed

All 3 are active and have been for around 3 years ?

I've had a look at adsiedit and all looks ok, the 3 DC's are the only entries in there ....

I've run a repadmin .syscall /force anyway but it doesn't seem to of done anything

cheers

April 22nd, 2015 10:10am

Hi

Please run each DC;

 "netdom resetpwd /s:server /ud:domain\user /pd:*

Detailed article

https://support.microsoft.com/en-us/kb/325850

"4 was retired invocaitons.0 were either:read-only replicas and not verifiably latent or,dc's no longer replicating this nc" means unavaible DC from FRS.please make sure about that.

Free Windows Admin Tool Kit Click here and download it now
April 22nd, 2015 10:36am

Thanks Burak 

I've run on all 3 ... I had to wait for permission ..

This is the Dcdiag /e  from the PDC
I can see a netlogon issue now ?

Domain Controller Diagnosis

 

Performing initial setup:

   Done gathering initial info.

 

Doing initial required tests

  

   Testing server: Cardiff\CH-DC1-2K8

      Starting test: Connectivity

         ......................... CH-DC1-2K8 passed test Connectivity

  

   Testing server: Cardiff\CH-DC2-2K8

      Starting test: Connectivity

         ......................... CH-DC2-2K8 passed test Connectivity

  

   Testing server: Cardiff\NA-DC1-2K8

      Starting test: Connectivity

         ......................... NA-DC1-2K8 passed test Connectivity

 

Doing primary tests

  

   Testing server: Cardiff\CH-DC1-2K8

      Starting test: Replications

         ......................... CH-DC1-2K8 passed test Replications

      Starting test: NCSecDesc

         ......................... CH-DC1-2K8 passed test NCSecDesc

      Starting test: NetLogons

         Unable to connect to the NETLOGON share! (\\CH-DC1-2K8\netlogon)

         [CH-DC1-2K8] An net use or LsaPolicy operation failed with error 67, Win32 Error 67.

         ......................... CH-DC1-2K8 failed test NetLogons

      Starting test: Advertising

         ......................... CH-DC1-2K8 passed test Advertising

      Starting test: KnowsOfRoleHolders

         ......................... CH-DC1-2K8 passed test KnowsOfRoleHolders

      Starting test: RidManager

         ......................... CH-DC1-2K8 passed test RidManager

      Starting test: MachineAccount

         ......................... CH-DC1-2K8 passed test MachineAccount

      Starting test: Services

         ......................... CH-DC1-2K8 passed test Services

      Starting test: ObjectsReplicated

         ......................... CH-DC1-2K8 passed test ObjectsReplicated

      Starting test: frssysvol

         ......................... CH-DC1-2K8 passed test frssysvol

      Starting test: frsevent

         There are warning or error events within the last 24 hours after the

         SYSVOL has been shared.  Failing SYSVOL replication problems may cause

         Group Policy problems.

         ......................... CH-DC1-2K8 failed test frsevent

      Starting test: kccevent

         ......................... CH-DC1-2K8 passed test kccevent

      Starting test: systemlog

         An Error Event occured.  EventID: 0x0000164A

            Time Generated: 04/22/2015   12:58:24

            Event String: The Netlogon service could not create server

         An Error Event occured.  EventID: 0x40000004

            Time Generated: 04/22/2015   12:58:47

            Event String: The Kerberos client received a

         An Error Event occured.  EventID: 0x40000004

            Time Generated: 04/22/2015   12:58:47

            Event String: The Kerberos client received a

         An Error Event occured.  EventID: 0xC0000040

            Time Generated: 04/22/2015   12:59:15

            Event String: The attempt to install printer Fax into an

         An Error Event occured.  EventID: 0xC0001B6E

            Time Generated: 04/22/2015   13:01:48

            (Event String could not be retrieved)

         An Error Event occured.  EventID: 0xC0001B6E

            Time Generated: 04/22/2015   13:01:50

            (Event String could not be retrieved)

         An Error Event occured.  EventID: 0x0000165B

            Time Generated: 04/22/2015   13:12:37

            Event String: The session setup from computer 'CH11864V1'

         An Error Event occured.  EventID: 0x000016AD

            Time Generated: 04/22/2015   13:14:37

            Event String: The session setup from the computer CH11864V1

         ......................... CH-DC1-2K8 failed test systemlog

      Starting test: VerifyReferences

         ......................... CH-DC1-2K8 passed test VerifyReferences

  

   Testing server: Cardiff\CH-DC2-2K8

      Starting test: Replications

         ......................... CH-DC2-2K8 passed test Replications

      Starting test: NCSecDesc

         ......................... CH-DC2-2K8 passed test NCSecDesc

      Starting test: NetLogons

         Unable to connect to the NETLOGON share! (\\CH-DC2-2K8\netlogon)

         [CH-DC2-2K8] An net use or LsaPolicy operation failed with error 67, Win32 Error 67.

         ......................... CH-DC2-2K8 failed test NetLogons

      Starting test: Advertising

         ......................... CH-DC2-2K8 passed test Advertising

      Starting test: KnowsOfRoleHolders

         ......................... CH-DC2-2K8 passed test KnowsOfRoleHolders

      Starting test: RidManager

         ......................... CH-DC2-2K8 passed test RidManager

      Starting test: MachineAccount

         ......................... CH-DC2-2K8 passed test MachineAccount

      Starting test: Services

         ......................... CH-DC2-2K8 passed test Services

      Starting test: ObjectsReplicated

         ......................... CH-DC2-2K8 passed test ObjectsReplicated

      Starting test: frssysvol

         ......................... CH-DC2-2K8 passed test frssysvol

      Starting test: frsevent

         There are warning or error events within the last 24 hours after the

         SYSVOL has been shared.  Failing SYSVOL replication problems may cause

         Group Policy problems.

         ......................... CH-DC2-2K8 failed test frsevent

      Starting test: kccevent

         ......................... CH-DC2-2K8 passed test kccevent

      Starting test: systemlog

         An Error Event occured.  EventID: 0x00000422

            Time Generated: 04/22/2015   12:56:12

            Event String: The processing of Group Policy failed. Windows

         An Error Event occured.  EventID: 0x40000004

            Time Generated: 04/22/2015   13:00:44

            Event String: The Kerberos client received a

         An Error Event occured.  EventID: 0x00000422

            Time Generated: 04/22/2015   13:01:13

            Event String: The processing of Group Policy failed. Windows

         An Error Event occured.  EventID: 0x00000422

            Time Generated: 04/22/2015   13:06:15

            Event String: The processing of Group Policy failed. Windows

         An Error Event occured.  EventID: 0x00000422

            Time Generated: 04/22/2015   13:11:17

            Event String: The processing of Group Policy failed. Windows

         An Error Event occured.  EventID: 0x0000164A

            Time Generated: 04/22/2015   13:15:44

            Event String: The Netlogon service could not create server

         An Error Event occured.  EventID: 0x00000422

            Time Generated: 04/22/2015   13:16:04

            Event String: The processing of Group Policy failed. Windows

         An Error Event occured.  EventID: 0x40000004

            Time Generated: 04/22/2015   13:16:18

            Event String: The Kerberos client received a

         An Error Event occured.  EventID: 0x40000004

            Time Generated: 04/22/2015   13:16:18

            Event String: The Kerberos client received a

         An Error Event occured.  EventID: 0xC0001B6E

            Time Generated: 04/22/2015   13:19:05

            (Event String could not be retrieved)

         An Error Event occured.  EventID: 0x00000422

            Time Generated: 04/22/2015   13:21:05

            Event String: The processing of Group Policy failed. Windows

         An Error Event occured.  EventID: 0x00000422

            Time Generated: 04/22/2015   13:22:29

            Event String: The processing of Group Policy failed. Windows

         An Error Event occured.  EventID: 0x0000164A

            Time Generated: 04/22/2015   13:26:28

            Event String: The Netlogon service could not create server

         An Error Event occured.  EventID: 0x40000004

            Time Generated: 04/22/2015   13:27:03

            Event String: The Kerberos client received a

         An Error Event occured.  EventID: 0x40000004

            Time Generated: 04/22/2015   13:27:04

            Event String: The Kerberos client received a

         An Error Event occured.  EventID: 0xC0001B6E

            Time Generated: 04/22/2015   13:29:50

            (Event String could not be retrieved)

         ......................... CH-DC2-2K8 failed test systemlog

      Starting test: VerifyReferences

         ......................... CH-DC2-2K8 passed test VerifyReferences

  

   Testing server: Cardiff\NA-DC1-2K8

      Starting test: Replications

         ......................... NA-DC1-2K8 passed test Replications

      Starting test: NCSecDesc

         ......................... NA-DC1-2K8 passed test NCSecDesc

      Starting test: NetLogons

         Unable to connect to the NETLOGON share! (\\NA-DC1-2K8\netlogon)

         [NA-DC1-2K8] An net use or LsaPolicy operation failed with error 67, Win32 Error 67.

         ......................... NA-DC1-2K8 failed test NetLogons

      Starting test: Advertising

         Warning: DsGetDcName returned information for \\ch-dc1-2k8.Companyname.local, when we were trying to reach NA-DC1-2K8.

         Server is not responding or is not considered suitable.

         ......................... NA-DC1-2K8 failed test Advertising

      Starting test: KnowsOfRoleHolders

         ......................... NA-DC1-2K8 passed test KnowsOfRoleHolders

      Starting test: RidManager

         ......................... NA-DC1-2K8 passed test RidManager

      Starting test: MachineAccount

         ......................... NA-DC1-2K8 passed test MachineAccount

      Starting test: Services

         ......................... NA-DC1-2K8 passed test Services

      Starting test: ObjectsReplicated

         ......................... NA-DC1-2K8 passed test ObjectsReplicated

      Starting test: frssysvol

         ......................... NA-DC1-2K8 passed test frssysvol

      Starting test: frsevent

         There are warning or error events within the last 24 hours after the

         SYSVOL has been shared.  Failing SYSVOL replication problems may cause

         Group Policy problems.

         ......................... NA-DC1-2K8 failed test frsevent

      Starting test: kccevent

         ......................... NA-DC1-2K8 passed test kccevent

      Starting test: systemlog

         An Error Event occured.  EventID: 0x40000004

            Time Generated: 04/22/2015   13:02:55

            Event String: The Kerberos client received a

         An Error Event occured.  EventID: 0x40000004

            Time Generated: 04/22/2015   13:05:34

            Event String: The Kerberos client received a

         An Error Event occured.  EventID: 0x40000004

            Time Generated: 04/22/2015   13:05:54

            Event String: The Kerberos client received a

         ......................... NA-DC1-2K8 failed test systemlog

      Starting test: VerifyReferences

         ......................... NA-DC1-2K8 passed test VerifyReferences

  

   Running partition tests on : Schema

      Starting test: CrossRefValidation

         ......................... Schema passed test CrossRefValidation

      Starting test: CheckSDRefDom

         ......................... Schema passed test CheckSDRefDom

  

   Running partition tests on : Configuration

      Starting test: CrossRefValidation

         ......................... Configuration passed test CrossRefValidation

      Starting test: CheckSDRefDom

         ......................... Configuration passed test CheckSDRefDom

  

   Running partition tests on : Companyname

      Starting test: CrossRefValidation

         ......................... Companyname passed test CrossRefValidation

      Starting test: CheckSDRefDom

         ......................... Companyname passed test CheckSDRefDom

  

   Running enterprise tests on : Companyname.local

      Starting test: Intersite

         Doing intersite inbound replication test on site Cardiff:

         ......................... Companyname.local passed test Intersite

      Starting test: FsmoCheck

         ......................... Companyname.local passed test FsmoCheck

April 22nd, 2015 12:53pm

Looks like you are missing the nelogon share

Check this https://support.microsoft.com/en-us/kb/947022

Free Windows Admin Tool Kit Click here and download it now
April 22nd, 2015 1:02pm

Hi

 After run net share restart the both dc's,and check reps.if still get logon error;

check this;

https://support.microsoft.com/en-us/kb/947022/en-us

April 22nd, 2015 1:04pm

Checked all 3 and they have the below

C:\>net share

Share name   Resource                        Remark

------------------------------------------------------------------------------
C$           C:\                             Default share
D$           D:\                             Default share
IPC$                                         Remote IPC
ADMIN$       C:\Windows                      Remote Admin
domain       C:\Windows\SYSVOL\domain
SYSVOL       C:\Windows\SYSVOL\sysvol        Logon server share
The command completed successfully.


Will look at your next post ... cheers

Free Windows Admin Tool Kit Click here and download it now
April 22nd, 2015 1:23pm

my post would be very similar to my previous one:

you are missing the netlogon share

make sure you have this folder

%SystemRoot%\sysvol\sysvol\{DOMAIN}\scripts 

then:

http://support.microsoft.com/default.aspx?scid=kb;EN-US;947022

  1. Stop File Replication Services
    1. Open services.msc
    2. Locate File Replication Services
    3. Stop the service

 

  1. Change the SysvolReady Flag
    1. Click Start, click Run, type regedit, and then click OK.
    2. Locate the following subkey in Registry Editor:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Netlogon\Parameters

  1. In the details pane, right-click SysvolReady Flag, and then click Modify.
  2. In the Value data box, type 0 and then click OK.
  3. Again in the details pane, right-click SysvolReady Flag, and then click Modify.
  4. In the Value data box, type 1, and then click OK.

NOTE: This will cause Netlogon to share out SYSVOL, and the scripts folder 

 

  1. Start File Replication Services
    1. Open services.msc
    2. Locate File Replication Services
    3. Stop the service

 

  1. Check that NETLOGON / SYSVOL comes back
    1. Open up a command prompt
    2. Run NET SHARE and confirm that the folders are there

 

NOTE: If the folders do not come back, check that the folder structure within the SYSVOL folder is correct.

April 22nd, 2015 1:35pm

From ch-dc1-2k8 (PDC)

Starting test: NetLogons
Unable to connect to the NETLOGON share! (\\CH-DC1-2K8\netlogon)
[CH-DC1-2K8] An net use or LsaPolicy operation failed with error 67, Win32 Error 67.
......................... CH-DC1-2K8 failed test NetLogons

From ch-dc2-2k8 I get this error

Starting test: NetLogons        
Unable to connect to the NETLOGON share! (\\CH-DC1-2K8\netlogon)         
[CH-DC1-2K8] An net use or LsaPolicy operation failed with error 67,  Win32 Error 67.          ......................... CH-DC1-2K8 failed test NetLogons      

from na-dc1-2k8 I get this error

Starting test: NetLogons         
Unable to connect to the NETLOGON share! (\\CH-DC1-2K8\netlogon)         
[CH-DC1-2K8] An net use or LsaPolicy operation failed with error 67,          Win32 Error 67.          ......................... CH-DC1-2K8 failed test NetLogons      

Free Windows Admin Tool Kit Click here and download it now
April 22nd, 2015 1:40pm

hey! looks like you are missing netlogon share ;)
April 22nd, 2015 2:01pm

YAH thanks Guys ....

It looks happier with no more netlogin errors :)

Run the checks (dcdiag /e) a few times on them now 

It looks like I''m down to just this error but its similar on all 3 servers ?

PDC (ch-dc1-2k8)
Starting test: frsevent
There are warning or error events within the last 24 hours after the SYSVOL has been shared.
Failing SYSVOL replication problems may cause Group Policy problems.

DC2 (ch-dc2-2k8)
Starting test: FrsEvent        
The event log File Replication Service on server          ch-dc1-2k8.companyname.local could not be queried, error 0x5          "Win32 Error 5"          ......................... CH-DC1-2K8 failed

DC3 (na-dc2-2k8)
Starting test: FrsEvent         
The event log File Replication Service on server ch-dc1-2k8.companyname.local could not be queried,
error 0x5          "Win32 Error 5"          ......................... CH-DC1-2K8 failed

Also when I go to look at administrative events all 3 give the same error "Access is denied"



  • Edited by aideyuk Wednesday, April 22, 2015 2:43 PM
Free Windows Admin Tool Kit Click here and download it now
April 22nd, 2015 2:42pm

1. verify you are running dcdiag from an elevated command prompt

2. run repadmin /syncall /APed and see if you get error

3. create a file inside the scripts folder and verify it gets replicated across all DCs

April 22nd, 2015 3:27pm

Hi Aperelli

1) Yes at an elevated prompt
2) Command works fine and no errors

C:\>repadmin /syncall /APED
Syncing all NC's held on localhost.
Syncing partition: CN=Schema,CN=Configuration,DC=companyname,DC=local
CALLBACK MESSAGE: The following replication is in progress:
    From: bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
    To  : abb03237-e91b-457f-ab16-788d5dc3930e._msdcs.companyname.local
CALLBACK MESSAGE: The following replication is in progress:
    From: bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
    To  : 2961b38b-570f-4a35-908f-9818a8080c0d._msdcs.companyname.local
CALLBACK MESSAGE: The following replication completed successfully:
    From: bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
    To  : abb03237-e91b-457f-ab16-788d5dc3930e._msdcs.companyname.local
CALLBACK MESSAGE: The following replication completed successfully:
    From: bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
    To  : 2961b38b-570f-4a35-908f-9818a8080c0d._msdcs.companyname.local
CALLBACK MESSAGE: SyncAll Finished.
SyncAll terminated with no errors.

Syncing partition: CN=Configuration,DC=companyname,DC=local
CALLBACK MESSAGE: The following replication is in progress:
    From: bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
    To  : abb03237-e91b-457f-ab16-788d5dc3930e._msdcs.companyname.local
CALLBACK MESSAGE: The following replication is in progress:
    From: bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
    To  : 2961b38b-570f-4a35-908f-9818a8080c0d._msdcs.companyname.local
CALLBACK MESSAGE: The following replication completed successfully:
    From: bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
    To  : abb03237-e91b-457f-ab16-788d5dc3930e._msdcs.companyname.local
CALLBACK MESSAGE: The following replication completed successfully:
    From: bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
    To  : 2961b38b-570f-4a35-908f-9818a8080c0d._msdcs.companyname.local
CALLBACK MESSAGE: SyncAll Finished.
SyncAll terminated with no errors.

Syncing partition: DC=companyname,DC=local
CALLBACK MESSAGE: The following replication is in progress:
    From: bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
    To  : abb03237-e91b-457f-ab16-788d5dc3930e._msdcs.companyname.local
CALLBACK MESSAGE: The following replication is in progress:
    From: bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
    To  : 2961b38b-570f-4a35-908f-9818a8080c0d._msdcs.companyname.local
CALLBACK MESSAGE: The following replication completed successfully:
    From: bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
    To  : abb03237-e91b-457f-ab16-788d5dc3930e._msdcs.companyname.local
CALLBACK MESSAGE: The following replication completed successfully:
    From: bfe39346-13d8-455a-a97a-2a33f9e779f5._msdcs.companyname.local
    To  : 2961b38b-570f-4a35-908f-9818a8080c0d._msdcs.companyname.local
CALLBACK MESSAGE: SyncAll Finished.
SyncAll terminated with no errors.


3) I put a file inside the scripts folder but it hasn't replicated across to the other 2 Domain Controllers :(

Free Windows Admin Tool Kit Click here and download it now
April 22nd, 2015 3:41pm

> C:\>repadmin /syncall /APED   Whatever repadmin does - it does NOT help if file replication is not working. Check FRS or DFSR event logs.  From what I saw in your dcdiag outputs, it "might" be the krbtgt password is out of sync, but this would affect AD replication as well. So let's ignore repadmin, let's ignore dcdiag, and focus on above event logs.  
April 22nd, 2015 4:02pm

" Whatever repadmin does - it does NOT help if file replication is not

working. Check FRS or DFSR event logs."

that's why I told him to create a file in the scripts folder....

Free Windows Admin Tool Kit Click here and download it now
April 22nd, 2015 4:04pm

Hi ..

DFS Replication log shows no errors/warnings

FRS replication gives me access denied when I try to look at the log ?
(I'm currently logged in as the Domain Administrator)

And the file I put into scripts has not copied across ...

April 22nd, 2015 4:18pm

Hi,

>>FRS replication gives me access denied when I try to look at the log ?
 (I'm currently logged in as the Domain Administrator)

Can we try another domain admin account to check this? Did we follow the article provided by aperelli to troubleshoot the issue?  Before going further, the following article can also be referred to as reference for troubleshooting.

Troubleshooting missing SYSVOL and NETLOGON shares on Windows domain controllers

https://support.microsoft.com/en-us/kb/257338

Best regards,
Frank Shen

Free Windows Admin Tool Kit Click here and download it now
April 24th, 2015 2:32am

Thanks All for the advice .....

FRS is now working on all 3 Domain controllers :)

After fixing the share issue and doing the burflags value 2

The errors reported by DCdiag have gone

April 24th, 2015 2:45am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics