Hi,

I am having problems with one site in our domain. Users of the site are complaining about slow logons. I've been troubleshooting the issue but I haven't found a solution yet.

One strange thing I found out... When pinging from workstations (at the site which is having this issue) our domain.net it resolves to an IP-address of different site's DC.

In AD Sites and Services we have following configuration:

SITE1
Subnets:
192.168.199.0/24 (this is subnet of the site which is having these issues)
192.168.200.0/24 (this is subnet where servers are located)

SITE2
Subnets:
192.168.105.0/24

SITE3:
Subnets:
192.168.107.0/24

For example sometimes when I ping domain.net from 192.168.199.0/24, DC from subnet 192.168.105.0/24 or 192.168.107.0/24 reply. Why is that so? Why doesn't DC from subnet 192.168.200.0/24 fromsame site (SITE1) reply always to requests coming from subnet 10.1.199.0/24?

We have also other sites that has DC (totally 27). SITE2 and SITE3 are just examples. DCs from other sites as well may also reply requests coming from .199 subnet.

I think this does slowness to logon because group policies and startup scripts are handled from other DCs... Is this correct?

Any ideas how to continue troubleshooting? Please let me know if you need any troubleshooting data...

Thanks for your help.

Best regards,

Toni Rantanen

Hi

How many DC are situvated in each site?

Is that all the site have DNS servers or which site is having DNS server?

Is it round robin and netmosk ordering is enabled in DNS properties?

Is the switch is confiared properly to route the requests?

Regards

Toni,
the slowness is typically due to DNS misconfiguration. You can verify this bychecking the authenticating domain controller on the client (LOGONSERVER environment variable). If client subnets have not been defined in AD,you wouldalso see events 5807 on the authenticating DC.
Verify that the SITE1 subnets are defined in AD and are assigned to SITE1 AD object. In addition, make sure that relevant SRV records of SITE1 DCs are registered on DNS server that clients in that site are using for name resolution (in particular, pay attention to _sites\SITE1\_tcp node in the DNS forward lookupzone representing your domain...

hth
Marcin

Hi Rajesh,

Each site have 1-2 DCs depending on the site's size.

All DCs are also DNS servers

Round robin and netmask ordering are both enabled.

Routing is ok.

Problem is that domain.net is randomly resolved to DCs in subnets that are not assigned for this site.For methis looks like AD Sites and Services or DNS configuration issue.

Rergards,

Toni,
have you checked the items I listed in my previous post (namely, the name of the authenticating domain controller on client computers and presence of site-specific SRV records on DNS servers they point to)?

hth
Marcin

Hi Marcin,

When entering command "echo %LOGONSERVER%" to Command Prompt, this will show DC from correct .200 subnet. So this is correct. But when I ping from same workstation to domain.net this will resolve to a DC in different subnet. For example subnet in SITE2 or SITE3 which both are over low bandwidth connections.

I can't find 5807 events from DCs.

Client subnet is defined in AD. When I take Properties from SITE1 in AD Sites and Services, I can see both subnets .199 and .200. Is this enough to "Verify that the SITE1 subnets are defined in AD and are assigned to SITE1 AD object."?

SRV records (_gc, _ldap and _kerberos) for relevant DCs can be found from_sites\SITE1\_tcp node in the DNS forward lookupzone.

Regards,

Hi Toni,

If %logonserver% shows a DC in the local site, the DC location should work properly. With DNS round robin enabled, it is possible that a DNS server returns an IP address of a DC in remote site. This does not necessarily indicate a problem in case the %logonserver% is correct.

For the slow logon issue, I suggest that you refer to the following KB to implement the PreferLogonDC registry value and check the result:

831201 An update for Windows Server 2003 and Windows 2000 Server makes it possible to put the logon server at the top of the DFS referrals list

http://support.microsoft.com/default.aspx?scid=kb;EN-US;831201

If the issue persists, please enable the userenv logging (http://support.microsoft.com/kb/221833) on a Windows XP or Windows Server 2003 client workstation, reproduce the issue and upload the userenv.log to the following space for further research.

https://sftasia.one.microsoft.com/choosetransfer.aspx?key=107470a7-94df-4796-bf49-2a51106b235a
Password: YYw8gd#-rH@%-

Thanks.

Joson Zhou

TechNet Subscriber Support in forum

If you have any feedback on our support, please contact tngfb@microsoft.com

Hi Joson,

Thanks for your answer.

I will install the hotfix first and I the problem still exists I will increase logging and post the log file as you instructed.

Before installing the hotfix I decided to test what DFS referrals list looks now. Please have a look at data below:

*********************

C:\>ipconfig

Windows IP Configuration

Ethernet adapter Local Area Connection:

Connection-specific DNS Suffix . : domain.net
IP Address. . . . . . . . . . . . : 192.168.199.201
Subnet Mask . . . . . . . . . . . : 255.255.255.0
Default Gateway . . . . . . . . . : 192.168.199.254

C:\>ping domain.net

Pinging domain.net [192.168.118.45] with 32 bytes of data:

Reply from 192.168.118.45: bytes=32 time=21ms TTL=125
Reply from 192.168.118.45: bytes=32 time=22ms TTL=125
Reply from 192.168.118.45: bytes=32 time=21ms TTL=125
Reply from 192.168.118.45: bytes=32 time=21ms TTL=125

Ping statistics for 192.168.118.45:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 21ms, Maximum = 22ms, Average = 21ms

C:\>ping -a 192.168.118.45

Pinging SITE4DC01.domain.net [192.168.118.45] with 32 bytes of data:

Reply from 192.168.118.45: bytes=32 time=21ms TTL=125
Reply from 192.168.118.45: bytes=32 time=22ms TTL=125
Reply from 192.168.118.45: bytes=32 time=21ms TTL=125
Reply from 192.168.118.45: bytes=32 time=22ms TTL=125

Ping statistics for 192.168.118.45:
Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 21ms, Maximum = 22ms, Average = 21ms

C:\>echo %logonserver%
\\SITE1DC01

C:\>dfsutil.exe /pktinfo

Microsoft(R) Windows(TM) Dfs Utility Version 4.2
Copyright (C) Microsoft Corporation 1991-2005. All Rights Reserved.

--mup.sys--
2 entries...
Entry: \domain.net\SysVol
ShortEntry: \domain.net\SysVol
Expires in 420 seconds
UseCount: 0 Type:0x1 ( DFS )
0:[\SITE1DC03.domain.net\SysVol] State:0x121 ( TARGETSET )
1:[\SITE1DC01.domain.net\SysVol] State:0x31 ( ACTIVE )
2:[\SITE5DC01.domain.net\SysVol] State:0x121 ( TARGETSET )
3:[\SITE6DC01.domain.net\SysVol] State:0x21 ( )
.
. // [Toni]: I have cut the list.
.
29:[\pltrzDC01.domain.net\SysVol] State:0x21 ( )

Entry: \domain.net\NETLOGON
ShortEntry: \domain.net\NETLOGON
Expires in 0 seconds
UseCount: 0 Type:0x1 ( DFS )
0:[\SITE1DC03.domain.net\NETLOGON] State:0x131 ( ACTIVE TARGETSET )
1:[\SITE1DC01.domain.net\NETLOGON] State:0x21 ( )
2:[\SITE7DC01.domain.net\NETLOGON] State:0x121 ( TARGETSET )
3:[\SITE8DC01.domain.net\NETLOGON] State:0x21 ( )
.
. // [Toni]: I have cut the list.
.
29:[\SITE4DC01.domain.net\NETLOGON] State:0x21 ( )

DfsUtil command completed successfully.

C:\>set logonserver
LOGONSERVER=\\SITE1DC01

C:\>

*********************

If I understood correctly DFS referrals list is correct already. DCs from SITE1 are in top of the list. Also DC from SITE1 is used as logon server. Still ping is referring to a DC in some other server. Based on my testing above, do you still see necessary to install the hotfix described in KB 831201?

Could it be possible that in our case logon is slow becausea useris authenticated by one domain controller in SITE1, andpolicies are downloaded from another domain controller in another site?

Regards,

Toni,
have you established what exactly is being processed during user logons? Do you happen to have references to specific servers (in sites other than SITE1) in your logon script or do you use GPO-based software installation with source files located in a remote site?

hth
Marcin

Hi Toni,

Your understanding is correct. According to the output of pktinfo, this workstation gets the policies from SITE1DC01. Therefore, I think the DC locator works properly. In this case, we do not need to perform the steps in KB831201. Please check the settings Marcin mentioned, and collect the userenv logging and event log to troubleshoot this performance issue.

In addition, please help confirm:

1. Does the slow logon issue happen on all users who log on to a computer in the site?

2. Does the same issue exist if the problematic user logon to a computer in another site?

3. Does this issue always exist or just happen intermittently?

Thanks.

Joson Zhou

TechNet Subscriber Support in forum

If you have any feedback on our support, please contact tngfb@microsoft.com

Hi Toni,

Hows everything going?

Just want to check if you have time to collect the information. If there is anything unclear, please feel free to let me know.

Thanks.

Joson Zhou

TechNet Subscriber Support in forum

If you have any feedback on our support, please contact tngfb@microsoft.com

Toni,
have you established what exactly is being processed during user logons? Do you happen to have references to specific servers (in sites other than SITE1) in your logon script or do you use GPO-based software installation with source files located in a remote site?

hth
Marcin

Hi Marcin,

First of all, sorry for my late reply.

I've been going through all GPOs and logon scripts and I haven't found any references to servers in other sites. Some logon scripts refers to files in share \\domain.net. This might be one thing causing these issues if domain.net does not refer to nearest DC...?

Is possible somehow find out from workstation end what it is exactly doing in startup? Is there any log that would contain all steps processed during logon?

Best regards,

Toni

Hi Joson,

First of all, I'm sorry for my late reply. Thanks, everything is just fine. I was week and a half on a holiday, so that's why my reply has been taking so long...

Userenv logging has been enabled. After slow logon happens next time... I will upload userenv.log it as you instructed in your message at Monday, September 07, 2009 8:01 AM.

Please find out answers for your earlier questions:

1. The slow logon issue happens randomly on most of the users. I would estimate that about 5% of all logons on SITE1 are "slow logons".
2. One of the user who has been having this problem every now and then, tested to log on remotely on a computer on other site and this problem did not happened. But...:
3. Unfortunately this problem happens only intermittently. That makes it even more difficult to troubleshoot.

I will upload you the userenv.log file after next time slow logon issue happens.

Thanks for your help! I'll get back to you after next slow logon.

Best regards,

Toni

Hi,

Thanks for your update.

Please let me know once the issue occurs again. Im glad to be of assistance.

Joson Zhou

TechNet Subscriber Support in forum

If you have any feedback on our support, please contact tngfb@microsoft.com

Hi Joson,

I have now uploaded userenv.log file for you as you instructed on above. A user who provided me the log file complained that "Stuck for a long time on Configuring Security Policy to the System and actual logon after giving credentials." The user also complained that when browsing from Windows Explorer domain.net it took a long time.

Thanks for helping me with this issue. Please let me know if you need any other information from me.

Best regards,

Hi Toni,

 

Thanks for your update.

 

I was leave from 9/28 to 10/5 due to our national holiday, and the useenv.log file was deleted from the space at 10/4 unfortunately. Could you please upload the useenv.log to the space again? In addition, as the system was stuck at “Configuring Security Policy”, please also collect MPSReport on the computer:

 

1. Download the executable file from the following URL
http://download.microsoft.com/download/b/b/1/bb139fcb-4aac-4fe5-a579-30b0bd915706/MPSRPT_DirSvc.EXE

2. Run the file on the computers.
3. After the tool finishes gathering the information, copy the cab file from the following folder:

C:\WINDOWS\MPSReports\DirSvc\cab

 

Thanks.

 

Joson Zhou

TechNet Subscriber Support in forum

Hi Joson,

I have now uploaded the userenv.log file again.

I will collect MPSReport also for you as soon as possible. I'll let you know when the report has been uploaded.

Thanks.

Best regards,

Hi Joson,

I have now uploaded %computername%_MPSReports.cab file for you.

Please let me know when you have investigated the log files provided. Thanks for your help.

Best regards,

Hi,

 

Thanks for your information and patience.

 

I have checked the files. According to the userenv.log, I found that the it took around 5 minutes to configuring security policy to the system.

 

USERENV(7b4.3d4) 10:28:47:831 MachinePolicyCallback: Setting status UI to Configuring security policy to the system.

USERENV(7b4.3d4) 10:33:14:316 MachinePolicyCallback: Setting status UI to Applying computer settings...

 

Then, I checked the gpresult.txt and winlogon.log in MPSReport and noticed that you configured File System policy but the setting could not apply to the computer:

 

        File System Settings

        --------------------

            GPO: **** PERMISSIONS

                ObjectName: C:\Program Files\***\****.NET

 

 

 

          Configure c:\program files\***\****.net.

Warning 3: The system cannot find the path specified.

          Error setting security on c:\program files\\***\****.net.

 

That could be the cause of slow logon. In fact, it is not recommended to modify file system ACLs via GPOs due to the performance impact. I suggest that you temporarily disable the policy and check the result.

 

Security configuration guidance support

http://support.microsoft.com/default.aspx?scid=kb;EN-US;885409

 

If there is anything unclear, please feel free to let me know.

 

Joson Zhou

TechNet Subscriber Support in forum

Hi Joson,

Thanks for this information.

I would like to explain the purpose of this particular GPO... My customer uses some 3rd party software that doesn't work without users having administrative rights on their workstations unless we modify security rights of that particular file. We need to give Modify rights to local Users group on that file and then the software works. If we disable this GPO then we have quite a lot of manual work to modify always that user rights of this particular file. Would like to recommend any better way (than File System GPO) to do this same action?

This user does not have the software installed (the file which permissions the GPO tried to modify does not exist) so that's why there were an error with him. I have now disabled this GPO for this particular computer by setting Delegation right with Apply Group Policy set to Deny for this user's computer.

So that will explain this particular slow logon issue. Thanks for solving this out!

Then we still have the original issue where ping domain.net does not refer to neareast DC. Could you help me to solve that issue as well? I have seen this issue only in one subnet (192.168.199.0/24 which belongs to SITE1). Users on this subnet are complaining that some 3rd party software are giving error messages like "no domain controller found" and browsing computers of the domain via My Network Places or with 3rd party software Dameware takes a long time. I suspect that this is because domain.net refers to a domain controller over slow network connection.

Best regards,

Hi Toni,

 

You may deploy a scheduled task on the computers to set the ACLs for the file.

 

As I stated earlier, we cannot identify whether the client is connecting to a DC in remote site based on the ICMP result. To analyze the issue, I suggest that we perform the following steps to collect the information when the issue occurs again:

 

1.    Please run the command Dfsutil /pktflush on the client computer.

2.    Please reproduce the issue and capture network traffic by using the network monitor utility.

Microsoft Network Monitor 3.3
http://www.microsoft.com/downloads/details.aspx?displaylang=en&FamilyID=983b941d-06cb-4658-b7f6-3088333d062f

How to use Network Monitor to capture network traffic
http://support.microsoft.com/kb/812953

3.    Run dfsutil /pktinfo.

4.    Please upload the Netmon file and the output of the pktinfo to the space.

Thanks.

 

Joson Zhou

TechNet Subscriber Support in forum

Hi Joson,

Sorry again for my late reply...

I have just tried to upload files but I receive an error "No workspace was found corresponding to the supplied password. The password or web page address (URL) may be incorrect. Please verify the password with a Microsoft Support Professional.". Perhaps the password I'm using has expired?

Best regards,

Hi Toni,

The space has been expired. Please upload the information to the this new one:

https://sftasia.one.microsoft.com/choosetransfer.aspx?key=f4bd0118-48d8-4f8e-9fea-be418a9d62bb

Password: kjMNk#NUfPEtF

Besides the files, please also let me know the IP address of the computers.

Thanks.

Joson Zhou

TechNet Subscriber Support in forum

If you have any feedback on our support, please contact tngfb@microsoft.com

Hi Joson,

I have uploaded Netmon.cap and Output.txt files for you. Netmon.cap contains capture result of Network Monitor and Output.txt contains ping results, ipconfig/all and dfsutil results.

As you can see from Output.txt "ping domain.net" does not answer from nearest DC. First it responds from other site DC (192.168.46.45) having pretty good network connection and then about half an hour later it responds from DC (192.168.30.45) in other site with not so good network bandwidth.

Please note that I have changed our IP subnets when posting here to forum (X.X.Y.0/24 -> 192.168.Y.0/24).

Please let me know if you need any other data for troubleshooting.

Thanks.

Best regards,

Hi Toni,

Im sorry if the description of the data collection steps in my previous reply was not clear, but I cannot identify the cause of the issue based on the files.

From the netmon file, we can see that when you ping the domain name domain.net, the client sent a DNS query to the DNS server for the IP address of domain.net because ICMP is not SRV-aware (Packet 78)

78 0.000000 {DNS:11, UDP:10, IPv4:9} 192.168.199.203 192.168.200.52 DNS DNS:QueryId = 0x3DBC, QUERY (Standard query), Query for domain.net of type Host Addr on class Internet

Then, it got the DNS response: (Packet 79)

79 0.000000 {DNS:11, UDP:10, IPv4:9} 192.168.200.52 192.168.199.203 DNS DNS:QueryId = 0x3DBC, QUERY (Standard query), Response - Success, 192.168.30.45, 192.168.167.45 ...

Please note that the 192.168.30.45 was at the top of the list. Therefore, the client sent ICMP request to the DC 192.168.30.45.

However, it does not mean that SRV-aware application (such as Group Policy, dclocator, DFS, nltest /dsgetdc, etc.) on the client will access the DC 192.168.30.45 to query/get the domain information. Thats why I say we cannot identify whether the client is connecting to a DC in remote site based on the ICMP result.

Next time, if you encounter the issue -- Users on this subnet are complaining that some 3rd party software are giving error messages like "no domain controller found" and browsing computers of the domain via My Network Places or with 3rd party software Dameware takes a long time., please perform the following steps to collect the network traffic in order that we can narrow down the cause of the issue:

1. Please start network monitor and access the sysvol share folder by using the UNC path \\domain.net\Sysvol.

2. After the sysvol folder appears, stop network monitor.

3. Please run dfsutil /pktinfo on the client machine.

4. Please upload the Netmon file and the output of the pktinfo to the space.

Edit: Please also run the following command on the client machine when the issue occurs:

nltest /dsgetdc:

If there is anything unclear, please feel free to let me know.

Joson Zhou

• Edited by Joson ZhouModerator Tuesday, November 03, 2009 1:02 AM Edit

I am having the same problem with pinging our domain name.  It resolves to a DC at another site.  If I do an IPCONFIG /FLUSHDNS, it likely resolves to another domain controller.  It seems to be happening at all sites.  The subnets are setup correctly in Active Directory Sites and Services.  Did this part of this problem ever get resolved?  Thanks.

Hi,

Back then when investigating this problem I understood that this is more like a property than an issue.

Best regards,

Toni

I am guessg its e to DNS Round Robin.

I have the same issue. Did anyone figure it out?

I do have the same situation, I confirmed the dns is working properly. It returned 4 IP of my DC in round-robin. DNS is good.

When ping received the respond from nslookup, it will have to decide one ip to use, we do not know the mechanism behind, but we shall know that nslookup and ping is not site-aware. Also, like Joson stated:"we cannot identify whether the client is connecting to a DC in remote site based on the ICMP

Also please refer to the link below:

https://social.technet.microsoft.com/Forums/windowsserver/en-US/c3a628ba-69be-4ef8-b988-3530b4692430/ping-domainnet-does-not-resolve-to-nearest-dc

And when will you use domain.local which is not FQDN where all the AD-aware applications leverage on the SRV records in DNS?

Kent