How do I troubleshoot AD LDS performance for userproxy objects? Functional, but LDAP Bind time is extremely high.

We have a Unix-hosted application (Business Objects) which is only able to do simple bind, is hosted internally. It needs to map users and group memberships for both Internal and DMZ, and authenticate users. DMZ trusts Internal, one way, AD LDS on 2008r2, server is in the DMZ (less trusted domain, obviously)

Firewall hole is in place (using 50636 and 50389) to the ADLDS Server from the internal B.O. servers which do ldap/s connections and then a simple bind using the internal LDS userproxy object for the application ID (the only way we could seem to make it work). Because (although it supposedly can chase referrals) the B.O. system was never able to  Directory includes Internal and external users and groups which do not overlap (e.g. external users and groups are all in unique OU's) and are created as UserProxy objects. B.O. allows us to specify specific DN's for searches, which I'm told has been done (i have no access to the Unix systems or the application configuration).

No 'expensive' searches have been revealed by LDAP  Interface Events level 4 or 6, w/ Field Engineering at 5 or any other levels.  DMZ is very small, with only the main Site and a DR site across town. AD LDS servers on subnet defined as main Site and NLTEST confirms they're directed to the two DC's in it.

Ldap bind time when we originally deployed was not noticably long, and ldp.exe could connect and do searches (Softerra also could). However when we turned it over to developers to begin testing, they found authentications were taking around 30 seconds to complete. About a month later *cough* we were informed there was a problem with logins taking too long.

Now when I go to the perfmon counter for the instance, and find every LDAP metric available is as low as we could possibly hope (e.g. searches log as complete in 0-12ms in the ADAM app log). But LDAP Bind time is off the charts, from 8000-14000ms from the moment we attempt a bind. Even with ldp.exe after connecting (SSL), not doing a query, just a simple bind takes about 10 seconds.

I feel like the only explanation is that the entire directory and every group is enumerated every time we do so. How can I tell? Any suggestions?

Much Obliged,

Trevor.



January 3rd, 2014 8:06pm

I've been in your situation before myself recently, but in another context, and found that the best insight was gained using packet tracing - using either NetMon 3.4 or WireShark.  I personally preferred NetMon, but WireShark seems to be the most commonly used, and your choice will depend on which O/S platform you are using - both are intuitive and are easy to set up.  Set this/these up on each server involved in the process (i.e. on your DC, on your Business Objects server, and your ADLDS server).

In one case I found that there was a broken DNS entry which would always time out after 30 seconds before a secondary search would be attempted which would always succeed.  The 30 seconds you mentioned above made me think of this ... something is timing out in a similar fashion.  Have you tried setting everything up using IP addresses instead?  Your 10 second LDP bind suggests that there may be more than one issue combining.

The other point I would make is that it is important to be able to understand exactly what LDAP queries are being constructed by your application - i.e. the LDAP "conversation" - don't assume anything.  It could be something as simple as the LDAP root not being specified optimally in all cases.  This article describes what you would expect to be happening in good detail - but it ultimately depends on what the application (Bus Objects in your case) is trying to do and how it is interpreting the results at each stage of the authentication process.

I'll keep thinking about this one in the meantime.  Interestingly I was troubleshooting an Optimal IdM VIS configuration designed to authenticate Cisco UCM identities at the time, which, I might add, is an excellent (and far simpler) alternative to the option of maintaining userProxyFull objects in an ADLDS instance which I too would have used in the

Free Windows Admin Tool Kit Click here and download it now
January 4th, 2014 8:44am

Thanks Bob, appreciated the quick response. I prefer Netmon from habit and because it meets my needs, personally. I'd narrowed to bind time specifically by just doing a BIND from softerra or ldp.exe with no queries.

The capture was much longer than I expected, and was confusing me with the LDS server showing attempts to connect to both the internal and eternal domain controllers while trying to authenticate an internal userproxy. I'd opened a case with MS around the same time as posting this, and walking through it they pointed out multiple attempts to do Kerberos auth directly from the LDS box back to the internal DC's (binding to internal userproxy object) and failing (port 88 isn't open).  Finally succeeding when failing back to NTLM and contacting the DMZ DC, which was able to query trustred internal domain controllers. So authentication always worked, but only on failback which delayed the process. 

What bugs me is that I must have overlooked a reference about DMZ LDS implementations needing to make connections through port 88 to internal domain controllers, or how to control authentication for an LDS instance...(anyone?). Needing the LDS server in the 'least trusted domain' is documented, as makes sense because my internal dc's dont trust the DMZ for lookups but the other way around. Anyone know of a reference page or manual that I missed for DMZ LDS?

In any case I'll post what we end up doing to resolve the situation. Thanks again!




January 8th, 2014 12:40am

Your LDS server is a domain member, using domain services for userProxy binds, and Kerberos / port 88 is included in the list of necessary ports in the standard "Active Directory + Firewall" documentation (example here).

IMHO Wireshark is 100x more useful than Netmon for its very powerful recording, filtering, and protocol dissection features.  I wouldn't even consider Netmon unless some policy prevented me from using Wireshark.

Free Windows Admin Tool Kit Click here and download it now
January 8th, 2014 4:53am

Have you noticed the same slowness when not going over SSL?  You might want to check the following:

--DNS (try using host file entry)

--Network Routing

--CRL on the SSL cert and ensure it is reachable and not causing the delay during authentication.

--Permissions on SSL cert

January 8th, 2014 11:28pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics