Connectivity issues through TMG 2010 inbound RDP, outbound https

Sorry this is long, but trying to get as much info into the initial post.

Basic configuration: Windows 2008 R2 server running TMG 2010 acting as passthrough (not NAT) router and firewall for an internally hosted application. External network of TMG is on our intranet; perimeter network hosts a W2K8R2 server running Terminal Services; internal network has DCs and certificate server for the environment. A few other servers for database, FTP and file services.

Users connect via secure RDP from outside the environment though TMG to Remote Desktop server. Once on that server they do various tasks, including download of data from secure FTP sites (over https) on our intranet.

Three main issues, which we believe are related and caused by TMG, but not able to find a root cause. These started approximately 1 month ago, but cannot see a connection with any changes in the environment or patches in May:

1. intermittent issues establishing RDP connection from clients on our intranet to the Remote Desktop server. Can fail when initially try to connect (generic "This computer can't connection with the remote computer" message). Sometimes they can get to enter their credentials, but then next stage when certificate for the secure connection is being checked they get same message and can't connect. In both instances within TMG log we see a Failed Connection with "The Object is shutting down" as the error message. Searching for help on this error message just comes back with lists of all the error messages on TMG and nothing useful to indicate what is actually shutting down. If they do make the connection it is stable - no reported dropouts or reconnections.

We have verified that making an RDP connection from the TMG to Remote Desktop server works each time, every time (and that a connection from outside the environment immediately after fails). Users working with the database and other internal FTP/file servers do not report any issues accessing these - all the issues seem to be with connection in to and out of the environment.

2. intermittent issues accessing externally hosted secure FTP sites from the Remote Desktop server to download data. These sites are accessible with no issues from outside the environment. Users have to authenticate with a smart card, select an option for the service they want and then either get to the FTP portal or get a "Service Unavailable" message (even though the service is definitely working). No Failure messages on the TMG. Again, once they do manage to get connected (which can take up to an hour after numerous attempts), connection is stable but can be slow.

3. intermittent issues accessing MS Exchange on our intranet from Outlook client running on the Remote Desktop server. Failed Connection message on the TMG for this https connection is "A socket operation was attempted to an unreachable network".

All the servers are VMs running on a single VMware ESXi 5.5 host. VMs have VMXNet3 NICs, so communicate at 10G between themselves. Physical NICs on host are 1G. There are many other VMs on the same host but no reported issues with any of these. The TMG and Remote Desktop VMs have sufficient CPU/memory, etc. with reservations set. The host has sufficient host and CPU.

We have run Wireshark from the client and Network Monitor on the TMG. In both we can see when making RDP connection you get Syn from client, Ack/Syn from RD, Ack from client, X224 request from client and then an Ack/Reset apparently from RD, but we are not seeing anything on the RD server indicating anything reaching it for the connection.

Can anyone make any suggestion on where to look next?

TIA



  • Edited by Techie Tim Tuesday, June 23, 2015 2:07 PM
June 23rd, 2015 10:12am

What do you mean by it is acting as a "passthrough" router? What kind of publishing rules are you using for RDP and Exchange? You will probably be best served breaking these out into separate individual issues because they may not all be related. Have you removed all 3rd party filters such as Antivirus to troubleshoot? What if you put a client just outside TMG (on external subnet), do you still have the problem? Was this all working find and then suddenly it started having intermittent issues?
Free Windows Admin Tool Kit Click here and download it now
July 1st, 2015 11:10am

What do you mean by it is acting as a "passthrough" router? What kind of publishing rules are you using for RDP and Exchange? You will probably be best served breaking these out into separate individual issues because they may not all be related. Have you removed all 3rd party filters such as Antivirus to troubleshoot? What if you put a client just outside TMG (on external subnet), do you still have the problem? Was this all working fine and then suddenly it started having intermittent issues?
July 1st, 2015 3:08pm

Passthrough -> routing is set up to pass client IP address

RDP -> uses standard TMG rule to allow RDP access from External to RDP server using RDP protocol/ports

Exchange -> my error on this one - users are accessing email on corporate network using webmail via browser on Terminal Server, not directly accessing it through Outlook client or similar. Rules are correct for the access to all websites associated with webmail.

Antivirus -> already eliminated this as cause.

Client just outside TMG -> already tested this with a VM running on the same host and same subnet as the TMG External gateway and trying to RDP in to the environment. This also gets the issue.

We've noticed a few things since original post:

we've found an issue with one of the CRLs on a root certificate being invalid. Since all the traffic on the wire is encrypted using certificates that link back to this certificate there's obvious latency being introduced whilst the client is waiting for the CA to respond and eventually timeout. We think that is happening all over the environment, not just with the initial RDP connection. We are working on a solution to this (most likely a temporary fix of using a link on the CA from the invalid URL to the correct CRL file).

Issue does get worse if there is a lot of activity on the Terminal Server. Not sure if this is related to the above, or just that the TS is struggling to accept new connections.

Users get "keyboard hammer" if they can't connect, so we see a large number of Failed connection attempts on TMG from the same IPs in quick succession (confirmed this is users and not malware - users have been notified and this has helped with a reduction in keyboard hammer). Doing a "netstat -ano" on the TMG when access is particularly bad and "keyboard hammer" was in effect we see a large number of UDP connections on the external gateway with destination as *:*.

We also see a large number of UDP connections to Internal gateway (which appear to correlate with connection attempts from things like Branch Cache Advertise and LDAP from servers on Internal network to external sites/DCs, which are blocked by default).

After a period of time once activity on the TS has gone down and the stray UDP connections on TMG are cleaned up, users are then able to more easily access the environment again.

The above is suggesting TMG struggling to cope with the number of UDP ports opened. I don't know if there is anything we could do to improve this, but would appreciate suggestions - not sure if it is standard TMG reaction to have these UDP ports from failed or denied connections, or if there is anything we could do to improve TMGs handling of them.

Next step for us is fixing the issue with the invalid CRL URL on the root cert (and Yes, I agree that it should not have been issued in that state - seems it has been that way for many years, but only now has anyone actually noticed!).

Free Windows Admin Tool Kit Click here and download it now
July 6th, 2015 6:03am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics