TCP frequent resets, etc.., mainly on https transactions
I'm noticing something on our Windows Server 2008 machine that is causing me a lot of concern.About half, or what seems like half, of the connections are getting the "bad" closure [RST, ACK] instead of [FIN, ACK]. This doesn't seem to bother various e-mail clients such as outlook 2003 or 2007 or Entourage. But it does seem to tick off the mail client on iphones, etc. It also seems to have no affect on OWA either.Is there any way to fix this error, if it is one, etc?Thanks in advance for any help or information posted.
February 1st, 2009 3:46am

hi there A tcp reset is always done by the application port which might be HTTP in your case. Would like to know if this is happening to your custom applications or any https site ? also i would like to know since when the issue has started . TCPip.sys file is responsible for making an tcp connection followed by the wininet.dll which will use the required API to communicate with IE , so i would like to check if the issue is happening only with IE . If you have firefox , you could check the behavior. Make sure you update the system with latest security patches and you can disable AV on the machine and check if the issue is occuring. I might require ethereal trace for analysissainath Windows Driver Development
Free Windows Admin Tool Kit Click here and download it now
February 2nd, 2009 12:01pm

It seems to be happening only on HTTPS when trying to use exchange's site. This issue has been going on for a while, but only really became an issue when the number of resets became far more than the successful transactions around the middle of December.The issue is predominately a problem when accessing exchange via our iPhones. Mostly happens when sending a new message, replying to an existing message, or trying to move messages around. Activesync push works fine.I am also now noticing resets on https connections from Entourage and some Outlook clients, but have not seen or been told of any ill effects.Firefox and IE seem to not be bothered by the resets.The system is fully patched, Exchange 2007 has all rollups applied, and have tested with the AV disabled. I get the same results.
February 2nd, 2009 5:32pm

hi there,as you said that your Iphone communicates with the exchange, i wanted to know which application is sending the reset ? is it exchange is reseting the connection or is it iphone application is resetting ?sainath Windows Driver Development
Free Windows Admin Tool Kit Click here and download it now
February 3rd, 2009 5:53pm

Most of the time the resets are coming from the exchange/IIS 7 side. I have seen on occasion where the iphone application resets it.
February 3rd, 2009 8:41pm

i would like to see the netmon / ethereal trace to understand the tcp behavior. is it possible for you to capture a trace while issue occurs and paste the frames ?sainath Windows Driver Development
Free Windows Admin Tool Kit Click here and download it now
February 4th, 2009 3:27pm

Here are two samples. Please let me know if you need more.No. Time Source Destination Protocol Info 27 5.379777 32.131.237.146 10.1.1.20 TLSv1 Change Cipher SpecFrame 27 (103 bytes on wire, 103 bytes captured)Ethernet II, Src: Cisco_e2:8c:e2 (00:12:01:e2:8c:e2), Dst: Ibm_a2:e9:a1 (00:1a:64:a2:e9:a1)Internet Protocol, Src: 32.131.237.146 (32.131.237.146), Dst: 10.1.1.20 (10.1.1.20)Transmission Control Protocol, Src Port: 51437 (51437), Dst Port: https (443), Seq: 1191, Ack: 2053, Len: 37Secure Socket LayerNo. Time Source Destination Protocol Info 28 5.379828 10.1.1.20 32.131.237.146 TCP https > 51437 [ACK] Seq=2053 Ack=1228 Win=63575 Len=0 TSV=5468356 TSER=442130768Frame 28 (66 bytes on wire, 66 bytes captured)Ethernet II, Src: Ibm_a2:e9:a1 (00:1a:64:a2:e9:a1), Dst: Cisco_e2:8c:e2 (00:12:01:e2:8c:e2)Internet Protocol, Src: 10.1.1.20 (10.1.1.20), Dst: 32.131.237.146 (32.131.237.146)Transmission Control Protocol, Src Port: https (443), Dst Port: 51437 (51437), Seq: 2053, Ack: 1228, Len: 0No. Time Source Destination Protocol Info 29 5.379895 32.131.237.146 10.1.1.20 TLSv1 Encrypted Handshake MessageFrame 29 (119 bytes on wire, 119 bytes captured)Ethernet II, Src: Cisco_e2:8c:e2 (00:12:01:e2:8c:e2), Dst: Ibm_a2:e9:a1 (00:1a:64:a2:e9:a1)Internet Protocol, Src: 32.131.237.146 (32.131.237.146), Dst: 10.1.1.20 (10.1.1.20)Transmission Control Protocol, Src Port: 51437 (51437), Dst Port: https (443), Seq: 1228, Ack: 2053, Len: 53Secure Socket LayerNo. Time Source Destination Protocol Info 30 5.383622 10.1.1.20 32.131.237.146 TCP https > 51437 [RST, ACK] Seq=2053 Ack=1281 Win=0 Len=0Frame 30 (54 bytes on wire, 54 bytes captured)Ethernet II, Src: Ibm_a2:e9:a1 (00:1a:64:a2:e9:a1), Dst: Cisco_e2:8c:e2 (00:12:01:e2:8c:e2)Internet Protocol, Src: 10.1.1.20 (10.1.1.20), Dst: 32.131.237.146 (32.131.237.146)Transmission Control Protocol, Src Port: https (443), Dst Port: 51437 (51437), Seq: 2053, Ack: 1281, Len: 0=========No. Time Source Destination Protocol Info 3046 650.656082 10.1.1.20 32.131.6.64 TCP https > 51521 [ACK] Seq=2053 Ack=1281 Win=63522 Len=0 TSV=5532884 TSER=442137212Frame 3046 (66 bytes on wire, 66 bytes captured)Ethernet II, Src: Ibm_a2:e9:a1 (00:1a:64:a2:e9:a1), Dst: Cisco_e2:8c:e2 (00:12:01:e2:8c:e2)Internet Protocol, Src: 10.1.1.20 (10.1.1.20), Dst: 32.131.6.64 (32.131.6.64)Transmission Control Protocol, Src Port: https (443), Dst Port: 51521 (51521), Seq: 2053, Ack: 1281, Len: 0No. Time Source Destination Protocol Info 3047 650.656192 10.1.1.20 32.131.6.64 TCP https > 51521 [RST, ACK] Seq=2053 Ack=1281 Win=0 Len=0Frame 3047 (54 bytes on wire, 54 bytes captured)Ethernet II, Src: Ibm_a2:e9:a1 (00:1a:64:a2:e9:a1), Dst: Cisco_e2:8c:e2 (00:12:01:e2:8c:e2)Internet Protocol, Src: 10.1.1.20 (10.1.1.20), Dst: 32.131.6.64 (32.131.6.64)Transmission Control Protocol, Src Port: https (443), Dst Port: 51521 (51521), Seq: 2053, Ack: 1281, Len: 0
February 4th, 2009 6:34pm

More samples... 255 112.158362 10.1.1.20 32.131.175.115 TCP https > 52665 [ACK] Seq=2053 Ack=1159 Win=63644 Len=0 TSV=8864964 TSER=442756182256 112.158373 32.131.175.115 10.1.1.20 TLSv1 Change Cipher Spec257 112.160273 32.131.175.115 10.1.1.20 TLSv1 Encrypted Handshake Message258 112.160305 10.1.1.20 32.131.175.115 TCP https > 52665 [ACK] Seq=2053 Ack=1249 Win=63554 Len=0 TSV=8864964 TSER=442756182259 112.160476 32.131.175.115 10.1.1.20 TLSv1 Encrypted Handshake Message260 112.160812 10.1.1.20 32.131.175.115 TCP [TCP segment of a reassembled PDU]261 112.160818 10.1.1.20 32.131.175.115 TLSv1 Encrypted Handshake Message262 112.162135 10.1.1.20 32.131.175.115 TCP https > 52665 [RST, ACK] Seq=2053 Ack=1249 Win=0 Len=0263 112.256174 32.131.175.115 10.1.1.20 TLSv1 Encrypted Handshake Message264 112.256443 10.1.1.20 32.131.175.115 TCP [TCP segment of a reassembled PDU]265 112.256448 10.1.1.20 32.131.175.115 TLSv1 Encrypted Handshake Message266 112.258797 32.131.175.115 10.1.1.20 TCP 52664 > https [ACK] Seq=829 Ack=2053 Win=65535 Len=0 TSV=442756186 TSER=8864829267 112.258922 32.131.175.115 10.1.1.20 TLSv1 Encrypted Handshake Message268 112.455079 10.1.1.20 32.131.175.115 TCP https > 52664 [ACK] Seq=2053 Ack=866 Win=63937 Len=0 TSV=8864993 TSER=442756188269 112.977340 32.131.175.115 10.1.1.20 TLSv1 Encrypted Handshake Message270 113.173886 10.1.1.20 32.131.175.115 TCP https > 52664 [ACK] Seq=2053 Ack=1159 Win=63644 Len=0 TSV=8865065 TSER=442756188271 113.198255 32.131.175.115 10.1.1.20 TLSv1 Change Cipher Spec272 113.201030 32.131.175.115 10.1.1.20 TLSv1 Encrypted Handshake Message273 113.201056 10.1.1.20 32.131.175.115 TCP https > 52664 [ACK] Seq=2053 Ack=1249 Win=63554 Len=0 TSV=8865068 TSER=442756188274 113.201067 32.131.175.115 10.1.1.20 TCP 52667 > https [ACK] Seq=579 Ack=176 Win=65535 Len=0275 113.201161 10.1.1.20 32.131.175.115 TCP https > 52664 [RST, ACK] Seq=2053 Ack=1249 Win=0 Len=0
Free Windows Admin Tool Kit Click here and download it now
February 9th, 2009 6:46pm

Just to post an update:Ok, finally got around to trying Windows Mobile emulator. Downloaded windows mobile emulator with 6.1 classic. Surfs the web just fine, connects to the owa side with IE. But fails to connect with the messaging option.When using the messaging feature, I'm still getting the [RST,ACK] packets as described earlier.Also, tried this just to help determine why the resets ( stumbled upon the command through the forums )Test-ActiveSyncConnectivity -MailboxCredential (Get-Credential mydomain\myuser) -UseAutodiscoverForClientAccessServer Inner error [System.IO.IOException]: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
March 10th, 2009 8:45pm

+ Few things i would like to highlight before i suggest you something,+ Ack Reset are not always bad, if ack reset generates after one party sends ack fin, it might be normal behavior..........................application can be design to close socket with Ack Reset instead of Ack fin..............why we do this? bcoz, we doesn't want, port to stay in "Time_Wait" for 4 minutes, instead we would like to make port available immediately for re-use.+ coming back to issue - in our case Ack Reset is send by server componentas source port is 443. + KB948830, talks about same issue and recommended to install rollup pack for Exch 2007.+ Update NIC drivers and disable offload feature under NIC properties.Verify that following registry key's set to value Zero.+HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\ParametersChange the following keys to '0' 'decimal'EnableTCPAEnableRSSEnableTCPChimney+ It can also caused by application, enable application logging to check if that provides more in-sights.Hope this helps.
Free Windows Admin Tool Kit Click here and download it now
March 11th, 2009 4:19am

It definitely gave me another point of view, so it helped.However, after going through the ideas here is my response to each one.Ack Reset I understand, but Reset Ack I don't. Everything I've ever read says this is bad if there isn't a Fin Ack before it.Would KB 948830 still apply if the Exchange 2007 install was the April 2008 release, and the updates started with Rollup 3?Nic Drivers are as updated as possible. Offloading is turned off as well.The following entries under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters do not exist:EnableTCPAEnableRSSEnableTCPChimneyApplication logging on the server is running. So far the only substantial events are the what appear to be normal event notifications, such as the Active Directory discovery, etc.
March 11th, 2009 5:01am

Ack Reset or Reset Ack is one and same..................depends how we read flags under TCP header. If you already have rollup 3, article will not apply to our scenario.Sorry, i overlooked OS..............server 2008 will not have 3 registry keys i mentioned above. Does we have any event logs specific to IIS/W3SVC services?Can you check what all third party filters we have on machine? For testing, is it possible to run box without those third party APP?Do we have any other app which interacts with IIS apart from Exchange?
Free Windows Admin Tool Kit Click here and download it now
March 11th, 2009 6:52am

The only filters I can think of are the default spam filters installed with Exchange 2007. In our case, most filtering is done before the mail reaches the exchange machine.The only third party application that is interacting with IIS is an installation of WSUS. But this problem was occuring before that was installed.
March 11th, 2009 2:42pm

hi there,Ok, finally got around to trying Windows Mobile emulatorby saying the above statement, i believe you have installed windows CE emulator ?, if so i believe you have developed your mobile application to communicate with the exchange ?so how have you handled the tcp communication in your custom application ?sainath Windows Driver Development
Free Windows Admin Tool Kit Click here and download it now
March 11th, 2009 6:23pm

The thing is I'm not developing a custom application. I am trying to get iPhones to communicate with our exchange setup. One of the suggestions was to install the windows mobile emulator and try connecting that way. I have the emulator surfing the web fine. It sees the exchange server, but won't sync.The most important issue here though, is that the iPhones seem to work half of the time, and not the other half. The only thing that seems to almost always work for me is the direct push.I have noticed, as I've said before that I only have errors on the phone when the [RST,ACK] flags are set and returned.This when I get a [RST,ACK] there is no or appears to be no log entry in the W3SVC logs for that attempt. Otherwise, there are entries in the log.
March 11th, 2009 8:36pm

+ Your lastsantence gives indication that its not IIS who is forcing socket closure and reason it doesn't log info. If its IIS, it will show that he called something and unable to read or corrupt or etc.+ Error are related to IO, which is more kernal based processing. It definitely looks filters on IIS box as IIS is generating Reset packets. Its encrypted packets, which mean we are not sure, what APP was doing at that point of time.+ On IIS box - open device manager---> click view, show hidden devices ----> expand non-plug and playCheck if we have third party drivers listed, if we have one, please unistalled those APP from box.Examples -- NIC teaming software, Anti Virus, third party firewall, spam filters, etchope this helps
Free Windows Admin Tool Kit Click here and download it now
March 12th, 2009 4:04am

Ok, Before I uninstall anything I want to make sure it needs to be. Other than the iphone tcp/ip resets everything else seems to be working fine, so I don't want to make it worse than it already is if it's not necessary.This is what I have under Non-Plug and Play Drivers:Ancillary Function Driver for WinsockCommon Log (CLFS)Crcdisk Filter DriverDISK VMBUS Acceleration Filter DriverDynamic Volume ManagerHTTPISA/EISA Class DriverKernel Mode Driver Frameworks serviceKSecDDLink-Layer Topology Discovery Mapper I/O DriverLink-Layer Topology Discovery ResponderMessage-oriented TCP/IP and TCP/IPv6 Protocol (SMB session)Mount Point ManagerNDIS System DriverNDProxyNETBTNetGroup Packet Filter DriverNETIO Legacy TDI Support DriverNSI proxy serviceNullPEAUTHQoS Packet SchedulerRDP Encoder Mirror DriverRDP Winstation DriverRDPCDDRemote Access Auto Connection DriverRemote Access IPv6 ARP DriverSecurity DriverSecurity Processor Loader DriverStorage VolumesTCP/IP Protocol DriverTCP/IP Registry CompatibilityTDTCPTerminal Services Security Filter DriverVgaSaveWindows Firewall Authorization Driver
March 12th, 2009 4:21am

+ List seems very much default. I don't see any third party filter driver which can cause issue.Only option i can think is to update TCPIP.sys and AFD.sys files or debug the box/App. I don't have any other action item.
Free Windows Admin Tool Kit Click here and download it now
March 12th, 2009 8:37am

+ Error are related to IO, which is more kernal based processing.I dont see any kernel based processing done , because both are win32 application, and reset of application is on the usermode. and not in the kernel modeYes i agree with the IO which has to happen with every OS , but there is no relation between socket and the Kernel IO.i would like to see if there are kernel mode errors and kernel based processingif there is kernel based processing would like to which module is communicating to which subsystem Questions=======Are there any IRQL errors or IRP which are failing ?sainath Windows Driver Development
March 12th, 2009 10:37am

I have never seen IRQL or IRP errors on this machine, no blue screens of death. The machine is not running virtualization.TCPIP.sys and AFD.sys are both version 6.0.6001.18000.
Free Windows Admin Tool Kit Click here and download it now
March 12th, 2009 2:33pm

Drivers are way too old - latest driver date are6.0.6001.22XXX according to KB 959816.TCPIP.sys is user mode drivers and AFD.sys is kernal mode drivers for TCPIP protocol suit. it will be worth updating both files, to latest version.
March 13th, 2009 3:38am

Good news... there are now fewer [RST,ACK]'s. I realize that we can't get rid of them completely, but man it sure would be nice to.So far, things are getting better. Again, Thanks for the all of the help. Do you have any other ideas to throw out that might eliminate the [rst,ack]'s more?
Free Windows Admin Tool Kit Click here and download it now
March 13th, 2009 4:50am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics