Hi

We have a Windows 2008 R2 SP1 (6.1.7601 Service Pack 1 Build 7601) serving as file server. Clients are Windows XP and Windows 7.

The files are being served happily and all of a sudden the server stops continuing on NEW SMB2 connections.

For ex,

A. time0 : connection 1 (and all connections before it) came in and is successfully established and is being served

B. time1: I assume something happens to the internals of server

C. time1: connection 2 comes in and tcp handshake is successful.

D. time+1msec: client sends SMB2 Negotiate

E. time+200 msec: server sends an ACK

F. time+59~sec: Server sends a RST

G. Now all the new connections from same or different clients have TCP handshake go thru and a reset from the server on NegotiateRequest!!!!

H. XP clients work fine to same server means SMBv1 or server resource is not an issue

I. If a client had an ongoing connection from BEFORE B (say connection 1). It still gets served but new connections get reset.

J. The only work around is to reboot the server!! Until it happens again!!

This sounds like something on Windows 2008 R2 SMB2 stack which goes into a state where it intentionally stops taking new connection. Some kind of anti-DDOS behavior or something??

Appreciate any help

Here is D (time+1msec: client sends SMB2 Negotiate)

NetBIOS Session Service
    Message Type: Session message (0x00)
    Length: 155
SMB (Server Message Block Protocol)
    SMB Header
        Server Component: SMB
        SMB Command: Negotiate Protocol (0x72)
        NT Status: STATUS_SUCCESS (0x00000000)
        Flags: 0x18
            0... .... = Request/Response: Message is a request to the server
            .0.. .... = Notify: Notify client only on open
            ..0. .... = Oplocks: OpLock not requested/granted
            ...1 .... = Canonicalized Pathnames: Pathnames are canonicalized
            .... 1... = Case Sensitivity: Path names are caseless
            .... ..0. = Receive Buffer Posted: Receive buffer has not been posted
            .... ...0 = Lock and Read: Lock&Read, Write&Unlock are not supported
        Flags2: 0xc853
            1... .... .... .... = Unicode Strings: Strings are Unicode
            .1.. .... .... .... = Error Code Type: Error codes are NT error codes
            ..0. .... .... .... = Execute-only Reads: Don't permit reads if execute-only
            ...0 .... .... .... = Dfs: Don't resolve pathnames with Dfs
            .... 1... .... .... = Extended Security Negotiation: Extended security negotiation is supported
            .... .0.. .... .... = Reparse Path: The request does not use a @GMT reparse path
            .... .... .1.. .... = Long Names Used: Path names in request are long file names
            .... .... ...1 .... = Security Signatures Required: Security signatures are required
            .... .... .... 0... = Compressed: Compression is not requested
            .... .... .... .0.. = Security Signatures: Security signatures are not supported
            .... .... .... ..1. = Extended Attributes: Extended attributes are supported
            .... .... .... ...1 = Long Names Allowed: Long file names are allowed in the response
        Process ID High: 0
        Signature: 0000000000000000
        Reserved: 0000
        Tree ID: 65535
        Process ID: 65279
        User ID: 0
        Multiplex ID: 0
    Negotiate Protocol Request (0x72)
        Word Count (WCT): 0
        Byte Count (BCC): 120
        Requested Dialects
            Dialect: PC NETWORK PROGRAM 1.0
                Buffer Format: Dialect (2)
                Name: PC NETWORK PROGRAM 1.0
            Dialect: LANMAN1.0
                Buffer Format: Dialect (2)
                Name: LANMAN1.0
            Dialect: Windows for Workgroups 3.1a
                Buffer Format: Dialect (2)
                Name: Windows for Workgroups 3.1a
            Dialect: LM1.2X002
                Buffer Format: Dialect (2)
                Name: LM1.2X002
            Dialect: LANMAN2.1
                Buffer Format: Dialect (2)
                Name: LANMAN2.1
            Dialect: NT LM 0.12
                Buffer Format: Dialect (2)
                Name: NT LM 0.12
            Dialect: SMB 2.002
                Buffer Format: Dialect (2)
                Name: SMB 2.002
            Dialect: SMB 2.???
                Buffer Format: Dialect (2)
                Name: SMB 2.???

Here is E (time+200 msec: server sends an ACK)

    [Time delta from previous captured frame: 0.201044000 seconds]

    Flags: 0x010 (ACK)
        000. .... .... = Reserved: Not set
        ...0 .... .... = Nonce: Not set
        .... 0... .... = Congestion Window Reduced (CWR): Not set
        .... .0.. .... = ECN-Echo: Not set
        .... ..0. .... = Urgent: Not set
        .... ...1 .... = Acknowledgment: Set
        .... .... 0... = Push: Not set
        .... .... .0.. = Reset: Not set
        .... .... ..0. = Syn: Not set
        .... .... ...0 = Fin: Not set

Here is F (F. time+59~sec: Server sends a RST)

    [Time delta from previous captured frame: 59.765376000 seconds]

    Flags: 0x014 (RST, ACK)
        000. .... .... = Reserved: Not set
        ...0 .... .... = Nonce: Not set
        .... 0... .... = Congestion Window Reduced (CWR): Not set
        .... .0.. .... = ECN-Echo: Not set
        .... ..0. .... = Urgent: Not set
        .... ...1 .... = Acknowledgment: Set
        .... .... 0... = Push: Not set
        .... .... .1.. = Reset: Set
            [Expert Info (Chat/Sequence): Connection reset (RST)]
                [Message: Connection reset (RST)]
                [Severity level: Chat]
                [Group: Sequence]
        .... .... ..0. = Syn: Not set
        .... .... ...0 = Fin: Not set

Hi,

Was the ACK packet received on client side? Could it be dropped by midterm device or the client got the packet but didn't respond? You need to capture the packets on both sides. 

Meanwhile, in order to test if it is dierctly related to SMB2.0, you could temporarily disable SMB2.0 and check the result.

And for more information about NEGOTIATE, please see:

Processing NEGOTIATE Request in the Server

http://msdn.microsoft.com/en-us/library/dd909345(v=office.12).aspx

How are things going? Please let us know if there is any progress.

Hi Shaon,

Thanks for the reply and sorry for delay in my response.

Let's run this again to make it clear.

• The Client sends NegotiateRequest packet to server.
• The request should get thru SRVNET.SYS/NTFS.SYS etc to SRV2.SYS
• Thru these means, SMB2 protocol (on server) is supposed to send a NegotiateResponse immediately.
• It does not send NegotiateResponse
  
• The TCP Stack waits for 200 msec and then sends and ACK. This is because of following

Normally TCP does not send an ACK the instant it receives data. Instead, it delays the ACK, hoping to have data going in the same direction as the ACK, so the ACK can be sent along with the data. (This is sometimes called having the ACK piggyback with the data.) Most implementations use a 200-ms delay-that is, TCP will delay an ACK up to 200 ms to see if there is data to send with the ACK

• So this is the 200 msec delayed ACK we see going from server to client
• The client is not required or supposed to ACK an ACK. Hence whether the client receives this ACK or not is irrelevant

• For factual purposes and to answer your question, the client does get an ACK but for reasons mentioned above doesn't do anything and continues to wait for a NegotiateResponse.
• The server DOES NOT generate a NegotiateResponse hence we never see it.
• It then hits its 60 sec timer and sends a RST.

The question here becomes why the server is NOT sending the NegotiateResponse

Another thing that makes itself obvious above, is that the network and TCP stacks are fully functional as the ACK is generated and delivered. Its obviously a server thing that the SMB2 protocol is going in a weird state that it is unable or unwilling to generate a response.

Is there any bug in SMB2 protocol where it may feel that I am getting attacked or something like that and it stops serving new clients?

I have a similar problem where I have a SBS 2011 standard server and a Win7 client. If I let the client sit idle for a while and then start IE then I can see that the client sends a "SMB2 Create Request File" which the server doesn't respond to, so the client resends the request a couple of times, and then times out. After 20s (from the initial request) the client sends a "SMB2 NegitiateProtocolRequest" which the server responds to immediately, and then Communication starts to flow as normal.

We are seeing the exact same thing here as well, but on random servers.  Of our 58 or so file servers, 3-5 servers frequently exhibit this behavior.  A restart temporarily helps but only for awhile.  Have you been able to find anything definitive yet?

Not yet. I'm trying to resolve another issue described by

http://social.technet.microsoft.com/Forums/en-US/smallbusinessserver/thread/f19b570d-8087-4fd7-b54c-a1a4391bf34c/

Who knows, they might be related :-)

We are facing the same type of issue. Were you able to come up with a solution?

I've come a little bit further, and it seems to be related to WLAN. Because if I connect my notebook via LAN then the problem doesn't occur, and on another machine which is connected via LAN then it doesn't occur and on yet another machine connected via WLAN there the problem occurs.

How is your machine connected to the network ?

We dont enable WLAN. We have the issue with a local LAN network. I wonder if it is somethign related to a bug in some of the advanced features in SMB 2.0.

I've now found the root cause to my SMB2-problems. It turned out that it was a Cisco WAP200 access-point which had a ARP-Proxy function which filtered out ARP-Requests from the server to the client when the client hadn't sent any data for some minutes. But the ARP-Proxy function didn't send an ARP-Reply back to the server which eventuallly caused the server to believe the client had gone, and closed all file handles related to the client. So when client tried to access the file where it believed it had a vallid filehandle then server didn't respond which caused error handling to start in client to recover from the situation.

When I Contact Cisco it turns out that this is a known problem with their products and they recommend me to turn of the ARP-Proxy function if possible. And if it isn't possible to turn it off then they don't have any solution. Searing for ARP-Proxy shows that this function is available in a number of different Cisco Products.

To conclude: If you experince long delays when opening files, opening IE or Outlook, then search for ARP communication and make sure your switch or accesspoint doesn't filter out the traffic.

Hi,

I am just curious if you were able to find a solution to this problem? We are experiencing the very same symptoms you have mentioned and can not find out the cause or fix for it. Thanks,

Is the problem happening on local lan or when client and server have WAN in between?

When the server stops responding, it does so for both the local and remote clients. A server reboot then resolves the issue till next time.

What was the cause in your scenario? Was it something WAN related for you?

• Proposed as answer by MWebjorn Sunday, October 13, 2013 8:07 AM
• Unproposed as answer by MWebjorn Sunday, October 13, 2013 8:07 AM

Hi,

I am just curious if you were able to find a solution to this problem? We are experiencing the very same symptoms you have mentioned and can not find out the cause or fix for it. Thanks,

The resolution was to throw out the Cisco AP's and buy a AP's from HP. Now Communication works fine

When the server stops responding, it does so for both the local and remote clients. A server reboot then resolves the issue till next time.

Try this: Run Netmon3 (http://www.microsoft.com/en-us/download/details.aspx?id=4865) on server NIC and client until problem occurs. Then filter captured traffic on ARP-protocol, and look for ARP-Requests from server to client. If your problem is caused by a malfunctioning ARP-Proxy in your LAN then you should see ARP-Requests from the server, but no ARP-Reply's from client. So then you'll just have to move your client closer and closer to the server until you've found the bad LAN-unit. Do you have any Cisco equipment in your LAN ? Note that it's not only Cisco AP's that have ARP-Proxy's. It can also be present in switches and firewalls.

My testcase was the following: I had a 10MB pdf-file which I opened on the client. The reason for a large pdf was so that the client wouldn't cache the entire file when it was opened. I then left the client sit idle for one hour. And when I tried to scroll the document it immediately failed. It was then easy to locate the point when client SMB2 error handling jumped in to resolve the communication problem. In my case a server reboot wasn't neccesary, but I had to close the pdf and open it again to see the co

What was the cause in your scenario? Was it something WAN related for you?

See my reply Monday, August 26, 2013 11

Is the problem happening on local lan or when client and server have WAN in between?

I've only seen this problem while attaching to my LAN via WLAN AP's, not when attaching to my LAN via a VPN. But the ARP-Proxy in my Cisco FW was turned off so I can't say what would happen if I turned it on. I've just spend enough time on this &%#  problem, so I didn't bother to poke around on the VPN side.

Hi,

Were you able to find the cause? You asked me if we are having the issue on LAN or across WAN. The clients are spread across 6 remote sites and also local where the data center is. When the server abruptly stops responding to SMB2 requests, it just stops servicing any clients (local or remote) only on SMB2. Packet captures show the exact behavior that you described where the server do not send response to SMB2 protocol negotiate.

Could you tell if you were able to resolve your issue and what was the reason in your case? Thanks.

Are you using any WAN Optimization or IPS device between client and server? In my case there was something in the middle changing the SMB2 Session Setup packets...

• Edited by 2cool2touch Thursday, October 17, 2013 2:32 PM

WAN optimization > Yes. No IPS though. So it was random change of SMB2 session packets in your case (random in time and random packet)? Since at other times SMB2 worked fine (I would assume during working times there was no changing of SMB2 session packets)? 

What WAN optimization do u have

Riverbed. Appliances are in the xx50 series. 

If you don't already have .. then enable SMB Signing Optimization support .. and / or go to RiOS 8.0.4 or any 8.5.x version

Was your issue Riverbed related and in the 8.0.x release?

There were other things environmental as well .. but enabling SMB signing and upgrading RiOS helped with this specific issue

Thanks for the feedback. SMB signing is enabled on our appliances but we are running 8.0.3.

Is it fixed? I thought there was no signature in SMB negotiation. 

  Signature: 0000000000000000

Hi,

I am also facing the exact issue which is mentioned in the topic "Windows 2008 R2 server stops responding to SMB2 Command: NegotiateProtocol".

The situation is, none of my windows 7 clients machines were connecting to the Windows 2008 R2 file server until the server was rebooted.

Network Monitoring tells that no negotiate reply go from server to client.

Can anyone help me?

Thanks

Siva

Hi Siva,

Is your client connected via WLAN? Have you tried the network monitoring technique described above to isolate the problem? If so, what did you see?