Hyper V file transfer kills the network

Dear All,

        We have 4 Hyper V VM installations on 2 HP DL 380 G8 servers and have they are connected to HP NSA 2040 SAN Storage through 6 Gbps Fibre channel. The physical machines are clustered.

Out of the 4, 2 are Clustered Hyper V's running Microsoft Dynamics AX 2012 R3 and we have 2 Clustered Hyper V for Database server running MS SQL 2012 R2.

The issue here is that the APP servers are slow in communicating with the DB server. The Hyper V's have a dedicated 4 * 1Gbps interface cards teamed and the packets are getting timed out. But from any other machine in the subnet the communications are proper.

When ever i try to copy a file from the DB server to the APP server using SMB share it kills my network, the copy will happen but after the copy is completed the network comes backup.

Its strange scenario. APP server 2 APP server smb copy is fine, and from DB to DB smb copy is fine but cannot copy between them.

Can anyone please help?

Thanks,
Sriram A Das

April 20th, 2015 5:20pm

Tell us your server nic card model . There's an issue with broadcom cards and windows server 2012r2
Free Windows Admin Tool Kit Click here and download it now
April 20th, 2015 6:14pm

Thanks a lot for your reply..!

PCI\VEN_14E4&DEV_1657&SUBSYS_169D103C&REV_01
PCI\VEN_14E4&DEV_1657&SUBSYS_169D103C
PCI\VEN_14E4&DEV_1657&CC_020000
PCI\VEN_14E4&DEV_1657&CC_0200


HP Ethernet 1Gb 4-port 331FLR Adapter

This is the HW model.

Thanks,
Sriram A  Das


April 20th, 2015 6:29pm

And we have one more card.

HP Ethernet 1Gb 4-port 331T Adapter

PCI\VEN_14E4&DEV_1657&SUBSYS_3383103C&REV_01
PCI\VEN_14E4&DEV_1657&SUBSYS_3383103C
PCI\VEN_14E4&DEV_1657&CC_020000
PCI\VEN_14E4&DEV_1657&CC_0200

  • Proposed as answer by Hugomedina Monday, April 20, 2015 6:40 PM
  • Unproposed as answer by Hugomedina Monday, April 20, 2015 6:40 PM
Free Windows Admin Tool Kit Click here and download it now
April 20th, 2015 6:32pm

Please download and install the driver from the official site . In your nic case is this  : 

https://www.broadcom.com/support/ethernet_nic/netxtreme_server.php  . 

Your network adapter is developed by broadcom  : 

https://www.broadcom.com/products/Ethernet-Controllers-and-Adapters/Enterprise-Adapters/HP

Tell me about your results . 

Grts.

April 20th, 2015 6:42pm

Thanks a lot Hugomedina. Will try it and update you.

Sorry for the late reply it was too late in the night for us. :)

Free Windows Admin Tool Kit Click here and download it now
April 21st, 2015 4:40am

Hello SriramaDas

As I understand your problem, you are experiencing problems with VMs that are running inside of a Hyper-V cluster and that file transfers are not running as smoothly as one would expect when utilising the hardware that is in place. For Broadcom-specific hardware, historically there has been a problem with VMs and the automatic enabling of the Virtual Machine Queue (VMQ) feature of the NICs in question. The functionality is there to properly queue network packages that are destined to a certain VM and to balance the load on the physical network interface to best accomodate the load.

As the NIC supports the functionality, Windows Server enables it to provide the optimal performance. Since there has been a problem with the Broadcom drivers for HP NIC implementations, disabling the VMQ functionality for troubleshooting purposes is a good step in troubleshooting.

The disabling has to be done both on the physical NIC though either PowerShell or through its properties page in Windows Server, as well as going into the Advanced Settings of the NIC on the VM and disabling the VMQ functionality there.

To verify whether the VMQ functionality is disabled, open an elevated PowerShell command and enter

Get-NetAdapterAdvancedProperty

This will list all your network adapters on the server in question. By filtering through the -InterfaceDescription noun, you can then determine whether the functionality VMQ is disabled for the NIC in question.

If the VMQ setting is still enabled, try disabling it and then rebooting the VM. Please post an update on whether this solution aided you on the matter.

Yours
Richard Johansson
Hyper-V Specialist Consultant

April 21st, 2015 7:14am

Hi Richard,

           Thanks a lot for the update really appreciated.

I did make the changes in the Host Adapters, disabled the VMQ feature in the there. But the VMs dont have that kind of feature disable in their properties.

PS C:\Users\administrator.INTERCOILINT> 
PS C:\Users\administrator.INTERCOILINT> Get-NetAdapterAdvancedProperty | find "Virtual Machine Queues"
Ethernet 8                Virtual Machine Queues         Disabled                       *VMQ            {0}          
Ethernet 7                Virtual Machine Queues         Disabled                       *VMQ            {0}          
Public                    Virtual Machine Queues         Disabled                       *VMQ            {0}          
Heartbeat                 Virtual Machine Queues         Disabled                       *VMQ            {0}          
Ethernet 4                Virtual Machine Queues         Disabled                       *VMQ            {0}          
Ethernet                  Virtual Machine Queues         Disabled                       *VMQ            {0}          
Ethernet 3                Virtual Machine Queues         Disabled                       *VMQ            {0}          
Ethernet 2                Virtual Machine Queues         Disabled                       *VMQ            {0}          

PS C:\Users\administrator.INTERCOILINT> 

 The output above is from the host machine

But still when i copy a big file between VMs its killing the whole network, but if i copy from APP1 to DB1 is working fine and i dont have any issues. The copy from DB to APP is the killer.

Can you please guide me through this? The MS partner who implemented is nowhere to be seen :).

Thanks,
Sriram A  Das

Free Windows Admin Tool Kit Click here and download it now
April 21st, 2015 8:27am

Hi Sir,

I want to confirm something :

1. there are 4 VMs (APP1,App2 , DB1 ,DB2 ) , they are running on two nodes hyper-v cluster .

2. these VMs are connecting to external virtual switch and getting IP addresses from your LAN .

3. App1 and App2 are clustered , DB1 and DB2 are clustered .

>>But still when i copy a big file between VMs its killing the whole network

Please explain the meaning of "killing the whole network" , killing the LAN or just you can not access that VM .

>>When ever i try to copy a file from the DB server to the APP server using SMB share it kills my network,

>>but if i copy from APP1 to DB1 is working fine

Please try to move APP1 and DB1 to same cluster node then try to copy file again .

Any further information please feel free to let us know .

Best Regards,

Elton Ji

April 23rd, 2015 2:33am

Hi Eton,

           Thanks a lot for your support, we moved the VM from here to there, tried a lots of stuff but never worked.

The finally we changed the NLB configuration in the APP servers from unicast to multicast and things are improving now.

Killing means, the whole LAN stops working. we lose the whole network. Which was strange.

Any idea on why that happened?

Thanks,
Sriram,

Free Windows Admin Tool Kit Click here and download it now
April 26th, 2015 3:00am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics