Hyper-V-VmSwitch EventID 106

I have Windows Server 2012 with Hyper-V role. There is a network team on the server.

When the server starts I receive a error:

Log Name:      System
Source:        Microsoft-Windows-Hyper-V-VmSwitch
Date:          2/4/2013 2:44:50 PM
Event ID:      106
Task Category: None
Level:         Error
Keywords:     
User:          SYSTEM
Computer:      Seventeen.aqa.com.ru
Description:
Available processor sets of the underlying physical NICs belonging to the LBFO team NIC /DEVICE/{F81E5C50-00B3-4105-9CBB-04919F058B9D} (Friendly Name: Microsoft Network Adapter Multiplexor Driver) on switch 8CBADD0E-F4A3-45D4-AC2D-874C7DE6C821 (Friendly Name: AQA) are not configured correctly. Reason: The processor sets overlap when LBFO is configured with sum-queue mode.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Hyper-V-VmSwitch" Guid="{67DC0D66-3695-47C0-9642-33F76F7BD7AD}" />
    <EventID>106</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8000000000000000</Keywords>
    <TimeCreated SystemTime="2013-02-04T10:44:50.175406400Z" />
    <EventRecordID>11345</EventRecordID>
    <Correlation />
    <Execution ProcessID="4" ThreadID="380" />
    <Channel>System</Channel>
    <Computer>Seventeen.aqa.com.ru</Computer>
    <Security UserID="S-1-5-18" />
  </System>
  <EventData>
    <Data Name="NicNameLen">46</Data>
    <Data Name="NicName">/DEVICE/{F81E5C50-00B3-4105-9CBB-04919F058B9D}</Data>
    <Data Name="NicFNameLen">44</Data>
    <Data Name="NicFName">Microsoft Network Adapter Multiplexor Driver</Data>
    <Data Name="SwitchNameLen">36</Data>
    <Data Name="SwitchName">8CBADD0E-F4A3-45D4-AC2D-874C7DE6C821</Data>
    <Data Name="SwitchFNameLen">3</Data>
    <Data Name="SwitchFName">AQA</Data>
    <Data Name="QueueMode">2</Data>
  </EventData>
</Event>

February 4th, 2013 10:59am

Hi Vadim!

You will have to:
1. - Breaking the Network Teaming
2. - No configure Ethernet
3. - Rebuild the Network Teaming
4. - Configuring Teaming with the chosen IP

Regards!

Free Windows Admin Tool Kit Click here and download it now
February 4th, 2013 11:09am

I did it several times.
February 5th, 2013 10:36am

It doesn't work
Free Windows Admin Tool Kit Click here and download it now
February 8th, 2013 4:43pm

You've misconfigured VMQ on the physical NICs in your team.  Please read the NIC teaming user's guide at http://www.microsoft.com/en-us/download/confirmation.aspx?id=30160.  The relevant section on NIC teaming and VMQ goes into detail about sum-mode vs min-mode VMQ and how to configure each correctly.  You may also find the section on Dynamic VMQ at http://msdn.microsoft.com/en-us/library/windows/hardware/jj248719.aspx to be helpful.

But I'll give you the high-level overview here.  The NIC Team is in VMQ sum-of-queues mode so you want to configure the processor sets on the underlying physical NICs in the team to not overlap.  As an example, if you have 16 CPUs and 2 NICs in your team, one correct configuration would be to assign CPUs 2-7 to the first NIC and CPUs 8-15 to the second NIC.


February 16th, 2013 12:03am

I suppose you meant these settings:

=====

PS C:\Windows\system32> Get-NetLbfoTeam|fl

Name                   : team0
 Members                : {Ethernet 3, Ethernet 2, Ethernet 4, Ethernet}
 TeamNics               : team0
 TeamingMode            : SwitchIndependent
 LoadBalancingAlgorithm : HyperVPort
 Status                 : Up

PS C:\Windows\system32> Get-NetAdapterRss|fl

Name                                            : Ethernet
 InterfaceDescription                            : Intel(R) I350 Gigabit Network Connection
 Enabled                                         : True
 NumberOfReceiveQueues                           : 2
 Profile                                         : Closest
 BaseProcessor: [Group:Number]                   : :4
 MaxProcessor: [Group:Number]                    : :63
 MaxProcessors                                   : 7
 RssProcessorArray: [Group:Number/NUMA Distance] :
 IndirectionTable: [Group:Number]                :

Name                                            : Ethernet 2
 InterfaceDescription                            : Intel(R) I350 Gigabit Network Connection #2
 Enabled                                         : True
 NumberOfReceiveQueues                           : 2
 Profile                                         : Closest
 BaseProcessor: [Group:Number]                   : :11
 MaxProcessor: [Group:Number]                    : :63
 MaxProcessors                                   : 7
 RssProcessorArray: [Group:Number/NUMA Distance] :
 IndirectionTable: [Group:Number]                :

Name                                            : Ethernet 4
 InterfaceDescription                            : Intel(R) I350 Gigabit Network Connection #4
 Enabled                                         : True
 NumberOfReceiveQueues                           : 2
 Profile                                         : Closest
 BaseProcessor: [Group:Number]                   : :25
 MaxProcessor: [Group:Number]                    : :7
 MaxProcessors                                   : 7
 RssProcessorArray: [Group:Number/NUMA Distance] :
 IndirectionTable: [Group:Number]                :

Name                                            : Ethernet 3
 InterfaceDescription                            : Intel(R) I350 Gigabit Network Connection #3
 Enabled                                         : True
 NumberOfReceiveQueues                           : 2
 Profile                                         : Closest
 BaseProcessor: [Group:Number]                   : :18
 MaxProcessor: [Group:Number]                    : :63
 MaxProcessors                                   : 7
 RssProcessorArray: [Group:Number/NUMA Distance] :
 IndirectionTable: [Group:Number]                :

Name                                            : team0
 InterfaceDescription                            : Microsoft Network Adapter Multiplexor Driver
 Enabled                                         : True
 NumberOfReceiveQueues                           : 0
 Profile                                         : NUMAStatic
 BaseProcessor: [Group:Number]                   : 0:0
 MaxProcessor: [Group:Number]                    : 0:30
 MaxProcessors                                   : 4
 RssProcessorArray: [Group:Number/NUMA Distance] : 0:0/0  0:2/0  0:4/0  0:6/0  0:8/0  0:10/0  0:12/0  0:14/0
                                                   0:16/0  0:18/0  0:20/0  0:22/0  0:24/0  0:26/0  0:28/0  0:30/0
 IndirectionTable: [Group:Number]                :
=====

I think there are no overlaped processor sets. But the error occurs.

I have another server with the same hardware and the same settings. It has overlaped processor sets by default. And the error does not occur on it.

What else should  I check on my "bad" server?

           
Free Windows Admin Tool Kit Click here and download it now
February 18th, 2013 10:39am

I was wrong (too many windows was opend at the moment when I copied the output of Get-NetAdapterRss cmdlet).

I reconfigured the NICs and the 106 error has gone.

=====

PS C:\Windows\system32> Get-NetAdapterRss|fl


Name                                            : team0
InterfaceDescription                            : Microsoft Network Adapter Multiplexor Driver
Enabled                                         : True
NumberOfReceiveQueues                           : 0
Profile                                         : NUMAStatic
BaseProcessor: [Group:Number]                   : 0:0
MaxProcessor: [Group:Number]                    : 0:30
MaxProcessors                                   : 4
RssProcessorArray: [Group:Number/NUMA Distance] : 0:0/0  0:2/0  0:4/0  0:6/0  0:8/0  0:10/0  0:12/0  0:14/0
                                                  0:16/0  0:18/0  0:20/0  0:22/0  0:24/0  0:26/0  0:28/0  0:30/0
IndirectionTable: [Group:Number]                :

Name                                            : NIC1
InterfaceDescription                            : Intel(R) I350 Gigabit Network Connection
Enabled                                         : True
NumberOfReceiveQueues                           : 1
Profile                                         : Closest
BaseProcessor: [Group:Number]                   : :2
MaxProcessor: [Group:Number]                    : :63
MaxProcessors                                   : 4
RssProcessorArray: [Group:Number/NUMA Distance] :
IndirectionTable: [Group:Number]                :

Name                                            : NIC3
InterfaceDescription                            : Intel(R) I350 Gigabit Network Connection #3
Enabled                                         : True
NumberOfReceiveQueues                           : 1
Profile                                         : Closest
BaseProcessor: [Group:Number]                   : :18
MaxProcessor: [Group:Number]                    : :63
MaxProcessors                                   : 4
RssProcessorArray: [Group:Number/NUMA Distance] :
IndirectionTable: [Group:Number]                :

Name                                            : NIC2
InterfaceDescription                            : Intel(R) I350 Gigabit Network Connection #2
Enabled                                         : True
NumberOfReceiveQueues                           : 1
Profile                                         : Closest
BaseProcessor: [Group:Number]                   : :10
MaxProcessor: [Group:Number]                    : :63
MaxProcessors                                   : 4
RssProcessorArray: [Group:Number/NUMA Distance] :
IndirectionTable: [Group:Number]                :

Name                                            : NIC4
InterfaceDescription                            : Intel(R) I350 Gigabit Network Connection #4
Enabled                                         : True
NumberOfReceiveQueues                           : 1
Profile                                         : Closest
BaseProcessor: [Group:Number]                   : :26
MaxProcessor: [Group:Number]                    : :63
MaxProcessors                                   : 4
RssProcessorArray: [Group:Number/NUMA Distance] :
IndirectionTable: [Group:Number]                :

=====


February 20th, 2013 10:14am

I am having this same issue.

What steps/commands did you use to correct this?

Free Windows Admin Tool Kit Click here and download it now
September 17th, 2013 10:29am

Great article about VMQ can be found here:

http://blogs.technet.com/b/networking/archive/2013/09/24/vmq-deep-dive-1-of-3.aspx


  • Edited by istvanh Sunday, December 01, 2013 12:47 PM
December 1st, 2013 12:47pm

Now I read a lot about VMQ but still have now clue about how to fix the problem. Anybody has a suggestion how to correct the issue? Thanks.
Free Windows Admin Tool Kit Click here and download it now
January 8th, 2014 9:11am

What you need to do is configure each of the NIC's in the Team that has generated the error so that they do not overlap on the same cpu cores.

For example, if your VM Team had 2 nic's called vm-nic-1 and vm-nic-2 and your server had 12 cores available you could configure each Nic to use 4 cores without overlapping like this:

Set-NetAdapterRss -name vm-nic-1 -BaseProcessorNumber 4 -MaxProcessorNumber 7

Set-NetAdapterRss -name vm-nic-2 -BaseProcessorNumber 8 -MaxProcessorNumber 12

  • Proposed as answer by Gary Hay [MSFT] Wednesday, January 08, 2014 9:21 AM
January 8th, 2014 9:19am

Gary, I will try that and let you know if that solved the issue.
I have indeed two 10 Gb NIC's. And the server has 12 core processors.
I see that one NIC uses cores from 4 to 7 (4 cores) and the other one from 8 to 12 (5 cores). Is that correct of should it read:
Set-NetAdapterRss -name vm-nic-2 -BaseProcessorNumber 8 -MaxProcessorNumber 11

Thanks for your help.

Free Windows Admin Tool Kit Click here and download it now
January 8th, 2014 3:55pm

Yep my bad, should be 8 -11 and not 8 - 12.
January 8th, 2014 4:05pm

Gary,
Thanks again. That solved my problem.

  • Proposed as answer by Jan_MarekMVP Wednesday, March 19, 2014 2:05 PM
Free Windows Admin Tool Kit Click here and download it now
January 9th, 2014 6:52am

Greate post. i solved my issu regarding this error:) I know its old. But this was the post witch came up when i was struggling.

i ahve 2 CPU 12 cores.

Set-NetAdapterRss -name vm-nic-1 -BaseProcessorNumber 4 -MaxProcessorNumber 7

Set-NetAdapterRss -name vm-nic-2 -BaseProcessorNumber 8 -MaxProcessorNumber 11

above fixed the problem. If you dont set this up, you will get alot of error when impleementing failover cluster for VM

November 13th, 2014 11:12am

Can anyone on this thread tell me what hardware this issue was showing up on?  Would not happen to be a HP c7000 series blade chassis, would it?
Free Windows Admin Tool Kit Click here and download it now
December 19th, 2014 12:56am

Just got this on a C7000 with HP Flex10s, trying to figure out the right path to fix.
January 2nd, 2015 7:25pm

I Have the same problem, i got c7000 with VC 4.31. my hyper-v farm have commuinicate problems
Free Windows Admin Tool Kit Click here and download it now
April 14th, 2015 1:56pm

Anyone with VMQ issues should post a new thread in this forum, i'll happily help out Where I can, all I ask is that you post which teaming method you are using, how many NICs are in the team, and how many sockets \ cores are in the host. also if you can do an initial screen grab of a GET-NETADAPTERVMQ.
April 15th, 2015 9:29am

Hello I was hoping for some help. I am seeing error ID 106 anytime that I create a checkpoint in hyper-v or if I do a live migration via cluster manager. Once I move all VM's of the 4th node of my Hyper-V cluster, I have no issues creating checkpoints or doing live migrations on the other 3 nodes. Only when they are running on node 4.  All 4 hosts were spec'd the same and have identical hardware. It has 2 dual port10gb HP 560SFP+ NICs and two teams configured. Each team("LS-WUCON" & "Team_Storage  is configured with switch independent, and dynamic load balancing.  The get-netadaptervmq command returns the same results on all 4 hosts. Why am I not having this very issue on the other 3 nodes. There were no driver/firmware updates installed and set-netadapterRSS was never ran on any node.

PS C:\Windows\system32> GET-NETADAPTERVMQ

Name                           InterfaceDescription              Enabled BaseVmqProcessor MaxProcessors NumberOfReceive
                                                                                                        Queues
----                           --------------------              ------- ---------------- ------------- ---------------
Ethernet 6                     HP Ethernet 1Gb 4-port 331FL...#4 True    0:0              16
Ethernet 2                     HP Ethernet 1Gb 4-port 331FL...#3 True    0:0              16
Ethernet 5                     HP Ethernet 1Gb 4-port 331FL...#2 True    0:0              16
Ethernet 3                     HP Ethernet 1Gb 4-port 331FLR ... True    0:0              16
LS-Wucon                       Microsoft Network Adapter Mu...#3 True    0:0                            126
Z1-6                           HP Ethernet 10Gb 2-port 560S...#4 False   0:0              16            0
Y1-7_Wucon                     HP Ethernet 10Gb 2-port 560S...#3 True    0:0              16            63
Y1-6                           HP Ethernet 10Gb 2-port 560S...#2 False   0:0              16            0
Z1-7_Wucon                     HP Ethernet 10Gb 2-port 560SFP... True    0:0              16            63
Team_Storage                   Microsoft Network Adapter Mult... False   0:0                            0

Any assistance would be appreciated.

Thank you

R.W.

Free Windows Admin Tool Kit Click here and download it now
July 10th, 2015 2:59pm

I have another update that I have learned since my last post. It appears that the issue with my vm's rebooting whenever I create a checkpoint is related to it being a role in the cluster. I removed it from the cluster and am able to create checkpoints via Hyper-V manager. I am not sure if this news helps or not, just thought it was worth mentioning. I have since put the 4th node of my hyper-v cluster in maintenance mode and have no issues doing live migrations between hosts. The only time I can get a VM to reboot is when I live migrate it from node4 to any of the other nodes in the cluster.

R.W.

July 10th, 2015 4:29pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics