I am trying to wrap my head around the configuration of SMB Multichannel on a Hyper-V cluster that is NOT using a SOFS cluster - so, using SMB Multichannel for Live Migration and CSV I/O redirection only.

I am seeing a lot of guidance online that says the possible NICs for SMB Multichannel should be restricted.  On the cluster, I would expect the restriction to only allow the Cluster/CSV and Live Migration networks for SMB Multichannel.  How is this achieved?  Everything that I see on setting a constraint with New-SmbMultichannelConstraint is setting the constraint against an SMB file server.  I do not have one to specify.

Additionally, I am using a converged network configuration, with teamed physical interfaces, a Hyper-V virtual switch, and virtual adapters off of that switch for management, cluster/CSV, and live migration networks.  Are these constraints only set on physical NICs?

Strange, when I run Get-SmbMultichannelConnection on any of my hosts, on the client IP it shows connections coming from the management network (10.1.80.x/24), connecting over to the cluster network (10.1.81.x/24) on other hosts.  The Live Migration network (10.1.82.x/24) appears to be functioning as expected.

Server Name    Selected       Client IP      Server IP
-----------    --------       ---------      ---------
fe80::ecad:... True           10.1.82.15     10.1.82.16
fe80::ecad:... True           10.1.80.15     10.1.81.16

Am I overthinking this?  I just don't want an I/O redirection to occur, with the redirection happening on a management interface that impedes our access to the system.

Because the cluster service initiates Live Migrations and Redirected Access and cluster heartbeating, it gets to decide which networks to use. As long as you've configured those correctly (Failover Cluster Manager does this best), then everything will probably be fine.

You may see cluster traffic across your management adapter because it is, by necessity, allowed for cluster communications. I'm guessing by your output that you made your cluster network routable? If so, remove any gateways from it (LM network too). This will have the primary desired goal of breaking the connection from the management network to the cluster network. The cluster also greatly prefers non-routed networks, which will have the secondary desired goal of keeping cluster communications off of your management network unless the cluster has no choice.

I don't know if just removing the gateway will be enough to get the cluster to automatically generate a new metric. After you've got the gateway off, run "Get-ClusterNetwork  | select Name, Metric". The cluster network should be reassigned something in the 30000s with the management network in the 70000s. Probably the easiest way to get it to regenerate is by de- and re-IPing the network on the

Thank you for the response, Eric.  No gateways are defined on the cluster networks.  The metrics and roles appear to be properly assigned, below.

One item of note - IPv6 is not enabled on all interfaces.  We intend on correcting this during a maintenance window over the weekend.  Still, I would not expect the management network to attempt SMB multichannel communication with the cluster network.

Another proposed suggestion is to set static metrics, potentially disable SMB Multichannel, and lock SMB to the management interface.  This does not seem feasible to me, as a MinimumBandwidthWeight is assigned on the management interface to restrict traffic.  Even with multichannel disabled, if we implemented this, redirected I/O would occur just on the management interface, correct?  The AutoMetric seems to be true to what is expected, and my thought is to leave it and SMB Multichannel in place.

My hope is enabling IPv6 on all of the interfaces stabilizes things.

Name          Metric  AutoMetric   Role
----          ------  ----------   ----
Cluster       39840   True         1
iSCSI         70384   True         0
Management    79840   True         3
Migration     39841   True         1

If you don't have gateways on the cluster network adapters, then how are the management adapters able to reach them at all? If 10.1.81.16 is not routable and not in the same subnet as 10.1.80.15, then your final line of output in your first listing does not make sense.

IPv6 won't help you.

Huh. Well, mine looks like yours. Never noticed that before. I need to dig a bit to see what that's all about.

Ours is working fine like this. Is there something in particular that you're having problems with or are you just worried that it's a problem?

Before I say anything else, I want to reiterate that the cluster service controls all of its inter-node communications. Messing with SMB-MC is not going to yield results for automatic inter-node traffic.

The management adapter must be enabled for cluster communications. Therefore, the cluster might use it for communications. Due to the metric that you have, it will use it only if it doesn't have another choice.

Now, as for why you see what you see in the multichannel output, it's most likely because a gateway exists in the same network as 10.1.81.16. Basically, 10.1.80.15 has talked to a local router that has talked to another router and between them, they figured out how to make a path to 10.1.81.16. The fact that you didn't notify 10.1.81.16 of a gateway is completely irrelevant to 10.1.80.15. It is only concerned about communications from 10.1.80.15 to 10.1.81.16. Communication from 10.1.81.16 to 10.1.80.15 is a problem for 10.1.81.16. Since the adapter that owns that IP is not aware of a gateway, then two-way communications cannot occur. If SMB-MC attempts to use this adapter pair, it will fail.

I do not see this second line in my home lab (which is the one I wrote my first response from) because I am in 100% control and have built a fully walled garden for my LM and cluster comm networks. No gateway exists at all in either network; there is no path in or out. So, if you want that line to go away, then you need to eliminate the ability for 10.1.80.15 to find a pathway into the network that 10.1.81.16 is a member of.

Or, you could just not worry about it. That would work pretty well, too.

In addition to what we are seeing with SMB Multichannel, we are having some communication issues between the nodes.  We can restart all nodes in the cluster, and communication is fine between them for a few days.  Then, suddenly some cluster nodes cannot connect to Hyper-V Manager on other nodes, or to the Failover Cluster Manager.  If managed from a remote system outside the cluster, either with Hyper-V Manager, Failover Cluster Manager, or SCVMM, all is fine.  The VMs have no communication issues.

Kind of aligns with what you were saying about a route or routes existing somewhere, and communication being attempted between the networks.  All of the networks remain up in Failover Cluster Manager, though, and no failover issues occur.

I also see this article - http://blogs.msdn.com/b/clustering/archive/2014/03/25/10510616.aspx.  Do we need to statically address our IPv6 interfaces?  The article states only one internal cluster network can use fe80 link-local addresses.

Have you specifically disallowed all non-management adapters to register in DNS? DNS doesn't fit wonderfully with your symptoms, but it's the first thing I check.

I'll have to look at my home lab, but I don't believe that I change anything with IPv6 anymore. I used to disable it because seriously, what is IPv6 on a network without a few million endpoints except cryptic overhead? But, the common practice is shifting toward leaving it on, so I'm pretty sure that's what I've been doing. Whatever I'm doing, I can tell you that I'm absolutely not using static assignments or DHCP reservations or anything like that.

I checked my home lab.

I have disabled IPv6 on the adapters that I use to connect via iSCSI and SMB3 to my back-end storage unit, which is just running WS2012R2. All other adapters have IPv6 enabled and in the default configuration. All storage adapters physical. The management adapter on my Hyper-V hosts is physical. The management adapter on my storage host is virtual. The adapters for cluster communications and Live Migration are virtual and live on a team of two physical adapters.

I have set up explicit constraints on SMB 3 from each of my Hyper-V hosts. The only server that they affect is the back-end storage host. The only reason I did that is because I have the back-end unit running Hyper-V so that I can test Replica, and it's got a Hyper-V virtual switch on its physical management adapter so that I can keep the storage lines physical. Because of that virtual adapter, SMB 3 thinks "Oh, neat, a 10GbE connection! Even though it doesn't have RSS support, let me use that instead of these two dedicated 1GbEs that do have RSS!" The underlying physical adapter is really only a lone 1GbE. So, constraints it is. Get-SMBMultichannelConstraint only lists that target. Get-SMBMultichannelConnection also only lists the back-end.

My MRTG graphs show that the Cluster and Live Migration networks are both being used evenly. The Management network also registers continuous traffic so I assume that it is also part of the communications but since I use it more regularly the chart is inconclusive. Now that I have a metering tool like this, I should probably do some more formal testing, but I can tell you that in the past I have never been drowned out of my host when I was doing cable-pull with Live Migration testing.

This is a daily graph in which I didn't do anything meaningful.