At wits end - WinRM errors

I'm still using SCVMM 2008 R2 , primarily to support a Hyper V Cluster running on 2008 R2 on a CSV.

I don't know what is up with WinRM, but it causes the SCVMM to stop working.  I've had to move the VMM service from server to server to get the thing to work and eventually, over time, a WinRM error will catch up to it and I have to move the thing somewhere else.

After first installing somewhere, things will be fine for weeks and then at some unknown point, the remote Libraries will have their items be reported as Missing. And servers in the admin view give some kind of error like unreachable or access denied.

And then I won't be able to add any new hosts if they were't already added. Usually administrating the hosts that are there continues to function. 

Today I tried to add a host, and then I got the usual WinRM error

A Hardware Management error has occurred trying to contact server CHI-DEV04.bridgenet.lan. Check that WinRM is installed and running on server CHI-DEV04.bridgenet.lan. For more information use the command "winrm helpmsg hresult".
ID: 2927. Details: Unknown error (0x8033809d)

I tried troubleshooting these WinRM errors, but I get nowhere, so I give up on it. This is completely frustrating and nothing solves my problem.

July 20th, 2015 7:00pm

Hi Sir,

Please try to run the following command and post the result here :

winrm g winrm/config
winrm e winrm/config/listener

Also please refer to this article regarding Winrm troubleshooting :

http://blogs.technet.com/b/jonjor/archive/2009/01/09/winrm-windows-remote-management-troubleshooting.aspx

Best Regards,

Elton Ji

Free Windows Admin Tool Kit Click here and download it now
July 21st, 2015 5:53am

I've read that link many times and I never was able to solve any problems with it.


Windows PowerShell
Copyright (C) 2012 Microsoft Corporation. All rights reserved.

PS C:\Windows\system32> winrm g winrm/config
Config
    MaxEnvelopeSizekb = 2000
    MaxTimeoutms = 600000
    MaxBatchItems = 20
    MaxProviderRequests = 4294967295
    Client
        NetworkDelayms = 5000
        URLPrefix = wsman
        AllowUnencrypted = false
        Auth
            Basic = true
            Digest = true
            Kerberos = true
            Negotiate = true
            Certificate = true
            CredSSP = false
        DefaultPorts
            HTTP = 5985
            HTTPS = 5986
        TrustedHosts = *
    Service
        RootSDDL = O:NSG:BAD:P(A;;GA;;;BA)(A;;GA;;;S-1-5-21-184783182-3202599787-2024604991-1025)S:P(AU;FA;GA;;;WD)(AU;S
A;GWGX;;;WD)
        MaxConcurrentOperations = 4294967295
        MaxConcurrentOperationsPerUser = 1500
        EnumerationTimeoutms = 240000
        MaxConnections = 50
        MaxPacketRetrievalTimeSeconds = 120
        AllowUnencrypted = false
        Auth
            Basic = true
            Kerberos = true
            Negotiate = true
            Certificate = false
            CredSSP = false
            CbtHardeningLevel = Relaxed
        DefaultPorts
            HTTP = 5985
            HTTPS = 5986
        IPv4Filter = *
        IPv6Filter = *
        EnableCompatibilityHttpListener = true
        EnableCompatibilityHttpsListener = false
        CertificateThumbprint
        AllowRemoteAccess = true
    Winrs
        AllowRemoteShellAccess = true
        IdleTimeout = 7200000
        MaxConcurrentUsers = 10
        MaxShellRunTime = 2147483647
        MaxProcessesPerShell = 25
        MaxMemoryPerShellMB = 1024
        MaxShellsPerUser = 30
================================================================================================
================================================================================================
================================================================================================
PS C:\Windows\system32> winrm e winrm/config/listener
Listener
    Address = *
    Transport = HTTP
    Port = 5985
    Hostname
    Enabled = true
    URLPrefix = wsman
    CertificateThumbprint
    ListeningOn = 127.0.0.1, 169.254.42.180, 169.254.125.12, 172.20.10.172, 172.20.10.173, ::1, fe80::5efe:172.20.10.172
%17, fe80::5efe:172.20.10.173%19, fe80::5eb:fe2e:30b4:4829%14, fe80::5de6:88b8:6355:7d0c%15, fe80::d40a:abe0:e802:edcf%2
1, fe80::f5f6:ecd9:8382:2ab4%12

Listener [Source="Compatibility"]
    Address = *
    Transport = HTTP
    Port = 80
    Hostname
    Enabled = true
    URLPrefix = wsman
    CertificateThumbprint
    ListeningOn = 127.0.0.1, 169.254.42.180, 169.254.125.12, 172.20.10.172, 172.20.10.173, ::1, fe80::5efe:172.20.10.172
%17, fe80::5efe:172.20.10.173%19, fe80::5eb:fe2e:30b4:4829%14, fe80::5de6:88b8:6355:7d0c%15, fe80::d40a:abe0:e802:edcf%2
1, fe80::f5f6:ecd9:8382:2ab4%12

PS C:\Windows\system32>

July 21st, 2015 5:57am

In my group policy, I have a GPO that's applied to all machines, EXCEPT THIS ONE, that has the Server setting as

CredSSP - Enabled
Allow remote
server management through WinRM Enabled

And client side has  CredSSP Enabled

The server the VMM is on now, also had the same policy. but after VMM failed on a 2nd server, I did some reading and saw that WinRM can cause problems for VMM, so when I decided to install on the server its on now, I excluded the GPO I made for WinRM that I linked to the  domain from that server.

[I had to make a few edits.. there was some bizaree copy and paste text editing problems]


  • Edited by Vince P 21 hours 10 minutes ago
Free Windows Admin Tool Kit Click here and download it now
July 21st, 2015 6:01am

In my group policy, I have a GPO that's applied to all machines, EXCEPT THIS ONE, that has the Server setting as

CredSSP - Enabled
Allow remote
server management through WinRM Enabled

And client side has  CredSSP Enabled

The server the VMM is on now, also had the same policy. but after VMM failed on a 2nd server, I did some reading and saw that WinRM can cause problems for VMM, so when I decided to install on the server its on now, I excluded the GPO I made for WinRM that I linked to the  domain from that server.

[I had to make a few edits.. there was some bizaree copy and paste text editing problems]


  • Edited by Vince P Tuesday, July 21, 2015 10:04 AM
July 21st, 2015 10:00am

In my group policy, I have a GPO that's applied to all machines, EXCEPT THIS ONE, that has the Server setting as

CredSSP - Enabled
Allow remote
server management through WinRM Enabled

And client side has  CredSSP Enabled

The server the VMM is on now, also had the same policy. but after VMM failed on a 2nd server, I did some reading and saw that WinRM can cause problems for VMM, so when I decided to install on the server its on now, I excluded the GPO I made for WinRM that I linked to the  domain from that server.

[I had to make a few edits.. there was some bizaree copy and paste text editing problems]


  • Edited by Vince P Tuesday, July 21, 2015 10:04 AM
Free Windows Admin Tool Kit Click here and download it now
July 21st, 2015 10:00am

In my group policy, I have a GPO that's applied to all machines, EXCEPT THIS ONE, that has the Server setting as

CredSSP - Enabled
Allow remote
server management through WinRM Enabled

And client side has  CredSSP Enabled

The server the VMM is on now, also had the same policy. but after VMM failed on a 2nd server, I did some reading and saw that WinRM can cause problems for VMM, so when I decided to install on the server its on now, I excluded the GPO I made for WinRM that I linked to the  domain from that server.

[I had to make a few edits.. there was some bizaree copy and paste text editing problems]


  • Edited by Vince P Tuesday, July 21, 2015 10:04 AM
July 21st, 2015 10:00am

In my group policy, I have a GPO that's applied to all machines, EXCEPT THIS ONE, that has the Server setting as

CredSSP - Enabled
Allow remote
server management through WinRM Enabled

And client side has  CredSSP Enabled

The server the VMM is on now, also had the same policy. but after VMM failed on a 2nd server, I did some reading and saw that WinRM can cause problems for VMM, so when I decided to install on the server its on now, I excluded the GPO I made for WinRM that I linked to the  domain from that server.

[I had to make a few edits.. there was some bizaree copy and paste text editing problems]


  • Edited by Vince P Tuesday, July 21, 2015 10:04 AM
Free Windows Admin Tool Kit Click here and download it now
July 21st, 2015 10:00am

In my group policy, I have a GPO that's applied to all machines, EXCEPT THIS ONE, that has the Server setting as

CredSSP - Enabled
Allow remote
server management through WinRM Enabled

And client side has  CredSSP Enabled

The server the VMM is on now, also had the same policy. but after VMM failed on a 2nd server, I did some reading and saw that WinRM can cause problems for VMM, so when I decided to install on the server its on now, I excluded the GPO I made for WinRM that I linked to the  domain from that server.

[I had to make a few edits.. there was some bizaree copy and paste text editing problems]


  • Edited by Vince P Tuesday, July 21, 2015 10:04 AM
July 21st, 2015 10:00am

SCVMM 100% relies on WinRM for communication to and from remote machines.

It is sensitive to anything that disrupts or delays WinRM traffic.  This can be anything from Group Policies to network QoS rules, to the Hyper-V Server management OS sharing the virtual switch and not bumping up its priority, to DNS resolution issues.

SCVMM will report anything that interferes with the management of anything it is in control of as a WinRM error.  And, it will always toss the error of checking WinRM.

And it is extremely rare that WinRM is not running - the root issue is generally something else related to the network and WinRM communication between the SCVMM Server and the other servers it manages.

Free Windows Admin Tool Kit Click here and download it now
July 21st, 2015 12:57pm

Thanks for the caution, but yeah I get it. VMM crapping out is something I been going through for about a year with the vague references to WinRM problems. What I never can find is any resolution to this. All I'm doing with WinRM is trying to actually make use of it.  Surely that's what it's there for. But I figured if using WinRM is such a hassle with VMM, then on the newest VMM server, I would exclude it and that's what I did, and there's problems.

So anyway, I have no clue what's wrong, I have no clue how to fix it.  I've read plenty of blogs from 5 years ago all warning about problems but there's no answers. I even called MS last year when the libraries started to go mysteriously offline and after watching the guy type winrm commands for two hours on my server the conclusion was bad Windows Update.

I'm stuck with this HyperV Cluster 2008 so I cant even upgrade the VMM (not that I really want to continue using this POS software)

If someone can tell just what the magical WinRM stable setting is then I can go put this SCVMM on a new server but I've never read just what that configuration looks like.

July 21st, 2015 1:10pm

With Hyper-V 2008 you should at least be able to move to SCVMM 2008 R2. (I can't recall the support matrix off the top of my head).

Beyond that, be sure to follow MSFT recommendations of the time.  Each Hyper-V server needs a dedicated physical NIC for management.

With clustering in the mix you have to ensure that you have the default gateways, DNS resolution, etc. configured properly to ensure that your management communication is not happening over any other NIC.

This can be especially messy if your servers are multi-homed (more than one interface on the same subnet, or if the SCVMM server can 'see' more than one IP of the target hosts.).

Incorrect default gateway configuration can cause SCVMM to send a WinRM request to one NIC of a Hyper-V Server and then the Hyper-V Server actually responds in a different NIC.  And being TCP traffic, the reply coming for a different MAC than the request causes the SCVMM server to toss away the reply.  Thus you have lost communications.

Lots of folks get themselves into this type of situation without realizing it.

Also, I helped MSFT work in these recommendations long ago: http://blogs.technet.com/b/vishwa/archive/2011/02/01/tuning-scvmm-for-vdi-deployments.aspx

Free Windows Admin Tool Kit Click here and download it now
July 21st, 2015 1:26pm

All of that good advice and we already follow it.  We are using SCVMM 2008 R2.

The cluster has two nodes, using a file share witness.  Each node has 4 NICs... a management port, a port dedicated to the VMs,  a heartbeat port (Ethernet cable plugs the nodes directly into each other) and a Live migration port (Ethernet cable plugs the nodes together).. They're sharing 4 daisy chained Dell storage enclosures configured as Cluster Shared Volume.

Two weeks I had an incident with MS Tech Support whereby we corrupted the stability of the configuration (she had me remove the virtual switch from one server, and that caused VMM to freak out)

Because the woman didn't know what she was doing, I was able to clear up the misconfigured VMs. I told her that I turned SCVMM off because SCVMM is too reactively to bad config.  And then she had someone from SCVMM team call me a few days afterwards. I asked what the best way to bring VMM back on line ,  because when we turned it off, there wree a lot of config problems.  He didn't have much for an answer and I told him i'll just unsintall SCVMM and reinstall it again.  And I did. and it was working fine until yesterday when I noticed the WinRM problems returned.

July 21st, 2015 4:12pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics