Use Hyper-V dynamic memory leads to full allocation of RAM in the Hyper-V guest since a few days, severe performance issues included

I have reported an issue to my Hyper-V hosting provider where I see a uncommon behaviour on my Hyper-V guest even when freshly deployed.

The Hyper-V Host system is Windows 2012 (assumingly Datacenter, clustered), the guest machine is a Generation 1 Windows 2012 R2, fully patched. Dynamic RAM is configured according to the Hyper-V Server provider.

The problem I see:

In my business environment (Windows 2012 R2 Host, Windows 2012 (R2) guests, with dynamic memory configured, aswell as in my testlab (Windows 10 CTP b10240 Host, Windows 2012 R2 guest generation 2) this problem does not occour.

After I have reported this to the Server provider, it now also occours in my Testlab. So it seems that a Windows Update is responsible for the problem.

Summary:

- RAM allocation in the taskmanager in the guest machine does not reflect the usage of the guest machine

- system performance incl. Windows UI is degraded in the guest OS, like it really would suffer full RAM, start swapping etc.

- this was not an issue before some days in my testlab, but now behaves the same. (all updates applied on the host, no updates applied on the guest)

- it does not matter if the guest OS is fully patched or not, it will show the same issue at the moment.


Evidence:
Normal behaviour of RAM consumption / allocation with Hyper-V dynamic RAM enabled on the Host (testlab, fresh installed machine, no updates on guest OS). The same situation exists in our productive environment Windows 2012 R2 Host, clustered but the hosts are not fully patched, yet.

http://imageshack.com/a/img909/5853/7ABr0Z.png (dynamic RAM enabled)
http://imageshack.com/a/img540/1921/9QDhAB.png (dynamic RAM disabled)

current behaviour of RAM consumption / allocation with Hyper-V dynamic RAM enabled on the Host (testlab, fresh installed machine, no updates on guest OS).

http://imageshack.com/a/img673/7048/K4Pgc2.png

current behaviour of RAM consumption / allocation with Hyper-V dynamic RAM enabled on the Host (Hosting provider, fresh installed machine, fully patched, incl. optional updates)

http://imageshack.com/a/img661/2030/xlBvZO.png

Because it is evident that the behaviour changed in the last few days, I hope this will raise awareness at Microsoft and hopefully help to identify the issue, moreover as this affects Hyper-V guests across plaforms.

What do you think?

cheers,

July 24th, 2015 3:20pm

edit: just noticed it was the same situation ever since, so excuse me if you think I am grasping straws here.

Nevertheless it prevents me to see the effective RAM usage on my guest because in case of my rented Hyper-V server I have no access to the Hyper-V host and may not see the current RAM load.

Nevertheless the performance on the rented machine is degraded.

Free Windows Admin Tool Kit Click here and download it now
July 24th, 2015 3:29pm

I am not fully clear on the symptom you are describing (please excuse me).  There are many different things that you are describing.

There was a Hyper-V specific patch released in this latest cycle.  That is true.  It does involve updates related to integration services.

Have you rebooted / patched and rebooted the VMs after the host was upgraded?  Or did they only suspend and continue to run while the host rebooted.

In regards to Dynamic Memory - I am not sure what you mean in that you cannot see effective RAM usage in your guest.  Or what you consider 'effective ram'.

Memory is only released to a VM as the applications in that guest request more (as identified by the virtual memory management service within the VM - a core windows function that all applications interact with).  And there is always a buffer that is given to the VM that it can burst into quickly, so that additional RAM can be background allocated and so on.  So your VM is always consuming the physical equivalent of allocated RAM plus buffer.

Your VM never knows what its ceiling is.  And any running application only knows that it can request more RAM. And the system either tells the application it can have more or it can't.  Then the application could page or the system could page to give more.

I just patched my test Hyper-V system and don't see any behavior that I would consider unusual, strange, or degraded.

That said, I also attempted to insert the integration services disk and see if the integration services needed to be upgraded.  The installer claimed that they didn't.

I went one level deeper and I manually forced an installation of the integration services CAB file and that succeeds.  (not knowing if any hard was done).  But I noticed no changes.

July 24th, 2015 4:10pm

There is this (https://support.microsoft.com/en-us/kb/3071740) optional update that is specific to VMs (8.1 and older) running on Windows 10 / Server 2016.

This could create issues if your host is not Windows 10, we have seen similar patterns with previous releases.

Free Windows Admin Tool Kit Click here and download it now
July 24th, 2015 4:14pm

Hello Brian, sorry that my description was not clear enough.

"In regards to Dynamic Memory - I am not sure what you mean in that you cannot see effective RAM usage in your guest.  Or what you consider 'effective ram'."

Yes, you have got me right. The effective RAM usage is not reported correctly in the task manager in the Hyper-V guest.

"Memory is only released to a VM as the applications in that guest request more (as identified by the virtual memory management service within the VM - a core windows function that all applications interact with).  And there is always a buffer that is given to the VM that it can burst into quickly, so that additional RAM can be background allocated and so on.  So your VM is always consuming the physical equivalent of allocated RAM plus buffer."

I understand this procedure like you and Microsoft decribed it. But somehow it affects me that I cannot estimate the current RAM consumption in the guest OS.

It maybe that I got something wrong here, as I can reproduce the situation with turning dynamic RAM / on or off so the situation in the taskmanager seems to be normal indeed.

I have digged the net for this "problem" and saw that people even reported this behaviour 2 years ago, either on Windows 7 or Windows 2008 Servers, so I guess at the end the current view in task manager is not unusual.

Thank you for your feedback in this re

July 24th, 2015 4:32pm

Feel free to ask questions.  That is what the forum is all about.

I will gladly attempt to describe what is happening as far as I am able.

The VM has no idea of how much of the host it is consuming, or if it is near its ceiling, or any information related to that.

That is not anything that the VM needs to know.  An operator of a VM might want to know that ( especially if they feel they are not getting what they are paying for ), but if he / she owns the host they can get it. 

As a consumer of capacity, a VM owner really should not know how much capacity ( of the host ) they are consuming.  Nor of the infrastructure as a whole.

You know the current allocation to the VM within the OS of the VM.  It is always a moment in time, that you can only track with some monitoring tool.

This would be the way to monitor and then prove that you are / are not getting the capacity you are paying for.

I have a feeling that you want to know the maximum RAM setting of Dynamic Memory from within the VM. 

But really, if your applications knew this, there are applications that would simply take it all and negate any reason to enable Dynamic Memory in the first place.

SQL is a total bad actor example in this case.  It would load everything in to RAM if it thought it could.

Free Windows Admin Tool Kit Click here and download it now
July 24th, 2015 4:54pm

Well that's why I posted it. The point you brought up that I cannot estimate if I get the amount I pay for is a good point but a different one I have not even handled, nor we handled possible overprovisioning on the Hyper V host.

If I run several RAM consuming application I would have to add their usage manually. (e.g. IIS, Java etc). The odd part is that I see the maximum amount of RAM the VM can get asset by the host (e.g 16 GB - http://imageshack.com/a/img661/2030/xlBvZO.png) but it is hard to track the usage in the guest VM when it is not shown as it would be case if dynamic RAM is disabled (http://imageshack.com/a/img540/1921/9QDhAB.png)

July 24th, 2015 5:03pm

I was just informed on another thread that Dynamic Memory is not working with Windows 10 / Server 2016.

I brought that up due to the wallpaper in all of the screen shots.

Does that mean it doesn't work, doesn't report properly, is it a management console issue, is it a VM issue, is it a host issue, it is a console issue.

If you are trying to gage if your host is saturated and your VM will thus be limited in order to support additional VMs - that is not possible from within a VM.  Unless you monitored and noticed that you VM was having RAM taken away or not getting more when your applications were requesting more.

But that is not really how things work with Hyper-V.  It will prevent new VMs from starting, then it will prevent VMs from asking for more.  It won't rebalance or take away unless clustering is involved and a high priority VM requires resources to run.  And this full host is the only option.

And it will always keep a percentage to keep the management OS healthy.

Free Windows Admin Tool Kit Click here and download it now
July 24th, 2015 5:22pm

I was just informed on another thread that Dynamic Memory is not working with Windows 10 / Server 2016.

I brought that up due to the wallpaper in all of the screen shots.


Hello Brian,

can you link that please?

"Does that mean it doesn't work, doesn't report properly, is it a management console issue, is it a VM issue, is it a host issue, it is a console issue."

This sentence somehow confuses me, can you explain further?

July 25th, 2015 5:15am

The post about it not working is in the nano server forum.

And my statement is that there could be one or many factors behind 'not working'.

Free Windows Admin Tool Kit Click here and download it now
July 25th, 2015 8:49am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics