Hyper-V: vmms.exe is creating two temp files in the System32 folder every five minutes
Hi,

I am running Windows 8.1 Pro (Version 6.3.9600) on an HP-Z800 workstation. Yesterday, I noticed that my C:\Windows\System32 folder contains more than 64,000 0-byte temp files. A pair of these files gets created every five minutes. Here is the tail end of a directory listing (dir vmg*.tmp /o-d):

<snip>
10/31/2013  11:04 AM                 0 vmgC7EC.tmp
10/31/2013  11:04 AM                 0 vmgC7ED.tmp
10/31/2013  10:59 AM                 0 vmg33EB.tmp
10/31/2013  10:59 AM                 0 vmg33EA.tmp
10/31/2013  10:54 AM                 0 vmg9FE8.tmp
10/31/2013  10:54 AM                 0 vmg9FD8.tmp
           64404 File(s)              0 bytes
               0 Dir(s)  148,178,698,240 bytes free

Notice from the listing that the first of these files was created last Halloween, 10/31/2013, at 10:54 AM.

I had applied three Windows updates at 10:30 AM that day. I can imagine that it took 20 minutes to install the updates and reboot, so the appearance of the first file *may* correspond to the application of one of the updates. Here are the updates I installed:
    Update for Windows 8.1 for x64-based Systems (KB2901549)
    nVidia - Graphics Adapter WDDM1.1 ... Quadro 2000
    nVidia - Graphics Adapter WDDM1.1 ... GeForce 210
    Definition Update for Windows Defender KB2267602 (Definition 1.161.1146.0)

Initially I suspected that KB2901549 was responsible (http://support.microsoft.com/kb/2901549), but I didn't see anything in the details that obviously explains this behavior.

Next I fired up Process Monitor, and started watching that folder. The culprit emerged as vmms.exe, Microsoft's Virtual Machine Management Service.

During the period of time I captured events, I saw that vmms.exe did the following:
    Creates the first temp file, vmg986C.tmp (Desired Access: Generic Read, Dis, ... OpenResult: Created)
    Calls WriteFile in a loop (maybe around 62 times?) and writes content into vmg986C.tmp
    Calls CloseFile on vmg986C.tmp
    Opens (calls CreateFile) on C:\Windows\System32\vmguest.iso, and then closes it (Desired Access: Read Attributes, Dis, OpenResult: Opened)
    Creates the second temp file (vmg989B.tmp) and then closes it (Desired Access: Generic Read, Dis, ... OpenResult: Created)
    Calls CreateFile again on vmguest.iso, and the result is SHARING VIOLATION (Desired Access: Read Attributes, Delete, Synchronize, Dis)
    Calls CreateFile on vmg986C.tmp, and then closes it (Desired Access: Read Attributes, Delete, ... OpenResult: Opened)
    Calls CreateFile again on vmguest.iso, and the result is SUCCESS. Then vmguest.iso is closed    (Desired Access: Read Attributes, ... OpenResult: Opened)
    Calls CreateFile on a third file, vmg98AC.tmp, and then closes it    (Desired Access: Generic Read, Dis, ... OpenResult: Created)
    It again tries to open vmguest.iso, and the result is SHARING VIOLATION    (Desired Access: Read Attributes, Delete, Synchronize, Dis, ...)

Next, Windows Defender notices the new files and does the following:
    Calls CreateFile on the first file, the one with content, and the result is NAME NOT FOUND    (Desired Access: Read Attributes, Dis)
    Calls CreateFile on the second file, vmg989B.tmp, and then closes it    (Desired Access: Read Attributes, Dis, ... OpenResult: Opened)
    Calls CreateFile on the third file, vmg98AC.tmp, and then closes it    (Desired Access: Read Attributes, Dis, ... OpenResult: Opened)

The result of all this is two new temp files, vmg989B.tmp and vmg98AC.tmp. I'm not sure when the first file was deleted. In five minutes there will be two more.

I use Hyper-V, with a Server 2012R2 guest VM that I use for development and testing. So it appears that vmms.exe being the culprit fits the facts.

This feels like a bug in vmms, rather than any kind of intentional or desired behavior. vmms.exe lives in C:\Windows\System32, so that could explain why the files are being written in that location.

Deleting the temp files is easy, and certainly I could write a script to clobber them every day, but I'd sure like to get to the root cause and prevent them from being created in the first place.

Has anybody encountered this before? Any suggestions for what to look at next?

Thanks in advance.

-Bruce Bauder
  • Moved by Marvin_Guo Monday, May 12, 2014 2:13 AM Virtualization issue
May 9th, 2014 7:46pm

The hyper-v forum is on technet.

This forum is for help in writing code.

Free Windows Admin Tool Kit Click here and download it now
May 9th, 2014 11:04pm

Hi,

This is abnormal, so far, I cannot reproduce this kind of phenomenon in my environment:

Try to disable and re-enable  the Hyper-V component to see if this issue persists.

Here is all the hotfixes about Hyper-V of Windows 8.1, installing them would get related newer system file:

http://blogs.technet.com/b/yongrhee/archive/2014/02/16/list-of-hyper-v-related-hotfixes-post-rtm-for-windows-8-1-rtm-and-windows-server-2012-r2-rtm.aspx

May 13th, 2014 9:16am

I've never seen this kind of behavior. 

Do you have something other than the default settings for %temp% or the user profile?

Free Windows Admin Tool Kit Click here and download it now
May 13th, 2014 12:51pm

Hi Bob, I do set %TEMP% to d:\temp, which is something I've done since dinosaurs roamed the earth. I would be surprised if that triggered the creation of these temp files last October, although I certainly would not rule that out at this point.

May 13th, 2014 5:01pm

Thank you for the suggestions. I scanned the article you linked and while nothing specifically addressed the behavior I am seeing, I will do some more digging and see if it might be worth grabbing any or all of them.

I will also try disabling and then re-enabling Hyper-V; I am currently using my VM to test a candidate patch release of our application so I can't do that immediately.

And, because I still find it awfully fishy that the first of these temp files appeared within 30 minutes of applying some Windows updates, I may try uninstalling one or more of them to see if that has any effect.

Free Windows Admin Tool Kit Click here and download it now
May 13th, 2014 5:22pm

I've seen stranger things with %temp% and Microsoft's virtualization products.  This isn't one of the problems I've seen so far.

It would be worth setting %temp% back to normal (and rebooting) to see if it still exhibits the same problem.  

Hyper-V has some different assumptions on permissions that maybe you're way of doing %temp% is butting up against some kind of limitation.

The updates in October may have been to the vmguest.iso and that might be why it showed up then.

May 13th, 2014 5:35pm

OK, that definitely sounds worth trying. Thanks again--I'll give it a shot.
Free Windows Admin Tool Kit Click here and download it now
May 13th, 2014 5:38pm

Well, I set %TEMP% back to %USERPROFILE%\AppData\Local\Temp and rebooted, and new temp files are still being created every five minutes in C:\Windows\System32 (arrgh).

Can you think of anything else to try next?

-Bruce

May 15th, 2014 1:55pm

Interesting!   I can't think of anything right now, but I'll see if I can get my setup to do the same.  I'll get back with you...    
Free Windows Admin Tool Kit Click here and download it now
May 15th, 2014 2:48pm

Do you see anything in the Hyper-V event logs?  (Event Viewer / Application and service logs / Hyper-V_*)

It may be some kind of permissions problem, but we need to figure out just what.

May 15th, 2014 3:19pm

Most of the Hyper-V categories listed informational messages, but Hyper-V-VMMS | Admin contains 2,468 errors, all identical, with Event ID 19510:

The Integration Services Setup Disk image could not be updated: The process cannot access the file because it is being used by another process. (0x80070020).

Unfortunately the messages don't specify which process is locking the file, but the errors are being logged at five-minute intervals, so this sure looks like it's related to the creation of the temp files.

I rebooted earlier this morning to change %PATH%, and it looks like a new version of vmguest.iso was generated when the system came up. This is what I see. I can mount the file and navigate its folders in Windows Explorer, and so it does not appear to be corrupted:

Directory of C:\Windows\System32
05/15/2014  06:17 AM        27,590,656 vmguest.iso

Oh (light bulb just went off), I think you already suspected this, but the naming convention for the temp files may indicate they're generated as part of the extraction operation from this ISO (vmg*). I had been thinking it just meant "Virtual Machine something-or-another". Perhaps vmms tries to extract some files from vmguest.iso, so it creates the temp files, discovers a permissions problem and bails out without cleaning up the temp files.

At any rate, I see the following permissions for vmguest.iso (Properties | Security tab in Explorer):

ALL APPLICATION PACKAGES: Read/execute and Read
SYSTEM:                                Full control
Local Administrators:              Full control
Users:                                   Read/Execute and Read

-Bruce

Free Windows Admin Tool Kit Click here and download it now
May 15th, 2014 4:03pm

This file could be locked if you have a VM started with it attached, but I don't know what's in vmms that would keep trying like that.    
May 15th, 2014 5:25pm

I noticed that vmguest.iso *was* mounted inside my running VM. I tried to eject it in Explorer and got an error message, so I did a mountvol /d and that worked. I'm guessing that it's been mounted ever since I built the VM and installed the integration services.

The temp files were still being created though, so I shut down my VM and then rebooted my host PC. After it came up, I left the VM in the "Off" state and watched for the appearance of temp files. Unfortunately, right on schedule, a new pair showed up within five minutes.

I punted and wrote a simple batch file that wakes up periodically and deletes these files, but I'll continue digging and see if I can't get to the root cause.

Thanks again for your help, I really appreciate it. Feel free to holler if you have other suggestions.

-Bruce

Free Windows Admin Tool Kit Click here and download it now
May 15th, 2014 8:20pm

Various curious!  I'll get back with you if I think of something.  About all we know is what it's not so far. :(    
May 15th, 2014 8:35pm

Hi,

I have just discovered the same issue on my developement Server 2012 R2 running with 2 VM (Windows 8.1 and CentOS 7). Will investigate and let you know in case I will find solution.

Regards,

Jacek

Free Windows Admin Tool Kit Click here and download it now
March 19th, 2015 1:50pm

Hi Jacek,

Take a look in Virtual Machine Connection and see if the Integration Services Setup Disk is currently mounted. I found that if you did that (from Action | Insert Integration Services Setup Disk), then you really need to eject it when you're done. Go to Media | DVD Drive | Eject.

It's been a while since I discovered this, and I completely forgot to update this thread at the time. But as silly as that sounds, having the Integration Services Setup Disk mounted was the cause of all those temp files.

I hope this helps.

-Bruce

March 19th, 2015 2:13pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics