When taking VSS snapshot with Hyper-V VSS writer on Windows 2012 R2, there are frequently some warning/error logs in Windows Event log. This will make the Hyper-V VSS writer goes to a bad state and in BCD, the BACKUP_METADATA is also missing for the problematic VM.

The logs in Windows Event log are as below:

In Windows Log->Application, there will be logs as:

A VSS writer has rejected an event with error 0x800423f4, The writer experienced a non-transient error.  If the backup process is retried,

the error is likely to reoccur.

. Changes that the writer made to the writer components while handling the event will not be available to the requester. Check the event log for related events from the application hosting the VSS writer.

Operation:

   PrepareForSnapshot Event

Context:

   Execution Context: Writer

   Writer Class Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}

   Writer Name: Microsoft Hyper-V VSS Writer

   Writer Instance ID: {fec29c4d-410b-4854-96b4-9637f3a24fdc}

   Command Line: C:\Windows\system32\vmms.exe

   Process ID: 2024

In Hyper-V-VMMS there are logs as:

Could not create backup checkpoint for virtual machine '********': Overlapped I/O operation is in progress. (0x800703E5). (Virtual machine ID 2AD8D389-65AE-45F7-89F8-FDF2F08CE3C2)

In Hyper-V-Worker there are logs as:

'******'has encountered a fatal error.  The guest operating system reported that it failed with the following error codes: ErrorCode0: 0x7A, ErrorCode1: 0x10EC78, ErrorCode2: 0xC000000E, ErrorCode3: 0x1BA09BE0, ErrorCode4: 0x21D8F000.  If the problem persists, contact Product Support for the guest operating system.  (Virtual machine ID 2AD8D389-65AE-45F7-89F8-FDF2F08CE3C2)

'******' has encountered a fatal error.  The guest operating system reported that it failed with the following error codes: ErrorCode0: 0xEF, ErrorCode1: 0x11F2780, ErrorCode2: 0x0, ErrorCode3: 0x0, ErrorCode4: 0x0.  If the problem persists, contact Product Support for the guest operating system.  (Virtual machine ID 16A22441-2ECB-474A-8998-ECA6102DE5AC)

'******'has encountered a fatal error.  The guest operating system reported that it failed with the following error codes: ErrorCode0: 0x7A, ErrorCode1: 0x119AEF8, ErrorCode2: 0xC000000E, ErrorCode3: 0x154C9860, ErrorCode4: 0x335DF592.  If the problem persists, contact Product Support for the guest operating system.  (Virtual machine ID A0CD6CF9-529F-4465-AD32-7A7D60069768)

The problematic VMs are with Windows OS, Windows 2008R2, Windows 2012 R2 and Windows 8.1.

My question is:

What is the cause of this problem? The event logs are not understandable at all.

How should I fix this problem?

Hi,

I am trying to involve someone familiar with this topic to further look at this issue. There might be some time delay. Appreciate your patience.

Thanks for your understanding and support.

Hi,

Please try the following command to enable automount feature and check if it works:

diskpart

automount enable

Best Regards,

Vincent Wu

Automount has already enabled in the Hyper-V server. So this should not be the cause. Moreover, this problem does not always happen for the same environment.

Besides, as my observation, Windows 2012 R2 seems to have a different manner from Windows 2012/2008. That is, it won't mount the VM's volume from the virtual disk file to Hyper-V host to do the auto-recovery. Please also help to confirm this.

Thanks.

Hi,

What is the result if we save the vms.

Best Regards,

Vincent Wu

I don't know the result if saving the VMs. Because the problem happened several days ago and we have not witnessed this problems recently.

Since there are already error codes in the event log can help to map these code to error messages? I think these error codes are not for me, the end user, since most of them are not regular Windows error codes. So what kind of these codes are?

Thanks.

Hi,

Please refer to the followig link:

http://technet.microsoft.com/en-us/library/jj614621

Best Regards,

Vincent Wu

Hi,

There is only introduction of the new features of Windows Server Backup in the link http://technet.microsoft.com/en-us/library/jj614621. Can you tell me what I can find related to my problem in the link? Furthermore, I am not using Windows Server Backup to take the VSS snapshot.

Hi,

Please check if there is any vss errors if we list writers and create snapshots. If not, please contact backup software vendor for help.

For your reference:

http://technet.microsoft.com/en-us/library/cc754968.aspx

Best Regards,

Vincent Wu

Hi,

Is there any update?

Best Regards,

Vincent Wu

Hi,

Has there been any movement on this? We are also experiencing random VSS failures for the Hyper VSS Writer on 2012 R2 when DPM takes backups.

Thanks

Andrew

any update on this, i am also experiencing the random VSS failures for the hyper-v VSS writer on 2012 R2 when DPM backups take place, furthermore in my case there maybe a VM on the same host that backs up successfully consistently but another which consistently fails

thanks

+1 here.

Backing up a virtualized Exchange Server 2013 running on Windows Server 2012 R2 crashes and reboots. Using DPM 2012 R2 running on Windows 2012 R2, also virtualized.

I am not backing up the VM... I am doing System state, bare metal, drives, Mailbox database, etc..

Only happens with this Exchange virtualization though (so far)

I have the same issue with Symantec BE 2014 on 2012R2 hyper-V.  I restarted media server and hyperV hosts so many times without luck.  Write Name "Microsoft Hyper-V VSS write" returns with "non-retryable error". 

Are there any errors in Windows events during the time of the backup job?  Make sure you have enough room on your C drive, try freeing up some space and then rerun the backup job.   

Hi

I am having the same issue. Try a backup on just 1 host, Hyper-V VSS writer State 8 failed Non-retryable error. Restart the Hyper v Virtual machine management service and the Hyper-V VSS writer goes back to good state.

Hi Mark,

First , please check if the integration services are running well in each VM .

You can try to use command below to check :

Get-VMIntegrationService -VMName  XXX

Please replace "XXX" to the VM's name .

Also you may try to follow this article to troubleshoot VSS issue:

https://technet.microsoft.com/en-us/library/ee692290%28v=ws.10%29.aspx?f=255&MSPPError=-2147217396

Best Regards,

Elton JI