DPM 2012 SP1 and Hyper-V 2012 Large System Volume information Folder

Over the past week we have seen a problem where the System Volume Information folder has been growing at the rate of 20GB - 30Gb a days and there are now 670GB of files in the folder. Thjis is not related to the DPM Filter Bitmap files of which there are a lot but the volume is small compared to the remaining 630 files sized between 700MB and 11GB in size.

our setup is as follows:

2 x Node Active Active Server 2012 cluster with a single 5TB CSV volume mounted for Hyper-V images. We have 75 images running accross the two nodes and these are backed up using DPM 2012 SP1. Within DPM the VM's are split into four protection groups with differnet protection and retention setups.

I have checked the Shadop copy for the volume and this is disabled, checking on DiskShadow CLI using List Shadows All it returns 'No Shadow Copies found in Syatem'

DPM has been performing well with not a single backup failure over the last three weeks but all of a sudden on the 7th August (date of the oldes file in System Volume) we are retaining huge amounts of data.

Checking the event log I can see the following errors:

we get the following 7-8 times per day

Volume Shadow Copy Service warning: Writer received a Freeze event more than two minutes ago. The writer is still waiting for either an Abort or a Thaw event.

Operation:
   Gathering Writer Data

Context:
   Writer Class Id: {66841cd4-6ded-4f4b-8f17-fd23f8ddc3de}

   Writer Name: Microsoft Hyper-V VSS Writer
   Writer Instance ID: {7e0fa2fb-7392-485c-bb90-966d0198bd10}

We get these errors about 7 - 10 times per day:

VDS Basic Provider

Unexpected failure. Error code: 48F@01000003

I am really stumped by this so any help greatly appreaciated.

Thanks,

John

August 15th, 2013 2:19pm

Hi,

Are the offending file names in the form of {GUID}-{GUID} ?

There is a known code defect in Windows 2012 where VSS snapshots marked for deletion are not deleted on CSV volumes.  This is going to be addressed in a upcoming Windows 2012 cluster update. 

Free Windows Admin Tool Kit Click here and download it now
August 15th, 2013 3:35pm

Hi,

that worked great, bit difficult assigning the correct level of security to enable the {guid}{guid} files to be deleted.

Thanks for the quick response

Cheers,

John

August 15th, 2013 4:40pm

Hi,

Are there any further details around this fix? Or a related KB describing the issue in detail? We are experiencing exactly this issue and is has become quite problematic; we have a CSV totally packed with old VSS snapshots that we are unable to get rid of.

Cheers

Free Windows Admin Tool Kit Click here and download it now
November 7th, 2013 3:39am

Hi,

The windows group has not released a fix for this issue yet. Please open a windows support case so they can help you delete them.

UPDATE 1-2-14 - a fix is now available - see announcement here: http://social.technet.microsoft.com/Forums/en-US/home?forum=dpmhypervbackup&announcementId=6a85772e-cc77-4c46-ba3b-bab89ca95db1

November 7th, 2013 5:12am

This is not a fix, its a statement, so to fix this issue can we remove the older dpmfilterbitmap files?
Free Windows Admin Tool Kit Click here and download it now
November 26th, 2013 5:20pm

Hi William See

<snip>
This is not a fix, its a statement, so to fix this issue can we remove the older dpmfilterbitmap files?
>snip<

The Windows VSS snapshots are not the same as DPM bitmap files.  Windows has not yet released a fix for snapshots that cannot be removed after deletion.

However - DPM 2012 Sp1 has a fix for large DPM bitmap files if you see that as being a problem. See http://www.microsoft.com/en-us/download/details.aspx?id=40318

November 26th, 2013 6:20pm

Hi,

is there a release Date or KB number available? The Microsoft Support could not help us on the phone.

Best regards

Stefan

Free Windows Admin Tool Kit Click here and download it now
November 27th, 2013 1:16pm

Is there already a fix?

If so: What is the KB number?

January 2nd, 2014 12:04pm

HI,

Yes, the fix was released by the Windows group that will address the snapshot cleanup issues.

2878635 - Update is available that improves the resiliency of the cloud service provider in Windows Server 2012: December 2013
http://support.microsoft.com/kb/2878635

Announcement updated here http://social.technet.microsoft.com/Forums/en-US/home?forum=dpmhypervbackup&announcementId=6a85772e-cc77-4c46-ba3b-bab89ca95db1

Free Windows Admin Tool Kit Click here and download it now
January 2nd, 2014 7:58pm

Hello Mike.

Can I delete files with names in the form of {GUID}-{GUID} too ? In the way like http://blogs.msdn.com/b/george_bethanis/archive/2014/09/09/dpm-2012-sp1-ru2-huge-system-volume-information-folder.aspx

September 29th, 2014 9:57am

Hi,

I am facing the similar issue with a Win server 2012 R2 Cluster. I am using Software VSS provider on all the nodes. I am protecting this cluster using DPM 2012 R2 with update rollup 5 installed.

Any suggestions would be appreciated.


Free Windows Admin Tool Kit Click here and download it now
March 12th, 2015 4:13am

Hi,

I am facing the similar issue with a Win server 2012 R2 Cluster. I am using Software VSS provider on all the nodes. I am protecting this cluster using DPM 2012 R2 with update rollup 5 installed.

Any suggestions would be appreciated.


March 12th, 2015 8:12am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics