Hi,

we have Windows Server 2012 standard, and SC DPM 2012 SP1 with latest updates.

When on Hyper-V host, we have several Linux / CentOS 6.3 or Ubuntu Server 12, with latest integration tools installed, virtualization is running fine.
We do not have failover clustor or shared storage.

However, when DPM agent, starts to backing up hyper-v host, all virtual machines (including windows servers on same host) stops responding, and virtual machines gets no access to disk IO

On Ubuntu console message is:

Read-error on swap-device (252:0:319496)

sd 2:0:0:0 > rejecting I/O to offline device

after backup of all virtual machines is finished everything again works.

Some things happen if you do not use DPM at all and use native Windows Server Backup ..

Also, while machine state is "Backing up.." , you can not modify and/or work with any virtual machine on this host (can not change network or any settings, shutdown virtual machines etc.) , however virtual machines are up and running inside ram and network (ping) is working, but they do not have disk access... . Also, on Hyper-V host, it seems that access to disk (where virtual machines is placed) is not available, and in resource monitor disk usage is at maximum.

On Hyper-V 2.0 / Windows Server 2008 R2 we did not have such a problems...

Problem with this persist on all our windows servers 2012 / hyper-v hosts where there exists virtual machines...

Please help, how to properly backup, including supported linux virtual machines, without downtime.

Kind regards

Hi Hrvoje,

DPM uses VSS to take the backups. If you backup a Non-Vss aware OS using DPM - We put it under saved state/turn it off and then back it up.

See this

http://social.technet.microsoft.com/Forums/en-US/dataprotectionmanager/thread/bc6a96d0-5a1f-4093-b84d-6d960afecf72/

Regards,

Siddharth Jha

I understand that there is no DPM Agent inside Linux, and that DPM Agent on Hyper-V host, must save state, backup and then restore.

However, during backup, I can not work with any other virtual machine in mean of change any settings or power on/off.., also, all of them (including virtual windows servers) does not have access to its virtual disk... :/

maybe problem is that VSS takes all available disk IO (in my case about 150MB/s constantly), and there is no IO capacity for other hosts.., is there a way for limiting disk IO ?

Thank you

Hi,

A couple of possible workarounds.

1) Use more CSV disks and put fewer guests per CSV.
2) Install and configure a VSS hardware provider for the SAN to offload snapshots from Windows.

Hi,

I do not have CSV, and I do not have Hyper-V cluster. All servers are stand alone with their local internal storage (RAID mirror with battery inside IBM server, there are 2x SAS 15k rpm drives) and total of 10 VMs on one physical server with about 128 GB RAM and two socket CPU. The load on virtual macines is not hight and there is no such a number of disk write IO operations...

Anyway is there a way to determine exactly where the problem is ? and / or to configure maximum usage of disk IO per virtual machine ?

thank you

Hi,

You would need to use performance monitor to see what the disk performance looks like.

You can try adding another disk - then configure VSS to use that other disk for shadow copies.

example:  Hyper-V guests reside on volume G:  - New disk added and new volume H: is created.

Open administrative command prompt and type:

C:\>vssadmin add shadowstorage /for=g: /on=h: /maxsize=unbounded 

This will take some write workload off the G: volume and maintain the snapshot on H:

Ok, thank you, I understand this new hardware (hdd) will take some write loads..., however, is there some software solution for limiting disk IO ?

I think we should try to duplicate this using diskshadow.exe to create a persistent snapshot and manually copy a VM.

1) On the hyper-V host open an administrative command prompt.
2) Run diskshadow.exe - and after it starts enter the following commands as written except for the drive letters used:

   Diskshadow> Set Verbose on
   Diskshadow> Set context Persistent
   Diskshadow> Add Volume X: ALIAS xvol    (Where X: is the drive letter of the volume containing the Hyper-V guests you want to make a snapshot for)
   Diskshadow> Begin backup
   Diskshadow> create                   
   Diskshadow>end backup
   DiskShadow> Expose %xvol% S:              (Where S: is a free drive letter)
  
3) Now Go into explorer and copy out VM's from the S: Drive which is the live shadow copy and see if that duplicates your problem.

When you are done, delete the shadow copy in diskshadow using the following command.
   
   
   DiskShadow>Delete shadows exposed S:  

In our environment were backing up "non-native machines" (a couple of Ubuntu VMs, actually) that reside on Windows 2012 host using free tool called VeeamZIP.

It allows us to perform either so-called offline backup (with a little downtime required for quiescing VMs prior to taking a volume snapshot) or crash-consistent one (without any downtime; since these VMs dont run any specific application, in overwhelming majority of cases crash-consistency is more than enough for us).

Kind regards, Leonardo.

• Edited by Leonardo Muller Monday, May 13, 2013 2:53 PM

In our environment were backing up "non-native machines" (a couple of Ubuntu VMs, actually) that reside on Windows 2012 host using free tool called VeeamZIP.

It allows us to perform either so-called offline backup (with a little downtime required for quiescing VMs prior to taking a volume snapshot) or crash-consistent one (without any downtime; since these VMs dont run any specific application, in overwhelming majority of cases crash-consistency is more than enough for us).

Kind regards, Leonardo.

• Edited by Leonardo Muller Monday, May 13, 2013 2:53 PM

hey did you disable backup under integration services in the vm configuration? this is not supported with non windows vm's.

also check out centos 6.4. It's a major improvement over 6.3 in terms of reliability with hyper-v.

stefan

• Edited by Stefan SP Saturday, May 18, 2013 7:56 PM

hey did you disable backup under integration services in the vm configuration? this is not supported with non windows vm's.

also check out centos 6.4. It's a major improvement over 6.3 in terms of reliability with hyper-v.

stefan

• Edited by Stefan SP Saturday, May 18, 2013 7:56 PM

Hi,

I have CentOS 6.3 (which is supported), I will plan upgrade to CentOS 6.4. I have disabled backup under integration services (and I understand this is not supported).

I have disabled backup under integration services in Hyper-V console. And issue is gone.