Hi All,

I'm using SCDPM 2012 R2 RU6  to protect several servers. protected servers are VMs hosted on VMwareand have 3 drives: C:, D. for applications and E: for swap file.

I have configured 1 protection group with volume backup scheduled on 6 pm and BMR backup on 8 pm. Protected Servers are unning Windows Server 2008 R2 and 2012

the issue is that BMR fail on some of the servers. if I replace BMR bya System State backup then it works fine.

on the protected server event log shows that

1/ WSb has failed because of network issues,

2/ Windows Error Reporting event indicate that WSB has crashed

Log Name:      Application
Source:        Microsoft-Windows-Backup
Date:          14/07/2015 21:22:10
Event ID:      517
Task Category: None
Level:         Error
Keywords:      
User:          SYSTEM
Computer:      computer.company.com
Description:
The backup operation that started at '2015-07-15T01:17:56.504790200Z' has failed with following error code '0x8078015B' (Windows Backup encountered an error when accessing the remote shared folder. Please retry the operation after making sure that the remote shared folder is available and accessible.). Please review the event details for a solution, and then rerun the backup operation once the issue is resolved.
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-Backup" Guid="{1DB28F2E-8F80-4027-8C5A-A11F7F10F62D}" />
    <EventID>517</EventID>
    <Version>1</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8000000000000000</Keywords>
    <TimeCreated SystemTime="2015-07-15T01:22:10.992739900Z" />
    <EventRecordID>39313</EventRecordID>
    <Correlation />
    <Execution ProcessID="5944" ThreadID="2704" />
    <Channel>Application</Channel>
    <Computer>computer.company.com</Computer>
    <Security UserID="S-1-5-18" />
  </System>
  <EventData>
    <Data Name="BackupTime">2015-07-15T01:17:56.504790200Z</Data>
    <Data Name="ErrorCode">0x8078015b</Data>
    <Data Name="ErrorMessage">%%2155348315</Data>
  </EventData>
</Event>

This e-mail, including attachments, is intended for the person(s) or company named and may contain confidential and/or legally privileged information.
Unauthorized disclosure, copying or use of this information may be unlawful and is prohibited. If you are not the intended recipient, please delete this message and notify the sender.
All incoming and outgoing e-mail messages are stored in the Swiss Re Electronic Message Repository.
If you do not wish the retention of potentially private e-mails by Swiss Re, we strongly advise you not to use the Swiss Re e-mail account for any private, non-business related communications.

Log Name:      Application
Source:        Windows Error Reporting
Date:          14/07/2015 21:22:11
Event ID:      1001
Task Category: None
Level:         Information
Keywords:      Classic
User:          N/A
Computer:      computer.company.com
Description:
Fault bucket , type 0
Event Name: Windows Server Backup Error
Response: Not available
Cab Id: 0

Problem signature:
P1: 4
P2: 1
P3: 0x8078015b
P4: 0x8007003b
P5: 0x8078015b
P6: 0x00000000
P7: 0x8078015b
P8: 0x00000000
P9:
P10:

Attached files:

These files may be available here:
C:\ProgramData\Microsoft\Windows\WER\ReportQueue\NonCritical_4_409ec6bd2146c21ada927c7f5969cbe324d2b3_21acb088

Analysis symbol:
Rechecking for solution: 0
Report Id: ea201e75-2a8f-11e5-9409-0050568640ee
Report Status: 4
Hashed bucket:
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Windows Error Reporting" />
    <EventID Qualifiers="0">1001</EventID>
    <Level>4</Level>
    <Task>0</Task>
    <Keywords>0x80000000000000</Keywords>
    <TimeCreated SystemTime="2015-07-15T01:22:11.000000000Z" />
    <EventRecordID>39312</EventRecordID>
    <Channel>Application</Channel>
    <Computer>computer.company.com</Computer>
    <Security />
  </System>
  <EventData>
    <Data>
    </Data>
    <Data>0</Data>
    <Data>Windows Server Backup Error</Data>
    <Data>Not available</Data>
    <Data>0</Data>
    <Data>4</Data>
    <Data>1</Data>
    <Data>0x8078015b</Data>
    <Data>0x8007003b</Data>
    <Data>0x8078015b</Data>
    <Data>0x00000000</Data>
    <Data>0x8078015b</Data>
    <Data>0x00000000</Data>
    <Data>
    </Data>
    <Data>
    </Data>
    <Data>
    </Data>
    <Data>C:\ProgramData\Microsoft\Windows\WER\ReportQueue\NonCritical_4_409ec6bd2146c21ada927c7f5969cbe324d2b3_21acb088</Data>
    <Data>
    </Data>
    <Data>0</Data>
    <Data>ea201e75-2a8f-11e5-9409-0050568640ee</Data>
    <Data>4</Data>
    <Data>
    </Data>
  </EventData>
</Event>

This e-mail, including attachments, is intended for the person(s) or company named and may contain confidential and/or legally privileged information.
Unauthorized disclosure, copying or use of this information may be unlawful and is prohibited. If you are not the intended recipient, please delete this message and notify the sender.
All incoming and outgoing e-mail messages are stored in the Swiss Re Electronic Message Repository.
If you do not wish the retention of potentially private e-mails by Swiss Re, we strongly advise you not to use the Swiss Re e-mail account for any private, non-business related communications.

Hi,

The main difference between SystemState (SS) backup that works and BMR backup that fails is the data transfer methods are different.

SystemState - DPMRA on the protected server initiates a system state backup using Windows Server Backup (WSB) to write the backup to C:\WindowsImageBackup.  Once WSB reports a successful SS backup, DPMRA copies the c:\Windowsimagebackup contents to the DPM replica volume.

BMR - DPMRA on the protected server initiates a BMR using Windows Server Backup (WSB) to write the BMR backup directly to the DPM Server replica volume via a share. 

The errorcode '0x8078015B' (Windows Backup encountered an error when accessing the remote shared folder) is showing that WSB had trouble, either network communications/ permissions (look at anti-virus) or some kind of disk IO problem.

You should start by looking at network configuration / connectivity / Anti-virus and maybe take a network trace while WSB BMR backup is in progress to see where the IO failure is occuring.

Hi Mike,

Thanks for your answer.

Here are some information that may clarify the issue:

• The issue has been observed on Windows server 2012 only so far. No 2008 R2 are concerned
• some server used to have successfull backup thaen started to fail every day (not a permission or configuration issue)
• failing server fail even if the backup job is trigered manually from DPM server (seems not to be an IO issue)
• For a given server the BMR backup will almost always fail after transferring the same amount of data. Some servers fail after less than 1 GB some others fails after few GB (no permission issue)
• The AV on the DPM server is already configured to exclude the root folder containing mounting points

In all cases this would not explain why WSB crashes ? it should logh an error but not crash !

Regards

Hi,

Please try running a manual BMR backup on the effected protected server (PS).

1) Set up a network share on a remote machine that has lots of free space: \\server\bmrshare

2) From an administrative command prompt on the PS, type:

              wbadmin.exe start backup -allcritical -backuptarget:\\server\bmrshare

This should show you the list of volumes included in the BMR backup and ask "Do you want to start the backup operation?. - Type Y to continue..

3) Observer if the backup is successful or also crashes.  Under the \\server\BMRshare share check the size of the files under the WindowsImageBackup. 

Hi Mike,

the backup is doing swell so far: 6 GB backed up and still going. Previously it does not go beyond 366 MB.

if it succeed what can be concluded ?

Regards

Hi,

Next step will be to perform a BMR backup to the DPM replica volume via the BMR share using SYSTEM account context. 

First - On the DPM Server get the path to the replica volume for the BMR backup for the same protected server by clicking on the "path to replica" while viewing data sources under the protection group.  Then type "NET SHARE" from an administrative command prompt.  The BMR share will just be a GUID but match it with the path from the above.

Next - in the protected server perform the following:

BMR volume on the DPM server is really locked down from a security standpoint, however DPMRA runs in system context so you need to do the same for this test.

1)      Download psexec.exe from www.sysinternals.com
2)      Run PSEXEC -s cmd.exe   (this will switch window to system context) 
3)      Afterwards - type WHOAMI it should return:

                  nt authority\system

4)  NET VIEW \\DMB_SERVER   - this should list the shares and match the share names seen on the DPM Server.

5) Run the BMR backup manually:

    C:\>wbadmin start backup -allcritical -backuptarget:\\DPM_SERVER\BMR_SHARE_GUID

6)  See if that has troubles compl

Hi Mike,

I picked up the share name from previously failing backup, Backup location field.

The backup has started hanged s short period at the usual value, 366 MB then continued.

Now it succeeded ?

Regards

Hi,

OK - now start a new DPM BMR backup for the same protected server.  There is no difference between what you just did with the manual backup using system account and what DPMRA executes during a scheduled backup.  DPM is simply acting as a scheduler and is depository for the same wbadmin.exe commands you ran manually.   

Hi,

I started a consistency check that has passed the critical ammount of transfer. will keep you informed.

if the current one succeed, how could you explain that more than 22 consecutive backup failed while this current one succeed ?

Regards

Hi,

The consistency check worked well. But the planned backup failed !

Regards

Hi,

I tried the same procedure with another failing server:

• the backup to a share using my account worked well
• using system account exhibited the same behaviour: fail after transferring 17 GB

Any idea ?

Hi

I confirm that every test using my account works whereas uding System Account fail !

Hi,

Are you using the same share for both tests ?  IE:  User backup up to \\server\share and System account backup to same \\server\share ?

Hi,

Definitely no: for my account it is \\server\testbackup and for system account it is the GUID like share that you mentionned in your previous post.