An unexpected error occurred while the job was running. (ID 104 Details: Access is denied (0x80070005))

We have a two node failover hyperV cluster running with CSV. We can backup hyperv servers on one server, but not on the other. Network is working fine, and we can backup normal files from both servers. SCDPM server is W2008R2 with SCDPM 2012R2 UR6, both clients are 4.2.1312.0. A virtual server can be backuped on the first server, then we live migrate it to the other server. The same virtual server then fails to backup.

We get this message in the SCDPM console:

Recovery point creation jobs for Microsoft Hyper-V \Offline\server on server.cluster.domain.com have been failing. The number of failed recovery point creation jobs = 1.
 If the data source protected has some dependent data sources (like a SharePoint Farm), then click on the Error Details to view the list of dependent data sources for which recovery point creation failed. (ID 3114)

An unexpected error occurred while the job was running. (ID 104 Details: Access is denied (0x80070005))

And at the same time, we get in the system eventlog the following DCOM event 10006 message:

DCOM got error "2147942405" from the computer xxxxx.domain.com when attempting to activate the server:
{DA6AA17A-D61C-4E9C-8CEA-DB25DEA52A95}

We backup only to disk. Normal backups of other servers are just working fine. The is no sharepoint farm on the server.

I tried changing the MSDTC rights (http://www.eventid.net/display-eventid-10006-source-DCOM-eventno-272-phase-1.htm), but that didn't solve it.

http://serverfault.com/questions/563088/dpm2010-windows-2012-dcom-errors-communicating-with-all-agents suggests to remove the node from the domain, and back again. This is not something I want to do because it might break the hyperv cluster, and I don't know how to fix that.

Do you have the same problem ? What actions did you do to solve this ?

June 4th, 2015 8:50am

Hi,

Please use the following blog to check DCOM permissions for the DPM RA Service using dcomcnfg.exe

 http://blogs.technet.com/b/dpmsupport/archive/2009/05/19/troubleshooting-agent-deployment-in-data-protection-manager-2007-dcom.aspx

Free Windows Admin Tool Kit Click here and download it now
June 4th, 2015 3:43pm

Thanks Mike for your idea. Unfortunately it did not locate the mistake. I checked all settings, and all settings  looked like described in the document. I can backup from both servers, but only files, no hyperv server in the cluster (through the cluster name on the scdpm server), if it resides on the broken server. I guess scdpm client is working fine. There must be some blocked access to the hyperv layer from the scdpm client. Some VSS access problem.

I looked through \program files\*dpm*\temp\dpmracurr.errlog on the clients, but I don't find the error (80070005) in this file. The only line I found which contained some kind of error:

1B44 14C8 06/05 11:00:03.350 03 miscellaneousutils.cpp(1042)   NORMAL Error:ERROR_UNKNOWN_PRODUCT, While detecting DPM version. Assuming DPM isnt installed.

Well DPM is installed, and I see the same error on the other working DPM client. False positive I guess.

Because I have two servers (one working, the other not), I compared the settings of the security of de DCOM service "DPM RA Service" / security / launch and activate". They are different, but effectively equal. On the working server, all 3 servers are named (scdpm server, server1 and server2), with their rights. On the broken server, only the two scdpm client computer (server1 & 2) are named, not the scdpm server computer. Instead two local groups are included, in which all 3 servers are named. Local group DPMRADCOMTrustedMachines containing the SCDPM server. Local group DPMRATrustedDPMRAs containing both scdpm client computers. It is strange that the permissions are so inconsistent between the servers, but effectively it is equal. (I did not changed it by hand, before the problem occurred. This is all done by software, automatically, unwater, hidden)

By hand I added the scdpm server in the security group, tried backup, rebooted, tried again, but it didn't change anything. I did not expect it, but I might just work. I removed the extra security line.

I think I might conclude that the problem is on the bad server. Both hyperv servers are W2012R2 DC with all latest patches. The working server is a HP gen9 server with SPP 2014-9, the other HP gen8 with SPP 2015-4.

The good was installed with W2012R2 DC and the hyperv+failover and then SCDPM client, the bad was W2012R2 DC, then scdpm client, After that the hyperv + failover role. I have no extra hardware to prove this is a bug and causing this error.

The location of the hyperv-cluster-ip address has no influence on the backup success or failure.

I have no clues left. In a month time, I will have an other server to add to the cluster. I hope this new one will backup fine, so I can reinstall this "bad" server, and work around this problem.


  • Edited by THofkamp 12 hours 41 minutes ago
June 5th, 2015 7:10am

Thanks Mike for your idea. Unfortunately it did not locate the mistake. I checked all settings, and all settings  looked like described in the document. I can backup from both servers, but only files, no hyperv server in the cluster (through the cluster name on the scdpm server), if it resides on the broken server. I guess scdpm client is working fine. There must be some blocked access to the hyperv layer from the scdpm client. Some VSS access problem.

I looked through \program files\*dpm*\temp\dpmracurr.errlog on the clients, but I don't find the error (80070005) in this file. The only line I found which contained some kind of error:

1B44 14C8 06/05 11:00:03.350 03 miscellaneousutils.cpp(1042)   NORMAL Error:ERROR_UNKNOWN_PRODUCT, While detecting DPM version. Assuming DPM isnt installed.

Well DPM is installed, and I see the same error on the other working DPM client. False positive I guess.

Because I have two servers (one working, the other not), I compared the settings of the security of de DCOM service "DPM RA Service" / security / launch and activate". They are different, but effectively equal. On the working server, all 3 servers are named (scdpm server, server1 and server2), with their rights. On the broken server, only the two scdpm client computer (server1 & 2) are named, not the scdpm server computer. Instead two local groups are included, in which all 3 servers are named. Local group DPMRADCOMTrustedMachines containing the SCDPM server. Local group DPMRATrustedDPMRAs containing both scdpm client computers. It is strange that the permissions are so inconsistent between the servers, but effectively it is equal. (I did not changed it by hand, before the problem occurred. This is all done by software, automatically, unwater, hidden)

By hand I added the scdpm server in the security group, tried backup, rebooted, tried again, but it didn't change anything. I did not expect it, but I might just work. I removed the extra security line.

I think I might conclude that the problem is on the bad server. Both hyperv servers are W2012R2 DC with all latest patches. The working server is a HP gen9 server with SPP 2014-9, the other HP gen8 with SPP 2015-4.

The good was installed with W2012R2 DC and the hyperv+failover and then SCDPM client, the bad was W2012R2 DC, then scdpm client, After that the hyperv + failover role. I have no extra hardware to prove this is a bug and causing this error.

The location of the hyperv-cluster-ip address has no influence on the backup success or failure.

I have no clues left. In a month time, I will have an other server to add to the cluster. I hope this new one will backup fine, so I can reinstall this "bad" server, and work around this problem.


  • Edited by THofkamp Friday, June 05, 2015 7:00 PM
Free Windows Admin Tool Kit Click here and download it now
June 5th, 2015 11:08am

Thanks Mike for your idea. Unfortunately it did not locate the mistake. I checked all settings, and all settings  looked like described in the document. I can backup from both servers, but only files, no hyperv server in the cluster (through the cluster name on the scdpm server), if it resides on the broken server. I guess scdpm client is working fine. There must be some blocked access to the hyperv layer from the scdpm client. Some VSS access problem.

I looked through \program files\*dpm*\temp\dpmracurr.errlog on the clients, but I don't find the error (80070005) in this file. The only line I found which contained some kind of error:

1B44 14C8 06/05 11:00:03.350 03 miscellaneousutils.cpp(1042)   NORMAL Error:ERROR_UNKNOWN_PRODUCT, While detecting DPM version. Assuming DPM isnt installed.

Well DPM is installed, and I see the same error on the other working DPM client. False positive I guess.

Because I have two servers (one working, the other not), I compared the settings of the security of de DCOM service "DPM RA Service" / security / launch and activate". They are different, but effectively equal. On the working server, all 3 servers are named (scdpm server, server1 and server2), with their rights. On the broken server, only the two scdpm client computer (server1 & 2) are named, not the scdpm server computer. Instead two local groups are included, in which all 3 servers are named. Local group DPMRADCOMTrustedMachines containing the SCDPM server. Local group DPMRATrustedDPMRAs containing both scdpm client computers. It is strange that the permissions are so inconsistent between the servers, but effectively it is equal. (I did not changed it by hand, before the problem occurred. This is all done by software, automatically, unwater, hidden)

By hand I added the scdpm server in the security group, tried backup, rebooted, tried again, but it didn't change anything. I did not expect it, but I might just work. I removed the extra security line.

I think I might conclude that the problem is on the bad server. Both hyperv servers are W2012R2 DC with all latest patches. The working server is a HP gen9 server with SPP 2014-9, the other HP gen8 with SPP 2015-4.

The good was installed with W2012R2 DC and the hyperv+failover and then SCDPM client, the bad was W2012R2 DC, then scdpm client, After that the hyperv + failover role. I have no extra hardware to prove this is a bug and causing this error.

The location of the hyperv-cluster-ip address has no influence on the backup success or failure.

I have no clues left. In a month time, I will have an other server to add to the cluster. I hope this new one will backup fine, so I can reinstall this "bad" server, and work around this problem.


  • Edited by THofkamp Friday, June 05, 2015 7:00 PM
June 5th, 2015 11:08am

I have done some more diagnostic. Any other installation sequence does not affect the working.

I have checked the security setting of DCOM on all hyperv servers, which looked ok. (http://blogs.technet.com/b/dpmsupport/archive/2009/05/19/troubleshooting-agent-deployment-in-data-protection-manager-2007-dcom.aspx)  This link is for non clustered members, where is the link to clustered member servers ? I asume this will be the same documentation.

The only problem with this is, that the "remote" computer in the eventlog of the dcom error event (10006) is from an other hyperv server !  Eg.  The working hyperv server, tries to start a SCDPM DCOM service on the other hyperv server ! Why does this hyperv server tries to start the SCDPM client ???

Our scdpm-server should do this, not a hyperv server. The strange thing is, that the server IS authorised to do this, but still there is an error. (computer accounts are in the the local security group)Log Name:      System
Source:        Microsoft-Windows-DistributedCOM
Date:          6-7-2015 10:34:29
Event ID:      10006
Task Category: None
Level:         Error
Keywords:      Classic
User:          SYSTEM
Computer:      hyperv2.domain.com
Description:
DCOM got error "2147942405" from the computer hyperv1.domain.com when attempting to activate the server:
{DA6AA17A-D61C-4E9C-8CEA-DB25DEA52A95}
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-DistributedCOM" Guid="{1B562E86-B7AA-4131-BADC-B6F3A001407E}" EventSourceName="DCOM" />
    <EventID Qualifiers="0">10006</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8080000000000000</Keywords>
    <TimeCreated SystemTime="2015-07-06T08:34:29.360864400Z" />
    <EventRecordID>4278</EventRecordID>
    <Correlation />
    <Execution ProcessID="892" ThreadID="4320" />
    <Channel>System</Channel>
    <Computer>hyperv2.domain.com</Computer>
    <Security UserID="S-1-5-18" />
  </System>
  <EventData>
    <Data Name="param1">2147942405</Data>
    <Data Name="param2">hyperv1.domain.com</Data>
    <Data Name="param3">{DA6AA17A-D61C-4E9C-8CEA-DB25DEA52A95}</Data>
  </EventData>
</Event>

In the documentation it looked like it is working ok. What do I have to do to make this work ? Why don't I see many more people with this error ?


  • Edited by THofkamp 21 hours 0 minutes ago
Free Windows Admin Tool Kit Click here and download it now
July 6th, 2015 6:44am

I have done some more diagnostic. Any other installation sequence does not affect the working.

I have checked the security setting of DCOM on all hyperv servers, which looked ok. (http://blogs.technet.com/b/dpmsupport/archive/2009/05/19/troubleshooting-agent-deployment-in-data-protection-manager-2007-dcom.aspx)  This link is for non clustered members, where is the link to clustered member servers ? I asume this will be the same documentation.

The only problem with this is, that the "remote" computer in the eventlog of the dcom error event (10006) is from an other hyperv server !  Eg.  The working hyperv server, tries to start a SCDPM DCOM service on the other hyperv server ! Why does this hyperv server tries to start the SCDPM client ???

Our scdpm-server should do this, not a hyperv server. The strange thing is, that the server IS authorised to do this, but still there is an error. (computer accounts are in the the local security group)Log Name:      System
Source:        Microsoft-Windows-DistributedCOM
Date:          6-7-2015 10:34:29
Event ID:      10006
Task Category: None
Level:         Error
Keywords:      Classic
User:          SYSTEM
Computer:      hyperv2.domain.com
Description:
DCOM got error "2147942405" from the computer hyperv1.domain.com when attempting to activate the server:
{DA6AA17A-D61C-4E9C-8CEA-DB25DEA52A95}
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-DistributedCOM" Guid="{1B562E86-B7AA-4131-BADC-B6F3A001407E}" EventSourceName="DCOM" />
    <EventID Qualifiers="0">10006</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8080000000000000</Keywords>
    <TimeCreated SystemTime="2015-07-06T08:34:29.360864400Z" />
    <EventRecordID>4278</EventRecordID>
    <Correlation />
    <Execution ProcessID="892" ThreadID="4320" />
    <Channel>System</Channel>
    <Computer>hyperv2.domain.com</Computer>
    <Security UserID="S-1-5-18" />
  </System>
  <EventData>
    <Data Name="param1">2147942405</Data>
    <Data Name="param2">hyperv1.domain.com</Data>
    <Data Name="param3">{DA6AA17A-D61C-4E9C-8CEA-DB25DEA52A95}</Data>
  </EventData>
</Event>

In the documentation it looked like it is working ok. What do I have to do to make this work ? Why don't I see many more people with this error ?


  • Edited by THofkamp Monday, July 06, 2015 10:39 AM
July 6th, 2015 10:38am

I have done some more diagnostic. Any other installation sequence does not affect the working.

I have checked the security setting of DCOM on all hyperv servers, which looked ok. (http://blogs.technet.com/b/dpmsupport/archive/2009/05/19/troubleshooting-agent-deployment-in-data-protection-manager-2007-dcom.aspx)  This link is for non clustered members, where is the link to clustered member servers ? I asume this will be the same documentation.

The only problem with this is, that the "remote" computer in the eventlog of the dcom error event (10006) is from an other hyperv server !  Eg.  The working hyperv server, tries to start a SCDPM DCOM service on the other hyperv server ! Why does this hyperv server tries to start the SCDPM client ???

Our scdpm-server should do this, not a hyperv server. The strange thing is, that the server IS authorised to do this, but still there is an error. (computer accounts are in the the local security group)Log Name:      System
Source:        Microsoft-Windows-DistributedCOM
Date:          6-7-2015 10:34:29
Event ID:      10006
Task Category: None
Level:         Error
Keywords:      Classic
User:          SYSTEM
Computer:      hyperv2.domain.com
Description:
DCOM got error "2147942405" from the computer hyperv1.domain.com when attempting to activate the server:
{DA6AA17A-D61C-4E9C-8CEA-DB25DEA52A95}
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
  <System>
    <Provider Name="Microsoft-Windows-DistributedCOM" Guid="{1B562E86-B7AA-4131-BADC-B6F3A001407E}" EventSourceName="DCOM" />
    <EventID Qualifiers="0">10006</EventID>
    <Version>0</Version>
    <Level>2</Level>
    <Task>0</Task>
    <Opcode>0</Opcode>
    <Keywords>0x8080000000000000</Keywords>
    <TimeCreated SystemTime="2015-07-06T08:34:29.360864400Z" />
    <EventRecordID>4278</EventRecordID>
    <Correlation />
    <Execution ProcessID="892" ThreadID="4320" />
    <Channel>System</Channel>
    <Computer>hyperv2.domain.com</Computer>
    <Security UserID="S-1-5-18" />
  </System>
  <EventData>
    <Data Name="param1">2147942405</Data>
    <Data Name="param2">hyperv1.domain.com</Data>
    <Data Name="param3">{DA6AA17A-D61C-4E9C-8CEA-DB25DEA52A95}</Data>
  </EventData>
</Event>

In the documentation it looked like it is working ok. What do I have to do to make this work ? Why don't I see many more people with this error ?


  • Edited by THofkamp Monday, July 06, 2015 10:39 AM
Free Windows Admin Tool Kit Click here and download it now
July 6th, 2015 10:38am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics