Hello All, I'm observing a strange problem in an AD 2008 R2 / Exchange 2010 SP2 environment: When creating a DAG and adding 1 or more servers to the DAG, the following error occurs: Summary: 2 item(s). 0 succeeded, 2 failed. Elapsed time: 00:00:05 <MAILBOX SERVER 1> Failed Error: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)) Exchange Management Shell command attempted: Add-DatabaseAvailabilityGroupServer -MailboxServer '<MAILBOX SERVER 1>' -Identity '<NAME DAG>' Elapsed Time: 00:00:02 <MAILBOX SERVER 2> Failed Error: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)) Exchange Management Shell command attempted: Add-DatabaseAvailabilityGroupServer -MailboxServer '<MAILBOX SERVER 2>' -Identity '<NAME DAG>' Elapsed Time: 00:00:02 There are no logs created on the Mailservers, so I have no more detailed information. Where to start with troubleshooting this issue? Edit: BTW I already checked the local admin membership of the "Exchange Trusted Subsystem" domain group. You know you're an engineer when you have no life and can prove it mathematically

Hi, Did you start EMC with elevated privileges (Run as Administrator)? If not, then that could explain the error you get.Martina Miskovic

Hi, Did you start EMC with elevated privileges (Run as Administrator)? If not, then that could explain the error you get.Martina Miskovic

Hi Martina, I tried both with EMS. The EMC is always opened with UAC prompt. So the answer is: Yes, I runned the command with elevated privileges. Regards, StephanYou know you're an engineer when you have no life and can prove it mathematically

Ok. Please check that "Exchange Trusted Subsystem" is a member of the local Administrator Group. Martina Miskovic

Hi Martina, I tried both with EMS. The EMC is always opened with UAC prompt. So the answer is: Yes, I runned the command with elevated privileges. Regards, StephanYou know you're an engineer when you have no life and can prove it mathematically

Ok. Please check that "Exchange Trusted Subsystem" is a member of the local Administrator Group. Martina Miskovic

Hi Martina, As stated in my first post: This is the case.You know you're an engineer when you have no life and can prove it mathematically

Hi Martina, As stated in my first post: This is the case.You know you're an engineer when you have no life and can prove it mathematically

Hi Stephan, How many Exchange servers do you have? Two-member or Four-member DAG ? In a single site or cross site? Please make sure you meet the requirements for the deployment first(Network Requirements... ) Planning for High Availability and Site Resilience http://technet.microsoft.com/en-us/library/dd638104 Did you create the DAG successfully? Create a Database Availability Group http://technet.microsoft.com/en-us/library/dd351172.aspx Please also run the cmdlet Get-DatabaseAvailabilityGroup DAGname -Status | fl and post the result here. Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.Frank Wang TechNet Community Support

Hi Stephan, How many Exchange servers do you have? Two-member or Four-member DAG ? In a single site or cross site? Please make sure you meet the requirements for the deployment first(Network Requirements... ) Planning for High Availability and Site Resilience http://technet.microsoft.com/en-us/library/dd638104 Did you create the DAG successfully? Create a Database Availability Group http://technet.microsoft.com/en-us/library/dd351172.aspx Please also run the cmdlet Get-DatabaseAvailabilityGroup DAGname -Status | fl and post the result here. Please remember to mark the replies as answers if they help and unmark them if they provide no help. If you have feedback for TechNet Subscriber Support, contact tnmff@microsoft.com.Frank Wang TechNet Community Support

Hi Frank, The DAG is not yet populated. When inserting the first server(s) in the freshly created DAG, the error appeared. [PS] C:\Program Files\Microsoft\Exchange Server\V14\Scripts>Get-DatabaseAvailabilityGroup Name Member Servers Operational Servers ---- -------------- ------------------- IICT-DAG-002 {} [PS] C:\Program Files\Microsoft\Exchange Server\V14\Scripts>Get-DatabaseAvailabilityGroup IICT-DAG-002 -Status | fl RunspaceId : cc985264-fa89-48f8-8aba-c1b0c89eb097 Name : IICT-DAG-002 Servers : {} WitnessServer : iict-srvp00-011.insourceict.local WitnessDirectory : C:\IICT-DAG-002 AlternateWitnessServer : AlternateWitnessDirectory : NetworkCompression : InterSubnetOnly NetworkEncryption : InterSubnetOnly DatacenterActivationMode : Off StoppedMailboxServers : {} StartedMailboxServers : {} DatabaseAvailabilityGroupIpv4Addresses : {10.100.0.54} DatabaseAvailabilityGroupIpAddresses : {10.100.0.54} AllowCrossSiteRpcClientAccess : False OperationalServers : PrimaryActiveManager : ServersInMaintenance : ThirdPartyReplication : Disabled ReplicationPort : 0 NetworkNames : {} WitnessShareInUse : AdminDisplayName : ExchangeVersion : 0.10 (14.0.100.0) DistinguishedName : CN=IICT-DAG-002,CN=Database Availability Groups,CN=Exchange Administrative Gro up (FYDIBOHF23SPDLT),CN=Administrative Groups,CN=InsourceICT,CN=Microsoft Exch ange,CN=Services,CN=Configuration,DC=insourceict,DC=local Identity : IICT-DAG-002 Guid : 71d5d869-03ac-4f8a-8de7-fc15bc6a0ae1 ObjectCategory : insourceict.local/Configuration/Schema/ms-Exch-MDB-Availability-Group ObjectClass : {top, msExchMDBAvailabilityGroup} WhenChanged : 8-6-2012 14:35:59 WhenCreated : 8-6-2012 13:35:21 WhenChangedUTC : 8-6-2012 12:35:59 WhenCreatedUTC : 8-6-2012 11:35:21 OrganizationId : OriginatingServer : IICT-SRV003.insourceict.local IsValid : True You know you're an engineer when you have no life and can prove it mathematically

Hello all, anyone a suggestion to help?You know you're an engineer when you have no life and can prove it mathematically

Hi Stephan, Please run the cmdlet Add-DatabaseAvailabilityGroupServer with parameter -verbose and post the result here. Please run Exbpa to do a "Health Check" and "Permission Check". I guess you had already try to remove and recreate the DAG object? Remove-DatabaseAvailabilityGroup http://technet.microsoft.com/en-us/library/dd335129Frank Wang TechNet Community Support

Hi Frank, Thanks for your reply. I indeed tried several times to remove and recreate the DAG. Here the verbose output: [PS] C:\Windows\system32>Add-DatabaseAvailabilityGroupServer -MailboxServer '<SERVERNAME>' -Identity '<DAGNAME>' -verbose VERBOSE: [11:41:25.169 GMT] Add-DatabaseAvailabilityGroupServer : Initializing Active Directory server settings for the remote Windows PowerShell session. VERBOSE: [11:41:25.169 GMT] Add-DatabaseAvailabilityGroupServer : Active Directory session settings for 'Add-DatabaseAvailabilityGroupServer' are: View Entire Forest: 'False', Default Scope: '<DOMAINNAME>', Configuration Domain Controller: '<DCNAME>.<DOMAINNAME>', Preferred Global Catalog: '<GCNAME>.<DOMAINNAME>', Preferred Domain Controllers: '{ <GCNAME>.<DOMAINNAME> }' VERBOSE: [11:41:25.169 GMT] Add-DatabaseAvailabilityGroupServer : Runspace context: Executing user: <DOMAINNAME>/Organization/Departments/IT/Users/Admins/Stephan van der Plas, Executing user organization: , Current organization: , RBAC-enabled: Enabled. VERBOSE: [11:41:25.169 GMT] Add-DatabaseAvailabilityGroupServer : Beginning processing & VERBOSE: [11:41:25.185 GMT] Add-DatabaseAvailabilityGroupServer : Instantiating handler with index 0 for cmdlet extension agent "Admin Audit Log Agent". VERBOSE: [11:41:25.216 GMT] Add-DatabaseAvailabilityGroupServer : Current ScopeSet is: { Recipient Read Scope: {{, }}, Recipient Write Scopes: {{, }}, Configuration Read Scope: {{, }}, Configuration Write Scope(s): {{, }, }, Exclusive Recipient Scope(s): {}, Exclusive Configuration Scope(s): {} } VERBOSE: [11:41:25.216 GMT] Add-DatabaseAvailabilityGroupServer : Searching objects "<SERVERNAME>" of type "Server" under the root "$null". VERBOSE: [11:41:25.232 GMT] Add-DatabaseAvailabilityGroupServer : Previous operation run on domain controller '<DCNAME>.<DOMAINNAME>'. VERBOSE: [11:41:27.513 GMT] Add-DatabaseAvailabilityGroupServer : Searching objects "<DAGNAME>" of type "DatabaseAvailabilityGroup" under the root "$null". VERBOSE: [11:41:27.513 GMT] Add-DatabaseAvailabilityGroupServer : Previous operation run on domain controller '<DCNAME>.<DOMAINNAME>'. VERBOSE: [11:41:27.529 GMT] Add-DatabaseAvailabilityGroupServer : Processing object "<DAGNAME>". VERBOSE: [11:41:27.529 GMT] Add-DatabaseAvailabilityGroupServer : Admin Audit Log: Entered Handler:Validate. VERBOSE: [11:41:27.529 GMT] Add-DatabaseAvailabilityGroupServer : Admin Audit Log: Entered ClassFactory:InitializeConfig. VERBOSE: [11:41:27.529 GMT] Add-DatabaseAvailabilityGroupServer : Admin Audit Log: Exited ClassFactory:InitializeConfig. VERBOSE: [11:41:27.544 GMT] Add-DatabaseAvailabilityGroupServer : Admin Audit Log: Exited Handler:Validate. VERBOSE: Adding mailbox server "<SERVERNAME>" to database availability group "<DAGNAME>". VERBOSE: [11:41:27.544 GMT] Add-DatabaseAvailabilityGroupServer : Resolved current organization: . VERBOSE: [11:41:27.576 GMT] Add-DatabaseAvailabilityGroupServer : Admin Audit Log: Entered Handler:OnComplete. VERBOSE: [11:41:27.576 GMT] Add-DatabaseAvailabilityGroupServer : Admin Audit Log: Exited Handler:OnComplete. Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)) + CategoryInfo : NotSpecified: (0:Int32) [Add-DatabaseAvailabilityGroupServer], UnauthorizedAccessException + FullyQualifiedErrorId : AA1F65A,Microsoft.Exchange.Management.SystemConfigurationTasks.AddDatabaseAvailabilityGroupServer VERBOSE: [11:41:27.576 GMT] Add-DatabaseAvailabilityGroupServer : Ending processing & Regards, Stephan You know you're an engineer when you have no life and can prove it mathematically

Hi Frank, Thanks for your reply. I indeed tried several times to remove and recreate the DAG. Here the verbose output: [PS] C:\Windows\system32>Add-DatabaseAvailabilityGroupServer -MailboxServer '<SERVERNAME>' -Identity '<DAGNAME>' -verbose VERBOSE: [11:41:25.169 GMT] Add-DatabaseAvailabilityGroupServer : Initializing Active Directory server settings for the remote Windows PowerShell session. VERBOSE: [11:41:25.169 GMT] Add-DatabaseAvailabilityGroupServer : Active Directory session settings for 'Add-DatabaseAvailabilityGroupServer' are: View Entire Forest: 'False', Default Scope: '<DOMAINNAME>', Configuration Domain Controller: '<DCNAME>.<DOMAINNAME>', Preferred Global Catalog: '<GCNAME>.<DOMAINNAME>', Preferred Domain Controllers: '{ <GCNAME>.<DOMAINNAME> }' VERBOSE: [11:41:25.169 GMT] Add-DatabaseAvailabilityGroupServer : Runspace context: Executing user: <DOMAINNAME>/Organization/Departments/IT/Users/Admins/Stephan van der Plas, Executing user organization: , Current organization: , RBAC-enabled: Enabled. VERBOSE: [11:41:25.169 GMT] Add-DatabaseAvailabilityGroupServer : Beginning processing & VERBOSE: [11:41:25.185 GMT] Add-DatabaseAvailabilityGroupServer : Instantiating handler with index 0 for cmdlet extension agent "Admin Audit Log Agent". VERBOSE: [11:41:25.216 GMT] Add-DatabaseAvailabilityGroupServer : Current ScopeSet is: { Recipient Read Scope: {{, }}, Recipient Write Scopes: {{, }}, Configuration Read Scope: {{, }}, Configuration Write Scope(s): {{, }, }, Exclusive Recipient Scope(s): {}, Exclusive Configuration Scope(s): {} } VERBOSE: [11:41:25.216 GMT] Add-DatabaseAvailabilityGroupServer : Searching objects "<SERVERNAME>" of type "Server" under the root "$null". VERBOSE: [11:41:25.232 GMT] Add-DatabaseAvailabilityGroupServer : Previous operation run on domain controller '<DCNAME>.<DOMAINNAME>'. VERBOSE: [11:41:27.513 GMT] Add-DatabaseAvailabilityGroupServer : Searching objects "<DAGNAME>" of type "DatabaseAvailabilityGroup" under the root "$null". VERBOSE: [11:41:27.513 GMT] Add-DatabaseAvailabilityGroupServer : Previous operation run on domain controller '<DCNAME>.<DOMAINNAME>'. VERBOSE: [11:41:27.529 GMT] Add-DatabaseAvailabilityGroupServer : Processing object "<DAGNAME>". VERBOSE: [11:41:27.529 GMT] Add-DatabaseAvailabilityGroupServer : Admin Audit Log: Entered Handler:Validate. VERBOSE: [11:41:27.529 GMT] Add-DatabaseAvailabilityGroupServer : Admin Audit Log: Entered ClassFactory:InitializeConfig. VERBOSE: [11:41:27.529 GMT] Add-DatabaseAvailabilityGroupServer : Admin Audit Log: Exited ClassFactory:InitializeConfig. VERBOSE: [11:41:27.544 GMT] Add-DatabaseAvailabilityGroupServer : Admin Audit Log: Exited Handler:Validate. VERBOSE: Adding mailbox server "<SERVERNAME>" to database availability group "<DAGNAME>". VERBOSE: [11:41:27.544 GMT] Add-DatabaseAvailabilityGroupServer : Resolved current organization: . VERBOSE: [11:41:27.576 GMT] Add-DatabaseAvailabilityGroupServer : Admin Audit Log: Entered Handler:OnComplete. VERBOSE: [11:41:27.576 GMT] Add-DatabaseAvailabilityGroupServer : Admin Audit Log: Exited Handler:OnComplete. Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)) + CategoryInfo : NotSpecified: (0:Int32) [Add-DatabaseAvailabilityGroupServer], UnauthorizedAccessException + FullyQualifiedErrorId : AA1F65A,Microsoft.Exchange.Management.SystemConfigurationTasks.AddDatabaseAvailabilityGroupServer VERBOSE: [11:41:27.576 GMT] Add-DatabaseAvailabilityGroupServer : Ending processing & Regards, Stephan You know you're an engineer when you have no life and can prove it mathematically

Hello all, anyone a suggestion to help?You know you're an engineer when you have no life and can prove it mathematically

Hi Stephan, Maybe this is an useless request, but could you please check whether Microsoft Exchange services are started? Specially for System Attendant service. Everyone is OK in ExBPA? If possible, please create another account which is a member of "Organization Management" Role Group to do the task to test(Logging on to Exchange server using the new account). Frank Wang TechNet Community Support

Hi Stephan, Maybe this is an useless request, but could you please check whether Microsoft Exchange services are started? Specially for System Attendant service. Everyone is OK in ExBPA? If possible, please create another account which is a member of "Organization Management" Role Group to do the task to test(Logging on to Exchange server using the new account). Frank Wang TechNet Community Support

Hi Stephan, Please run the cmdlet Add-DatabaseAvailabilityGroupServer with parameter -verbose and post the result here. Please run Exbpa to do a "Health Check" and "Permission Check". I guess you had already try to remove and recreate the DAG object? Remove-DatabaseAvailabilityGroup http://technet.microsoft.com/en-us/library/dd335129Frank Wang TechNet Community Support

Hi Frank, The system Attendant service is running on the Mailbox servers. ExBPA is giving no errors (only some warnings about the temp file path). I created a user account and placed it in the domain admins and in the organiztion management group. I logged on with this account to the management console on the DAG witness server and the same error appeared when trying to add a mailbox server to the DAG.You know you're an engineer when you have no life and can prove it mathematically

Hi Frank, The system Attendant service is running on the Mailbox servers. ExBPA is giving no errors (only some warnings about the temp file path). I created a user account and placed it in the domain admins and in the organiztion management group. I logged on with this account to the management console on the DAG witness server and the same error appeared when trying to add a mailbox server to the DAG.You know you're an engineer when you have no life and can prove it mathematically

Hi Stephan, I'm researching the issue and will update it very soon. Sorry for bringing inconvenience. Thanks.Frank Wang TechNet Community Support

Hi Frank, No inconvenience at all! I'm appreciating your help. I hope you have some more ideas to solve this. Regards, StephanYou know you're an engineer when you have no life and can prove it mathematically

Hi Frank, any update or anything I can try yet? Sorry for my impatiance. Regards, StephanYou know you're an engineer when you have no life and can prove it mathematically

Hi Frank, ExBPA is giving no errors (only some warnings about the temp file path). Hi Steve, Did you also run the ExBPA to do a Permission Check? No warnings as well? "I created a user account and placed it in the domain admins and in the organization management group" Could you please remove the account from domain admins group to test? If possible, please use the account to log on to one DAG member server(to create a new Windows Profile) to do the task. If you have a dedicated Hub server, please use it as the FSW, thus you don't need to add the Exchange Trusted Subsystem USG to the local Administrators group on the FSW server. New-DatabaseAvailabilityGroup http://technet.microsoft.com/en-us/library/dd351107.aspx Please disable the firewall to test as well. Frank Wang TechNet Community Support

Hi Frank, Here you can see the output of the permissions check: Active Directory container for 'Role Assignment Policies' found : The Active Directory container for 'Role Assignment Policies' found as expected at CN=Policies,CN=RBAC,CN=<domainname>,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=<domainname>,DC=local. Active Directory container for 'Role Assignments' found : The Active Directory container for 'Role Assignments' found as expected at CN=Role Assignments,CN=RBAC,CN=<domainname>,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=<domainname>,DC=local. Active Directory container for 'Roles' found : The Active Directory container for 'Roles' found as expected at CN=Roles,CN=RBAC,CN=<domainname>,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=<domainname>,DC=local. Active Directory container for 'Scopes' found : The Active Directory container for 'Scopes' found as expected at CN=Scopes,CN=RBAC,CN=<domainname>,CN=Microsoft Exchange,CN=Services,CN=Configuration,DC=<domainname>,DC=local. Active Directory container for 'Security Groups' found : The Active Directory container for 'Security Groups' found as expected at OU=Microsoft Exchange Security Groups,DC=<domainname>,DC=local. Default management role assignment policy found as expected : A default management role assignment policy was found as expected. When I remove the user from the domain admins group, it is not allowed to log on interactively (via RDP or physically on the console ) to the server. I am indeed using a dedicated Hub Server. You know you're an engineer when you have no life and can prove it mathematically

When I remove the user from the domain admins group, it is not allowed to log on interactively (via RDP or physically on the console ) to the server. Hi Steve, Please modify the Local Security Policy on the server: Administrative Tools->Local Security Policy->Local Policies->User Rights Assignment->Allow log on locally->Add User or GroupFrank Wang TechNet Community Support

Frank, I placed the account in this local policy and in the "Remote Desktop Users" group, was able to log on to the DAG witness (hub) server, start the EMC, start the Manage database availabilty group membership wizartd, but the same error appears. So no luck.You know you're an engineer when you have no life and can prove it mathematically

Hi Stephan, I had already involved someone to solve the problem since June 29. Maybe the reponse is delayed due to US Independence Day. If the issue is urgent, suggest you open a case with PSS. Sorry for the inconvenience again. http://support.microsoft.com/default.aspx?id=fh;en-us;offerprophoneFrank Wang TechNet Community Support

Hi Frank, As I don't think the answer will come via this forum, I'll open a case tomorrow. I'll update you via this forum. Regards, StephanYou know you're an engineer when you have no life and can prove it mathematically

Hello Stephan, Make sure that you do not have any other AD object with same name as the node that we are attempting to add. We had seen a similar problem where node had same name as a user in the domain. Regards, Pradeep

Hello Pradeep, Thanks for your reply. I'm actually quite sure there is no other object in AD with the smae name. To be sure, I installed a new server, installed the Exchange 2010 Mailbox server role and upgraded to SP2 and RU3. But also this new machine cannot be placed in the DAG.You know you're an engineer when you have no life and can prove it mathematically

Hi; Was the DAG Account (CNO) pre-staged? Try and give the new mailbox Server permission to the CNO Object. See the below Technet Article for instructions. Pre-stage the Cluster Network Object for a Database Availability Group http://technet.microsoft.com/en-us/library/ff367878Martina Miskovic

I was about to ask the same question. If it's not prestaged I would try creating it manually per the above article (dont forget to disable it) and ensure that a DNS records is created for it. Then try adding your member. Additionally, I would check to be sure your FSW server has the DAG Exchange Trusted Subsystem group in the local admin's group. I know you mentioned it above but unless I misread something I don't see where you said what access, permissions, or otherwise that groups was granted. Chris Morgan

Martina and Chris, The DAG account does exist (it is created with the New-DatabaseAvailabilityGroup cmdlet). It is a disabled computer account. Following the article you suggest, I gave the Exchange Trusted Subsystem group full control on this object. I waited for replication, and tried the Add-DatabaseAvailabilityGroupServer cmdlet again, but still with no luck (still this message: Access is denied. (Exception from HRESULT: 0x80070005 (E_ACCESSDENIED)) Regards, StephanYou know you're an engineer when you have no life and can prove it mathematically

See the article again on how to give the new mailbox server permission to the object.Martina Miskovic

See the article again on how to give the new mailbox server permission to the object. Martina Miskovic As mentioned: Following the article you suggest, I gave the Exchange Trusted Subsystem group full control on this object. I now also gave the computer account of the to-be-first-DAG-node server Full Control. But still the same error.You know you're an engineer when you have no life and can prove it mathematically

I understand that my suggestion wasn't clear. Can you give your (to be) second node the permissions and try?Martina Miskovic

Still the same situation.You know you're an engineer when you have no life and can prove it mathematically

Still the same situation. You know you're an engineer when you have no life and can prove it mathematically Hmm, odd. If you did create a case with Microsoft as you wrote earliear that you were going to do, I hope the problem will be solved soon.Martina Miskovic

And the Exchange Trusted Subsystem shows up in the local admins group of the mailbox server you are trying to add as well right? It is by default when installing Exchange but if something is ran (like restrictive groups in group policy) it could remove it.Chris Morgan

I had the same issue. I don't control the build for our organization and found that the local group policy identified the Log on as a service policy. I added Exchange Trusted Subsystem to this policy and it removed this error. Security Settings/Local Policies/User Rights Assignment/Log on as a service

This needs more investiagtion. If you are still facing the same problem please open a ticket with PSS as suggested earlier.