Failed to add a 2nd node to Exchange 2013 DAG
Hi
Im try to add a Second server to DAG cluster but i always get the same error:
I have checked:
Exchange Trusted SubSystem is member of all local admin groups on the server.Rights on the FileShare WittnesAdd the server througe GUI and a Administrator PS session.Checked all DNS record
I have followed this and checked all the settings:
http://technet.microsoft.com/en-us/library/dd638104.aspx#feedback
error
A server-side database availability group administrative operation failed. Error The operation failed. CreateCluster errors may result from incorrectly configured static addresses. Error: An error occurred while attempting a cluster operation.
Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed..
[Server: SRV010-STO-VM.ad.contoso.com]
Log File:
dd-databaseavailabiltygroupserver started on machine SRV011.
[2013-03-28T10:04:57] add-dagserver started
[2013-03-28T10:04:57] commandline: $scriptCmd = {& $wrappedCmd @PSBoundParameters }
[2013-03-28T10:04:57] Option 'Identity' = 'NCEX-STO-DAG1'.
[2013-03-28T10:04:57] Option 'MailboxServer' = 'SRV011'.
[2013-03-28T10:04:57] Option 'DatabaseAvailabilityGroupIpAddresses' = ''.
[2013-03-28T10:04:57] Option 'WhatIf' = ''.
[2013-03-28T10:04:57] Process: w3wp w3wp.exe:3572.
[2013-03-28T10:04:57] User context = 'NT AUTHORITY\SYSTEM'.
[2013-03-28T10:04:57] Member of group 'Everyone'.
[2013-03-28T10:04:57] Member of group 'BUILTIN\Users'.
[2013-03-28T10:04:57] Member of group 'NT AUTHORITY\SERVICE'.
[2013-03-28T10:04:57] Member of group 'CONSOLE LOGON'.
[2013-03-28T10:04:57] Member of group 'NT AUTHORITY\Authenticated Users'.
[2013-03-28T10:04:57] Member of group 'NT AUTHORITY\This Organization'.
[2013-03-28T10:04:57] Member of group 'BUILTIN\IIS_IUSRS'.
[2013-03-28T10:04:57] Member of group 'LOCAL'.
[2013-03-28T10:04:57] Member of group 'IIS APPPOOL\MSExchangePowerShellAppPool'.
[2013-03-28T10:04:57] Member of group 'BUILTIN\Administrators'.
[2013-03-28T10:04:57] Updated Progress 'Validating the parameters.' 2%.
[2013-03-28T10:04:57] Working
[2013-03-28T10:04:58] Mailbox server: value passed in = SRV011, mailboxServer.Name = SRV011, mailboxServer.Fqdn = SRV011.ad.contoso.com
[2013-03-28T10:04:58] LogClussvcState: clussvc is Stopped on SRV011.ad.contoso.com. Exception (if any) = none
[2013-03-28T10:04:58] The IP addresses for the DAG are (blank means DHCP): 192.168.5.165
[2013-03-28T10:04:58] Looking up IP addresses for NCEX-STO-DAG1.
[2013-03-28T10:04:58] NCEX-STO-DAG1 = [ 192.168.5.165 ].
[2013-03-28T10:04:58] Looking up IP addresses for SRV011.
[2013-03-28T10:04:58] SRV011 = [ ::1, 172.30.10.21, 192.168.5.159 ].
[2013-03-28T10:04:58] Looking up IP addresses for SRV011.ad.contoso.com.
[2013-03-28T10:04:58] SRV011.ad.contoso.com = [ ::1, 172.30.10.21, 192.168.5.159 ].
[2013-03-28T10:04:58] DAG NCEX-STO-DAG1 has 1 servers:
[2013-03-28T10:04:58] DAG NCEX-STO-DAG1 contains server SRV010.
[2013-03-28T10:04:58] Updated Progress 'Checking if Mailbox server 'SRV011' is in a database availability group.' 4%.
[2013-03-28T10:04:58] Working
[2013-03-28T10:04:58] GetRemoteCluster() for the mailbox server failed with exception = An Active Manager operation failed. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"OpenCluster(SRV011.ad.contoso.com) failed with
0x6d9. Error: There are no more endpoints available from the endpoint mapper"' failed.. This is OK.
[2013-03-28T10:04:58] Ignoring previous error, as it is acceptable if the cluster does not exist yet.
[2013-03-28T10:04:58] DumpClusterTopology: Opening remote cluster NCEX-STO-DAG1.
[2013-03-28T10:04:59] Dumping the cluster by connecting to: NCEX-STO-DAG1.
[2013-03-28T10:04:59] The cluster's name is: NCEX-STO-DAG1.
[2013-03-28T10:04:59] Groups
[2013-03-28T10:04:59] group: Available Storage [not a CMS]
[2013-03-28T10:04:59] OwnerNode: SRV010.ad.contoso.com
[2013-03-28T10:04:59] State: Offline
[2013-03-28T10:04:59] group: Cluster Group [Cluster Main Group]
[2013-03-28T10:04:59] OwnerNode: SRV010.ad.contoso.com
[2013-03-28T10:04:59] State: Online
[2013-03-28T10:04:59] Resource: Cluster IP Address [Online, type = IP Address, PossibleOwners = SRV010 ]
[2013-03-28T10:04:59] Address = [192.168.5.165]
[2013-03-28T10:04:59] EnableDhcp = [0]
[2013-03-28T10:04:59] Network = [Cluster Network 1]
[2013-03-28T10:04:59] Resource: Cluster Name [Online, type = Network Name, PossibleOwners = SRV010 ]
[2013-03-28T10:04:59] NetName = [NCEX-STO-DAG1]
[2013-03-28T10:04:59] Nodes
[2013-03-28T10:04:59] node: SRV010.ad.contoso.com [ state = Up ]
[2013-03-28T10:04:59] Subnets
[2013-03-28T10:04:59] Name(Cluster Network 1), Mask(192.168.5.0/24), Role(ClusterNetworkRoleInternalAndClient)
[2013-03-28T10:04:59] NIC 192.168.5.158 on Node SRV010 in State=Up
[2013-03-28T10:04:59] Name(Cluster Network 2), Mask(172.30.10.0/24), Role(ClusterNetworkRoleInternalUse)
[2013-03-28T10:04:59] NIC 172.30.10.20 on Node SRV010 in State=Up
[2013-03-28T10:04:59] Opening the cluster on nodes [SRV010].
[2013-03-28T10:04:59] Other mailbox servers in the DAG are already members of cluster 'NCEX-STO-DAG1'
[2013-03-28T10:04:59] The server SRV011 does not belong to a cluster, and the other servers belong to NCEX-STO-DAG1.
[2013-03-28T10:04:59] Successfully resolved the servers based on the stopped servers list.
[2013-03-28T10:04:59] The following servers are in the StartedServers list (The list is the StartedServers property of the DAG in AD):
[2013-03-28T10:04:59] The following servers are in the StoppedServers list:
[2013-03-28T10:04:59] Verifiying that the members of database availability group 'NCEX-STO-DAG1' are also members of the cluster.
[2013-03-28T10:04:59] Verifying that the members of cluster 'NCEX-STO-DAG1' are also members of the database availability group.
[2013-03-28T10:04:59] According to GetNodeClusterState(), the server SRV011 is NotConfigured.
[2013-03-28T10:04:59] The CNO is currently Online.
[2013-03-28T10:04:59] InternalValidate() done.
[2013-03-28T10:05:08] Updated Progress 'Adding server 'SRV011' to database availability group 'NCEX-STO-DAG1'.' 6%.
[2013-03-28T10:05:08] Working
[2013-03-28T10:05:08] Updated Progress 'Adding server 'SRV011' to the cluster.' 8%.
[2013-03-28T10:05:08] Working
[2013-03-28T10:11:27] The following log entry comes from a different process that's running on machine 'SRV010.ad.contoso.com'. BEGIN
[2013-03-28T10:11:27] [2013-03-28T10:05:08] Opening a local AmCluster handle.
[2013-03-28T10:05:08] Updated Progress 'Adding server 'SRV011' to database availability group 'NCEX-STO-DAG1'.' 2%.
[2013-03-28T10:05:08] Working
[2013-03-28T10:05:08] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = SRV011, dwStatus = 0x0 )
[2013-03-28T10:05:09] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNodeState, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 12, szObjectName = , dwStatus = 0x0 )
[2013-03-28T10:05:09] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseGettingCurrentMembership, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 25, szObjectName = NCEX-STO-DAG1, dwStatus
= 0x0 )
[2013-03-28T10:05:09] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseGettingCurrentMembership, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 25, szObjectName = , dwStatus = 0x0 )
[2013-03-28T10:05:09] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseAddNodeToCluster, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 37, szObjectName = SRV011, dwStatus = 0x0 )
[2013-03-28T10:05:09] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseAddNodeToCluster, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 37, szObjectName = , dwStatus = 0x0 )
[2013-03-28T10:05:09] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNetft, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 50, szObjectName = SRV011, dwStatus = 0x0 )
[2013-03-28T10:05:09] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateNetft, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 50, szObjectName = , dwStatus = 0x0 )
[2013-03-28T10:05:09] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateClusDisk, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 62, szObjectName = SRV011, dwStatus = 0x0 )
[2013-03-28T10:05:09] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseValidateClusDisk, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 62, szObjectName = , dwStatus = 0x0 )
[2013-03-28T10:05:09] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseConfigureClusSvc, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 75, szObjectName = SRV011, dwStatus = 0x0 )
[2013-03-28T10:05:09] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseConfigureClusSvc, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 75, szObjectName = , dwStatus = 0x0 )
[2013-03-28T10:05:09] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseStartingClusSvc, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 87, szObjectName = SRV011.ad.contoso.com, dwStatus
= 0x0 )
[2013-03-28T10:05:10] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseStartingClusSvc, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 87, szObjectName = , dwStatus = 0x0 )
[2013-03-28T10:05:10] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseNodeUp, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = SRV011, dwStatus = 0x0 )
[2013-03-28T10:08:10] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseNodeUp, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseFatal, dwPercentComplete = 100, szObjectName = SRV011, dwStatus = 0x5b4 )
[2013-03-28T10:08:10] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseStart, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = SRV011, dwStatus = 0x0 )
[2013-03-28T10:11:26] ClusterSetupProgressCallback( eSetupPhase = ClusterSetupPhaseFailureCleanup, ePhaseType = ClusterSetupPhaseEnd, ePhaseSeverity = ClusterSetupPhaseInformational, dwPercentComplete = 100, szObjectName = , dwStatus = 0x0 )
[2013-03-28T10:11:27] The preceding log entry comes from a different process running on computer 'SRV010.ad.contoso.com'. END
[2013-03-28T10:11:27] The operation wasn't successful because an error was encountered. You may find more details in log file "C:\ExchangeSetupLogs\DagTasks\dagtask_2013-03-28_10-04-57.598_add-databaseavailabiltygroupserver.log".
[2013-03-28T10:13:59] WriteError! Exception = Microsoft.Exchange.Cluster.Replay.DagTaskOperationFailedException: A server-side database availability group administrative operation failed. Error The operation failed. CreateCluster errors may result from incorrectly
configured static addresses. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This operation returned because the timeout period expired"' failed..
---> Microsoft.Exchange.Cluster.Replay.AmClusterApiException: An Active Manager operation failed. Error: An error occurred while attempting a cluster operation. Error: Cluster API '"AddClusterNode() (MaxPercentage=100) failed with 0x5b4. Error: This
operation returned because the timeout period expired"' failed. ---> System.ComponentModel.Win32Exception: This operation returned because the timeout period expired
--- End of inner exception stack trace ---
at Microsoft.Exchange.Cluster.ClusApi.AmCluster.AddNodeToCluster(AmServerName nodeName, IClusterSetupProgress setupProgress, IntPtr context, Exception& errorException, Boolean throwExceptionOnFailure)
at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
--- End of inner exception stack trace (Microsoft.Exchange.Cluster.Replay.AmClusterApiException) ---
at Microsoft.Exchange.Cluster.Replay.DagHelper.ThrowDagTaskOperationWrapper(Exception exception)
at Microsoft.Exchange.Cluster.Replay.DagHelper.AddDagClusterNode(AmServerName mailboxServerName, String& verboseLog)
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.RunRpcServerOperation(String databaseName, RpcServerOperation rpcOperation)
--- End of stack trace on server (SRV010.ad.contoso.com) ---
at Microsoft.Exchange.Data.Storage.Cluster.HaRpcExceptionWrapperBase`2.ClientRethrowIfFailed(String databaseName, String serverName, RpcErrorExceptionInfo errorInfo)
at Microsoft.Exchange.Cluster.Replay.ReplayRpcClientWrapper.RunAddNodeToCluster(AmServerName serverName, AmServerName newNode, String& verboseLog)
at Microsoft.Exchange.Management.SystemConfigurationTasks.AddDatabaseAvailabilityGroupServer.JoinNodeToCluster()
[2013-03-28T10:14:24] Updated Progress 'Done!' 100%.
[2013-03-28T10:14:24] COMPLETED
add-databaseavailabiltygroupserver explicitly called CloseTempLogFile().
anyone else experiencing the same problem?
March 28th, 2013 8:12am
I'm having the same problem.
Free Windows Admin Tool Kit Click here and download it now
March 28th, 2013 12:47pm
I hope both exchange servers are running on same windows version!
Check this article
http://blogs.perficient.com/microsoft/2011/03/incorrectly-configured-static-address-when-creating-a-dag/Gulab Prasad,
Exchange Ranger
Z-Hire Employee Provisioning App
March 28th, 2013 11:58pm
I think this is because of you don't have enough rights to create objects in AD.
You can try following:
- Create computer account in AD with the same name as your DAG
- Grant full access permissions to this computer account for "Exchange Trusted Subsystem Group" group and for your mailbox servers (it would be computer accounts).
- Disable this computer account and retry creating DAG / joining servers to DAG
Hope this provided you some workaround.
Good luck!
Free Windows Admin Tool Kit Click here and download it now
March 29th, 2013 3:17am
Can you share the output of get-databaseavailabilitygroup and get-databaseavailabilitygroupnetwork
March 29th, 2013 7:32am
Hi Pewee,
Any updates?
If you have any feedback on our support, please click
here
Frank Wang
TechNet Community Support
Free Windows Admin Tool Kit Click here and download it now
March 31st, 2013 10:44pm
Yes !
Windows server 2012 STD all updates.
/pw
April 2nd, 2013 3:17am
Im Syrry tha did not help. I have prestage the computer object in Active Directory Before i created the DAG.
/p
Free Windows Admin Tool Kit Click here and download it now
April 2nd, 2013 3:18am
Name Member Servers
Operational Servers
---- --------------
-------------------
NCEX-STO-DAG1 {SRV010-STO-VM}
Identity ReplicationEnabled
Subnets
-------- ------------------
-------
NCEX-STO-DAG1\MapiDagNetwork True
{{192.168.5.0/24,Up}}
NCEX-STO-DAG1\ReplicationDagNetwork01 True
{{172.30.10.0/24,Up}}
April 2nd, 2013 3:21am
This problem is now solved.
There is a bug in MS network teaming
software.
The solotion is either
breaks Teaming or removing the
"offloading" on the network
adapter and the Team.
April 23rd, 2013 4:40pm
This problem is now solved.
There is a bug in MS network teaming
software.
The solotion is either
breaks Teaming or removing the
"offloading" on the network
adapter and the Team.
Hi Per
Wiberg,
How exactly did you fix this problem? I have been trying to address the same issue since months but to no avail.
Could you please elaborate on the steps you followed to fix this?
Thanks & Regards,
Rahul
Free Windows Admin Tool Kit Click here and download it now
May 14th, 2013 6:36am