Hi there,

Setting up a test platform running on 2012R2 with Exchange 2013SP1, and whilst all looks ok I simply cannot add any server to a new DAG, I can create the dag without any issues and tried using DAG with an IP, and a simplified DAG without IP however simply cannot add any of my 2 member servers with the following error;

Exchange 2013 SP1 A server-side administrative operation has failed. 'GetNetworkConfig' failed on the server. Error: The NEtworkManager has not yet initialized.

LAN - 10.21.129.111 & 112 / 24

REPLICATION - 192.168.129.111 & 112 / 24

I've checked everything! All sits on vCloud, removed ALL NICs and replaced, removed Symantec, no loss between REPL NICs, all seems fine.  Not a lot in the logs other than these last results;

[2014-03-05T12:51:10] Updating 'MDB001'.MasterServerOrAvailabilityGroup to 'DAG001.
[2014-03-05T12:51:10] database[ 0 ] (MBDB01)'s MsOrDag=DAG001.
[2014-03-05T12:51:10] database[ 1 ] (MDB001)'s MsOrDag=DAG001.
[2014-03-05T12:51:10] Updated Progress 'Successfully updated the membership of server 'SRV111' in Active Directory group 'DAG001'.' 18%.
[2014-03-05T12:51:10] Working

I'm at the end of the road and cannot find anything to help me ;(

That sounds like the Microsoft Exchange Replication service is having trouble starting on the machines. When they're not in a Dag can you run Get-MailboxDatabaseCopyStatus and Test-ReplicationHealth?

What is the status of IPv6 on both server? Is it disabled or enabled?

Where have you created the FSW?

HI Jared,

I agree, have a feeling it's that bit but not sure how to check if working or re-install perhaps.

Copy status is ALL ok;

CopyQueueLength - 0
ReplayQueueLEnght - 0
ContentIndexState - Healthy

This is without DAG.  For replication Health fails on last 2 points on both VMs, could it be missed failover cluster bit or didn't install correctly?

DatabaseRedundancy

DatabaseAvailability

RunspaceId       : d4597641-ae09-4687-8e66-cfbef67bc31f
Server           : SRV111
Check            : DatabaseRedundancy
CheckDescription : Verifies that databases have sufficient redundancy. If this check fails, it means that some
                   databases are at risk of losing data.
Result           : *FAILED*
Error            : Failures:
                       There were database redundancy check failures for database 'MBDB01' that may be lowering its
                   redundancy and putting the database at risk of data loss. Redundancy Count: 1. Expected Redundancy
                   Count: 2. Detailed error(s):

                           SRV111:
                           Database 'MBDB01' does not have enough copies configured to meet the validation criteria.

                       There were database redundancy check failures for database 'MDB001' that may be
                   lowering its redundancy and putting the database at risk of data loss. Redundancy Count: 1.
                   Expected Redundancy Count: 2. Detailed error(s):

                           SRV111:
                           Database 'MDB001' does not have enough copies configured to meet the validation
                   criteria.

Identity         :
IsValid          : True
ObjectState      : New

RunspaceId       : d4597641-ae09-4687-8e66-cfbef67bc31f
Server           : SRV111
Check            : DatabaseAvailability
CheckDescription : Verifies that databases have sufficient availability. If this check fails, it means that some
                   databases are at risk of losing service.
Result           : *FAILED*
Error            : Failures:
                       There were database availability check failures for database 'MBDB01' that may be lowering its
                   availability. Availability Count: 0. Expected Availability Count: 2. Detailed error(s):

                           SRV111:
                           Couldn't read the database availability group for server
                   'SRV111.ad.totalitycloud.com' from Active Directory. Error: Could not find the database
                   availability group object for server SRV111 in Active Directory.

                       There were database availability check failures for database 'MDB001' that may be
                   lowering its availability. Availability Count: 0. Expected Availability Count: 2. Detailed
                   error(s):

                           SRV111:
                           Couldn't read the database availability group for server
                   'SRV111.ad.totalitycloud.com' from Active Directory. Error: Could not find the database
                   availability group object for server SRV111 in Active Directory.

Identity         :
IsValid          : True
ObjectState      : New

HI Liaqat,

OK IPv6 is enabled on MAPI AND REPLI, I did disabled on REPL but no different.  FSA is  a 2012R2 VM on same subnet/domain etc.. and has file share enabled along with Exchange Trusted permissions etc...

You can leave IPv6 enabled. If it is disabled then it should be completely disabled through registry.

How about firewall setting on 2012R2 VM having FSW?

Since Get-MailboxDatabaseCopyStatus and Test-ReplicationHealth are at least returning it means that the Replay service is up and running.

So you create the Dag. Can you add the first member to the Dag, or is this happening when adding the second?

If it's happening on the first does running New-Cluster -AdministrativeAccesPoint:None -Name <some name> succeed?

Please check and verify the cluster core resources are up.

Hi!

Firewall is disabled as part of Group Policy on ALL domain, private and public, so all servers automatically have it disabled. 

Hi Jared,

Happens to both, whether I add first or 2nd VM which could be a server issue.  I haven't run new cluster  but only new dag.

How do I do that!

Also do you see any error event log for cluster service?

Few more updates, when I add to DAG I get error message but click ok and adds however status isn't right.

If I add it using PowerShell I get the following;

WARNING: An unexpected error has occurred and a Watson dump is being generated: Some or all identity references could not be translated.

Hi,

I would like to clarify the following things for troubleshooting:

1. Each member of the DAG should be running the same operating system.

2. Each Mailbox server in a DAG should be a member server in the same domain.

Please check if the servers you are adding meet the above requirements.

What's more, here is a helpful article for your reference.

Planning for High Availability and Site Resilience

http://technet.microsoft.com/en-us/library/dd638104(v=exchg.150).aspx

Hope it helps.

Best regards,
Amy

Hi Amy, thanks.

1 - 100% - Based on the same VM image

2 - 100% - It's a newly created domain in 2012R2.

The ONLY thing I haven't done is open up firewall ports etc.. but I've tested internal Outlook access.  REally strange and annoying!!!

Did you update the OS with latest patches/updates?

Hi Shoaib,

Absolutely! 100%, used latest Exchange SP1 2013 unless something not right with that as I know it's very newly released...

In Active Directory take the property of the Computer objet for the DAG, under the attribute editor tab make sure it has the fully qualified domain name under dnshostname property.

Make sure the first computer account for the first member server you are trying to add has full control over the computer object for the dag.

and that the compuer account for the DAG is disabled before adding the first server to the DAG.

Can you upload a snapshot for the error.

HI Shoaib,

I've tried both methods, which is creating a predefined CNO and a dag with an IP as well as the new method where you can create a DAG without an IP and both have the same error.  I've done what you mentioned for predefined DAG. Uploaded snapshot of what happens when I add member to a DAG;

Hi,

Did you happen to go through the below article?

http://blogs.technet.com/b/scottschnoll/archive/2014/02/25/database-availability-groups-and-windows-server-2012-r2.aspx

Hi Shoaib, Thx, i have and ive tested creating a dag both with aap IP and new style without. I get the same problem when adding either exchange to dag :( Must be something on the os ive missed because cant find the problem at all!, so annoying!!

Not sure how relevant, but I left it a few weeks, I applied all new updates, rebooted entire environment.

I then redid everything, I created a DAG (using CLI) gave me error and as normal created the DAG anyway and added x2 servers.  I added the mailbox to the DAG and no errors at all, everything looks ok on web portal and ran command below

Get-DatabaseAvailabilityGroup Identity DAG001 Status

All looks ok but what's the best way to see all is 100%?  I am concerned I got errors but still able to do everything, when I run

Test-ReplicationHealth

SRV112  DatabaseRedundancy         *FAILED*   Failures:...
SRV112  DatabaseAvailability       *FAILED*   Failures:...

Everything else is ok, I'm going to test it fully but curious to see what people think!

Everything looks 100% which is strange despite errors when I added the servers to the DAG!! Ran ReplicationHealth and have below.S

Should I be concerned about previous error messages??  Any concrete test I can do other than ones I've done?

[PS] C:\Windows\system32>Test-ReplicationHealth

Server          Check                      Result
------          -----                      ------
SRV112  ClusterService             Passed
SRV112  ReplayService              Passed
SRV112  ActiveManager              Passed
SRV112  TasksRpcListener           Passed
SRV112  TcpListener                Passed
SRV112  ServerLocatorService       Passed
SRV112  DagMembersUp               Passed
SRV112  ClusterNetwork             Passed
SRV112  QuorumGroup                Passed
SRV112  DatabaseRedundancy         Passed
SRV112  DatabaseAvailability       Passed
SRV112  DBCopySuspended            Passed
SRV112  DBCopyFailed               Passed
SRV112  DBInitializing             Passed
SRV112  DBDisconnected             Passed
SRV112  DBLogCopyKeepingUp         Passed
SRV112  DBLogReplayKeepingUp       Passed

You'll often see transient errors when adding servers to the DAG or making changes. Once everything settles down if you seeing all passed then you should be good to go.

Everything looks ok, very strange...I had issues adding mailboxes to DAG via portal, worked 100% better via shell...  I'm still little concerned despite everything working ok!!

I am having trouble also creating a DAG in 2012R2 and Exchange 2013SP1.  I don't have trouble in the GUI/Portal creating it, and I can also add both my mailbox servers in to the DAG (there are only two).  My problem comes checking the DAG just after adding the servers to the DAG and forming the cluster.  The GUI/Portal shows all is fine but when I run:

Get-DatabaseAvailabilityGroup DAG1 -Status | Fl

It shows pretty much everything OK except that the FSW shows not in use.

Partial output is:

RunspaceId                             : a6c17fdc-d7c6-4593-b10a-a0d2be53cf6e
Name                                   : DAG1
Servers                                : {EPEX2, EPEX1}
WitnessServer                          : epfs.domain.net
WitnessDirectory                       : C:\DAG1
AlternateWitnessServer                 :
AlternateWitnessDirectory              :
NetworkCompression                     : InterSubnetOnly
NetworkEncryption                      : InterSubnetOnly
ManualDagNetworkConfiguration          : False
DatacenterActivationMode               : DagOnly
StoppedMailboxServers                  : {}
StartedMailboxServers                  : {EPEX2.domain.net, EPEX1.domain.net}
DatabaseAvailabilityGroupIpv4Addresses : {10.1.10.11}
DatabaseAvailabilityGroupIpAddresses   : {10.1.10.12}
AllowCrossSiteRpcClientAccess          : False
OperationalServers                     : {EPEX1, EPEX2}
PrimaryActiveManager                   : EPEX1
ServersInMaintenance                   : {}
ServersInDeferredRecovery              : {}
ThirdPartyReplication                  : Disabled
ReplicationPort                        : 64327
NetworkNames                           : {MapiDagNetwork, ReplicationDagNetwork01}
WitnessShareInUse                      : None
DatabaseAvailabilityGroupConfiguration :
AutoDagSchemaVersion                   : 1.0
AutoDagDatabaseCopiesPerDatabase       : 1
AutoDagDatabaseCopiesPerVolume         : 1
AutoDagTotalNumberOfDatabases          : 0
AutoDagTotalNumberOfServers            : 0
AutoDagDatabasesRootFolderPath         : C:\ExchangeDatabases
AutoDagVolumesRootFolderPath           : C:\ExchangeVolumes
AutoDagAllServersInstalled             : False
AutoDagAutoReseedEnabled               : True
AutoDagDiskReclaimerEnabled            : True
ReplayLagManagerEnabled                : False
AdminDisplayName                       :
ExchangeVersion                        : 0.10 (14.0.100.0)

Notice the "WinessShareInUse:   None"  when it should be "Primary"

And if I look on the witness server, the witness folder hasn't been created.

I did add the "Exchange Trusted Subsystem" to the local administrators group on the FSW server but still now proper witness.  It wouldn't have let me create the DAG to begin with if the Exch Trusted Sub wasn't in the group to begin with.

Pjmartins,

Have you checked to see of the witness is in use and working?  On mine, failover cluster manager shows "Witness: none" but otherwise seems fine!

I've been struggling to get the witness to work but haven't been able to.  A search led me to your issue and I thought it might be the same.

Hi Andy,

Mine is the same, bar a few exceptions, nice to have someone else in the same boat!  The WFS is strange because I see no directory etc...

ManualDagNetworkConfiguration          : True

This is because I've set my replication network to just replication LAN.

DatacenterActivationMode               : Off

My nodes are in the same DC, so I don't think I need this one until I get an off site Exchange VM added later on.

DatabaseAvailabilityGroupIpv4Addresses : {255.255.255.255}
DatabaseAvailabilityGroupIpAddresses   : {255.255.255.255}

I used the new function without creating an IP or AD Computer etc....  Basically no administrative access point

WitnessShareInUse                      : None

I also have this!

Maybe it's all normal but still strange, doing a load of testing and so far ok. Waiting to test VIP Load Blanacing..

This is very late but for others coming afterwards.  Doublecheck the Mailbox Database Copy at Servers/Databases/.../AddDatabaseCopy.

Just added a test database and got the response Jared got.  Reran the Test-ReplicationHealth cmdlt and showed Passed.