Two Mailbox Databases in a DAG - one with Content Index Failed and one with Content Index FailedandSuspended

Hello,

I have two Server 2012 R2 servers both running Exchange 2013 with SP1 (CU4).  I know that CU4 is outdated now and I do plan to upgrade to CU9 very soon, but I was hoping to get this issue addressed first before attempting the upgrade.

I have one mailbox database and one DAG.  The active copy of the database is on EXCH1 and the passive is on EXCH2.  The database status on EXCH1 is Mounted and the database status on EXCH2 is Healthy.  But on EXCH1, the Content Index State is Failed and on EXCH2, the Content Index State is FailedandSuspended.

I have found some references on how to rebuild the Content Index when the database is in a DAG using this command (Get-MailboxDatabaseCopyStatus * | where {$_.ContentIndexState -eq "Failed"} | Update-MailboxDatabaseCopy -CatalogOnly) but this seems to only apply when there's at least one copy that has a health Content Index.  Since both of my copies have failed Content Indexes, it doesn't seem that I can use this method.

I have also found some references that say you can stop the Microsoft Exchange Search service and the Microsoft Exchange Search Host Controller service, then either rename or delete the folder with the GUID name in the mailbox database path location, then restart both of those services and the Content Index will rebuild itself.  I tried that on my active copy and when I restarted the services and then ran Get-MailboxDatabaseCopyStatus on EXCH1, I did see a new GUID folder get created but that folder is only about 16MB in size and it has not grown at all since I restarted the services.  And on EXCH1, the Content Index state remains at Failed and I never saw it change to Crawling.  I then tried stopping the Search services on EXCH2, renaming the GUID folder, and then restarting the services and once again, the GUID folder did get recreated, but the size remained at about 16MB and the Content Index State for that database copy still says FailedandSuspended.

On EXCH1, when I run Get-MailboxDatabaseCopyStatus | fl, I am seeing ContentIndexState Failed and ContentIndexErrorMessage The database has been dismounted.  On EXCH2, when I run Get-MailboxDatabaseCopyStatus | fl, I am seeing ContentIndexState FailedandSuspended and ContentIndexErrorMessage Reseeding of the index is required.

Also on EXCH2, I am seeing the following when running Get-MailboxDatabaseCopyStatus | fl:

IncomingLogCopyingNetwork  {EXCH1,MapiDagNetwork,An error occurred while communicating with server 'EXCH1'. Error: Unable to write data to the transport connection: An established connection was aborted by the software in your host machine.}  I am uncertain if this error is somehow related to the issue with the Content Indexes or if this is a separate issue.  I am only seeing this condition on the server with the passive copy of the mailbox database.

Thank you for taking the time to read this post!

Daniel

July 30th, 2015 1:07pm

I had the same thing with one of my Dag DB's (we have 24 DB's). I tried all the steps you mentioned to try to fix the index in addition to trying to use the 2013 exchange scripts to reset the index (They were 2010 versions and didn't work with 2013).

I wound up creating a new DB and moving the users to the new DB.  At that point I discovered that I had a mailbox that had a corrupted folder and wouldn't move.  After going through many different attempts to try to fix the mailbox, I wound up exporting the mailbox to PST, deleting and recreating it.  After that the content index on that DB changed to healthy.  I was able to move all the mailboxes back to the original DB.

You might want to check your application event log for any warnings or errors related to Database Maintenance.

Free Windows Admin Tool Kit Click here and download it now
July 30th, 2015 3:19pm

Hi Daniel,

Please create a new AD group named ContentSubmitters and grant Admistrators and NetworkService full access to it.

Thereafter restart "Microsoft Exchange Search" & "Microsoft Exchange Search Host Controller" services to have a try.

For detailed information about it, please refer to:

https://support.microsoft.com/en-us/kb/2807668

Regards,

July 31st, 2015 5:27am

Hi Corey,

I ended up opening up a case with MS Support on this issue and we tried various things to get the Content Indexes healthy on both sides of the DAG.  We were successful in getting the CI to a Healthy state on the Active copy of the DB but on the Passive side, we could never get it back to a Healthy state.  Once the Active side was Healthy, we tried re-seeding the Passive side using the -CatalogOnly switch but the Passive side reverted back to a FailedandSuspended state after the re-seed operation.  We also tried re-seeding the entire database (mailboxes and CI data) twice but both times we ended up with a Healthy mailbox database copy but FailedandSuspended for the CI on the Passive side.

So, Microsoft did ultimately suggest that we begin the process of migrating our mailboxes to a new database.  I haven't had the time to start that process yet but I hope to get to it soon.  Since you had similar success with this method, I wanted to post this update to let you and others know that is what MS Support has suggested as well.  Once I know for sure that fixes the issue (and I have a pretty good feeling it will), I will come back and mark your reply as the Answer.

Thank you very much for taking the time to respond!

Daniel

Free Windows Admin Tool Kit Click here and download it now
August 25th, 2015 12:16pm

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics