Unable to login to ECP

Exchange 2013 SP1 with a 2 member DAG. Starting today, administrator is not able to login to ecp. When you put your username and password in and click sign in, the page flashes the password field is cleared with no error message. The same thing happens of the server with the passive copy. I followed the advise on this thread:

https://social.technet.microsoft.com/Forums/office/en-US/a07fa978-81cf-4722-b222-eaa981e680d2/exchange-server-2013-ecp-login?forum=exchangesvrdeploy

to run 

set-ecpvirtualdirectory -Identity "ecp (default web site)" -windowsauthentication $true -formsauthentication $false

do an ipconfig /flushdns on your desktop.

log in to your ecp with https://yourservername/ecp/?exchclientver=15

and the same thing happens. I restarted the server with the active copy, same problem. Email is flowing both ways (send and receive). On the server with the active copy, these errors are in the application log after the restart:

Microsoft Exchange Server Locator Service failed to find active server for database 'b167e9f8-98a2-4635-a6da-17cc153dadc8'. Error: An Active Manager operation failed. Error: Invalid Active Manager configuration. Error: The Cluster service is not running.

But the cluster service IS running and cluster manager reports 0 issues. Cluster service IS running on both servers. More errors:

A whole bunch of postgresql errors:

The description for Event ID 0 from source PostgreSQL cannot be found. Either the component that raises this event is not installed on your local computer or the installation is corrupted. You can install or repair the component on the local computer.If the event originated on another computer, the display information had to be saved with the event.The following information was included with the event: FATAL:  terminating connection due to administrator command


Along with event id 1012:

Data loss occurred in RetentionAgent: RetentionAgent: Data loss occurred. The size of this folder C:\Program Files\Microsoft\Exchange Server\V15\Logging\Diagnostics\DailyPerformanceLogs has reached the max size allowed - 5120 MB. Some files will be purged.


Perfnet 2004 event id's, 1008 perflib and event id 1032:

The EDS Job manager failed to start the following jobs: 
Job: 'TransportSyncHealthHubLog' is poisoned.

Event id 4105:

The Microsoft Exchange Replication service failed to RPC to the local Information Store service. This often means that the Information Store service is not running, but it may also be unresponsive. Error:MapiExceptionNetworkError: Unable to make admin interface connection to server. (hr=0x80040115, ec=-2147221227)
Diagnostic context:
    Lid: 65256  
    Lid: 49064   dwParam: 0x1
    Lid: 12514   Win32Error: 0x6D9
    Lid: 62184  
    Lid: 16280   dwParam: 0x0 Msg: EEInfo: ComputerName: n/a
    Lid: 8600    dwParam: 0x0 Msg: EEInfo: ProcessID: 4040
    Lid: 12696   dwParam: 0x0 Msg: EEInfo: Generation Time: 0415-05-08T01:38:19.3970000Z
    Lid: 10648   dwParam: 0x0 Msg: EEInfo: Generating component: 2
    Lid: 14744   dwParam: 0x0 Msg: EEInfo: Status: 1753
    Lid: 9624    dwParam: 0x0 Msg: EEInfo: Detection location: 883
    Lid: 13720   dwParam: 0x0 Msg: EEInfo: Flags: 0
    Lid: 11672   dwParam: 0x0 Msg: EEInfo: NumberOfParameters: 0
    Lid: 62184  
    Lid: 16280   dwParam: 0x0 Msg: EEInfo: ComputerName: n/a
    Lid: 8600    dwParam: 0x0 Msg: EEInfo: ProcessID: 4040
    Lid: 12696   dwParam: 0x0 Msg: EEInfo: Generation Time: 0415-05-08T01:38:19.3970000Z
    Lid: 10648   dwParam: 0x0 Msg: EEInfo: Generating component: 2
    Lid: 14744   dwParam: 0x0 Msg: EEInfo: Status: 1753
    Lid: 9624    dwParam: 0x0 Msg: EEInfo: Detection location: 900
    Lid: 13720   dwParam: 0x0 Msg: EEInfo: Flags: 0
    Lid: 11672   dwParam: 0x0 Msg: EEInfo: NumberOfParameters: 0
    Lid: 62184  
    Lid: 16280   dwParam: 0x0 Msg: EEInfo: ComputerName: n/a
    Lid: 8600    dwParam: 0x0 Msg: EEInfo: ProcessID: 4040
    Lid: 12696   dwParam: 0x0 Msg: EEInfo: Generation Time: 0415-05-08T01:38:19.3970000Z
    Lid: 10648   dwParam: 0x0 Msg: EEInfo: Generating component: 2
    Lid: 14744   dwParam: 0x0 Msg: EEInfo: Status: 1753
    Lid: 9624    dwParam: 0x0 Msg: EEInfo: Detection location: 501
    Lid: 13720   dwParam: 0x0 Msg: EEInfo: Flags: 0
    Lid: 11672   dwParam: 0x0 Msg: EEInfo: NumberOfParameters: 4
    Lid: 8856    dwParam: 0x0 Msg: EEInfo: prm[0]: Unicode  string: ncalrpc
    Lid: 8856    dwParam: 0x0 Msg: EEInfo: prm[1]: Unicode  string: 
    Lid: 12952   dwParam: 0x0 Msg: EEInfo: prm[2]: Long val: 2306091726
    Lid: 12952   dwParam: 0x0 Msg: EEInfo: prm[3]: Long val: 382312662
    Lid: 24060   StoreEc: 0x80040115
    Lid: 23746  
    Lid: 31938   StoreEc: 0x80040115
    Lid: 19650  
    Lid: 27842   StoreEc: 0x80040115
    Lid: 20866  

and a whole bunch of event id 106's:

Performance counter updating error. Counter name is Time in Resource per second, category name is MSExchange Activity Context Resources. Optional code: 2. Exception: The exception thrown is : System.InvalidOperationException: Instance 'ad-powershell-defaultdomain' already exists with a lifetime of Process.  It cannot be recreated or reused until it has been removed or until the process using it has exited.
   at System.Diagnostics.SharedPerformanceCounter.FindInstance(Int32 instanceNameHashCode, String instanceName, CategoryEntry* categoryPointer, InstanceEntry** returnInstancePointerReference, Boolean activateUnusedInstances, PerformanceCounterInstanceLifetime lifetime, Boolean& foundFreeInstance)
   at System.Diagnostics.SharedPerformanceCounter.GetCounter(String counterName, String instanceName, Boolean enableReuse, PerformanceCounterInstanceLifetime lifetime)
   at System.Diagnostics.SharedPerformanceCounter..ctor(String catName, String counterName, String instanceName, PerformanceCounterInstanceLifetime lifetime)
   at System.Diagnostics.PerformanceCounter.InitializeImpl()
   at System.Diagnostics.PerformanceCounter.get_RawValue()
   at Microsoft.Exchange.Diagnostics.ExPerformanceCounter.get_RawValue()
Last worker process info : System.ArgumentException: Process with an Id of 14036 is not running.
   at System.Diagnostics.Process.GetProcessById(Int32 processId)
   at Microsoft.Exchange.Diagnostics.ExPerformanceCounter.GetLastWorkerProcessInfo()
Processes running while Performance counter failed to update: 
3148 MSExchangeMigrationWorkflow
4920 MSExchangeHMWorker
5248 MSExchangeSubmission
3932 noderunner
1172 taskhost
776 svchost
380 smss
2152 conhost

Any help is much appreciated!

May 7th, 2015 10:22pm

After posted the above, i ran set-ecpvirtualdirectory -Identity "ecp (default web site)" -windowsauthentication $true -formsauthentication $true on the passive copy server. Reset IIS and was able to login to ECP on the active copy server. Went to look at the DB "state" and see content index = failed:

Drives still have much free space. C drive which is where the logs are has about 100GB free and the D drive which is where the DB is has over 700GB free.

Free Windows Admin Tool Kit Click here and download it now
May 7th, 2015 10:39pm

Hi,

According to your post, I understand that cannot login ECP and DAG failure.
If I misunderstand your concern, please do not hesitate to let me know.

Please try to run below command to check the state of DAG:
Get-MailboxDatabaseCopyStatus * | ft auto
If the status of database copy in question is failed, then you have to rebuild the Index:
1. Suspend replication for the mailbox database copy on this server: Suspend-MailboxDatabaseCopy Identity <mailbox database name>\<server name>
2. Next run Update-MailboxDatabaseCopy -Identity "Mailbox Database 01\EX2" -DeleteExistingFiles to reseed the database with a new copy.

Also, please run command to double check the state of Exchange system component: Get-ServerComponentstate -Identity servername
If the state is Inactive, please run below command to active relevant component:
Set-ServerComponentState <Identity> -Component component name -Requester HealthAPI -State Active

More details about Server Component States in Exchange 2013, please refer to:
http://blogs.technet.com/b/exchange/archive/2013/09/26/server-component-states-in-exchange-2013.aspx

Thanks

May 11th, 2015 2:59am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics