- 2 nodes cluster
- 3 SQL Server instances clustered
- Virtual machines, W2K8R2 on VMWare
- SAN using iSCSI

We have one instance that will start from the services.msc (both the engine and the agent) but not from the Failover Cluster Manager when attempting to bring the service online.  In reality the services start because during the 'Online pending' I am able to connect and query the databases on that instance although it is in the 'Online pending' state.  

The error messages suggest (at least to me) that the cluster is unable to connect to the SQL Server services that are up, therefore it goes into Failed state.  I can figure out why it can't connect to it.

The error message we are getting are the following;

[sqsrvres] ODBC sqldriverconnect failed

[sqsrvres] checkODBCConnectError: sqlstate = 08001; native error = ffffffff; message = [Microsoft][SQL Server Native Client 10.0]SQL Server Network
Interfaces: Error Locating Server/Instance Specified [xFFFFFFFF].

[sqsrvres] ODBC sqldriverconnect failed

[sqsrvres] checkODBCConnectError: sqlstate = HYT00; native error = 0; message = [Microsoft][SQL Server Native Client 10.0]Login timeout expired

[sqsrvres] ODBC sqldriverconnect failed

[sqsrvres] checkODBCConnectError: sqlstate = 08001; native error = ffffffff; message = [Microsoft][SQL Server Native Client 10.0]A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online.

[sqsrvres] ODBC sqldriverconnect failed

[sqsrvres] checkODBCConnectError: sqlstate = 08001; native error = ffffffff; message = [Microsoft][SQL Server Native Client 10.0]SQL Server Network Interfaces: Error Locating Server/Instance Specified [xFFFFFFFF].

[sqsrvres] checkODBCConnectError: sqlstate = HYT00; native error = 0; message = [Microsoft][SQL Server Native Client 10.0]Login timeout expired

[sqsrvres] ODBC sqldriverconnect failed

[sqsrvres] checkODBCConnectError: sqlstate = 08001; native error = ffffffff; message = [Microsoft][SQL Server Native Client 10.0]A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online.

[sqsrvres] ODBC sqldriverconnect failed

[sqsrvres] checkODBCConnectError: sqlstate = 08001; native error = ffffffff; message = [Microsoft][SQL Server Native Client 10.0]SQL Server Network Interfaces: Error Locating Server/Instance Specified [xFFFFFFFF].

[sqsrvres] ODBC sqldriverconnect failed

[sqsrvres] checkODBCConnectError: sqlstate = HYT00; native error = 0; message = [Microsoft][SQL Server Native Client 10.0]Login timeout expired

[sqsrvres] ODBC sqldriverconnect failed

[sqsrvres] checkODBCConnectError: sqlstate = 08001; native error = ffffffff; message = [Microsoft][SQL Server Native Client 10.0]A network-related or instance-specific error has occurred while establishing a connection to SQL Server. Server is not found or not accessible. Check if instance name is correct and if SQL Server is configured to allow remote connections. For more information see SQL Server Books Online.

Fault bucket , type 0
Event Name: WSFC Resource Deadlock
Response: Not available
Cab Id: 0

Problem signature:
P1: SQL Server (MVSTG)
P2: SQL Server
P3: ONLINERESOURCE
P4:
...
P10:

Attached files:
These files may be available here:
C:\ProgramData\Microsoft\Windows\WER\ReportQueue\Critical_SQL Server (MVST_f6c6c38b8673478d1cca2c1659ac3f41af00e9_134c2c06

Analysis symbol:
Rechecking for solution: 0
Report Id: c8cfec1e-ef81-11e0-a201-0daf324d05056be606f
Report Status: 4

The content of the the file/log refered to in the Resource Deadlock message is the following:

Version=1
EventType=WSFC Resource Deadlock
EventTime=129623137530033421
ReportType=1
Consent=1
ReportIdentifier=c8cfec1e-ef81-11e0-a201-005056be606f
Response.type=4
Sig[0].Name=Resource Name
Sig[0].Value=SQL Server (MVSTG)
Sig[1].Name=Resource Type
Sig[1].Value=SQL Server
Sig[2].Name=Call Type
Sig[2].Value=ONLINERESOURCE
DynamicSig[1].Name=OS Version
DynamicSig[1].Value=6.1.7600.2.0.0.274.10
DynamicSig[2].Name=Locale ID
DynamicSig[2].Value=4105
FriendlyEventName=WSFC Resource Deadlock
ConsentKey=WSFC Resource Host Monitor
AppName=Failover Cluster Resource Host Subsystem
AppPath=C:\Windows\Cluster\rhs.exe
ReportDescription=WSFC Resource Deadlock

Can anyone help?  Thanks in advance!!!

 

 

 

What is the status of the instance on services.msc (started or starting).

Also confirm the "Startup type" of the instance.It should be manual for the clustered instance. 

What if you failover the instance to the other node? Is it runnng fine on the other node?

Please share the last SQL srver errorlog as well.

Please note for the clustered environment it is always advisible to start/stop the SQL instance from Failover Cluster Administrator(online /offline)

 

-Please mark it as answer if you are satisfied with the resolution.

 

 

 

1. The status of the instance in services.msc is started for both services, SQL Server and Agent.

2. Startup is manual on both.

3. If I failover the service does not come up on the other node either and we get the same error in the application log.

4. Here is the sql server log when attempting to start the service through the FCM.

2011-10-05 20:24:57.21 Server      Microsoft SQL Server 2008 R2 (RTM) - 10.50.1600.1 (X64)
 Apr  2 2010 15:48:46
 Copyright (c) Microsoft Corporation
 Enterprise Edition (64-bit) on Windows NT 6.1 <X64> (Build 7600: ) (Hypervisor)

2011-10-05 20:24:57.21 Server      (c) Microsoft Corporation.
2011-10-05 20:24:57.21 Server      All rights reserved.
2011-10-05 20:24:57.21 Server      Server process ID is 8644.
2011-10-05 20:24:57.21 Server      System Manufacturer: 'VMware, Inc.', System Model: 'VMware Virtual Platform'.
2011-10-05 20:24:57.21 Server      Authentication mode is MIXED.
2011-10-05 20:24:57.21 Server      Logging SQL Server messages in file 'R:\ERRORLOGS\SQLServer\ERRORLOG'.
2011-10-05 20:24:57.21 Server      This instance of SQL Server last reported using a process ID of 4512 at 05/10/2011 8:23:51 PM (local) 05/10/2011 11:23:51 PM (UTC). This is an informational message only; no user action is required.
2011-10-05 20:24:57.21 Server      Registry startup parameters:
  -d D:\MSSQL10_50.MVSTG\MSSQL\DATA\master.mdf
  -e R:\ERRORLOGS\SQLServer\ERRORLOG
  -l L:\MSSQL10_50.MVSTG\MSSQL\DATA\mastlog.ldf
2011-10-05 20:24:57.22 Server      SQL Server is starting at normal priority base (=7). This is an informational message only. No user action is required.
2011-10-05 20:24:57.22 Server      Detected 4 CPUs. This is an informational message; no user action is required.
2011-10-05 20:24:59.55 Server      Using dynamic lock allocation.  Initial allocation of 2500 Lock blocks and 5000 Lock Owner blocks per node.  This is an informational message only.  No user action is required.
2011-10-05 20:24:59.92 Server      Node configuration: node 0: CPU mask: 0x000000000000000f:0 Active CPU mask: 0x000000000000000f:0. This message provides a description of the NUMA configuration for this computer. This is an informational message only. No user action is required.
2011-10-05 20:25:00.00 spid1s      A significant part of sql server process memory has been paged out. This may result in a performance degradation. Duration: 0 seconds. Working set (KB): 9732, committed (KB): 51344, memory utilization: 18%.
2011-10-05 20:25:00.04 spid5s      Starting up database 'master'.
2011-10-05 20:25:00.14 spid5s      19 transactions rolled forward in database 'master' (1). This is an informational message only. No user action is required.
2011-10-05 20:25:00.16 spid5s      0 transactions rolled back in database 'master' (1). This is an informational message only. No user action is required.
2011-10-05 20:25:00.16 spid5s      Recovery is writing a checkpoint in database 'master' (1). This is an informational message only. No user action is required.
2011-10-05 20:25:00.46 spid5s      CHECKDB for database 'master' finished without errors on 2011-10-03 18:00:05.983 (local time). This is an informational message only; no user action is required.
2011-10-05 20:25:00.87 spid5s      Resource governor reconfiguration succeeded.
2011-10-05 20:25:00.87 spid5s      SQL Server Audit is starting the audits. This is an informational message. No user action is required.
2011-10-05 20:25:00.87 spid5s      SQL Server Audit has started the audits. This is an informational message. No user action is required.
<{39B75A24-0837-4CEC-AFDF-B960027AE07E}>RsFxNso initialized. InstanceId = 00000039
2011-10-05 20:25:00.93 spid5s      FILESTREAM: effective level = 1, configured level = 1, file system access share name = 'MVSTG'.
2011-10-05 20:25:00.97 spid5s      SQL Trace ID 1 was started by login "sa".
2011-10-05 20:25:00.99 spid5s      Starting up database 'mssqlsystemresource'.
2011-10-05 20:25:01.01 spid5s      The resource database build version is 10.50.1600. This is an informational message only. No user action is required.
2011-10-05 20:25:01.19 spid10s     Starting up database 'model'.
2011-10-05 20:25:01.19 spid5s      Server name is 'PSSQLDVCL1\MVSTG'. This is an informational message only. No user action is required.
2011-10-05 20:25:01.19 spid5s      The NETBIOS name of the local node that is running the server is 'SWV78SQLD01'. This is an informational message only. No user action is required.
2011-10-05 20:25:01.42 spid13s     A new instance of the full-text filter daemon host process has been successfully started.
2011-10-05 20:25:01.61 spid13s     Starting up database 'msdb'.
2011-10-05 20:25:01.62 spid14s     Starting up database 'MVdb'.
2011-10-05 20:25:01.65 spid15s     Starting up database 'MVAD01'.
2011-10-05 20:25:01.66 spid16s     Starting up database 'MVdb01'.
2011-10-05 20:25:01.66 spid17s     Starting up database '_MVdbMVADtest'.
2011-10-05 20:25:01.68 spid20s     Starting up database 'MVdb04'.
2011-10-05 20:25:01.69 spid19s     Starting up database 'MVdb03'.
2011-10-05 20:25:01.70 spid18s     Starting up database 'MVdb02'.
2011-10-05 20:25:01.71 Server      A self-generated certificate was successfully loaded for encryption.
2011-10-05 20:25:01.77 spid21s     Starting up database 'MVdb05'.
2011-10-05 20:25:01.77 spid22s     Starting up database '_MVdbIREtest'.
2011-10-05 20:25:01.77 spid23s     Starting up database '_MVdbBATCHtest'.
2011-10-05 20:25:01.78 spid24s     Starting up database 'ASPState'.
2011-10-05 20:25:01.78 spid25s     Starting up database 'MVPurge_dev'.
2011-10-05 20:25:01.79 spid26s     Starting up database 'MVdb06'.
2011-10-05 20:25:01.79 spid10s     CHECKDB for database 'model' finished without errors on 2011-10-03 18:00:14.950 (local time). This is an informational message only; no user action is required.
2011-10-05 20:25:01.80 spid27s     Starting up database 'MVAD01_empty'.
2011-10-05 20:25:01.80 spid10s     Clearing tempdb database.
2011-10-05 20:25:01.86 spid14s     1 transactions rolled forward in database 'MVdb' (5). This is an informational message only. No user action is required.
2011-10-05 20:25:01.86 spid16s     1 transactions rolled forward in database 'MVdb01' (7). This is an informational message only. No user action is required.
2011-10-05 20:25:01.88 spid17s     1 transactions rolled forward in database '_MVdbMVADtest' (8). This is an informational message only. No user action is required.
2011-10-05 20:25:01.93 spid24s     1 transactions rolled forward in database 'ASPState' (15). This is an informational message only. No user action is required.
2011-10-05 20:25:01.97 spid5s      0 transactions rolled back in database 'MVdb' (5). This is an informational message only. No user action is required.
2011-10-05 20:25:01.97 spid5s      Recovery is writing a checkpoint in database 'MVdb' (5). This is an informational message only. No user action is required.
2011-10-05 20:25:01.99 spid23s     1 transactions rolled forward in database '_MVdbBATCHtest' (14). This is an informational message only. No user action is required.
2011-10-05 20:25:01.99 spid25s     1 transactions rolled forward in database 'MVPurge_dev' (16). This is an informational message only. No user action is required.
2011-10-05 20:25:02.01 Server      Server is listening on [ 142.139.123.60 <ipv4> 7902].
2011-10-05 20:25:02.01 Server      Server local connection provider is ready to accept connection on [ \\.\pipe\SQLLocal\MVSTG ].
2011-10-05 20:25:02.01 Server      Server named pipe provider is ready to accept connection on [ \\.\pipe\$$\PSSQLDVCL1\MSSQL$MVSTG\sql\query ].
2011-10-05 20:25:02.05 spid14s     CHECKDB for database 'MVdb' finished without errors on 2011-10-04 08:00:02.810 (local time). This is an informational message only; no user action is required.
2011-10-05 20:25:02.06 spid16s     CHECKDB for database 'MVdb01' finished without errors on 2011-10-03 22:00:02.703 (local time). This is an informational message only; no user action is required.
2011-10-05 20:25:02.07 spid27s     1 transactions rolled forward in database 'MVAD01_empty' (18). This is an informational message only. No user action is required.
2011-10-05 20:25:02.07 spid5s      0 transactions rolled back in database 'MVdb01' (7). This is an informational message only. No user action is required.
2011-10-05 20:25:02.07 spid5s      Recovery is writing a checkpoint in database 'MVdb01' (7). This is an informational message only. No user action is required.
2011-10-05 20:25:02.08 spid14s     0 transactions rolled back in database '_MVdbMVADtest' (8). This is an informational message only. No user action is required.
2011-10-05 20:25:02.08 spid14s     Recovery is writing a checkpoint in database '_MVdbMVADtest' (8). This is an informational message only. No user action is required.
2011-10-05 20:25:02.08 spid14s     0 transactions rolled back in database 'ASPState' (15). This is an informational message only. No user action is required.
2011-10-05 20:25:02.08 spid14s     Recovery is writing a checkpoint in database 'ASPState' (15). This is an informational message only. No user action is required.
2011-10-05 20:25:02.08 Server      The SQL Server Network Interface library could not register the Service Principal Name (SPN) for the SQL Server service. Error: 0x2098, state: 15. Failure to register an SPN may cause integrated authentication to fall back to NTLM instead of Kerberos. This is an informational message. Further action is only required if Kerberos authentication is required by authentication policies.
2011-10-05 20:25:02.08 spid16s     0 transactions rolled back in database 'MVPurge_dev' (16). This is an informational message only. No user action is required.
2011-10-05 20:25:02.08 Server      SQL Server is now ready for client connections. This is an informational message; no user action is required.
2011-10-05 20:25:02.08 spid24s     CHECKDB for database 'ASPState' finished without errors on 2011-10-04 00:10:02.017 (local time). This is an informational message only; no user action is required.
2011-10-05 20:25:02.08 spid16s     Recovery is writing a checkpoint in database 'MVPurge_dev' (16). This is an informational message only. No user action is required.
2011-10-05 20:25:02.13 spid16s     0 transactions rolled back in database '_MVdbBATCHtest' (14). This is an informational message only. No user action is required.
2011-10-05 20:25:02.13 spid16s     Recovery is writing a checkpoint in database '_MVdbBATCHtest' (14). This is an informational message only. No user action is required.
2011-10-05 20:25:02.15 spid25s     0 transactions rolled back in database 'MVAD01_empty' (18). This is an informational message only. No user action is required.
2011-10-05 20:25:02.16 spid25s     Recovery is writing a checkpoint in database 'MVAD01_empty' (18). This is an informational message only. No user action is required.
2011-10-05 20:25:02.49 spid21s     1 transactions rolled forward in database 'MVdb05' (12). This is an informational message only. No user action is required.
2011-10-05 20:25:02.52 spid26s     1 transactions rolled forward in database 'MVdb06' (17). This is an informational message only. No user action is required.
2011-10-05 20:25:02.54 spid22s     1 transactions rolled forward in database '_MVdbIREtest' (13). This is an informational message only. No user action is required.
2011-10-05 20:25:02.57 spid5s      0 transactions rolled back in database 'MVdb05' (12). This is an informational message only. No user action is required.
2011-10-05 20:25:02.57 spid5s      Recovery is writing a checkpoint in database 'MVdb05' (12). This is an informational message only. No user action is required.
2011-10-05 20:25:02.57 spid21s     CHECKDB for database 'MVdb05' finished without errors on 2011-10-03 22:40:02.013 (local time). This is an informational message only; no user action is required.
2011-10-05 20:25:02.59 spid10s     Starting up database 'tempdb'.
2011-10-05 20:25:02.59 spid21s     0 transactions rolled back in database '_MVdbIREtest' (13). This is an informational message only. No user action is required.
2011-10-05 20:25:02.59 spid21s     Recovery is writing a checkpoint in database '_MVdbIREtest' (13). This is an informational message only. No user action is required.
2011-10-05 20:25:02.61 spid21s     0 transactions rolled back in database 'MVdb06' (17). This is an informational message only. No user action is required.
2011-10-05 20:25:02.61 spid21s     Recovery is writing a checkpoint in database 'MVdb06' (17). This is an informational message only. No user action is required.
2011-10-05 20:25:02.61 spid15s     1 transactions rolled forward in database 'MVAD01' (6). This is an informational message only. No user action is required.
2011-10-05 20:25:02.63 spid21s     0 transactions rolled back in database 'MVAD01' (6). This is an informational message only. No user action is required.
2011-10-05 20:25:02.63 spid21s     Recovery is writing a checkpoint in database 'MVAD01' (6). This is an informational message only. No user action is required.
2011-10-05 20:25:02.68 spid20s     1 transactions rolled forward in database 'MVdb04' (11). This is an informational message only. No user action is required.
2011-10-05 20:25:02.71 spid14s     The Service Broker protocol transport is disabled or not configured.
2011-10-05 20:25:02.71 spid14s     The Database Mirroring protocol transport is disabled or not configured.
2011-10-05 20:25:02.78 spid20s     CHECKDB for database 'MVdb04' finished without errors on 2011-10-03 22:30:02.247 (local time). This is an informational message only; no user action is required.
2011-10-05 20:25:02.80 spid20s     0 transactions rolled back in database 'MVdb04' (11). This is an informational message only. No user action is required.
2011-10-05 20:25:02.80 spid20s     Recovery is writing a checkpoint in database 'MVdb04' (11). This is an informational message only. No user action is required.
2011-10-05 20:25:02.80 spid14s     Service Broker manager has started.
2011-10-05 20:25:03.04 spid19s     1 transactions rolled forward in database 'MVdb03' (10). This is an informational message only. No user action is required.
2011-10-05 20:25:03.12 spid5s      0 transactions rolled back in database 'MVdb03' (10). This is an informational message only. No user action is required.
2011-10-05 20:25:03.12 spid5s      Recovery is writing a checkpoint in database 'MVdb03' (10). This is an informational message only. No user action is required.
2011-10-05 20:25:03.15 spid19s     CHECKDB for database 'MVdb03' finished without errors on 2011-10-03 22:20:01.417 (local time). This is an informational message only; no user action is required.
2011-10-05 20:25:03.20 spid18s     1 transactions rolled forward in database 'MVdb02' (9). This is an informational message only. No user action is required.
2011-10-05 20:25:03.30 spid18s     CHECKDB for database 'MVdb02' finished without errors on 2011-10-03 22:10:02.100 (local time). This is an informational message only; no user action is required.
2011-10-05 20:25:03.32 spid5s      0 transactions rolled back in database 'MVdb02' (9). This is an informational message only. No user action is required.
2011-10-05 20:25:03.32 spid5s      Recovery is writing a checkpoint in database 'MVdb02' (9). This is an informational message only. No user action is required.
2011-10-05 20:25:03.89 spid13s     1 transactions rolled forward in database 'msdb' (4). This is an informational message only. No user action is required.
2011-10-05 20:25:03.96 spid5s      0 transactions rolled back in database 'msdb' (4). This is an informational message only. No user action is required.
2011-10-05 20:25:03.96 spid5s      Recovery is writing a checkpoint in database 'msdb' (4). This is an informational message only. No user action is required.
2011-10-05 20:25:04.01 spid13s     CHECKDB for database 'msdb' finished without errors on 2011-10-03 18:00:15.387 (local time). This is an informational message only; no user action is required.
2011-10-05 20:25:04.04 spid5s      Recovery is complete. This is an informational message only. No user action is required.

I'm no expert yet but is it correct that it seems as though the SQL Browser is not providing the correct response to request for connections?

 

 

As indicated in the SQL error log there is no issue with the SQL service.

Try out the following:

1. Click Start, click Run, type regedit, and then click OK.
2. Locate and then click the following registry subkey:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa
3. Right-click Lsa, point to New, and then click DWORD Value. Note: In a 64 bit VM ,when you click new , DWORD(32-bit) value needs to chosen. For 32 bit VM ,only DWORD value will be shown .

 4. Type DisableLoopbackCheck, and then press ENTER. 

 5. Right-click DisableLoopbackCheck, and then click Modify.
6. In the Value data box, type 1, and then click OK. 
7. Restart the BOX.

8. Now the SQL server resource should come online after restart.

Please let me know the result.

-Please mark it as answer if you are satisfied with the resolution.

 

 

Thanks Deepesh,

1. The registry entry was already present with the correct value on node 1, however it was missing on node 2.

2. I added the entry on node 2 with the value you specified, rebooted the node but the services still won't start.

3. The series of errors/entries (in Application Log & SQL Server log) resulting from attempting to start the service from the FCM are the same as in my first post.

Where would you look next?

Thanks again!

Few more questions:

Is it a new instance that you are currently installing? or an exsisting one which is running fine earlier.

Please do a configuration check from the FCM of this instance with the rest of the instances which are running fine.

Please also upload the system and application logs to some fileshare.

 

-Please mark it as answer if you are satisfied with the resolution.

 

Deepesh,

This is an existing instance that has worked perfectly for about a year.

I have actually found the problem and resolved it.  About 6 months ago I have set the 'Hide instance' option to Yes.  Everything worked perfectly fine, failovers were working flawlessly and startups as well. 

In my fact finding it was clear that the problem was that the cluster service (I assume here) or something other was not able to find the instance once it was up and running in order to confirm it was alive, then would go into a failed state.  I then decided to try changing this setting to No and the instance came up through FCM without any problems.

What I don't understand is why this worked before and quite suddenly stopped working.  Does someone have addtional information on this 'Hide Instance' feature and how it works?  Although my instances are up I have to get to the bottom of this and undestand what happened.  Any information on this would be great.

Thank you Deepesh for answers and guidance, it was much appreciated.

• Marked as answer by Beneque79 Thursday, October 06, 2011 4:41 PM

Deepesh,

This is an existing instance that has worked perfectly for about a year.

I have actually found the problem and resolved it.  About 6 months ago I have set the 'Hide instance' option to Yes.  Everything worked perfectly fine, failovers were working flawlessly and startups as well. 

In my fact finding it was clear that the problem was that the cluster service (I assume here) or something other was not able to find the instance once it was up and running in order to confirm it was alive, then would go into a failed state.  I then decided to try changing this setting to No and the instance came up through FCM without any problems.

What I don't understand is why this worked before and quite suddenly stopped working.  Does someone have addtional information on this 'Hide Instance' feature and how it works?  Although my instances are up I have to get to the bottom of this and undestand what happened.  Any information on this would be great.

Thank you Deepesh for answers and guidance, it was much appreciated.

• Marked as answer by Beneque79 Thursday, October 06, 2011 4:41 PM

I had this same problem after running the CU 7 for SQL Server 2008 R2 SP1.  I couldn't start my clustered named instance until I turned off hide instance... Anyone know why??

Maybe because its hidden from the Browser Service.

Note, Cluster Service will attempt to connect to the SQL service every few minutes (setting in SQL cluster resource) for the IsAlive check, if this fails then the SQL resource is restarted even if the instance was online.

There was an issue with the Native Client which was fixed, e.g. older version SQL 2005 against SQL 2008 native client.

Create a test ODBC connection on the cluster node and see if it works that way.

Answer was Firewall..

http://blogs.msdn.com/b/sql_protocols/archive/2008/06/17/troubleshooting-sql-server-cluster-installations.aspx

I've experienced this a few times after removing permissions from public.   I found the following error in the Cluster Log: "The EXECUTE permission was denied on the object 'sp_server_diagnostics', database 'mssqlsystemresource', schema 'sys'. (229)".  I started SQL Server from Services on the active node and granted Execute to 'sp_server_diagnostics'.  This resolved my issue and allowed SQL Server to come online and failover successfully.