Hi experts,
I found one database in availability group state change automatically. Please help me to identify the source of the problem. Thanks in advanced.
I found ERR started at
0000233c.00002848::2015/09/11-03:51:09.840 ERR [RES] SQL Server Availability Group: [hadrag] ODBC Error: [HYT00] [Microsoft][SQL Server Native Client 11.0]Query timeout expired (0)
and I use goole to search and found the link below. It indicated a network issue. Is it the source of the problem. By the way, My failure Codition Level is 1 and I have increased Health Check Timeout from 30,000ms to 150,000ms just now.
--- SQLDIAG ---
cluster log in UTC+8:
0000233c.00002848::2015/09/11-03:51:09.840 INFO [RES] SQL Server Availability Group: [hadrag] SQLMoreResults() returns -1 with following information
0000233c.00002848::2015/09/11-03:51:09.840 ERR [RES] SQL Server Availability Group: [hadrag] ODBC Error: [HYT00] [Microsoft][SQL Server Native Client 11.0]Query timeout expired (0)
0000233c.00002848::2015/09/11-03:51:09.840 ERR [RES] SQL Server Availability Group: [hadrag] ODBC Error: [01000] [Microsoft][SQL Server Native Client 11.0][SQL Server] (0)
0000233c.00002848::2015/09/11-03:51:09.840 INFO [RES] SQL Server Availability Group: [hadrag] No more diagnostics results
0000233c.00002848::2015/09/11-03:51:09.840 INFO [RES] SQL Server Availability Group: [hadrag] Discard the pending result sets
0000233c.00002848::2015/09/11-03:51:09.840 ERR [RES] SQL Server Availability Group: [hadrag] ODBC Error: [24000] [Microsoft][SQL Server Native Client 11.0]Invalid cursor state (0)
0000233c.00002848::2015/09/11-03:51:09.840 INFO [RES] SQL Server Availability Group: [hadrag] Diagnostics is stopped
0000233c.00003250::2015/09/11-03:51:09.840 INFO [RES] SQL Server Availability Group: [hadrag] Change diagnostics interval worker is stopped
0000233c.00002848::2015/09/11-03:51:09.840 INFO [RES] SQL Server Availability Group: [hadrag] Disconnect from SQL Server
0000233c.00002848::2015/09/11-03:51:10.854 INFO [RES] SQL Server Availability Group: [hadrag] Connect to SQL Server ...
0000233c.000047c8::2015/09/11-03:51:11.525 ERR [RHS] s_RhsRpcCreateResType: ERROR_NOT_READY(21)' because of 'Startup routine for ResType MSMQ returned 21.'
00001760.000018f8::2015/09/11-03:51:11.525 INFO [RCM] result of first load attempt for type MSMQ: 21
00001760.000018f8::2015/09/11-03:51:11.525 WARN [RCM] Failed to load restype 'MSMQ': error 21.
0000233c.000047c8::2015/09/11-03:51:11.525 ERR [RHS] s_RhsRpcCreateResType: ERROR_NOT_READY(21)' because of 'Startup routine for ResType MSMQTriggers returned 21.'
00001760.000018f8::2015/09/11-03:51:11.525 INFO [RCM] result of first load attempt for type MSMQTriggers: 21
00001760.000018f8::2015/09/11-03:51:11.525 WARN [RCM] Failed to load restype 'MSMQTriggers': error 21.
00001760.000018f8::2015/09/11-03:51:11.525 ERR [RCM] [GIM] ResType Virtual Machine has no resources, not collecting local utilization info
00001760.000018f8::2015/09/11-03:51:11.525 INFO [RCM] [GIM] Scheduling Local Node Crawler to run in 300000 millisec.
0000233c.00002848::2015/09/11-03:51:11.899 INFO [RES] SQL Server Availability Group: [hadrag] The connection was established successfully
0000233c.000047c8::2015/09/11-03:51:12.929 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.00004028::2015/09/11-03:51:13.194 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.000036f0::2015/09/11-03:51:13.194 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.000036f0::2015/09/11-03:51:13.194 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.000036f0::2015/09/11-03:51:13.194 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.0000442c::2015/09/11-03:51:13.225 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.000036f0::2015/09/11-03:51:13.225 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.000036f0::2015/09/11-03:51:13.225 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.000036f0::2015/09/11-03:51:13.225 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
0000233c.000047c8::2015/09/11-03:51:17.937 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.0000442c::2015/09/11-03:51:18.202 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.000036f0::2015/09/11-03:51:18.202 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.000036f0::2015/09/11-03:51:18.202 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.000036f0::2015/09/11-03:51:18.202 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.00003d7c::2015/09/11-03:51:18.233 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.00004028::2015/09/11-03:51:18.233 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00004028::2015/09/11-03:51:18.233 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00004028::2015/09/11-03:51:18.233 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00001760.00003bf8::2015/09/11-03:51:18.608 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003bf8::2015/09/11-03:51:18.608 INFO [API] s_ApiGetQuorumResource final status 0.
0000233c.000047c8::2015/09/11-03:51:22.945 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.00004028::2015/09/11-03:51:23.210 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.00003d7c::2015/09/11-03:51:23.210 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00003d7c::2015/09/11-03:51:23.210 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00003d7c::2015/09/11-03:51:23.210 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.00004028::2015/09/11-03:51:23.242 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.00003d7c::2015/09/11-03:51:23.242 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00003d7c::2015/09/11-03:51:23.242 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00003d7c::2015/09/11-03:51:23.242 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00001760.00003d14::2015/09/11-03:51:26.502 INFO [NM] Received request from client address DL980-3.
00001760.000018f8::2015/09/11-03:51:26.502 INFO [NM] Received request from client address DL980-3.
00001760.00003bf8::2015/09/11-03:51:26.502 INFO [NM] Received request from client address DL980-3.
0000233c.000047c8::2015/09/11-03:51:27.953 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.00004028::2015/09/11-03:51:28.218 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.000036f0::2015/09/11-03:51:28.218 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.000036f0::2015/09/11-03:51:28.218 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.000036f0::2015/09/11-03:51:28.218 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.00003d7c::2015/09/11-03:51:28.250 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.000036f0::2015/09/11-03:51:28.250 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.000036f0::2015/09/11-03:51:28.250 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.000036f0::2015/09/11-03:51:28.250 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00001760.000018f8::2015/09/11-03:51:29.357 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:51:29.357 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:51:29.545 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:51:29.545 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:51:29.545 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:51:29.545 INFO [API] s_ApiGetQuorumResource final status 0.
0000233c.00001b04::2015/09/11-03:51:32.961 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.000036f0::2015/09/11-03:51:33.226 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.00003d7c::2015/09/11-03:51:33.226 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00003d7c::2015/09/11-03:51:33.226 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00003d7c::2015/09/11-03:51:33.226 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.000036f0::2015/09/11-03:51:33.258 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.00003d7c::2015/09/11-03:51:33.258 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00003d7c::2015/09/11-03:51:33.258 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00003d7c::2015/09/11-03:51:33.258 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00001760.000010b8::2015/09/11-03:51:36.066 INFO [NM] Received request from client address DL980-3.
00001760.000018f8::2015/09/11-03:51:36.955 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:51:36.955 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000010b8::2015/09/11-03:51:37.158 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000010b8::2015/09/11-03:51:37.158 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000010b8::2015/09/11-03:51:37.158 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000010b8::2015/09/11-03:51:37.158 INFO [API] s_ApiGetQuorumResource final status 0.
0000233c.000047c8::2015/09/11-03:51:37.969 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.000036f0::2015/09/11-03:51:38.234 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.00003d7c::2015/09/11-03:51:38.234 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00003d7c::2015/09/11-03:51:38.234 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00003d7c::2015/09/11-03:51:38.234 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.0000442c::2015/09/11-03:51:38.266 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.000036f0::2015/09/11-03:51:38.266 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.000036f0::2015/09/11-03:51:38.266 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.000036f0::2015/09/11-03:51:38.266 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
0000233c.00001b04::2015/09/11-03:51:42.977 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.00004028::2015/09/11-03:51:43.180 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.0000442c::2015/09/11-03:51:43.180 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.0000442c::2015/09/11-03:51:43.180 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.0000442c::2015/09/11-03:51:43.180 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.0000442c::2015/09/11-03:51:43.242 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.00004028::2015/09/11-03:51:43.242 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00004028::2015/09/11-03:51:43.242 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00004028::2015/09/11-03:51:43.242 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00001760.00003830::2015/09/11-03:51:45.395 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:51:45.395 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:51:45.411 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:51:45.411 INFO [API] s_ApiGetQuorumResource final status 0.
0000233c.00002848::2015/09/11-03:51:45.411 INFO [RES] SQL Server Availability Group: [hadrag] Run 'EXEC sp_server_diagnostics 10' returns following information
0000233c.00002848::2015/09/11-03:51:45.411 ERR [RES] SQL Server Availability Group: [hadrag] ODBC Error: [HYT00] [Microsoft][SQL Server Native Client 11.0]Query timeout expired (0)
0000233c.00002848::2015/09/11-03:51:45.411 ERR [RES] SQL Server Availability Group: [hadrag] Failed to run diagnostics command. See previous log for error message
0000233c.00002848::2015/09/11-03:51:45.411 INFO [RES] SQL Server Availability Group: [hadrag] Disconnect from SQL Server
00001760.00003830::2015/09/11-03:51:45.427 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:51:45.427 INFO [API] s_ApiGetQuorumResource final status 0.
0000233c.000047c8::2015/09/11-03:51:47.985 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.000036f0::2015/09/11-03:51:48.188 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.00004028::2015/09/11-03:51:48.188 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00004028::2015/09/11-03:51:48.188 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00004028::2015/09/11-03:51:48.188 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.00004028::2015/09/11-03:51:48.250 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.000036f0::2015/09/11-03:51:48.250 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.000036f0::2015/09/11-03:51:48.250 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.000036f0::2015/09/11-03:51:48.250 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00001760.000010b8::2015/09/11-03:51:48.282 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000010b8::2015/09/11-03:51:48.282 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000010b8::2015/09/11-03:51:48.297 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000010b8::2015/09/11-03:51:48.297 INFO [API] s_ApiGetQuorumResource final status 0.
0000233c.00002848::2015/09/11-03:51:50.419 INFO [RES] SQL Server Availability Group: [hadrag] Connect to SQL Server ...
0000233c.00002848::2015/09/11-03:51:51.605 INFO [RES] SQL Server Availability Group: [hadrag] The connection was established successfully
0000233c.000047c8::2015/09/11-03:51:52.993 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.0000442c::2015/09/11-03:51:53.196 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.00003d7c::2015/09/11-03:51:53.196 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00003d7c::2015/09/11-03:51:53.196 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00003d7c::2015/09/11-03:51:53.196 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.00003d7c::2015/09/11-03:51:53.258 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.0000442c::2015/09/11-03:51:53.258 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.0000442c::2015/09/11-03:51:53.258 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.0000442c::2015/09/11-03:51:53.258 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
0000233c.000047c8::2015/09/11-03:51:57.907 INFO [RES] SAP Resource <SAP TCP 00 Instance>: IsAlive request.
00002338.00004028::2015/09/11-03:51:58.204 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.00003d7c::2015/09/11-03:51:58.204 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00003d7c::2015/09/11-03:51:58.204 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00003d7c::2015/09/11-03:51:58.204 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.00003d7c::2015/09/11-03:51:58.266 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.00004028::2015/09/11-03:51:58.266 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00004028::2015/09/11-03:51:58.266 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00004028::2015/09/11-03:51:58.266 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00001760.000010b8::2015/09/11-03:52:02.073 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000010b8::2015/09/11-03:52:02.073 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:02.073 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:02.073 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:02.073 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:02.073 INFO [API] s_ApiGetQuorumResource final status 0.
0000233c.00002848::2015/09/11-03:52:02.089 INFO [RES] SQL Server Availability Group: [hadrag] Diagnostics is started
0000233c.00003d08::2015/09/11-03:52:02.089 INFO [RES] SQL Server Availability Group: [hadrag] Change diagnostics interval worker is started
0000233c.00002848::2015/09/11-03:52:02.089 INFO [RES] SQL Server Availability Group: [hadrag] SQL Server component 'system' health state has been changed from '' to 'warning' at 2015-09-11 11:52:02.077
0000233c.00001b04::2015/09/11-03:52:02.104 INFO [RES] SQL Server Availability Group: [hadrag] Availability Group with resource ID '0c81ffc5-c2fe-4beb-8fa1-0e2d9df8f6f5' did not receive healthinformation before HealthCheckTimeout
0000233c.00001b04::2015/09/11-03:52:02.104 ERR [RES] SQL Server Availability Group <AGTCP>: [hadrag] Availability Group is not healthy with given HealthCheckTimeout and FailureConditionLevel
0000233c.00001b04::2015/09/11-03:52:02.104 ERR [RES] SQL Server Availability Group <AGTCP>: [hadrag] Resource Alive result 0.
0000233c.00001b04::2015/09/11-03:52:02.104 INFO [RES] SQL Server Availability Group: [hadrag] Availability Group with resource ID '0c81ffc5-c2fe-4beb-8fa1-0e2d9df8f6f5' did not receive healthinformation before HealthCheckTimeout
0000233c.00001b04::2015/09/11-03:52:02.104 ERR [RES] SQL Server Availability Group <AGTCP>: [hadrag] Availability Group is not healthy with given HealthCheckTimeout and FailureConditionLevel
0000233c.00001b04::2015/09/11-03:52:02.104 ERR [RES] SQL Server Availability Group <AGTCP>: [hadrag] Resource Alive result 0.
0000233c.00001b04::2015/09/11-03:52:02.104 WARN [RHS] Resource AGTCP IsAlive has indicated failure.
00001760.000038f4::2015/09/11-03:52:02.104 INFO [RCM] HandleMonitorReply: FAILURENOTIFICATION for 'AGTCP', gen(0) result 1/0.
00001760.000038f4::2015/09/11-03:52:02.104 INFO [RCM] Res AGTCP: Online -> ProcessingFailure( StateUnknown )
00001760.000038f4::2015/09/11-03:52:02.104 INFO [RCM] TransitionToState(AGTCP) Online-->ProcessingFailure.
00001760.00003830::2015/09/11-03:52:02.104 INFO [GEM] Sending 1 messages as a batched GEM message
00001760.000038f4::2015/09/11-03:52:02.104 INFO [RCM] rcm::RcmGroup::UpdateStateIfChanged: (AGTCP, Online --> Pending)
00001760.000038f4::2015/09/11-03:52:02.104 ERR [RCM] rcm::RcmResource::HandleFailure: (AGTCP)
00001760.000038f4::2015/09/11-03:52:02.104 INFO [RCM] resource AGTCP: failure count: 0, restartAction: 0 persistentState: 1.
00001760.000038f4::2015/09/11-03:52:02.104 INFO [RCM] numDependents is zero, auto-returning true
00001760.000038f4::2015/09/11-03:52:02.104 INFO [RCM] Will queue immediate restart (500 milliseconds) of AGTCP after terminate is complete.
00001760.000038f4::2015/09/11-03:52:02.104 INFO [RCM] Res AGTCP: ProcessingFailure -> WaitingToTerminate( DelayRestartingResource )
00001760.000038f4::2015/09/11-03:52:02.104 INFO [RCM] TransitionToState(AGTCP) ProcessingFailure-->[WaitingToTerminate to DelayRestartingResource].
00001760.000038f4::2015/09/11-03:52:02.104 INFO [RCM] Res AGTCP: [WaitingToTerminate to DelayRestartingResource] -> Terminating( DelayRestartingResource )
00001760.000038f4::2015/09/11-03:52:02.104 INFO [RCM] TransitionToState(AGTCP) [WaitingToTerminate to DelayRestartingResource]-->[Terminating to DelayRestartingResource].
00001760.000021b4::2015/09/11-03:52:02.104 INFO [GEM] Sending 1 messages as a batched GEM message
0000233c.00001b04::2015/09/11-03:52:02.104 INFO [RES] SQL Server Availability Group: [hadrag] Stopping Health Worker Thread
0000233c.00003d08::2015/09/11-03:52:02.104 INFO [RES] SQL Server Availability Group: [hadrag] Health worker was asked to terminate
00002338.00004028::2015/09/11-03:52:02.104 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.00003d7c::2015/09/11-03:52:02.104 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
0000233c.00002848::2015/09/11-03:52:02.104 WARN [RES] SQL Server Availability Group: [hadrag] Failed to retrieve data column. Return code -1
0000233c.00003d08::2015/09/11-03:52:02.104 INFO [RES] SQL Server Availability Group: [hadrag] Change diagnostics interval worker is stopped
0000233c.00002848::2015/09/11-03:52:02.104 ERR [RES] SQL Server Availability Group: [hadrag] ODBC Error: [HY008] [Microsoft][SQL Server Native Client 11.0]Operation canceled (0)
0000233c.00002848::2015/09/11-03:52:02.104 INFO [RES] SQL Server Availability Group: [hadrag] Diagnostics is stopped
0000233c.00002848::2015/09/11-03:52:02.104 INFO [RES] SQL Server Availability Group: [hadrag] Disconnect from SQL Server
0000233c.00002848::2015/09/11-03:52:02.104 INFO [RES] SQL Server Availability Group: [hadrag] Extended Event logging is stopped
0000233c.00002848::2015/09/11-03:52:02.104 INFO [RES] SQL Server Availability Group: [hadrag] Extended Event target state:
0000233c.00002848::2015/09/11-03:52:02.104 INFO [RES] SQL Server Availability Group: [hadrag] Extended Event session summary: dropped buffers = 0, dropped events = 0
0000233c.00001b04::2015/09/11-03:52:02.104 INFO [RES] SQL Server Availability Group: [hadrag] Stopping Change Diagnostics interval Worker Thread
0000233c.00001b04::2015/09/11-03:52:02.120 INFO [RES] SQL Server Availability Group <AGTCP>: [hadrag] Connect to SQL Server ...
00001760.00002008::2015/09/11-03:52:02.120 INFO [RCM] ignored non-local state Pending for group AGTCP
00001760.00003830::2015/09/11-03:52:02.120 INFO [NM] Received request from client address DL980-3.
00001760.000010b8::2015/09/11-03:52:02.120 INFO [NM] Received request from client address DL980-3.
00002338.000036f0::2015/09/11-03:52:02.120 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.00004028::2015/09/11-03:52:02.120 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
0000233c.00001b04::2015/09/11-03:52:02.135 INFO [RES] SQL Server Availability Group <AGTCP>: [hadrag] The connection was established successfully
00002338.000036f0::2015/09/11-03:52:02.135 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.00003d7c::2015/09/11-03:52:02.135 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
0000233c.00001b04::2015/09/11-03:52:02.198 INFO [RES] SQL Server Availability Group <AGTCP>: [hadrag] Disconnect from SQL Server
0000233c.00001b04::2015/09/11-03:52:02.198 INFO [RES] SQL Server Availability Group <AGTCP>: [hadrag] Offline call successful in Terminate function
00001760.000038f4::2015/09/11-03:52:02.198 INFO [RCM] HandleMonitorReply: TERMINATERESOURCE for 'AGTCP', gen(1) result 0/0.
00001760.000038f4::2015/09/11-03:52:02.198 INFO [RCM] Res AGTCP: [Terminating to DelayRestartingResource] -> DelayRestartingResource( StateUnknown )
00001760.000038f4::2015/09/11-03:52:02.198 INFO [RCM] TransitionToState(AGTCP) [Terminating to DelayRestartingResource]-->DelayRestartingResource.
00001760.000038f4::2015/09/11-03:52:02.198 WARN [RCM] Queueing immediate delay restart of resource AGTCP in 500 ms.
00001760.000010b8::2015/09/11-03:52:02.276 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.0000460c::2015/09/11-03:52:02.276 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000038f4::2015/09/11-03:52:02.713 INFO [RCM] Delay-restarting AGTCP and any waiting dependents.
00001760.000038f4::2015/09/11-03:52:02.713 INFO [RCM-rbtr] giving default token to group AGTCP
00001760.000038f4::2015/09/11-03:52:02.713 INFO [RCM-rbtr] giving default token to group AGTCP
00001760.000038f4::2015/09/11-03:52:02.713 INFO [RCM] Res AGTCP: DelayRestartingResource -> OnlineCallIssued( StateUnknown )
00001760.000038f4::2015/09/11-03:52:02.713 INFO [RCM] TransitionToState(AGTCP) DelayRestartingResource-->OnlineCallIssued.
00001760.0000460c::2015/09/11-03:52:02.713 INFO [GEM] Sending 1 messages as a batched GEM message
00001760.00003d14::2015/09/11-03:52:02.713 INFO [RCM] HandleMonitorReply: ONLINERESOURCE for 'AGTCP', gen(1) result 997/0.
00001760.00003d14::2015/09/11-03:52:02.713 INFO [RCM] Res AGTCP: OnlineCallIssued -> OnlinePending( StateUnknown )
00001760.00003d14::2015/09/11-03:52:02.713 INFO [RCM] TransitionToState(AGTCP) OnlineCallIssued-->OnlinePending.
00001760.000021b4::2015/09/11-03:52:02.713 INFO [GEM] Sending 1 messages as a batched GEM message
0000233c.00000e98::2015/09/11-03:52:02.713 INFO [RES] SQL Server Availability Group <AGTCP>: [hadrag] The DeadLockTimeout property has a value of 300000
0000233c.00000e98::2015/09/11-03:52:02.713 INFO [RES] SQL Server Availability Group <AGTCP>: [hadrag] The PendingTimeout property has a value of 180000
0000233c.00000e98::2015/09/11-03:52:02.713 INFO [RES] SQL Server Availability Group <AGTCP>: [hadrag] Connect to SQL Server ...
00002338.000036f0::2015/09/11-03:52:02.713 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.00003d7c::2015/09/11-03:52:02.713 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
0000233c.00000e98::2015/09/11-03:52:02.728 INFO [RES] SQL Server Availability Group <AGTCP>: [hadrag] The connection was established successfully
0000233c.00000e98::2015/09/11-03:52:02.728 INFO [RES] SQL Server Availability Group <AGTCP>: [hadrag] Current SQL Instance is not part of Failover clustering
0000233c.00000e98::2015/09/11-03:52:02.728 INFO [RES] SQL Server Availability Group: [hadrag] Starting Health Worker Thread
0000233c.0000459c::2015/09/11-03:52:02.744 INFO [RES] SQL Server Availability Group: [hadrag] XEvent session MSSQLSERVER is created with RolloverCount 10, MaxFileSizeInMBytes 100, and LogPath 'C:\Program Files\Microsoft SQL Server\MSSQL11.MSSQLSERVER\MSSQL\LOG\'
0000233c.0000459c::2015/09/11-03:52:02.744 INFO [RES] SQL Server Availability Group: [hadrag] Extended Event logging is started
0000233c.0000459c::2015/09/11-03:52:02.744 INFO [RES] SQL Server Availability Group: [hadrag] Health worker started for instance DL980-3
0000233c.0000459c::2015/09/11-03:52:02.744 INFO [RES] SQL Server Availability Group: [hadrag] Connect to SQL Server ...
00002338.000036f0::2015/09/11-03:52:02.744 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
0000233c.0000459c::2015/09/11-03:52:02.744 INFO [RES] SQL Server Availability Group: [hadrag] The connection was established successfully
00002338.00003d7c::2015/09/11-03:52:02.744 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
0000233c.0000459c::2015/09/11-03:52:02.759 INFO [RES] SQL Server Availability Group: [hadrag] Diagnostics is started
0000233c.0000459c::2015/09/11-03:52:02.759 INFO [RES] SQL Server Availability Group: [hadrag] SQL Server component 'system' health state has been changed from '' to 'warning' at 2015-09-11 11:52:02.747
0000233c.000030f8::2015/09/11-03:52:02.759 INFO [RES] SQL Server Availability Group: [hadrag] Change diagnostics interval worker is started
0000233c.0000459c::2015/09/11-03:52:02.775 INFO [RES] SQL Server Availability Group: [hadrag] SQL Server component 'resource' health state has been changed from '' to 'clean' at 2015-09-11 11:52:02.747
0000233c.0000459c::2015/09/11-03:52:02.775 INFO [RES] SQL Server Availability Group: [hadrag] SQL Server component 'query_processing' health state has been changed from '' to 'warning' at 2015-09-11 11:52:02.747
0000233c.0000459c::2015/09/11-03:52:02.775 INFO [RES] SQL Server Availability Group: [hadrag] SQL Server component 'io_subsystem' health state has been changed from '' to 'clean' at 2015-09-11 11:52:02.747
0000233c.0000459c::2015/09/11-03:52:02.775 INFO [RES] SQL Server Availability Group: [hadrag] SQL Server component 'events' health state has been changed from '' to 'unknown' at 2015-09-11 11:52:02.747
0000233c.00004230::2015/09/11-03:52:02.915 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.0000442c::2015/09/11-03:52:03.212 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.00004028::2015/09/11-03:52:03.212 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00004028::2015/09/11-03:52:03.212 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00004028::2015/09/11-03:52:03.212 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.00004028::2015/09/11-03:52:03.274 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.0000442c::2015/09/11-03:52:03.274 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.0000442c::2015/09/11-03:52:03.274 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.0000442c::2015/09/11-03:52:03.274 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.0000442c::2015/09/11-03:52:03.586 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.00004028::2015/09/11-03:52:03.586 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00001760.000010b8::2015/09/11-03:52:05.224 INFO [GEM] Sending 1 messages as a batched GEM message
00001760.000010b8::2015/09/11-03:52:05.224 INFO [GUM] Node 1: Processing RequestLock 1:21794
00001760.000021b4::2015/09/11-03:52:05.224 INFO [GUM] Node 1: Processing GrantLock to 1 (sent by 2 gumid: 150394)
00001760.000021b4::2015/09/11-03:52:05.224 INFO [GEM] Sending 1 messages as a batched GEM message
000024a8.0000417c::2015/09/11-03:52:05.833 INFO [RES] Physical Disk <Cluster Disk 1>: VolumeIsNtfs: Volume
\\?\GLOBALROOT\Device\Harddisk13\ClusterPartition2\ has FS type NTFS
00001760.000010b8::2015/09/11-03:52:06.441 INFO [GEM] Sending 1 messages as a batched GEM message
00002338.00004028::2015/09/11-03:52:06.441 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00001760.00003d14::2015/09/11-03:52:06.441 INFO [GEM] Sending 1 messages as a batched GEM message
00002338.0000442c::2015/09/11-03:52:06.457 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00001760.00003d14::2015/09/11-03:52:06.457 INFO [GEM] Sending 1 messages as a batched GEM message
0000233c.00000e98::2015/09/11-03:52:06.457 INFO [RES] SQL Server Availability Group <AGTCP>: [hadrag] Disconnect from SQL Server
00001760.00004390::2015/09/11-03:52:06.457 INFO [RCM] rcm::RcmApi::OnlineGroup: (AGTCP, 0)
00001760.00004390::2015/09/11-03:52:06.457 INFO [RCM] rcm::RcmGum::SetGroupPersistentState(AGTCP,1)
00001760.00004390::2015/09/11-03:52:06.457 INFO [GEM] Sending 1 messages as a batched GEM message
0000233c.00000e98::2015/09/11-03:52:06.457 INFO [RHS] Resource AGTCP has come online. RHS is about to report status change to RCM
00001760.0000460c::2015/09/11-03:52:06.457 INFO [RCM] HandleMonitorReply: ONLINERESOURCE for 'AGTCP', gen(1) result 0/0.
00001760.0000460c::2015/09/11-03:52:06.457 INFO [RCM] Res AGTCP: OnlinePending -> Online( StateUnknown )
00001760.0000460c::2015/09/11-03:52:06.457 INFO [RCM] TransitionToState(AGTCP) OnlinePending-->Online.
00001760.0000460c::2015/09/11-03:52:06.457 INFO [RCM] rcm::RcmGroup::UpdateStateIfChanged: (AGTCP, Pending --> Online)
00001760.000010b8::2015/09/11-03:52:06.457 INFO [GEM] Sending 1 messages as a batched GEM message
00001760.00003d14::2015/09/11-03:52:06.457 INFO [RCM] moved 0 tasks from staging set to task set. TaskSetSize=0
00001760.00003d14::2015/09/11-03:52:06.457 INFO [RCM] rcm::RcmPriorityManager::StartGroups: [RCM] done, executed 0 tasks
00001760.000021b4::2015/09/11-03:52:06.457 INFO [GEM] Sending 1 messages as a batched GEM message
00001760.00002008::2015/09/11-03:52:06.457 INFO [RCM] ignored non-local state Online for group AGTCP
00001760.00004390::2015/09/11-03:52:06.472 INFO [RCM] Group AGTCP: done going through resources, returning true
00002338.00004028::2015/09/11-03:52:06.488 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.0000442c::2015/09/11-03:52:06.488 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00001760.000038f4::2015/09/11-03:52:06.519 INFO [NM] Received request from client address DL980-3.
00002338.00004028::2015/09/11-03:52:06.535 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.0000442c::2015/09/11-03:52:06.535 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.00004028::2015/09/11-03:52:06.550 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.0000442c::2015/09/11-03:52:06.566 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.00004028::2015/09/11-03:52:06.566 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.000036f0::2015/09/11-03:52:06.582 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.00004028::2015/09/11-03:52:06.597 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.000036f0::2015/09/11-03:52:06.597 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.00004028::2015/09/11-03:52:06.628 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.000036f0::2015/09/11-03:52:06.628 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.00004028::2015/09/11-03:52:06.660 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.000036f0::2015/09/11-03:52:06.675 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.00004028::2015/09/11-03:52:06.706 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.000036f0::2015/09/11-03:52:06.706 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00001760.000018f8::2015/09/11-03:52:07.003 INFO [RCM] rcm::RcmApi::AddPossibleOwner: (AGTCP, 1)
00001760.000018f8::2015/09/11-03:52:07.003 INFO [RCM] rcm::RcmGum::AddPossibleOwner(AGTCP,1)
00001760.000018f8::2015/09/11-03:52:07.128 ERR [GUM] Node 1: Local Execution of a gum request /rcm/gum/AddPossibleOwner resulted in exception ERROR_OBJECT_ALREADY_EXISTS(5010)' because of 'The specified node is already a possible owner.'
00001760.000018f8::2015/09/11-03:52:07.128 ERR [RCM] rcm::RcmApi::AddPossibleOwner: ERROR_OBJECT_ALREADY_EXISTS(5010)' because of 'The specified node is already a possible owner.'
00001760.000018f8::2015/09/11-03:52:07.128 INFO [RCM] rcm::RcmApi::RemovePossibleOwner: (AGTCP, 2)
00001760.000018f8::2015/09/11-03:52:07.128 INFO [RCM] rcm::RcmGum::RemovePossibleOwner(AGTCP,2)
00001760.000018f8::2015/09/11-03:52:07.128 INFO [RCM] Removing node 2 from resource 'AGTCP'.
00001760.000018f8::2015/09/11-03:52:07.128 INFO [GEM] Sending 1 messages as a batched GEM message
00002338.00004028::2015/09/11-03:52:07.128 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00001760.000018f8::2015/09/11-03:52:07.128 INFO [RCM] rcm::RcmApi::AddPossibleOwner: (AGTCP, 2)
00001760.000018f8::2015/09/11-03:52:07.128 INFO [RCM] rcm::RcmGum::AddPossibleOwner(AGTCP,2)
00001760.000018f8::2015/09/11-03:52:07.128 INFO [RCM] Adding node 2 to resource 'AGTCP'.
00001760.000018f8::2015/09/11-03:52:07.128 INFO [GEM] Sending 1 messages as a batched GEM message
00002338.000036f0::2015/09/11-03:52:07.128 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00001760.000018f8::2015/09/11-03:52:07.143 INFO [RCM] rcm::RcmGum::SetGroupPreferredOwners(AGTCP,<vector len='2'>
00001760.000018f8::2015/09/11-03:52:07.143 INFO <item>1</item>
00001760.000018f8::2015/09/11-03:52:07.143 INFO <item>2</item>
00001760.000018f8::2015/09/11-03:52:07.143 INFO </vector>
00001760.000018f8::2015/09/11-03:52:07.143 INFO )
00001760.000018f8::2015/09/11-03:52:07.143 INFO [GEM] Sending 1 messages as a batched GEM message
00002338.0000442c::2015/09/11-03:52:07.159 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
00002338.000036f0::2015/09/11-03:52:07.159 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
0000233c.00001b04::2015/09/11-03:52:07.923 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.0000442c::2015/09/11-03:52:08.189 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.000036f0::2015/09/11-03:52:08.189 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.000036f0::2015/09/11-03:52:08.189 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.000036f0::2015/09/11-03:52:08.189 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.0000442c::2015/09/11-03:52:08.220 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.000036f0::2015/09/11-03:52:08.220 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.000036f0::2015/09/11-03:52:08.220 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.000036f0::2015/09/11-03:52:08.220 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
0000233c.00001b04::2015/09/11-03:52:12.931 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.0000442c::2015/09/11-03:52:13.197 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.000036f0::2015/09/11-03:52:13.197 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.000036f0::2015/09/11-03:52:13.197 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.000036f0::2015/09/11-03:52:13.197 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.0000442c::2015/09/11-03:52:13.228 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.000036f0::2015/09/11-03:52:13.228 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.000036f0::2015/09/11-03:52:13.228 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.000036f0::2015/09/11-03:52:13.228 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
0000233c.00001b04::2015/09/11-03:52:17.939 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.000036f0::2015/09/11-03:52:18.205 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.0000442c::2015/09/11-03:52:18.205 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.0000442c::2015/09/11-03:52:18.205 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.0000442c::2015/09/11-03:52:18.205 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.0000442c::2015/09/11-03:52:18.236 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.000036f0::2015/09/11-03:52:18.236 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.000036f0::2015/09/11-03:52:18.236 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.000036f0::2015/09/11-03:52:18.236 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00001760.000018f8::2015/09/11-03:52:21.216 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:21.216 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:21.216 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:21.216 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:21.247 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:21.247 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000038f4::2015/09/11-03:52:21.262 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000038f4::2015/09/11-03:52:21.262 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000038f4::2015/09/11-03:52:21.262 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:21.262 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000038f4::2015/09/11-03:52:21.262 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:21.262 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:21.403 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:21.403 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:21.715 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:21.715 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:21.746 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:21.746 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:22.729 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:22.729 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:22.729 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:22.729 INFO [API] s_ApiGetQuorumResource final status 0.
0000233c.0000459c::2015/09/11-03:52:22.807 INFO [RES] SQL Server Availability Group: [hadrag] SQL Server component 'query_processing' health state has been changed from 'warning' to 'clean' at 2015-09-11 11:52:22.790
0000233c.00004230::2015/09/11-03:52:22.947 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00001760.000018f8::2015/09/11-03:52:23.103 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:23.103 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:23.103 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:23.103 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:23.134 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:23.134 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:23.134 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:23.134 INFO [API] s_ApiGetQuorumResource final status 0.
00002338.000036f0::2015/09/11-03:52:23.212 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.0000442c::2015/09/11-03:52:23.212 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.0000442c::2015/09/11-03:52:23.212 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.0000442c::2015/09/11-03:52:23.212 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.00003d7c::2015/09/11-03:52:23.244 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.0000442c::2015/09/11-03:52:23.244 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.0000442c::2015/09/11-03:52:23.244 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.0000442c::2015/09/11-03:52:23.244 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00001760.00003830::2015/09/11-03:52:23.337 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:23.337 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:23.368 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:23.368 INFO [API] s_ApiGetQuorumResource final status 0.
0000233c.00004230::2015/09/11-03:52:27.955 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.0000442c::2015/09/11-03:52:28.220 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.00003d7c::2015/09/11-03:52:28.220 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00003d7c::2015/09/11-03:52:28.220 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00003d7c::2015/09/11-03:52:28.220 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.00004028::2015/09/11-03:52:28.252 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.00003d7c::2015/09/11-03:52:28.252 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00003d7c::2015/09/11-03:52:28.252 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00003d7c::2015/09/11-03:52:28.252 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00001760.00003830::2015/09/11-03:52:31.778 INFO [NM] Received request from client address DL980-3.
00001760.000018f8::2015/09/11-03:52:32.277 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:32.277 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:32.464 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:32.464 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:32.464 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.00003830::2015/09/11-03:52:32.464 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:32.636 INFO [API] s_ApiGetQuorumResource final status 0.
00001760.000018f8::2015/09/11-03:52:32.636 INFO [API] s_ApiGetQuorumResource final status 0.
0000233c.00004230::2015/09/11-03:52:32.963 INFO [RES] SAP Resource <SAP TCP 00 Instance>: LooksAlive request.
00002338.00005c20::2015/09/11-03:52:33.228 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:6e26fe17-09c2-4f54-8aae-52678b351486:Netbios
00002338.00004028::2015/09/11-03:52:33.228 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00004028::2015/09/11-03:52:33.228 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00004028::2015/09/11-03:52:33.228 INFO [RES] Network Name <AGTCP_tccdb4>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.00004028::2015/09/11-03:52:33.260 INFO [RES] Network Name: Agent: Sending request Netname/RecheckConfig to NN:bbc2b407-7d1e-46fa-a32d-8ab9cfcefd9c:Netbios
00002338.00005c20::2015/09/11-03:52:33.260 INFO [RES] Network Name <tccprd1>: Netbios: Slow Operation, FinishWithReply: 0
00002338.00005c20::2015/09/11-03:52:33.260 INFO [RES] Network Name: [NN] got sync reply: 0
00002338.00005c20::2015/09/11-03:52:33.260 INFO [RES] Network Name <tccprd1>: Netbios: End of Slow Operation, state: Initialized/Idle, prevWorkState: Idle
00002338.00004028::2015/09/11-03:52:36.115 INFO [RES] Network Name <AGTCP_tccdb4>: Getting Read/Write private properties
- Edited by dennislee124 7 minutes ago
Other recent topics
