Hi Everyone,

I'm having issues getting the web worker role deployed. It keeps failing and going into repair mode.

Here is a snippet from the logs. If anyone can give me an idea of what may be happening.

I have followed the deployment document to the letter and have all roles running etc.

Id = 1420, Date = Mon Apr 15 20:07:18 UTC+1200 2013, Level = 2, Server Name = ww-azure-02.cloud.local, Message = Attempting autorepair (attempt 26 of 2147483647).
Id = 1419, Date = Mon Apr 15 20:07:17 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Forcing server reboot due to multiple unsuccessful repairs.
Id = 1418, Date = Mon Apr 15 20:07:17 UTC+1200 2013, Level = 1, Server Name = ww-azure-02.cloud.local, Message = Server marked 'Unhealthy' due to failure to run operation 'AutoRepair'
Id = 1417, Date = Mon Apr 15 20:07:17 UTC+1200 2013, Level = 1, Server Name = ww-azure-02.cloud.local, Message = Failed to run operation 'AutoRepair'.  Operation failed to complete.  System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.GetResponse()
   at HostingController.Helper.CheckWorkerHttpAvailability(String& errorMessage).
Id = 1416, Date = Mon Apr 15 20:07:17 UTC+1200 2013, Level = 1, Server Name = ww-azure-02.cloud.local, Message = Failed to run operation 'Repair'.  Operation failed to complete.  System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.GetResponse()
   at HostingController.Helper.CheckWorkerHttpAvailability(String& errorMessage).
Id = 1415, Date = Mon Apr 15 20:07:17 UTC+1200 2013, Level = 1, Server Name = ww-azure-02.cloud.local, Message = Failed to run operation 'SampleRuntimeData'.  Operation failed to complete.  System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.GetResponse()
   at HostingController.Helper.CheckWorkerHttpAvailability(String& errorMessage).
Id = 1414, Date = Mon Apr 15 20:07:17 UTC+1200 2013, Level = 1, Server Name = ww-azure-02.cloud.local, Message = Failed to run operation 'WorkerPeriodicHttpHealthCheck'.  Operation failed to complete.  System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.GetResponse()
   at HostingController.Helper.CheckWorkerHttpAvailability(String& errorMessage).
Id = 1413, Date = Mon Apr 15 20:07:17 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Changing service 'ResourceMetering' from state 'Stopped' to 'Running'
Id = 1412, Date = Mon Apr 15 20:07:17 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Changing service 'ResourceMetering' from state 'Running' to 'Stopped'
Id = 1411, Date = Mon Apr 15 20:07:16 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Changing service 'dwassvc' from state 'Stopped' to 'Running'
Id = 1410, Date = Mon Apr 15 20:07:09 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Changing service 'dwassvc' from state 'Running' to 'Stopped'
Id = 1409, Date = Mon Apr 15 20:07:04 UTC+1200 2013, Level = 4, Server Name = ww-azure-02.cloud.local, Message = Running command '"%windir%\microsoft.net\framework64\v2.0.50727\CasPol.exe" -q -m -ag 1.3 -url file://* FullTrust'.
Id = 1408, Date = Mon Apr 15 20:07:04 UTC+1200 2013, Level = 4, Server Name = ww-azure-02.cloud.local, Message = Running command '"%windir%\microsoft.net\framework\v2.0.50727\CasPol.exe\" -q -m -ag 1.3 -url file://* FullTrust'.
Id = 1407, Date = Mon Apr 15 20:07:04 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Changing service 'ResourceMetering' from state 'Stopped' to 'Running'
Id = 1406, Date = Mon Apr 15 20:07:03 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Changing service 'ResourceMetering' from state 'Running' to 'Stopped'
Id = 1405, Date = Mon Apr 15 20:07:03 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Changing service 'W3SVC' from state 'Stopped' to 'Running'
Id = 1404, Date = Mon Apr 15 20:07:03 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Installing product 'Web Farm Framework 1.1'
Id = 1403, Date = Mon Apr 15 20:07:03 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Installing product 'WFastCGI Gateway for IIS and Python 2.7'
Id = 1402, Date = Mon Apr 15 20:07:00 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Installing product 'WCF 4.5 for Web Hosting'
Id = 1401, Date = Mon Apr 15 20:06:58 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Downloading product 'Web Farm Framework 1.1'
Id = 1400, Date = Mon Apr 15 20:06:58 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Installing product 'ASPNET 3.5'
Id = 1399, Date = Mon Apr 15 20:06:58 UTC+1200 2013, Level = 3, Server Name = ww-azure-02.cloud.local, Message = Downloading product 'WFastCGI Gateway for IIS and Python 2.7'
Id = 1398, Date = Mon Apr 15 20:05:57 UTC+1200 2013, Level = 2, Server Name = ww-azure-02.cloud.local, Message = Attempting autorepair (attempt 25 of 2147483647).
Id = 1397, Date = Mon Apr 15 20:05:57 UTC+1200 2013, Level = 1, Server Name = ww-azure-02.cloud.local, Message = Server marked 'Unhealthy' due to failure to run operation 'AutoRepair'
Id = 1396, Date = Mon Apr 15 20:05:57 UTC+1200 2013, Level = 1, Server Name = ww-azure-02.cloud.local, Message = Failed to run operation 'AutoRepair'.  Operation failed to complete.  System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host

Thanks, Rob

Hi Rob,

It is not clear from the log what is causing the worker provisioning to fail. Coudl you please provide more informaiton?

- Anythign interestign you can see in the event logs on the web worker server?

- Is teh worker a standaloen VM or shres another different role?

Thank you

Hi,

I managed to eliminate all but one of the errors. I found some logs on the web worker that was complaining that the web worker account on the sql server was disabled.

Fixed this and now have only the last error left.

Id = 9734, Date = Tue Apr 16 09:59:40 UTC+1200 2013, Level = 2, Server Name = 192.168.7.22, Message = Attempting autorepair (attempt 2 of 2147483647).
Id = 9733, Date = Tue Apr 16 09:59:40 UTC+1200 2013, Level = 1, Server Name = 192.168.7.22, Message = Server marked 'Unhealthy' due to failure to run operation 'AutoRepair'
Id = 9732, Date = Tue Apr 16 09:59:40 UTC+1200 2013, Level = 1, Server Name = 192.168.7.22, Message = Failed to run operation 'AutoRepair'.  One or more health checks failed.
Id = 9731, Date = Tue Apr 16 09:59:40 UTC+1200 2013, Level = 1, Server Name = 192.168.7.22, Message = Failed to run operation 'Repair'.  One or more health checks failed.
Id = 9730, Date = Tue Apr 16 09:59:40 UTC+1200 2013, Level = 1, Server Name = 192.168.7.22, Message = Failed to run operation 'HealthCheck'.  One or more health checks failed.
Id = 9729, Date = Tue Apr 16 09:59:40 UTC+1200 2013, Level = 1, Server Name = 192.168.7.22, Message = System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   --- End of inner exception stack trace ---
   at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
   at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
   --- End of inner exception stack trace ---
   at System.Net.HttpWebRequest.GetResponse()
   at HostingController.Helper.CheckWorkerHttpAvailability(String& errorMessage)
Id = 9728, Date = Tue Apr 16 09:59:43 UTC+1200 2013, Level = 3, Server Name = 192.168.7.22, Message = Changing service 'ResourceMetering' from state 'Stopped' to 'Running'

This one is persistent and there are no further logs giving any pointers.

Also to note I can install the frontend and publisher roles to this machine fine just not the web worker.

Regards, Rob

• Edited by slowboy Monday, April 15, 2013 10:06 PM update

Hi Everyone,

I'm answering my own question.

DONT!!! Disable ipv6 on the web worker roles.

After a lot of digging around this is what it turned out to be.

Regards, Rob

• Marked as answer by slowboy Thursday, April 25, 2013 1:28 PM

Hello,

I was successful, after disabling IPV6 from web worker server.

Thanks a lot.