I've been trying to migrate a VM from a private cloud environment to Azure using System Center 2012 R2 App Controller and managed to get the deployment task going.  However, after about 8 minutes (wall clock) the task failed with "Server operation did not finish within user specified timeout '90' seconds".

As the VHD is about 8 GB, I can believe it would take longer than this to actually upload.

Is there any way to increase this timeout using App Controller or will it be necessary to do the upload using PowerShell to increase the blob timeout?  If it will be necessary to use PowerShell, what is the best way to handle the VHD and associated information in the Stored VM folder for the VM?

- Mark

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant 19 hours 59 minutes ago

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM
• Unmarked as answer by L. Mark Pilant Monday, August 11, 2014 7:42 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM
• Unmarked as answer by L. Mark Pilant Monday, August 11, 2014 7:42 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM
• Unmarked as answer by L. Mark Pilant Monday, August 11, 2014 7:42 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM
• Unmarked as answer by L. Mark Pilant Monday, August 11, 2014 7:42 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM
• Unmarked as answer by L. Mark Pilant Monday, August 11, 2014 7:42 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM
• Unmarked as answer by L. Mark Pilant Monday, August 11, 2014 7:42 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM
• Unmarked as answer by L. Mark Pilant Monday, August 11, 2014 7:42 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM
• Unmarked as answer by L. Mark Pilant Monday, August 11, 2014 7:42 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM
• Unmarked as answer by L. Mark Pilant Monday, August 11, 2014 7:42 PM

OK.  It seems the problem is the dedicated T1 line simple wasn't fast enough.  I tried the same operation using a higher speed (30 Mbps fiber connection) and it uploaded fine.  Although, as expected, it did take a bit to upload 8 GB.

- Mark

• Marked as answer by L. Mark Pilant Wednesday, February 19, 2014 4:17 PM
• Unmarked as answer by L. Mark Pilant Monday, August 11, 2014 7:42 PM

Well it would seem I was a bit hasty.

After upgrading the "pipe" considerably (4 T1s instead of just a single) the problem is still occurring.

Watching the network throughput for the entire time, most of the time there was in excess of 5 Mbps of traffic being sent to Azure.  There were a couple of places where the network activity dropped to about zero, but that may have been because there were two VHDs to transfer.

So the REALLY big question is:  How do I troubleshoot this.  All I have on this end (at the moment) is the failure information presented by App Controller.

I'm going to start poking around the Event log and for other log files, but I'm not too hopeful.

- Mark

In the Application and Services Logs -> Microsoft -> CloudManager -> Infrastructure ->Operational I found the following entry:

Request Failed By Provider. RequestID: 54d20e74-84de-4b93-b44c-14617c69bf78 InputUrl: /api/cloudsystems/management.core.windows.net/clouds/my-azure-subscription-guid/hostedservices/PilantAzureTest/deployments/Production/roles InputVerb: POST User: CILCLOUD\mpilant ProviderUrl: CILCLOUD\mpilant ProviderVerb: http://localhost:18622/azure/cloudsystems/management.core.windows.net/clouds/c7da2faa-5a78-4625-a7ad-a5060495beae/hostedservices/PilantAzureTest/deployments/Production/roles HttpStatusCode: POST ExceptionDetails: 500

The HTTP status of 500 has me concerned.  If I remember correctly, this is a "Server Error" which usually indicates something misconfigured or without the correct permissions.  Any ideas as to how I might see if this is the case?

- Mark

It is also curious this error occurs after the VHDs appear to have been moved to the cloud.  So I appears as though the problem may be some amount of handshaking at the "end".

Another bit of possibly useful info.

In the case where I am able to migrate to Azure, the App Controller services are logged on a "Network Service".  In the case where I cannot migrate, the services are logged on as a specific account (courtesy the PowerShell Deployment Toolkit).  I wonder is this may be a permission problem where the account created by PDT for App Controller isn't a member in all the groups in which it is needed.

- Mark