Uploading file to Azure

Hi,

I feel sure there must be an answer to this already, but how do I reliably transfer images to Azure?

I wrote a small program to transfer the same file to Azure storage, and I intermittently get the following error:

Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host.

It seems to take about 5 minutes before I get this error back, even when trying to transfer a file <1MB. I tried adding a retry policy but it didn't seem to do anything. I have read about Transient Fault Handling and that looks rather complicated for what I am trying to achieve.

Here is the example C# code that fails:

using System;
using System.Drawing;
using System.IO;
using Microsoft.WindowsAzure;            /* ADD REFERENCE */
using Microsoft.WindowsAzure.StorageClient;    /* ADD REFERENCE */
using System.Threading;

namespace ConsoleApplication1
{
    class Program
    {
        static void Main(string[] args)
        {
            var mimeType = "image/jpeg";
            var containerName = "containername";    /* AMEND HERE */
            var accountUsername = "ACCOUNTUSERNAME";    /* AMEND HERE */
            var accountPassword = "ACCOUNTPASSWORD";    /* AMEND HERE */
            var origfilename = "abc.jpg";
            Uri uri;
                    
            for (int x = 1; x < 100; x++)
            {
                for (int n = 1; n < 31; n++)
                {
                    try
                    {
                        using (var imageStream = FetchImage("http://SERVER.COM/FILENAME.JPG").ToStream(System.Drawing.Imaging.ImageFormat.Jpeg))    /* AMEND HERE */
                        {
                            using (var mainImage = Image.FromStream(imageStream))
                            {
                                using (Stream mainStream = mainImage.ToStream(System.Drawing.Imaging.ImageFormat.Jpeg))
                                {
                                    string filename = Path.GetFileNameWithoutExtension(origfilename) + n.ToString() + Path.GetExtension(origfilename);
                                        
                                    uri = UploadFile(accountUsername, accountPassword, containerName, filename, mainStream, mimeType);
                                    Console.WriteLine("Loop:" + x.ToString() + " File No:" + n.ToString() + " Processed: " + uri.ToString());
                                }
                            }
                        }
                    }
                    catch (Exception ex)
                    {
                        Console.WriteLine(ex.Message);
                    }
                }
            }
            Console.ReadKey();
        }

        public static System.Drawing.Image FetchImage(string url)
        {
            var httpWebRequest = (System.Net.HttpWebRequest)System.Net.HttpWebRequest.Create(url);
            httpWebRequest.AllowWriteStreamBuffering = true;
            httpWebRequest.Timeout = 20000;
            using (Stream webStream = httpWebRequest.GetResponse().GetResponseStream())
            {
                return Image.FromStream(webStream);
            }
        }

        public static Uri UploadFile(string accountUsername, string accountPassword, string containerName, string filename, Stream fileStream, string mimeType)
        {
            var cloudStorageAccount = CloudStorageAccount.Parse("DefaultEndpointsProtocol=http;AccountName=" + accountUsername + ";AccountKey=" + accountPassword);
            var blobClient = cloudStorageAccount.CreateCloudBlobClient();
            var blobContainer = blobClient.GetContainerReference(containerName);
            
            blobClient.Timeout = new TimeSpan(0, 0, 30);
            blobContainer.CreateIfNotExist();

            blobContainer.SetPermissions(new BlobContainerPermissions { PublicAccess = BlobContainerPublicAccessType.Blob });

            var blob = blobContainer.GetBlobReference(filename);
            
            if (!string.IsNullOrEmpty(mimeType)) blob.Properties.ContentType = mimeType;
    
            blob.UploadFromStream(fileStream);        /* THIS LINE ERRORS */

            Console.WriteLine(blob.Uri);
            return blob.Uri;
        }
    }
}


  • Edited by Zorro2103 Friday, September 13, 2013 2:57 PM
September 13th, 2013 5:29pm

Hi,

I can't say where your code is failing, but this is the code that I use to make a WCF service running as an Azure web role upload an image to an Azure storage Blob:

//Uploads image to Azure Blob
        internal static string UploadImageToBlob(string locationID, Stream imageToUpload)
        {
            try
            {
//Create a unique GUID based on locationID + new random GUID to use as filename
                string randomGUID = locationID
                    + "-"
                    + Guid.NewGuid().ToString();

                //Retrieve storage account from application settings
                CloudStorageAccount storageAccount = GetStorageAccount();

                //Create blob client
                CloudBlobClient blobClient = storageAccount.CreateCloudBlobClient();

                //Retrieve reference to images container
                CloudBlobContainer container = blobClient.GetContainerReference(
                    RoleEnvironment.GetConfigurationSettingValue("BlobContainer"));                    

                //Retrieve references to the blob inside the container
                CloudBlockBlob blockBlob = container.GetBlockBlobReference(randomGUID);

                blockBlob.UploadFromStream(imageToUpload);

                return blockBlob.Uri.ToString();
            }
            //TODO
            catch(Exception)
            { return null; }
        }

This code works very well and image transfers are super fast (average file size is 4MB).

Perhaps your memory stream is closed before/during the execution of UploadFromStream? This error has happened to me before, where the application will try to access a memory stream that was already previously closed.

Free Windows Admin Tool Kit Click here and download it now
September 15th, 2013 6:34pm

Hi luisdel,

Thank you for the reply. The code for uploading the file looks rather similar to the code I posted. Does your code sometimes fail and return null? If so, Do you have any idea how long you have to wait from the point of attempting the upload of the file?

The code I posted works for the majority of transfers, on the failed ones it takes around 5 minutes plus before the exception is caught. I tried using timeout options but they seemed to make no difference. The files I send usually take less than 3 seconds to transfer, I would therefore like to know how to get timeout working correctly if the file hasn't transferred within 30 seconds.

Thanks,

Zorro2013

September 16th, 2013 10:19am

Have you monitored if there are problems withint the Azure service? You can enable monitoring and see if you are getting errors or warning from the storage service, see screenshot. Go to the configure Tab on your storage service in the adminportal and select to monitor all options.
Free Windows Admin Tool Kit Click here and download it now
September 21st, 2013 3:01am

This topic is archived. No further replies will be accepted.

Other recent topics Other recent topics