Windows Azure $30 demo saving $127,970

I recently presented at the Windows Azure User group in Manchester. Also presenting was Alan Smith. I wanted to share my finding from Alan’s presentation which was a demo of a ray tracing render farm hosted in Windows Azure using worker roles and the Azure Storage service.

At the start of the presentation he deployed an Azure application that uses 256 worker roles to render a 1,500 frame 3D ray-traced animation in one hour at the end of the presentation, when the animation was complete, he simply played the animation and then deleted the Azure deployment.

What was amazing about this presentation and demo is that it highlights one of the great benefits of cloud computing, you pay for what you use, and if you need massive compute power for a short period of time using Windows Azure can work out very cost effective.

A fully documented write up of the entire presentation is here https://geekswithblogs.net/asmith/archive/2012/06/25/150043.aspx

The virtual pin board contains 11,481 pins, with the scene file containing 2,000 scene files will be created via a Microsoft Kinect device.

The tracing time for the test frame was 4 minutes 27 seconds, which means rendering the 2,000 frames in the animation would take over 148 hours, or a little over 6 days.

The completed animation can be viewed in 1280x720 resolution

Cost Savings of the Cloud

The cost of creating an on-premise render farm to render animations increases in proportion to the number of servers. The table below shows the cost of servers for creating a render farm, assuming a cost of $500 per server.

The Windows Azure compute services provide worker roles, which are ideal for performing processor intensive compute tasks. With the scalability available in Windows Azure a job that takes 256 hours to complete could be perfumed using different numbers of worker roles. The time and cost of using 1, 16 or 256 worker roles is shown below.

Using worker roles in Windows Azure provides the same cost for the 256 hour job, irrespective of the number of worker roles used. Provided the compute task can be broken down into many small units, and the worker role compute power can be used effectively, it makes sense to scale the application so that the task is completed quickly, making the results available in a timely fashion. The task of rendering 2,000 frames in an animation is one that can easily be broken down into 2,000 individual pieces, which can be performed by a number of worker roles.

Effective Use of Resources

The monitor statistics the animation took 6 days, 7 hours and 22 minutes CPU to render, this works out at 152 hours of compute time, rounded up to the nearest hour. With the average CPU usage across all instances is 93.27%, with over 99% used when all the instances are up and running. This shows that the worker role resources are being used very effectively.

Grid Computing Scenarios

Windows Azure provides a great platform for developing these types of grid computing applications, and can work out very cost effective.

· Windows Azure can provide massive compute power, on demand, in a matter of minutes.

· The use of queues to manage the load balancing of jobs between role instances is a simple and effective solution.

· Using a cloud-computing platform like Windows Azure allows proof-of-concept scenarios to be tested and evaluated on a very low budget.

· No charges for inbound data transfer makes the uploading of large data sets to Windows Azure Storage services cost effective.

Tips for using Windows Azure for Grid Computing Scenarios

Alan suggested the following tips when implementing a grid computing project in Windows Azure.

· Using an Azure Storage queue to load-balance the units of work across multiple worker roles is simple and very effective. The design I have used in this scenario could easily scale to many thousands of worker role instances.

· Windows Azure accounts are typically limited to 20 cores. If you need to use more than this, a call to support and a credit card check will be required.

· Be aware of how the billing model works. You will be charged for worker role instances for the full clock our in which the instance is deployed. Schedule the workload to start just after the clock hour has started.

· Monitor the utilization of the resources you are provisioning, ensure that you are not paying for worker roles that are idle.

· If you are deploying third party applications to worker roles, you may well run into licensing issues. Purchasing software licenses on a per-processor basis when using hundreds of processors for a short time period would not be cost effective.

· Third party software may also require installation onto the worker roles, which can be accomplished using start-up tasks. Bear in mind that adding a startup task and possible re-boot will add to the time required for the worker role instance to start and activate. An alternative may be to use a prepared VM and use VM roles.

· Consider using the Windows Azure Autoscaling Application Block (WASABi) to autoscale the worker roles in your application. When using a large number of worker roles, the utilization must be carefully monitored, if the scaling algorithms are not optimal it could get very expensive!