NOTE: While these instructions may work, they are fairly old. If you are using Visual Studio 2013, there's a much quicker way to accomplish the same end goal. See: http://azure.microsoft.com/en-us/documentation/videos/azure-identity-application-to-authenticate/
In this video, I walk you through how to configure Single Sign On (SSO) across CRM Online and Windows Azure using Visual Studio 2010. My walkthrough is based on the Web Single Sign-On with .NET and Windows Azure Active Directory walkthrough, but adds the nuances of getting this working with Windows Azure & Visual Studio 2010. If you are using Visual Studio 2012, follow the instructions in the link. If you are using Windows Azure Web Sites instead of a Web Role, then there is an even easier way to configure SSO: http://www.asp.net/vnext/overview/latest/windows-azure-authentication.
In the video, I use Fiddler to redirect https traffic to the Windows Azure Compute Emulator. You can download it here: