RDP failures on recycling role

Many times we want to login to an Azure VM when the role is recycling so that we can troubleshoot further. In such case if the RDP is not already enabled then you would first need to enable it. In some cases the RDP fails with below error,

Your role instances have recycled a number of times during an update or upgrade operations. This indicates that the new version of your service or the configuration settings you provided when configuring the service prevent the role instances from running. verify your code does not throw unhandled exceptions and that your configuration settings are correct and then start another update or upgrade operation.

I implore you not to give up at this point. If you reboot the VM on recycling role after this error then it will allow you to log in via RDP when it come back up again.