Coffee Break – keeping stuff in the cloud


In the last few Coffee Break posts we have been looking at virtual hard disks (VHDs) and virtual machines (VMs). Now let’s combine the two in the cloud.

Coffee Break 11 – Provisioning – Using Microsoft Azure storage to keep VHDs and other useful stuff in the cloud

Provisioning a new instance of Microsoft Dynamics NAV is like putting individual bits together. In this Coffee Break post, we will prepare some of these bits by building a VHD up front and then upload it to Azure. Then every new virtual machine (VM) can pick a copy of this disk. A VHD like this can contain anything such as the Microsoft Dynamics NAV product DVD, application objects, add-on databases, documents, and so on. And we can save a bit of time and bandwidth compared to uploading a new DVD, for example, every time we provision a new VM in the cloud.

Customer Story:

The customer is frequently creating new VMs. And every new VM they create will need access to common data, such as the Microsoft Dynamics NAV product DVD.

Prerequisites:

1 VHD (local) and 1 VM (In Azure):

Create a VHD: Coffee Break – Windows PowerShell and creating a Hyper-V disk

Create a VM: Coffee Break – Using Windows PowerShell to provision virtual machines

Get the VHD to the VM

Once we have a VHD (locally) and a VM in Azure (whether this was created manually or via PowerShell), let’s combine the two. First, upload the VHD to Azure. Connect your ISE PowerShell environment to your Azure subscription:
Add-AzureAccount

Create a new storage account to hold our VHD:

$SourceStorageAccount = “mysharedstuff

First test that the storage name is actually available:

Test-AzureNameStorage $SourceStorageAccount

If it is (the line above returns False), then create it:

New-AzureStorageAccountStorageAccountName $SourceStorageAccountLocation “West Europe”

 

Optionally we can add containers to this storage, just to keep things organized. First, let’s create a context that includes access information for our new storage:

$Key = (Get-AzureStorageKeyStorageAccountName $SourceStorageAccount).Primary
$Context = New-AzureStorageContextStorageAccountName $SourceStorageAccountStorageAccountKey $Key

This $Context variable gives us access to the storage, and is one we will use a couple of times.

Create a container in our storage to store vhds:

New-AzureStorageContainerContext $ContextName vhds

Upload the VHD

Let’s say you have a local VHD here: C:\temp\mydisctest.vhd. Then copy that up to your storage:
Add-AzurevhdLocalFilePath C:\temp\mydisctest.vhd -Destination “https://$SourceStorageAccount.blob.core.windows.net/vhds/mydisk.vhd”

If you get this error: “The process cannot access the file ‘C:\temp\mydisctest.vhd’ because it is being used by another process.” then make sure that the disk is not already mounted to your local machine. Note that if you happen to double-click on the VHD, Windows may automatically mount it. You can dismount it like this:

Dismount-VHDPath C:\temp\mydisctest.vhd

(side note:) If you should ever need to copy it back, use  Save-AzureVhd.

The Add-Azurevhd cmdlet does a clever copy by checking that this is a valid VHD, and by copying only actual data. So if the disk itself is 3GB but is only 5% full, then only a file the size of the 5% will actually be copied.

Make a copy

Every time we want a new disk, then we can make a new copy of this VHD, rather than attaching it directly in which case we could only use it for one VM. First create a new container for our copy. This container could relate somehow to the VM that we will be attaching the VHD to:

New-AzureStorageContainerContext $ContextName myvm

Then copy mydisk.vdh from vhds to myvm:

$SourceURI = “https://$SourceStorageAccount.blob.core.windows.net/vhds/mydisk.vhd

$State = Start-AzureStorageBlobCopyAbsoluteUri $SourceURIDestContainer myvm -DestBlob NewVHD.vhd -Context $Context

 

This will start an asynchronous copy. To check status before using the copy:

$state | Get-AzureStorageBlobCopyState

 

Get it together

Now we have a fresh copy of the VHD in Azure, let’s attach it to a VM, using the cmdlet Add-AzureDataDisk. The two main ways we can use this cmdlet are:

  • ImportFrom – to import an existing VHD – the one we will use here
  • CreateNew – to create a new raw disk

The syntax of the cmdlet is:
$vm | Add-AzureDataDisk | Update-AzureVM


So, we need to get the VM, then pipe it through in that way. First check that you have the right VM. See which ones you have available:

Get-AzureVM
Then pick one:
$VM = get-azurevmServiceName mytestserviceabcdefName NAVPC

Check that you still have the location for your VHD and then store that in another new variable:
$NewVHD = “https://$SourceStorageAccount.blob.core.windows.net/myvm/NewVHD.vhd”

Then put $VM through the pipe (or edit the line above if you want to have it all in just one line):
$VM | Add-AzureDataDiskImportFromDiskLabelMyStuff” –MediaLocation $NewVHDLUN  3 | Update-AzureVM

If you want to watch as it is happening, connect (RDC) to one of your Azure VMs and run File Explorer. Leave that in the background – it will update automatically and you can see when the disk is added. The disk is ready to use, and the files you put on it back when the disk was still on your local machine, are available now in your VM.

To remove it again, run this:

$VM | Remove-AzureDataDiskLUN 3 -DeleteVHD | Update-AzureVM

Check what disks (if any) are still attached:

$VM | Get-AzureDataDisk

 

Note:

If you attach and then detach the disk, Azure may still hold a lease on this disk, and it may not be free to use yet or even to be deleted. This is the error you may get when trying to attach the disk again or delete it: “The MyDisk VHD is already registered with image repository as the resource with ID XYZ”. In this case we may not be able to delete the disk automatically, which is why in the example above, if we remove the disk, then delete the disk as part of that operation.

So, just to avoid problems like this: When we detach a disk, then just to keep things clean, use -DeleteVHD to remove the disk completely.

 

 

Jasminka Thunes, Escalation Engineer Dynamics NAV EMEA

Lars Lohndorf-Larsen, Escalation Engineer Dynamics NAV EMEA

Bas Graaf, Senior Software Engineer Dynamics NAV

 

Comments (13)

  1. Sooooo tired of constant fees and charges... says:

    How much does cloud storage cost now, and what is Microsoft forecasting the charge to be in the near, middle, and 5+ year future?

  2. Pallea says:

    It's quite hard to take you serious when you don't tell whom you are.

    Here in Denmark we can calculate the price here:

    azure.microsoft.com/…/calculator

    But I am not sure if you are REALLY asking og just trying to do a complaint…

    Thanks

    PalleA

  3. Sooooo tired of constant fees and charges... says:

    Both I guess.

    Why is it that every time I turn around I'm paying some new gatekeeper a fee?

    Microsoft's revenues are declining, this just looks like a way for them to stick their hands into yet another pocket.

    What exactly is wrong with local storage?  

    I really am tired of having to log in to this system, that system, remember 15 passwords just to do a day's work, and constantly hearing about license issues.

  4. Pallea says:

    What I stil don't get is that you are hiding yourself – That makes it hard to take you serious. Having said that..Microsoft Azure is NOT mandatory. Dynamics NAV runs quite fine on a local server. So you really dont need any gatekeepers, etc etc.

    You said "What exactly is wrong with local storage?" – The answer to that questions is quite simple Nothing, but some prefers the freedom that Azure or other cloudservices offers them. Just to give you some examples:

    • Reduce of costs on server management (IT People)
    • Reduce of costs of "Mirroring" your critial hardware/software

    • Reduce of costs of electricity (thats a huge bill, from where I live).

    • The option of up- or downscaling memory, disc-space, processors when needed/not needed.

    • Predefined/installed images (VHDs) that saves me as a reseller, and my customers A LOT of money

    • Remote Access, anywhere, anytime, anyplace which means A LOT for many customers.

    But again its hard to explain when you dont reveal yourself or give more information on what you are sepcifically complaining about.

    To me what is happening to Microsoft, Dynamics NAV, tablets etc. is the most positive from Microsoft in more than a decade.

    Thanks

    PalleA

  5. Jens Glathe says:

    Nothing gets reduced, only more complex. You have easier access (on a global scale), but also a very real security issue. And I haven't even mentioned spying agencies, data-greedy corporations and governments yet. That alone is a reason to never have global web access to sensitive datasets and applications. Also, do you know what the bandwidth cost is? Certain setups do have prohibitive bandwidth costs. Also, a "backup" is only a backup when you have complete control over it, and can rebuild your infrastructure completely. Anything else will get you into hot water some day. So, all in all, only an option when you're operating on a global scale. I don't.

  6. John says:

    The funniest thing is that these guys don't even respond to the people asking MS questions in it's own official blog…

    Don't have much to say beside what I've said before – NAV MVPs army (MS marketing department #2 – let's call things the way they are) will feed this business for some time, but not sure for how long…

  7. Pallea says:

    @Jens I have customers that work only locally in Denmar and they "do it in the cloud" 😉 why? for several of the reasons that I have mentioned earlier.

    Actually you can download your working VHD using a powershell-script. Just as easy as doing an upload, so you are not lost or stuck with data in the cloud. Its "just" Hyper-V images.

    @John Ofcourse they are not responding to unknown users with silly comments and not real questions. If anyone of the users here have a real honest question then anyone can give their PAM or KAM at Microsoft and get all the answers they might have – Been there, done that, and very satisfied.

    I am not an MVP – In order to become a MVP you have to help alot in the community and I don't do that. I mainly work with customers and Danish Microsoft Partners. I do love the Dynamics NAV product – Dynamics NAV 2015 is the best product in many years. Yes, there are some issues with things that could be better, but overall…I am impressed over how far the product has come the last 2 years.

    Thanks

    PalleA

  8. Sooooo tired of constant fees and charges... says:

    NOT impressed by so-called 'improvements' made to NAV over the last 2 years.

    All that has happened is Microsoft has tied NAV more firmly into using their own product lines (next: BING for help searches?) and reduced the ability of actual users to 'do for themselves'.

    We're using a literally 30 year old system that is bought and paid for, with extensions added on to keep it up to date.  Thing is, in addition to the updates, we've also IMPROVED the underlying program.  Something that has not happened with NAV.  There are still KNOWN 10 year old bugs in NAV that haven't been fixed.

    No licensing, no fees, one password, it works way better than NAV without the additional cost of consultants who just recommend spending more and more money, with them, with Microsoft, for little actual benefit.

    What's wrong with that model?  Other than we're not dealing with a Microsoft constantly churning the waters in a flailing effort to stay relevant and pull still more loose change out of our pockets.

  9. Pallea says:

    Comon, you are blaming Microsoft for linking their products together??? Ofcourse they do! Dynamics NAV works much better together with e.g. Microsoft Excel – and because the users requested the link!

    Any customer can do exactly the same amount of development/modifications like the last 20 years, so either you don't know Dynamics NAV 2015 or you don't know what you are working with.

    What improvements in the last 2 years do you not like?

    What bugs do you believe that have not been fixed for the last 10 years?

    What have you done to get the bugs fixed and whan response did you get?

    What exactly kind of socalled underlying improvements have you done?

    Stop "hiding" yourself behind a silly name – If you REALLY are THAT sure that you are right, then you also must be strong enough to reveal yourself.

    Thanks

    PalleA

  10. Sooooo tired of constant fees and charges... says:

    The 30 year old workhorse is not Microsoft.

    It does allow us to export Excel worksheets to use, .pdf reports, heck, we can even export Lotus spreadsheets.

    The simple fact is that ERP programs haven't really changed much over the past 30 years in their fundamental goals. This particular one I hear was almost as big a piece of junk as NAV when they first got it (BPCS).  Thing is, they worked on improving it, extending its reach with data warehousing.

    Constant improvement instead of constant application of lipstick to a corpse.

    They decided to experiment with NAV since we didn't have lot control and tracking developed.  Gave it a shot.  What a sorry excuse for a 21st century ERP system it is.  Can't depend on one report matching another for the same transaction on the same day, more time spent reconciling than using data.

    The only problem now is finding RPG programmers.

    I realize that just like new car dealers, Microsoft has to keep driving demand with 'new models' and 'features' else they'll go out of business.  But as a consumer, I don't really see the benefits of all the higher tech in cars, which just leads to higher maintenance costs and less chance of finding repair parts farther down the road.  Ditto NAV.  Did I mention my car has 200K miles and is almost 20 years old? Still running strong.

  11. As mentioned in comments above: a) A good place to check what charges to expect is the Azure pricing calculator, and b) Azure is entirely optional.

    Cloud is not for everybody. But a lot of people are very interested in exploring what it can do for them. So I am happy to say that we are planning many more coffee breaks to come. Some coffee breaks will be regarding cloud, some not. Hopefully they will help in a small way to explore some of the options that come with NAV today, whether they are narrow and specific options or more widely applicable.

    Thanks for all comments!

    Lars

  12. John says:

    Can you guys, when you have a few more coffee breaks – make optional arguments for procedures/function?

    21 century is here, come on…

    Believe me, community will appreciate this "coffee-effort".