Playing with transportable snapshots

Transportable shadow copies are another practical consequence of a new paradigm - the data is stored in the (SAN) cloud. The data availability is not anymore tied to your machine. This simple fact has far reaching consequences. One consequence is the ability to control the lifetime of your data without affecting the performance of your existing servers. Another consequence is the ability to manage multiple independent copies of your data volumes across several machines.

Let's take the simples example where the "data" is tied to a certain machine. For example, a SQL Server database is used 24/7 by a production server. The problem here is that you cannot do any experiments, upgrades, or CPU-consuming data mining queries on this database without affecting your production server. And, in some cases, this is unacceptable.

Wouldn’t be nice to have a “clone” of your SQL database that you can play with from a separate machine? Without affecting the original server at all? This is what transportable shadow copies are used for.

Transportable snapshots for the masses

The transportable shadow copies technology was added in Windows Server 2003. Basically, you have the ability to “clone” images of your disks, and surface them later on a separate machine. This is done in real-time, with no apparent effect on the production server. But on the other side, this usually requires specialized hardware support from vendors.

Here is how it works, at least at a very high level:

1) First, we need a VSS requestor – i.e. a program that is using the VSS API to create a transportable shadow copy. This API is independent on the actual method used to create the shadow copy under the cover, and therefore, it is vendor independent.

2) The VSS requestor sends to VSS a shadow copy creation request, say, on drives X: and Y:, containing your SQL database and log files.

3) VSS creates the shadow copy under the cover. Under the cover, VSS talks with a custom, hardware-dependent “plug-in” component that actually creates the shadow copy (this is the “VSS Hardware provider”). The shadow copy preparation time might take a while, but it takes less than 10 seconds to create the actual shadow copy image. The original "production" server should remain unaffected.

4) The shadow copy is now created, but not yet visible on any machine in the SAN.

5) Now, we start a process called “Import” that will surface the shadow copy on a separate machine. The two LUNs for the created snapshots will appear on the import machine as two new disks. On top of them you can define two separate drive letters, for the database and log volumes.

6) The final step is to make these two volumes writable (because shadow copies are read-only by default) and mount the SQL database.

7) Voila! You have now a separate database to play with on a separate machine. This database is completely independent from the original database from the production server. It doesn’t matter what you do with this other database, in general, the impact on the production server is almost zero.

This technique can be used with great success to create “clones” of online databases in matter of minutes, irrespective of the database size. Depending on the hardware support, you are able to clone a terabyte-size SQL database or a Microsoft Exchange in a few minutes. Also, the API used to perform this operation is largely hardware-independent, which is a key advantage if you do not want to learn the specific proprietary methods of creating a snapshot with your SAN model.

Finally, VSS ensures integration with the applications or services that are storing data on these shadow copies. SQL Server, Exchange or Active Directory stores are properly flushed and brought in a consistent state right before the snapshot creation.

Preparations

First, you need two separate machines running Windows Server 2003 Advanced Server or Datacenter edition. Both machines must be connected (through a SAN configuration) to a storage array that supports VSS hardware shadow copies. Then, on each machine, a VSS hardware provider must be installed. Some VSS HW providers require a special a manual setup process such that transportable shadow copies can be created on the first machine and imported on the second machine.

But if you do not have a SAN, the best way to get started with a transportable shadow copy creation is to use a VSS provider that does not require a SAN in the back-end. There are at least two ways to do this:

a) You can use the simulated VSS Hardware Provider which is included in the VSS SDK 7.1 (which can be freely downloaded from the web from here). The problem with the simulated provider is that it works best only with very small volumes/disks.

b) Alternatively, you can use the VSS Hardware Provider for Wintarget. Wintarget is an software-emulated iSCSI target from String Bean Software that also has a VSS hardware provider. You can download the trial versions from this link.

To exercise the VSS API, we use a freely downloadable tool called VSHADOW.EXE, also supplied with the VSS SDK.

Let's see it in action

Let's describe the practical steps in simple sequences of commands.

1) [Machine1] Creating a persistent transportable shadow copy of the X: and Y: volumes:

VSHADOW -p -t=bc1.xml -script=sc1.cmd X: Y:

We also generated a script file (sc1.cmd) that contains the definitions of the shadow copy (set) IDs as a script that sets some predefined environment variables. We will see later why we need this.

2)  [Machine1 -> Machine2] Copy the generated backup extensions document (bc1.xml). This contains information about the “shadow copy” that lives somewhere in the cloud (as I mentioned above, these shadow copies are not visible on any machine – yet).

3) Also, we need to copy the sc1.cmd file that contains the definitions of the shadow copy (set) IDs as a script that sets some predefined environment variables. We will see later why we need this.

4) [Machine2] Import the transportable shadow copy

VSHADOW -i=bc1.xml

5) [Machine2] Assign drive letters to the new shadow copy disks. To do this, we need first to know the shadow copy IDs – and we will just execute the “sc1.cmd” script above:

Sc1.cmd

Then, we need to expose these shadow copies as new drive letters. We reuse the same drive letters as on the production server (Machine 1). Optionally, you could expose these shadow copies as drive letters or mount points:

VSHADOW –el=%VSHADOW_ID_1%,X:

VSHADOW –el=%VSHADOW_ID_2%,Y:

6) [Machine2] Break the shadow copy set into writable disks. At this point, these new disks are not anymore under the control of VSS. They appear in the system as completely independent disks.

VSHADOW –bw=%VSHADOW_SET_ID%

7) [Machine2] Finally, we mount the new SQL database.

Done!

More resources

1) A simple introduction in these concepts, on Technet

2) The VSS SDK section on MSDN talks in detail about transportable shadow copies.

3) Not directly related, but here is a link for the documentation of the Microsoft Exchange VSS writer.

4) Highly technical details about implementation detail of VSS Hardware Providers: here.