How you can uniquely identify a LUN in a Storage Area Network

A SAN is a complex animal

From the first minute you try to understand it, you might be overwhelmed by the cascade of new things: fibre switches, multi-path software, redundant controllers, disk enclosures etc. But the idea is pretty simple. In a few words, the primary role of a SAN is to expose virtual disk units (called LUNs) to various machines. A very high-level picture of a simple SAN configuration is presented below:

Here we see two machines (A and B) connected to a single storage array (X). This array implements three virtual disk units (LUNs) that are exposed to these machines: the first one sees the first LUN as the disk labelled "Disk 1", while the second machine sees the other two LUNs, as disks "Disk 2" and "Disk 3".

The need for a standard

But here comes a basic problem. If I can expose the same LUN to one or more machines, then how could I address it? In other words, how can safely I distinguish between one LUN and another? This seems to be a really trivial problem. Just stick a a unique GUID to each LUN and you are done! Or, stick a unique number. Or... a string... but hold on, things are not that easy. What if storage array maker ABC assigns GUIDs to each LUN and another vendor assigns 32-bit numbers? We have a complete mess.

To add to the confusion, we have this other concept - the serial number attached to a SCSI disk. But this doesn't work all the time. For example, some vendors assign a serial number for each LUN, but this serial number is not guaranteed to be unique. Why, some SCSI controllers are even returning the same serial number for all exposed LUNs!

And this was exactly the state of things until recently. Every hardware vendor had a more-or-less proprietary method to identify LUNs exposed to a system. But if you wanted to write an application that tried to discover all the LUNs you had a hard time since your code was tied to the specific model of each array. Not to mention that you did nto have the guarantee that these LUN IDs were unique! What if two vendors had a conflicting way to assign IDs to LUNs? You could end up with two LUNs having the same ID!

Fortunately, hardware vendors are already converging on a standard. The standard is described in detail in the latest SPC-3 draft, which describes the latest incarnation of the SCSI protocol. The link is this one: https://www.t10.org/ftp/t10/drafts/spc3/spc3r21b.pdf. For more details, go to section 7.6.4 Device Identification VPD page. 

The principle is simple - we have this SCSI Inquiry command that can be used to retrive some "metadata" of some target LUN. This metadata includes various components, like the product ID, serial number, etc. A SCSI initiator (a regular machine in our case) can use this command to discover more details about various LUNs exposed by the storage array. Now, this SCSI command was enhanced to return optional data through Page #83. If this page is requested, you will get a list of related identifiers for this LUN.

According to this standard, a LUN might have one or more structure identification descriptor structures. This structure is well documented in the above standard. The structure has a number of relevant fields, described below:

1) Association field, describes to what is the identifier attached: 0 (the identifier is related to a device), 1 (related to the path between the device and the port), 2 (related to the SCSI target).

2) Identifier type field. The value of this field describes the particular format/semantics of the actual identifier. Zero (0) means a proprietary vendor format, and this means that the vendor is not guaranteed to be globally unique. A value of 3 means that the identifier is a FPCH name, which is guaranteed to be unique. And so on...

3) Identifier data field. This is the actual identifier, and it consists in an array of bytes.

4) Identifier data size field - the size in bytes of the actual identifier.

Here is how it works: every LUN can have one or more identifiers as already mentioned above. Each identifier will have a different type. So, in the first picture in this post, the LUN 1 might have three identifiers. One of type 0 (a vendor-specific, proprietary identifier), another one of type 1 (T10 identifier), and a third one, of type 3 identifier (FPCH Name). This is a clever way that allows storage vendors to assign multiple identifiers to a LUNs going forward, while maintaining backward compatibility with the old, proprietary way of identifying LUNs. So if a storage array has already a proprietary ID for each LUN, he can simply add a new globally-unique identifier to in the next version of its firmware. And, in the end, all hardware vendors will converge to a common addressing scheme.

Note however that not all identifier types above are considered globally unique. An identifier of type 0 is obviously not, as mentioned above. But an identifier of type 1,2,3 or 8 is guaranteed to be unique, as long as the vendor follows the standard completely.

I haven't mentioned yet another important field - the association field. This field describes the purpose of the identifier. For example, when the association field is "0", this indicates that the specified identifier is tied to the actual LUN device. When the association is "1", the identifiers is tied to the specific path between the LUN/controller port, not to the LUN itself. Finally, the association "2" describes an identifier associated to the SCSI target. But in the context of our discussion we are interested only in LUN identifiers with association being always "0".

One word of caution, though: not every Fibre Channel/iSCSI storage array will implement Page #83 identifiers for a LUN, and even if it does, older models might not return unique identifiers! As far as I can tell, it appears that most of the latest SAN storage arrays, from various vendors, are implementing correctly this standard. Anyway, I would recommend you to double-check this with with your storage vendor - the availability of SCSI unique identifiers might depend on your array model, or even firmware version.  

Finally, I would like to point one thing: the identification issue mentioned above appears usually when you have several LUNs exposed simultaneously to multiple HBAs or multiple machines. So the whole problem won't really be an issue when you don't need a SAN, and you have only direct attach storage. This is why you won't see the Page #83 feature implemented in regular SCSI adapters.

What about the world of Windows?

The importance of the globally-unique identifiers is growing, and it already affected our Windows world. In the upcoming Windows Server SP1 thre are a number of improvements, and some one of them are specifically dealing with unique identifiers. A less-known FAQ (presented here) details various improvements of Microsoft Clusters in SP1. A nice snippet is this one (I underlined the relevant part):

So not only that SP1 will recommend compliant hardware that supports unique identifiers, but it also uses this feature to offer an advanced facility - individual LUN resets.

But you might be wondering - do I need to understand all this whacky business of manually issuing SCSI commands directly to the controller? Do I have to compute bits here and there as the SPC-3 standard mentions? Fortunately no - the storage stack already implements most of this heavy stuff for you.

In our Windows world, a LUN usually appears as a single disk in a given system (there are exceptions there with multi-paths, but I won't go there right now). So how can you programatically take a look to the SCSI unique ID of your disk device? The answer is the IOCTL_STORAGE_QUERY_PROPERTY command, available both from kernel-mode and user-mode. You can easily get the list of unique identifiers in this way, if the SCSI controller that manages your disk supports Page 83 and unique SCSI identifiers. (note, however, that storport.sys is recommended when you want to deal with the Page #83 identifiers)

Here is the process needed to obtain the list of storage identifiers for a disk:

1) Open a handle to the disk device (\\.\PHYSICALDRIVEnnn) with read/write access (probably GENERIC_READ will be enough)

2) Fill in the STORAGE_PROPERTY_QUERY structure.
- Set the QueryType field to PropertyStandardQuery
- Set the PropertyId field to StorageDeviceIdProperty (don't confuse it with StorageDeviceProperty which will cause the IOCTL to return other things, for example the Page #80 data, etc.)

3) Issue the IOCTL_STORAGE_QUERY_PROPERTY with the structure above as its input parameters. After a succesful completion, cast the output buffer to a STORAGE_DEVICE_ID_DESCRIPTOR structure - you will have a BYTE array containing the identifiers, obviously, in the Identifiers field. If this fails, then maybe this is not a SCSI disk, or maybe the SCSI controller doesn't support Page 83. Make sure to supply enough data for the output buffer, otehrwise you will get ERROR_MORE_DATA/ERROR_INSUFFICIENT_BUFFER.

4) Parse the identifiers. Note that not all of these identifiers are associated with the LUN. Some identifiers might be actually related to the path between your HBA and the LUN, others might be related to the SCSI target. Here is the returned structure (see ntddstor.h from the Windows DDK for more details):

typedef struct _STORAGE_DEVICE_ID_DESCRPTOR {
ULONG Version;
ULONG Size;
ULONG NumberOfIdentifiers;
UCHAR Identifiers[1];
} STORAGE_DEVICE_ID_DESCRIPTOR, *PSTORAGE_DEVICE_ID_DESCRIPTOR;

The returned byte array contain a list of STORAGE_IDENTIFIER C++ structures used by Windows:

typedef struct _STORAGE_IDENTIFIER {
STORAGE_IDENTIFIER_CODE_SET CodeSet;
STORAGE_IDENTIFIER_TYPE Type;
USHORT IdentifierSize;
USHORT NextOffset;
STORAGE_ASSOCIATION_TYPE Association;
UCHAR Identifier[1];
} STORAGE_IDENTIFIER, *PSTORAGE_IDENTIFIER;

We notice that the layout of the structure is a little bit different with the structure mentioned above, but the most important fields (Type, Association, CodeSet, Identifier) are exactly the same.

Parsing the identifiers can be done in this way:
4.1) Initialize a BYTE pointer to the address of this the Identifiers field above.
4.2) Cast this pointer to a STORAGE_IDENTIFIER structure.
4.2) Get the association of this identifier. If not zero, go to the next identifier (step 4.5)
4.3) Get the type of this identifier. If not 1,2,3 or 8, go to the next identifier (step 4.5)
4.4) You obtained a unique identifier! Print it, etc.
4.5) Add NextOffset to the current BYTE pointer and go to step 4.2)

Here is some associated sample code that I wrote:

 #include <windows.h>#include <stdio.h> #include <ntddstor.h> // Query the Page 80 informationvoid QueryPage80(LPWSTR pwszDiskDevice, HANDLE hDiskDevice){     // The input parameter     STORAGE_PROPERTY_QUERY query;    query.PropertyId = StorageDeviceProperty;    query.QueryType = PropertyStandardQuery;     // Prepare a large output buffer (good enough in our sample code)    BYTE bOutputBuffer[8192];    DWORD returnedLength;    if (!DeviceIoControl(        hDiskDevice,                        IOCTL_STORAGE_QUERY_PROPERTY,        &query,        sizeof( STORAGE_PROPERTY_QUERY ),        &bOutputBuffer,                           sizeof(bOutputBuffer),                              &returnedLength,              NULL                            ))    {        wprintf(L"\nERROR: Cannot request Page #80 information for device '%s'.\n "            L"[DeviceIoControl() error: %d]\n",             pwszDiskDevice, GetLastError());         return;    }     //    //  Get some basic data about our disk device    //     STORAGE_DEVICE_DESCRIPTOR *pDesc = (PSTORAGE_DEVICE_DESCRIPTOR) bOutputBuffer;     //  Get the Page 80 information    //  This code assumes zero-terminated strings, according to the spec     if (pDesc->VendorIdOffset != 0)        wprintf(L"- Page80.VendorId: %hs\n", (PCHAR)((PBYTE)pDesc + pDesc->VendorIdOffset));     if (pDesc->ProductIdOffset != 0)        wprintf(L"- Page80.ProductId: %hs\n", (PCHAR)((PBYTE)pDesc + pDesc->ProductIdOffset));     if (pDesc->ProductRevisionOffset != 0)        wprintf(L"- Page80.ProductRevision: %hs\n", (PCHAR)((PBYTE)pDesc + pDesc->ProductRevisionOffset));     if (pDesc->SerialNumberOffset != 0)        wprintf(L"- Page80.SerialNumber: %hs\n", (PCHAR)((PBYTE)pDesc + pDesc->SerialNumberOffset));}  // Query the Page 83 informationvoid QueryPage83(LPWSTR pwszDiskDevice, HANDLE hDiskDevice){     // The input parameter     STORAGE_PROPERTY_QUERY query;    query.PropertyId = StorageDeviceIdProperty;    query.QueryType = PropertyStandardQuery;     // Prepare a large output buffer (good enough in our sample code)    BYTE bOutputBuffer[8192];    DWORD returnedLength;    if (!DeviceIoControl(        hDiskDevice,                        IOCTL_STORAGE_QUERY_PROPERTY,        &query,        sizeof( STORAGE_PROPERTY_QUERY ),        &bOutputBuffer,                           sizeof(bOutputBuffer),                              &returnedLength,              NULL                            ))    {        wprintf(L"\nERROR: Cannot request SCSI Inquiry VPD Page #83 information for device '%s'.\n"            L"Maybe it doesn't support Page 83?\n"            L"[DeviceIoControl() error: %d]\n",             pwszDiskDevice, GetLastError());         return;    }     STORAGE_DEVICE_ID_DESCRIPTOR *pDesc = (PSTORAGE_DEVICE_ID_DESCRIPTOR) bOutputBuffer;     // Listing all identifiers...    wprintf(L"\n- Page83.NumberOfIdentifiers: %d\n", pDesc->NumberOfIdentifiers);     STORAGE_IDENTIFIER *pId = (PSTORAGE_IDENTIFIER) pDesc->Identifiers;    for(UINT i = 0; i < pDesc->NumberOfIdentifiers; i++)    {        // Checks if this Identifier is unique        bool isUnique = false;        if (pId->Association == StorageIdAssocDevice)        {            if (((INT)pId->Type == StorageIdTypeVendorId)                || ((INT)pId->Type == StorageIdTypeEUI64)                || ((INT)pId->Type == StorageIdTypeFCPHName)                || ((INT)pId->Type == StorageIdTypeScsiNameString))            {                isUnique = true;            }        }         wprintf(L"\n- Page83.Identifier\n", i);        wprintf(L"   - Type: %d\n", pId->Type);        wprintf(L"   - Association: %d\n", pId->Association);        wprintf(L"   - Size: %d\n", pId->IdentifierSize);        wprintf(L"   - IsGloballyUnique? %s\n", isUnique? L"TRUE": L"FALSE");         wprintf(L"   - Data: ");        for(int j = 0; j < pId->IdentifierSize; j++)            wprintf(L"%02hx ", pId->Identifier[j]);        wprintf(L"\n");         // move to next identifier        pId = (PSTORAGE_IDENTIFIER) ((BYTE *) pId + pId->NextOffset);    }}  // Entry pointextern "C" __cdecl wmain(int argc, WCHAR ** argv){    if (argc != 2)    {        wprintf(L"- You must specify a disk device! as argument!\n");         return;    }     WCHAR * pwszDiskDevice = argv[1];     wprintf(L"Querying information for disk '%s' ... \n", pwszDiskDevice);     // Opening a handle to the disk      HANDLE hDiskDevice = CreateFile(        pwszDiskDevice,        GENERIC_READ | GENERIC_WRITE,       // dwDesiredAccess        FILE_SHARE_READ | FILE_SHARE_WRITE, // dwShareMode        NULL,                               // lpSecurityAttributes        OPEN_EXISTING,                      // dwCreationDistribution        0,                                  // dwFlagsAndAttributes        NULL                                // hTemplateFile        );     if (hDiskDevice == INVALID_HANDLE_VALUE)    {        wprintf(L"\nERROR: Cannot open device '%s'. "            L"Did you specified a correct SCSI device? "            L"[CreateFile() error: %d]\n",             pwszDiskDevice, GetLastError());         return;    }     // Query the Page 80 information    QueryPage80(pwszDiskDevice, hDiskDevice);     // Query the Page 83 information    QueryPage83(pwszDiskDevice, hDiskDevice);} 

The program above is a simple console application, which issues the IOCTL_QUERY_STORAGE_PROPERTIES twice to the same disk. First time, to get the Page #80 information, and the second time for the Page #83 information, to retrieve the list of identifiers. The program takes only one parameter, which is the disk device. You need, however, the Microsoft DDK to ge the latest version of the ntddstor.h header.

Let's take it for a test drive

I compiled the program utility drive above and ran it to some disk drives. First thing to note - the program accepts a disk device as its parameter. You can obtain the disk ID of a certain volume by looking into the Disk Management (diskmgmt.msc). Programatically, you can get the same information through the high-level WMI interfaces (see this link for some sample code in VBScript).

Looking in diskmgmt.msc we can see the association between disks and volumes. For example, the disk with the ID "0" corresponds to the C:\ volume, while the disk with the ID 1 is associated not associated with any volume. These disk IDs - these are unique numbers assigned at each disk rescan operation (and implicitly at each boot time). Therefore, these IDs might change at the next reboot.

There is one important thing to note about these disk IDs. Any disk with ID "n" will have an associated device (in the form of a DOS symbolic link) in the format "\\.\PHYSICALDRIVEn". For example, the disk "0" above will have the device \\.\PHYSICALDRIVE0, and so on.

Let's run our program on the C:\ volume. We get something like this:

C:\>identifier.exe \\.\PHYSICALDRIVE0
Querying information for disk '\\.\PHYSICALDRIVE0' ...
- Page80.VendorId: FUJITSU
- Page80.ProductId: MAM3184MC
- Page80.ProductRevision: 5A01
- Page80.SerialNumber: UKL3P2904PCH

ERROR: Cannot request SCSI Inquiry VPD Page #83 information for device '\\.\PHYS
ICALDRIVE0'.
Maybe it doesn't support Page 83?
[DeviceIoControl() error: 50]

This actually makes sense, since the Disk 0 was tied to the local C:\ volume, which is a local SCSI drive. Its SCSI adapter does not implement Page 83. Now, how about Disk 1?

C:\>identifier.exe \\.\PHYSICALDRIVE1
Querying information for disk '\\.\PHYSICALDRIVE1' ...
- Page80.VendorId: COMPAQ
- Page80.ProductId: HSV110 (C)COMPAQ
- Page80.ProductRevision: 2001
- Page80.SerialNumber: P4889B59IM604V

- Page83.NumberOfIdentifiers: 1

- Page83.Identifier
- Type: 3
- Association: 0
- Size: 16
- IsGloballyUnique? TRUE
- Data: 60 05 08 b4 00 01 4a 11 00 01 90 00 87 a1 00 00

This time we have more luck. The Disk 1 (implemented in a LUN in a HSV 110 HP EVA array) has one unique identifier of type 3 (FP-CH). In conclusion, our LUN has a globally unique identifier.

If you have a SAN around, give it a try the same code on your machines - you might get slightly different results, depending on the hardware model or even the firmware version. Make sure that you have Windows Server 2003 installed, with the latest storport.sys QFE installed (KB 883646 at the time of the writing). And please let me know (in the comments section) if you found an old hardware that doesn't support yet globally unique identifiers...