Partner Center API and Azure Resource Utilization

Thanks to the hard work from the Partner Center team we are now able to query Azure usage records directly from the Partner Center API. With this update to the Partner Center API we can now perform everything that the CREST API can and a whole lot more! Numerous partners ask how they can generate an itemized invoices for their customer’s Azure consumption. Until now there was not a good way to accomplish this. With this post I want to focus on how to utilize Azure usage records to create an itemized invoice for Azure consumption. The remaining sections of this post will introduce the concepts and provide some sample code. All sample code provided in this post is provided as-is and with no warranty.

Tags

Tags in regards to Azure are key value pairs that identity resources with properties that you define. This feature allows us to logically group resource groups and resources within the Azure. You can add tags using the Azure Resource Manager API, command line interface, management portal, PowerShell, or templates. With all of these options to assign tags it is easy to design a provisioning workflow that will ensure all resources have the appropriate tags assigned.

One of the more common use cases for tags is to generate an itemized invoice. With a direct Azure subscription you could follow the documentation available here, however, those instructions will not work with Cloud Solution Provider (CSP) subscriptions. Tags applied to resource groups are not inherited by the resources contained with  So in order to construct the invoice correctly you will need to be sure that the appropriate tags have been applied to all of the resources.

In order to demonstrate how tags can be utilized to construct the invoice I have provisioned an instance of App Service and Redis Cache. The instance of App Services has a costCenter tag assigned with a value of Operations, and the instance of Redis Cache has a costCenter tag assigned with a value of Infrastructure.  

Azure Utilization Records

Azure utilization records provide information pertaining to the consumption of Azure services. When you supplement these records with details from the Azure RateCard it is possible to calculate the total charges that have been incurred. When we query the Partner Center API for usage records, using the utilization records feature, it returns JSON that is similar to the following

 
{
  "usageStartTime": "2016-12-12T16:00:00-08:00",
  "usageEndTime": "2016-12-13T16:00:00-08:00",
  "resource": {
    "id": "9cb0bde8-bc0d-468c-8423-a25fe06779d3",
    "name": "Standard IO - Table Write Operation Units (in 10,000s)",
    "category": "Data Management",
    "subcategory": ""
  },
  "quantity": 0.0146,
  "unit": "10,000s",
  "infoFields": {},
  "instanceData": {
    "resourceUri": "/subscriptions/8e4357c6-bd40-4a49-b169-ced26d105d70/resourceGroups/securitydata/providers/Microsoft.Storage/storageAccounts/myaccount",
    "location": "ussouthcentral",
    "partNumber": "",
    "orderNumber": ""
  },
  "attributes": { "objectType": "AzureUtilizationRecord" }
}

As you can use the record being returned contains a wealth of information. If you are using the Partner Center Managed API then it will serialize the response into an object that you can perform operations against. I would like to point out that the resource group name is not one of the properties included with a usage record. However, you can obtain the resource group name with a little bit of work by parse the resourceUri property. The following snippet of code shows one way to obtain all of the usage records for all of the subscription that the specified customer has

 
/// <summary>
/// Gets all Azure usage records for the specified customer. 
/// </summary>
/// <param name="customerId">Identifier for the customer of interest.</param>
/// <returns>A collection of Azure usage records.</returns>
/// <exception cref="ArgumentException">
/// customerId
/// </exception>
public Dictionary<string, List<AzureUtilizationRecord>> GetUsageRecords(string customerId)
{
    Dictionary<string, List<AzureUtilizationRecord>> usageRecords;
    ResourceCollection<Subscription> subscriptions;

    if (string.IsNullOrEmpty(customerId))
    {
        throw new ArgumentException(nameof(customerId));
    }

    try
    {
        subscriptions = PartnerCenter.Customers.ById(customerId).Subscriptions.Get();
        usageRecords = new Dictionary<string, List<AzureUtilizationRecord>>();

        foreach (Subscription s in subscriptions.Items.Where(x => x.BillingType == BillingType.Usage))
        {
            ResourceCollection<AzureUtilizationRecord> records =
                PartnerCenter.Customers.ById(customerId).Subscriptions.ById(s.Id).Utilization.Azure.Query(
                DateTime.Now.AddMonths(-1), DateTime.Now);

            usageRecords.Add(s.Id, new List<AzureUtilizationRecord>(records.Items));
        }

        return usageRecords;
    }
    finally
    {
        subscriptions = null;
    }
}

Now that we have a way to obtain Azure usage records the remaining steps in the process, of creating the appropriate invoice, are rather straight forward. With this post I am considering an itemized based invoice as one that groups resources based upon a tag with a key of costCenter. With all of this is in mind we can leverage code similar to the following to generate the invoice for a specific customer.

 
/// <summary>
/// Generates an itemized based invoice for the speicifed customer.
/// </summary>
/// <param name="customerId">Identifier assign to the customer of interest.</param>
/// <remarks>
/// This function will group resources based upon the value of the costCenter tag that 
/// is applied to the resource. All resources that do not have a costCenter tag will be 
/// grouped together and reported accordingly.
/// </remarks>
public void GenerateInvoice(string customerId)
{
    AzureRateCard rateCard;
    Dictionary<string, List<AzureUtilizationRecord>> usageRecords;
    List<AzureUtilizationRecord> costCenterRecords;
    List<AzureUtilizationRecord> records;
    ICollection<string> costCenters;

    if (string.IsNullOrEmpty(customerId))
    {
        throw new ArgumentException(nameof(customerId));
    }

    try
    {
        rateCard = PartnerCenter.RateCards.Azure.Get();
        usageRecords = GetUsageRecords(customerId);

        foreach (string subscription in usageRecords.Keys)
        {
            // Obtain the collection of usage records that either has no costCenter tag or no tags.
            records = usageRecords[subscription].Where(x => x.InstanceData.Tags == null ||
                x.InstanceData != null && !x.InstanceData.Tags.ContainsKey("costCenter")).ToList();

            // Process the usage records with no tags or no costCenter tag.
            ProcessUsageRecords(rateCard, subscription, string.Empty, records);

            // Obtain the collection of usage records that have a costCenter tag.
            records = usageRecords[subscription].Where(
                x => x.InstanceData.Tags != null && x.InstanceData.Tags.ContainsKey("costCenter")).ToList();

            // Obtain a list of the cost centers utilized.
            costCenters =
                records.Select(x => x.InstanceData.Tags)
                    .Where(x => x.Keys.Contains("costCenter"))
                    .Select(x => x.Values).SingleOrDefault();

            if (costCenters == null)
            {
                continue;
            }

            // Process the usage records with a costCenter tag. 
            foreach (string cs in costCenters)
            {
                costCenterRecords = records.Where(x =>
                    x.InstanceData.Tags.Keys.Contains("costCenter") &&
                    x.InstanceData.Tags["costCenter"].Equals(cs, StringComparison.CurrentCultureIgnoreCase)).ToList();

                ProcessUsageRecords(rateCard, subscription, cs, costCenterRecords);
            }
        }
    }
    finally
    {
        costCenterRecords = null;
        rateCard = null;
        records = null;
        usageRecords = null;
    }
}

private void ProcessUsageRecords(AzureRateCard rateCard, string subscription, string costCenter, IEnumerable<AzureUtilizationRecord> usageRecords)
{
    AzureMeter meter;
    decimal quantity = 0, rate = 0, total = 0;

    if (string.IsNullOrEmpty(subscription))
    {
        throw new ArgumentException(nameof(subscription));
    }

    if (usageRecords == null)
    {
        return;
    }

    try
    {
        Console.WriteLine(string.IsNullOrEmpty(costCenter)
            ? "Usage records with no cost center tag..."
            : $"Usage records for {costCenter} cost center...");

        foreach (AzureUtilizationRecord record in usageRecords)
        {
            // Obtain a reference to the meter associated with the usage record.
            meter = rateCard.Meters.Single(x => x.Id.Equals(record.Resource.Id));
            // Calculate the billable quantity by substracting the included quantity value.
            quantity = record.Quantity - meter.IncludedQuantity;

            if (quantity > 0)
            {
                // Obtain the rate for the given quantity. Some resources have tiered pricing
                // so this assignment statement will select the appropriate rate based upon the
                // quantity consumed, excluding the included quantity. 
                rate = meter.Rates
                    .Where(x => x.Key <= quantity).Select(x => x.Value).Last();
            }

            total += quantity * rate;

            Console.WriteLine("{0}, {1}", meter.Name, (quantity * rate));
        }

        Console.WriteLine($"Total Charges {total}");
    }
    finally
    {
        meter = null;
    }
}

Since all of the code in this post is considered sample code the invoice data is simply being written to the console. The following figure shows sample when you invoke the code

In a real world scenario I would recommend that you leverage something like an Azure WebJob to generate and store invoices in your particular billing system. Hopefully this information will help you the necessary invoices for your customers.