Retrieving Resource Metrics via the Azure Insights API

  • Article

There are many options for configuring monitoring and alerting for Azure resources. The Azure portal will show some default metrics. It is also possible to enable more advanced or custom diagnostic settings. The links below provide additional details on enabling diagnostics and monitoring for two popular Azure compute resources, Azure Web Apps and Azure Virtual Machines. Visual Studio Application Insights can also be leveraged to monitor the usage and performance of applications.

As previously mentioned, the Azure portal will show some default metrics for many Azure resources. For example, the screenshot below shows the monitoring tile for an Azure Web App, which has been configured to display three key values: Average Response Time, CPU Time, and Requests.

azure_web_app_default_metric_tile

The metric values displayed on this tile are not retained indefinitely. For an Azure Web App, the retention policy is as follows:

  • Minute granularity – 24 hour retention
  • Hour granularity – 7 day retention
  • Daily granularity – 30 day retention

By using the Azure Insights API it is possible to programmatically retrieve the available default metric definitions (the type of metric such as CPU Time, Requests, etc.), granularity, and metric values. With the ability to programmatically retrieve the data, comes the ability to save the data in a data store of your choosing. For example, that data could be persisted to Azure SQL Database, DocumentDB, or Azure Data Lake. From there you could perform whatever additional analysis is desired.

It should be noted that the Azure Insights API is not the same as Application Insights.

Besides working with various metric data points, the Insights API allows you to manage things like alerts, autoscale settings, usage quotas, and more. Check out the full list via the Azure Insights REST API Reference documentation.

The remainder of this post will discuss using the Insights API to learn more about the default metrics available for Azure resources.

Investigating Available Metrics via the Insights REST API

There are three basic steps for working with the Insights REST API:

  1. Authenticate the Azure Insights request
  2. Retrieve the available metric definitions
  3. Retrieve the metric values

The first step is to authenticate the Azure Insights API request. As the Azure Insights API is an Azure Resource Manager based API, it requires authentication via Azure Active Directory (Azure AD). The easiest way (in my opinion at least) to set up authentication is by creating an Azure AD service principal and retrieve the authentication (JWT) token. The sample script below demonstrates creating an Azure AD service principle via PowerShell. For a more detailed walkthrough, please reference the guidance at https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal/#authenticate-service-principal-with-password---powershell. It is also possible to create a service principle via the Azure portal.

Create a Service Principle

  1. # Instructions at https://azure.microsoft.com/en-us/documentation/articles/resource-group-authenticate-service-principal/

  3. $pwd = "[your-service-principle-password]"

  4. $subscriptionId = "[your-azure-subscription-id]"

  6. Login-AzureRmAccount

  7. Select-AzureRmSubscription -SubscriptionId $subscriptionId

  9. $azureAdApplication = New-AzureRmADApplication `

  10.                         -DisplayName "Collier Web Metrics Demo" `

  11.                         -HomePage "https://localhost/webmetricdemo" `

  12.                         -IdentifierUris "https://localhost/webmetricdemo" `

  13.                         -Password $pwd

  15. New-AzureRmADServicePrincipal -ApplicationId $azureAdApplication.ApplicationId

  16. New-AzureRmRoleAssignment -RoleDefinitionName Reader -ServicePrincipalName $azureAdApplication.ApplicationId

  18. $subscription = Get-AzureRmSubscription -SubscriptionId $subscriptionId

  19. $creds = Get-Credential -UserName $azureAdApplication.ApplicationId -Message "Please use your service principle credentials"

  21. Login-AzureRmAccount -Credential $creds -ServicePrincipal -TenantId $subscription.TenantId

Once the authentication setup step is complete, it is possible to execute queries against the Azure Insights REST API. There are two helpful queries:

  1. List the metric definitions for a resource
  2. Retrieve the metric values

Details on listing the metric definitions for a resource is documented at https://msdn.microsoft.com/en-us/library/azure/dn931939.aspx. For an Azure Web App, the metric definitions should look similar to example screenshot below.

azure_web_app_metric_definitions_with_pointers

Once the available metric definitions are known, it is easy to retrieve the required metric values. Use the metric's name 'value' (not the 'localizedValue') for any filtering requests (e.g. retrieve the 'CpuTime' and 'Requests' metric data points). The request / response information for this API call do not appear as an available task at https://msdn.microsoft.com/en-us/library/azure/dn931930.aspx. However, it is possible to do so, and the request URI is very similar to that of listing the metric definitions.

Method Request URI
GET https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/{resource-provider-namespace}/{resource-type}/{resource-name}/metrics?api-version=2014-04-01&$filter={filter}

For example, to retrieve just the Average Response Time and Requests metric data points for an Azure Web App for the 1 hour period from 2016-02-18 20:26:00 to 2016-02-18 21:26:00, with a granularity of 1 minute, the request URI would be as follows:

 https://management.azure.com/subscriptions/{subscription-id}/resourceGroups/collierwebapi2016/providers/Microsoft.Web/sites/collierwebapi2016/metrics?api-version=2014-04-01&$filter=%28name.value%20eq%20%27AverageResponseTime%27%20or%20name.value%20eq%20%27Requests%27%29%20and%20timeGrain%20eq%20duration%27PT1M%27%20and%20startTime%20eq%202016-02-18T20%3A26%3A00.0000000Z%20and%20endTime%20eq%202016-02-18T21%3A26%3A00.0000000Z

The result should be similar to that of the screenshot below.

azure_web_app_metric_values

Using the REST API is very helpful in terms of understand the available metric definitions, granularity, and related values. This information can be very helpful when using the Azure Insights Management Library.

Retrieving Metrics via the Insights Management Library

Just like working with the REST API, there are three basic steps for working with the Insights Management Library:

  1. Authenticate the Azure Insights request
  2. Retrieve the available metric definitions
  3. Retrieve the metric values

The first step is to authenticate by retrieving the JWT token from Azure AD. Assuming the Azure AD service principle is already configured, retrieving the token can be as simple as shown in the code sample below.

Get Auth Token

  1. private static string GetAccessToken()

  2. {

  3.     var authenticationContext = new AuthenticationContext(string.Format("https://login.windows.net/{0}", _tenantId));

  4.     var credential = new ClientCredential(clientId: _applicationId, clientSecret: _applicationPwd);

  5.     var result = authenticationContext.AcquireToken(resource: "https://management.core.windows.net/", clientCredential: credential);

  7.     if (result == null)

  8.     {

  9.         throw new InvalidOperationException("Failed to obtain the JWT token");

  10.     }

  12.     string token = result.AccessToken;

  13.     return token;

  14. }

 

The primary class for working with the Insights API is the InsightsClient. This class exposes functionality to retrieve the available metric definitions and metric values, as seen in the sample code below:

Get Metric Data

  1. private static MetricListResponse GetResourceMetrics(TokenCloudCredentials credentials, string resourceUri, string filter, TimeSpan period, string duration)

  2. {

  3.     var dateTimeFormat = "yyy-MM-ddTHH:mmZ";

  5.     string start = DateTime.UtcNow.Subtract(period).ToString(dateTimeFormat);

  6.     string end = DateTime.UtcNow.ToString(dateTimeFormat);

  8.     StringBuilder sb = new StringBuilder(filter);

  9.     if (!string.IsNullOrEmpty(filter))

  10.     {

  11.         sb.Append(" and ");

  12.     }

  13.     sb.AppendFormat("startTime eq {0} and endTime eq {1}", start, end);

  14.     sb.AppendFormat(" and timeGrain eq duration'{0}'", duration);

  16.     using (var client = new InsightsClient(credentials))

  17.     {

  18.         return client.MetricOperations.GetMetrics(resourceUri, sb.ToString());

  19.     }

  20. }

  22. private static MetricDefinitionListResponse GetAvailableMetricDefinitions(TokenCloudCredentials credentials, string resourceUri)

  23. {

  24.     using (var client = new InsightsClient(credentials))

  25.     {

  26.         return client.MetricDefinitionOperations.GetMetricDefinitions(resourceUri, null);

  27.     }

  28. }

 

For the above code, the resource URI to use is the full path to the desired Azure resource. For example, to query against an Azure Web App, the resource URI would be:

 /subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/Microsoft.Web/sites/{site-name}/

 

It is also possible to query the metric data for a classic Azure Virtual Machine – just change the request URI to be appropriate for the classic VM:

 /subscriptions/{subscription-id}/resourceGroups/{resource-group-name}/providers/microsoft.classiccompute/virtualmachines/{vm-name}/

 

To find the resource URI for a desired resource, one approach is to use the https://resources.azure.com tool. Simply browse to the desired resource and then look at the URI shown, as in the screenshot below.

csablog-resource_explorer

 

For the full code sample, please see my GitHub repository available at https://github.com/mcollier/microsoft-azure-insights-demo/.

Thanks to Yossi Dahan's blog post at https://yossidahan.wordpress.com/2015/02/13/reading-metric-data-from-azure-using-the-azure-insights-library/ for the inspiration. azure_web_app_metric_value_listing