Intro to Diagnostics for Azure Stream Analytics


Azure Stream Analytics offers a number of diagnostics tools to help investigate problems with streaming jobs:

1.       Management Services (Also known as : Operations Logging)

2.       Monitoring metrics

3.       No-Output Diagnostics

4.       Alerts On Metrics

5.       Alerts On Operations Logs

 

1.       Management Services: This is a standard way in Azure services to supply operational logging messages to users. In general, this is where you would find details related to all management operations of Azure Stream Analytics. A link to the corresponding logs of an Azure Stream Analytics (ASA) job is displayed on the job’s Dashboard tab. If you click on that link, it will set the filters in a way that it shows latest logs for that specific ASA job.

ASA writes various operations and their statuses to this log. Examples of Operation names are (this is not complete list):

  • Receive Events
  • Send Events
  • Process Events
  • Serialize Events
  • ASA Job Management Operations such as:
    • Start job ‘ASA-Job-Name’
    • Stop job ‘ASA-Job-Name’
    • Update job ‘ASA-Job-Name’

Possible statuses are:

  • Validated
  • Starting
  • Started
  • Completed
  • Success
  • Failed
  • Stopping
  • Stopped

User can select on any of the events logged, and click on the Details button at the bottom of the window, it shows more details on that event. These details also contain information that is required to debug service side issues if you were to contact Microsoft Support for help.

An example of the Management Services OR Operations Logging service is shown here:

  
   

     

2. Monitoring Metrics: Just like any other Azure service, Monitoring metrics are surfaced in Monitor tab of the Azure Stream Analytics Job. The charts of metrics over a period of time are shown on this tab. The main use of the metrics is to understand the trends, to set alerts if they reach certain thresholds, and to correlate them to other diagnostics information to debug issues.

Here is an example of the Monitor tab.

 

 

3. No-Output Diagnostics: This feature is specific to Azure Stream Analytics jobs. The main usage of this feature is to surface the errors related to Input or Output of the Job that can cause the job to stop generating output. They often need user’s immediate attention, so they are show to the user on the job’s dashboard itself. These error are also logged to Operations Logs.

Diagnostics messages are listed on the Dashboard tab and details of these messages are listed in Inputs, Outputs Tabs and you can drill into these issues as follows.

 

Drilling into issues listed in Outputs tab as an example:

Once you click on that resource (Output), you will see more information such as below.

 

4. Alerts On Metrics : It is described in the following link on how to set alerts on azure services and it applies to Azure Stream Analytics as well: How to: Receive Alert Notifications and Manage Alert Rules in Azure

 

5. Alerts On Operations Logs : There is no UX support for creating alerts on operational logs. However, you can create them programmatically like you do for other Azure services. Example on how to do this for Azure services is explained in below article and that applies to Azure Stream Analytics service as well.

https://code.msdn.microsoft.com/windowsazure/Receive-Email-Notifications-199e2c9a

Comments (0)

Skip to main content