Processing Configurable Threshold Based Rules in Azure Stream Analytics

This post covers the usage of Azure Stream Analytics to process configurable threshold based rules. Canonical scenarios where an alert is to be generated when an event with a certain value occurs or when an aggregated value exceeds a certain threshold can be articulated as threshold based rules in Azure Stream Analytics. These queries are…


Troubleshooting Azure Stream Analytics jobs with SELECT INTO

Azure Stream Analytics is a fully managed service to do real time processing of data with a flexible SQL-like language. You can easily construct queries to complete complex analysis. However, it also means sometimes the system can be hard to troubleshoot when it does not run as expected. Here is one trick that can help…


How to deal with missing events in streaming data?

Streaming data is often not perfect – some of the events can be missing and some can be generated or received with delay. At the same time downstream applications may require input data within regular intervals (e.g. every 5 seconds) Some customers asked us – how can Azure Stream Analytics be used to convert a…


Handling Json array in Stream Analytics Query

This week, I am sharing a query question asked in stackoverflow to illustrate how to handle json array in Stream Analytics Query:   Problem   I retrieve some weatherdata from an external API. This is returned as JSON and send to an Azure IoT hub. Stream analytics processes the json into a proper format, but I…


Learn more about Azure Stream Analytics Time Skew Policies

In Stream Analytics, all data stream events have a timestamp associated with them. As all events are temporal in nature and timing of arrival of the event is how the timestamp is assigned, considerations exists for both the tolerance of out of order events and the late arrival of events to the Stream Analytics job….


Azure Streaming Units – how do I monitor? how do I scale?

Our CSS team does a fabulous job reaching out to us with customer questions, so we can draft out some documentation to address common questions.. Bill Carroll, Senior Escalation Engineer reached out to us today with this question. I have a customer who is monitoring the SU% as reported in management web app’s monitor widget. He…


Querying JSON array with Azure Stream Analytics

Kent Weare has written a nice post on querying Json array with Azure Stream Analytics. He is getting device reads off of an Azure Event Hub. These reads are being aggregated on the publisher side and placed into a single message/event. Since the publisher is creating a message structure that contains many device reads for that specific interval Kent…


Stream Analytics query to check when no data has arrived from a given device in a configurable time window

Paolo Salvatori describes well here how you might write a stream analytics query to check when no data has arrived from a given device in a configurable time window. To solve this problem, the idea is to correlate the data stream containing real-time events (e.g. sensor readings) with device reference data. Thanks again Paolo for…