Processing Configurable Threshold Based Rules in Azure Stream Analytics

This post covers the usage of Azure Stream Analytics to process configurable threshold based rules. Canonical scenarios where an alert is to be generated when an event with a certain value occurs or when an aggregated value exceeds a certain threshold can be articulated as threshold based rules in Azure Stream Analytics. These queries are…


Troubleshooting Azure Stream Analytics jobs with SELECT INTO

Azure Stream Analytics is a fully managed service to do real time processing of data with a flexible SQL-like language. You can easily construct queries to complete complex analysis. However, it also means sometimes the system can be hard to troubleshoot when it does not run as expected. Here is one trick that can help…


How to deal with missing events in streaming data?

Streaming data is often not perfect – some of the events can be missing and some can be generated or received with delay. At the same time downstream applications may require input data within regular intervals (e.g. every 5 seconds) Some customers asked us – how can Azure Stream Analytics be used to convert a…


Handling Json array in Stream Analytics Query

This week, I am sharing a query question asked in stackoverflow to illustrate how to handle json array in Stream Analytics Query:   Problem   I retrieve some weatherdata from an external API. This is returned as JSON and send to an Azure IoT hub. Stream analytics processes the json into a proper format, but I…


Querying JSON array with Azure Stream Analytics

Kent Weare has written a nice post on querying Json array with Azure Stream Analytics. He is getting device reads off of an Azure Event Hub. These reads are being aggregated on the publisher side and placed into a single message/event. Since the publisher is creating a message structure that contains many device reads for that specific interval Kent…


Stream Analytics query to check when no data has arrived from a given device in a configurable time window

Paolo Salvatori describes well here how you might write a stream analytics query to check when no data has arrived from a given device in a configurable time window. To solve this problem, the idea is to correlate the data stream containing real-time events (e.g. sensor readings) with device reference data. Thanks again Paolo for…


Tumbling, Sliding and Hopping Windows with Azure Streaming

I found these series of posts by @DesertIsleSQL very well written describing Tumbling, Sliding and Hopping Window functions in Azure Streaming queries. Incorporating Azure Streaming Analytics with Azure ML – Part 1 Using Tumbling Windows to Select Data from Azure Stream Analytics – Part 2 Azure Stream Analytics Hopping – Part 3    


Query Pattern of the Week: Send data to multiple outputs

Have you checked out our team’s collection of Common Stream Analytics Query Patterns? This location acts as a repository for query patterns commonly used by our customers. One pattern that frequently comes up in real-world applications is directing job data to multiple outputs to enable both a hot path and a cold path for data….

0

Query Pattern of the Week: Specify logic for different cases

Last week we kicked off a new blog series to highlight a query pattern with a real world example every week.  This week we examine how to use CASE statements to specify conditional logic.  For more query patterns, check out the Common Stream Analytics Query Patterns page. Description: Evaluate one of multiple results based on…

0

Working with complex data types in Azure Stream Analytics

Azure Stream Analytics supports processing events in a variety of data formats (CSV, JSON, Avro). JSON and Avro can contain complex types such as nested objects (records) or arrays. Many Stream Analytics customers use nested data and we have received various questions related to processing events containing records and/or arrays. This blog post provides an…