Event Processing Ordering Design Choices for Azure Stream Analytics

In this blog, I would like to explain the design choices we made with Azure Stream Analytics for: ‘When’ in processing-time results are materialized, and ‘How’ the results are materialized. These design choices reduce implementation complexities and run time resource usage while meeting customers’ requirements for their scenarios. Firstly, I would like to point out…


SU Utilization Metric

In order to achieve low latency streaming processing, Azure Stream Analytics jobs perform all processing in memory. When running out of memory, the streaming job fails. As a result, for a production job, it’s important to monitor a streaming job’s resource usage, and make sure there is enough resource allocated, in order to keep the…


Using Azure Stream Analytics JavaScript UDF to lookup values in JSON array

JavaScript UDF (User-Defined Function) allows you to handle complex JSON schema and keep your query clean. In this blog you will learn how to handle nested JSON arrays with a JavaScript UDF. Below is an example event generated by an IoT gateway. There is an array of two devices – device01 and device02; each device…

0

Processing Configurable Threshold Based Rules in Azure Stream Analytics

This post covers the usage of Azure Stream Analytics to process configurable threshold based rules. Canonical scenarios where an alert is to be generated when an event with a certain value occurs or when an aggregated value exceeds a certain threshold can be articulated as threshold based rules in Azure Stream Analytics. These queries are…


How to achieve exactly-once delivery for SQL output

Azure Stream Analytics guarantees exactly-once processing within the processing pipeline. However, it currently doesn’t ensure end to end exactly-once delivery to output sink. Instead, it guarantees at-least-once delivery to output sink. When using SQL output, we can achieve exactly-once semantics if the following requirements are met: all output streaming events have a natural key, i.e….