Troubleshooting Azure Stream Analytics jobs with SELECT INTO

Azure Stream Analytics is a fully managed service to do real time processing of data with a flexible SQL-like language. You can easily construct queries to complete complex analysis. However, it also means sometimes the system can be hard to troubleshoot when it does not run as expected. Here is one trick that can help…


How to configure Azure Stream Analytics outputs to skip events that cannot be written due to conversion errors or schema mismatch

By default, when output events cannot be written to the external storage due to being non-conforming data (null columns when destination column is non-nullable, longer strings than the destination column support, values with types that cannot be written in the output etc.) they cause the job to stop processing new events and continuously retry the…


Why aren’t my ASA job results showing up in Azure SQL Server table? How do I debug?

Often we receive questions from users of Azure Stream Analytics which help us improve our documentation or error handling. These Q&A’s also help other users. Here is one such case of figuring out why ASA results may not be appearing in Azure SQL server table. Question: I’m having trouble outputting my ASA job results into…


Learn more about Azure Stream Analytics Time Skew Policies

In Stream Analytics, all data stream events have a timestamp associated with them. As all events are temporal in nature and timing of arrival of the event is how the timestamp is assigned, considerations exists for both the tolerance of out of order events and the late arrival of events to the Stream Analytics job….


Azure Streaming Units – how do I monitor? how do I scale?

Our CSS team does a fabulous job reaching out to us with customer questions, so we can draft out some documentation to address common questions.. Bill Carroll, Senior Escalation Engineer reached out to us today with this question. I have a customer who is monitoring the SU% as reported in management web app’s monitor widget. He…


Notify users of data received from sensors or other systems

How many times have you gone to a vending machine, just to find that the one snack bar you wanted out of the 50,000 things in the machine has run out?  If you are really hungry you could kick the machine, but well that wouldn’t help. Say you write a note to the vending machine company….


Querying JSON array with Azure Stream Analytics

Kent Weare has written a nice post on querying Json array with Azure Stream Analytics. He is getting device reads off of an Azure Event Hub. These reads are being aggregated on the publisher side and placed into a single message/event. Since the publisher is creating a message structure that contains many device reads for that specific interval Kent…


Stream Analytics query to check when no data has arrived from a given device in a configurable time window

Paolo Salvatori describes well here how you might write a stream analytics query to check when no data has arrived from a given device in a configurable time window. To solve this problem, the idea is to correlate the data stream containing real-time events (e.g. sensor readings) with device reference data. Thanks again Paolo for…