Query Pattern of the Week: Send data to multiple outputs


Have you checked out our team’s collection of Common Stream Analytics Query Patterns? This location acts as a repository for query patterns commonly used by our customers. One pattern that frequently comes up in real-world applications is directing job data to multiple outputs to enable both a hot path and a cold path for data.  Read on for more details!

Description: In a single job, send processed data to multiple outputs

Example: Analyze a stream of vehicle data and alert over a given condition, while archiving all events to long term storage.

Input:

Make

Time

Honda

2015-01-01T00:00:01.0000000Z

Honda

2015-01-01T00:00:02.0000000Z

Toyota

2015-01-01T00:00:01.0000000Z

Toyota

2015-01-01T00:00:02.0000000Z

Toyota

2015-01-01T00:00:03.0000000Z

Test File:

Download from GitHub: MultipleOutputs.json

Query:

SELECT

    *

INTO

    ArchiveOutput

FROM

    Input TIMESTAMP BY Time

 

 

SELECT

    Make,

    System.TimeStamp AS Time,

    COUNT(*) AS[Count]

INTO

    AlertOutput

FROM

    Input TIMESTAMP BYTime

GROUP BY

    Make,

    TumblingWindow(second, 10)

HAVING                

    [Count] >= 3

 

Outputs:

ArchiveOutput:

Make

Time

Honda

2015-01-01T00:00:01.0000000Z

Honda

2015-01-01T00:00:02.0000000Z

Toyota

2015-01-01T00:00:01.0000000Z

Toyota

2015-01-01T00:00:02.0000000Z

Toyota

2015-01-01T00:00:03.0000000Z

AlertOutput:

Make

Time

Count

Toyota

2015-01-01T00:00:10.0000000Z

3

Explanation:

The INTO clause tells Stream Analytics which of the outputs to direct data to.  The first query is a pass-through of all data received by the job. The second query does some simple aggregation and filtering and could potentially send results to a downstream alerting system.

Looking for more examples?

To learn about more patterns, check out the Common Stream Analytics Query Patterns page.  Looking to solve a problem that you don’t see captured yet?  Let us know via our msdn forum or by tweeting us @AzureStreaming.

Comments (0)

Skip to main content