Pulling data from either public or private sources to Azure Event Hub where the data is exposed as a web service or feed

In typical Internet of Things (IoT) scenarios, you have devices that you can program to push data to Azure, either to an Azure Event Hub or an IoT hub. Both of those hubs are entry points into Azure for storing, analyzing, and visualizing with a myriad of tools made available on Microsoft Azure. However, they both require that you push data to them, formatted as JSON and secured in specific ways. This brings up the following question. What do you do if you want to bring in data from either public or private sources where the data is exposed as a web service or feed of some sort, but you do not have the ability to change how the data is published? Consider the weather, or traffic, or stock quotes - you can't tell NOAA, or WSDOT, or NASDAQ to configure a push to your Event Hub. To solve this problem, Spyros Sakellariadis and Dinar Gainitdinov have written and open-sourced a small cloud sample that you can modify and deploy that will pull the data from some such source and push it to your Event Hub. From there, you can do whatever you want with it, subject, of course, to the license terms from the producer. You can find the application here.

Check out the technical documentation and the GenericWebToEH solution code to try it out on some of your favorite data feeds.

For other scenarios there are a number of documentation articles and code samples on pushing data from devices you control to Azure and for analyzing in combination with other streaming or static data.

Comments (0)

Skip to main content