Consuming data from Codename “Dallas”
Codename “Dallas” as part of Windows Azure is an Information Service offering which makes it easy to find and consume data from a number of different languages, both managed such as C# and VB.NET, and other web technologies such as javascript and php due to the interface it provides.
For a good overview and the developer portal for Dallas can be found at:
https://www.microsoft.com/windowsazure/dallas/
In this post I am going to summarise some of the methods of consuming data from Dallas, as mentioned in the Dallas Learning Course on Channel 9:
https://channel9.msdn.com/learn/courses/Azure/Dallas/
and Zach Owen’s part of the Dallas presentation from PDC 09:
https://microsoftpdc.com/Sessions/SVC29
For details on navigating the Dallas Catalog and Datasets the first lab on Channel 9 is very helpful to get to know what is available:
So you have found your data source and want to consume some data…
Using the Generated Proxy from C#
The first way to consume data is using the Generated Proxy Class and use managed code such as C# to access the data.
On the Dallas portal when you have subscribed to a Dataset you should see a list of credentials (Account Key and Unique User ID) and below that a link to Download C# service classes. Download and add this class to your project where you are going to consume the data, and in your consumer class add the using statement:
using Microsoft.Dallas.Services;
Build the solution, and you should be ready to start accessing the services.
In this example I will be using the UN dataset “Free subscription for Food and Agricultural Data”.
Using the code below, populating the accountKey and uniqueUserId with the values from your subscription in the Dallas Portal, this will create a service using the generated FAO3510Service class and read all values into a strongly typed List to iterate over.
// Specify the account key and unique user ID to use for the query
string accountKey = "";
string uniqueUserId = "";
// Create a service proxy with which to execute the query. Note that the
// service proxy requires the
// account key and unique user ID
FAO3510Service service = new FAO3510Service(accountKey, new Guid(uniqueUserId));
// Invoke the query. Note that the Invoke method can take country ID and Year
// parameters. By passing in null values
// for these parameters we are not specifying any criteria for our search.
// Note also that by default, the search will return the first 100 results
// that satisfy the query.
List<FAO3510Item> results = service.Invoke(null, null);
// Set up a header row for the results
Console.WriteLine("{0,13}{1,24}{2,18}{3,7}{4,10}","Series Code", "Country or Area Code", "Country or Area", "Year", "Value");
// Iterate through the result set
foreach (FAO3510Item item in results)
{
Console.WriteLine("{0,13}{1,24}{2,18}{3,7}{4,10}", item.SeriesCode, item.CountryOrAreaCode, item.CountryOrArea, item.Year, item.Value);
}
Console.ReadLine();
As these invoke queries go into strongly typed and LINQ enabled data structures, it is easy to filter and do calculations on the data. And using LINQ’s Join, connect multiple data sets together.
A good example of this is in Zach’s video at around 17:15 https://microsoftpdc.com/Sessions/SVC29
Using RESTful calls over HTTP in Managed code
The second way of consuming data is to use HTTP calls, in this example I am going to use Managed C#.
From the Dallas portal subscription, along with the generated proxies (as above) there is also the URL which may be called along with the AccountKey and UniqueUserID. This is essentially the same as what the proxy abstracted in the above example.
The only difference here is that we also need the URL to be declared along with the AccountKey and UniqueUserId:
string url = "https://api.sqlazureservices.com/UnService.svc/FAO/3510?$format=atom10";
string accountKey = "";
string uniqueUserId = "";
// Establish a Web Request to the Dallas Dataset Url
WebRequest request = WebRequest.Create(url);
// Specify the request headers containing our account key and unique user ID
request.Headers.Add("$accountKey", accountKey);
request.Headers.Add("$uniqueUserID", uniqueUserId);
// Get the response
HttpWebResponse response = (HttpWebResponse)request.GetResponse();
// Write the status of the response to the Console
Console.WriteLine(response.StatusDescription);
// Wait for input from the user
Console.ReadLine();
// Get the stream containing content returned by the server.
Stream dataStream = response.GetResponseStream();
// Open the stream using a StreamReader for easy access.
StreamReader reader = new StreamReader(dataStream);
// Read the content.
string responseFromServer = reader.ReadToEnd();
// Display the content to the Console.
Console.WriteLine(responseFromServer);
// Cleanup the streams and the response.
reader.Close();
dataStream.Close();
response.Close();
// Wait for input from the user before closing the Console window
Console.ReadLine();
This sets up the Web Request, specifies the accountKey and uniqueUserID in the header of the request, gets the response from that request and reads it back as a Stream.
The same thing can be done in VB.NET:
Dim url As String = "https://api.sqlazureservices.com/UnService.svc/FAO/3510?$format=atom10"
Dim accountKey As String = ""
Dim uniqueUserId As String = ""
' Create a request for the URL.
Dim request As WebRequest = WebRequest.Create(url)
' Specify the headers containing our account key and unique user ID
request.Headers.Add("$accountKey", accountKey)
request.Headers.Add("$uniqueUserID", uniqueUserId)
' Get the response.
Dim response As HttpWebResponse = CType(request.GetResponse(), _HttpWebResponse)
' Write the status of the response to the Console.
Console.WriteLine(response.StatusDescription)
' Wait for input from the user
Console.ReadLine()
' Get the stream containing content returned by the server.
Dim dataStream As Stream = response.GetResponseStream()
' Open the stream using a StreamReader for easy access.
Dim reader As New StreamReader(dataStream)
' Read the content.
Dim responseFromServer As String = reader.ReadToEnd()
' Display the content to the Console.
Console.WriteLine(responseFromServer)
' Cleanup the streams and the response.
reader.Close()
dataStream.Close()
response.Close()
' Wait for input from the user
Console.ReadLine()
Using RESTful calls with Javascript
As Zach specified in the PDC talk, the real power in Dallas is that it can be consumed by a number of different languages, in this example Javascript will be used with an XMLHTTP request.
var url = "https://api.sqlazureservices.com/UnService.svc/FAO/3510?$format=atom10";
var accountKey = "";
var uniqueUserId = "";
var xmlRequest = new ActiveXObject("Msxml2.XMLHTTP");
xmlRequest.open("GET", url);
xmlRequest.setRequestHeader('$accountKey', accountKey);
xmlRequest.setRequestHeader('$uniqueUserID', uniqueUserId);
xmlRequest.onreadystatechange = function() {
if (xmlRequest.readyState == 4)
{
var response = xmlRequest.responseXML;
// retrieve any values here and format them
var values = response.getElementsByTagName('d:value');
Document.Write(values);
}
}
xmlRequest.send(null);
The main point here is that the pattern in the code is the same across languages, we setup the URL and credentials, setup a HTTP/Web request, add the accountKey and uniqueUserId to the request headers, set up a listener for the response and then do the request.