Apache Hadoop on Windows Azure Part 9 – Using Interactive JavaScript for Data Visualization
Apache Hadoop on Windows Azure is integrated with a web-based interactive JavaScript console, which allows you to perform
- Perform HDFS operations, including uploading/reading files to/from the HDFS
- Run MapReduce programs from .js scripts or JAR files, and monitor their progress
- Run a Pig job specified using a fluent query syntax in JavaScript, and monitor its progress
- Visualize data with graphs built using HTML5
Once you have built your Hadoop cluster on Windows Azure, you can launch Interactive JavaScript web console directly as below:
After the Interactive JavaScript is launched you can enter a set of commands or your own JavaScript directly from the browser:
For our example we will do the following:
- Upload a data file to HDFS
- Run JavaScript to parse the data
- Run Graph Visualizer with parsed data
Let’s collect some real time data about mobile market share for 2011:
https://www.netmarketshare.com/operating-system-market-share.aspx?qprid=9&qpcustomb=1
Create a txt file locally as mobilesharedata.txt locally:
iOS 52.10
JavaME 21.27
Android 16.29
Symbian 5.76
BlackBerry 3.51
Other 1.07
Upload this file to HDFS from Interactive JavaScript console:
js> fs.put()
File uploaded.
Verify the file is uploaded:
js> #ls
Found 2 items
drwxr-xr-x - avkash supergroup 0 2012-01-02 20:37 /user/avkash/.oink
-rw-r--r-- 3 avkash supergroup 81 2012-01-02 20:52 /user/avkash/mobilesharedata.txt
Let’s read the data this
js> file = fs.read("/user/avkash/mobilesharedata.txt")
iOS 52.10
JavaME 21.27
Android 16.29
Symbian 5.76
BlackBerry 3.51
Other 1.07
Now parse the data as below:
js> data = parse(file.data, "OS, MarketShare:long")
[
0: {
OS: "iOS"
MarketShare: 52
}
1: {
OS: "JavaME"
MarketShare: 21
}
2: {
OS: "Android"
MarketShare: 16
}
3: {
OS: "Symbian"
MarketShare: 5
}
4: {
OS: "BlackBerry"
MarketShare: 3
}
5: {
OS: "Other"
MarketShare: 1
}
]
Set the graph visualization settings as below:
js> options = { title: "Mobile OS Market Share in 2011", orientation: 20, x: "OS", y: "MarketShare" }
{
title: "Mobile OS Market Share in 2011"
orientation: 20
x: "OS"
y: "MarketShare"
}
Now let’s visualize the data in Bar Graph:
js> graph.bar(data, options)
Now let’s visualize the data in Pie Graph:
js> graph.pie(data, options)
Keywords: Windows Azure, Hadoop, Apache, BigData, Cloud, MapReduce