Microsoft Live Labs Pivot and Stack Overflow Data via API

  • Article

Recently, I've been playing around with Microsoft Live Labs Pivot. Pivot is a tool from Microsoft Live Labs that allows you to visualize massive amounts of data. Through its interface, you can ask questions of the data to gain new insights.

Here's a quick overflow of what Pivot is and how it works:

Introducing the Stack Exchange API

I'm a fan of Stack Overflow because it provides a great way for developers to get answers to the questions that they have about their use of algorithms, code, and technology. Recently, Jeff Atwood announced the public beta of the Stack Exchange API. You can use this API to query a number of popular Stack Exchange sites like stackoverflow.com and serverfault.com. If you go to stackapps.com, you'll find a site where you can browse against the API, ask questions, and get inspired to write applications that target this data. And, if that's not enough, you could win some fabulous prizes as part of the Stack Exchange API Contest:

The entire Stack Overflow, Inc team will ultimately decide the winners based on order of awesomeness. And lest you think we don’t know awesome when we see it, we built Stack Overflow. I’m just sayin'. (But seriously, please understand that our decisions will be based on a variety of factors, some of which may be entirely subjective.)

Microsoft Live Labs Pivot and Stack Overflow, Together at Last!

Armed with this data, I wrote a console application in C# that generated a Pivot collection (i.e. so.cxml) for the top 500 users on Stack Overflow based on reputation:

The solution took me about an hour to write. Of course, the StackOverlow.Net project (hosted on CodePlex) made my life a little easier by providing a mechanism to quickly interface with the Stack Exchange API via a class library.

How Do I Get Started with Microsoft Live Labs Pivot?

First, go download and install Microsoft Live Labs Pivot. Take a look and check out some of the collections that are provided to get a better sense of how it works. Second, check out the developer info for Pivot. Recently, the team has published two tools (one for the command line and one for Excel) that make it simple to create your own collections and view them in Pivot. Third, build a custom collection and see if you can gain better insights from the data you're interested in.