Azure HDInsight Job Logging

We’ve made a nice fix to the Templeton job submission service that runs on the HDInsight clusters for remote job submission.  We’ve talked with a number of customers who want to be able to get access to the logs for the jobs remotely as well.  This typically requires access directly to the cluster.  We’ve updated…


Microsoft HDInsight Installation & Dependency Management

It’s a rainy Saturday afternoon here in Seattle, and the kids are keeping themselves busy running around the Christmas tree, so I’ve got a little time to put together a post that addresses some questions that have come up a few times in the forums as well as in our internal discussion aliases for our…


Updating HDInsight Preview

Today we’ve shipped an update to the single node HDInsight Server Preview that is installable via Web Platform Installer.   What Are We Doing? Every two weeks, we’re going to take a snapshot of the work in progress and update the public installer with this (provided it passes a basic level of validation).  This gives us…


Getting Up and Running with Piggybank on HDInsight

It’s been a fun couple of weeks launching HDInsight, and I’m going to be getting back into doing some more technical blogging.  There are a few easy topics off the bat that we’ve heard requested from customers.  The first one involves Piggybank, which is a user contributed collection of useful Pig user defined functions (UDF’s)….


Every Day is a Good Day When You Paint

Slightly off-topic, but a friend of mine posted a video on Facebook that struck me as being very relevant to our space.    At the same time I saw this, another friend and co-worker was having a fairly crummy day with lots of crazy meetings and requests from co-workers, but that day was made a…


Writing a Map/Reduce Job for Hadoop using Windows PowerShell

I had a little bit of time on my hand and wanted to whip up a quick sample using PowerShell for a Hadoop job. This uses the Hadoop streaming capability, which essentially allows for mappers and reducers to be written as arbitrary executables that operate on standard input and output. The .ps1 scripts are pretty…


Talking About Hadoop on Windows

A few folks have asked, so I decided to put the data in one place.  We’ll be talking more about Hadoop at TechEd North America and Hadoop Summit next week, and then later in the month at TechEd Europe.   Here are the sessions we’re presenting: TechEd (NA and Europe, links are to NA sessions)…


Tap, tap, tap, is this thing on?

I’ll be back to bloging here in a bit.  Since you last visited, I’ve had a fun journey working further on Workflow, Azure and now I’ve landed in the SQL team working on Hadoop, in particular, the developer story. Not a lot to say right now, but make sure to check out !


Thoughts on Flowchart

Last night I saw that Maurice had a few tweets that caught my attention about flowchart, but this is the one that I want to talk about: Come to think of it I also really mis a parallel execution in a flowchart. But other than that flowcharts rock! #wf4 #dmabout 13    hours ago via web…


Improving ModelItem with Dynamic

Hi all, I’m back, and apologize for the delay, things have been crazy here and I’ve taken a little time off following shipping VS2010, so I’m getting back into it.  I’d like to talk about one of my favorite late additions to the WF4 designer programming model.  We added this change sometime in the RC/RTM…