Implementing a MapReduce Join with Hadoop and the .Net Framework

I have often been asked how does one implement a Join whilst writing MapReduce code. As such, I thought it would be useful to add an additional sample demonstrating how this is achieved. There are multiple mechanisms one can employ to perform a Join operation, and the one to be discussed will be a Reduce…

1

Framework for .Net Hadoop MapReduce Job Submission V1.0 Release

It has been a few months since I have made a change to the “Generics based Framework for .Net Hadoop MapReduce Job Submission” code. However I was going to put together a sample for a Reduce side join and came across a issue around the usage of partitioners. As such I decided to add support…

0

Hadoop Streaming and F# MapReduce

And now for something completely different. As you may know Microsoft has recently announced plans for a Hadoop adoption for both Windows Server and Windows Azure. You can find out more about Hadoop and Windows Azure at Apache Hadoop-based Services for Windows Azure and Availability of Community Technology Preview (CTP) of Hadoop based Service on…

3

F# Parallel Array Sorting

In previous posts I have presented code to perform Parallel Sorting of arrays using 2 different methods: Merge sort using Barrier: http://blogs.msdn.com/b/carlnol/archive/2011/07/17/f-array-parallel-sort-functions-demonstrating-a-merge-sort-using-barrier.aspx Quicksort: http://blogs.msdn.com/b/carlnol/archive/2011/07/17/f-an-array-parallel-quicksort-implementation.aspx As the code is spread out over these post I thought it would be useful to wrap up the code into a single solution for downloading: http://code.msdn.microsoft.com/FSharp-Parallel-Array-Sort-0833cf30 Although the two implementations…

0

F#: An Array.Parallel Quicksort Implementation

As I mentioned in my previous post, Array.Parallel sort functions demonstrating a Merge Sort using Barrier, I wanted to continue the parallel sort examples with a Quicksort using the Task Parallel Libraries. F#, as do all functional languages, lend themselves easily to Quicksort implementations. There are many variations of an F# Quicksort, here are a…

0

F# Array.Parallel sort functions demonstrating a Merge Sort using Barrier

If you follow the excellent Parallel Programing with .Net blog, you will have read a recent post by Emad Omara demonstrating a Parallel Merge Sort using Barrier. While there may be more efficient parallel sorting options, as this post notes, this is a good demonstration of the usage of a Barrier, and presents a reasonable…

0