PLINQ is coming up soon (PFX)

Article
10/14/2007

I've been hearing about PLINQ (Parallel Linq) since the first days of announcing LINQ, the idea of making use of the new functional style programming provided in DotNet 3.5 in order to give better performance on Multi Core machines, the idea sounds cool since first day, and it now comes true in a new name Parallel FX or PFX

The programming model provided is quite simple and utilizes the same LINQ model, the new assembly is called System.Concurrency.dll which is the library that contains the new interface called IParallelEnumerable<T>, also it adds an extension methods for all collections and arrays that implement old IEnumerable, the extension method is called AsParrallel<T> which converts any collection to a Parallel enabled collection of type IParallelEnumerable<T>

Take a look of the following code

 IEnumerable<int> data = new int[] {1, 2, 3, 4, 5, 6}; // or whatever complex data collection

 var q = data.Where(x => x > 4).OrderBy(x=>x).Select(x => x);

 foreach (var i in q) ....

This code is what you already write in C# 3.0, now if you want to add PLINQ support all what you have to do is adding the AsParallel function call before using any query operation so the previous code will look like this

 IEnumerable<int> data = new int[] {1, 2, 3, 4, 5, 6}; // or whatever complex data collection

 var q = data.AsParallel().Where(x => x > 4).OrderBy(x=>x).Select(x => x);

 foreach (var i in q) ....

or you can write it in the LINQ query style

 IEnumerable<int> data = new int[] {1, 2, 3, 4, 5, 6}; // or whatever complex data collection

 var q = from i in data.AsParallel()

         where i > 4

         orderby i

         select i;

 foreach(var i in q) ....

Once you've added the AsParallel function call, PLINQ will be ready to execute transparently all the OrderBy, Where, Select, GroupBy ... etc on all the available processors.

You don't need to explicitly create threads, locks and manage concurrent execution (Unless you are making something big).

This doesn't mean that you can make use of the PLINQ power on anything other than Queries, the ParallelEnumerable class also adds some extra extension methods like the ForAll method

The ForAll method is useful if you are applying some kind of operation on all the members of a certain collection, so the ForAll function will do this operation in parallel for all the members of the collection

 IEnumerable<int> data = new int[] {1, 2, 3, 4, 5, 6}; // or whatever complex data collection

 data.ForAll(i=>Console.WriteLine(i));

The previous code sample will print out all the members of the array, if you imagined calling more complex function that does some heavy work on each array member, the ForAll will give you extra power to do the job faster by making use of the parallel data processing techniques.

This is not all the new stuff introduced in the System.Concurrency library, the new Parallel class is also a nice addition, it provides some extra general purpose parallel execution, so it is not related to LINQ, the most important part is the Parallel.For function, which as you expected from the name executes a parallel loop.

Check the following code

 void ParMatrixMult(int size, double[,] m1, double[,] m2, double[,] result)

   Parallel.For( 0, size, delegate(int i) {

     for (int j = 0; j < size; j++) {

       result[i, j] = 0;

       for (int k = 0; k < size; k++) {

         result[i, j] += m1[i, k] * m2[k, j];

});

This is an example I got from here, it illustrates a Matrix Multiplication using Parallel.For, as you can see the Parallel.For method accepts the start index and the length, then a delegate to execute.

There is also Parallel.Aggregate function which can be used to aggregate a certain data item over a parallel loop safely.

This all what I could write in one post, however the System.Concurrency contains more cool API's.

Here comes the resources for further readings.

PLINQ is coming up soon (PFX)

Additional resources