Parallel Extensions to the .NET FX CTP


The shift to multi- and many-core processors that is currently underway presents an exciting opportunity for everyone in the software industry.  With an expected increase of 10 to 100 times today’s compute processing power, the opportunities to deliver powerful and immersive new user experiences and business value are just awesome.


 


Today we released an early preview of the Parallel Extensions to the .NET Framework technology, available for download on MSDN.  This release contains new APIs to make programming on the .NET Framework simpler as well as supporting documentation and samples. 


 


Parallel Extensions runs on .NET FX 3.5, and relies on features available in C# 3.0 and VB 9.0 and includes:



  • Imperative data and task parallelism APIs, including parallel for and foreach loops, to make the transition from sequential to parallel programs simpler.

  • Declarative data parallelism in the form of a data parallel implementation of LINQ-to-Objects.  This allows you to run LINQ queries on multiple processors.

  • First class tasks that can be used to schedule, wait on, and cancel parallel work.

  • New concurrency runtime used across the library to enable lightweight tasks and effectively map and balance the concurrency expressed in code to available concurrent resources on the execution platform.

  • Several great examples of how to use parallelism in real world problems to obtain impressive speedups, including a raytracer, Sudoku puzzle generator, and other simple puzzle solvers and smaller samples.


Along with the Parallel Extensions release, we have also launched a new MSDN dev center dedicated to concurrent programming.  It has a collection of whitepapers (including a whitepaper that describes the broader vision for parallel computing at Microsoft) and will be used to announce and house future developments in the parallel computing space.


 


Our work on concurrent programming is another great example of close, ongoing collaboration between product teams and Microsoft Research.  Although we understand the shift to parallel computing is a gradual road ahead for our whole industry, we are excited by the prospect and believe that the Parallel Extensions library is a large step in the right direction. 


 


Looking forward to hearing your feedback as you start getting your feet wet in this space.


 


Namaste!


 


 

Comments (62)

  1. twoelfer says:

    Soma,

    the dev center link is dead.

    WM_FYI

    -thomas woelfer

  2. twoelfer says:

    the download link also.

  3. Cory Nelson says:

    links do not work

  4. Kay Lee says:

    I just started foaming at the mouth to work with PLINQ and the link is dead as the other commented.  I feel as sad as a 10 year old on christmas who opened a present thinking it was an XBox360.. to find underwear.

  5. Fabrice says:

    Looks like someone forgot to turn a switch on, or this post has been sent too son…

  6. You’ve been kicked (a good thing) – Trackback from DotNetKicks.com

  7. Somasegar says:

    Sorry for the confusion, the links are now live.

  8. Noah Coad says:

    The Parallel Computing Platform team has launched a new Parallel Computing Dev Center along with our

  9. The Parallel Computing Platform team has launched a new Parallel Computing Dev Center along with our

  10. Veeral Patel says:

    Good to finally see the Parrallel FX library out, cant wait to get to work to try this out. I have been waiting for a while for this to come out.

    Would you know if Microsoft planning to release a full production version of the Microsoft Research Accelerator framework which offers true parallel processing capabilities on the x86 architecture?

  11. joedu says:

    Veeral: we know that there are many benefits to using vector instructions to execute certain kinds of data parallel operations, like floating point arithmetic.  Accelerator is one exciting example out of MSR that shows that you can use GPGPU capabilities to acheive general purpose vectorization, and, as you point out, x86 offers SSE which can do similar things.  There has been a surge in research in this area — in fact, the winner of Jim Gray’s pennysort contest last year was a GPU-based sort algorithm.

    With that said, we’re not committing to adding any such support to ParallelFX at this point, though we’re certainly aware of the technologies and possibilities.  Thanks for the question, and if you have additional feedback, please let us know.

    Thanks,

    —joe duffy

  12. Dataland says:

    .NET FX CTP could be really useful!  Hopefully, this won’t automagically turn piggy code into code that is 10 or 100 times more piggy.  I really hope you guys consider incorporating some sort of higher level heuristics into the IDE (or runtime), where something like auto-profiling is built-in and developers can keep a handle on slow code that now uses more cores (IMHO if you write a Piggy Code Enhancer, this severely defeats some of multi-core and many-core main benefits).

  13. pirklk says:

    I cant find the link for the "new MSDN dev center dedicated to concurrent programming." (Parallel Computing Dev Center)

  14. Parallel Extensions to the .NET FX CTP(Somasegar’s WebLog)より 前から少し話に出ていた PLINQ(Parallel LINQ) の初の CTP が公開になっていますね。

  15. Somasegar’s WebLog : Parallel Extensions to the .NET FX CTP Joe Duffy’s Weblog – Parallel Extensions

  16. Somasegar's WebLog : Parallel Extensions to the .NET FX CTP Joe Duffy's Weblog – Parallel Extensions

  17. This is just awesome, I’ve been waiting to play with these APIs :)

  18. Time for another weekly round-up of developer news that focuses on .NET, agile and general development

  19. Ananth B. says:

    Hi Somasegar (and Veeral, Joe). I was really excited about Accelerator too, but kinda disappointed with how it actually worked, I thought the solution was too clunky. In any case, I have an open source effort for a LINQ-to-GPU going on here

    http://brahma.ananthonline.net. You may find this approach interesting.

    -Ananth

  20. Veeral Patel says:

    @Ananth,

    I had a quick peek at your site, you have taken an interesting approach with LINQ. I will download your solution over the weekend to tinker with. I admit accelerator is a bit rough but the idea is there which is very important.

    @Joe Is MS going to develop any good debugging tools for multicore programming similar to what is available on the MAC OS X with its default IDE shipping with the OS. From memory I maybe incorrect here but I do recall their debugger allowed you to trace processes running on individual cores etc.

  21. joedu says:

    Hi Veeral,

    Parallel programming tool support is certainly an area that you’ll see improvements in over time.  Debugging parallel programs poses some unique challenges around things like reactive debugging (e.g. exception handling), correctness (e.g. race conditions), and liveness (e.g. deadlocks).  Profiling parallel performance is also substantially different, since the way work is divided up has as much to do with your results as the number of CPU cycles used.

    Visual Studio 2008 already offers great support for see threads, freezing and resuming individual ones, and most of the infrastructure (e.g. thread pools, CLR locks) support rich debugging information.  But this is just the tip of the iceberg of what’s possible.  We’d love to hear your thoughts on what would be useful: https://connect.microsoft.com/site/sitehome.aspx?SiteID=516.

    Cheers,

    —joe

  22. Nice.. one of the features that i can’t wait to use..

    Regards

  23. MegP_MS says:

    pirklk – The devcenter link is now linked to in the post above.

  24. The CLR Team has been working with the Parallel Computing Platform Team for the past year on some innovative

  25. The CLR Team has been working with the Parallel Computing Platform Team for the past year on some innovative

  26. I have a question. Does ParallelFX offer a "task stealing" mechanism such as was implemented in CILK and is implemented in Threading Building Blocks? I didn’t see anything about this in my initial perusal of the documentation.

    If ParallelFX doesn’t currently have a task stealing mechanism, is it planned to introduce this later on? Without such a mechanism, it would seem like cases where particular tasks take much longer than other tasks (for example, problems where the computation ends when a limit condition is reached) will leave processor cores unnecessarily idle, reducing scalability.

    If ParallelFX does currently offer task stealing, can you point us to any documentation on how it works?

  27. Introduction November 29: Soma has announced a CTP release of the Parallel Extensions to the .NET framework

  28. Microsoft Parallel Extensions to .NET Framework 3.5, December 2007 Community Technology Previewが公開されています。

  29. MSDN magazine October 2007 had a nice little gem tucked away in an article about Parallel LINQ [1] (emphasis

  30. MSDN magazine October 2007 had a nice little gem tucked away in an article about Parallel LINQ [1] (emphasis

  31. MSDN magazine October 2007 had a nice little gem tucked away in an article about Parallel LINQ [1] (emphasis

  32. MSDN magazine October 2007 had a nice little gem tucked away in an article about Parallel LINQ [1] (emphasis

  33. Collected from Mahesh Kumar http://cyberiafreak.spaces.live.com/ I. 20 Ways to Come Up With Amazing Ideas

  34. Keith Patrick says:

    Anyone have any luck viewing content in the .chm on the download page? I’ve got the index, but every page shows "Page cannot be displayed" errors

  35. joedu says:

    Hi Kevin,

    Yes, the scheduler that underlies the whole library does in fact use dynamic work stealing, a la MIT CILK, to ensure good load balance and scalable queue management.  Please see this article on MSDN for a bit more detail on this: http://msdn.microsoft.com/msdnmag/issues/07/10/Futures/.

    Thanks,

    —joe duffy

  36. joedu says:

    Keith,

    You have to first save the CHM file to disk, and then open it from there.  This seems to be a limitation with CHM files (or at least the way we have generated ours).

    Hope that works for you,

    —joe duffy

  37. Keith Patrick says:

    Joe, that’s how I’m trying to read it. No content even when I have it saved to my desktop.  Maybe it’s that it’s using an ancient-looking help viewer? HHA.dll is 4.74 & the others are 5.02, but I didn’t catch anything in the requirements about a particular version of the help viewer

  38. essey says:

    Keith,

    Thank you for reporting this CHM issue.  

    There are occassionally problems with copying a CHM to Windows Vista due to the heightened security measures regarding Web pages.  There’s a step in that process that was possible to compromise, so Vista takes the safest approach for you, which is to not render any content.  To fix this, we will wrap it in a .zip file and repost.

    For the time being, you can download and install the MSI to get the exact same CHM without the security issue.

    Thanks,

    Ed Essey – MSFT

  39. Microsoft has released an early preview of ParallelFX (Parallel Extensions to the .Net Framework) , a

  40. Keith Patrick says:

    Ed,

    I’m seeing this on an XP machine, not Vista.  Also, I’m on a corporate workstation that disallows installations (I have yet to run an installer from MS that does not require admin rights). Makes software development quite a pain, I know, but the days of the "XCopy Install" never quite made it, and the reality is that lots of companies turn off admin access to its employees.

  41. No me cansaré de repetirlo, esta gente de Microsoft no para. Ahora le llega el turno a Microsoft Parallel

  42. Keith Patrick says:

    Just on a whim, I tried downloading it again this morning, and all the content is coming through fine now!

  43. Z chwilą kiedy zostałem zaproszony na konferencję Intela na temat procesorów wielordzeniowych, śledzę

  44. Z chwilą kiedy zostałem zaproszony na konferencję Intela na temat procesorów wielordzeniowych, śledzę

  45. Adam Durling says:

    For those of you having problems reading the help file – try clicking ‘Unblock’ on the general tab on the file’s property pages.  This worked for me.

  46. Nedávno se objevilo CTP paralelních extenzí pro .NET Framework 3.5 ("oficiální" oznámení např. zde ),

  47. A couple of weeks ago saw the release of the CTP of the Parallel Extensions to the .NET Framework ( download

  48. A couple of weeks ago saw the release of the CTP of the Parallel Extensions to the .NET Framework ( download

  49. I’d prefer reading in my native language, because my knowledge of your languange is no so well. But it was interesting! Look for some my links:

  50. Don Syme’s WebLog on F# and Other Research Projects has this post: A couple of weeks ago saw the release

  51. Don Syme's WebLog on F# and Other Research Projects has this post: A couple of weeks ago saw the

  52. Using Parallel Extensions from F# A couple of weeks ago saw the release of the CTP of the Parallel Extensions…

  53. I’d prefer reading in my native language, because my knowledge of your languange is no so well. But it was interesting! Look for some my links:

  54. Redwerb says:

    Multi-threading just got a little bit easier

  55. 2007 was another great year for software. Here are a few software releases that caught my eye and imagination

  56. Redwerb says:

    2007 was another great year for software. Here are a few software releases that caught my eye and imagination

  57. Para(lle?)lised says:

    I am not sure if this is a good place to ask this question (where is the good place?), but here it is.

    I am trying to translate some of the procedures I wrote to parallel.

    Let’s assume I have following procedure to implement.

    I have a list of strings which need to be searched using regular expressions. The procedure returns after finding first matching string.

    If I use Parallel.For method to implement the procedure, it seems there can be some problems.

    List<string> SList = new List<string>();

    Slist.add("aaa bbb ccc"); //many times

    Parallel.For(0, SList.Count, i =>

               {

     bool found=SearchStringUsingRegularExpsessions(SList[i]);

    if (found) break; //obviously, it will not work

               });

    If this Parallel.For does not preserve order of i parameter (it does not, right? And I mean the order of calling the delegate with i parameter, not the order in which the delegate finishes calculating), then it looks like all the strings may be searched before the match is found (worst case).

    Also I don’t know (an elegant way) how to mimic "break" instruction, when there is a match and the procedure should break (and return if no matches were found previously in other threads – this also needs some kind of synchronisation).

    There is also this method

    (from help file)

    public static void ForEach<TSource, TLocal>(

    IEnumerable<TSource> source,

    Func<TLocal> threadLocalSelector,

    Action<TSource, int, ParallelState<TLocal>> body

    )

    The body delegate is provided with the item from *source being enumerated* as well as with the index of the item as it was taken from the enumerable and a ParallelState<(Of <(TLocal>)>) instance which may be used to *stop the loop* before all iterations have completed. It may also be used to store per-thread data to avoid synchronization overhead in the loop body.

    It seems that this method adresses these problems (marked with **) and they can be solved without resorting to complicated synchronisation methods.

    So, does anybody have an idea how to deal elegantly with this all, or better yet, could anybody provide an example of using ForEach?

    (yes, I googled, to no effect, and examples do not help too).

  58. joedu says:

    Hi Para(lle?)lised,

    I’m happy to respond to questions here, but our team’s blog is http://blogs.msdn.com/pfxteam/ and we have forums on MSDN at http://msdn.microsoft.com/concurrency/.  If you post in the forum, someone in the community might be able to respond more quickly.

    Now, to answer your question… there are a few things to consider:

    – Yes Parallel.For and ForEach reserve the right to execute all iterations in any order, including all at once (given enough processors).  So you can’t make any assumptions as you would in a sequential for loop regarding order of execution.

    – The Parallel.For and ForEach methods offer overloads that give you a ParallelState object.  This object can be used to stop the loop, much like you would with a break statement in sequential loops.  To do so, just call Stop method on it.  This is a "best effort" to prevent subsequent iterations from running, and the call to For or ForEach will not return until they are guaranteed to be done.

    – The .NET Regex class, while thread-safe, is less scalable than we’d like.  (We’re working on that.)  In the meantime, you can use the overload of For or ForEach that allows a thread-local state object to be created.  Just create a single Regex per thread.

    With all of this in mind, here’s a little snippet that should do what you’re looking for:

    List<string> slist = new List<string>(…);

    int indexOfMatch = -1;

    Parallel.For<Regex>(

       0,

       slist.Count,

       () => new Regex(…),

       delegate(int i, ParallelState<Regex> ps) {

           if (ps.ThreadLocalState.IsMatch(slist[i])) {

               indexOfMatch = i;

               ps.Stop();

           }

       }

    );

    if (indexOfMatch != -1) { … }

    Notice that the body of the loop stores directly to indexOfMatch.  Technically this is a race condition.  But given that you’re simply trying to find one possible match, you’re guaranteed that all writes represent a valid match.  So you shouldn’t care if it’s written to multiple times.

    I hope this helps.  If not, please don’t hesitate to ask for clarifications.

    Thanks,

    — joe duffy (dev lead, parallel extensions)

  59. Para(lle?)lised says:

    Ok, the code compiles but this is not exactly what I had in mind.

    It’s because of one condition I forgot to mention.

    It is possible to consider "break" as "break when you find match in any of items in the list", however I consider "break" here rather as "break when you find the first occurence from the top of the list".

    And this is quite common problem when searching for example in a text editor – you have to find the first occurence, not any occurence, because you’d have to keep track of all found occurences in order not to find them again. And the user would go crazy anyway. People need it to be done sequentially.

    This implies a few interesting things (that is how I think it should be – I think it is not possible to do it with current parallel extensions library so I guess it’s a feature request… but maybe it’s too naive).

    We’re trying to find the first occurence in a list of strings (from the top of the list). Indexing varialble changes from 1 to 10.

    1. We need to execute all iterations in sequential order. It is allowed to execute them all at once (or almost), as long as there are no skips, ie. (for quad core processor):

    a) Run iteration 1,2,3,4

    b) If iteration 1 is finished (or goes into IO wait) you can run only iteration 5, then 6, then 7, but you can’t run iteration 7 right after iteration 4.

    c) Ideally iterations 1,2,3,4;   2,3,4,5;   3,4,5,6; and so on run together.

    2. If iteration 1 goes into IO wait, and iteration 5 is immediately started, if iteration 1 gets out of IO wait, then iteration 5 should be put to sleep, and the iteration 1 should start running. This ensures the fastest possible execution of all iterations starting from the top of the list (however switching between iterations may be time-consuming).

    It also means that iteration 1 has highest priority, iteration 2 a little smaller and so on.

    3. Let’s assume there are matches in iteration 4 and 5, and iteration 5 is finished before iteration 4. So in the code above it is possible that iteration 5 will provide a match. But this is not what we want – we want a match returned from iteration 4.

    So how should it be done?

    Iteration 5, after finding the match, should call function StopAllIterationsAfter(5). This would cause all iterations with higher numbers (6, 7, …) to stop, however iterations 4 could still run.

    Then iteration 5 should wait until all iterations with higher number end and it should assign the match to the final value.

    Iteration 4 would do exactly the same – wait for iteration 5 to end and assign the final value. And this would be the final value that we really want. Thus we also avoid race condition you mentioned.

    ————

    So we need the following features (can theese be  implemented in the parallel extensions?):

    – execution of iterations without skips, trying to execute first iterations from the top of the list.

    – stopping iteratons with a given number

    – waiting for any iteraton to end

    It would be best if there was an easier way to ensure correct behaviour, like using some flags in Parallel.For.

    The rest of this functionality can be implemented using simple locking.

    So how about that? Or maybe there is another way to do it? Looks like tasks have some of the necessary functions, but not all. But I’d rather do it with Parallel.For. It’s fun.

  60. This is the first in a series of posts about the improvements we are making to the CLR thread pool for

  61. Hunain Durrani says:

    Hi Somasegar,

    The download link is being removed, can you help me with new link i will be great full.

    Thanks

  62. phuff says:

    Hunain, Parallel Extensions are now part of the .NET Framework in .NET 4.  You can download Beta 2 of .NET 4 from http://msdn.microsoft.com/en-us/vstudio/dd582936.aspx.

    Polita Paulus

    Developer Division