MS Open Tech Open Sources Rx (Reactive Extensions) – a Cure for Asynchronous Data Streams in Cloud Programming

From:
Erik Meijer, Architect, Microsoft Corp.
Claudio Caldato, Lead Program Manager, Microsoft Open Technologies, Inc.

Updated with a quote from Ferranti Computer Systems NV

Updated: added quotes from Netflix and BlueMountain Capital Management

If you are a developer that writes asynchronous code for composite applications in the cloud, you know what we are talking about, for everybody else Rx Extensions is a set of libraries that makes asynchronous programming a lot easier. As Dave Sexton describes it, “If asynchronous spaghetti code were a disease, Rx is the cure.”

Reactive Extensions (Rx) is a programming model that allows developers to glue together asynchronous data streams. This is particularly useful in cloud programming because helps create a common interface for writing applications that come from diverse data sources, e.g., stock quotes, Tweets, computer events, Web service requests.

Today, Microsoft Open Technologies, Inc., is open sourcing Rx. Its source code is now hosted on CodePlex to increase the community of developers seeking a more consistent interface to program against, and one that works across several development languages. The goal is to expand the number of frameworks and applications that use Rx in order to achieve better interoperability across devices and the cloud.

Rx was developed by Microsoft Corp. architect Erik Meijer and his team, and is currently used on products in various divisions at Microsoft. Microsoft decided to transfer the project to MS Open Tech in order to capitalize on MS Open Tech’s best practices with open development.

There are applications that you probably touch every day that are using Rx under the hood. A great example is GitHub for Windows.

According to Paul Betts at GitHub, "GitHub for Windows uses the Reactive Extensions for almost everything it does, including network requests, UI events, managing child processes (git.exe). Using Rx and ReactiveUI, we've written a fast, nearly 100% asynchronous, responsive application, while still having 100% deterministic, reliable unit tests. The desktop developers at GitHub loved Rx so much, that the Mac team created their own version of Rx and ReactiveUI, called ReactiveCocoa, and are now using it on the Mac to obtain similar benefits."

And Scott Weinstein with Lab49 adds, “Rx has proved to be a key technology in many of our projects. Providing a universal data access interface makes it possible to use the same LINQ compositional transforms over all data whether it’s UI based mouse movements, historical trade data, or streaming market data send over a web socket. And time based LINQ operators, with an abstracted notion of time make it quite easy to code and unit test complex logic.”

Netflix Senior Software Developer Jafar Husain explained why they like Rx. "Rx dramatically simplified our startup flow and introduced new opportunities for performance improvements. We were so impressed by its versatility and quality, we used it as the basis for our new data access platform. Today we're using both the Javascript and .NET versions of Rx in our clients and the technology is required learning for new members of the team."

And Howard Mansell, Quantitative Strategist with BlueMountain Capital Management added, “We are very pleased that Microsoft are Open-Sourcing the Reactive Extensions for .NET. This will allow users to better reason about performance and optimize their particular use cases, which is critical for performance and latency sensitive applications such as real-time financial analysis.”

From Belgium, Guido Van de Velde, Director  MECOMS Product Organisation for Ferranti Computer Systems NV, explains how Rx is important for their global company: “Ferranti uses Rx in its vertical solution for the utility market, MECOMS™, to process and manage all data and events from the Smart Grid. Its architecture allows the setup of data processing pipelines which can scale and deliver excellent performance. Performance testing together with Microsoft showed that this architectures supports up to hundreds of millions of smart meters and other sensors, running on  commodity hardware. Thanks to Rx we can focus on component functionalities and don’t have to worry about interfaces and connections between the different components saving significant development time.”

Part of the Rx development team will be on assignment with the MS Open Tech Hub engineering program to accelerate the open development of the Rx project and to collaborate with open source communities. Erik will continue to drive the strategic directions of the technology and leverage MS Open Tech Hub engineering resources to update and improve the Rx libraries. With the community contribution we want to see Rx be adopted by other platforms. Our goal is to build an open ecosystem of Rx-compliant libraries that will help developers tackle the complexity of asynchronous programming and improve interoperability.

We are also happy to see that our decision is welcome by open source developers.

“Open sourcing Rx just makes sense. My hope is that we’ll see a couple of virtuous side-effects of this decision. Most likely will be faster releases for bug fixes and performance improvements, but the ability to understand the inner workings of the Rx code should encourage the creation of additional tools and Rx providers to remote data sources,” said Lab 49’s Scott Weinstein.

According to Dave Sexton, http://davesexton.com/blog, “It’s a solid library built around core principles that hides much of the complexity of controlling and coordinating asynchrony within any kind of application. Opening it will help to lower the learning curve and increase the adoption rate of this amazing library, enabling developers to create complex asynchronous queries with relative ease and without any spaghetti code left over.”

Starting today, the following libraries are available on CodePlex:

  • Reactive Extensions
    • Rx.NET: The Reactive Extensions (Rx) is a library for composing asynchronous and event-based programs using observable sequences and LINQ-style query operators.
    • RxJS: The Reactive Extensions for JavaScript (RxJS) is a library for composing asynchronous and event-based programs using observable sequences and LINQ-style query operators in JavaScript which can target both the browser and Node.js.
    • Rx++: The Reactive Extensions for Native (RxC) is a library for composing asynchronous and event-based programs using observable sequences and LINQ-style query operators in both C and C++.
  • Interactive Extensions
    • Ix: The Interactive Extensions (Ix) is a .NET library which extends LINQ to Objects to provide many of the operators available in Rx but targeted for IEnumerable<T>.
    • IxJS: An implementation of LINQ to Objects and the Interactive Extensions (Ix) in JavaScript.
    • Ix++: An implantation of LINQ for Native Developers in C++.
  • Bindings
    • Tx: a set of code samples showing how to use LINQ to events, such as real-time standing queries and queries on past history from trace and log files, which targets ETW, Windows Event Logs and SQL Server Extended Events.
    • LINQ2Charts: an example for Rx bindings. Similar to existing APIs like LINQ to XML, it allows developers to use LINQ to create/change/update charts in an easy way. We would love to see more Rx bindings like this one.
    • With these libraries we are giving developers open access to both push-based and pull-based data via LINQ in Microsoft’s three fundamental programming paradigms (native, JScript and Managed code).

We look forward to seeing you guys use the library, share your thoughts and contribute to the evolution of this fantastic technology built for all you developers.