Here’s to the first release from MS Open Tech: Redis on Windows


The past few weeks have been very busy in our offices as we announced the creation of Microsoft Open Technologies, Inc. Now that the dust has settled it’s time for us to resume our regular cadence in releasing code, and we are happy to share with you the very first deliverable from our new company: a new and significant iteration of our work on Redis on Windows, the open-source, networked, in-memory, key-value data store.

The major improvements in this latest version involve the process of saving data on disk. Redis on Linux uses an OS feature called Fork/Copy On Write. This feature is not available on Windows, so we had to find a way to be able to mimic the same behavior without changing completely the save on disk process so as to avoid any future integration issues with the Redis code.

The version we released today implements the Copy On Write process at the application level: instead of relying on the OS we added code to Redis so that some data structures are duplicated in such a way that Redis can still serve requests from clients while saving data on disk (thus achieving the same effect of Fork/Copy On Write does automatically on Linux).

You can find the code for this new version on the new MS Open Tech repository in GitHub, which is currently the place to work on the Windows version of Redis as per guidance from Salvatore Sanfilippo, the original author of the project. We will also continue working with the community to create a solid Windows port.

We consider this not to be production ready code, but a solid code base to be shared with the community to solicit feedback: as such, while we pursue stabilization, we are keeping the older version as default/stable on the GitHub repository. To try out the new code, please go to the bksavecow branch.

In the next few weeks we plan to extensively test the code so that developers can use it for more serious testing. In the meantime, we will keep looking at the ‘save on disk’ process to find out if there are other opportunities to make the code perform even better. We will promote the bksavecow branch to master as soon as we (and you!) are confident the code is stable.

Please send your feedback, file suggestions and issues to our GitHub repository. We look forward to further iterations and to working with the Redis community at large to make the Windows experience even better.

Claudio Caldato

Principal Program Manager

Microsoft Open Technologies, Inc.

A subsidiary of Microsoft Corporation.

Comments (16)

  1. A .NET developer says:

    Will the new ReFS filesystem in Windows 8 have a copy-on-write feature?

  2. Alex says:

    One question: why?

  3. L. says:

    Since the Windows kernel *has* support for fork() with copy-on-write, you could also have taken the opportunity to expose it in Win32.

  4. noocyte says:

    I would encourage you to implement Redis as an Azure service, that would really rock!

  5. Claudio Caldato says:

    @ A .NET developer

    I don't know the answer to your question but here the problem is about copy-on-write of memory pages.

  6. Claudio Caldato says:

    Even if Windows Kernel could support a fork-like functionality, making it available through public APIs is not a trivial task.  

  7. @noocyte

    Already in our roadmap :), we need first to get the code through extensive testing to make sure we have a stable code base then we can start looking into what it takes to get Redis to run as a service on Azure.

  8. Just A OS Theorist says:

    @Claudio

    There are 'many' examples of ways to do the fork/copy on write in memory using public NT APIs.  

    Even back in 2000, there was a book by Gary (?) on Native NT APIs that engaged this conversation, and it has been beat to death with a ton of examples around the web in the past 12 years.

    If 'Windows' was NOT capable of doing this, the SUA would NOT have worked. Right?

    Maybe state that the Win32 APIs doesn't offer an equivalent, instead of saying Windows.

    Especially since people get confused about this stuff easily and it is misleading to limit the concept of Windows NT to just the Win32 subsystem.

    PS

    You do not need to publish these comments, just offering a couple of tips, and I do not want to portray your work as reductive or appear to be condescending, as it is not my intent.

    *IP hopping on purpose*

  9. Want to use redis on Windows says:

    Hi!

    If there any plans to implement local communication method, similar to unix sockets? E.g. named pipe.

    Thank you.

  10. @Want to use redis on Windows  

    It is an interesting idea, please add an issue to our GitHub project so we can consider this feature for future releases (github.com/…/redis)

  11. Shriram says:

    Can we use the VS 2008 or Pro instead of 2010 t Express and still attain a working version of this ?

  12. @Shriram

    we provide a VS solution for VS2010 and now we also release binaries. If you want to use VS2008 you may have to rebuild the solution but that should be a simple task.

  13. Matthias Götzke says:

    It seems that ms stopped working on this project. That's the reason I did not use it to begin with. A really nice idea, but ms is just not committed to stay with it. This happened a number of times lately and is a serious failure. When you  start projects like this and do not intend to maintain them, just leave it or at least write something as to why it was abandoned . Had you kept with it I would have used it in my next archive architecture, this way I am more inclined to continue on my path  of trying to get off of windows server . 🙁

  14. Tim Sanders says:

    This is another BS PR stunt from Microsoft?

  15. @Matthias Götzke

    I'm very sorry that we gave you the impression we are not working on the project anymore. Even though you don't see any apparent activity on the public repo it does not mean we are not working on the project.

    As I explained in one of the issues opened on GitHub, we decided to work on a private repo in order to avoid too much randomization. For people that wants to start testing Redis on Windows they can rely on a stable version of the code on the public repo. With the next release (coming very soon) we are adding 64bit support and we think the code should be in a state where we can switch to the public repo as our main development branch so it will be easier to track progress and see what is happening.

    I hope this will help you to reconsider your decision to get off Windows Server. And BTW: contributions are always welcome 🙂

  16. Matthias Götzke says:

    @claudio .. Well , its all about trust, so lets see 🙂 ms has been rather inconsistent before, maybe its culture can change though. I for one would appreciate it. I am using a lot of smartos now, but win server has some advantages still.