Managed interface to Esent released


I just published the first release of the Esent .NET managed interface on Codeplex.


http://www.codeplex.com/ManagedEsent


This is an interop layer can be used to write managed applications that use esent. There is a straightforward translation of the unmanaged API and some helper methods/objects built on top of those methods. I want to develop a comprehensive object model on top of the interop API (which is still very ‘C’-style), but that is a long-term project. With the interop layer a primary concern was that this code should be high-quality so it has tracing, good error checking and a comprehensive set of tests that I developed along with the code. The test project currently has ~250 small tests, and there are a few scripts to check the output of the test/utility programs.


I consider the code beta quality right now and plan to push it to ‘stable’ in a few weeks.


 


Comments (7)

  1. laurionb says:

    Internally we use a C++/CLI layer to access Esent. This has two major drawbacks (probably the same ones you found):

    1. The compiled code is specific to a 32-bit/64-bit architecture.

    2. There is no way to compile C++/CLI and C# into the same DLL, so we ended up with an extra binary.

    Windows 2000 support is attractive, but you have to deal with the instance problem. The W2K version of esent.dll only supports one instance per process and the JetCreateInstance API isn’t present in the DLL. That presents quite a severe restriction on client applications.

    In general I would be interested to see how you deal with API availability. The unmanaged code just uses conditional compilation with a #define in esent.h to allow client applications to select the level of functionality that they want. I had though of annotating classes/methods to indicate availability, but that wouldn’t generate an error until runtime.

    The dispose chaining problem is painful. The idea of having the instance clean up the sessions (and the sessions clean up the tables, etc.) sounds like a good approach. Watch out for multi-threading issues when keeping a per-instance list of sessions though.

  2. CoqBlog says:

    Vous avez probablement déjà vu, au cours de vos recherches dans Process Explorer par exemple, que certains

  3. Stephen Cleary says:

    The C++/CLI layer also requires the C++ RTL redistributable, which is quite large. (BTW, I believe there is a way to compile C++/CLI and C# into the same DLL – you just have to do it via the command line because VS doesn’t support it).

    For Windows 2000 support, I currently have W2K-compatible clients do "using (Instance instance = Instance.Global())", and XP+-compatible clients do "using (Instance instance = new Instance("name", "displayName"))". The Instance class does (thread-safe) checking and will throw an exception if both are done in the same process. Instance’s methods then check the handle so they know which P/Invoke method to call (when dealing with the *Instance APIs).

    API availability:

    I’d love for an easy way to do this at compile time, but it’s just not easy. So, I’m doing it at runtime. I do have an "OSAttribute" defined, so that the OS checking can be at least somewhat automated. As you said, it is still a runtime check, though. I’m being careful to explicitly note all platform variances in the XML comments, but I’ll also throw NotImplemented if they are used anyway. (Of course, this only applies if there isn’t a graceful fallback).

    I’m being careful with multi-threading issues on the instance’s list of sessions. Actually, right now I’m dealing with multi-threading issues on the instance’s list of callbacks. 🙂 (I’m keeping a GCHandle for each registered callback).

    Yes, I decided to support callbacks. Ambitious, I suppose. 🙂

    I’ve done a lot of multi-threaded work in the past, and a fair amount of complex C# interop. This project has been a lot of fun – actually challenging!

  4. laurionb says:

    Stephen,

    If you do find a way to compile C# and C++ into the same DLL, even by the commandline please do let me know.

    Callbacks are very ambitious. We have used them with our internal C++ interop layer and they can be tricky. Be careful not to let clients throw an exception ‘through’ esent because that will corrupt our internal state. If a managed callback throws an exception, you should catch it at the managed/unmanaged transition and return an error to esent. When esent returns to the managed code you can rethrow the exception (perhaps store it in thread-local storage?).

    I hope you will be able to make the code available when finished.

  5. laurionb says:

    Oops, I removed Stephen’s first comment when I was removing a spam link. Sorry!

  6. Stephen Cleary says:

    C# and C++ can be combined; you have to compile the C++ to a netmodule, then pass /addmodule to the C# compiler when compiling the C# netmodule, and finally link them together. I did it as a proof of concept back in the day, but ended up not going that direction due to the platform binding (x86/x64/IA64) and redistribution problems.

    MSDN: http://msdn.microsoft.com/en-us/library/168k2ah5.aspx

    Mike Stall comments on this blog entry that MDbg uses this to compile IL and C# code together: http://blogs.msdn.com/junfeng/archive/2005/02/12/371683.aspx

    Regarding the "exception problem" with callbacks: this is something I’m aware of. (I have a long history of writing OO wrappers for C APIs). Currently, I’ve got two kinds of exceptions defined, EseException for any exception from the managed API, and JetException for unexpected errors from the unmanaged API (JetException inherits from EseException). So, I have a callback wrapper that catches any JetException and extracts the error code and returns that instead; any other exception would result in returning ErrorCode.CallbackFailed (JET_errCallbackFailed).

    I’m not planning to re-throw the exception. For one thing, not all callbacks are called from native code, so the behavior would be different for different callbacks.

    I’ve worked with callbacks for other APIs, and I find the best practice is to recommend "don’t explicitly throw" during callbacks. Designs expecting exception propogation through callbacks are usually less than ideal.

    I do treat JetException specially. This is the type thrown by my P/Invoke layer if an unexpected error is returned, so by default, a callback invoking Jet APIs would propogate any errors.

    Oh, and yes, I hope to publish the code when it is complete.

    I was just about to post my callback code now (for the status callbacks), but on second thought it’s pretty long and this isn’t really the best place to display code. And the GCHandle register isn’t in place yet, either.

    Would you like me to post what I have? MSDN has a place for open source now. I would appreciate your comments as I go along.

  7. sambo99 says:

    Laurion,

    I just started a new project on github that will provide ActiveRecord on top ESE (and will allow you to swap in sqlite etc..) http://github.com/sambo99/simplestorageengine/tree/master its using your managed ESE wrapper.

    Thanks heaps.

    Sam