Container hierarchies in MEF: shaping it for the future (maybe)


Recapping: MEF supports two creation policies: shared (think
singleton) or non-shared (think transient, prototype or a fancy new operator).
MEF also recognizes types that implement IDisposable and correctly “implement
the protocol” for calling Dispose on them in the right time.

This is all goodness, but as applications grow in complexity,
this is hardly enough. You can imagine a MDI application that wants distinct
singleton instances for each document. That’s actually why we decided to use
the name Shared instead of Singleton. A component is shared within a context.
Another historical fact is that in the early days, MEF’s container used to be
called CompositionDomain. Thinking about, I believe it was the perfect name:

  • Each “domain” represents an isolated island of
    components and their wirings (composition)
  • Each “domain” holds ownership of components it
    instantiates
  • Each “domain” reclaim resources off what it owns
    once it is disposed
  • Optionally, a “domain” can be linked to a parent
    domain, so dependencies can be fulfilled by components owned by the parent

The last bullet leads to an important compromise: hierarchy
must be established in a way that the depth means shorter lifetime
. In other
words, the top level container must be the one that lives the longest, and the
bottom container the one that lives the shortest. If you change this, hell
breaks loose.

For some reason the class was renamed (before I joined the
team) to CompositionContainer, but the principles remain. They allow you to
support requirements such as the typical “a container per-web-request” or “multi-tenancy”
where each container represents customizations for each tenant, and finally MDI
apps where each document means a new container; to name just a few.

To surprise some of you, we have always supported containers
hierarchy in MEF. The API were always there:

 

var parent = new
CompositionContainer(catalog);

var child = new
CompositionContainer(parent);

 

Unfortunately, the above doesn’t accomplish much. The only
way for MEF to discover components is through catalogs. So in order to control
what each container will own you have to manage the catalogs accordingly, and specify
the right ones when creating containers:

 

var parent = new
CompositionContainer(globalCatalog);

var child = new
CompositionContainer(childCatalog,
parent);

 

But the vast majority of applications start with a single
catalog, say DirectoryCatalog. What to do in this case?

Aware of this, I published a FilteredCatalog sample code on
Codeplex. While it helps, it is still hard to get filtering right.

The problem in practice

For the next release we are considering several options to
free – or at least diminish – the pain for users of MEF who need container
hierarchies in order to get “scoping”. As most of the pain is based on how to
deal with catalogs, we are currently focused on those. Note that this is a hard
problem and we may reject all options and table this for yet another release.
:-\

Say you’re writing a MVC app using MEF. You also want
controllers and related artifacts to be isolated in a per-web-request basis. Your
starting point is a catalog that has everything plus the kitchen sink. You need
to select only the components that should be in the request level container.
How would you do it?

  1. Components (parts in the MEF lingo) have exports
    – remember the [Export]?
  2. Each controller is likely to export IController
  3. Nailed it!

 

var catalog = new
DirectoryCatalog(binFolder);

var perRequestCatalog =
catalog.Where( component -> component.Exports<IController>() );

 

This looks sufficient, right? The perRequestCatalog contains
all components that happen to export IController.

What to do next? So far you have two pieces of information
in the form of sets: the catalog, which is a DirectoryCatalog and contains
everything (including the controllers) and a perRequestCatalog which contains
all controllers.

Your gut reaction may be to write the following:

 

var catalog = new
DirectoryCatalog(binFolder);

var perRequestCatalog =
catalog.Where( component -> component.Exports<IController>() );

 

var parent = new
CompositionContainer(catalog);

var perRequestContainer =
new CompositionContainer(perRequestCatalog, parent);

 

Looks reasonable, right? Except that it’s wrong wrong wrong…
Now the parent container points to a catalog that has everything, and the
perRequestContainer has just the catalogs. If you ask the latter for all
controllers, guess what will you get? All controllers. Twice.

To fix that the parent container should point to a catalog
that has everything BUT the controllers. So this is a general rule for scoping
in MEF, you usually won’t want overlapping sets unless you know what you’re
doing.

Going back to code, imagine we could write the following:

 

var catalog = new
DirectoryCatalog(binFolder);

var catalogPair =
catalog.Where( component -> component.Exports<IController>() );

 

var perRequestCatalog =
catalogPair.Matching;

var appLevelCatalog =
catalogPair.Unmatching;


Now the .Where call returned two sets: the true and the
false. The false set happens to be the input set minus the true set, aka the
difference, aka the complement. With that it’s easy to get it right:

 

var parent = new
CompositionContainer(appLevelCatalog);

var perRequestContainer =
new CompositionContainer(perRequestCatalog, parent);


Now you will get the expected behavior, since the sets do
not overlap.

Shaping it

Then it comes to how to expose it. We’re exploring a number
of ways. First, let’s break it in three correlated APIs:

  1. Filter the catalog based on a predicate – method name?
  2. Hold the true set and the false set – type name?
  3. Expose the false set – property name?

For the first, we are considering: Catalog.Filter,
.Partition and .Where. Remember, there’s no clear winner, all options have
drawbacks. Filter does not convey that the result is two sets. Partition is
quite mathematical, and may not elude that it can also be used for pure
filtering scenarios. Where brings the mental model of Linq, but doesn’t return
IEnumerable<> so it may be abusing the mental model.

For the second, we’ve been debating over FilteredCatalog, PartitionedCatalog,
Tuple<Catalog,Catalog>,  and
PairedCatalog. Note that the result of the aforementioned method is itself a
catalog, but a catalog with “extra” information, the false set.

Finally, for the false set we have considered a property
like Complement, PartsNotFiltered, DisjointSet.

Hard! How would you design it?

 

Comments (6)

  1. rjperes says:

    The FilteredCatalog link is wrong!

  2. Marc Jacobi says:

    Isn't the problem the fact that the exports of both the parent and child catalogs are returned (IControler example)? I mean, why not allow a catalog to perform a function on whether or not to pass on the request to the parent catalog (skip-when-found, merge-results-no doubles, defer-to-parent etc)?

    In my view it is all you need. If I would be able to set the behavior of passing requests to parent catalogs/containers then I think you have a nice solution.

    I once wrote a service container framework that did the same thing and I was able to use in a composite application to great success (each module got its own container and parented an application wide container).

    Hope it helps.

  3. Chris says:

    What about an overload on DirectoryCatalog / Assembly catalog that lets you pass Predicate<PartDefinition> typeSelector and / or typeExcluder ??

    Maybe not part definition, but something that makes it fairly easy to query the part and see if you want it in the catalog.

    DirectoryCatalog app = new DirectoryCatalog("bin",typeSelector: x => {

     if(x.HasContract<IController>()) {

         return false;

     }

     if(x.Metadata.ContainsKey["Request"])) {

          return false;

      }

      return true;

    });

    DirectoryCatalog request = new DirectoryCatalog("bin",typeSelector: x => app.Parts.Contains(x) == false);

    Also,

    DirectoryCatalog all = new DirectoryCatalog("bin");

    ComposablePartCatalog[] catalogs = all.Split(x => x.Metadata.ContainsKey("request"));

    Pair or Tuple is probably equally as good as []

    Not sure what it would be like calling split on a shared container instance every web request though…

    >>The last bullet leads to an important compromise: hierarchy must be established in a way that the >>depth means shorter lifetime.

    If we make that assumption shouldn't we be able to assume that any parts in parent that are CreationPolicy.NonShared can resolve dependencies using the child container?

    That would /really/ help

  4. @rjperes, fixed

    @Marc, this is certainly an option, but seems more complex and with more hairy implications for edge-cases.

    @Chris, Split is interesting. will bring this up! thanks. WRT to NonShared being able to use child containers, actually not. You can have a Shared depending on a NonShared which would depend on component that will go away before the parent.

  5. Morten Mertner says:

    I'd second both the Predicate on the catalog constructor and the Split option.

    Maybe you could introduce a CompartmentCatalog as a companion to the AggregateCatalog – but instead of aggregating it separates based on some criteria:

    var all = new DirectoryCatalog( "." );

    var catalogs = new CompartmentCatalog( all, new Compartment( "controllers", <predicate to select> ), new Compartment( "plugins", <another predicate ), … );

    Naming the compartments might not be needed or beneficial, but does provide a mechanism for referencing each compartment. Also not sure if having multiple compartments is a useful addition.

  6. Andrey says:

    My few cents 🙂

    1. DirectoryCatalog(folder, predicate) means you have to go over all the types within each assembly in the folder every time an instance of DirectoryCatalog is instantiated? Not good. I believe the FilteredCatalogs are the way to go. Traverse the assemblies once and then expose the different views of the whole set.

    2. var catalogPair = catalog.Where( component -> component.Exports<IController>()) assumes I want to split the catalog into 2 parts, why? Not good. I personally want 3 parts/levels for my app: shell, document, view.