Implementing custom provider for WCF Data Services is complicated enough. The fact that for the really interesting providers it’s also necessary to implement a custom IQueryable makes things just so much more complicated. Based on feedback from multiple parties trying to implement the IQueryable for WCF Data Services one of the most challenging tasks is to support projections and expansions. These are hard not only because the expression trees generated for them are complicated but also because they require the query to return results in different shape than usual. In this post we’ll walk through a simple case of dealing with projection and expansions (with some limitations).
First let’s assume that you’re familiar with the expression trees generated by WCF Data Services for projections and expansions. These are described in the Data Services Expressions series part 8 and part 9. Also some familiarity with IQueryable, its behavior and the general approaches to its implementation is required. There are multiple places to learn about these, but the classic source is Matt’s blog series.
Since the solution we’re going to try out is simplistic it only solves certain limited scenarios. Projections ($select) are meant to limit the number of properties returned by a query for any given entity in it. This is definitely useful for the client since it can download only the properties it’s going to need. But the server can take advantage of this as well, since if the client only asks for subset of properties the server can also only load a subset of properties from the underlying data store. The simple solution described in this post assumes that the second optimization is not necessary, that it’s OK to load all properties for each entity from the data store. The projections then only happen inside the service and are easier to deal with.
If your provider wants to support expansions as well, these are even more complicated. This simple solution will only support expansions if the entity instances can load the target of any given navigation property on demand (when the navigation property is accessed). Some data sources might be able to do this easily, but for other it might be hard or expensive. I will discuss possible solutions to those in some later post.
The general idea of dealing with projections in our provider will be this:
- Take the entire query and split it in two parts, the projection and expansion itself and the source query for the projection and expansion which returns the results as plain entities.
- Then run the source query against the underlying data source (we will assume we already have a solution for this).
- Take the results of the source query and apply projections and expansions on it inside the service.
- Return the new results to WCF Data Services.
For the purposes of this sample we will use a LINQ to SQL based data source. So we create a new web application, add a database to it with two tables Products and Categories (similar schema as in the sample service on odata.org). Then we create a LINQ to SQL classes over these tables which will give us a SampleDataContext class with two properties Products and Categories.
Note: I chose LINQ to SQL since it can deal with all the other parts of the query (filters, sorting and so on). It also supports on-demand loading of navigation properties by just accessing them. And last but not least, it generates the proxy classes, so we can use reflection provider to setup the metadata for us. In custom provider implementations this would obviously not be the case and most of these would have to solve in our code, but for the purposes of this sample such an approach is enough.
Custom query provider
We need a custom query provider, that is IQueryable and IQueryProvider implementations. Implementation of IQueryable is pretty simple (as usual):
When a query is to be execute this only calls ExecuteQuery method on the query provider. Now let’s add a stock implementation of the query provider like this:
Our query provider is based on a query provider for the underlying data source. As noted above this assumes that we already have an IQueryable implementation which can deal with filters, sorting and alike.
And now to the real interesting part, how to implement the ExecuteQuery:
The ExecuteQuery implementation only determines if the query contains projections/expansions and if so it invokes the ProcessProjection method. That method takes the source of the projection and, executes it against the data source (in our case this would query IQueryable against the LINQ to SQL and run it). Then it takes the results and applies the projections/expansions in-memory on it.
The main trick used here is to take a lambda expression (System.Linq.Expressions.LambdaExpression) and call the Compile method on it. This generates a lightweight dynamic method which is the body of the .Select and can be called just like any other method. Then we just simply call this method on each result and return to WCF Data Services.
And now in our sample we simply wrap the IQueryable returned by LINQ to SQL with the one we implemented above. Like this:
And that’s it. Give it a try and run a query for example like service/Products?$select=Name. It should only return products with Name property. To verify that it really worked, set a breakpoint into the ProcessProjection method on the line which create dataSourceQueryResults and run the query again. The dataSourceQuery is the IQueryable which will execute against the data source. In this sample it should show in the debug window a simple table scan for table Products (this is provided by LINQ to SQL for us). If the query would contain filters or sorting this query would contain all of those and LINQ to SQL would generate the appropriate SQL for it.
If we would like to use this solution as part of a bigger custom provider the query provider would need to be able to handle all of our custom provider needs on top of this. There are also some technical limitations. In particular the solution requires that IDataServiceQueryProvider.NullPropagationRequire returns true, otherwise the function generated by lambda.Compile would not correctly handle null values. This solution is also not suitable for wrapping EF providers since the expression trees generated for EF provider are not fully compatible with lambda.Compile either. And last, if our custom provider uses untyped properties, those would have to be “resolved” in the lambda expression before the .Compile is called. Not to mention how to solve the expansions for real and don’t lazy load the navigation properties one by one. All this will have to wait for another post.
And here’s the full sample as a VS 2010 solution.