Oops, we did it again


A new update to LINQ preview is available at MSDN site.


What’s the coolest new feature?  IMHO, its IQueryable<T>. 


 DLINQ’s query mechanism has been generalized and available for all to use as part of System.Query.  It implements the Standard Query Operators for you using expression nodes to represent the query. Your queries can now be truly polymorphic, written over a common abstraction and translated into the target environment only when you need it to.


    public int CustomersInLondon(IQueryable<Customer> customers) {


        int count = (from c in customers


                     where c.City == London


                     select c).Count();


        return count;


    }


Now you can define a function like this and it can operate on either an in memory collection or a remote DLINQ collection (or you own IQueryable for that matter.)  The query is then either run entirely locally or remotely depending on the target. 


If its a DLINQ query a count query is sent to the database.


SELECT COUNT(*) AS [value]


FROM [Customers] AS [t0]


WHERE [t0].[City] = @p0


If its a normal CLR collection, the query is executed locally, using the System.Query.Sequence classes definitions of the standard query operators.  All you need to do is turn your IEnumerable<Customer> into IQueryable<Customer>.  This is accomplished easily with a built-in ToQueryable() method.


  List<Customer> customers = …;


  CustomersInLondon(customers.ToQueryable());


Wow!  That was easy.  But, how is this done?  How can you possible turn my List<T> into some queryable thingamabob?


Good question.  Glad you asked.


Check out this little gem: 


  Expression<Func<Customer,bool>> predicate = c => c.City == London;           


  Func<Customer,bool> d = predicate.Compile();


 


Now you can compile lambda expressions directly into IL at runtime!


ToQueryable() wraps your IEnumerable<T> in IQueryable<T> clothing, uses the Queryable infrastructure to let you build up your own expression tree queries, and then when you enumerate it, the expression is rebound to refer to your IEnumerable<T> directly, the operators rebound to refer to System.Query.Sequence, and the resulting code is compiled using the built-in expression compiler.  That code is then invoked producing your results.


Amazing, but true.


But wait, there is more!  Yes, it is true, not only can you slice, but you can also dice!


That’s not the end of IQueryable<T>’s little bag of tricks.  What else could there be? How about dynamic queries, ones you can build up at runtime? Yes? No?  Say it isn’t so!


It’s true.  IQueryable<T> is fully dynamic.  Sort of.  Each IQueryable<T> has a method CreateQuery<S>() that creates a new IQueryable<S> (of the same concrete type) with a single Expression tree argument.  That is, if I have an expression tree representing the code that defines a query, I can make an IQueryable<S> out of an old IQueryable<T>.


That’s great you say, but you probably don’t statically know what the ‘T’ and ‘S’ are in your program if you are building dynamic queries at runtime.  True, true, we’ve thought of that too. You see, IQueryable<T> has a smaller cousin, IQueryable, just like IEnumerable<T> has its little friend IEnumerable.  IQueryable is non-generic, but its ever so much a query as IQueryable<T>.  In fact, it has its own CreateQuery() method that builds a new IQueryable so you can do all this query building without all that static info getting in your way.


So how do you build the expression trees that make up a query.  Check out the System.Query.QueryExpression class.  It is a collection of static methods that correspond to the standard query operator pattern, but are more loosely typed, taking arguments that are Expression’s instead of Expression<T>’s.  These methods do all the work necessary to build up the call nodes that represent calls to the more fully typed IQueryable<T> query operator methods. 


Here’s how you can dynamic build up the query in the prior example.


IQueryable q = …;


 


ParameterExpression p = Expression.Parameter(typeof(Customer), “c”);


Expression body = Expression.EQ(


    Expression.Property(p, typeof(Customer).GetProperty(“City”)),


    Expression.Constant(London)


    );


LambdaExpression predicate = QueryExpression.Lambda(body, p);


Expression where = QueryExpression.Where(q.Expression, predicate);


 


q = q.CreateQuery(where);


 


You can then call GetEnumerator() on ‘q’ to execute the query.


Too hard you say?  Do I have to write all that code?  What if I just want my users to type in a little filter expression.  Do I have to turn that into all those nodes?


Ah, well, we’ve thought of that too.  Try this out. 


IQueryable q = …;


 


ParameterExpression p = Expression.Parameter(typeof(Customer), “c”);


LambdaExpression predicate =
    QueryExpression.Lambda(“c.City = ‘London‘”, p);


Expression where = QueryExpression.Where(q.Expression, predicate);


 


q = q.CreateQuery(where);


Or if you don’t want them to have to type silly dot’s, just don’t give the parameter a name.


IQueryable q = …;


 


ParameterExpression p = Expression.Parameter(typeof(Customer), “”);


LambdaExpression predicate =
    QueryExpression.Lambda(“City = ‘London‘”, p);


Expression where = QueryExpression.Where(q.Expression, predicate);


 


q = q.CreateQuery(where);


 


Viola!  Instant filter. Instant dynamic query. Run it here. Run it there. Why, you can run it everywhere.


But that’s enough from me. Why are you wasting time reading this when you could be downloading the preview and trying it out for yourself?


Matt


 

Comments (32)

  1. Matt,

    this stuff is so cool. linq rocks…

    WM_THX

    thomas woelfer

  2. ransom says:

    OMG! This is so cool. I don’t know what it is, but it is so cool.  

  3. Marc Brooks says:

    Show me how that last one can be parameterized to avoid injection issues…

  4. mattwar says:

    You wouldn’t use that text based filter if you were assembling user input with other pieces, you’d use the node construction pattern. The text based filter would only be used if the entire predicate is supplied by the user.  The user cannot cause injection because this expression is parsed, turned into expression nodes that must statically check as a correct predicate. DLINQ automatically converts the user supplied literal into an ADO.Net parameter, so there is no injection on that end either.

  5. I mentioned this in my last blog entry, but thought I should make it more discoverable. We’ve released…

  6. I mentioned this in my last blog entry, but thought I should make it more discoverable. We’ve released…

  7. Pedro Felix says:

    A new LINQ CTP is available. Looking for a reason to install it? See&amp;nbsp;this.

  8. mikep says:

    Matt, introduction of IQueriable<T> seams a logical step to bridge different linq implementations of PDC timeframe. It seams next step would make explicit IEnumerable<T> based query operators implementation not necessary?! What do you think?

  9. mattwar says:

    It would seem that you don’t need IEnumerable<T> operators at all anymore.  However, you still need them as actual methods to invoke when the IQueryable<T> is executed locally, and its has additional overhead due to the expression tree representation and the runtime compilation.

  10. There’s an update to the LINQ extensions for the .NET framework available on MSDN. Check it out on…

  11. Mike Griffin says:

    See (scroll down to the bottom for better samples)

    http://www.entityspaces.net/portal/Documentation/QueryAPISamples/tabid/80/Default.aspx

    Anyway, LINQ looks very cool Matt, what level are you involved with it?  

  12. Mike Griffin says:

    AggregateTestCollection aggTestColl = new AggregateTestCollection();

    aggTestColl.Query.es.CountAll = true;

    aggTestColl.Query

       .Select (aggTestColl.Query.IsActive,

                aggTestColl.Query.DepartmentID)

       .Where  (aggTestColl.Query.IsActive.Equal(true))

       .GroupBy(aggTestColl.Query.IsActive,

                aggTestColl.Query.DepartmentID)

       .OrderBy(aggTestColl.Query.DepartmentID.Ascending,             aggTestColl.Query.IsActive.Ascending);

    aggTestColl.Query.es.WithRollup = true;

    aggTestColl.Query.Load();

    Yields

    ==============

    SELECT [IsActive],[DepartmentID] ,COUNT(*) AS ‘Count’ FROM [AggregateTest]

    WHERE ([IsActive] = @IsActive1 )

    GROUP BY [IsActive],[DepartmentID] WITH ROLLUP

    ORDER BY [DepartmentID] ASC,[IsActive] ASC

    Same binary code runs on all databases.

  13. mattwar says:

    Mike, you could certainly add LINQ to your product. It would be relatively simple to translate IQueryable<T> expression trees into your query API, so it would simply bolt on top of what you have now.

    By implementing IQueryable<AggregateTest> on your AggregateTestCollection class your users could write LINQ queries.

  14. Mike Griffin says:

    That’s exactly what we were thinking. We provide a whole other superset of functionality but thought exactly as you suggested, implement the IQueryable<> syntax. We’re releasing 1.4 this weekend which adds MySQL and some other cools stuff. Then 1.5 adds full hierarchical support. Of course we generate it all from your db schema. What’s cool is that our NUnit test suite runs the exact same binary code against Oracle, Access, Microosft SQL, and MySQL the only difference being a connection string (of course, you have to use a schema that works in all databases but it’s very forgiving). Anyway, we’re thinking along the same lines …

  15. Добавлены новые возможности: Поддержка запросов времени выполнения, соединений т

  16. One of the biggest Linq improvements from the last CTP is IQueryable&amp;lt;T&amp;gt; which allows polymorphic…

  17. Marco Russo says:

    Qualche mese fa scrivevo qualche commento relativamente al ruolo di LINQ. Il recente rilascio di una…

  18. After false steps of ObjectSpaces and non-existent O/R mapping tools from Microsoft, the LINQ family…

  19. After false steps of ObjectSpaces and non-existent O/R mapping tools from Microsoft, the LINQ family…

  20. Christopher Shain says:

    I do a good amount of work in the  multidimentional (MDX) query space.  Will LINQ be extended for that paradigm also?

  21. gay rape says:

    We are wellocme to it’s configuration.

  22. This is soo cool.

    We are currently implementing Linq support for our opensource mapper (NPersist)

    http://blogs.wdevs.com/phirephly/archive/2006/06/05/13420.aspx#13422

    it works lovely..

    kudos to the Linq team :-)

  23. The documents on ADO.NET vNext that were previously pulled from MSDN have been republished by Microsoft.I…

  24. This is the first in a series of posts on C# and LINQ. These posts will describe a natural, easy to understand

  25. By Matt Duffin. So, you’re creating an application… no surprise there. As part of that application, you

  26. rod says:

    following the example, i get the error ->

    The type or method has 2 generic parameter(s), but 1 generic argument(s) were provided. A generic argument must be provided for each generic parameter

    from the call -> q.CreateQuery(where);

    Im Trying to build a  runtime hack to get around the fact linq doesnt support array.contains by building my where clause manually.