N-Tier Improvements for Entity Framework


The first version of Entity Framework provides convenient ways to load, manipulate and persist objects and relationships. As with many other O/RMs, Entity Framework has a state manager that tracks every change made. Existing objects are typically loaded first from the database, later modified, and finally the changes are saved back to the store.


Another feature, full graph serialization, makes it very easy for developers to ship around object graphs representing snapshots of the current state of the world, across execution boundaries.


The next version of Entity Framework will also support Persistence Ignorance. Therefore object graphs can now be made of POCO instances, which WCF now also supports.


Nonetheless, there is a task that belongs in any N-Tier application that still requires a great amount of work for developers using EF:  decoding messages that represent state changes performed by the client, meant to be processed or persisted by the service.


The major pain point in the first version is the number of intricate steps a program needs to perform in order to setup the state manager in the appropriate state according to the changes encoded in the message. For instance, for any but the simplest scenarios, in order to use the basic graph manipulations APIs in ObjectContext, like AddObject and AttachTo, it is necessary to “shred” the object graphs contained in the message, which destroys important information about relationships between object instances that therefore has to be kept somewhere else (as an illustration of this approach, see Danny Simmons’s EntityBag in project Perseus).


In addition, given that the message typically contains all the necessary information and that concurrency is usually a concern, the program shouldn’t have to re-query the database. Instead, it should be easier to “tell” the state manager about the original state (that came from the database previously) and about what changes were made.


There are a few approaches to N-Tier that are very popular in the industry. Customers using Entity Framework for the first time often carry the expectation that it will support a similar experience:


DataSet


The original ADO.NET data access technology includes the DataSet, a versatile data cache that can be serialized as XML and be used to transport collections of rows and relationships and, also diffgrams representing changes.


DataSets make many N-Tier scenarios extremely easy to implement, and over time, have proven helpful for many customers. In spite of this, they are not appropriate for some scenarios and architectural patterns:




  1. The lack of a widely accepted standard serialization format has hindered the development of implementations that could produce or consume datasets in other platforms.



  2. Diffgrams represent sets of CREATE, UPDATE and DELETE operations of arbitrary depth, which is not adequate for architectural patterns that conceive operations with more constrained semantics (i.e. PlaceOrder, CreateCustomer).



  3. No simple ways to control the application of batch CUD operations contained in diffgrams makes them unsuitable any time there is a trust boundary between the client and the server.



  4. Also, the data-centered perspective of DataSets leads to an anti-pattern known as the “anemic domain model”, which is characterized by a separation between the data aspect and the behavioral aspect of domain objects.


Data Transfer Objects


The approach typically used in the DDD and SOA communities to address  multi-tier scenarios is to hand-build a service with well defined operation semantics together with the a conceptual message format, usually composed of data transfer objects (DTOs).  Implementing DTOs often also involves writing the logic that translates DTOs back and forth to persistent objects.


Here is a simplified set of rules that characterizes the use of DTOs:




  1. Actual entities are not serialized, just DTOs, which are value-only objects (without behavior) are exposed in the boundaries.



  2. The client does not depend on the types defined on the server. The shapes of the DTOs are the sole contract, and the types used in the client are for the client only.



  3. The context or container is not sent together with the message.



  4. The service decides what to do with the contents of the message. It is not the message who decides (as an extreme counter example, imagine a message that tells the service to delete all the contents of the database).


Those rules are especially important in SOA scenarios, in which the client and the server can be separated by a trust boundary, such as the one that exists between two different companies.


There are other permutations of layered architectures, though, in which the level of decoupling that DTOs provide may not justify the increased complexity, for example:




  1. A multi-tier application in which the client is fully trusted.



  2. A rich internet application in which an infrastructure service is purely used for data persistence (is by design a CRUD-only service) and therefore types exposed do not contain behavior anyway.



  3. Any architecture in which the persistence framework is actually used to provide and manage DTOs and not the actual entities.


REST


ADO.NET Data Services provides a rich framework that developers can easily use to expose any Entity Data Model as a REST-style resource collection. Simple HTTP verbs like GET, PUT, POST and DELETE are used in combination with URIs to represent CRUD operations against the persistence service.


This pattern is suitable for many applications and can easily be combined with service operations with richer semantics in the same application.


ADO.NET Data Services, however, may not be compelling for those customers who want to define their services as operations with richer semantics entirely.


Goals


ADO.NET Data Services goes a long way addressing the needs of customers doing RESTful services, and so there is no need for us to improve Entity Framework experience on that space.


With the N-Tier improvements for Entity Framework, we want to address some of the same problem space as DataSet, but we want to avoid the primary issues with it.


Ideally, we would like to provide building blocks that are appealing for developers building solutions on a wide range of architectures. For instance, we would like to provide fine enough control for DTO proponents, but at the same time reduce the level of pain that those trying to address simpler scenarios experience today.


Design


If we just wanted to provide a DataSet like experience on top of Entity Framework, the most straightforward (although very expensive to test) way would probably be based on full serialization of the state manager. Basically, the internal representation of ObjectStateManager includes collections of objects to be deleted, inserted, and modified, and also relationships to be added and deleted, as well as objects and relationships that should remain unchanged.


Such a model would lead to very fine grain control of the state manager, but you would lose some of the benefits of operating on the domain objects and the higher level APIs. For instance, it would make it extremely easy for the program to set the state manager in a completely invalid state.


An alternative representation for such an experience would be to ship a “change log” containing the history of all operations performed on the client. This representation has the disadvantage of being more verbose than the previous one, but it would make “point in time” rollbacks pretty easy to implement.


Besides these two, there are a few more interesting generic representations for changes in a graph, but in general, they all suffer from the same disadvantage: providing a solution for them does not give the user the level of control that the most complicated scenarios and sophisticated patterns require.


Based on this fact, we decided to adopt the following goal for the design:


Entity Framework won’t define its own unique representation for the set of changes represented in an N-Tier application. Instead, it will provide basic building block APIs that will facilitate the use of a wide range of representations.


This has the desirable consequence that Entity Framework won’t impose a pattern for N-Tier. Both DTO-style and DataSet-like experiences can be built on top of a minimal set of building blocks APIs. It is up to the developer to select the pattern that better suits the application.


Here is a first cut of the new methods that we are proposing to add to the ObjectContext. The new APIs will work with EntityObjects, IPOCO and POCO entities:


public partial class ObjectContext
{
    /// <summary>
    /// Apply property values to original state of the entity
    /// </summary>
    /// <param name=”entitySetName”>name of EntitySet of root</param>
    /// <param name=”original”>entity with original values</param>
    public void ApplyOriginalValues(
        string
entitySetName,
        object
original) { }

   
    /// <summary>
    /// Changes the state of the entity and incident relationships
    /// </summary>
    /// <param name=”entity”>entity to change state of</param>
    /// <param name=”state”>new state</param>
    public void ChangeObjectState(
        object
entity,
        EntityState
state) { }


    /// <summary>

    /// Changes state of relationship
    /// </summary>
    /// <param name=”source”>source object of relationship</param>
    /// <param name=”target”>target object of relationship</param>
    /// <param name=”relationshipName”>name of relationship</param>
    /// <param name=”sourceRole”>role of source object</param>
    /// <param name=”targetRole”>role of target object </param>
    /// <param name=”state”>new state</param>
    public void ChangeRelationshipState(
        object
source,
        object
target,
        string relationshipName,
        string
sourceRole,
        string
targetRole,
        EntityState state) { }

    /// <summary>
    /// Changes state of relationship represented by navigation
    /// property

    /// </summary>
    /// <param name=”source”>source object of relationship</param>
    /// <param name=”target”>target object of relationship</param>
    /// <param name=”navigationProperty”>navigation property</param>
    /// <param name=”state”>new state</param>
    public void ChangeRelationshipState(
        object
source,
        object
target,
        string navigationProperty,
        EntityState
state) { }

   
    /// <summary>
    /// Changes state of relationships represented by lambda
    /// expression

    /// </summary>
    /// <typeparam name=”TSource”>type of source entity</typeparam>
    /// <param name=”source”>source entity of relationship</param>
    /// <param name=”target”>target entity of relationship</param>
    /// <param name=”selector”>property selector expression</param>
    /// <param name=”state”>new state</param>
    public void ChangeRelationshipState<TSource>(
        TSource source,
        object
target,
        Expression<Func<TSource, object>> selector,
        EntityState state) { }

 
}

ApplyOriginalValues


Similar to the existing ApplyPropertyChanges, but this API keeps the tracked entity intact, only affecting the corresponding original values. This operation only works for entities in Modified, Unchanged or Deleted states. Added entities by definition don’t have original state.


ChangeObjectState


This method will transition the state of a tracked entity passed as argument, but will also have side-effects on incident relationships. In case the new state is modified, it will also mark all properties as modified, regardless of the original and current values.


ChangeRelationshipState


Similar to ChangeObjectState, but sets the relationship between two tracked entities to the provided state. Relationships cannot be in the modified state, but this method can be used to report which relationship should be added, deleted, or unchanged.


The relationship doesn’t need to exist in the context, but the entities do. If the relationship doesn’t exist, it can be created in the new state. For instance, when this code is invoked, a new relationship can be created between customer1 and order1 in the added state:


context.ChangeRelationship(
    customer1,
    order1,
    c=>c.Orders,

    EntityState
.Added);

The relationship is typically represented by the navigation property but the metadata names of the relationship and roles can also be used in a different overload.


Design notes




  • All parameters named “entity”, “source” and “target” accept either entities or EntityKeys.



  • We could choose to deprecate the ApplyPropertyChanges API and replace it with ApplyCurrentValues, which is more consistent name-wise with the new ApplyOriginalValues method.



  • For convenience, we should add similar APIs in other types like ObjectStateEntry, RelatedEnd and others as appropriate. For instance, the ObjectStateEntry will also have a ChangeState  API that will only take the target state parameter.



  • We could choose to put ChangeObjectState() and the various ChangeRelationshipState() in ObjectStateManager rather than in ObjectContext, since this is a lower level API than everything else currently in the ObjectContext.


Scenario tests


To get a sense of how the new API can be used, we have tried several different approaches and representations. We also have plans to release sample code that will show in more detail different ways to use the new API. In the meanwhile, here are a few simple cases.


General purpose Attach with state resolution delegate


In this scenario between a client and mid-tier, the intention is to allow a client to query for a Customer entity with the Customer’s Order entities. The client then makes a variety of changes to the Customer entity graph including modifying the Customer, and adding or removing Orders.  The client would like to send the updated Customer entity graph to the mid-tier so that it can be processed and persisted.


This system will use a simple mechanism to help with change tracking on DTO entities. Each DTO will implement a small interface to help determine the state of the entity using three properties: IsNew, IsModified, and IsDeleted. In this example, original values are not tracked by the client.


public interface IEntityWithChanges
{
    bool IsNew { get; }
    bool IsModified { get; }
    bool IsDeleted { get; }
}

When the client makes changes to the Customer entity graph, the only requirement in this scenario is that when an entity changes, the appropriate property will return true to help determine the state of the entity.


The mid-tier will expose a service method that implements the logic to update the Customer entity graph. Rather than traverse the Customer, her Orders and OrderLines looking for changes, the service method will use a convention based mechanism that knows how to deal with the IEntityWithChanges interfaces. In this scenario, this mechanism takes the form of an extension method to the ObjectContext that will attach an entity graph to the context, and use a delegate to map each entity that implements IEntityWithChanges to an EntityState so that it can be used with the new ChangeObjectState APIs.


First we can define a method to map from an IEntityWithChanges to an EntityState:


/// <summary>
/// A mapping from an IEntityWithChanges to an EntityState
/// </summary>
public EntityState GetEntityState(object entity)
{
    IEntityWithChanges entityWithChanges =
       entity as IEntityWithChanges;

    if (entityWithChanges != null)
    {
        if (entityWithChanges.IsNew)
        {
            return EntityState.Added;
        }
        else if (entityWithChanges.IsModified)
        {
            return EntityState.Modified;
        }
        else if (entityWithChanges.IsDeleted)
        {
            return EntityState.Deleted;
        }
    }
    return EntityState.Unchanged;
}

Next we will define the extension method on an ObjectContext that can attach an entity graph and use a mapping delegate to determine the state of each entity in the graph.


/// <summary>
///
Attach a graph of entities using a supplied mapping function to
/// determine the entity’s state.
/// In this example, the state of relationships between entities is
/// determined by the rules of ChangeObjectState:
/// – if one entity in the relationship is Added, the relationship is
///   Added
/// – if an entity is Deleted or Detached, all incident relationships
///   are Deleted or Detached
/// </summary>
///
<param name=”entitySetName”>name of EntitySet of root</param>
///
<param name=”entityGraph”>graph of entities to attach</param>
///
<param name=”entityStateMap”>EntityState resolver</param>
public static void Attach(
    this ObjectContext context,
    string entitySetName,
    object entityGraph,
    Func<object, EntityState> entityStateMap)
{
    context.AttachTo(entitySetName, entityGraph);

    // Create a list of unique entities in the graph
    var allEntities =
        context
        .ObjectStateManager
        .GetObjectStateEntries(EntityState.Unchanged)
        .Where(entry => !entry.IsRelationship)
        .Select(entry => entry.Entity);
       
    // Apply the entityStateMap to each entity
    foreach (object entity in allEntities)
    {
        context.ChangeObjectState(entity, entityStateMap(entity));
    }
}

Finally we can define our service method that can update a Customer entity graph using the mapping delegate and the Attach extension method.


/// <summary>
/// A service method to update a Customer’s entity graph which
/// can include modifications to the Customer entity,
/// new Orders, and removal of Orders
/// </summary>
/// <param name=”c”></param>
public void UpdateCustomerOrder(Customer customer)
{
    using (NorthwindEntities context = new NorthwindEntities())
    {
        // Call the Attach extenion method with the mapping delegate
        // that goes from IEntityWithChanges to EntityState
        context.Attach(“Customers”, customer, GetEntityState);
        context.SaveChanges();
    }
}


ChangeOrder Service Operation


In this scenario, the mid-tier exposes a method to perform a very specific operation: changing the Customer that owns an Order. The client can set the Order’s Customer to a new Customer or to one that already exists in the store. In either case, the service method uses the old Customer so that it can remove the relationship between the Order and that Customer.


/// <summary>
///
Updates an Order by changing the relationship between the Order
/// and a Customer in this case
/// </summary>
///
<param name=”updatedOrder”>order to update</param>
///
<param name=”oldCustomer”>previous customer</param>
public static void ChangeOrder(
    Order
updatedOrder,
    Customer
oldCustomer)
{
    using (NorthwindEntities context = new NorthwindEntities())
    {
        // Tell the context about the entities to persist
        context.AttachTo(“Orders”, updatedOrder);
  
        // Change the state of the new customer
        // In this example, this is done by the convention:
        // – Customer.CustomerID is null –> customer is ‘Added’
        // – Customer.CustomerID is not null –> it already exists
        if(updatedOrder.Customer.CustomerID == “”)
        {
            // When state of customer changes to ‘Added’, state
            // of the relationship between updatedOrder and
            // pdatedOrder.Customer is also changed to ‘Added’
            context.ChangeObjectState(
                updatedOrder.Customer,
                EntityState.Added);
        }
        else
        {
            if (updatedOrder.Customer.CustomerID !=
                oldCustomer.CustomerID)
            {
                // only relationship needs to be marked as ‘Added’
                context.ChangeRelationshipState(
                    updatedOrder,
                    updatedOrder.Customer,
                    o => o.Customer,
                    EntityState.Added);

                // Report removal of relationship between
                // pdatedOrder and oldCustomer

                context.ChangeRelationshipState(
                    updatedOrder,
                    oldCustomer,
                    “Customer”,
                    EntityState.Deleted);

             }
        }
       
        // Persist the changes to the store
        context.SaveChanges();
    }
}

As is always the case, your feedback on this topic is welcome!


Jeff Derstadt
Jaroslaw Kowalski
Diego Vega
and The Object Services Team


This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post.

Comments (54)

  1. Frans Bouma says:

    I find this approach a really easy way out for you, but it doesn’t solve the problem for developers at all: they still have to babysit change tracking, something which is the job of the O/R mapper framework: an O/R mapper is there to do entity management, not the developer.

    The only true way to solve this is to place the change tracking inside the entity objects. This way, wherever the entity goes, the tracked changes go too. This requires more code, but that’s doable. As you’re saying you’re going to support poco, I do suspect you do that through dynamic subclasses generated in IL at runtime like nhibernate does? If so, you can generate the change tracking directly into the subclass.

    If not, then… it’s tough luck but you will have a hard time convicing that what you propose is really a solution. The main thing is: with change tracking inside the entity, one can send the entity to a different tier, not having the context available there, alter it and send it back and it is immediately saveable: the context doesn’t have to do anything, nor does the developer have to do anything.

    LLBLGen Pro (my o/r mapper framework, you should be familiar with it, MS did usability tests with it) does it that way, and the change tracking is completely transparent: one can fetch an entity graph on the server, send it over the wire to a client (xml webservices, wcf, remoting, whatever) and change the graph on the client. Then send the graph back to the server and simply persist it. No graph traversing, no babysitting on whatever data a context (which isn’t there anymore) produced. It’s been taken care of, without the developer needs to worry.

    But feel free to proceed with this of course :). I just want to let you know that if you listen too much to the POCO fanatics, you might lose sight on what’s really important, namely: software which just works, frameworks which just take care of the difficult stuff, so the developer can focus on business logic, not babysitting change sets.

  2. Jeff says:

    @Frans

    Good point to bring-up. It is certainly the case that this set of APIs does not address the actual change tracking that needs to be done on the client. One of our goals was to avoid defining a partcular wire-level format for changes sent from client to another tier.

    That being said, this API allows users and framework developers the ability to report changes back to the state manager in a general way. This gives users flexibility in how they choose to do change tracking on the client, whether it is via an ObjectContext, another O/RM technology, self tracking entities (which with a code generated template could be possible), or another technique. This is certianly not the only set of changes or additions that we hope to provide for N-tier in the Entity Framework; this was just a step. There is a huge value in supporting just-works out-of-the-box solutions, which as you say is what many developers would prefer. I expect a set of common patterns to emerge as recommended ways to help in the client change tracking space.

  3. We have recently published an article on API changes that we want to make in next version of Entity Framework

  4. JasonBSteele says:

    @Frans and Jeff,

    You’re both right! 🙂

    If these APIs are a step on the way to a framework that supports self tracking objects then I believe itt’s a step in the right direction!

    However I hope MS will support such a framework and not expect us to all re-invent the wheel by hacking together our own custom solutions.

    Just my opinion

  5. mhnyborg says:

    The entity framework team will never learn instead of making something that works there throwing more API at a dead end.

    But I am glad to read this because now I can safely remove the EF framework from my list of data access  choices.

  6. Kristofer says:

    I’m sure that somewhere in a dusty cabinet you will find the design documentation for good old ADO 2.x.

    Read that, read it twice, sleep on it and then discuss among yourselves: why make n-tier support any more complicated [for the users of EF/…] any more complicated than that?

    And then re-read what Frans Bouma wrote in his comment.

    Build change tracking into the entity classes. That will make n-tier support work as elegantly as it did ten years ago in ADO. Make generation of the change tracking code optional (but default) so those who want/need it can use it but those who don’t want/need it can simply switch it off and get clean light-weight objects…

    JMHO

    Kristofer

  7. Erik says:

    How does this IEntityWithChanges fit with Jimmy Nilssons " things you should not have to do in persistence ignorance…Provide mandatory specific fields?

  8. Andrej Burger says:

    When somebody said me, that MS added detached scenario to EF I was vary sad. MS again implemented something I found out a time ago. But after reading the article I am "happy". I dont think this solves much. So we can continue to earn money with wrapper above EF that solves all the problems with detached entities. OK we do not support POCO, we created base class for entities, we have own code generator, but customer do not cares. He has state tracking, original and current values, references and that is exactly what he needs and do not need to implement himself. He just loads, changes and saves the object.

    I understand that you cannot supply only high level solution. There are projects that need special care. So create layers, where the lowest layer is for special projects and the highest layer is for easy projects. Not one solution for everybody.

    Then developer can choose what layer will he use. That is much worth, than possibility to implement it himself even with published common patterns. But if he do not like it, he can use the lowest layer and implement it as he wants.

    Andrej

  9. More than 1.5 year ago (!) I wrote an article about why change tracking of changes of an entity should

  10. Diego Vega says:

    @Jason, Kristofer and Andrej

    Thanks for your comments, and keep them coming!

    As Jeff said, this is just a first step. As you can see in the explanation of the DataSet approach, and also in the initial post of this blog about the Transparent Design Process (http://blogs.msdn.com/efdesign/archive/2008/06/23/transparency-in-the-design-process.aspx), we are aware that many of our customers expect us to simplify the experience much more than this. We will keep working to satisfy this need, and here we think we are laying out the foundation for that.

    In any case, we want to make sure the building block API is public, because we think it augments Entity Framework as a platform that other developers can use to provide their own solutions for these problems. Hopefully Andrej will find it useful!

    On the other side, I have to say most of the customers we see asking questions in our forums about how to do N-Tier development using EF, seem not to be held back by having to write the kind of code that we show in the patterns here. They seem to hit a wall, however, when they see how the current API provides only all or nothing contract on graphs, and that therefore they need to shred entities and relationships.

    It would be very cool to hear more on how you would expect the experience to be.

  11. Diego Vega says:

    @Erik,

    IEntityWithChanges interface that you see in the code sample, is just that, sample code. Nothing like that is required in the entities.

    The intention is to show how you could define a contract yourself on how to represent state, and then use it in your own entities, to later write the piece of code that would use the contract for something useful.

    I probably don’t know Jimmy well enough to tell what he would think :), but pesonally I don’t mind using an interface like this in POCO, as long as it is either my interface or a very general purpose interface in the .NET Framework libraries.

  12. JasonBSteele says:

    @Diego

    Thanks for the feedback. Whilst I would appreciate that most of the people posting in the forums "seem not to be held back by having to write the kind of code that we show in the patterns here", I would say that they probably would rather not! 😉

    I would also so say that they are likely to be more advanced programmers and if you are looking for the ubiquitous adoption of EF then you will need to appeal to junior and intermediate programmers as well.

    In my opinion they are not going to be that concerned about whether its POCO or not – they will want something that makes sense and works out of the box. And by "works" I mean change tracking across tiers.

    Oh and BTW – that means in Astoria as well. At present the client side code generated doesn’t support change tracking. this means that when you bind it to an editable Silverlight grid say, you have to write event handlers for the grid to explicity say what data will have been changed. See http://mtaulty.com/CommunityServer/blogs/mike_taultys_blog/archive/2008/06/30/10549.aspx for an example.

    Its hard not to see this as a step backwards.

    Thanks for listening.

  13. Weekly digest of interesting stuff

  14. See Entity Framework Team Abandons Unified N-Tier Architecture for v2 (http://oakleafblog.blogspot.com/2008/11/entity-framework-team-abandons-unified.html) for my take on this travesty.

    Hopefully Matt Warren will revive his mini-connectionless DataContext for LINQ to SQL or Danny Simmons will productize Perseus.

    –rj

  15. Kristofer says:

    Diego,

    At the design/development stage it should be no more complex than a checkbox or a messagebox asking "include automagic change tracking?". As I already mentioned, something along the lines of the ADO 2.x implementation of n-tier support should be the minimum supported at the simple end. Then expose every interface you see fit for those who want to be able to customize or implement their own change tracking.

  16. lynn says:

    I’ve read this several times and I think you make a set of good points after getting past my initial frustrations of ‘NO CHANGE TRACKING’. The ‘ChangeOrder Service Operation’ scenario kind of reminds me of kind of rule enforcement that is possible in Astoria.

    Personally I have no problem with Astoria across the wire, especially if joins and projects become possible.  I can leverage Astoria’s API to enforce rules. However, I just need simple state tracking between ObjectContext and session state or ObjectContext and local store on the client. Are you considering supporting these kind of simple scenarios with a set of out-of-the-box tools built on your proposed tracking layer?

    (If they are already there please let me know becasue I have missed something.)

    Thanks for your patience and consideration,

    Lynn

  17. &quot;Tidak ada klien yang harusnya bisa menerima bahwa tim yang di-hire untuk mengerjakan program aplikasi

  18. Yoot says:

    Entity Framework => Entity Base Class Library

  19. Syl_74 says:

    I agree with Frans Bouma. Your approach is a really easy way out for you, but it doesn’t solve the problem for developers at all: they still have to babysit change tracking.

    By the way, in the first scenario with Customer+Orders (using IEntityWithChanges), could you explain how the server will detect "Deleted" orders, because theses orders are deleted, they won’t be in the Customer’s Order collection ?

  20. Meta-Me says:

    Well I went to Vega for Dev Connections. What an experience. Never been there before, but for first two

  21. Pop Catalin says:

    I too agree with Frans Bouma, the amount of plumbing that EF v1 requires is ridiculous, add this on top of a rigid API, and EF doesn’t come close to what it was promised to be.

    Automatic change tracking is a core requirement of any data access technology newer then 10 years, but EF is not supposed to be yet another data access technology is supposed to be something more, under these circumstances I fail too see how it can miss on such a basic requirement.

    Add to the mix improper support for data binding, and EF is not any longer a great "tool", even if academically speaking it may be  a great "technology". We the end developers need great tools to do out jobs, not great technologies. DataSets were and are great tools to work with in spite of their shortcomings as a technology, because they offered so much functionality built in, and didn’t require endless tweaking and plumbing to accomplish lots of different tasks.

    I really don’t want to use another EF version that instead of making my life easy, it gives me endless pains covered by the promises of database independence, which are not quite there yet (No migrations, no model first).

    I would like to see EF v2 launched and having some "serious" use cases covered with demos from Microsoft. (and by serious I don’t mean one form with a master detail on it, but samples that touch every aspect of the framework for web and desktop development, from 2 tier to multitier).

    In my shop we are already considering a switch to nHibernate, which is kind of funny because what pushed us in that direction was working with EF.

    I hope you use this blog for the power of good, and give another thinking about what EF should offer by default.

  22. leriksen71 says:

    Further thoughts –

    I agree with Pop Catalin. If EF doesn’t hit a home run in 2.0 than its a big loss. Put the decision to open up the change tracking API on the ObjectContext is a great choice and there will be many that appreciate it. But there are also quite a few of us that need to map a database to POCO, go across tiers and have everything -just work-. Otherwise EF and DataSets are a trade off between easy DB access or easy state management. EF needs to be a total win in all areas.

    I think the multiple out-out-of-the-framework choices along with the open change tracking model would be a good idea. No choices to go along the open tracking model is an incomplete solution.

  23. alphus says:

    I agree with Frans.

    90%+ of your user base will not be writing a framework on top of EF; they will be *using* it for data access. Please support change tracking inside entities.

  24. Diego Vega says:

    @Lynn:

    I am not sure I understand your question completely, but if you are ok with the REST style, and you want to synchronize state between a service and a local store, it is very likely that you will be interested in something the Astoria Team is working on: Astoria Offline

    You can learn more about it here:

    http://blogs.msdn.com/astoriateam/archive/2008/11/03/alpha-preview-of-project-codename-astoria-offline-coming-soon.aspx

  25. efdesign says:

    @Pop Catalin, leriksen71, alphus

    Thanks for the feedback! The message that we need to do more than just provide this API has been received and welcomed.

    We are actually looking hard into some high level approaches we would like to make part of the experience of using EF. It is not clear yet whether or which are going to make it into the product in the next version. In any case, we have the alternative of releasing them in source code form as EF Extensions (see http://code.msdn.microsoft.com/adonetefx).

    There is a reason we need to take this first step. Some time ago, we started several prototypes of a general end-to-end solution for N-Tier, and we also started talking to internal partners and customers that usually deal with this kind of problem. We found a wide spectrum of possibilities and even conflicting points of view, but they all had in common the need for a more expressive API.

    I think Alex James puts it very elegantly. At the core, we are trying to improve the Entity Framework API so that it becomes possible to express intent:

    http://blogs.msdn.com/alexj/archive/2008/11/24/state-vs-statement-of-intent.aspx

    In retrospective, we should have made clearer that this was the first blog post on N-Tier improvements, and not the last one.

    Thanks,

    Diego

    PS: I would also be interested in hearing what the pain points you have found on databinding with Entity Framework are.

  26. leriksen71 says:

    @Diego

    Thanks for the clarification! Looking forward to more discussion.

  27. 墙头草 says:

    社区对于ADO.NETEntityFramework和LINQtoSQL的最大不满,就是它不支持更改跟踪。但只有在你连接到上下文对象的时候,你才可以修改对象并把它们保存回数据库。就像数据库连接…

  28. Jon Rista says:

    Well, this is another dissapointing article from the EF team. I have to agree with Frans, this is an easy way out for you guys. I’ll take it a step further, and say its a complete and total joke. There is absolutely NO WAY, EVER, that I would want to write my own change tracking plumbing…unless I was writing my own O/RM. That is kind of the whole point of POCO, PI, and snapshot-based change tracking…to completely eliminate change tracking as an issue for the developer. I don’t understand why its so hard to accomplish, either. The concept is very simple:

    1) Retrieve object graph, and take snapshot.

    2) Detach object graph.

    3) Modify object graph…locally, remotely, wherever the heck I please.

    4) Attach object graph, comparing the attached graph with the corresponding snapshot (identified by the root objects key, an EntityKey, some key I provide in addition to the graph, whatever).

    5) Save changes.

    I don’t want to have to deal with change tracking….EVER. Neither do I want to deal with EF beyond the level of retrieving/detaching a graph or attaching a graph. Everything between a retrieval/detach and attach should be 100% isolated from the O/RM…because thats all it is….the layer that translates from my object model to my database model.

    It shouldn’t matter if I detach and use the object graph locally, send it accross the wire from a web service, use it from a REST service, etc. How I use my object graphs should not be a concern for my object persistence layer. Neither should the persistence layer be a concern for the activities performed on a graph between detach and attach. Thats the whole idea, the meaning behind "isolation" and "decoupling" and "ignorance". The only thing EF should be concerned about is knowing what the state of that graph was when I detached it, and what changes to take into account when I attach it again. And it should be able to determine what those changes are on its own…without any help from me (I completely disagree that change tracking should be built into objects…thats a persistence concern that breaks the whole idea behind POCO and PI).

    Thats what I need. Thats what anyone who is looking for an O/RM needs. I need to be able to build domain objects that do their job, regardless of how the data they represent is stored. I need a persistence layer that can take my domain objects, individually or as composite graphs, map them to my database tables, and take care of the ugly mess of translation to and from for me.

    I DO NOT need to have more work created for me because the EF team is too lazy to figure out how to compare a snapshot of a graph to the current state of a real graph.

    I DO NOT need to have more work created for me because the EF team wants to create some all-pervasive framework that permeates every corner of my code, instead of creating a neat, isolated, efficient, compact O/RM which would solve real-world problems without getting in my face every time I turn around.

    Come one guys…get a clue. Keep it simple. Keep it isolated. Solve real world problems. This new API does none of that, and it defeats the goal of supporting POCO and PI…change tracking is a PERSIETENCE CONCERN.

  29. I too, am a bit baffled that it is now unclear whether n-tier support will make it into v2.

    It is my opinion that a good ORM tool should provide N-tier support out of the box. REST Data Services are cool, but microsoft has something called WCF, which supports tcp, http, msmq and custom bindings. I want to be able to, out of the box, track changes over any of these bindings, including entities with relationships.

    While it may be cool that the EF can be highly customized it is essential that out of the box solutions are provided so that these basic needs are available.

  30. There is a saying in spanish:

    "Arbol que nace torcido jamas su tronco endereza pues hace naturaleza con el tronco que a crecido."

    I don’t know the translation! 🙂

  31. Tony Sneed says:

    Hey guys,

    I actually think this is a step in the right direction, simply because the proposed ChangeObjectState method makes it possible to persist updates to an object graph without needing to re-query the database.  In fact, the example of using IEntityWithChanges to track entity state across execution boundaries bears close resemblance to the approach I offer in my MSDN Magazine article on applying L2S and EF to n-tier application architectures. 🙂  For readers of this blog, here is the link: http://msdn.microsoft.com/en-us/magazine/dd263098.aspx.

    Of course, this initial step is not going to be sufficient to satisfy the throngs of developers out there who crave a fully functioning n-tier solution out-of-the-box.  But to deliver on this promise, there are three more steps you need to take:

    1) Propose a standard schema for change-tracking across service boundaries – and work with a standards body for industry acceptance.  While it may take some time for the standard to be established, at least we’ll be moving toward an platform-independent wire format that can be consumed by any code on either side of the wire.

    2) Write a client-side change tracker that is decoupled from any specific wire format, but also supply an implementation that uses the above wire format.

    3) Write an API for the EF  that can interpret state changes coming over the wire and persist them.

    The goal here is to arrive at a turn-key solution, which enables developers to build n-tier apps with POCO’s without having to re-invent the wheel or write a great deal of plumbing code.  I can understand your desire not to commit to a specific wire format, but without the existence of a standard schema, we’re going to see different formats emerge that are incompatible with one another.  It’s also important to provide a change tracker that is decoupled from the wire format and can track POCO’s without any dependence on EF or any other persistence stack.

    I hope these suggestions are helpful. 🙂

    Cheers,

    Tony Sneed

    http://blog.tonysneed.com

  32. unbornchikken says:

    Hi,

    I think you guys don’t understand what is our real problem.

    Here is the scenario:

    You release v(n). There are many stuff that is not working (aka broken). We add plumping and infrastructure code to get things working as espected.

    You hear our cries. Release v(n+1), where you address out problems and implement all that plumping stuff that we implemented on top of v(n). But our v(n) based plumping code won’t be compatible with v(n+1), we have to rewrite our solutions to get things working again. In v(n+1) you add a lot of new, but half implemented stuff, we have to write infrastructure code again to get things working in real world. You hear our cries … GOTO 10.

    Pain. A lot of pain.

    But. I think this is a great idea to split n-tier and WCF story out out of EF because those are another level of application design. I’ve seen Jamie Cool’s Silverlight LOB presentation at PDC where he showed us an n-tier framework prototype that working on top of the EF vNext. This is promising, great stuff! I hope MS dev. teams’ right hand knows what left hand doing. 🙂

    And please, EF Team,  pretty please! Make a real world LOB application sample around your stuff, and you WILL face exactly the same problems that we face day to day. Sad but true EF v1 has nothing to do with real world applications today. Nothing.

    Anyway, thanks for the good work that you’ve already done! And I hope my funny English is understandable enough. 🙂

    Regards,

    Gabor Mezo

    Visual C# MVP

    Hungary

  33. Mikinho says:

    Good concepts but as others have mentioned until it is further along it is a liability to use.

    Just one comment…use best practices in your code examples.  It is hard to take code serious when you post things like the below

    if(updatedOrder.Customer.CustomerID == "")

    String.IsNullOrEmpty is available for a reason, use it.

  34. Luke says:

    We currently are developing an application using Linq to SQL in the middle tier. We have a data access service that exposes some basic CRUD operations as well as some for relationship maintenance. We perform security checks in this tier and throw exceptions / clear returned values when the user does not have permission to a given object or property.

    One thing that would be nice is the ability to do projections. With LINQ to SQL, this is all but impossible without using database-dependent sql or attempting to serialize an expression tree across the service boundary, which is fairly non-portable. With Entity Framework, we could serialize our projection as Entity SQL, but unfortunately there is no way to go from an expression tree to Entity SQL and/or back again, which is what we would want to do.  

    We still want to be able to use expression trees against a local IQueryable, which would then convert them to EntitySQL to be sent across the service boundary. Then the service would convert to the EntitySQL back to an expression tree, walk the tree to make sure permissions are OK, and execute the query. Also, to help facilitate this, why can’t there be an extensible ObjectContext that we could override on the client-side to intercept calls to query and save changes as forward them to the middle tier?

    So again, the two things I think would be useful for middle tier would be an EntitySQL parser/generator and a service-aware ObjectContext (or something like that).

  35. Hi,

    I have put some effort too into providing support for Entity Framework, version 1, to provide a general attach method which can handle most common scenario’s in attaching an object graph and applying changes as added, modified or deleted entities.

    The nice thing about the solution, in my opinion, is that it does not require change tracking. The AttachObjectGraph receives a detached entity graph and will – while attaching it – apply all changes it can detect within a specified scope.

    You’ll find a detailed description of this solution including code and sample on:

    http://www.codeproject.com/KB/architecture/attachobjectgraph.aspx

    Kind regards,

    Rudi Breedenraedt

  36. Tanveer Badar says:

    I have to agree with everyone else here. EF is a half-baked solution much worse than enums in C++.

    I don’t understand why it is so hard for you guys to do this right. ChangeObjectState or not, do this in the framework with no part left to the end developer. Why should I hook up your infrastructure and dictate to EF how to persist a detached entity? It isn’t my job, really. If I had to do that, I am better off with a mature ORM.

    Do you know how nHibernate does that?

    1- Compare entity key to unsaved-value attribute.

    2- Issue insert [table] [values] blah blah blah if they match.

    3- Issue update [table] [values] blah blah blah where key = entity key if they don’t.

    Why should it be so hard to do this yourselves?

  37. Background One of the biggest pieces of feedback we received from the N-Tier Improvements for Entity

  38. Meta-Me says:

    Lots of the feedback we got on the EF design blog about our early N-Tier plans , highlighted that lots

  39. A couple of years ago I was involved in a couple of threads about doing change-tracking in client-side

  40. Cet article a pour objectif de décrire notre approche à propos de l&#39;accès aux données. En préambule

  41. davin says:

    之前还不了解wcf的时候,就常常在EF Design team blog上和adodotnetentityframework社区看到一些关于EF在wcf场景中的应用的话题和文章。因为一直很关注EF,所以很想知道在wcf中是怎样在应用的。想从互联网上找些代码示例看看,不过没有什么收获。之后在学习wcf时,在new WCF features in .NET 3.5 SP1 了解到 WCF完全支持POCO(Support for ADO.NET Entity Framework entities in

  42. Building applications that work across tiers is a core part of many application architectures. In .NET

  43. Shaleen Chugh says:

    I my opionion Microsoft should keep things simple (KISS). All the dirty work of CHANGE TRACKING once the entity has been be retrived and changed should be managed by the framework and not by the developers. It should be intelligent enough to find out what changes have been made to the object graph and apply them and keep all the dirty logic of how it has been done shielded away from the developers so developers can build applications quicker without having to bother about complex change tracking mechanics and applying them. Come on Microsoft its the 21st century no the 20th…….

  44. I received a mission to explore entity framework as a DAL for a new project, this project is intended

  45. &#160; Building applications that work across tiers is a core part of many application architectures

  46. Rajesh Kumar says:

    ChangeRelationshipState

    >> … In either case, the service method uses the old Customer so that it can remove the relationship between the Order and that Customer.

    Visualize a nTier distributed app. Middle tier ships initial data to GUI, which modifies the order’s customer and ships the Object graph back to Application server.

    1) Does the Application server hold on to state? For how long? If it stores the state, then in a disconnected/occassionally connected scenario, the data might come back after days…

    2) Ideally i think Entities should Self track changes (could be automated via Base class + DLR hacks)…

    So each layer (GUI, Communication, Integration and BL) can figure out their changes and transmit ALL upstream (DL->BL->….->GUI) and transmit ONLY the changes downstream (GUI->…->BL->DL).

    This will reduce the payloads and remove "state" requirements for other layers.

    Since Object Graph itself is "Data + State", which leads us back to the old concept of Data Grams but in Entities/Object graph level.

  47. Marg says:

    @ John Risto: Where is this ‘snapshot’ supposed to be stored in your step 1 ‘Take a snapshot of the graph’ ?? Don’t you know you’re not supposed to hold state between web service calls.

  48. Harish Suhanda says:

    I am working in a N-Tier web application where I’ve a separate project for EntityModel (.edmx). I’ve another Tier which I call DataProvider Layer, I perform all the CRUD operation in EntityModel via DataProvider. There is a separate DataProvider for all the Entities. All the DataProviders derive from the BaseProvider which has an object of type XXXEntities. This XXXEntities is availabe to all the DataProviders and I use this entity to perform the CRUD operations.

    I followed this approach because I do not want to create multiple objects of XXXEntities.

    How far is this approach correct? Is there any specific Architecture that I can/should follow to design my application/solution that is based on EntityFramework?

    Inbetween, I love EntityFramework 🙂

  49. Harish says:

    Inbetween, Since I am using a single XXXEntities object throughout the DataProvider Layer, I don’t have to attach the objects that I want to Update/delete.

    But I am still not confident about this approach.

  50. Anoosh says:

    بابا دمتون گرم ای ول مطالبتون خیلی توپه

  51. Gopi V says:

    Is the Entity Framework 4 has covered the Change Tracking Inside the Entities ? Or we still have to write our own frameworks to do that ?. Anyhow am very new to this EF.

    Any guidance towards the patterns, best practices from the EF Team would be much appreciated.

    Thanks.

  52. Ralph Krausse says:

      I have a Entity Framework question perhaps you could help me with. If I had 3 entities that each have an association to each other, how would add data if each has a constraint of a foreign key? IE, some data has to exist before data can be added because of the association?

    Thanks

    Ralph

  53. lucylitho says:

    <p><a href=" http://www.stickerprintersuk.com&quot; > flyer printers uk </a></p>  is a business leader in

    handling  any type of print projects for marketing needs.  

    Our products are brochures, catalogs, postcards,

    Business cards, banners, plastic cards, leaflet , sticker etc.

    Affordability, high quality and fast turnarounds are our area of focus.