N-Tier Improvements for Entity Framework

The first version of Entity Framework provides convenient ways to load, manipulate and persist objects and relationships. As with many other O/RMs, Entity Framework has a state manager that tracks every change made. Existing objects are typically loaded first from the database, later modified, and finally the changes are saved back to the store.

Another feature, full graph serialization, makes it very easy for developers to ship around object graphs representing snapshots of the current state of the world, across execution boundaries.

The next version of Entity Framework will also support Persistence Ignorance. Therefore object graphs can now be made of POCO instances, which WCF now also supports.

Nonetheless, there is a task that belongs in any N-Tier application that still requires a great amount of work for developers using EF:  decoding messages that represent state changes performed by the client, meant to be processed or persisted by the service.

The major pain point in the first version is the number of intricate steps a program needs to perform in order to setup the state manager in the appropriate state according to the changes encoded in the message. For instance, for any but the simplest scenarios, in order to use the basic graph manipulations APIs in ObjectContext, like AddObject and AttachTo, it is necessary to “shred” the object graphs contained in the message, which destroys important information about relationships between object instances that therefore has to be kept somewhere else (as an illustration of this approach, see Danny Simmons’s EntityBag in project Perseus).

In addition, given that the message typically contains all the necessary information and that concurrency is usually a concern, the program shouldn’t have to re-query the database. Instead, it should be easier to “tell” the state manager about the original state (that came from the database previously) and about what changes were made.

There are a few approaches to N-Tier that are very popular in the industry. Customers using Entity Framework for the first time often carry the expectation that it will support a similar experience:

DataSet

The original ADO.NET data access technology includes the DataSet, a versatile data cache that can be serialized as XML and be used to transport collections of rows and relationships and, also diffgrams representing changes.

DataSets make many N-Tier scenarios extremely easy to implement, and over time, have proven helpful for many customers. In spite of this, they are not appropriate for some scenarios and architectural patterns:

  1. The lack of a widely accepted standard serialization format has hindered the development of implementations that could produce or consume datasets in other platforms.

  2. Diffgrams represent sets of CREATE, UPDATE and DELETE operations of arbitrary depth, which is not adequate for architectural patterns that conceive operations with more constrained semantics (i.e. PlaceOrder, CreateCustomer).

  3. No simple ways to control the application of batch CUD operations contained in diffgrams makes them unsuitable any time there is a trust boundary between the client and the server.

  4. Also, the data-centered perspective of DataSets leads to an anti-pattern known as the “anemic domain model”, which is characterized by a separation between the data aspect and the behavioral aspect of domain objects.

Data Transfer Objects

The approach typically used in the DDD and SOA communities to address  multi-tier scenarios is to hand-build a service with well defined operation semantics together with the a conceptual message format, usually composed of data transfer objects (DTOs).  Implementing DTOs often also involves writing the logic that translates DTOs back and forth to persistent objects.

Here is a simplified set of rules that characterizes the use of DTOs:

  1. Actual entities are not serialized, just DTOs, which are value-only objects (without behavior) are exposed in the boundaries.

  2. The client does not depend on the types defined on the server. The shapes of the DTOs are the sole contract, and the types used in the client are for the client only.

  3. The context or container is not sent together with the message.

  4. The service decides what to do with the contents of the message. It is not the message who decides (as an extreme counter example, imagine a message that tells the service to delete all the contents of the database).

Those rules are especially important in SOA scenarios, in which the client and the server can be separated by a trust boundary, such as the one that exists between two different companies.

There are other permutations of layered architectures, though, in which the level of decoupling that DTOs provide may not justify the increased complexity, for example:

  1. A multi-tier application in which the client is fully trusted.

  2. A rich internet application in which an infrastructure service is purely used for data persistence (is by design a CRUD-only service) and therefore types exposed do not contain behavior anyway.

  3. Any architecture in which the persistence framework is actually used to provide and manage DTOs and not the actual entities.

REST

ADO.NET Data Services provides a rich framework that developers can easily use to expose any Entity Data Model as a REST-style resource collection. Simple HTTP verbs like GET, PUT, POST and DELETE are used in combination with URIs to represent CRUD operations against the persistence service.

This pattern is suitable for many applications and can easily be combined with service operations with richer semantics in the same application.

ADO.NET Data Services, however, may not be compelling for those customers who want to define their services as operations with richer semantics entirely.

Goals

ADO.NET Data Services goes a long way addressing the needs of customers doing RESTful services, and so there is no need for us to improve Entity Framework experience on that space.

With the N-Tier improvements for Entity Framework, we want to address some of the same problem space as DataSet, but we want to avoid the primary issues with it.

Ideally, we would like to provide building blocks that are appealing for developers building solutions on a wide range of architectures. For instance, we would like to provide fine enough control for DTO proponents, but at the same time reduce the level of pain that those trying to address simpler scenarios experience today.

Design

If we just wanted to provide a DataSet like experience on top of Entity Framework, the most straightforward (although very expensive to test) way would probably be based on full serialization of the state manager. Basically, the internal representation of ObjectStateManager includes collections of objects to be deleted, inserted, and modified, and also relationships to be added and deleted, as well as objects and relationships that should remain unchanged.

Such a model would lead to very fine grain control of the state manager, but you would lose some of the benefits of operating on the domain objects and the higher level APIs. For instance, it would make it extremely easy for the program to set the state manager in a completely invalid state.

An alternative representation for such an experience would be to ship a “change log” containing the history of all operations performed on the client. This representation has the disadvantage of being more verbose than the previous one, but it would make “point in time” rollbacks pretty easy to implement.

Besides these two, there are a few more interesting generic representations for changes in a graph, but in general, they all suffer from the same disadvantage: providing a solution for them does not give the user the level of control that the most complicated scenarios and sophisticated patterns require.

Based on this fact, we decided to adopt the following goal for the design:

Entity Framework won’t define its own unique representation for the set of changes represented in an N-Tier application. Instead, it will provide basic building block APIs that will facilitate the use of a wide range of representations.

This has the desirable consequence that Entity Framework won’t impose a pattern for N-Tier. Both DTO-style and DataSet-like experiences can be built on top of a minimal set of building blocks APIs. It is up to the developer to select the pattern that better suits the application.

Here is a first cut of the new methods that we are proposing to add to the ObjectContext. The new APIs will work with EntityObjects, IPOCO and POCO entities:

public partial class ObjectContext
{
    /// <summary>
    /// Apply property values to original state of the entity
    /// </summary>
    /// <param name="entitySetName">name of EntitySet of root</param>
    /// <param name="original">entity with original values</param>
    public void ApplyOriginalValues(
string entitySetName,
object original) { }
   
    /// <summary>
    /// Changes the state of the entity and incident relationships
    /// </summary>
    /// <param name="entity">entity to change state of</param>
    /// <param name="state">new state</param>
    public void ChangeObjectState(
object entity,
EntityState state) { }

/// <summary>
    /// Changes state of relationship
    /// </summary>
    /// <param name="source">source object of relationship</param>
    /// <param name="target">target object of relationship</param>
    /// <param name="relationshipName">name of relationship</param>
    /// <param name="sourceRole">role of source object</param>
    /// <param name="targetRole">role of target object </param>
    /// <param name="state">new state</param>
    public void ChangeRelationshipState(
object source,
object target,
string relationshipName,
string sourceRole,
string targetRole,
EntityState state) { }

    /// <summary>
    /// Changes state of relationship represented by navigation
/// property
    /// </summary>
    /// <param name="source">source object of relationship</param>
    /// <param name="target">target object of relationship</param>
    /// <param name="navigationProperty">navigation property</param>
    /// <param name="state">new state</param>
    public void ChangeRelationshipState(
object source,
object target,
string navigationProperty,
EntityState state) { }
   
    /// <summary>
    /// Changes state of relationships represented by lambda
/// expression
    /// </summary>
    /// <typeparam name="TSource">type of source entity</typeparam>
    /// <param name="source">source entity of relationship</param>
    /// <param name="target">target entity of relationship</param>
    /// <param name="selector">property selector expression</param>
    /// <param name="state">new state</param>
    public void ChangeRelationshipState<TSource>(
TSource source,
object target,
Expression<Func<TSource, object>> selector,
EntityState state) { }
 
}

ApplyOriginalValues

Similar to the existing ApplyPropertyChanges, but this API keeps the tracked entity intact, only affecting the corresponding original values. This operation only works for entities in Modified, Unchanged or Deleted states. Added entities by definition don’t have original state.

ChangeObjectState

This method will transition the state of a tracked entity passed as argument, but will also have side-effects on incident relationships. In case the new state is modified, it will also mark all properties as modified, regardless of the original and current values.

ChangeRelationshipState

Similar to ChangeObjectState, but sets the relationship between two tracked entities to the provided state. Relationships cannot be in the modified state, but this method can be used to report which relationship should be added, deleted, or unchanged.

The relationship doesn’t need to exist in the context, but the entities do. If the relationship doesn’t exist, it can be created in the new state. For instance, when this code is invoked, a new relationship can be created between customer1 and order1 in the added state:

context.ChangeRelationship(
customer1,
order1,
c=>c.Orders,
EntityState.Added);

The relationship is typically represented by the navigation property but the metadata names of the relationship and roles can also be used in a different overload.

Design notes

  • All parameters named “entity”, “source” and “target” accept either entities or EntityKeys.

  • We could choose to deprecate the ApplyPropertyChanges API and replace it with ApplyCurrentValues, which is more consistent name-wise with the new ApplyOriginalValues method.

  • For convenience, we should add similar APIs in other types like ObjectStateEntry, RelatedEnd and others as appropriate. For instance, the ObjectStateEntry will also have a ChangeState  API that will only take the target state parameter.

  • We could choose to put ChangeObjectState() and the various ChangeRelationshipState() in ObjectStateManager rather than in ObjectContext, since this is a lower level API than everything else currently in the ObjectContext.

Scenario tests

To get a sense of how the new API can be used, we have tried several different approaches and representations. We also have plans to release sample code that will show in more detail different ways to use the new API. In the meanwhile, here are a few simple cases.

General purpose Attach with state resolution delegate

In this scenario between a client and mid-tier, the intention is to allow a client to query for a Customer entity with the Customer’s Order entities. The client then makes a variety of changes to the Customer entity graph including modifying the Customer, and adding or removing Orders.  The client would like to send the updated Customer entity graph to the mid-tier so that it can be processed and persisted.

This system will use a simple mechanism to help with change tracking on DTO entities. Each DTO will implement a small interface to help determine the state of the entity using three properties: IsNew, IsModified, and IsDeleted. In this example, original values are not tracked by the client.

public interface IEntityWithChanges
{
    bool IsNew { get; }
    bool IsModified { get; }
    bool IsDeleted { get; }
}

When the client makes changes to the Customer entity graph, the only requirement in this scenario is that when an entity changes, the appropriate property will return true to help determine the state of the entity.

The mid-tier will expose a service method that implements the logic to update the Customer entity graph. Rather than traverse the Customer, her Orders and OrderLines looking for changes, the service method will use a convention based mechanism that knows how to deal with the IEntityWithChanges interfaces. In this scenario, this mechanism takes the form of an extension method to the ObjectContext that will attach an entity graph to the context, and use a delegate to map each entity that implements IEntityWithChanges to an EntityState so that it can be used with the new ChangeObjectState APIs.

First we can define a method to map from an IEntityWithChanges to an EntityState:

/// <summary>
/// A mapping from an IEntityWithChanges to an EntityState
/// </summary>
public EntityState GetEntityState(object entity)
{
    IEntityWithChanges entityWithChanges =
entity as IEntityWithChanges;
    if (entityWithChanges != null)
    {
        if (entityWithChanges.IsNew)
        {
            return EntityState.Added;
        }
        else if (entityWithChanges.IsModified)
        {
            return EntityState.Modified;
        }
        else if (entityWithChanges.IsDeleted)
        {
            return EntityState.Deleted;
        }
    }
    return EntityState.Unchanged;
}

Next we will define the extension method on an ObjectContext that can attach an entity graph and use a mapping delegate to determine the state of each entity in the graph.

/// <summary>
/// Attach a graph of entities using a supplied mapping function to
/// determine the entity's state.
/// In this example, the state of relationships between entities is
/// determined by the rules of ChangeObjectState:
/// - if one entity in the relationship is Added, the relationship is
/// Added
/// - if an entity is Deleted or Detached, all incident relationships
/// are Deleted or Detached
/// </summary>
/// <param name="entitySetName">name of EntitySet of root</param>
/// <param name="entityGraph">graph of entities to attach</param>
/// <param name="entityStateMap">EntityState resolver</param>
public static void Attach(
this ObjectContext context,
string entitySetName,
object entityGraph,
Func<object, EntityState> entityStateMap)
{
context.AttachTo(entitySetName, entityGraph);

// Create a list of unique entities in the graph
var allEntities =
context
.ObjectStateManager
.GetObjectStateEntries(EntityState.Unchanged)
.Where(entry => !entry.IsRelationship)
.Select(entry => entry.Entity);

// Apply the entityStateMap to each entity
foreach (object entity in allEntities)
{
context.ChangeObjectState(entity, entityStateMap(entity));
}
}

Finally we can define our service method that can update a Customer entity graph using the mapping delegate and the Attach extension method.

/// <summary>
/// A service method to update a Customer's entity graph which
/// can include modifications to the Customer entity,
/// new Orders, and removal of Orders
/// </summary>
/// <param name="c"></param>
public void UpdateCustomerOrder(Customer customer)
{
    using (NorthwindEntities context = new NorthwindEntities())
    {
        // Call the Attach extenion method with the mapping delegate
        // that goes from IEntityWithChanges to EntityState
        context.Attach("Customers", customer, GetEntityState);
        context.SaveChanges();
    }
}


ChangeOrder Service Operation

In this scenario, the mid-tier exposes a method to perform a very specific operation: changing the Customer that owns an Order. The client can set the Order’s Customer to a new Customer or to one that already exists in the store. In either case, the service method uses the old Customer so that it can remove the relationship between the Order and that Customer.

/// <summary>
/// Updates an Order by changing the relationship between the Order
/// and a Customer in this case
/// </summary>
/// <param name="updatedOrder">order to update</param>
/// <param name="oldCustomer">previous customer</param>
public static void ChangeOrder(
Order updatedOrder,
Customer oldCustomer)
{
using (NorthwindEntities context = new NorthwindEntities())
{
// Tell the context about the entities to persist
context.AttachTo("Orders", updatedOrder);

// Change the state of the new customer
// In this example, this is done by the convention:
// - Customer.CustomerID is null --> customer is 'Added'
// - Customer.CustomerID is not null --> it already exists
if(updatedOrder.Customer.CustomerID == "")
{
// When state of customer changes to 'Added', state
// of the relationship between updatedOrder and
// pdatedOrder.Customer is also changed to 'Added'
context.ChangeObjectState(
updatedOrder.Customer,
EntityState.Added);
}
else
{
if (updatedOrder.Customer.CustomerID !=
oldCustomer.CustomerID)
{
// only relationship needs to be marked as 'Added'
context.ChangeRelationshipState(
updatedOrder,
updatedOrder.Customer,
o => o.Customer,
EntityState.Added);

// Report removal of relationship between
// pdatedOrder and oldCustomer
                context.ChangeRelationshipState(
updatedOrder,
oldCustomer,
"Customer",
EntityState.Deleted);
             }
}

// Persist the changes to the store
context.SaveChanges();
}
}

As is always the case, your feedback on this topic is welcome!

Jeff Derstadt
Jaroslaw Kowalski
Diego Vega
and The Object Services Team

This post is part of the transparent design exercise in the Entity Framework Team. To understand how it works and how your feedback will be used please look at this post .