Messages vs. Documents in a Workflow System

  • Article

Using workflow systems can be interesting.  Everyone has a good idea of how to solve the "workflow problem," and it usually involves systems with various different basic features... however most are similar in that the problem doesn't allow for a lot of variety in how it is solved.

In fact, a rather nice reference (if you can tolerate the formality of the language), is a dissertation that compares the features of various commercial workflow systems from a standpoint of the which workflow patterns are supported by the system.  A summary and the link to the original document can be found at https://tmitwww.tm.tue.nl/research/patterns/products.htm

One thing that is missed in a discussion of patterns, though, is the distinction between workflow messages, workflow documents and target documents.  I find this troubling, because this is one of the key points that caused a rather embarassing project failure (for me personally), when I was attempting to create a workflow solution using Microsoft Biztalk Server.

Definition: Workflow Message

A workflow message is a relatively small data object that encapsulates the "command" pattern.  The object contains enough data to associate the command to an existing data item in another computer system, and then issues a command against that data item. 

For example: in Healthcare, if you visit a hospital, the institution will send a claim to your insurance company asking to be paid.  This transaction is called the HIPAA 837-I.  This could take anywhere from a few hours to a few months for the insurance company to respond with payment.  At any point along the way, the hospital can send a "request for status" to the insurance company, to find out what is going on.  This request is called a HIPAA 276.  The response is a HIPAA 277, and provides some information about what the status is. 

Both the 276 and the 277 are workflow messages.  They contain enough information to refer to both the healthcare claim and the patient.  However, the message itself is not the business data.  It never has to be maintained.  It exists at a single point in time, and represents a command (in this case: a query for status).  To summarize, a workflow message is:

  • comparatively small in size
  • contains information that relates the message to a business object
  • expresses information about a business object or encapsulates a command to be performed on that object
  • is immutable... it exists at a single point in time, and it does not change.

Definition: Workflow Document

A workflow document is a data object that contains enough information to trigger or drive a business process.  While it is similar to a message in that it usually refers to other data objects, the workflow document can be changed during the course of the business process.  In fact, it nearly always has to be maintained in some fashion during the course of the workflow, and is often kept as a record of activity after the workflow process has completed.  This document may have attachments placed in it, and the list of attachments may change during the course of the workflow process.  A workflow document is a construction of the workflow process itself.  If there was no workflow process, this document would not come into being.  The format of the workflow document is nearly always constrained by the workflow process, and is often expressed as a form with fixed fields.

For example: let's say that you work in a busy PR department, and one of the department's responsibilities is to generate press releases.  You will usually have a workflow process to generate the press release.  In our example, the press release is generated by one of the company divisions and sent to a PR coordinator in your group.  That coordinator will then behave like a reporter, calling the source contact and collecting information about related products and detail on the information to be released.  The coordinator may collect the original PR request, one or two supporting documents, and a spreadsheet containing a release schedule.  The coordinator will track the date that the release should go out, checkpoint dates when the business has an opportunity to change their minds, and signatures from various folks showing that they agree with the content.  All of this information may change periodically during the course of the process.  The coordinator may share this document with other members of his or her team, and even assign it to someone when they go on vacation.

In summary: a workflow document is:

  • A creation of the workflow process itself -- it does not exist seperately
  • It is used to capture the data needed by the process to enforce business rules
  • It may change many times during the course of the business process
  • It may need to be saved when the process is completed as a record
  • It may be transferred from person to person
  • It may have attachments that may or may not be directly related to the final product of the process
  • It has a specific format or set of fields.  It is often a form.

Definition: Target Document

A target document is a document in any format (spreadsheet, word processing, picture, diagram, whatever...) that is consumed by, created by, or managed as a result of the business process.  In our example of the PR department above, the final press release itself would be a target document.  So would the initial request for a press release if it existed in a different format than the form itself.  So would any supporting documents collected by the PR coordinator. 

The thing to understand about a target document is this: a target document doesn't care about the business rules that surround it.  It is not specifically aware of it's own status in the workflow process, who it is assigned to, and how long it has been there.   A target document is nearly always the "product" of a workflow process, but is nearly never aware of the process itself.  (The exception to this rule is when an "indicator" is placed on a document to indicate things like completness ("Draft" vs. "Final").

In summary, a target document

  • is of any format that makes sense to the business users
  • contains very few fields or indicators of its own status within the workflow process
  • contains little or no indication of who it is assigned to or for how long
  • is managed by a workflow process at some point during its lifetime
  • may exist before the workflow process begins.  May continue to exist, and be edited, modified, and managed, after the workflow process has completed.

Why create the distinctions?

So, I defined three terms and hopefully did so in a clear manner.  The terms are fairly clear, but why are they important?

Because some systems have support for messages, but not workflow documents.  Others have support for workflow documents but not messages, and still others are entirely focussed on the target document alone.  Very few have support for, and understanding of, all three types of documents.

This is what bit me when I attempted to create a document management workflow using Biztalk alone.  Biztalk is an excellent message processing system and EAI middleware product from Microsoft that can be used to coordinate collaborative processes both within an organization and between organizations.  Let me make it clear: I'm not dissing BTS.  (I have often used the handle 'biztalkbum' in the online forums).

On the other hand, Biztalk is not a document management system.  It has no concept of a Workflow Document and nearly no concept of a Target Document.  There is no central formatted repository of data that is maintained through the steps of the workflow, and which can contain the documents that are collected as artifacts along the way.  If you need that, you have to add it.  I needed it.  In fact, it was so critical to the success of my project, and would take so much time to add it to Biztalk, that it would have been easier to start with a system that had support for shared documents, and add a workflow engine to it.  (That's exactly the decision we made, by the way, but it cost us a month of delay as we tried to solve the problem with Biztalk).

So, by writing this down, I'm doing a couple of things:

  • I'm collecting my thoughts in one place, so that I can have a common term of reference as I go forward,
  • I'm sharing a set of terms that I hope others can use and adopt, so we can scrutinize and compare workflow solutions using criteria like "do you support the concept of workflow documents?",
  • I'm coming to terms with the mistake I made, and trying to understand the distinctions I need to make to avoid making it again, and
  • I'm sharing my experience with those of you in the outside world who stumble upon this article, to help you avoid this mistake.

More articles on collaboration, workflow, and representation of the same, all to come...