Is workflow best represented by something other than a graph?
A few days ago, I blogged about the three layers of abstraction for workflow items. Now, I'm going to think aloud about a theoretical problem... is workflow best represented by a single graph?
Is the analysis of the problem leading to another problem?
Most workflow processes are described using one of two types of visual paradigms: "flowchart" style graphs and petri-net graphs. Both are directed graphs that start in one place and follow to another. Every workflow that I've seen attempts to capture 100% of the workflow steps in a single diagram. Even those folks who attempted to be "sophisticated" would limit themselves to only two levels of abstraction (a high level process, and individual low-level processes that "plug in" to the high level process at specific places).
The problem I've been running into is that these abstractions are extraordinarily leaky. The graphs usually combine details from more than one layer of abstraction. Error handling is described in different terms depending on your location within the workflow, and the ability to modify the workflow in an ad-hoc manner is not covered at all. This is "directed workflow" and, honestly, doesn't model the interactions that people truly expect to perform in a business environment.
While analysts have no difficulty describing systems from the standpoint of a high-level graph, it does not appear to be a good implementation paradigm. As my former coworkers in the hardware space might say: there is impedance in the system.
Impedance
The concept of Impedance comes from Electronics, where it is a carefully defined measurement. In Computing, in recent years, this term has been used to describe the "resistance" between a relational database modeling system, and an object oriented middle tier in an application, requiring complicated logic in the data access layer to map data from one storage paradigm to another.
In my context, I am using this term to refer to the fact that humans follow processes very well, as long as they make room for human activities in a particular human manner. Our methods of creating a workflow diagram, on the other hand, do a poor job of actually capturing the requirements of human beings, working with individual capacities and on individual cases.
To overcome this impedance, we either create overly simplistic workflow designs or overly complex designs. The simplified designs simply show assignments and require that the individuals interacting in the workflow would simply "know" the steps that they need to follow when work has been assigned to them. This is perfectly human, but not terribly flexible from a business standpoint. The overly complex designs attempt to model every possible exception or flow. The problem is that people, in the act of using the workflow, will inevitably choose a path that was meant only as a very rare exception... leaving some work item stranded down a rabbit hole. Overcoming this requires extensive training... and once again, limits business flexibility.
Sometimes, our systems attempt to overcome the impedence for us. A few workflow systems attempt to hide this impedence under a layer of compiler transformations that seek to take the flowcharts and petri nets and convert them to executable models that can be interpreted at runtime.
I'm beginning to wonder if this is an effective approach. If the analysis method doesn't match the area of human activity, let's stop trying to force it to fit. Let's just change the analysis model.
Adapting the "multiple views" concept to Workflow
Just as software development requires multiple views in order to describe a design, I believe that workflow analysis should require multiple views in order to describe a process.
The static view
The static view would be a fairly simple Finite State Automata diagram, where work items remain in a state until an event occurs to move it to another state. This view type is well understood, so I won't describe the idea in detail here. My only addition would be that one or more diagrams may appear at EACH level of abstraction, and that an analyst should limit themselves to describing only a single level of abstraction in each diagram.
The dynamic view
Another view would be to approach the model as an activity stack. Essentially a "porous" LIFO stack, a workflow would start with a set of activities, a series of evaluation expressions, and a collection of variables in their initial states.
The each activity would define behavior for pushing messages out. After initialization is complete, the each activity is evaluated, and any outbound messages are sent to a partner (usually an application, a managed component, or a transmitted form like Infopath).
The system then waits for zero or more messages to arrive. (Note: the object may subscribe to one or more time-based events).
When an event arrives, predefined behavior will dictate that the event will either perform an operation or modify one of the state variables. (If it performs an operation, the operation may indirectly modify one of the state variables). When the event is complete, each of the conditions on the stack is evaluated. All conditions that are true are dropped from the stack. (This is the porous part... all items are evaluated. Therefore, on a seven element stack, items 1, 4, 5, and 7 may all be satisfied by a single event, leaving only items 2, 3, and 6 remaining on the stack).
The top activity would be re-evaluated. Any outbound messages will be transmitted, and the system returns to waiting for an event.
This is not altogether dissimilar from petri nets. The difference is that Petri nets use mathematical weights to determine paths and outcomes, making them inherently difficult for a business analyst to construct. Difficulty of use dramatically reduces the effectiveness of a solution (Assembly language is fast, but very few programmers use it... same reason).
The conditions on the stack would represent various patterns of interaction. These exist at the pattern level (what I call the "business process" level of abstraction). The interaction conditions would include things like various combinations of approvals, weighted voting, time and termination conditions, etc. Dr. Van der Aalst produced an excellent paper on workflow patterns that could serve as an initial inventory of primitives for this stack object.
The collection of state variables must be queryable at all times from the outside. This is a major deficiency of some workflow systems (that will go nameless)... that the data that is used to derive decisions or track the status of an item is not visible by a program that wishes to find the item on the basis of that information.
For example: if a workflow item manages the process of approving an invoice, and the invoice has some fields like quantity, supplier, requestor, cost center, total value, and even business priority, then shouldn't these fields be visible to anyone who wishes to query the list of workflow items to find, sort, and report on the items in the queue?
That's a lot of work... why do it?
One advantage of looking at workflow as a stack: it is fairly easy to move a workflow item from one process to another. The constructor for a workflow would define the necessity for a set of variables and initial conditions. If a variable already exists, no need to create it again. The existing stack is completely replaced with the new one, or stacked on top, and the process begins as described above... in many circumstances, there may not need to be any data conversion or gyrations at all.
Another advantage: ad hoc events could ADD items to the stack. This would allow someone to add a primitive that could seek special case approvals, for example, or interact with a system that is only used occasionally. New variables could be added as well. This would allow a workflow item to truly change as it moves, yet remain deterministic.
Of course, when a condition is "removed" from the stack, it isn't really "removed" but placed into a bin of conditions. The events are also stored in the bin. That way, if compensation is needed to roll back the workflow, you can walk the bin, finding the events in reverse order and executing the compensation on each event. Also, the bin can be used for audit tracking and generic time-based reporting.
Conclusion
I'm not quite sure what diagramitics to use to model a dynamic view, or even if this view is a useful description from the design standpoint. I can say, however, that I would have found it useful if this kind of concept were built-in to workflow systems, because I could use it to reduce the amount of code I'd have to write in order to implement a solution.