QUESTION: Can I use DefaultWorkflowSchedulerService instead of ManualWorkflowSchedulerService in my IIS hosted WorkflowRuntime?
ANSWER: Do so at your own risk. It's like swimming in an unguarded pool or skydiving ...
Using the DefaultWorkflowSchedulerService in an IIS host (specifically ASP.NET) is one of those good/bad scenarios depending on point of view. Windows Workflow Foundation (WF) has no qualms about using the DefaultWorkflowSchedulerService in an ASP.NET hosted scenario and everything will work just fine from a WF point of view. The problem is that ASP.NET does NOT like people using up all its threads.
Our ManualWorkflowSchedulerService was written specifically for the ASP.NET scenario to make sure that we were running on the minimal number of threads. The call to RunWorkflow is an explicit "gifting" of the current thread (the request thread in the IIS case) to run as much of a specific workflow instance as possible. When you introduce the DefaultWorkflowSchedulerService you are, by default, allowing as many as 4 threads to run workflow instances.
Is that a bad thing? In most cases, no. Don't quote me on these numbers, but .NET supports something like 25 threads per processor per Process. If you are doing anything with Transactions, then at least one of those threads needs to be "available" to avoid deadlock in some cases. The number of threads ASP.NET requires depends on the situation - if you are running a single web service in an isolated AppPool then I can't imagine ever running into an issue.
So, what's the "official word" - the statement that covers everyone's butt and will hopefully steer users to a design that is free from danger?
First, examine your scenario. Ask yourself why you need to be doing the processing in the background (non-request thread)? Is all processing tied to a request? Does the processing take very long? Is new processing ever driven by a "delay" in the workflow or some event not tied to an inbound request?
If you are doing a lot of processing then it is often better to model this as a windows service. Using Windows Communication Foundation (WCF) you can directly expose your windows service as a consumable web service. Alternately you can use WCF or .NET Remoting (last generation technology at this point) to communicate between your current web service and your windows service.
If you aren't doing much processing and it is all tied to the request, then consider using the ManualScheduler. While this will add additional latency to your responses, it can be considered a constant value as opposed to the unpredictable response times you'll see when you've got an unknown number of active workflow instances vying for processor time.
In general, try to avoid using the DefaultWorkflowSchedulerService in ASP.NET/IIS. If you do use it, use it with caution and expect that there may be a few unknown problems which pop up during stress. There is always an alternate design and in general the tradeoffs will be maintainability/complexity versus request/response speed versus stability under stress.