Some more stuff to remember when dealing with self-replicating tasks. (See my earlier post for an introduction to Parallel FX and self-replicating tasks):
- Self-replicating tasks should have an inter-replica communication mechanism for communicating the progress/details of the activity. This depends on what the activity is trying to achieve. See here for an example.
- Self-replicating tasks should have an inter-replica communication mechanism for communicating the completion of the overall activity.
- Only use when the cost of this communication and the management of partitions is considerably less than the potential benefit gained from parallelism
- Do not assume that the task is always replicated. It is only replicated if there are available resources. For the same reason also, do not assume that there will be a specific number of replicas.
- In some instances, the number of replicas could far exceed the number of cores in the machine.
- You may choose to use optimistic concurrency when it is possible to correctly deal with multiple executions of the same step.
- In general replicating tasks are an advances feature that can be very useful in specific scenarios. Use with caution.
(All based on the first Parallel Extensions CTP)