Managed ThreadPool vs Win32 ThreadPool (pre-Vista)
The following is a conversation between me and a CLR dev. The conversation is very informative so I quote it here.
From:
Sent:
To:
Subject: RE: ThreadPool.QueueUserWorkItem
There might be some confusion here around the meaning of the term "I/O Thread." In the Windows thread pool (the old one, not the new Vista thread pool), an "I/O thread" is one that processes APCs queued by other threads, or by I/O initiated from the I/O threads. The "non-I/O" threads get their work from a completion port, either as a result of QueueUserWorkItem, or I/O initiated on a handle bound to the threadpool with BinIoCompletionCallback. So they are both geared toward processing I/O completions, but they just use different mechanisms.
In the managed ThreadPool, we use the terms "worker thread" and "I/O thread." In our case, an I/O thread is one that waits on a completion port; i.e., it's exactly equivalent to Windows' non-I/O thread. How confusing! Our "worker threads" wait on a simple user-space work queue, and never enter an alertable state (unless user code does so), and so explicitly do not process APCs. Managed "worker threads" have no equivalent in the Windows thread pool, just as Windows "I/O threads" have no managed equivalent.
The managed QueueUserWorkItem queues work to the "worker threads" only. UnsafeQueueNativeOverlapped queues to the I/O threads, as do completions on handles that have been bound to the ThreadPool via BindHandle.
Why don't we support APCs as a completion mechanism? APCs are really not a good general-purpose completion mechanism for user code. Managing the reentrancy introduced by APCs is nearly impossible; any time you block on a lock, for example, some arbitrary I/O completion might take over your thread. And they don't scale well, except in certain very constrained scenarios, because there's no load-balancing of completions across threads. You can, of course, implement your own load balancing, but you'll never do better in user-space than the kernel does with completion ports. So we provide a rich async I/O infrastructure based on completion ports, and nothing else.
From:
Sent:
To:
Subject: RE: ThreadPool.QueueUserWorkItem
With ThreadPool.QueueUserWorkItem, the callback is not called on an I/O thread. You might be looking for ThreadPool.UnsafeQueueNativeOverlapped.
From: Junfeng Zhang
Sent:
To:
Subject: ThreadPool.QueueUserWorkItem
The native kernel32 QueueuserWorkItem has a flag to indicate to schedule the callback in I/O thread or not. The flag is missing in the managed ThreadPool.QueueUserWorkItem.
Why is so? Is it because the callback is always scheduled on a I/O thread?