Why is processor affinity inherited by child processes?


Consider why a typical program launches child processes. (Shell programs like Explorer aren't typical.) It's because the task at hand is being broken down into sub-tasks which for whatever reason has been placed into a child process. An Example of this would be, say, a multi-pass compiler/linker, where each pass is implemented as a separate process in a pipeline.

Now consider why you might want to set a process's affinity mask to restrict it to a single processor. One reason is that the program may have bugs that cause it to crash or behave erratically on multi-processor machines. This was common for older programs that were written for uni-processor versions of Windows or when multi-processor machines were still prohibitively expensive. In this case, you would launch the program in a suspended state, by passing the CREATE_SUSPENDED flag to the CreateProcess function, then set the processor affinity mask for that process to a single processor, then resume the process's main thread.

But what if the problem was in a child process of the process you're launching? Since you don't have control over how the process launches its child, you have no way to sneak in and set the child process's processor affinity mask. That's why the processor affinity mask is inherited: If you set it on the parent process, this covers all the child helper processes that process may launch as part of its execution.

Another reason why you might want to set a process's affinity mask is to restrict its CPU usage. (For example, you might restrict a CPU-intensive application to a single processor of your dual-processor machine.) And again, if the process launches child processes, you want those child processes to be subject to the same restriction as their parent so that the task as a whole remains restricted to a single processor.

That's why processor affinity is inherited by child processes. Because it's nearly always what you want.

Comments (23)
  1. Adrian says:

    Where can I read more about processor affinity? I’ve gotten bits and pieces from the SDK docs and articles like yours, but I still have fundamental questions about the topic.

  2. Dave says:

    Google for SetProcessAffinityMask.

    That API requires the PROCESS_SET_INFORMATION access right. Is that granted to non-admin users (e.g., "limited users" on Windows XP)? I know that everyone runs as admin but I thought I’d ask.

  3. Jim says:

    How do I set the mask for an MFC app?

  4. Peter Hornby says:

    When I was developing a resource manager for large multi-processor machines, we quickly ran into your "Shell programs aren’t typical" special case.

    To be specific, if you have some rule-based system which arranges for a non-default affinity mask for explorer.exe, cmd.exe or services.exe, you’ll find pretty much every process on the system running with an inherited, non-default affinity mask, which can be seriously counter-intuitive for the unwary user.

  5. Daev says:

    Although the Unix fork/exec model is one of the oddest and most counter-intuitive ways to create a new process, it has one small advantage: you can fiddle with the execution environment of your new process before it begins.

    Since the portion of the code after the ‘fork’ and before the ‘exec’ actually executes in the new process, the programmer can make any initial process-context changes ‘from the inside’ before the child executable takes over. Typically this is spent mucking around with signals and other Unix-specific things, but you could also set processor affinity, standard handles, and the like.

    In Win32 terms, this would mean supplying a function that runs during CreateProcess, in the new process’s context. Because no such thing exists, CreateProcess has become laden down with lots of extra parameters, structures, and flags for customizing process creation.

    What would be the consequences of taking a page from the Unix model? Yes, I know, the answer to "Why doesn’t it work that way?" is "what would happen if it did?" Are there security-descriptor gotchas lurking here?

  6. Vladimir says:

    Is there any way to set processor affinity using .manifest file or something like that.

    So i can do this for my application without changing application itself.

    Looks like one of the apps is having problem running in true multithreaded environment, and while vendor is trying to fix that, i have to go to Task Manager and set affinity for this app manualy.

    What i’m looking for is a configuration way of setting affinity

    Thanks.

  7. Daev says:

    Implementing fork() would probably require redesigning Win32 from the bottom up to be more like Unix. Same goes for signals. (Probably the converse would be something like adding Win32 kernel objects to a Unix.)

    You could do a bustling business selling bait from all the cans of worms that idea opens…

    The ability to customize process startup (which fork/exec gives you) could be gained by having a function like those used in DLL injection or API hooking, which runs in the address space of the new process before CreateProcess returns.

  8. Bryan says:

    Plus, something like fork() would make porting some Unix tools *much*, *much* easier… ;-)

  9. oldnewthing says:

    Solve a problem by introducing a bigger problem. Your callback function is in the wrong address space.

  10. Bryan says:

    It’d be called back in the child process address space, though. Before the child process starts executing.

    It might still require access to some of the data in the parent’s address space, true, but that’s where copy-on-write pages come in handy.

  11. Bryan says:

    Or am I missing what you meant?

  12. oldnewthing says:

    Ah, so you’re really asking for the fork() model, not just a dinky callback function. Now things get interesting. Will DLLs be able to handle the return value from GetCurrentProcessId() changing after the fork()? What happens to USER and GDI and COM objects – are they forked too? I don’t know the answer – I’m just pointing out that your answer creates far more questions…

  13. Bryan says:

    Well… It is the fork model in kernel space, yes. The API could be done with just a "dinky callback".

    As for the DLLs calling GetCurrentProcessId… hmm. Presumably it’ll only matter for function calls made into the DLLs (threads are not copied into the new process, so it’s not like a single thread will suddenly have a different PID). As for how many of those exist… probably at least one, which means it isn’t going to change. Rats…

    (The same issue exists on Linux — if you write a shared lib, the return from getpid() can change at any time. But nothing relies on it not changing, because it’s been that way from pretty much the beginning.)

    I’m not sure about the kernel representation of USER and GDI objects (if there is one), but if they’re memory-only, then no big deal. I’m pretty sure in-process COM objects are memory-only. If that’s true, then the callback would have access to a "copy" (copy-on-write, that is) of them.

    But out-of-process COM objects start to get more interesting. The proxy could be duplicated easily enough, but then the stub in the server would have to be also, and that’d be quite difficult.

    More questions is right. ;-)

  14. oldnewthing says:

    (Of course, if you don’t copy threads, then entering a critical section might hang because you’re waiting for a thread that doesn’t exist… Copying the kernel object behind a window is the easy part; the hard part is resolving thread affinity, what happens if somebody sends a message to that window – does it go to the old window or the new one? If you give the new window a new handle, what happens to all the code that caches window handles?)

  15. Bryan says:

    Err… yikes.

    X does "everything" over a socket, and the socket is shared between processes (so both would get the message). But if you do the fork() and exec() within one Xlib callback, the child will never get to Xlib’s select() call, so it’ll never get the message. If not, however… that’s probably why a program I wrote a while ago failed miserably every once in a while. ;-)

    The critical sections are hairy too — if a thread other than the one calling CreateProcess held the critical section, then no big deal (because they’re shared between processes). Both processes would have a handle to the same kernel object, and when the thread (in the other process) let go of it, one of the two processes would be able to get it. (Until the callback returns, anyway. Then it’s gone out of the child.)

    But if the thread holding the section calls CreateProcess, then which process gets control of it? Both is clearly wrong. Choosing one or the other would break some code, I’m sure.

    OK, never mind. I’d be a nice way to set the affinity of a child process (or change file handles), but there are obviously a bunch of things I haven’t thought of. :-)

  16. oldnewthing says:

    (Actually the critical section isn’t shared between processes. Or at least not completely. Critical sections are a mix of user-mode and kernel-mode. You’re sharing the kernel half but cloning the user-mode half…)

  17. Bryan says:

    Basically, adding a callback function parameter to the CreateProcess API function (maybe rename it CreateProcessEx, for compatibility) could allow this. It shouldn’t even be all that difficult.

    Signals, yes, that would probably be a huge redesign if they were done in the kernel. But Cygwin has support for at least SIGINT and SIGTERM (but not SIGHUP), so doing some of them in userspace is possible. (I’m not sure how correct it is, though.)

    "Adding Win32 kernel objects to a Unix" — what do you mean by kernel objects? Stuff like SysV semaphores, shared memory, and message queues? (all these can be shared between processes) That’s already been done. ;-) Event objects, no, but I *think* you can emulate them with semaphores.

  18. Bryan says:

    Oops, I’m remembering wrong.

    I thought critical sections could be named (and were therefore cross-process). Turns out they can’t (and aren’t).

    As far as cloning only one half goes — it sounds like the page holding the kernel’s side of the critical section isn’t associated with a process then? If it was (and got set to copy-on-write), would that help? I’d think so, but since I know nothing about the internals, I don’t know for sure. Maybe it’s impossible given the way other stuf is set up, though.

  19. oldnewthing says:

    But how does your *code* get into the child process? And are you suggesting mapping the parent processes’ data into the child process too? Wouldn’t that interfere with the child process’s address space (and be a security hole)? Do you intend to call kernel32 in the child process before kernel32 has been initialized?

  20. Bryan says:

    Well, the model I have is the way Linux does it.

    Basically, after you call fork(), you get two processes, both executing from immediately after the call to fork(). Both still have all libraries loaded and initialized. But both are mapped into memory using copy-on-write pages, so that any changes made by either of them are contained to the process that made the change.

    This is the point where the callback would be invoked. In effect, memory reads would come from the parent, but writes would only affect the child. (Subsequent reads of a previously-written address would return the value that was previously written. Changing the value at an address from the parent or child after the fork does not mean that the value in the other process would change, because the page would get reallocated somewhere else when the process did the write.)

    The kernel (or whoever) would call the callback function before clearing or overwriting any pages (which is code originally from the parent, or one of its DLLs, yes, but it’s still in memory because the contents of pages haven’t been changed). The callback can call any functions that the parent could call (including stuff in kernel32), because the address space is still the same; kernel32 doesn’t need to be initialized, because it already was when the parent was started.

    In Linux, when you call exec(), *that’s* when your process is cleared from memory and overwritten by the new process. (That’s also when most of the new executable’s pages get allocated, because that’s when most of them are usually written to.) In this callback model, that would happen right after the callback returned. That’s when kernel32 would be loaded into the new executable’s space (and presumably re-initialized).

    The callback can’t interfere with the child process’s address space, because after the callback returns, the entire address space gets overwritten with the new EXE and DLLs anyway. So any changes made by the callback (to memory) would be lost at that point.

    The point, though, is that changes to things like file descriptors persist. (Anything that isn’t memory.)

    Also, the callback wouldn’t necessarily have to ever return — it could call exit() (or ExitProcess()), which means that the new EXE and DLLs would never be loaded. It’d be a way to do "threading" without having to worry about protecting shared memory — because any writes are contained to the process that made them.

  21. Daev says:

    I was the one who just wanted a dinky callback.

    I agree that fork() is utterly insane; it has to be designed in from the ground up. (Just read MGrier’s account of how the NT loader works with DLLs, and the very thought of adding fork will terrify you. http://blogs.msdn.com/mgrier/default.aspx )

    But Jeff Richter talks extensively about running your code in someone else’s process in his chapter on DLL injection, so I assume the kernel is prepared to handle the memory context issues. It does look increasingly complex the harder you look at it, though.

  22. Daev says:

    Since we’re talking about it, I’ve seen one problem where fork() is the only possible solution. Since it relates directly to something Raymond has said several times about exceptions versus explicit handling of error codes, I’ll describe it here.

    Suppose you have a big, complex number-crunching program, and you want to be able to abort a computation in it. (Perhaps your program is a worker thread that receives shutdown orders, or perhaps you want to impose a time constraint on a particular algorithm.) Your design doesn’t use exceptions, but simulates their effect by constantly checking a global flag and returning when it’s time to abort.

    So:

    (1) Every function that takes any significant amount time needs to call should_I_quit() regularly, and if it returns TRUE then the function immediately returns.

    (2) Just as importantly, any place in the code that calls one of the functions in (1) needs to call should_I_quit() soon afterwards, and if it’s TRUE then we again immediately return. (And so forth, up the call stack.)

    If anyone drops the ball anywhere, then you have a situation where the abort fails. How do you test for these bugs?

    Here’s how it’s tested on Unix systems with fork. (I think it’s called "fault injection testing.") Create an abort-debugging build of the program with two special properties:

    (1) All successful aborts cause the program to exit.

    (2) The should_I_quit() function is changed to use fork. Fork clones the whole process, and in the child process should_I_quit() returns TRUE, which should lead to the process exiting. Meanwhile, the parent process waits for the child to exit — and if after a few seconds it’s still alive, we can report an abortability bug. Otherwise, the parent process returns FALSE from should_I_quit() and everything proceeds normally onward to the next potential failure point, the next should_I_quit call, where the whole ordeal repeats itself.

    Only by forking the whole process — that is, only by the process returning & cleaning up & exiting AND simultaneously continuing on as though nothing had happened — can we mechanically test all possible failure points. Using threads won’t cut it.

    I’ve thought about it for years and never come up with a Windows equivalent. Any ideas?

Comments are closed.