SimpleScript Part Six: Threading Technicalities

Refresher Course

Before you read this, you might want to take a quick refresher on my original posting on the script engine threading model.  That was a somewhat simplified (!) description of the actual script engine contract.  Let me just sum up:

  • free threaded objects can be called from any thread at any time; the object is responsible for synchronizing access to shared resources
  • apartment threaded objects can have multiple instances on multiple threads but once an object is on a thread, it can only be called from that thread.  The caller is responsible for always calling an object on the thread on which it was created.  The object is responsible for synchronizing access to resources shared across instances.
  • rental threaded objects can be called on any thread, but the caller is responsible for ensuring that the object is only called from one thread at a time.
  • an initialized engine is apartment threaded, with a couple exceptions — InterruptScriptThread and Clone can both be called from any thread on an initialized engine.
  • an uninitialized engine is free-threaded

Things Get More Complicated

That’s actually not quite right.  It would be more accurate to say that an uninitialized engine is rental threaded.  Why?  Because otherwise it would be legal to do really dumb things like call Close on two different threads at the same time.  If you look carefully at the code, you’ll see that most of the methods are not robust in the face of full-on multithreading.  It’s the “the host isn’t a bozo” threading model!

This is a ridiculously complex threading contract, I know.  From COM’s point of view, the script engine is free threaded — the restrictions are so arcane that clearly the engine has got to be the one enforcing the rules, not COM, which is why you’ll see lots of calls to check that the caller is on the right thread in my code.

If you take a look at the registration code, you’ll see that I register the script engine as “Both”, which means “either free threaded or apartment threaded, we’ll sort it out”.  What the heck is up with that?  Why not just call it “free threaded” and be done with it?

Because again, I oversimplified the description of an apartment in my original posting.  A single-threaded-apartment (STA) object is created on a thread and is always called on that thread.  Think of a person (an object) in a room (a thread) — you want to talk to them (call a method), you go to their room.  You can put as many people in a room as you’d like, and build as many rooms as you’d like, but you want to call a method, you do it from the thread where the callee lives. That’s all fine and good.

But there is also a multi-threaded apartment!  Imagine that you take some of the rooms and you knock holes in the walls.  If you’re in one room and you want to talk to someone in another room, you don’t have to go there, you can just yell through the hole.  The guy listening to you is responsible for synchronizing all the shouting going on, but at least he knows that no one is going to be trying to talk through a wall with no hole in it. 

In any given process there is one “main” STA, possibly many more secondary STAs, and one MTA.  You can’t have two distinct sets of rooms that mutually communicate but don’t talk to each other.

I briefly described in an earlier post the way that COM marshals calls “through the walls” from one apartment to another.  Well, if we register the script engine as a free threaded object, COM is going to think that it lives in the default MTA, and that any STA objects created by the process live in their own STAs, and therefore, all calls between the two apartments are going to have to be marshaled by the Free Threaded Marshaler.  That extra indirection really screws up your performance numbers, lemme tell ya.  We register as “both” threaded, and COM says “OK, you sort it out then if you’re so smart“, which we do by requiring that the host call us on the right thread and give us objects that are on the right thread. 

Even that is still a considerable oversimplification — there’s still the Neutral Threaded Apartment that I haven’t talked about yet, and the interactions between the CLR threading model and the COM threading model, and how marshaling really works, but that’s getting out of my depth.  Go ask Christopher Brumme if you’ve got questions about that stuff.

Honouring Our End Of The Script Engine Contract

Let’s take a look at the script engine contract through the example of one of its objects that we’ve already implemented — the named item list.  There are only four operations that can be performed on the named item list, and we know when they can be performed according to our contract and implementation:

  • Add can only be called on an initialized engine, and hence only from the engine threadAdd writes to the named item list.
  • Reset can only be called on an initialized engine as it is being moved to uninitialized state, hence only from the engine thread.  Reset writes to the named item list by removing non-persisting entries.
  • Clear is only called when the engine is going to closed state.  The engine might already be in uninitialized state and hence, this can be called on any thread if the engine is uninitialized.  However if the engine is initialized then it can only be called from the main engine threadClear writes to the named item list by removing all entries.
  • Clone can be called on any thread, and reads from the named item list.

What then are the possible threading conflicts?  The three writers, Add, Reset and Clear cannot be called at the same time on different threads by virtue of the fact that the engine must be initialized if Add or Reset are being called.  There are a few cases which for completeness we should get right, but are in reality extremely unlikely.  Why would any host be so dumb as to Clone an engine while in the middle of a call to Add?  Or worse, Clone during Close?  I won’t assume that hosts won’t pull shens like that, even though they are very unlikely.

What about two Clones at the same time on two different threads?  On the one hand, they’re only reading, so why should they block?  On the other hand, boy, do I ever not want to implement single-writer-multi-reader mutexes just to make that extremely unlikely case marginally faster. 

Therefore, we’ll do it the easy way.  The only thing we really need to worry about practically is one thread doing a Clone while another thread is doing a Reset, but we’ll get it right for all the cases.  The first thing we’ll do when we enter any of those methods is enter a critical section, and the last thing we’ll do before we leave is exit it.  Rather than mess around with the operating system’s somewhat gross critical section code in the object itself, I define a handy object to wrap it.  See mutex.cpp for the implementation.

Something to note about this implementation is that it uses InitializeCriticalSectionAndSpinCount to initialize the critical section.  The comments there and here are required reading if you need to make critical sections work on heavily loaded pre-Windows-XP boxes. Earlier versions throw exceptions rather than returning error codes, which means that every entry and exit to a critical section has to be protected with __try blocks and you then have to get the exception handling right!  The VBScript and JScript engines have all kinds of totally gross code in them to handle the edge case where a heavily loaded server runs out of memory just as a critical section is about to be entered.  (Yes, it happens. Every single out-of-memory case will eventually be exercised by a sufficiently loaded server, I know this from painful experience.) 

I’m going to skip all that totally gross code here and assume that we all live in the happy world of Windows XP, where the operating system actually returns sensible errors.

I’m still trying to sort out how all this is going to work once code blocks are throw into the mix.  I’ll try out a few things and see how it goes.  More bulletins as events warrant.

Comments (19)

  1. Nicholas Allen says:

    I’m trying to plan out how you’re going to support the thread model for InterruptScriptThread and Clone. Let me know if you’re looking at a different direction.

    The basic problem is that InterruptScriptThread and Clone can be called from any thread at any time on an initialized engine. These unruly calls look like the hardest part of the threading model to provide.

    For InterruptScriptThread it doesn’t seem that bad, maybe. Calling InterruptScriptThread trips a latch and then returns successfully. Scripts periodically pause and run a callback to the engine after chunks of work. One of the things this callback does is check the latch, and if it was tripped the script is killed. And, stopping a script (interrupt or normal termination) resets the latch.

    Does InterruptScriptThread guarantee that the script stops before the function returns or just at some hopefully near point in the future? If it has to stop right away, I guess after tripping the latch, InterruptScriptThread could wait for a signal from the callback that the script has stopped before returning.

    Clone is a little harder because it has to have the return result right away. I’m going to guess it uses the second approach where Clone sleeps until the next callback and then copies the state. This looks safe because from outside the engine, it’s impossible to distinguish that delay from the script already being at the callback and Clone just taking a while. The assumption here is that there’s no practical way to Clone while a script is actually executing. That is just madness, right, since the script could mutate some of the engine state while you’re copying?

  2. Nicholas Allen says:

    > boy, do I ever not want to implement single-writer-multi-reader mutexes

    Is that really so bad? A basic one is just a semaphore if we cheat and only allow a bounded number of simultaneous readers, right? And a writer preference read-write lock is just a mutex+semaphore. Or did you mean a robust, extremely scalable, high performance implementation in which case it probably would suck to do given the win32 sync primitives.

    I’d be surprised if they didn’t turn out to be useful in at least a couple more places.

  3. Eric Lippert says:

    Re: InterruptScriptThread

    You’re on the right track. I’ll cover all this stuff in a whole lot more detail when we get there, but basically ISC sets a threadsafe "halt soon" flag, VBScript and JScript check the flag at the beginning of every statement.

  4. Eric Lippert says:

    Re: Clone — you need to be able to clone a running engine. The trick there will be the fact that once the engine has compiled state, that state is read-only, and hence can be easily shared amongst engines. I’ll have to build a mutex into the code block list similar to the named item list.

  5. Eric Lippert says:

    Re: SWMR mutexes — yeah, it’s not that hard, but I’m doing this in my spare time, and the straightforward critsec code is easy.

  6. Mike Dimmick says:

    I thought that the ‘Both’ threading model meant that if created from a STA-model thread (I can’t remember the correct terminology – it called CoInitialize, OleInitialize or CoInitializeEx with the COINIT_APARTMENTTHREADED flag) the object ends up in the STA, whereas if created from an MTA-model thread (called CoInitializeEx with COINIT_MULTITHREADED) it ends up in the MTA.

  7. Eric Lippert says:

    Correct — like I said, I was simplifying my description of the model. The important point is that both-threaded objects do not automatically aggregate the free threaded marshaler.

  8. Nicholas Allen says:

    Does "beginning of every statement" mean once per language statement? So if I call in JScript:


    and a happens to have 100M elements I have to end task the engine? How about if I do:

    a.sort(function f(x,y){return x-y;});

    Does each invocation of f count as a statement so that interrupt works as expected?

  9. Nicholas Allen says:

    I think I may have the wrong impression of what Clone does in a running engine. Does it:

    a) Copy the engine state such as code and items but not the executing scripts. The new engine has no activity initially after Clone.

    b) Copy the engine state and the executing scripts. The new engine is off and running at the exact same instruction point as before the Clone.

    I had in my mind that it was b) which doesn’t seem to support the implementation you describe. Specifically, if I’m doing an eval and want a b) type Clone I need to bring along the compiled eval block so the state isn’t really read-only. If I’m doing an eval and want a a) type Clone, well it’s no problem since the eval code is an anonymous block which doesn’t need to be copied. Closures would probably have the same problem.

    Or is a b) type Clone really possible by being careful enough about when eval can happen?

  10. Eric Lippert says:

    By "beginning of every statement" I mean just that. Before we start a statement, we check to see if the halt flag is set. In your second example, since "return" is a statement, we’ll check once every comparison.

    In your first example, you’re out of luck. You call a method that takes half an hour to come back, you don’t get to halt the script for half an hour.

    Of course, an array would never have 100 million entries in JScript — that would take up more memory than the entire process space, and forget sorting — _garbage collection_ would take forever.

    If you have a hundred million entries to sort, I recommend using a rather more buff tool, like SQL Server.

  11. Eric Lippert says:

    Re: what does clone do:

    Clone does (a). Clone is motivated by Active Server Pages. You have an engine compiled and running which, when executed, serves up foo.asp. A second request for foo.asp comes in on another thread while the first one is still running. Without clone you have two choices:

    1) wait for the first one to finish, uninitialize the engine, reinitialize it on the new thread — thereby serializing access to that page.

    2) create a new engine on the new thread and recompile the state, which is potentially expensive.

    Clone mitigates both these problems. A cloned engine doesn’t have the cost of compiling the same program all over again and it can run at the same time as the clonee on another thread.

    Read for more details.

  12. Nicholas Allen says:

    True, 100M is probably out of reach for 32 bit code. But even 100K is going to take a very long time to complete. If I’m browsing the web and some malicious page decides to chew my CPU, I don’t get to choose the tool.

    As for GC taking forever, I’m guessing that’s because you’ve got a stop the world, mark and sweep collector in mind. Which is an adequate tool for the job but similarly not industrial strength.

    re: Clone. Thanks, that removes a lot of the hairy problems I had dreamed up.

  13. Eric Lippert says:

    Re: DOS: Correct — though we have some defences in place against simple denial-of-service attacks, it is possible for a hostile web page to chew up a whole lot of CPU and memory, and make IE look hung.

    Re: GC: Correct, JScript uses a stop-the-world, mark-n-sweep collector and runs it every time 256 variants, 4096 array slots or 64KB of strings are allocated, so allocating large chunks of memory becomes an order n-squared operation as n becomes large. JScript was designed for simple scripts on simple pages, not for implementing databases.

    See for details.

    I haven’t decided what kind of GC to put into SimpleScript yet.

  14. Eric Lippert says:

    > Does InterruptScriptThread guarantee that the script stops before the function returns or just at some hopefully near point in the future?

    InterruptScriptThread guarantees nothing as far as timing goes. For example, what if IST is called while the engine is executing the LAST statement? Then we never check the halt flag and never return SCRIPT_E_ABORTED back to the host. We do however put the engine into "halted mode".

    We’ll get into all this in way more detail later, I promise, so hold your horses for now. There are all kinds of questions that we’ll have to consider: what happens when you attempt to halt an already halted engine? What if you attempt to call ParseScriptText on a halted engine? How do you reset an engine out of halted mode? Etc, etc, etc.

  15. Raymond makes other people discuss stuff so he doesn’t have to.