How do I get mouse messages faster than WM_MOUSEMOVE?


We saw some time ago that the rate at which you receive WM_MOUSE­MOVE messages is entirely up to how fast your program calls Get­Message. But what if your program is calling Get­Message as fast as it can, and it's still not fast enough?

You can use the Get­Mouse­Move­Points­Ex function to ask the window manager, "Hey, can you tell me about the mouse messages I missed?" I can think of two cases where you might want to do this:

  • You are a program like Paint, where the user is drawing with the mouse and you want to capture every nuance of the mouse motion.

  • You are a program that supports something like mouse gestures, so you want the full mouse curve information so you can do your gesture recognition on it.

Here's a program that I wrote for a relative of mine who is a radiologist. One part of his job consists of sitting in a dark room studying medical images. He has to use his years of medical training to identify the tumor (if there is one), and then determine what percentage of the organ is afflicted. To use this program, run it and position the circle so that it matches the location and size of the organ under study. Once you have the circle positioned properly, use the mouse to draw an outline of the tumor. When you let go of the mouse, the title bar will tell you the size of the tumor relative to the entire organ.

(Oh great, now I'm telling people to practice medicine without a license.)

First, we'll do a version of the program that just calls Get­Message as fast as it can. Start with the new scratch program and make the following changes:

class RootWindow : public Window
{
public:
 virtual LPCTSTR ClassName() { return TEXT("Scratch"); }
 static RootWindow *Create();
protected:
 LRESULT HandleMessage(UINT uMsg, WPARAM wParam, LPARAM lParam);
 void PaintContent(PAINTSTRUCT *pps);
 BOOL WinRegisterClass(WNDCLASS *pwc);

private:
 RootWindow();
 ~RootWindow();
 void OnCreate();
 void UpdateTitle();
 void OnSizeChanged(int cx, int cy);
 void AlwaysAddPoint(POINT pt);
 void AddPoint(POINT pt);
 void OnMouseMove(LPARAM lParam);
 void OnButtonDown(LPARAM lParam);
 void OnButtonUp(LPARAM lParam);

 // arbitrary limit (this is just a demo!)
 static const int cptMax = 1000;
private:
 POINT  m_ptCenter;
 int    m_radius;
 BOOL   m_fDrawing;
 HPEN   m_hpenInside;
 HPEN   m_hpenDot;
 POINT  m_ptLast;
 int    m_cpt;
 POINT  m_rgpt[cptMax];
};

RootWindow::RootWindow()
 : m_fDrawing(FALSE)
 , m_hpenInside(CreatePen(PS_INSIDEFRAME, 3,
                                  GetSysColor(COLOR_WINDOWTEXT)))
 , m_hpenDot(CreatePen(PS_DOT, 1, GetSysColor(COLOR_WINDOWTEXT)))
{
}

RootWindow::~RootWindow()
{
 if (m_hpenInside) DeleteObject(m_hpenInside);
 if (m_hpenDot) DeleteObject(m_hpenDot);
}

BOOL RootWindow::WinRegisterClass(WNDCLASS *pwc)
{
 pwc->style |= CS_VREDRAW | CS_HREDRAW;
 return __super::WinRegisterClass(pwc);
}

void RootWindow::OnCreate()
{
 SetLayeredWindowAttributes(m_hwnd, 0, 0xA0, LWA_ALPHA);
}

void RootWindow::UpdateTitle()
{
 TCHAR szBuf[256];

 // Compute the area of the circle using a surprisingly good
 // rational approximation to pi.
 int circleArea = m_radius * m_radius * 355 / 113;

 // Compute the area of the region, if we have one
 if (m_cpt > 0 && !m_fDrawing) {
  int polyArea = 0;
  for (int i = 1; i < m_cpt; i++) {
   polyArea += m_rgpt[i-1].x * m_rgpt[i  ].y -
               m_rgpt[i  ].x * m_rgpt[i-1].y;
  }
  if (polyArea < 0) polyArea = -polyArea; // ignore orientation
  polyArea /= 2;
  wnsprintf(szBuf, 256,
           TEXT("circle area is %d, poly area is %d = %d%%"),
           circleArea, polyArea,
           MulDiv(polyArea, 100, circleArea));
 } else {
  wnsprintf(szBuf, 256, TEXT("circle area is %d"), circleArea);
 }
 SetWindowText(m_hwnd, szBuf);
}

void RootWindow::OnSizeChanged(int cx, int cy)
{
 m_ptCenter.x = cx / 2;
 m_ptCenter.y = cy / 2;
 m_radius = min(m_ptCenter.x, m_ptCenter.y) - 6;
 if (m_radius < 0) m_radius = 0;
 UpdateTitle();
}

void RootWindow::PaintContent(PAINTSTRUCT *pps)
{
 HBRUSH hbrPrev = SelectBrush(pps->hdc,
                                    GetStockBrush(HOLLOW_BRUSH));
 HPEN hpenPrev = SelectPen(pps->hdc, m_hpenInside);
 Ellipse(pps->hdc, m_ptCenter.x - m_radius,
                   m_ptCenter.y - m_radius,
                   m_ptCenter.x + m_radius,
                   m_ptCenter.y + m_radius);
 SelectPen(pps->hdc, m_hpenDot);
 Polyline(pps->hdc, m_rgpt, m_cpt);
 SelectPen(pps->hdc, hpenPrev);
 SelectBrush(pps->hdc, hbrPrev);
}

void RootWindow::AddPoint(POINT pt)
{
 // Ignore duplicates
 if (pt.x == m_ptLast.x && pt.y == m_ptLast.y) return;

 // Stop if no room for more
 if (m_cpt >= cptMax) return;

 AlwaysAddPoint(pt);
}

void RootWindow::AlwaysAddPoint(POINT pt)
{
 // Overwrite the last point if we can't add a new one
 if (m_cpt >= cptMax) m_cpt = cptMax - 1;

 // Invalidate the rectangle connecting this point
 // to the last point
 RECT rc = { pt.x, pt.y, pt.x+1, pt.y+1 };
 if (m_cpt > 0) {
  RECT rcLast = { m_ptLast.x,   m_ptLast.y,
                  m_ptLast.x+1, m_ptLast.y+1 };
  UnionRect(&rc, &rc, &rcLast);
 }
 InvalidateRect(m_hwnd, &rc, FALSE);

 // Add the point
 m_rgpt[m_cpt++] = pt;
 m_ptLast = pt;
}

void RootWindow::OnMouseMove(LPARAM lParam)
{
 if (m_fDrawing) {
  POINT pt = { GET_X_LPARAM(lParam), GET_Y_LPARAM(lParam) };
  AddPoint(pt);
 }
}

void RootWindow::OnButtonDown(LPARAM lParam)
{
 // Erase any previous polygon
 InvalidateRect(m_hwnd, NULL, TRUE);

 m_cpt = 0;
 POINT pt = { GET_X_LPARAM(lParam), GET_Y_LPARAM(lParam) };
 AlwaysAddPoint(pt);
 m_fDrawing = TRUE;
}

void RootWindow::OnButtonUp(LPARAM lParam)
{
 if (!m_fDrawing) return;

 OnMouseMove(lParam);

 // Close the loop, eating the last point if necessary
 AlwaysAddPoint(m_rgpt[0]);
 m_fDrawing = FALSE;
 UpdateTitle();
}

LRESULT RootWindow::HandleMessage(
                          UINT uMsg, WPARAM wParam, LPARAM lParam)
{
 switch (uMsg) {
  case WM_CREATE:
   OnCreate();  
   break;

  case WM_NCDESTROY:
   // Death of the root window ends the thread
   PostQuitMessage(0);
   break;

  case WM_SIZE:
   if (wParam == SIZE_MAXIMIZED || wParam == SIZE_RESTORED) {
    OnSizeChanged(GET_X_LPARAM(lParam), GET_Y_LPARAM(lParam));
   }
   break;

  case WM_MOUSEMOVE:
   OnMouseMove(lParam);
   break;

  case WM_LBUTTONDOWN:
   OnButtonDown(lParam);
   break;

  case WM_LBUTTONUP:
   OnButtonUp(lParam);
   break;
 }

 return __super::HandleMessage(uMsg, wParam, lParam);
}

RootWindow *RootWindow::Create()
{
 RootWindow *self = new(std::nothrow) RootWindow();
 if (self && self->WinCreateWindow(WS_EX_LAYERED,
       TEXT("Scratch"), WS_OVERLAPPEDWINDOW,
       CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT, CW_USEDEFAULT,
       NULL, NULL)) {
      return self;
  }
 delete self;
 return NULL;
}

This program records every mouse movement while the button is down and replays them in the form of a dotted polygon. When the mouse button goes up, it calculates the area both in terms of pixels and in terms of a percentage of the circle.

This program works well. My relative's hand moves slowly enough (after all, it has to trace a tumor) that the Get­Message loop is plenty fast enough to keep up. But just for the sake of illustration, suppose it isn't. To make the effect easier to see, let's add some artificial delays:

void RootWindow::OnMouseMove(LPARAM lParam)
{
 if (m_fDrawing) {
  POINT pt = { GET_X_LPARAM(lParam), GET_Y_LPARAM(lParam) };
  AddPoint(pt);
  UpdateWindow(m_hwnd);
  Sleep(100);
 }
}

Now, if you try to draw with the mouse, you see all sorts of jagged edges because our program can't keep up. (The Update­Window is just to make the most recent line visible while we are sleeping.)

Enter Get­Mouse­Move­Points­Ex. This gives you all the mouse activity that led up to a specific point in time, allowing you to fill in the data that you missed because you weren't pumping messages fast enough. Let's teach our program how to take advantage of this:

class RootWindow : public Window
{
...
 void AlwaysAddPoint(POINT pt);
 void AddMissingPoints(POINT pt, DWORD tm);
 void AddPoint(POINT pt);
...
 POINT m_ptLast;
 DWORD m_tmLast;
 int   m_cpt;
};

void RootWindow::AddMissingPoints(POINT pt, DWORD tm)
{
 // See discussion for why this code is wrong
 ClientToScreen(m_hwnd, &pt);
 MOUSEMOVEPOINT mmpt = { pt.x, pt.y, tm };
 MOUSEMOVEPOINT rgmmpt[64];
 int cmmpt = GetMouseMovePointsEx(sizeof(mmpt), &mmpt,
                            rgmmpt, 64, GMMP_USE_DISPLAY_POINTS);

 POINT ptLastScreen = m_ptLast;
 ClientToScreen(m_hwnd, &ptLastScreen);
 int i;
 for (i = 0; i < cmmpt; i++) {
  if (rgmmpt[i].time < m_tmLast) break;
  if (rgmmpt[i].time == m_tmLast &&
      rgmmpt[i].x == ptLastScreen.x &&
      rgmmpt[i].y == ptLastScreen.y) break;
 }
 while (--i >= 0) {
   POINT ptClient = { rgmmpt[i].x, rgmmpt[i].y };
   ScreenToClient(m_hwnd, &ptClient);
   AddPoint(ptClient);
 }
}

void RootWindow::AlwaysAddPoint(POINT pt)
{
...
 // Add the point
 m_rgpt[m_cpt++] = pt;
 m_ptLast = pt;
 m_tmLast = GetMessageTime();
}

void RootWindow::OnMouseMove(LPARAM lParam)
{
 if (m_fDrawing) {
  POINT pt = { GET_X_LPARAM(lParam), GET_Y_LPARAM(lParam) };
  AddMissingPoints(pt, GetMessageTime());
  AddPoint(pt);
  UpdateWindow(m_hwnd);
  Sleep(100); // artificial delay to simulate unresponsive app
 }
}

Before updating the the current mouse position, we check to see if there were other mouse motions that occurred while we weren't paying attention. We tell Get­Mouse­Move­Points­Ex, "Hey, here is a mouse message that I have right now. Please tell me about the stuff that I missed." It fills in an array with recent mouse history, most recent events first. We go through that array looking for the previous point, and give up either when we find it, or when the timestamps on the events we received take us too far backward in time. Once we find all the points that we missed, we play them into the Add­Point function.

Notes to people who like to copy code without understanding it: The code fragment above works only for single-monitor systems. To work correctly on multiple-monitor systems, you need to include the crazy coordinate-shifting code provided in the documentation for Get­Mouse­Move­Points­Ex. (I omitted that code because it would just be distracting.) Also, the management of m_tmLast is now rather confusing, but I did it this way to minimize the amount of change to the original program. It would probably be better to have added a DWORD tm parameter to Add­Point instead of trying to infer it from the current message time.

The Get­Mouse­Move­Points­Ex technique is also handy if you need to refer back to the historical record. For example, if the user dragged the mouse out of your window and you want to calculate the velocity with which the mouse exited, you can use Get­Mouse­Move­Points­Ex to get the most recent mouse activity and calculate the velocity. This saves you from having to record all the mouse activity yourself on the off chance that the mouse might leave the window.

Comments (11)
  1. Douglas says:

    "This saves you from having to record all the mouse activity yourself on the off chance that the mouse might leave the window."

    Or capture the mouse in button down and release capture in button up so that you get all mouse messages, even if they leave the window.

  2. SI says:

    Isn't the multimonitor adjustment equivalent to casting to signed short? Just kind of strange that the conversion for input points and output points is not symmetrical.

    [I didn't spend any time studying it. I just made a note and moved on, since it's not the point of the article. -Raymond]
  3. Joshua says:

    If you want your paint application to work over remote desktop, now you know how.

  4. Bob says:

    Why does GetMouseMovePointsEx require the x and y coords, isn't the timestamp enough?

    [That assumes that there can be at most one mouse event per millisecond. -Raymond]
  5. Bob says:

    > That assumes that there can be at most one mouse event per millisecond

    Okay, so what measures does GetMouseMovePointsEx take to handle the ABA problem then? I assume windows is coded to make sure that 2 equal timestamps can't share the same x,y coords?

    [Yes, there is an ABA race here, but it's better than nothing. -Raymond]
  6. laonianren says:

    I like the polygon area algorithm: it's very simple and it copes with overflows in its accumulator.  The only downside is daft results if the curve isn't simple (like a figure-eight).

    I guess it's very standard but I've not seen it before.

  7. "Oh great, now I'm telling people to practice medicine without a license."

    Funny … when I read that, my first thought was remembering some years ago when my vacation job included setting up web servers – one of which happened to be a medical one (a European mirror site for Yale medical school's 'GASNet', the Global Anesthesiology Server Network; the server hardware was stolen in 1999, though the DNS entry is still there today!) We didn't have rsync in those days, and even universities didn't have the bandwidth to spare to tar and copy the whole site each night, so I had to hack a Perl script to do the job. (It *almost* worked, but not quite.) The site admin thanked me on the front page, complete with email address (spam not being widespread that far back!)

    At which point, I started getting email asking medical advice: "I have a stabbing pain in … what should I do?". "Er, try a doctor, not a student moonlighting as sysadmin?"

    My second thought was "hm, that might be useful in the endoscopy project I'm working on now". (Measurement is rather more difficult there than with X-ray or MRI images because of the angles, though.)

    On the ABA problem: the default USB mouse polling rate is 125 Hz, which would put each coordinate 8 ms apart – and since neither the radiologist's monitor nor his hand-eye coordination will be operating even that fast; if you're getting more than one mouse position per millisecond, the vast majority won't relate to actual movements by the operator anyway!

  8. asklar says:

    [I know, unrelated to the API being discussed but I couldn't help myself :)]

    Don't you need to add a term to the sum to account for x[n]*y[0] – x[0]*y[n]?

    [x[n] == x[0] and y[n] == y[0], so the missing term is zero. -Raymond]
  9. @jas88: Aren't there higher-performance mice out there that use high-speed USB 2.0 to achieve a faster sample rate (i.e. < 1 ms)?  Seems like I remember seeing a review of one awhile back.

  10. Joshua says:

    [Yes, there is an ABA race here, but it's better than nothing. -Raymond]

    I would assume the results are returned in collection order so there is no ABA race.

    [Suppose the mouse move from x,y to z,w, then back to x,y all within a single millisecond. Which x,y is the one you are looking for? -Raymond]
  11. Worf says:

    @JamesJohnston: Unfortunately, that's not possible. USB is a host-polled bus, so the host decides how fast a given device wants to be polled. If the Windows HID driver polls for reports at 125Hz, it only gets reports at 125Hz. There's no way for the mouse to push HID reports any faster than that – it can only do it when the host asks for the report.

    Now, USB allows for polling rates up to 1kHz, but almost no OS implements it that fast – usually it's around 4ms per transaction or so.

    A high resolution mouse simply reports larger delta X/Y values every report.

Comments are closed.