Multitouch in VE3D

Ever since I got a Touchsmart at home I’ve been very interested in the desktop touch experience, and so in the last couple days I decided to play with how I could incorporate it into VE3D.  The code itself isn’t really ready for consumption, but if you’d like to see where I’m at, I’ve built a very preliminary plug-in that you can install and play with.


This requires Windows 7 and a machine that has multitouch support, such as a Touchsmart or Latitude XT.  The package is a self-extracting zip, just run it and accept the defaults.  The next time you launch VE3D, the plug-in will load shortly after startup.  There’s no UI or anything.  The gestures will just quietly start working.  The plug-in will work on any app that uses GlobeControl (and loads it), but since it calls native code it does require installation.

Currently supported are two-finger zoom and two-finger rotate.  I’m using the built-in gesture engine from Win7, which makes things very easy, but in the future I would like to use direct touch input instead.  This should eliminate a few artifacts like a momentary delay between when you start input and when the map starts moving.  It should also allow support of some more interesting types of inputs, like a two-finger pan, combining rotate and zoom into a single fluid motion, etc.  The point is that there’s a lot of potential beyond what I’ve done here.

The touch interface in Win7 relies on Windows Messages and a few native functions.  To listen to Windows Messages, use Host.CommunicationManager.AttachToMessage.  To interact with the native gesture API, here’s something to get you started (this code will work just as well in any managed application).  After that it’s a matter of redirecting gestures to existing navigation Actions using bindings.  To really do it right you’d probably also want to write a few things in your own CameraController.

Here’s the gesture documentation on MSDN.  Here’s the raw touch docs.

Note:  there’s a bug in the current GlobeControl where the lParam and wParam values are reversed in the handler to AttachToMessage.  Whoops!  This is fixed in the next version, so be prepared.  I will be releasing my code as a sample at that time, and will update the installer as well.

Note 2:  I noticed on my Touchsmart that the rotate gesture has some trouble when your fingers are diagonal relative to the screen.  I think this is a driver thing, since it doesn’t happen on the Latitude XT, which uses a different touch technology and, obviously, different drivers.  However, it looks like they are releasing improved drivers on Windows Update, so I hope this is just a bleeding edge thing.

Edit:  the “Download” link has been updated to a version that is compatible with the current build on Virtual Earth 3D.  Please uninstall the old version first and then install the new one.  Functionality is still the same, that is to say, less than ideal.  From what I have seen, using WM_TOUCH instead of WM_GESTURE will provide a far superior experience.  Stay tuned.

Comments (11)

  1. KSubramani says:

    Awesome!!!  I’m working on a VE 3D app for a 5-point multitouch device.  Does this plugin account for an arbitrary number of touches?  Also, I’m very curious to know if you’ve come up with a gesture for changing the tilt of the map.  Given that my device has more than two touch points, I’m thinking maybe to add a three finger gesture where motioning across the screen with three or more fingers will change the tilt of the map.

  2. NikolaiF says:

    This plug-in just uses WM_GESTURE, not WM_TOUCH, so it does whatever the gesture engine does.  I think it’s just two points.  The gesture engine doesn’t support anything that I thought would make a good tilt gesture.  If you’re near the ground you can tilt/turn by grabbing the edges of the screen, so I left it at that for now.

    The things I’ve thought about are two-finger pans, plant with one finger and drag up/down with another.  But I’m sure there are more sophisticated ideas to be had.

  3. stevehead says:

    This plugin seems to have been disabled by the latest ve3d enforced update.

  4. NikolaiF says:

    I’ve rebuilt and updated the download link.  FYI, the old installer does not show up in the programs list unless you log in as an administrator.  Not sure as yet why.  Anyway, enjoy!

  5. grovlan says:

    hi, greath plug-in, but how detect wm_touch message in your code to detect all fingers. i’ve tried to attach message GesturesInterop. WM_TOUCH = 0x0240; but it don’t go in the handleGesture function, maybe i don’t understand enought the code, but if you can help me, thanks.

  6. NikolaiF says:


    You need to specifically turn on touch inputs to get touch messages — otherwise all you get are gestures.

    Also, keep in mind that this code was written against the Win7 beta, and is at least partially out of date for RC/RTM.  Here’s a few web resources:

    I haven’t had a chance to play much more with this code yet, though I certainly plan to and you’ll see mention of it here.

  7. grovlan says:

    OK thank you, i watched and i’ve just found the samples of w7 … (bang!!) .

    It helps me a lot.Thanks

    Do you know if there is a documention or examples for CameraControl in virtual earth. Waiting your answer i’ve simulated fingers with TUIO client and i’ve combinated zoom and rotate with Turn and ApproachTarget, but i want to combinate zoom, rotate AND Translation. I think i have to use PointerDrag, but i don’t know how to define my own pointer or my own anchor.

    Thank you for your help.

    And sorry for my poor english, i’m french and for the story, i’m working on a multitouch device for my internships.

  8. grovlan says:

    huuum, sorry,  i’ve an other question…  if i understand to use wm_touch and not wm_gesture i’ve to use RegisterTouchWindow(handle, 0) with an handle, but the handle i print in Handlegesture is not the current windows handle (i use getForeGroundWindow).. So, how can i get this handle ?? you used this in handleGesture, but how??

    thank you very much.

    i’m lost…

  9. NikolaiF says:

    Hi Grovlan,


    Samples (including camera):

    Defining your own camera controller provides the most power, but is correspondingly the most difficult.  My time is crunched right at the moment, but I will try to look at it more myself when I can.

    I think the handle you want is host.Control.Handle.  It is the handle of the control itself, not necessarily the top level window handle.

  10. grovlan says:

    Ok, it’s perfect. Thank you very much, i’ll send you my plugin if you want (urk, i’ve forgetten how to conjugate subjonctive), when it’ll be finished.