Microsoft Research: How we watch the computer, how it watches us

image At least once each year I trek on over to visit my colleagues at Microsoft Research.  It's actually not much of a trek.  The group's headquarters are really just a few blocks from my own office on the Microsoft campus.   I was especially looking forward to this year's visit since I hadn't yet had an opportunity to see MSR's new building.  The group moved into sparkling new facilities back in November.  The building hosts a four-story atrium with dazzling open spaces, state of the art conference rooms, and a soothing Northwest decor.  It's the perfect place to think and collaborate.

Visiting MSR has become part of my routine  because it's a way for me to image preview some of our latest technologies and then speculate how they might one day be applied to health and healthcare.  Two years ago, I was privileged to see Surface computing long before it became a commercial product.  As inventor-researcher, Andy Wilson, and I played with Surface I couldn't help but speculate on all the ways this revolutionary user interface would find its way into clinical medicine. 

image On this year's tour we take a look at six different projects in a special two-part video edition of my popular House Calls for Healthcare Professionals series. Several of these projects have a bit of a common thread in that they explore how we watch the computer, and how the computer might do a better job of watching us.  In Part One,  I think you'll particularly enjoy something called "situated interaction" in which an on-screen avatar of sortsinteracts with individuals or groups of people to perform routine tasks such as scheduling our on-campus shuttle service.  Perhaps one day many of the mundane tasks now handled by front desk "assistants" in our clinics and medical centers will be performed by such machines.  We'll also see the way researchers are using eye-tracking software imageto study how humans interact with what's on a computer screen to help guide the design of better user interfaces.  Once again, one might imagine how this could be applied to designing better clinical systems where too much data often obscures what we really need to know.  On the last stop of the first program, we once again drop  in to visit Andy Wilson.  This time Andy shows us what may be the next evolution in Surface computing and display technology.  It gets BIG!

Part Two of my video visit to Microsoft Research carries on with the theme byimage exploring how visualization is helping researchers study the HIV virus.  This work, by Dr. David Heckerman and colleagues, got its start in research originally designed to help filter spam in corporate e-mail.  It turns out that spam headers morph over time in much the same way that a virus mutates.  Bingo!  Researchers designed algorithms and mapping solutions that help predict how and where HIV mutates, a discovery that has great potential for speeding vaccine research.  In another lab, we explore howanimation helps bring data to life thereby improving our image comprehension of what's going on.  Finally, I visit with a research intern who demonstrates yet another way we may one day interact with the computer, using electromyographic signals.  This technology may lead to better assistive technologies for people with disabilities or to ways that busy clinicians, who are scrubbed into surgery or otherwise occupied, might navigate the computer screen and enter data.

I hope you enjoy the shows.  Special thanks to my friend and professional colleague, Dr. Eric Horvitz (most recently of "6.6 degrees of separation" fame) for his help in facilitating our video shoot with his fellow MSR researchers.  Thanks also to Laura Foy and Bob Snyder from Microsoft Channel 10.  Click on the links below to watch the videos.

A Visit to Microsoft Research Part One

A Visit to Microsoft Research Part Two

Bill Crounse, MD    Senior Director, Worldwide Health   Microsoft Corporation

Comments (9)

  1. The videos are well worth watching as well as fascinating.  These are not just "techs"

  2. Thanks to Dr. Crounse for previewing some of this at the HUG meeting in Redmond recently, we have already

  3. sully says:

    Amazing videos, thank you so much

  4. I have been wanting to write this for quite some time, with as much technology as I post here, so here

  5. There are also plans to visit Google and Facebook.  Of course e-health was right up there being

  6. WulfCry says:

    Strange I’ve seen this picture before.

    And what does this piece of text mean.

    "I think you’ll particularly enjoy something called "situated interaction" in which an on-screen avatar of sorts interacts with individuals or groups of people"

    What is interact defined at to where and which. Sounds vaguely lamented for A.I as situated or assisted why.

  7. hlthblog says:


    Thanks for writing.  The situated interaction technology from Microsoft Research is ineed a kind of AI.  Using very powerful computers, the team has designed an Avatar that listens, watches, and inteprets things going on in the environment in front of her.  The demo shows the Avatar helping people reserve a shuttle service on the Microsoft campus.  She asks where you need to go, who you are traveling with, and a few other questions.  She knows how many people are waiting in line and even if some of those people are becomming inpatient.  It is really quite remarkable.

    Bill Crounse, MD

  8. WulfCry says:

    The video’s are still interesting I can see how far technology can be developed although you wont see any of this yet due to complications to implement them in some industries. One thing within the video got mine interest ”dynabizz” or ”dinabiss” I cant find any of this kind of data visualization back and in what Microsoft product it is integrated.    

  9. hlthblog says:

    Thanks for your comment.  The graphic visualizaiton you saw in the video is likely to be seen in future releases of Microsoft Office and other solutions where such displays help people consume and understand data.

    Bill Crounse, MD

Skip to main content