POINT. TALK. TOUCH. Welcome to the wonderful world of natural user interfaces

For the last four decades, using a computer meant sitting down, looking at a screen and using a keyboard and, sometimes, a mouse. Today, new technology enables much more intuitive ways of interacting with computers. If you think touch screens and Kinect-controlled video games were state of the art, think again. What’s coming next is going to make that look like nostalgic old hat.


Tap, flick, fling. Tap open the bonnet of a car that hasn’t come off the production line yet. Flick a virtual ball and watch it roll towards the virtual hands of a colleague a thousand miles away. Fling a coin down on a table and watch information on investment plans appear.
Act as though the hardware of technology was not there, and you will begin to understand the possibilities of natural user interface, or NUI.

Early incarnations of NUI are already ubiquitous and extremely profitable. Touch-powered tablets like the Microsoft Surface and operating systems like Windows 8 have changed the way we interact with computers. In the living room, Microsoft’s Kinect for Xbox broke records by selling eight million devices within its first 60 days.  And this is just the beginning.

A Babel fish moment

Before 1967, in order to use a computer, you had to speak another language. That prehistoric method of instruction was known as command line interface and it was understood by only a specialised few. No ordinary person could just sit down at a terminal and know what to do.

Then in 1967 Douglas Englebert filed the patent for the first computer mouse. This moment marked the beginning of a second era of human-computer interaction, known as graphic user interface.

The mouse meant anybody could sit down at a computer, point and click, and their action would be translated into whichever programming language was needed to complete their command.

It might have taken another 30 years for the full impact of that Babel fish moment to ripple out from the labs via the first Macintosh and Windows 95 computers, but it empowered more people to use computers than ever before.

Should we meet?

“Natural interface is about really immersing the user; it’s about bringing the physical world into the digital domain.” That’s how Shahram Izadi, head of NUI research at Microsoft Research Labs, describes the emerging relationship with technology in the NUI era.

The problem with the mouse was it still required people to go to their computer, and interact with a piece of hardware in order to access the benefits of technology. Our current era of human-computer interaction, natural user interface, is about making technology part of our surroundings: it’s about bringing technology to us. Imagine it as the difference between speaking to someone on the phone and meeting them in person. The latter experience is richer with body language, touch and eye contact: communication is more intuitive.

In practical terms this means using touch, gesture and voice as methods of command, and allowing everyday physical objects to function as portals to technology. The idea is to take what comes naturally to us, and make it digital.

How can I help you?

The ability that natural user interface has to deliver a richer relationship with technology is an important development for business. It means companies can deliver a better customer experience, and a happier customer is a more loyal profitable customer.

A big majority of consumers (85 percent) say they would pay more for a better customer experience, according to a report by RightNow Technologies. In fact, more than a quarter is willing to pay as much as 15 percent more. 

The future, today

Already companies are using developments like multi touch screens and gesture recognition technology to their advantage. For example, Microsoft is working with Nissan to reinvent the car showroom.

Nissan had a new model launch to prepare for, but no model to actually display or demonstrate with. So, they developed the Pathfinder app that used Kinect’s gesture recognition technology to allow customers to explore the vehicle on screen. People could open the doors, move the seats and change the interior by utilising the same movements they would use with a real display model.

Pathfinder proved so successful that Nissan rolled it out to 16 dealerships across the USA. “It’s a powerful pre-sales tool that has the power to revolutionize the dealer experience,” says John Brancheau, vice president of marketing at Nissan North America.

Use your imagination

Of course every business is different. Making the most of NUI means understanding just how your customers interact with your product or service, and applying the right technologies in the right places. Automatic doors were right for supermarkets, with customers carrying lots of bags, but not for high-security bank vaults.

The vast potential of NUI technologies like Microsoft Kinect and 3D digital rendering is already beginning to emerge as imaginative innovators and Microsoft partnerships begin to play with the possible:

  • One Kinect hack has resulted in a shopping cart that loyally (and automatically) follows a disabled shopper.
  • Heart surgeons are using Kinect’s gesture technology to access patient files during surgery without the need to scrub in and out during complex procedures.
  • Dr. Dre and Snoop Dogg were even able to bring Tupac back to life in concert at the Coachella festival last year using hologram technology.

Take the lead

It’s early days for NUI but already it promises innovators a competitive advantage. That’s why more than 200 organisations, including 25 Fortune 500 companies, are already working with Microsoft to explore Kinect’s commercial possibilities.
“Just having the functionality these days is table stakes. Everyone’s got the functionality,” says Peter O’Blenis, chief operating officer at Flick Software. “It’s how you interact with it that makes it stand out.”

Skip to main content