Manifesto: a new fluid UI for desktop/LOB data-entry

  • Article

I'm fed up with clunky dialog-based UIs.

 

Here are two apps, the Windows 7 Calculator and Windows 10 People App. How many of you use Calculator by clicking on buttons with the mouse? And how many do free-form typing with the keyboard?  answer: everyone prefers the keyboard. I think that free-form typing is the Holy Grail of LOB apps. Dialogs look cluttered and force users to waste their time on UI rather than directly on data entry.

calculator peopleapp

 

 

How about medical record software? Whenever I'm at the doctor's office, I peer over shoulders to see what they're using. In the consulting room, nurses and doctors usually seem to use free-form text entry to edit a template. The reception/billing system is usually a clunky dialog filled with too many buttons.

ims   vetinarygate

 

I want free-form text-based data entry. (As for free-form text-based commanding, that's already popular. Here are some examples. The final one is an exciting new indie game called Duskers where you control droids by text-based commanding.)

vscommand  vscode monkey game

 

 

Example of free-form text-based data entry

Let's re-imagine the Windows 10 People app. How could it get away from dialogs, and embrace free-form text entry? Here's my attempt...

(1) A simple text-entry dialog with grayed out suggestions for what to type:

pic1

(2) While you're typing, heuristics in the editor (e.g. "this line has overtyped the grey Name suggestion") infer that this is the name. The caption is updated to reflect this live while you type:

pic2

(3) When you hit ENTER to commit a line, the editor pretty-prints it. Pretty-printing is important to show what's been inferred:

pic3

(4) While I type the next line, more heuristics in the editor (e.g. "typing what looks like a phone number even though it's in the Address field") infer that this is the telephone number:

pic4

(5) When you hit ENTER, the editor pretty-prints it. In this case there are several kinds of phone number available, but the editor picked the most likely one and offered a way to change that (using the mouse or hitting Ctrl+Dot):

pic5

(6) I haven't shown any examples, but I'd also like the editor to show an intellisense completion list with suggestions of what it would expect and understand for you to type right now.

 

 

Manifesto: fluid UIs for text-based data entry

We should abandon clunky forms with lots of fields.

We should embrace free-form text entry.

This requires a smart text editor with natural language processing, heuristics, and most importantly intellisense.

 

 

 

What's next? My friend Klaus Löffelman had a great insight. He observed that the same characteristics that make for free-form text entry are also well suited to dictation and voice control. You could use them for Cortana/Siri/Alexa. You'd actually use the same heuristics, NLP and AI in for free-form data entry as you do bots in Skype or Facebook Messenger.

Update: there was some discussion on reddit. I appreciated this comment from @grauenwolf: "I have built several medical record applications over the years, and I can say without hesitation that there are lots of places where this would have been a perfect fit. "