A Mobile Digital Whiteboard Device

  • Article

Overview

Pundits have been predicting the paperless office to be "just around the corner" for decades, but it has yet to arrive. The ease with which digital data can be turned into hardcopy has arguably had the opposite effect, resulting in even more paper in the office than might otherwise exist. Part of the reason for this is the ease of use advantage paper has over computer displays.

As far as this idea is concerned, the most important advantage paper has over computer displays is the ability to have multiple pieces of paper strewn around a workspace and to refer to all of them simultaneously. Computers can emulate this to some extent by attaching multiple monitors to a single computer, but this is not really the same. The number of monitors that may be attached is often limited, either explicitly by the operating system or implicitly by running out of expansion slots for addition video cards. Further, with paper it is easy to detach a set of papers from each other (e.g. remove a paperclip or staple, take individual pages out of a notebook or binder) and arrange them any which way that may be desired; doing something similar with data on a computer is difficult (e.g., you must know to size the application appropriately and then create a new view, or launch a second instance of the application) and often impossible.

This idea removes this particular advantage of paper over computer displays. It allows any number of displays to be easily attached to a computer with no configuration required. It provides a standard gesture for "tearing" UI from an application to a display. These displays support inking and other interactions via a stylus, similar to Tablet PCs. Pages on a display retain a live link to their application, so edits performed via the display are automatically persisted back to the source document. Displays can be stacked and arranged as the user desires, just tike so many pieces of paper.

The Device

The device is essentially just a digitizer display, similar to that used on Tablet PCs. A CPU and system RAM will be necessary in order to handle communication with the host and to render the display, but the capabilities of these components do not need to be as powerful as would be required for a full-fledged computer. (The levels required would be roughly equivalent to those required by the now-defunct Smart Displays.)

The device is likely to be rigid and look similar to the slate-style Tablet PCs, but the flexible displays being developed by various companies would allow the device to look and act a lot more like the paper it is meant to replace. Tablet PCs, Pocket PCs, and other "smart" devices can also pretend to be the device by means of a special application or device driver.

A user interacts with the display via a stylus (again, a la a Tablet PC); one stylus or multiple styli can be used across any number of devices regardless of whether they are all connected to the same host computer. (More restrictive schemes, such as requiring each device to have its own styles, or requiring a unique stylus to be registered with each host computer and used only with those devices connected to that host, or allowing only a single stylus to be used per host computer, can certainly be imagined, but are likely too restrictive to be of any use.)

In addition to the screen and display the device sports several buttons:

  • A security button to lock/unlock the device. The device is unlocked by a conventional password or by inking a user-defined gesture.
  • A button that locks/unlocks the current page onto the device. When a page is locked to the device it continues to be usable and editable on the device even when the device is disconnected from the host computer (e.g., by moving it out of communication range). This allows usages such as locking a document onto the device and then using the disconnected device in a meeting to review and annotate the document, or using the device as a sort of mobile clipboard that enables data to be transferred from one host computer to another.
  • A button that displays UI for a) connecting to one of the recognized host computers, and b) loading specific content from the current host computer.

The device is perhaps most useful when sized similarly to a sheet of letter paper (which is to say approximately 8.5" x 11"), given that it is intended to replace exactly that. However just as paper sizes ranging from extremely tiny to wall-size can be useful, so may other sizes of devices be advantageous in certain situations. (For example, I would like a 1" x 1.5" device I could glue onto my Tablet PC and use as a sort of digital sticky note.)

The device connects to a host computer via a wireless protocol such as Wi-Fi, Bluetooth, or Wireless USB. The device can also talk to peripherals such as a wireless keyboard or printer via these same wireless protocols. Voice input may be interesting and useful as well.

The device could have a video out port to enable it to be connected to a video projector.

Multiple devices may be connected to a single host computer. Each device is connected to a single host at any one time but this host may be changed at will to any other currently recognized host.

Discovery Of And Interaction With Host Computers

The device works with a host computer via the following process:

  1. The device periodically sniffs its connection (e.g., Wireless USB) for hosts that indicate they can converse with it.
  2. When such a host is detected (e.g., its user enters a room containing a desktop computer so configured) the device notifies its user (preferably via a subtle alert such as a "toast" window or change in status of a sidebar tile) that a new host has been recognized. (The device can also be configured to automatically connect to a newly recognized host depending on whether it is locked, whether it currently has a page locked to it, and whether it is already connected to a host computer.)
  3. The user instructs the device to connect to a particular host computer. The device notifies the user (e.g., by changing a status icon) when the connection has been established.
  4. The user loads a document, page, window, pane, or other data onto the device by variously
    1. "tearing" the page, pane, area, etc. from a display on the host computer and dragging it onto the device
    2. using a dialog, task pane, menu item, or other UI on the host computer to identify the data to load and the device onto which to load it
    3. using a dialog, task pane, menu item, or other UI on the device to select a particular page, pane, etc. on the host computer to load onto the device
    4. "tearing" the page, etc. from the device to another device
    5. using a dialog or other UI on the device to move its data to another device or to move the data from another device to it
  5. The user manipulates the data as she wishes and the application controlling the data allows, for example:
    1. Manipulating widgets and editing data in a pane that affects other parts of the application on the host computer (e.g., details affecting the text that is currently selected in a document on the host).
    2. Editing a view onto data on the host computer (e.g., grabbing a magnified view onto a bitmap and directly editing the pixels in that view, or editing a OneNote page).
    3. Annotating the data on the device without affecting the corresponding data on the host computer.
  6. The user may disconnect from the host computer and optionally connect to a different host computer at any time. Whether the data currently loaded onto the device is lost or maintained depends on instructions from the user when he requests the disconnection/connection, whether the data is locked onto the device, and/or the configuration of the device and (old and new) host. This allows for example the device to serve as a mobile clipboard, wherein data is locked to the device, the device is connected to a different host computer, and then the data is downloaded to the new host computer.
  7. When a recognized host is lost (e.g., its user leaves the room) the device notifies its user that the host has been lost. The device defaults to locking the page it is currently displaying to itself but this behavior is configurable.

Using The Device While Connected To A Host Computer

There exists on the host computer UI whereby the user can find panes, pages, etc. that can be loaded onto the device, see a list of all devices connected to the host computer, and then loaded a particular pane, etc. onto a particular device.

There exists on the device UI whereby the user can find panes, pages, etc. on the host computer and each other device connected to the host computer and then loaded a particular pane, etc. onto a particular device.

There exists on the device UI whereby the user can disconnect from the current host computer and connect to a different host computer.

The device supports a user gesture whereby the user can copy or move a pane, page, etc. currently loaded on the device onto another device.

The host computer supports a user gesture whereby the user can identify a window, pane, page, area, etc. in/of an application and "tear" that window, pane, etc. onto the device.  Note that a particular window, pane, etc. or area thereof may be loaded onto multiple devices; for example an artist using a graphics editing application might tear the color palette onto one device, the brush settings palette onto another device, an zoomed-out view of the picture being edited onto another device, a specific highly magnified portion of the picture being edited onto another device, and a different portion of the of the picture being edited at the same or different zoom level onto another device.

Using The Device While Disconnected From A Host Computer

There exists on the device a hardware button and UI whereby the user can lock the currently loaded data onto the device, such that said data remains on the device even if the device is disconnected from the host computer.  Depending on the type of data and the capabilities of the device the data may be editable to a greater or lesser degree; for example a multi-page (screen) document (e.g., a OneNote page) locked onto the device may be scrolled and edited, or the data may be converted into what is effectively a screenshot of the data which may be annotated on the device but which annotations cannot be uploaded to the host computer other than as a digital image.

If the application supports such functionality the device can serve as a sort of mobile clipboard, wherein data is locked onto the device, the device is connected to a different host computer, and then the data is uploaded to an application on the new host.

Devices can recognize other devices even in the absence of a host computer.  Data locked onto one device can be transferred and locked onto another device; again, the facility and flexibility with which this may be done depends on the capabilities of the device (e.g., a simple device whose screen happens to be smaller than the source device may crop the data or display it a different zoom factor, whereas a Tablet PC functioning as a device may support scrolling and editing the data).

Enabling Application Support For The Device

An application can take specific actions to support the device; i.e., implementing particular interfaces or calling APIs to inform the operating system it supports the device and including specific functionality to interact with the device.

Frameworks used to build application (e.g., the .Net Framework or the new layout system code-named Avalon) could greatly simplify supporting the device by providing built-in support throughout the framework.  For example, Avalon could implement support for the device such that any panel or any page of a XAML document could automatically be torn off to the device.

By default data locked to the device supports some basic functionality, to wit that the data is effectively turned into a screenshot which can be annotated on the device but not merged back into the originating data.  An individual application can take steps to enhance the capabilities of locked data, such that the data can be manipulated via a special viewer app or can be merged back into the originating data, for example.