Showcasing Windows 7 Platform with Applets

About every decade we make the big decision to update what we refer to as the applets (note we’ll use applet, application, program, and tool all interchangeably as we write about these) in Windows—historically Calc (Calculator), Paint (or MS Paint, Paint Brush) and WordPad (or Write), and also the new Sticky Notes applet in Windows 7. As an old-timer, whenever I think of these tools I think of all the history behind them and how they came about. I’m sure many folks have seen the now “classic” video of our (now) CEO showing off Windows to our sales force (the last word of this video is the clue that this video was done for inside Microsoft). Windows 7 seems like a great time to update these tools. The motivation for updating the applets this release is not the 10-year mark or just time to add some applet-specific features, but the new opportunities for developers to integrate their applications with the Windows 7 desktop experience. While many use the applets as primary tools, our view of these is much more about demonstrating the overall platform experience and to provide guidance to developers about how to integrate and build on Windows 7, while at the same time providing “out of box” value for everyone. There’s no real “tension” over adding more and more features to these tools as our primary focus is on showing off what’s new in Windows—after all there are many full-featured tools available that provide similar functionality for free. So let’s not fill the comments with request for more bitmap editing features or advanced scientific calculator features :-).

The APIs discussed in this post are all described on MSDN in the updated developer area for Windows 7 where you can find the Windows 7 developer guide. Each of the areas discussed is also supported by the PDC and WinHEC sessions on those sites.

This post was written by several folks on our applications and gadgets team with Riyaz Pishori, the group program manager, leading the effort. --Steven

This blog post discusses some of the platform innovations in Windows 7 and how Windows 7 applications have adopted and showcased these innovations. This post details some of the platform features that developers and partners can expect in Windows 7 and how Windows 7 programs have showcased these innovations. This post also discusses how applications have been given a facelift both in terms of their functionality as well as their user experience by focusing on key Windows design principles and platform innovations. Finally, this post can serve as a pointer or guide to application developers and ISVs to get themselves familiar with some of the key new Windows platform innovations, see them in action and then figure out how they can build on these APIs for their own software.

The post is organized by each subsystem, and how Windows applets are using that particular subsystem.

Windows Ribbon

The Windows Ribbon User Interface is the next generation rich, new user interface for Windows development. The Windows Ribbon brings the now familiar Office 2007 Ribbon user interface to Windows 7, making it available to application developers and ISVs.

There are several advantages of adopting the Windows Ribbon user interface, many of which have been talked about in the Office 2007 blogs. The Ribbon provides a rich, graphical user interface for all commands in a single place, without the need to expose various functions and commands under different menus or toolbars. The Ribbon UI is direct and self-explanatory, and has a labelled grouping of logically related commands. While using an application that is built on the Ribbon UI platform, the user only needs to focus on his workflow and the context of his task, rather than worry about where a particular function is located or accessible. The Ribbon UI also takes care of layout and provides consistency as compared to toolbars which the user can customize in terms of their sizes, location and contents. It also has built-in and improved keyboard accessibility, and making the application DPI and theme aware becomes easier by using the Ribbon. Finally, development and changes to the user interface is very quick and rapid due to the XML-markup based programming model for the Ribbon User Interface.

Paint and Wordpad are two of the first consumers of the Windows Ribbon UI Platform. In Windows 7, both these applications are enhanced with a set of new features, and the user interface of these applications also required to be brought up to the Windows 7 experience and standards. The Windows Ribbon UI is a great fit for these applications to revamp their user experience and make it consistent, and make these applications rich, fun and easy to use. The tasks and commands in these applications were amenable to be applied to the Ribbon UI framework, and it also served as an opportunity for popular native Windows applications to showcase the Windows Ribbon UI platform to consumers, as well as developers and ISVs. Many has asked about the Windows Explorer and IE also using the ribbon, which we did not plan on for Windows 7. Our Windows 7 focus was on the platform and demonstrating the platform for document-centric applications such as Paint and Wordpad.

Both these applications showcase several elements of the Windows UI Ribbon. The Application Menu of both Paint and Wordpad exposes Application related commands that are typically available thru the ‘File’ menu of an application. Both the applications have a core tab set that consists of ‘Home’, which exposes most of the commands in the application, and ‘View’ which exposes the image or document viewing options in the application. The commands in both these tabs are laid out logically in groups of related functionality.

A quick access toolbar (QAT) is provided by both Paint and Wordpad, which comes with certain defaults like Save, Undo and Redo that are meaningful to the application. The user can customize the QAT by using the QAT drop-down, or right-click on any command or group in the ribbon and add it to the QAT.

Several ribbon commands are used in both these applications, like command buttons, split buttons, galleries, drop-downs, check boxes and toggle buttons.

Paint and WordPad ribbon

Paint Application Menu

Further, both applications provide a ‘Print preview’ mode which shows a print preview of the image or the document in context. In a mode, all the core tabs are removed and only the mode is displayed for the user to interact with. On exiting a mode, the user is returned to the core tab set.

Paint also exposes a contextual tab for the Text tool, which is displayed only when a text control is drawn on canvas. A contextual tab shown next to the core tab set when the text tool is in focus, and removed when the text is applied to the image on the canvas. The contextual tab set contains the tools that are specific and relevant only to the text tool.

Both the applications provide live previews through ribbon galleries, for example the font size and font name for Wordpad and Paint while formatting text, bullets and lists in Wordpad, and color selection, outline size selection and outline and fill styles for shapes in Paint. A live preview allows the user to see the changes instantaneously on mouse hover, and then apply those changes on a selection. These previews are one of the key elements of the ribbon UI and demonstrate why the metaphor is much more than a “big toolbar” but a new interaction style.

By adopting the Ribbon User Interface, both the applications inherit built-in keyboard accessibility support using ribbon Keytips, have tooltips on all commands, and have ready support for DPI and Windows themes.

Paint and Wordpad can serve as examples of how the Ribbon UI can be easily used in MFC applications. The Windows Ribbon presents new opportunities and options for developers and ISVs to develop applications with the Ribbon User Interface. The Windows Scenic Ribbon programming model and architecture emphasizes the separation of the markup file and the C++ code files to help developers decouple the presentation and customization of the UI from the underlying application code. The platform also promotes developer-designer workflow, where the developer can focus on the application logic, while the designer can work on the UI presentation and layout. The ribbon UI is a significant investment for us and you should expect to see us continue to use it more throughout Microsoft, including an implementation in the .NET Framework as was demonstrated by Scott Guthrie at the PDC, which will be built on Windows 7 natively in the future.

Multi-touch platform

Windows 7 provides support for multi-touch input data, as well as supporting multi-touch in Win32 via Windows messages. The investments in the multi-touch platform include the developer platform that exposes touch APIs to applications, enhancing the core user interface in Windows 7 to optimize for touch experiences, and providing multi-touch gestures for applications to consume. Developers on Windows 7 can build on these APIs decide on the appropriate level of touch support they would like to provide in their software.

Wordpad enhances the document reading experience by using the multi-touch platform and using the zoom and pan gestures. Zooming, panning and inertia lets the user get to a particular piece of content very quickly in an intuitive fashion. By using the zoom gesture, the user can zoom in or zoom out of the document which is akin to using the zoom slider at the right of the Wordpad status bar. On multi-touch capable hardware, the user can zoom in and out of the document by placing his fingers anywhere within the document window and executing the zoom gesture. Wordpad also supports the pan gesture to pan thru the pages of a document that is open in Wordpad. By executing the pan gesture, the user can scroll-down or scroll-up a document similar to using the scroll bar of the Wordpad application.

In Paint multi-touch data is used to allow users to paint with multiple fingers. It is an example of an application that allows multi-touch input without the usage of gestures. For Paint’s functionality, providing multiple finger painting ability was more compelling and enriching than allowing for zoom, pan, rotate or other gestures that act on the picture in a read-only mode and not in an edit-mode. New brushes in Paint are multi-touch enabled, and they handle touch inputs via multiple fingers and allow the user to simultaneously draw strokes on canvas on finger drag. These brushes are also pressure-sensitive, thereby providing a realistic experience with touch by varying the stroke width based on the pressure on the screen. While adopting the multi-touch platform to enhance the end-user experience in Paint, conscious design decisions were made to preserve the single touch experience for functionalities where a multi-touch scenario does not apply such as the color picker, magnifier and text tool.

By building with the multi-touch APIs, Paint and Wordpad have created more natural and intuitive interfaces on touch-enabled hardware and show “out of the box” how different capabilities can be exposed by developers in their software.

Taskbar

Sticky Notes (or just Notes) is an extension of a TabletPC applet available in Windows 7. One of the things which was key to the Notes experience on the desktop was to have the ability to quickly take all the notes away and get them back, but still making sure it is really easy to create a new note. We achieved this by having a single top level window for the sticky notes application. You can minimize all your notes and view a stack of notes in the preview on the command bar with a single click. The stacked preview has been achieved using the new thumbnail preview APIs that enable apps to override the default taskbar previews that are essentially a redirected snapshot of the top level application window, and provide their own. This enables applications to decouple their previews from the top level application window and provide a more productive preview based on the scenario. For example, this was very valuable in Sticky Note scenarios where a quick peek at a note that was last touched provides for quite a productive workflow. Taskbar also caches the preview thumbnail images so once the preview is given to the Taskbar, the application does not need to keep it around – the application does however need to send an updated preview whenever it changes.

 Sticky Notes preview on Taskbar

Another nifty customization end-point on the task bar is the destination menu (aka jumplist). This menu comes up when a user right clicks on the application in the taskbar or hovers over the application icon in the Start Menu. The Sticky Notes application does not have a single main application window – this makes the application feel really light weight and fits in well into the Windows 7 philosophy of creating simple and powerful user experiences. The challenge then was exposing functionality such as the ability to create a new note from a central location or potentially other custom “tasks”. The destination menu helped exposing these scenarios in a simple yet discoverable way.

Sticky Notes destination menu

Sticky Notes destination menu

The new taskbar functionality and extensibility built in that has the potential to make it a lot easier for people to work with applications/scenarios in a more productive and efficient manner when developers integrate their software with the Windows desktop.

Building on the long history of Search in Windows and the significant enhancements in Windows 7, there are APIs available to developers to deeply integrate their content types with the desktop search user experience affordances in Windows 7. Sticky Notes shows one example of how these APIs can be used.

The Sticky Notes application now allows users to get back to their notes by simply searching for content through Windows Inline search within the start menu. This is in line with allowing users to reach the relevant note as quickly as possible even when the application is closed. Even though search could be done for both Text and Ink content, it is restricted to text because of lower success rates with varied handwriting styles in ink. The application registers a protocol handler that generates a URL for each Note. The Sticky Notes Filter handler gets asked for the content associated with each note that is then indexed by the Search infrastructure. These indexes are then used to perform a quick lookups when the user searches the Search interfaces provided by the Windows Shell. When a user clicks on a result, Search invokes the associated application with the URL corresponding to the one that the protocol handler had generated that the Filter handler associated with the content it sent to the Search indexer.

 

Search from start menu and a result displayed from Sticky Notes

The search platform also has the ability to enable the filter handler to specify the language of each chunk of content passed on to it that overrides the default Search heuristics used to compute the language - this increases Search accuracy manifold and thereby enhances internationalization support of the entire ecosystem.

The reason Sticky Notes implemented a protocol handler in addition to a Filter handler was because it implements its own integrated storage schema on top of the Windows File system - all the notes are represented by a single .snt file. The protocol handler generates URL's to individual entities (in this case - notes); the filter handler picks out content for each of these URL's and gives it to Search for indexing.

This demonstrates the ease in which applications can plug into the search platform in Windows 7, and add search handlers which can enhance the overall user experience from the App as well as the platform.

Real-Time Stylus

Real-Time Stylus (RTS) is infrastructure that provides access to the stylus events coming from pen or touches digitizers. It provides information about strokes and points and provides access to ink-related events. Using RTS, applications can get access to stylus information and develop compelling end-user scenarios and experiences.

Sticky Notes now allows the users to Ink and Type on notes depending on the availability of inking hardware. Users can use keyboard input to type on notes and use the stylus to ink on notes. Though the experience has been designed keeping in mind that users will either use either ink or text on a particular note, it does allow users to ink and type on the same note. However these surfaces are maintained agnostic to each other. Sticky Notes also auto-grows the note while inking on the note, providing a real-time experience of the note adjusting its size to fit the inked content.

Real Time Stylus (RTS) is used for inking features provided in Sticky Notes. Inking gestures are also available to applications, and the scratch out gesture has been implemented in Sticky Notes to delete content.

Scratch gesture

Sticky Note with both typed text and inked content

In addition, Paint uses RTS to get a stream of positional input from mouse, stylus or touch which are used for drawing strokes on canvas. Paint also captures additional input variables like pressure and touch surface area when such input is available from the digitizer, and maps these inputs into the stroke algorithms that are used to generate Paint strokes on canvas. Using this algorithm, the user is able to modulate stroke width and other parameter based on the pressure or touch area on canvas.

Paint with new brush strokes

Using RTS allows the development of applications and software that can build on the inking platform and provide ways to interact with the application that go beyond mouse or keyboard. Using stylus, inking and gestures, developers can create interactive experiences for end-users.

Restart and Recovery

The Windows Error Reporting (WER) infrastructure is a set of feedback technologies that is built into Windows 7 and other earlier versions of Windows client and server. WER allows applications to register for application failures and capture this data for end-users who agree to report it. This data can be accessed and analyzed and can be used to monitor error trends and download debug information to help developers and ISVs determine the root cause for application failures.

WER can add value to software development at various stages: during development, during beta testing by getting early feedback from end-users, after the release of the product by analysing and prioritizing the top fixes, and at end of life of the product.

Related to failure recovery, Applications can also register with WER for restart on application of a Windows patch that terminates the application and on application of an update that reboots the computer, as well as failure caused due to an application crash or hang or not responding state. Applications can optionally register for recovery of lost data, can develop their own mechanism for recovery.

Several Windows applications adopt the WER infrastructure to collect and analyze data. Calculator, Paint and Wordpad register for restart and additionally recover the current data in the sessions of the application that were running. Sticky Notes also registers for restart and recovery, and returns the user to the set of notes open on the desktop. Using WER, end-users would allow Windows to capture and collect problem data and then would be returned to the applications in the same state that they were in earlier.

As you can see, our primary effort for the applets in Windows 7 is to showcase some of the new platform APIs and innovations available to developers. As you get to try out these applications you will see that while showcasing the Windows 7 platform innovations, we have also added some commonly requested features and functionality. Some of them are: Check and correct, calculation modes and templates in Calculator, New brushes, shapes and multi-touch support in Paint, Open standards support in Wordpad and Ink and text, taskbar and search integration in Sticky notes. Maybe we won’t wait 10 years to update these again :-)

--Riyaz Pishori and team