Is Visual Studio 2010 really as evolutionary as we think?

Article
05/28/2009

As mentioned in my post Why would anyone park his Windows START button top left? I eluded to the fact that I would evaluate the Visual Studio User Interface, from my perspective, using heuristic evaluation, a topic that I covered in the book I read and referred to (“Interaction Design, beyond human-computer interaction”, by John Whiley and Sons, Ltd). For years we have been focusing our whitepapers, our gee-wizz it’s .NET 1.0 events, our presentations and our blogs to the technology, the bits and bytes. We all know the immense evolution of features arriving with Visual Studio team System 2010, which Brian Harry has and will be covering, as will many of the technology evangelists.

What I would like to do in this post is document my explorations through the world of evaluation approaches and methods, investigate whether the Visual Studio Team System 2010 is really as evolutionary as I think it is and conclude with my findings. I must reiterate, that the findings are mine and the opinions represented in this blog post mine as well.

Before we begin, what is heuristic evaluation?

Heuristic evaluation is a usability inspection technique, which was developed by Jakob Nielsen and his colleagues and guided by a set of usability principles … commonly referred to as “heuristics”. Two sites I found with good summaries include: https://www.useit.com/papers/heuristic/ and good old Wiki https://en.wikipedia.org/wiki/Heuristic_evaluation … plus a million and one other sites and books :)

What are we trying to achieve?

In this exercise the goals include, from my perspective::

Practise what I have read, namely heuristic evaluation.
Evaluate Visual Studio 2010 BETA-1.
Investigate and decide whether the new user interface is as evolutionary as I personally believe it is.

Evaluating Visual Studio 2010 BETA-1

To ensure that the exploration and evaluation is focused, I decided to define the following user scenario as a guideline

Create a class library project Interfaces, defining a interface IChatter, which defines methods Hello() and GoodBye().
Create a class library project ConsoleChatter, implementing the IChatter interface.
Create a console application project, referencing ConsoleChatter, and calling Hello() and GoodBye() methods.
Create a Models project, adding a sequence diagram showing the complete logic flow, starting from Main().

I am not quite sure if it is proper heuristic evaluation procedure, but I completed the exercise first using Visual Studio Team System 2008 and then Visual Studio team System 2010 BETA-1, to get a comparative view on the current and future releases respectively. My notes with the 2008 excursion were the following, most of which are based on commonly reported issues, but implicitly enforced in the walkthrough scenario to set the scene and determine if the user interface interaction has improved accordingly:

Start page appears cluttered with information … where should the user start?
When adding an interface, the interface defaults to private access unless decorated with public. No user hint or error is raised in the Error List window, alerting the user that the interface could be kind of meaningless in its default state, until a reference is made. Thereafter an error appears.
When defining the class and adding the interface inheritance, there is no hint on what implicit and explicit interface implementation is. User typically tries both and then selects the appropriate option.
Warning that an incorrect type, for example a class library, cannot be launched directly when console is not start-up program is raised as user tries to run the test application.
When removing a document from the tab list, often by mistake, it is unclear how to return it to the tab list at a first glance. Same applies to windows in collections … once removed from their home, users are puzzled as how to get it back.
Obviously I could not complete step 4, as the functionality does not exist in 2008.

Here are my journey notes, looking at VSTS 2010 BETA-1, remembering the journeys with Developer Studio (the era before VS), Visual Studio 97 – 2008 and before. I understand that because of my long-term involvement with Visual Studio, my evaluation may be biased as I am likely to be more sensitive to enhancements and insensitive to known issues than other users. I understand that because of my long-term involvement with Visual Studio, my evaluation may be biased as I am likely to be more sensitive to enhancements and insensitive to known issues than other users :)

Heuristics Category Heuristics Evaluation Notes Improvement Suggestions 5 star rating

Nielsen’s Heuristics Visibility of system status Visual Studio keeps the user informed about the state of the application through status bar feedback (1), the use of specialised windows, such as the Error List (2) and associated code highlighting, such as the wavy underline to indicate an error in the source code.

Match between system and the real world The application speaks the language of Windows platform specific developer, using words, phrases, descriptions and concepts familiar to the user. At first the list of options and properties per option appears overwhelming to the user, but the logical ordering of the options and sub-options allows for rapid adoption and understanding. Keep history of the settings and options changed by the user, presenting both the complete and a subset of commonly referred to and accessed options. This would allow users to find their “typical” settings much quicker and improve the “recognition rather than recall” heuristic further down as well.

User control and freedom User is able to undo and redo most actions, including global impact changes, such as code refactoring. Created a VB, instead of C#, project by accident as I failed to notice that the project creation wizard defaulted to VB. The undo’ing of this step involved manual un’doing and deletion of files, which can be time consuming and confusing to the novice user. Support the un’doing of a project creation action, including the deletion of all created artefacts.

Consistency and standards User interface is consistent with 2008 … look & feel, feature set. The additional features blend into the environment and can be reviewed by the user through exploration. In other words if you know the 2008 environment you will be familiar with 2010, as the look & feel is consistent and as the use of terminology, actions, menu and wizards is consistent with previous releases and with the Windows platform. Some of the project settings (2008 defaulted to AnyCPU, 2010 defaults to X86) have changed and may|will cause problems with users, especially those that have been working with defaults when creating Team Builds. Allow user to define the start-up defaults, do not enforce them.

Error preventions Visual Studio performs consistency and validity checking while the user edits code. For example when I purposefully deleted the public access modifier keyword from the interface declaration, the system warned that this action would make the interface inaccessible. This pro-actively advises the user and avoids error-prone actions. Actions, such as deletions, present the user with a warning and confirmation option, before committing the action. The system does not warn the user when declaring a private (which is default with no accessor specified) interface, until it is referenced somewhere in the codebase. Often a user declares interfaces and cannot reference them, spending valuable time on tracking down the missing public|protected accessor declaration. While a private interface is not an error, it is a potential error condition that warrants a warning.

Recognition rather than recall The system promotes the concept of information remembering in a number of ways, such as clear and simple dialogs, ability to step back and forward in wizards before committing the action, and context sensitive help. The new dock|float features, plus the dock navigation, makes undocking and docking of windows a breeze.

Flexibility and efficiency of use The solution caters for the novice user, the power user, the mouse biased person and the keyboard shortcut user, with other productivity features such as code snippets as shown. Both inexperienced and experienced users can therefore tailor their integrated development environment to suit their styles and to tailor frequent actions. The toolbars are known and trusted, consistent with previous versions, which is a good feature from a Visual Studio usability point of view. The question that is raised is whether the “Ribbon” concept adopted in many other popular Windows applications, for example Office, should not be introduced for consistency. Give the user the choice, as ribbons potentially clutter the working area for Visual Studio focused users, but would introduce consistency for users working Visual Studio and Office.

Aesthetics and minimalistic design
The new start (home) page is aesthetically pleasing. There is no irrelevant information, it does not appear cluttered as its predecessors and has intuitive images and labels for activities, such as walkthroughs, feedback, etc.

The rest of the system was consistent in terms of showing only the relevant information, hiding the rarely or advanced information, but allowing access to it through “Advanced” links.

All the encountered dialogs were simple and consistent.

Allow the user to easily customise the home page, allowing organisations to create tailored, specialised and focused home pages.

Help users recognise, diagnose and recover from errors Error messages are expressed both in plain message and Windows specific code, as well as reference to where in the code the problem was encountered. The precise indication of the problem, as well as constructive suggestions are huge productivity features.

Help and documentation The offline, online and context sensitive help with Visual Studio is one of the best I have personally worked with, offering easy and comprehensive indexing, searching and supplementary samples and “how to” walkthroughs and videos. The BETA-1 unfortunately is limited to online help, as the offline help will only be reintroduced in the next BETA release.

Custom Heuristics Sufficient information design
The default use of colours emphasises the areas, such as properties, solution explorer and error list, and there is no excessive use of colours or excessive use of the red and green.
New search and filter features, for example ability to search for installed templates, or to filter out noise when working in Options, is a huge productivity feature that allows user to see only the useful and relevant information and focus the attention on the pertinent information.

Easy transition to more in-depth information All detailed and advanced information is abstracted and hidden from the user, avoiding clutter and complexity. Advanced buttons allow the users to drill down to more detailed and advanced information when needed.

Intuitive environment Probably the most important usability feature is an intuitive user interface, which “entices” the user to explore, which the user enjoys using and which is intuitive, both for inexperienced and experienced users. Throughout the test scenario I found the environment intuitive, and familiar based on experience from previous versions.

THE FINAL AVERAGE

59.5/65

De-briefing … findings

The heuristics are focused on usability of the application, in our case Visual Studio Team System 2010 BETA-1. In terms of usability my findings are obviously biased and represent my personal opinion. However, based on the years of Visual Studio, watching the progress and ignoring the obvious issues such as performance and missing features, which are part of any BETA program, I believe that we are looking at major evolution. Not only is the new user interface intuitive, informative and aesthetic, but it is an inviting interface and lots of fun. It entices me to fire it up and write more code … which I guess is a good thing and therefore the BETA-1 passes and receives the evolutionary stamp from me :)

I would, however, like to highlight one feature issue I picked up around the use of the dynamic types and the sequence diagram. The new C#4.0 feature potentially allows us to write code that calls methods on the type, which may not actually exist, resulting in runtime debugging errors and associated adventure journeys. Sequence diagrams in community technology previews (CTP) seemed to agree with my fear of dynamic, as they fail to resolve the final type and find it best to just present an empty sequence diagram to the user. In BETA-1 the sequence diagram is no longer an empty black (or should that be white) hole, but shows the instantiation of the the known type as per the code. The method calls are not shown, which makes sense as those are only resolved at run-time. In summary, be careful when using dynamic keyword and generating sequence diagrams, as the results may not be as expected … depending on your opinion/view on dynamic.

The “humble” Conclusion

So what is my “humble” conclusion? Visual Studio Team System 2010 BETA-1 “rocks” and deserves a second glance by everyone to see what is looming on the horizon and to give the product group constructive feedback.

See you next time for another journey through the wonders of VSTS 2010 BETA-1 …

Heuristics Category	Heuristics	Evaluation Notes	Improvement Suggestions	5 star rating
Nielsen’s Heuristics	Visibility of system status	Visual Studio keeps the user informed about the state of the application through status bar feedback (1), the use of specialised windows, such as the Error List (2) and associated code highlighting, such as the wavy underline to indicate an error in the source code.
	Match between system and the real world	The application speaks the language of Windows platform specific developer, using words, phrases, descriptions and concepts familiar to the user. At first the list of options and properties per option appears overwhelming to the user, but the logical ordering of the options and sub-options allows for rapid adoption and understanding.	Keep history of the settings and options changed by the user, presenting both the complete and a subset of commonly referred to and accessed options. This would allow users to find their “typical” settings much quicker and improve the “recognition rather than recall” heuristic further down as well.
	User control and freedom	User is able to undo and redo most actions, including global impact changes, such as code refactoring. Created a VB, instead of C#, project by accident as I failed to notice that the project creation wizard defaulted to VB. The undo’ing of this step involved manual un’doing and deletion of files, which can be time consuming and confusing to the novice user.	Support the un’doing of a project creation action, including the deletion of all created artefacts.
	Consistency and standards	User interface is consistent with 2008 … look & feel, feature set. The additional features blend into the environment and can be reviewed by the user through exploration. In other words if you know the 2008 environment you will be familiar with 2010, as the look & feel is consistent and as the use of terminology, actions, menu and wizards is consistent with previous releases and with the Windows platform. Some of the project settings (2008 defaulted to AnyCPU, 2010 defaults to X86) have changed and may\|will cause problems with users, especially those that have been working with defaults when creating Team Builds.	Allow user to define the start-up defaults, do not enforce them.
	Error preventions	Visual Studio performs consistency and validity checking while the user edits code. For example when I purposefully deleted the public access modifier keyword from the interface declaration, the system warned that this action would make the interface inaccessible. This pro-actively advises the user and avoids error-prone actions. Actions, such as deletions, present the user with a warning and confirmation option, before committing the action.	The system does not warn the user when declaring a private (which is default with no accessor specified) interface, until it is referenced somewhere in the codebase. Often a user declares interfaces and cannot reference them, spending valuable time on tracking down the missing public\|protected accessor declaration. While a private interface is not an error, it is a potential error condition that warrants a warning.
	Recognition rather than recall	The system promotes the concept of information remembering in a number of ways, such as clear and simple dialogs, ability to step back and forward in wizards before committing the action, and context sensitive help. The new dock\|float features, plus the dock navigation, makes undocking and docking of windows a breeze.
	Flexibility and efficiency of use	The solution caters for the novice user, the power user, the mouse biased person and the keyboard shortcut user, with other productivity features such as code snippets as shown. Both inexperienced and experienced users can therefore tailor their integrated development environment to suit their styles and to tailor frequent actions. The toolbars are known and trusted, consistent with previous versions, which is a good feature from a Visual Studio usability point of view. The question that is raised is whether the “Ribbon” concept adopted in many other popular Windows applications, for example Office, should not be introduced for consistency.	Give the user the choice, as ribbons potentially clutter the working area for Visual Studio focused users, but would introduce consistency for users working Visual Studio and Office.
	Aesthetics and minimalistic design	The new start (home) page is aesthetically pleasing. There is no irrelevant information, it does not appear cluttered as its predecessors and has intuitive images and labels for activities, such as walkthroughs, feedback, etc. The rest of the system was consistent in terms of showing only the relevant information, hiding the rarely or advanced information, but allowing access to it through “Advanced” links. All the encountered dialogs were simple and consistent.	Allow the user to easily customise the home page, allowing organisations to create tailored, specialised and focused home pages.
	Help users recognise, diagnose and recover from errors	Error messages are expressed both in plain message and Windows specific code, as well as reference to where in the code the problem was encountered. The precise indication of the problem, as well as constructive suggestions are huge productivity features.
	Help and documentation	The offline, online and context sensitive help with Visual Studio is one of the best I have personally worked with, offering easy and comprehensive indexing, searching and supplementary samples and “how to” walkthroughs and videos. The BETA-1 unfortunately is limited to online help, as the offline help will only be reintroduced in the next BETA release.
Custom Heuristics	Sufficient information design	The default use of colours emphasises the areas, such as properties, solution explorer and error list, and there is no excessive use of colours or excessive use of the red and green. New search and filter features, for example ability to search for installed templates, or to filter out noise when working in Options, is a huge productivity feature that allows user to see only the useful and relevant information and focus the attention on the pertinent information.
	Easy transition to more in-depth information	All detailed and advanced information is abstracted and hidden from the user, avoiding clutter and complexity. Advanced buttons allow the users to drill down to more detailed and advanced information when needed.
	Intuitive environment	Probably the most important usability feature is an intuitive user interface, which “entices” the user to explore, which the user enjoys using and which is intuitive, both for inexperienced and experienced users. Throughout the test scenario I found the environment intuitive, and familiar based on experience from previous versions.
			THE FINAL AVERAGE	59.5/65