Usability and the Web: An Overview

August 1999

An organization’s Web site is a gateway to its information, products and services. As such, it should ideally be a reflection of the needs of the clients it serves. Unfortunately, Web site design and development is often driven by technology or by organizational structure or business objectives, rather than by user needs. In recent years however, Web site owners and developers have gradually begun to acknowledge and address the issue of usability.

The key to Web site usability is ensuring that the site is both useful and usable for the intended audience.

This paper will review usability issues as they apply to the Web, and provide an overview of different usability techniques that can be used in Web design and development.

The discussion as to what constitutes a " usable " Web interface is ongoing. To a certain degree usability depends upon the purpose and target audience of a particular site. However, there is general agreement that a usable Web interface is one that is accessible, appealing, consistent, clear, simple, navigable and forgiving of user blunders.

Web users are notoriously impatient and fickle – if they are frustrated on a site they will quickly go somewhere else and likely not return. For commercial sites this is critical. A site that is confusing or difficult to use may result in the loss of clients or reduced revenues due to unsuccessful transactions. For informational sites, lack of attention to usability principles may result in users being unable to find the information they need. In recent Web usability studies, users were able to find the correct answers to test questions only 42 percent of the time. ¹

Usability engineering for the Web grew out of the software development discipline of Human Computer Interaction (HCI). However, the Web is different from software, and the nature of the Web poses new challenges to designers and developers who are trying to incorporate usability into their sites.

• Due to the global nature of the Web and the wide ranging demographics of people accessing the Internet, a target audience can be difficult to define.
• Diversity in end user configurations (hardware, software, browsers, connectivity and bandwidth) means that users may have wildly different experiences of the same site.
• Inflated user expectations of Internet technology can be difficult to satisfy.
• The rapidly changing nature of the Web results in short development schedules, making it difficult to incorporate user-centered design techniques.
• Unlike a software package, the user has not made an investment in a particular site, and other options are easily available and accessible.

Before employing any usability techniques, the first challenge is to define the Web site’s target audience. Whether the site is at the beginning stage of development, at the user testing stage, or due for a redesign, it is critical for the designers to have as much knowledge and understanding as possible of the site’s users. The second challenge is then to find actual representative users to participate in design or testing activities.

Identifying a Web site’s target audience is generally very straightforward. A software company’s target audience is developers; a sales company has a marketing profile of their ideal customer; an academic library’s target audience is made up of faculty and students. However, simply identifying a site’s audience does not provide the information about user needs, characteristics and abilities that is needed to create an audience definition.

There are a number of methods of collecting information for audience definition, including focus groups, individual interviews, demographic research and collection of feedback from a pre-existing site.

One of the most common ways used to collect information for Web site audience definition is a user survey. On-line surveys can be posted on a pre-existing site, e-mailed directly to known users, or posted to newsgroups and mailing lists.

An audience definition survey may collect any or all of the following information:

• user profile (demographic information, job or recreational preferences),
• surfing profile (how do they use the Web),
• site usage (likes, dislikes, task requirements), and
• level of technology (hardware, browser type, connection speed).

The survey method is relatively quick and inexpensive but the drawback to this method is that it may not result in a representative sample. For example, the respondents’ may only be those dissatisfied with a current site or sophisticated enough to use an on-line form. For this reason, information gathered in an audience definition survey should ideally be supplemented by other sources.

There are a number of different approaches to finding representative users to participate in site design or testing activities. If the focus is on performing small, " quick and dirty " tests to provide some fast results, acquaintances or colleagues may be suitable. For more formal, extensive or complex activities, the following sources may be used:

• customer lists (including marketing mailing lists or records of sales),
• related organizations or associations,
• e-mail discussion lists or newsgroups,
• conferences and events,
• temp agencies or focus group companies.

4.0 User-Centered Design

The key to building a useful and usable Web site is to involve the user in the development process from the beginning. In recent software development, user-centered design has generally led to improvements in software interfaces and a higher likelihood that the software will actually deliver what the user wants and needs. Computer users are now expecting that Web sites will likewise be easy to use.

4.1 Focus Groups

The traditional focus group consists of a small group of users who discuss issues and concerns, guided by a moderator. The discussion is recorded, sometimes on videotape, and then analyzed. Focus groups can also be conducted electronically, via groupware or informally through e-mail or pre-existing newsgroups.

Focus groups are primarily a market research technique. In Web site design, the best use of a focus group is to determine users wants, expectations or perceived needs.

Focus groups are not a good method for assessing interface usability because they are a forum for opinions only and don’t show how the user would actually interact with the site.

Card sorting exercises involve the user in determining site structure and content organization.

Individual users are provided with index cards containing content descriptions and are asked to organize the content into rational groupings then provide labels or descriptions for their groups. Users may also be given blank cards to create content categories that they feel are missing. The site designers then analyze the different groupings for consistency and develop an overall site structure.

Category identification, category description and category labeling are follow-up activities to card sorting. These activities help to ensure that the content categories, labels and descriptions that have been developed meet user expectations, and alert the designer to changes that may be required.

Category identification involves presenting users with the content groups and labels, and asking them to indicate which label they would expect to " click on " to find each content group.

Category description involves presenting a user group with the content labels and asking them to describe the content they would expect to find behind a particular label.

Category labeling involves presenting users with several labels for some sample content and asking them to choose the most appropriate label.

In wire-frame validation, the data gathered from the card-sorting and follow-up exercises is used to create one or more simple structural models or " wire-frames " of the site. A wire-frame model consists of simple page mock-ups that show the site structure and location of content. Wire-frame models can then be tested by users to determine whether the site structure makes sense from a user perspective.

Because no design elements are involved, wire-frames can be quickly assembled and tested. This allows for iterative development, i.e., wire-frames can be revised and re-tested a number of times until the site structure is finalized. ²

" Greek pages " can be used to provide insight into whether a particular Web page design and layout will be effective in helping users to navigate the site and find content.

On a Greek page all the text has been replaced with nonsense. Users are then asked to see if they can identify specific content elements of the page, such as the page title, main content items or navigation elements by relying solely on the page layout and graphic design.

Usability inspection methods are based on having evaluators inspect a user interface to identify usability problems in the design.

Some inspection methods also rate the severity of usability problems as well as evaluating the overall usability of an entire system. The various methods can be distinguished in two ways:

• the evaluation criteria used by the inspectors, and
• the process they use to reach their judgments.

Depending on the method, an interface may be inspected by one evaluator at a time, or by a group working together. Evaluators are often usability specialists, but can also be knowledgeable developers or users.

Usability inspection methods have a number of advantages:

• Because these methods are based on inspection rather than actual testing, they don’t require a working system. They can be done on prototypes, even paper.
• They can be performed early in the development cycle, and repeated easily at different stages of development.
• They can be integrated easily into many different development methodologies.
• They are relatively cheap to use, and don’t require special equipment or labs.
• An inspection by a usability specialist can be a simple way to transfer usability knowledge to a larger group.
• Expertise is not required to conduct a simple inspection. An effective, basic inspection can be done with as little as one hour of training.
• They provide quick, concrete results.

In the world of Web site design, the most popular usability inspection method is heuristic evaluation. ³

Heuristic evaluation involves having a small group of evaluators (usually three to five) examine an interface and compare it against a set of recognized usability principles called heuristics.

Depending on the circumstances, the site designers and evaluators may choose to use a pre-existing set of heuristics, or may develop a customized set of principles to address specific site issues. ⁴ A search of the Web will turn up literally hundreds of guidelines, standards, and usability criteria that can be used to develop a list of general principles for evaluation purposes. ⁵

In a heuristic evaluation the evaluators work independently, identifying usability problems and matching each problem to the usability principle that it violates. In some cases, scenarios (list of steps the user would take) are used to assist evaluators to understand realistic tasks. When the evaluators are finished all the results are merged, and the problems are rated according to severity.

The drawback to heuristic evaluation is that it does not provide design solutions, it simply identifies usability problems. Also, it does not address the positive aspects of the design. However, the general usability principles can be used as guidelines for fixes or re-designs and a debriefing or brainstorming session at the end of the process may also help to generate design ideas.

Preconceptions that user testing is expensive, difficult and time-consuming prevent many Web designers from doing any testing at all. However, there is general consensus on the Web that doing any user testing, however minimal, is better than no testing at all, and that useful results can be obtained in a short amount of time with a relatively small number of users.

While large, complex corporate sites may use elaborate user testing methods such as usability labs, videotaping, eye-tracking etc., it is also easy to do quick, cheap and simple tests that will give valuable results.

Basic user testing involves having real users performing real tasks while explaining their decisions by thinking out loud. Test users should be representative of the site’s target audience and the test tasks should represent the way users will actually be using the site. Having users give a running commentary as they are proceeding through the test tasks provides valuable insights into the problems that they may encounter.

Jakob Nielsen estimates that an average usability study using traditional methodology takes 39 hours total, including planning and final reporting. ⁶ Simple tests can be conducted in a couple of hours.

Because users must be able to perform tasks, user testing requires that a working system of some sort be in place, whether a prototype of a new site, or a pre-existing site that is being tested for the purpose of a redesign.

Some additional steps can be taken to maximize the effectiveness of user testing:

• If possible, it may be advantageous to test at the user’s site. Observing the user operating the site using their own equipment in their own environment will give a truer picture of the user’s experience of the site.
• Testing is usually carried out on a one-to-one basis to allow the facilitator to closely observe the user’s behaviour. A second facilitator may be useful for recording purposes.
• The inclusion of a second test subject has also been found to produce useful test results. The dialogue between the two subjects encourages thinking aloud. This phenomenon, known as co-discovery learning, is similar to brain storming.
• The facilitator or tester must be able to be objective. For this reason, it is often more effective to hire an outside facilitator.

User testing does not generally produce statistically significant results, nor does it need to in order to be useful. The key to interpreting the results of user testing is to look for general trends and behaviour patterns that indicate problems with the usability of the site.

Once a site has been posted live to the Web, it is useful to continue to collect user feedback to monitor ongoing user satisfaction with the site. The most common electronic methods of collecting user feedback are electronic mail and on-line questionnaires. It is important to remember though, that these methods have limitations:

• Users must have the ability to submit their response, e.g., an e-mail form might require that the user’s e-mail address be configured within their browser or that their Javascript be enabled.
• Users must be willing to respond. Because the respondents are self-selected, correspondence will, as a rule, be very positive or very negative. There will be very few responses from individuals whose attitude toward the site ranges from mildly negative to mildly positive.
• Users must have the appropriate software configurations, e.g., technologies that require a certain type of browser or plug-in will limit responses.
• The location of the feedback mechanism within the site will also affect the response rate.
• A user who is responding to an on-site feedback mechanism will not have the same level of commitment and attention as someone who is participating in a user test. As a result, feedback received through such mechanisms is often too generic to be useful.

Given the limitations of these feedback mechanisms, they are most effectively used as general indications of user satisfaction. Before proceeding with a major site redesign, other usability techniques, such as user testing, should also be employed.

A log is a file to which a server’s transactions are logged. It allows a Web administrator to track file use and server traffic. A log file includes such information as: the date of the transaction, the name of the computer requesting the file and the name and size of the file transferred. Logs may also include information about the user’s browser and previous sites visited. ⁷

Log file analysis is commonly used to track site usage. Increasingly, site administrators will also use the data to draw conclusions about the popularity and/or usability of certain pages or areas of the site.

However, while there are some valid uses of log files, the data can also be highly misleading. For example, a substantial portion of logged transactions is the result of the Web robots or spiders that probe sites for pages to index, usually for search engines. Even when non-human visitors are eliminated, effective log file analysis requires context. To understand user behaviour it is important to know the sequence of actions taken by the user, and whether the information they found was of value to them. Neither is well documented by log files.

Given the unique nature and staggering rate of growth of the Web, it is clear that usability is an issue that is becoming increasingly important for Web site owners and designers to address.

To a certain extent, Web site users expect instant gratification. Successful Web sites then, whether they be commercial or informational, will increasingly be those that meet the usability criteria of being clear, appealing and easy to use.

To meet these objectives, there are many usability methodologies available that can be used quickly, cheaply and easily to provide fast and useful information to improve Web site usability. For best results it is recommended that a variety of methodologies be used and that Web site development and design be treated as an iterative and ongoing process.

Fleming, Jennifer. " User Testing: How to Find out What Users Want. " June 15, 1998.
<http://ahret.com/guides/design/199806/0615jef.html>

Fuccella, Jeanette and Jack Pizzolato. " Creating Web Site Designs Based on User Expectations and Feedback. " Internetworking ITG newsletter. June 1998.
<http://www.sandia.gov/itg/newsletter/june98/web_design.html>

Fuller, Rodney and Johannes J. de Graaff. " Measuring User Motivation from Server Log Files. " October, 1996.
<http://www.microsoft.com/usability/webconf/fuller/fuller.htm>

Instone, Keith. " Conducting Your First User Test. " Webreview.com. May, 1997.
<http://webreview.com/pub/97/05/30/usability/index.html>

Instone, Keith. " Site Usability Evaluation. " Webreview.com. October, 1997.
<http://webreview.com/pub/97/10/10/usability/index.html>

Kalin, Sari. " Mazed and Confused. " CIO WebBusiness Magazine. April 1, 1999.
<http://www.cio.com/archive/webbusiness/040199_use.html>

Kirakowski, Jurek. " Questionnaires in Usability Engineering: A List of Frequently Asked Questions (2^nd Ed.) ".
<http://www.ucc.ie/hfrg/resources/qfaq1.html>

Levi, Michael D. and Frederick G. Conrad. " Usability Testing of World Wide Web Sites. " May 7, 1998.
<http://stats.bls.gov/ore/htm_papers/st960150.htm>

Marketwave. " Web Mining: Going Beyond Web Traffic Analysis. " 1999.
<http://www.marketwave.com/press/whitepaper.htm>

Nielsen, Jakob. " Cost of User Testing a Website. " Alertbox. May 3, 1998.
<http://www.useit.com/alertbox/980503.html>

Nielsen, Jakob. " Guerrilla HCI: Using Discount Usability Engineering to Penetrate the Intimidation Barrier. " 1994.
<http://useit.com/papers/guerrilla_hci.html>

Nielsen, Jakob. " Heuristic Evaluation. "
<http://www.useit.com/papers/heuristic/>

Nielsen, Jakob. " Technology Transfer of Heuristic Evaluation and Usability Inspection. " November 27, 1995.
<http://www.useit.com/papers/heuristic/learning_inspection.html>

Nielsen, Jakob. " Testing Whether Web Page Templates Are Helpful. " Alertbox. May 17, 1998.
<http://www.useit.com/alertbox/980517.html>

Nielsen, Jakob. " Top Ten Mistakes of Web Management. " Alertbox. June 15, 1997.
<http://www.useit.com/alertbox/9706b.htm>

Nielsen, Jakob. " The Use and Misuse of Focus Groups. " 1997.
<http://www.useit.com/papers/focusgroups.html>

Nielsen, Jakob. " Users First: Survey Your Users. " DevHead. February 18, 1999.
<http://www.zdnet.com/devhead/stories/articles/0,4413,2211547,00.html>

Nielsen, Jakob. " User Testing. " DevHead. October 5, 1998.
<http://zdnet.com/devhead1stories/articles/0,4413,2137666,00.html>

Nielsen, Jakob. " What Is Usability? " DevHead. September 29, 1998.
<http://www.zdnet.com/devhead/stories/articles/0,4413,2137671,00.html>

Spool, Jared. " Why On-Site Searching Stinks. " 1997.
<http://world.std.com/~uieweb/searchar.htm>

Yu, Jack J., Prasad V. Prabhu and Wayne C. Neale. " A User-Centred Approach to Designing a New Top-Level Structure for a Large and Diverse Corporate Web Site. " June 5, 1998.
<http://www.research.att.com/conf/hfweb/proceedings/yu/index.html>

__________
Notes

¹ Spool, Jared. " Why On-Site Searching Stinks. " June, 1997. http://world.std.com/~uieweb/searchar.htm

² The techniques of card-sorting, category identification, description and labeling, and wire-frame validation are components of IBM Corporation's Web development methodology and are described in detail in the article Creating Web Site Designs Based on User Expectations and Feedback, by Jeanette Fuccella and Jack Pizzolato, June 1998. http://www.sandia.gov/itg/newsletter/june98/web_design.html

³ For a brief overview of other usability inspection methods, see Jacob Nielsen's Summary of Usability Inspection Methods at http://www.useit.com/papers/heuristic/inspection_summary.html. Constantine and Lockwood's Web Usability Inspections slide show from the User Interface '99 conference also provides a good general summary http://www.foruse.com/Presentations/WebInspectUI99/index.htm.

⁴ The original software development heuristics developed by Jakob Nielsen (with annotations for the Web by Keith Instone) are available at http://webreview.com/wr/pub/97/10/10/usability/sidebar.html.

⁵ An example of a simple set of general Web usability principles is available at http://www.foruse.com/Presentations/WebInspectUI99/sld017.htm.

⁶ See Jakob Nielsen's Alertbox for May 3, 1998, Cost of Testing a Web Site http://www.useit.com/alertbox/980503.html

⁷ For a more detailed discussion of log file analysis, see: Haigh, Susan and Janette Megarity. " Measuring Web Site Usage: Log File Analysis. " Network Backgrounder #57. September 11, 1998. http://www.nlc-bnc.ca/pubs/netnotes/notes57.htm