Usability testing of DFAIT website

POR #: 419- 07
Contract Date: February 7, 2008

Prepared for:
Foreign Affairs and International Trade Canada
Lester B. Pearson Building
125 Sussex Drive
2^nd floor
Ottawa (Ontario) K1A 0G2
barry.nesbitt@international.gc.ca

Contract number: 08170-070584/001/CY

Introduction

• Background
  • Foreign Affairs and International Trade Canada (DFAIT) continues to make changes to its websites with the goal of creating a unified web presence to better address the needs of both domestic and international clients
  • This work includes implementation of new design, navigation and content; plus development of a new information architecture
• Objectives
  • Test the usability of the current design and information architecture plus some proposed wire frames
  • Gather user feedback to provide direction for the future evolution of the websites tested
  • Test the ease with which users can navigate the websites to find information, the suitability of the link labels and terminology, and the overall appeal of the website designs

Method

• Recruiting
  • 26 participants in each of 5 audiences were recruited and tested on
    • General: Adult, international traveller ( Consular website)
    • University: Policy students/professors ( eDiscussion and START )
    • Business: Involved in international trade ( Trade Negotiations )
    • Youth: Between 18-25 years ( International Youth Program )
    • Russian: New arrivals in Canada ( Russia mission site )
    different DFAIT content:
• Remote usability testing
  • All participants were tested in their own environments (home or office) using remote testing software (UserVue from TechSmith).
    • Screen interactions and participant audio was captured for later analysis
    • Participants were asked to try and complete 5 to 7 typical tasks
    • Measured ability to complete tasks (with or without hints) and their level of satisfaction with the way the website supported the task
    • Testing took place from March 10^th-14^th, 2008
    • Participants were paid an honorarium for their participation

For details, see DFAIT Facilitator Guide v1.0 Master 09Mar08.doc

Strengths - Across all areas tested

Scorecard

• Comparison of task completion, satisfaction and usability issues

Consular summary

• Task performance overview
  • 7 tasks tested - 1 Critical and 1 Major task problem which included:
    • Difficulty finding potential issues associated with travelling with dual citizenship
    • Not finding warning information in Travel Report because some people did not scroll
• Usability overview
  • 1 Critical, 6 Major and 10 Minor issues - Critical and Major included:
    • Content missed because it was not where expected
    • Travel Report information missed by some users
      • Country list not visible without scrolling
      • Table of contents overlooked
    • Too many competing and redundant links on the Home page
    • Difficult to discriminate some sections
      • Key Information vs. Essential Resources vs. Quick Links
    • Verbose and unstructured text in some areas

Consular key recommendations

• Key recommendations
  • Re-examine information architecture in a few key areas to ensure content is where it is expected to be
  • Change layout of the Travel Reports country listing to ensure people see the start of country listings without having to scroll
  • Eliminate redundant links from the Home page and organize it into clearly differentiated sections
    • Build on the existing categories of before, during or returning from travel
  • Help people focus on the country of interest and the stage in their travel
  • Provide links to Offices Abroad or Embassies on the Home page
  • Visually integrate content on the right hand side to make it more noticeable

eDiscussion and START summary

• Task performance overview
  • 7 tasks tested - 1 Critical and 5 Major task problems which included:
    • Not able to find START program from main DFAIT Home page
    • Not able to find specific funding details for Global Peace and Security Fund
    • Difficulty locating information on how to submit a Policy Position Paper
    • Difficulty finding the current eDiscussion and using the email alert form
    • Difficulty finding video about Children at War
    • Not able to access specific contact information about Sudan activities
• Usability overview
  • 1 Critical, 6 Major and 9 Minor issues - Critical and Major included:
    • No links to funding eligibility information
    • Title of current eDiscussion difficult to find
    • Policy Paper information not expected under eDiscussions
    • No links from summary of Policy Position Papers to actual papers
    • Annotation of Video Library link is misleading
    • eNewsletter subscription form confusing to use
    • Global Issues not an obvious link to look for the START program

eDiscussion and START key recommendations

• Key recommendations
  • Support key tasks associated with looking for funding information
    • Provide easy access to Eligibility and How to Apply information
  • Highlight the current eDiscussion title more prominently
  • Clarify the relationship between eDiscussions and Policy Position Papers
    • Change the Policy eDiscussions section to Policy Discussions to better accommodate the range of content found there
    • Policy Position Papers, Online Discussions, Video Interviews, etc.
  • Provide cross-links to related information
    • e.g. - Link to How to Submit a Policy Position Paper from the summary listing of papers
  • Make link labels match user expectations regarding destination content
  • Simplify the eNewsletter subscription form
  • Re-evaluate where the START program is located or change the Global Issues link label to something which would better convey the notion of Programs or Canada's Involvement abroad
  • Ensure navigation breadcrumbs are used consistently throughout the site

Trade negotiations summary

• Task performance overview
  • 5 tasks tested - 2 Critical task problems which included:
    • Unable to associate finding information about business travel with Trade Negotiations and Agreements
    • Difficulty finding information about trade barriers
• Usability overview
  • 3 Critical, 5 Major and 9 Minor issues - Critical and Major included:
    • Business travel regulations not expected under Trade Negotiations link
    • Business travel information inconsistent in different parts of website
    • Trade barriers not associated with the link label of Market Access Report
    • ExportSource and InfoExport terms do not communicate well
    • Valuable right hand content and links often overlooked
    • Links often not arranged for ease of scanning, nor in the user's language
    • Search was not working during testing

Trade negotiations key recommendations

• Key recommendations
  • Provide cross-links to business travel regulations on the Services for Travellers pages
  • Ensure consistency of information by taking a more database approach - maintain only one valid source for Business Travel Requirements which may be accessed from several locations
  • Consider changing the Market Access Report label to Trade and Investment Barriers
  • Replace internal terms, like ExportSource and InfoExport, with more meaningful terms
  • Visually integrate information in right hand column with central content
  • Arrange links into bulleted lists so they can be scanned easily
  • Ensure links are provided at the point where the content or resources are described - e.g. Trade Barrier descriptions should link to database
  • Reduce the amount of text and structure content for easy scanning

International youth program summary

• Task performance overview
  • 5 tasks tested - 1 Critical and 0 Major task problems which included:
    • Unable to find link to the International Youth Program from DFAIT home page
• Usability overview
  • 2 Critical, 4 Major and 9 Minor issues - Critical and Major included:
    • International Youth Program not expected under Foreign Policy section
    • Program descriptions provide no links to more details or country opportunities
    • Link to International Job Opportunities in Engineering goes to a generic University website Home page
    • Content being described is often missing links to actually get to the described information
    • Key information missing from French version of How Our Programs Work page
    • Right side navigation is often overlooked

International youth key recommendations

• Key recommendations
  • Provide cross-links from the Travel and Business sections and consider changing the name to something like Work or Study Abroad
  • Link program titles on Program Descriptions page to more specific information about the program and participating countries
  • Allow people to choose a program of interest and then see what countries support it OR choose a country of interest and then see what programs are supported
  • Ensure links to resources are provided at the point when the resources are being described
  • Ensure the same information is available in both official languages
  • Visually integrate right hand links so it is clear that they are related to the central content
  • Relocate the Country drop-down menu to a more noticeable position and change the prompt to Select Country
  • Place related links at relevant points throughout the sub-site rather than grouping them under the vague label Other Links

Russian mission site summary

• Task performance overview
  • 5 tasks tested - 2 Critical task problems which included:
    • Difficulty getting to the Trade area of the website
    • Difficulty determining what documentation is required for moving from Armenia to Canada
• Usability overview
  • 4 Critical, 0 Major and 4 Minor issues - Critical included:
    • Link to Trade area of website is missing from main menu
    • Key links to the Trade area of the site are not visible because they are unexpectedly on the right hand side
    • Not all English menu items are on the Russian-language version of menu
    • Links on the right hand side are not evident to users

Russian mission site key recommendations

• Key recommendations
  • Put a link to the Trade area on the Russian-language version
  • Map menu items exactly between the Russian and English versions to facilitate people learning the corresponding English terminology to use
  • Move key links from the right hand side into the central area and visually tie in the content in the right hand column
  • Put links to the Trade Commissioner service in the central area
  • Ensure all links are evident to the users - underline if necessary
  • Highlight the currently active link on the left hand menu when appropriate
  • Ensure the link labels correspond to the headings on the destination pages

Future directions

• Continue to develop a task-based structure to the websites
  • Structure text and lists of links so they are easy to scan
  • Directly link to the resources being described
  • Examine user's steps in achieving their goals and help them complete each step towards their goal
  • Remove verbose descriptions and allow people to get to the actions as quickly as possible
  • Test all terminology for it's ability to communicate as intended, especially link labels - consider doing a Customer Carewords analysis
    • www.CustomerCarewords.com
• Further improve navigation and user experience
  • Explore opportunities to integrate and simplify information, maintaining a consistent look and feel across the multiple sub-sites
  • Continue to refine the effectiveness of search by managing search based on analysis of search logs and ongoing user testing

Next steps

1. Address the critical and major usability issues from this testing
2. Address typical user experience issues
   1. Put the users' most frequent and important tasks front and centre
   2. Move departmental content out of the way
   3. Reduce clutter by eliminating redundant links
   4. Adopt common style sheets to be used across all sub-sites
   5. Visually integrate the right hand content better with central content
   6. Provide context-sensitive and specific contact information
   7. Provide a search box on every page
3. Consider conducting a full Customer Carewords analysis www.customercarewords.com to optimize terminology, information architecture, and users' task flow management