Executive summary
DFAIT usability testing results
Selected websites of Foreign Affairs and International Trade Canada
Prepared for:
Foreign Affairs and International Trade Canada (DFAIT)
Contract number: 08227-068533/001/CY
POR-486-06
Award date: 2007-03-08
Table of Contents
- Introduction
- Method
- Usability test results summary
- Future directions
- Next steps
- Neo Insight contact information
1 Introduction
1.1 Background
The Department of Foreign Affairs and International Trade (DFAIT) has been making a number
of changes to its websites with the goal of creating a unified web presence to better address the
needs of both domestic and international clients. This work has included the implementation of
new designs and navigation on several departmental websites, plus development of a new
information architecture with associated wire frames.
1.2 Objectives
The objectives of this project were to:
- Test the usability of the current design, the proposed information architecture and proposed
wire frames.
- Gather user feedback to provide direction for the future evolution of the websites tested.
- Test the ease with which users can navigate the websites to find information, the suitability
of the link labels and terminology, and the overall appeal of the website designs.
- Deliver results which will help the department refine its overall web presence.
2 Method
2.1 Recruiting
A total of 20 participants were recruited and tested - 5 people in each of 4 audiences testing
different DFAIT content sites:
- General: Adult, international traveller (Voyage website)
- Business: Involved in international trade (DFAIT commerce site)
- University: Policy students/professors (Canada in the World)
- Chinese: New arrivals in Canada (Chinese mission site)
2.2 Remote usability testing
All participants were tested in their own environments (home or business) using remote testing
software (UserVue from TechSmith).
- Screen interactions and participant audio was captured for later analysis
- Participants were asked to try and complete 8 to 9 typical tasks
- Measured ability to complete tasks (with or without hints) and their level of satisfaction
with the way the website supported the task
- Testing took place from March 19th-24th, 2007
- Participants were paid an honorarium for their participation
3 Usability test results summary
3.1 Strengths
The table summarizes the main strengths of the websites and pages tested.
| Website or web page |
Strength |
| DFAIT Home page |
Was perceived as having strong organization; being visually appealing. |
| Assistance for Travellers |
Enables users to quickly accomplish tasks related to frequently asked questions or top countries; "I'm pretty impressed". |
| Services for Business |
Has valuable resources to expand business, market abroad, export, and import. |
| Canada in the World |
Focuses on international policy; holds possibilities for
specific audiences: students and professors, policy
researchers, media professionals. |
| China Mission site |
Offers similarly structured pages in Chinese, English and French. Divides content by audience and location. |
3.2 Scorecard
None of the websites had any Critical usability issues. However, each site had a small number
of Major issues and a larger number of Minor issues, as shown in the table:
| Website or web page |
Major Issues |
Minor issues |
| DFAIT Home page |
3 |
5 |
Assistance for Travellers |
5 |
13 |
Services for Business |
5 |
10 |
Canada in the World |
3 |
13 |
China Mission site |
1 |
17 |
3.3 DFAIT home page - Usability issues and recommendations
Major usability issue
Recommendations
Red headers in navigation bar are not perceived as links.
Redesign main navigation bar to ensure all links are clearly perceived as links and that they show visited status.
Second level pages are often missed due to link confusion.
Explore options for combining functionality of the top 2 levels of page or for preventing skip-over behaviour.
Canada in the World link is not associated with some of its content.
Explore more appropriate link labels or separate out Policy content from some of the other content related to programs and resources.
3.4 Assistance for Travellers - Usability issues and recommendations
Major usability issue
Recommendations
Multiple competing links confuse users and clutter screen.
Provide one, salient entry point to the Emergency information page with cross-links to other related information. Apply similar strategy to passport and visa information.
Critical links and functionality are not noticed on the right side.
Move critical elements to the top of the left navigation bar or top and centre.
Inconsistent link labels cause users to miss important information.
Use consistent term or phrase for content, such as Travel Warnings, to ensure user can follow the path to the desired information.
Cross-links redirect users away from desired information.
Combine Travel Warnings as part of Travel Report so there is no ambiguity and then highlight the Travel Warning as the first thing in the report.
3.5 Services for Business - Usability issues and recommendations
Major usability issue
Recommendations
Confusion between ExportSource and InfoExport.
Use more descriptive names and include previews of the types of information subsumed under each.
Virtual Trade Commissioner feature not understood nor found.
Replace internal jargon with more familiar terminology and focus on what the feature can do for the user.
Look and feel changes constantly with no easy way back to Home.
Explore options to evolve more consistent look and feel with partner sites with cross-links back to the main commerce Home page.
Screen real estate wasted on duplicate links and non-priority content.
Explore design options for providing a single set of links and moving all frequent and critical links towards the top of the page - above the fold.
Links in red bars on main commerce page do nothing.
Make it clear what is an active link and what is not. Track visited links to help user know where they have been.
3.6 Canada in the World - Usability issues and recommendations
Major usability issue
Recommendations
Unclear expectations for primary and secondary navigational links.
Communicate a clear relationship between items in the right hand side, the left hand navigation bar, and items featured in the centre of the main page.
Track visited links in the left-hand-menu by changing their representation according to common industry conventions.
Difficult to find valuable assets such as videos.
Move link to Resources higher up in navigation bar and relabel it with a more meaningful term.
Link to videos and podcasts from Feature Video or Quick Links.
Inadequate support for people arriving from external search.
Let them know where they are, what they can do, how to move up in the hierarchy, and where else they can go.
3.7 China Mission site - Usability issues and recommendations
Major usability issue
Recommendations
Multiple instances of links and mis-matched ordering.
Reduce duplication of links across right-hand, left-hand, and on page content.
If redundant links are necessary, put them in the same order.
Difficulties in understanding how location and services taxonomies interrelate causes confusion and slows completion of tasks.
The left hand menu lists services by embassy - as well as at a top level - creates conflicting taxonomies, making people look for services-then-embassies-then-services.
Organize the taxonomy For Chinese primarily around services.
Consider moving embassy services presently under For Chinese into Presence in China .
4 Future directions
4.1 Refine content to better address user needs
- Track usage information, information requests, search phrases, etc. to understand the priority content people are looking for.
- Work with partners to develop task-oriented information - e.g. steps to take to start
exporting, how to determine import permit requirements, etc.
- Restructure content pages to put the headlines and task-oriented information upfront
followed by more descriptive and support information, rather than the other way round.
- Test labelling for features such as the Virtual Trade Commissioner or ExportSource to
ensure they are communicating to the target audience.
4.2 Refine navigation and user experience
- Explore options for making the home page more of a directory with numerous direct links
to frequent or critical content.
- Explore opportunities to integrate and simplify information, as described in more detail
throughout this report.
- Work with other groups, especially within DFAIT, to achieve a more unified experience
when transitioning from website to website.
- Develop a strategy to better support search - more effective search engine plus content
and metadata changes to optimize performance.
5 Next steps
5.1 Build momentum in defining audiences and tasks
- Establish benchmark usability metrics for key tasks determined by goals of target
audiences and departmental objectives.
- Set goals for improvement and take measures annually and/or after major revisions.
- Review and refine benchmark metrics annually.
- As primary targets are achieved and ROI reduces, extend metrics to secondary
audiences and tasks to improve other aspects of site.
5.2 Verify key terms with target audiences
- Verify terminology and language with the intended audiences.
- Explore alternate terms for Canada in the World.
- Rework link labels so people have accurate expectations of the associated content and
make the first click the correct click.
5.3 Refine the information architecture
- Put the users' most frequent and important tasks front and centre.
- Move content they view as departmental out of their way.
- Reduce clutter caused by redundant links.
- Layer information (headlines, then detail) working with partners.
- Combine common policy headings that people can't discriminate and use the more
specific word eDiscussions.
5.4 Establish common guidelines for page structure
- Adopt some common styles to be used across the sites which will aid users as they navigate from site to site.
- Adopt common conventions for representation of visited links, breadcrumbs, and
indication of where the current page is in the left hand menus.
- Define guidelines for when right hand side features are to be used and for how labels
that are links should behave.
5.5 Extend strengths of individual sites to the other sites
- Extend the Most Popular features on Voyage.gc.ca to other sites.
- Adopt a common approach to offering contact information.
5.6 Improve performance for users who search
- Provide a search box on every page.
- Review and refine search engine performance as part of regular usability testing of
target audiences and key tasks.
5.7 Make access to common or important information more direct
- Help users quickly get at "how-to" information and quickly determine what needs to be done - e.g. forms, approvals, etc.
- Move critical links to primary scanning areas on the left and centre, using the right side
only for related or supporting information.
6 Neo Insight contact information
Please contact us if you have any questions regarding this research:
Prime:
Dr. Gord Hopkins Email:
gord@neoinsight.com
Office: (613) 836-0660
Cell: (613) 292-1972
Fax: (866) 232-6968
Backup:
Scott Smith Email:
scott@neoinsight.com
Office: (613) 271-3001
Cell: (613) 863-5397
Fax: (613) 232-6968