Publishing and Depository Services Directorate (PDSD)
Client Satisfaction Survey - Report

Ce rapport est aussi disponible en français.

The views expressed herein are those of the supplier/authors and do not necessarily reflect those of Public Services and Procurement Canada.

© Her Majesty the Queen in Right of Canada, as represented by the Minister of Public Works and Government Services Canada

Information contained in this publication or product may be reproduced, in part or in whole, and by any means as stipulated in the Open Government Licence – Canada.

For additional information, contact:
Public Services and Procurement Canada Communications
11 Laurier Street, Phase III, Place du Portage
Gatineau, Quebec
Canada K1A 0S5

Departmental Contact Information: questions@pwgsc-tpsgc.gc.ca

Table of Contents

• 1.0 EXECUTIVE SUMMARY
  • 1.1 RESEARCH PURPOSE AND OBJECTIVES
  • 1.2 STUDY AND ANALYSIS FRAMEWORK
  • 1.3 PRINCIPAL FINDINGS
    • 1.3.1 Flows into the Publications Website
    • 1.3.2 Importance of Government Publications
    • 1.3.3 Satisfaction with the publications.gc.ca Experience
    • 1.3.4 Satisfaction Among Those Who Contacted the PDSD Directly
  • 1.4 CONCLUSIONS
• 2 BACKGROUND AND OBJECTIVES
  • 2.1 PROJECT BACKGROUND
  • 2.2 OBJECTIVES
• 3.0 METHODOLOGY
  • 3.1 QUALITATIVE
    • 3.1.1 Individual in-depth interviews with PDSD staff
    • 3.1.2 Bulletin boards with Library Professionals
  • 3.2 ON-LINE SURVEY AMONG LIBRARY PROFESSIONALS
    • 3.2.1 Target Population
    • 3.2.2 Fieldwork Dates, Completion and Response Rate
    • 3.2.3 Weighting
    • 3.2.4 Questionnaire
  • 3.3 ON-LINE SURVEY AMONG SITE VISITORS
    • 3.3.1 Target Population
    • 3.3.2 Topics Addressed
    • 3.3.3 Fieldwork Dates, Completion and Response Rate
    • 3.3.4 Weighting
    • 3.3.5 Questionnaire
• 4 DETAILED RESULTS
  • 4.1 SITE VISITORS: DIAGNOSIS FLOWS INTO PUBLICATIONS.GC.CA
  • 4.2 SITE VISITORS: PUBLICATION SEEKERS
    • 4.2.1 Information Sought by Publication Seekers
    • 4.2.2 Reasons for Searching Among Publication Seekers
    • 4.2.3 Navigation to and Usage of publications.gc.ca by Publication Seekers
    • 4.2.4 Other Resources Used by Publication Seekers
  • 4.3 SITE VISITORS: NON-PUBLICATION SEEKERS
    • 4.3.1 Information Sought by Non-Publication Seekers
    • 4.3.2 Navigation to and Usage of the Site by Non-Publication Seekers
  • 4.4 IMPORTANCE OF PDSD FUNCTIONS FOR PUBLICATION SEEKERS
  • 4.5 ASSESSMENT OF SITE FUNCTIONALITY BY PUBLICATION SEEKERS
  • 4.6 CONTACT AND SATISFACTION WITH PDSD DIRECT SERVICE
  • 4.7 PROFILE OF THE SITE VISITOR SAMPLE
    • 4.7.1 Gender and Age
    • 4.7.2 Education
    • 4.7.3 Country of Residence
    • 4.7.4 Types of Institutions Represented
  • 4.8 RESEARCH WITH LIBRARY PROFESSIONALS
    • 4.8.1 On-Line Bulletin Board
    • Importance of Government of Canada Publications
    • Library Professionals' Diagnostic of the Current Situation
    • Imperatives as Identified by Library Professionals
    • Solutions as Identified by Library Professionals
  • 4.9 E-MAIL SURVEY OF LIBRARY PROFESSIONALS
    • 4.9.1 Satisfaction with publications.gc.ca and Other Aspects of PDSD Service
    • 4.9.2 Direct Contacts with PDSD and Related Quality of Service Ratings
    • 4.9.3 Usage and Evaluation of Alternative Sources
    • 4.9.4 Profile of the Library Professionals Email Sample
  • 4.10 IN CONCLUSION

APPENDICES

• A: Discussion Guide
  • 1 - PART 1 - INTRODUCTIONS & MORE
    • 1.1 - INTRODUCTIONS
    • 1.2 - A LITTLE BIT ABOUT YOU
    • 1.3 - AREA
    • 1.4 - EXPERIENCE
    • 1.5 - NATURE OF YOUR LIBRARY
    • 1.6 - YOUR ROLE
  • 2 - IMPORTANCE OF GOC PUBLICATIONS
    • 2.1 - IMPORTANCE
    • 2.2 - PUBLICATIONS: OF SPECIFIC OR GENERIC IMPORTANCE?
  • 3 - FINDING AND GETTING PUBLICATIONS
    • 3.1 - THE PROCESS…
    • 3.2 - CHANGES OVER THE LAST YEARS
    • 3.3 - DO YOU KNOW…
    • 3.4 - PERSONAL CONTACTS?
  • 4 - PART 1 ENDS
    • 4.1 - THAT'S A GOOD START FOR TODAY!
  • 5 - PART 2 STARTS
    • 5.1 - LET'S EVALUATE THE PROCESS
    • 5.2 - PROCESS RELATED QUESTIONS
    • 5.3 - HOW EASY/DIFFICULT
    • 5.4 - PROBLEMS AND ISSUES?
    • 5.5 - SPECIFIC ISSUES
    • 5.6 - SUGGESTIONS AND EXAMPLES
    • 5.7 - EXAMPLES FROM OTHER GOVERNMENTS & DEPARTMENTS
    • 5.8 - PART 2 ENDS
  • 6 - PART 3: LOOKING FORWARD
    • 6.1 - COMING CHANGES?
    • 6.2 - "OPEN GOVERNMENT"?
    • 6.3 - OTHER SUGGESTIONS LOOKING FORWARD
    • 6.4 - A FEW WRAP UP QUESTIONS
    • 6.5 - ADVISORY COMMITTEE (INFORMATION ONLY)
    • 6.6 - WE'RE DONE!
• B: SITE VISITOR QUESTIONNAIRE
• C: LIBRARY PROFESSIONAL QUESTIONNAIRE

1 Executive summary

1.1 Research Purpose and Objectives

The Publishing and Depository Services Directorate (PDSD) maintains a central database of Government of Canada publications that is searchable and available to the public. All institutions of the Government of Canada identified in Schedules I, I.1 and II of the Financial Administration Act are obligated to provide their portable electronic publications to PDSD as identified in the Treasury Board Secretariat of Canada (TBS) Procedures for Publishing.

The Publishing and Depository Services Directorate (PDSD) is responsible for cataloguing publications from Government of Canada departments and agencies. The Directorate's collection includes more than 355,000 bibliographic records, with over 235,000 downloadable electronic publications which can be accessed through the online catalogue via publications.gc.ca. The Weekly Acquisitions List (WAL) is generated at the end of each week based on newly catalogued records from PDSD. A URL linking to the WAL is distributed each Monday via the INFODEP mailing list which consists of clients from public, academic, legislative, special, and government libraries to inform them of the newly catalogued titles. In 2016-2017, the publications website had approximately 150,000 visitors a month and approximately 100,000 publications were downloaded on a monthly basis. Visitors can visit the publications.gc.ca website, contact PDSD by phone, my mail, by email or by fax.

The Directorate has been in transition for some time. In 2014, the Directorate moved to an exclusively electronic publication environment. More recently with the introduction of the Web Renewal Initiative, and the Open Government Initiative, many changes have occurred and there is ambiguity as to how Government of Canada information is disseminated and through which platform. In this context, the Directorate has had to adapt quickly to a changing environment.

Given these challenges, the PDSD seeks information and insight from key audiences, notably library professionals and other users of the publications.gc.ca website in order to better position itself for its future role, and to highlight which aspects of its service delivery to focus on. Information is sought in the following key areas for each target audience:

Libraries and library professionals

• How important are government publications to various kinds of libraries, and why?
• How easy or difficult is it for library professionals to locate government publications at present? What issues and problems do they confront?
• Where do library professionals look for Government of Canada publications? Do they consult other websites to find Government of Canada publications?
• How do they assess the dissemination of government publications generally and the PDSD's role in specific? Do they understand the role of the PDSD, and how do they assess its collection, its specific role and the services it provides?
• How do they assess the services and information the PDSD currently provides?
• What concerns do library professionals have about the future, specifically with respect to the changes to government web services and the advent of the Open Information Portal?
• How might the PDSD improve its services and set priorities for the future?

General public visiting publications.gc.ca

• How many visitors coming to publications.gc.ca are looking for a publication, and what other motives have brought them to the site? Do visitors understand what the publications website offers?

General public visiting publications.gc.ca for publications

• Who are the people downloading publications from the site, and what are their specific needs?
• How well does the site and the PDSD generally serve their needs?
• To what extent do these various kinds of clientele value the services provided by the PDSD? Which specific services, features and communications are valued, and why?
• Do these various kinds of clientele find what they are looking for?
• To what extent do these various kinds of clientele recognize and value the services provided by PDSD?
• Where do they look for Government of Canada publications? Do they consult other websites to find Government of Canada publications?
• How might the PDSD improve its services and set priorities for the future?

The methodology employed in this study consisted of both qualitative and quantitative components. Qualitative interviews by telephone were conducted with PDSD staff to gather information from an internal perspective about the nature of the clientele at PDSD and some of their challenges. Library professionals were queried with on-line bulletin boards, followed by a quantitative survey. Finally, visitors to the publications website (publications.gc.ca) were queried via an on-line survey.

1.2 Study and Analysis Framework

One of the immediate and most important findings in this study is that more than half of the visitors to publications.gc.ca and roughly half of those contacting the PDSD via telephone, email and through other more direct channels are not looking for a publication, but rather a variety of other things – information on general topics, government forms, information on products and services and so on. In this light, adjustments to the analysis framework were necessary. Very obviously, the mixed nature of this clientele has obvious implications for quality of service measurement, not least of which is the need to separate publication seekers from the rest. This is necessary because each clientele is looking for different things and as a result would not evaluate the work of the PDSD in comparable terms. Consequently, the analysis of data from the quantitative survey was done with visitors to publications.gc.ca to ensure that each topic was assessed with the appropriate clientele.

Accordingly, the different topics addressed in the study were broken down as follows:

• Diagnosis of who is coming to the PDSD, either via the website or other channels was assessed via discussions with PDSD staff and on the site visitor pop-up survey
• The importance of government publications and other aspects of the PDSD's mission (of maintaining a self-serve capability for example) were assessed via qualitative research with library professionals, and among publication-seeking respondents who answered the site visitor pop-up survey
• Experience and satisfaction with the web experience was assessed with both clientele on the site visitor pop-up survey
• Experience and satisfaction with the core quality of service as it relates to the delivery of publications and related services was assessed in the site visitor pop-up survey among publication seekers, and among library professionals both in the qualitative bulletin board and in the email survey
• Finally, satisfaction with the PDSD's responsiveness and overall quality of service vis-à-vis people who contacted the Directorate directly was covered in the site visitor pop-up survey, with all respondents, although publication and non-publication seekers are analyzed separately

1.3 Principal Findings

The principal findings in this study are summarized as follows and presented in the same order as above:

1.3.1 Flows into the Publications Website

From this study, we have learned that a minority (42%) of visitors to publications.gc.ca indicate that they were looking for a publication, and that among this group, almost a third indicated that they were also looking for other kinds of information. This means many users of the publications.gc.ca site were in fact coming to the site in search of something it does not provide. The group of publication seekers is composed of 34% general public users, another 28% of users representing other government organizations, 20% of users from educational institutions, and another 16% from other sources. The general public makes up a bigger (55%) proportion of non-publication seekers, but also includes sizable proportions of government employees (20%), users from education institutions (13%) and another 9% from other places.

From this basic data, the survey seems to indicate that the various conduits to the site, be they search engines, links and so on are sub-optimal. Although our study gives some indication of what pathways these groups used to find the site, we do not have sufficient or the right data to properly diagnose the source of the problem. Our data does suggest, however, that the problem is being experienced by a wide variety of both experienced and inexperienced users, and that it is most likely attributable to several root causes.

1.3.2 Importance of Government Publications

All audiences who contributed their perspective to this study agree (to only slightly different degrees) that government publications are highly important. Publication-seeking respondents to the pop-up site survey and library professionals indicate as much in their numerical ratings around this question, which in most cases are rated 4 or above on a 5-point scale. The recognition of importance extends not only to publications themselves, but also to aspects of the PDSD mission, notably its self-service focus, maintaining a catalogue and pursuing the digitization of paper publications, maintaining resources necessary to allow users to contact the PDSD directly and so on. Only the maintenance of a weekly listing of publications fell below this threshold, although other qualitative results suggest that it is the frequency of this list that is less important and not the list itself.

Library professionals, for their part, endorse the importance of these aspects even more strongly in the survey, but it is their qualitative perspective that provided the most resonant articulation of the importance of Government of Canada publications and what the PDSD does in their regard: Library professionals deem government publications “unique”, of particular resonance, part of an important historical record, and essential to government accountability, an engaged citizenry and the nation's capacity to learn from the past. Those aspects of the PDSD's mission, notably its work to identify and catalogue publications, its efforts to ensure compliance with Treasury Board directives and its expertise were also deemed highly important, in particular because of the constant evolutions in the digital world. Library professionals articulated a strong and emphatic imperative around the need for continuity in this record, for comprehensive accounting for the entire body of publications, for durable links and change-resistant archiving, and for consideration of challenges to come and long-term planning to ensure that this material is preserved for posterity. The fact that the majority of surveyed libraries catalogue publications and point to the publications.gc.ca URL reinforces the point.

1.3.3 Satisfaction with the publications.gc.ca Experience

Overall satisfaction with the publications website were 3.6/5 among all publication seekers, and dropping to scores in or around 3.2/5 for “in the end I got what I needed”, “the basic search engine capabilities were effective”, and finally “It was easy to find what I was looking for.” These lower scores may reflect the fact that the PDSD doesn't have what the visitor was looking for ( i.e. something other than a document), or that the PDSD isn't in possession of a document that exists, which can happen for a variety of reasons.

Library professionals, for their part, tend to be more generous in their appraisal of the publications.gc.ca web experience (overall satisfaction 3.9/5 from a sample of 116 respondents), and favorably inclined about “I am able to find the Government of Canada publications I am looking for” (3.7/5).

1.3.4 Satisfaction Among Those Who Contacted the Publishing and Depository Services Directorate Directly

Our data shows that 88 respondents rated their satisfaction following direct contact with the PDSD, either by phone (6% of contacts), email (10% of contacts), by regular mail (1% of contacts) or in person (2% of contacts). This group of 88 respondents further divides between publication-seekers (N=49, or 55%) and non-publication seekers (N=39, or 45%). Though this sample base is too small to rely on for anything other than directional findings, the data suggests that the PDSD staff satisfy groups about equally, even though the requests from non-publication seekers are not what PDSD staff are trained for.

Library professionals, for their part, tend to be more satisfied with the quality of service delivered by PDSD staff for requests made directly: 4.2/5 on the “overall level of satisfaction with the quality of services received.”, and 4.1/5 on “the amount of time it took to receive the service”.

1.4 Conclusions

Quality of Service

This study has highlighted the importance of publications to users, and has identified some of the issues and obstacles that the PDSD's clientele faces in their quest for Government of Canada publications. In particular, the study shows that those seeking publications encountered challenges with aspects of the web experience, and with things such as the search capabilities and the overall ease in finding their desired publication. When these users make direct contact with the Directorate, however, they indicate relatively high levels of satisfaction with the personal aspects of the service. All of this suggests that finding a publication and other information in the Government of Canada web environment is a process that can be improved.

Larger Web Structures and Navigation Toward the Publications Site

Although the underlying reasons are not clear, this study has demonstrated that the larger web structures surrounding publications.gc.ca and the various tools and methods that are used to direct users toward the right kinds of government information are sub-optimal. While a diagnosis on how and why this is happening is beyond the scope of this project and the data, it is important to recognize the potential impact of so many non-publication seekers on our estimation of the quality of service the PDSD provides. It has been necessary to carefully differentiate users between publication and non-publication seekers, and to ensure that only those users with an intent to find a publication evaluate the PDSD's service. The root causes of this mixed clientele are not illuminated by our data. That said, the fact that non-publication seekers come from government organizations (20%) and from educational institutions (13%) as well as the general public (55%) suggests that the problem is experienced by all comers. Additional research (which would presumably be part of the government's ongoing efforts to perfect its web offer) on how users of all kinds think of and decode the offer of “publications” relative to the specific topics they seek information on would warrant consideration.

The Importance of Government Publications and Library Professionals' Contribution

Finally, this study has provided rich and compelling evidence from library professionals about the importance of Government of Canada publications and especially of preserving this body of information for posterity. The library professionals who participated in this study, either by way of the bulletin board or the survey articulate similar imperatives:

• Comprehensive capture of existing publications – be they in print form or electronic is important, especially in light of the constant and predictable pace of evolution in information technology and Internet structures.
• Stability in how these files are referenced (links) and stored over the long term is essential.
• Work to ensure that this body of information is more easily and centrally discoverable is important. A general theme to emerge is that a singular source (“one-stop shop”) is ideal, and that the PDSD is, so far, the organization that comes closest to achieving this aim despite its relative imperfections.

Finally, library professionals provided a wealth of suggestions and recommendations on how to better achieve these aims which are accessible in the annex of this report.

2 BACKGROUND AND OBJECTIVES

2.1 Project Background

The Publishing and Depository Services Directorate (PDSD) maintains a central database of Government of Canada publications that is searchable and available to the public. All institutions of the Government of Canada identified in Schedules I, I.1 and II of the Financial Administration Act are obligated to provide their portable electronic publications to PDSD as identified in the Treasury Board Secretariat of Canada (TBS) Procedures for Publishing.

The Publishing and Depository Services Directorate (PDSD) has evolved from dealing almost entirely with printed publications to one responsible for cataloguing all publications from Government of Canada departments and agencies in electronic form. The Directorate's collection includes more than 355,000 bibliographic records, with over 235,000 downloadable electronic publications which can be accessed through the online catalogue via publications.gc.ca. The Weekly Acquisitions List (WAL) is generated at the end of each week based on newly catalogued records from the PDSD. A URL linking to the WAL is distributed each Monday via the INFODEP mailing list which consists of clients from public, academic, legislative, special, and government libraries to inform them of the newly catalogued titles. In 2016-2017, the publications website had approximately 150,000 visitors a month and approximately 100,000 publications were downloaded on a monthly basis. Visitors can visit the publications.gc.ca website, contact PDSD by phone, my mail, by email or by fax.

Challenges Facing the PDSD

The PDSD is facing a number of challenges:

• Evolutions in the services offered: As a result of shifts toward electronic documents, actual physical publishing and printing was removed from the Directorate's responsibilities as of April of 2014. This eliminated the Directorate's responsibilities in ordering, shipping, and warehousing, all of which were once core functions, and brought a new focus on intangible forms of communications. As a result, the Directorate can be said to have shifted from providing tangible products for clients to providing services, functions and electronic products for key stakeholders and partners.
• Evolutions in related government structures: Other important challenges arise as a function of changes to the structures and regimes the Government of Canada deploys to disseminate information, including a migration of the PDSD's data to the new Integrated Library System, the institution of the new Open Information Portal, and substantive changes to the government's web structures. The Government of Canada Web Renewal Initiative is seeing the integration of departmental websites into the common canada.gc.ca website. As a result, departments are losing their individual real estate making it more difficult to locate publications.
• Limited understanding and compliance among departments: Although mandated by Treasury Board requirements to provide all publications to PDSD, departments provide their publications at different rates, in different formats, and with imperfect understanding of new (web accessibility) standards and some do not comply with the requirement which requires PDSD to harvest publications.
• Limited resources (budgets) and high costs associated with making all publications easily available.
• Limited insight into the nature of clientele seeking publications. As the Directorate evolves toward increasingly impersonal modes of communication and web-based services, the nature of its clientele and their needs is increasingly harder to discern.
• As a result of the fast-paced changing Government of Canada landscape related to information management and more specifically publishing, government departments no longer have a clear understanding of their responsibilities associated with publishing and citizens have an imperfect awareness and understanding of the role played by each organization responsible for disseminating information.

2.2 Objectives

The PDSD seeks information and insight from key audiences, notably library professionals and users of the publications.gc.ca website in order to better position itself for its current and future role. Information is sought in the following key areas for each target audience:

Libraries and Library Professionals

• How important are government publications to different kinds of libraries, and why?
• How easy or difficult is it for library professionals to locate government publications at present? What issues and problems do they confront?
• Where do library professionals look for Government of Canada publications? Do they consult other websites to find Government of Canada publications?
• How do they assess the dissemination of government publications generally? Do they understand the role of the PDSD, and how do they assess its collection, its specific role and the services it provides?
• How do they assess the services and information the PDSD currently provides?
• What concerns do library professionals have about the future, specifically with respect to the changes to government web services and the advent of the Open Information Portal?
• How might the PDSD improve its services and set priorities for the future?

General Public Visiting publications.gc.ca

• How many visitors coming to publications.gc.ca are looking for a publication, and what other motives have brought them to the site? Do visitors understand what the publications website offers?

General Public Visiting publications.gc.ca for Publications

• Who are the people downloading publications from the site, and what are their specific needs?
• How well does the site and the PDSD generally serve their needs?
• To what extent do these different kinds of clientele value the services provided by the PDSD? Which specific services, features and communications are valued, and why?
• Do these different kinds of clientele find what they are looking for?
• To what extent do these different kinds of clientele recognize and value the services provided by PDSD?
• Where do they look for Government of Canada publications? Do they consult other websites to find Government of Canada publications?
• How might the PDSD improve its services and set priorities for the future?

3.0 METHODOLOGY

The methodology employed in this study consisted of both qualitative and quantitative components that were executed in the following sequence:

1. Qualitative interviews by telephone were conducted with PDSD staff to gather information from an internal perspective about clientele at PDSD and some of their challenges;
2. Library professionals were queried with on-line bulletin boards;
3. Invitations were sent to library professionals via email to request their participation in the on-line quantitative survey;
4. Finally, visitors to the publications website (publications.gc.ca) were queried via an on-line survey.

3.1 Qualitative

3.1.1 Individual in-depth interviews with Publishing and Depository Services Directorate staff

A series of four informal interviews were conducted with PDSD staff to help the research team learn about how the PDSD functions and to absorb some of the realities of the PDSD's operations. These interviews were conducted over the phone and were not recorded.

3.1.2 Bulletin boards with Library Professionals

A second qualitative exercise was conducted via an on-line bulletin board with library professionals recruited from the PDSD subscriber lists. A total of 16 library professionals participated, 8 in English and 8 in French (although individuals participated to different degrees). The on-line bulletin board was held over a period of 5 days between October 31 and November 4, 2016. An on-line bulletin board involves daily on-line exchanges regarding a set of questions. Participating libraries were able to answer questions, see other participants' answers and comment/build on them.

Our discussions involved the following aspects:

• their perceptions of the role of the PDSD;
• their perceptions of the challenges and imperatives surrounding the archiving of government publications;
• their perceptions of the value of the PDSD's work in this regard;
• their understanding of the PDSD's role in light of evolutions in technology and government website architecture;
• aspects identified as areas of improvement.

A discussion guide and email recruiting letter were developed and submitted for client input prior to the sessions. Copies of the discussion guides are appended to this report in their original language.

3.2 On-line survey among library Professionals

3.2.1 Target Population

All library professionals listed in PDSD INFODEP subscriber list, which includes 580 contact points, were invited by email to complete the survey. Those who had taken part in the bulletin board exercise were excluded from the invitation.

The list of topics addressed in the survey of library professionals was as follows:

• their general use of publications.gc.ca;
• their level of satisfaction with publications.gc.ca overall and on a series of aspects;
• communication with PDSD Client Service;
• their level of satisfaction with the Client Service overall and on a series of aspects;
• the importance of the various aspects of the PDSD's services;
• general questions for statistical purposes.

3.2.2 Fieldwork Dates, Completion and Response Rate

The survey for library professionals was conducted between February 2 and March 5, 2017. A total of 116 library professionals completed the survey questionnaire with a response rate of 20%. The average length of completion was 8 minutes.

3.2.3 Weighting

The results were weighted according to library type and region to ensure that the resulting data sample reflected the nature of libraries as identified by PDSD. The following table shows the distribution of the unweighted sample compared to the distribution of clients in the list. As the table below illustrates, the distribution of respondents in the survey sample was proximate to the distribution of libraries on the PDSD list, and thus required minimal adjustments.

Results of this survey can be extrapolated to the entire population of libraries subscribing to INFODEP, with a margin of error of +/- 8.1%.

3.2.4 Questionnaire

The questionnaire used for this survey is appended (see Appendix C) to the present report. It was written in both official languages and was submitted to PDSD for approval.

3.3 On-line survey among Site Visitors

3.3.1 Target Population

The target population for this part of the study were visitors of the publications.gc.ca site, including people from the general public, private companies, educational entities and governmental organizations. Those who identified themselves as library professionals at the first question of the survey were redirected to the survey for library professionals. A pop-up invitation was posted on the publications.gc.ca website inviting the visitors of the site to complete a survey on their experience using the publicationgs.gc.ca website.

3.3.2 Topics Addressed

Topics addressed were as follows:

• their general use of publications.gc.ca;
• their level of satisfaction with publications.gc.ca overall and on a series of aspects;
• communication with PDSD Client Service;
• their level of satisfaction with the Client Service overall and on a series of aspects;
• the importance they ascribe to various aspects of the PDSD mission and mode of operation;
• general questions for classification purposes.

3.3.3 Fieldwork Dates, Completion and Response Rate

Fieldwork was conducted between February 2 and March 5, 2017.  846 site visitors completed the survey questionnaire. The average length of completion was 5 minutes. Since this was an intercept survey, it is not possible to calculate a response rate. Also, since the respondents elected to participate or not to this survey, the results of this survey can not be extrapolated to the population of publications.gc.ca visitors.

3.3.4 Weighting

Since the profile of the entire population of those who visit the site is unknown, it was not possible to weigh the data to reflect the population of visitors. This data is therefore unweighted.

3.3.5 Questionnaire

The questionnaire used for this survey is appended (see Appendix B) to the present report. It was designed in both official languages and was submitted to PDSD for approval.

4 Detailed Results

A significant part of this research exercise consisted of a survey of visitors to the publications.gc.ca website, and was designed as the principal instrument for obtaining quality of service ratings. This survey also focused on who respondents were and how many were on the site for reasons having to do with a publication. Additionally, a block of questions focused on the importance of what the PDSD does and how well the website serves the needs of the people there. Finally, the survey measured the number of users who make direct contact with the PDSD and how well those people feel they were served in that process. In sum, then, this study was initially conceived and designed to approach quality of service as if it were a relatively straight-forward aspect. That said, interviews undertaken with PDSD staff at the onset of this research project had highlighted that an apparently robust (but not quantified) proportion of visitors to the website were not looking for a publication but rather for some other kind of information. As a result, the survey was adjusted to assess the flows of people into the site, where they come from, who they are, what they are looking for and so on.

Section 4.1 of this report focuses on what our study tells us about these flows and some of the implications therein for PDSD and the quality of service it provides.

4.1 Site Visitors: Diagnosis Flows into Publications.gc.ca

As the table above illustrates, 42% are looking for a publication, whereas a majority (58% are looking for something other than a publication. For the purposes of this report, the former group will be identified as “publication seekers” and the latter “non-publication seekers”, noting that a good proportion of this group of visitors were in fact looking for several things. The publication seekers comprise members of the general public, who in fact make up the biggest sub-group (34%), followed by people representing other government organizations (28%), then by people from educational organizations (20%) and finally “others” (17%)Footnote 1. Non-publication seekers comprise an even larger proportion of general public visitors (55%), and also include some 20% of people from other governmental organizations and 13% from educational institutions.

From this basic data, several observations and/or conclusions can be drawn:

• Very obviously, there is something in the manner that users are pointed to the publications.gc.ca site that is sub-optimal: That the largest proportion of people coming to the site are looking for something other than a publication is symptomatic of issues that cannot be accurately diagnosed in this study.
• Given that non-publication seekers are composed of people coming from both other government institutions and educational institutions, we can surmise that inexperience or lack of “web savviness” are not the root of the problem. The problems contributing to this situation are apparently significant enough to affect users who we might expect to have the sufficient experience and familiarity with web architecture to navigate successfully.
• Finally, the fact that general public are those that are most common among this group of non-publication seekers suggests that whatever the underlying issues, they may be most challenging to users in the general public.

4.2 Site Visitors: Publication Seekers

4.2.1 Information Sought by Publication Seekers

TABLE 2: TYPES OF INFORMATION SOUGHT (BY USER TYPE)

PUBLICATION SEEKERS (N=352)	Total	Type of publication seeker
		The general public	A government organization	An educational institution	Other
Q2) What brought you to our website?	n=352	n=121	n=100	n=71	n=55
Searching for a publication	100%	100%	100%	100%	100%
SUBTOTAL: Other	28%	32%	16%	42%	22%
Looking for general information on a subject	24%	28%	15%	35%	20%
Looking for a government form	4%	5%	1%	3%	7%
Looking for a Government of Canada program / service	4%	8%	0%	4%	2%
DNK/DNA	0%	1%	0%	0%	0%

The previous table shows that among the 352 individuals surveyed who came to the publications website looking for a document, roughly one quarter (28%Footnote 2) also mentioned looking for other things, notably “general information on a subject”, forms or information on a program or service. Although there were no provisions in this study to explore how users define “publications”, this data does suggest that at least this near-third of the intended users are thinking about content in addition to container, or perhaps that they are thinking of “publication” in ways that are not accounted for in search engines or other kinds of web content. A publication is of course a distinct form of information, but it can contain information on just about any topic. By the same token, the very nature of what constitutes a publication is itself in flux: a publication has traditionally taken on a rather stable nature in physical form, but what constitutes an electronic publication is something less stable, and perhaps much less clear to many users. Our survey provides very limited insight on how users think of these terms, but such an understanding would likely contribute to improved information and web architecture.

4.2.2 Reasons for Searching Among Publication Seekers

TABLE 3: REASONS FOR SEARCHING (BY USER TYPE)

	Total	Type of respondent
		The general public	A government organization	An educational institution	Other
Q8) Generally, why do you search for Government of Canada publications?	n=352	n=121	n=100	n=71	n=55
Personal purposes	12%	31%	2%	1%	0%
Professional purposes	59%	37%	89%	35%	80%
Educational purposes	27%	29%	9%	63%	13%
Legal purposes	1%	1%	0%	0%	4%
Other (Please specify)	1%	3%	0%	0%	4%
Both personal and professional purposes	1%	1%	0%	0%	2%
For all purpose	1%	2%	0%	0%	0%
Educational and legal purposes	0%	0%	0%	0%	2%

The nature of publication seekers is revealed in questions about the purposes behind their search. The table above shows that a strong majority (59%) are looking for professional purposes, a proportion that increases to 89% among those coming from other government organizations, and which drops to 35% among people from educational institutions. Respondents of “Other” types appear to be motivated more by legal or professional purposes, likely in some proportion people working in various kinds of businesses.

4.2.3 Navigation to and Usage of publications.gc.ca by Publication Seekers

TABLE 4: PATHWAYS TO THE SITE (BY USER TYPE)

PUBLICATION SEEKERS (N=352)	Total	Type of publication seeker
		The general public	A government organization	An educational institution	Other
Q3) How did you navigate to publications.gc.ca?	n=352	n=121	n=100	n=71	n=55
By directly accessing [publications.gc.ca]	20%	15%	31%	20%	9%
From another government webpage	16%	20%	26%	4%	4%
By using a search engine	56%	58%	36%	70%	73%
SUBTOTAL: Other	7%	6%	6%	4%	11%
Link from a colleague / instructor	1%	3%	0%	1%	0%
Link through email	3%	1%	2%	1%	7%
Through another website / link from another website	1%	2%	1%	1%	0%
Letter/newsletter from PMC	1%	0%	1%	0%	2%
Followed a link sent by someone in the Publication group	1%	1%	2%	0%	0%
Facebook	0%	0%	0%	0%	2%
DNK/DNA	2%	2%	1%	1%	4%

The table above illustrates something of the different pathways that publication seekers took to make their way to publications.gc.ca. The biggest source of traffic to the site – some 56% of publication seekers – got there “by using a search engine”, either one contained on another government site, or a publicly available engine such as Google or Bing. The table also shows that some 20% of respondents in the survey got there by directly accessing the site, presumably because they are repeat users, or because they were made aware of the site by other means. Another 16% of users accessed the site by a link from another government page.

TABLE 5: FREQUENCIES OF VISITS TO SITE AND OF PUBLICATION SEARCHES

PUBLICATION SEEKERS (N=352)	Total	Type of publication seeker
		The general public	A government organization	An educational institution	Other
Q5) How often do you visit publications.gc.ca?	n=352	n=121	n=100	n=71	n=55
SUBTOTAL: Weekly or more	18%	12%	28%	21%	9%
Daily	5%	3%	9%	7%	2%
Weekly	13%	10%	19%	14%	7%
SUBTOTAL: More than once a year but not weekly	48%	42%	45%	54%	64%
Monthly	16%	11%	15%	16%	29%
A few times a year	33%	31%	30%	38%	35%
Once a year or so	6%	12%	4%	1%	2%
This is my first visit to publications.gc.ca	24%	31%	19%	23%	24%
I don’t know	3%	3%	4%	1%	2%
Q6) How often do you look for Government of Canada publications?	n=352	n=121	n=100	n=71	n=55
SUBTOTAL: Weekly or more	22%	10%	34%	27%	22%
Daily	8%	2%	17%	9%	4%
Weekly	14%	8%	17%	18%	18%
SUBTOTAL: More than once a year but not weekly	55%	55%	57%	51%	60%
Monthly	21%	13%	26%	24%	26%
A few times a year	34%	41%	31%	27%	35%
Once a year or so	7%	12%	3%	3%	6%
This is my first time	15%	22%	4%	18%	13%
I don’t know	1%	2%	2%	1%	0%

Finally, our survey shows that first-time visitors to the site are most common among general public respondents, whereas the most frequent users (weekly or more) are found among respondents coming from other government organizations. Respondents from educational institutions are a middling group in both respects – 23% of them were on the site for the first time, whereas 21% are frequent (weekly or more users). This pattern of visiting to the publications.gc.ca matches the frequency at which these groups search for publications: The general public searches least often, government users most often and users from academic institutions at rates in-between the two.

4.2.4 Other Resources Used by Publication Seekers

Given that the survey sample was generated by means of a “pop-up” to users on the publications website, our data cannot shed reliable light on how usage of this site compares to the use of other resources available to potential publication seekers. What the results do show, however, is that “the Internet” (presumably traditional search engines) is the source cited most often (61%), even more than publications.gc.ca (53%). Canada.gc.ca (61%) is the third-most used source, followed by individual departmental websites (50%), LAC (32%) and finally the Open Government website (16%).

4.3 Site Visitors: Non-Publication Seekers

4.3.1 Information Sought by Non-Publication Seekers

Our data shows that most non-publication seekers (55%) have come to the site for “general information on a subject”, a proportion which increases to 64% among those coming from an educational institution. 58% of general public users are looking for general information, as the case for 45% of those coming from another government organization, and 48% for “others”. Other information sought included topics such as a Government of Canada program or service (16%), a form (10%), and then “more specific information or data (6%), translations for terms or to use Termium (2%).

The motives for these searches tend to match the user types: “personal purposes” is the most common motive for this group as a whole (39%), and is the most common among members of the general public (61%). “Professional purposes” is the second-most common motive for this group as a whole (30%), and is the most prominent motive for those coming from other government organizations, and then finally “educational purposes” is the third-most common motive for this group as a whole (28%), and is the most prominent motive for those coming from educational institutions.

4.3.2 Navigation to and Usage of the Site by Non-Publication Seekers

TABLE 6: PATHWAYS TO THE SITE (BY USER TYPE)

	Total	Looking of publications - Type of user
		The general public	A government organization	An educational institution	Other
Q3) How did you navigate to publications.gc.ca?	n=270	n=157	n=45	n=41	n=22
By directly accessing [publications.gc.ca]	22%	25%	16%	22%	14%
From another government webpage	18%	19%	29%	5%	9%
By using a search engine	50%	47%	49%	61%	59%
SUBTOTAL: Other	6%	5%	4%	12%	14%
Link from a colleague / instructor	0%	1%	0%	0%	0%
Link through email	1%	0%	2%	2%	0%
Through another website / link from another website	4%	3%	2%	7%	9%
Letter/newsletter from PMC	0%	0%	0%	0%	5%
Followed a link sent by someone in the Publication group	1%	1%	0%	0%	0%
Facebook	0%	0%	0%	2%	0%
DNK/DNA	4%	5%	2%	0%	5%

The table above shows that search engines are the most common pathway to the publications site (50%), followed by directly accessing the site (22%), linkage from other government webpages (18%), and finally another 6% for a variety of reasons.

TABLE 7: FREQUENCY OF SITE VISITATION (BY NON-PUBLICATION SEEKING USER TYPE)

	Total	Looking of publications - Type of user
		The general public	A government organization	An educational institution	Other
Q5) How often do you visit publications.gc.ca?	n=270	n=157	n=45	n=41	n=22
SUBTOTAL: Weekly or more	13%	10%	11%	22%	14%
Daily	3%	3%	0%	5%	9%
Weekly	10%	8%	11%	17%	5%
SUBTOTAL: More than once a year but not weekly	40%	40%	51%	37%	32%
Monthly	14%	15%	16%	15%	9%
A few times a year	26%	25%	36%	22%	23%
Once a year or so	6%	7%	7%	0%	0%
This is my first visit to publications.gc.ca	35%	36%	29%	34%	50%
I don’t know	7%	8%	2%	7%	5%

Finally, the table above illustrates that only 35% of non-publication-seeking visitors to the site are first timers, whereas some 13% come at a weekly or more frequent rate. This suggests that people who have used the site in the past have perhaps not grasped its specific focus on publications, or perhaps understand the term “publication” more broadly than it is defined by the PDSD.

4.4 Importance of Publishing and Depository Services Directorate Functions For Publication Seekers

In its efforts to enhance and preserve long-term access to Government of Canada electronic publications, the Directorate maintains an online searchable catalogue and publishes a Weekly Acquisitions List. Publication seekers are also fortunate to have PDSD staff who can assist them by phone or by email if needed. The survey results show that most of these are judged to be important by publication seekers, albeit with varying levels of endorsement from different kinds of users.

The data supports the following observations and conclusions:

• Users coming from educational institutions are the most likely to endorse the importance of these functions and characteristics, and do so at statistically-significant higher levels. This finding is consistent with what we heard from the on-line bulletin board discussions with library professionals. Educational institutions are places where organizational culture and vocation contribute to greater sensitivity to the imperative, difficulty and benefit of maintaining documented knowledge. Our qualitative discussions also demonstrated that library professionals have a clearer understanding of publications, which likely contributes to clearer expectations about what the PDSD's responsibilities and limits are.
• Among the different aspects assessed, maintaining self-service access to Government of Canada publications was deemed the most important (a mean score of 4.6 out of 5), followed by ensuring long-term access (4.5/5), ensuring an ability to access older publications in digital format (4.3/5), which suggests support for the digitization of publications that were issued only in print form. The levels of endorsement around these service imperatives were generally consistent across different user types.
• Other aspects deemed important, but not quite at the same levels are having a single access point for all Government of Canada publications (4.0/5), ensuring access to information about publications not available in digital form (3.9/5) among all of those who came to the site seeking a publication, and increasing to 4.1 among those coming from educational institutions, and finally having the ability to contact staff who can assist in locating a publication (3.7/5), and again, increasing significantly among those coming from educational institutions.

Finally, among these dimensions measured for importance, we see that having a listing of publications published in a given week is the one deemed relatively least important (3.0/5, with slightly higher levels among those coming from educational institutions (3.3/5). That said, it is reasonable to conclude that it is the (weekly) frequency of this publication that is relatively less important, and not the listing itself. Again, this is a view that was echoed in our qualitative research sessions with library professionals who were consistent in their estimation of the value of such lists, but are often inclined to consult them at lower frequencies. This reflects both confidence in the PDSD's effectiveness in catering to its mission, but also the possibility that demand for these documents is ebbing somewhat relative to earlier years.

4.5 Assessment of Site Functionality by Publication Seekers

One of the several aspects of the PDSD's service delivery assessed in this survey is the adequacy of the publications.gc.ca website. As earlier sections of this report demonstrate, however, more than half of the people surveyed in this exercise were not people looking for publications, but rather people looking for other things but who ended up on the site by error. The reader should note here that only the ratings of publication seekers are cited here.

The data support the following observations and conclusions:

• Overall satisfaction with publications.gc.ca website was good (3.7/5) for the total sample, but also indicating that there is room for improvement. This rating level is consistent across type of respondent and among those searching for different reasons, and varies only slightly across other dimensions such as gender, age, and so on.
• Something slightly higher than a majority agrees (somewhat + strongly) that “in the end I got what I needed”. This assessment was highest (60%) among visitors coming from other government organizations, and dropped to 47% among the general public visitors. The mean score on this 5-point scale was 3.2, and remains consistent (with no statistically-significant variation) across type of respondent and among those searching for different reasons.
• The data suggest that some of this mitigated endorsement of the publications.gc.ca website functionality may be attributable to the search engine, as two dimensions – “The basic search engine capabilities on the website are effective” and “It was easy to find what I was looking for” are the two lowest rated aspects of the site (3.3/5 and 3.2/5 respectively). Here too, these ratings do not appear to vary across user types or motives for searching.

4.6 Contact and Satisfaction with Publishing and Depository Services Directorate Direct service

While PDSD's main function is to deal with publications – their identification, cataloguing and listing on publications.gc.ca and so on – the Directorate is also organized to answer direct requests from people who have visited the site. These requests may come in several ways: telephone calls, email, regular mail or in-person visits. The staff who are tasked with handling these requests are nominally experts in government publications and there to help people identify and locate publications that are presumably not available on the website or not easily found. Our survey shows that 19% of publication seekers (N=49) respondents made some form of direct contact with the PDSD, and a little more than half were looking for publications. The reader is cautioned that the discussion that follows should be treated with some discretion given the small sample sizes obtained.

That said, the data shows that almost 42% of those making entreaties to the Directorate are from the group of non-publication seekers. This of course is an important factor in the assessment of the Directorate's quality of service performance: It highlights the plain fact that PDSD staffers who are trained for one thing are in fact dealing with a substantial number of requests that are outside of the bounds of their mandate and their expertise. We may also presume that dealing successfully with these extraneous requests (or at least to the satisfaction of the originating person) requires additional time, effort and different skills than would be used for finding publications. Finally, to the extent that these premises are valid, we may also presume that dealing with these non-publication requests requires the PDSD to expend more resources than might would otherwise be necessary.

In fact, interviews that were conducted with PDSD staffers as part of this research study lend credence to these premises. Although staffers were unable to quantify just how many people they deal with are looking for other things, they were quite clear about having to deal with a lot of people and many varied requests: Their experience is that these non-publication seekers are highly heterogeneous in provenance, and looking for information that cuts across a large swath of topics associated with government – things as varied as lost pension checks, inquiries on passports, agriculture-related requests and the like. The variety of these requests, and the fact that so many of them end up in the hands of the Directorate is likely a direct by-product of transitions in the canada.gc.ca architecture, and perhaps of the generic quality of the term “publications”. This term may simply be recognized as something of general relevance to any kind of information in an otherwise less than limpid web environment, or perhaps standing out as the most promising of “leads” presented in search engine results.

The data from the survey supports the following observations and conclusions relative to how well PDSD satisfies these two groups:

• Some 42% of the people surveyed who reported contacting Government of Canada Publications were from the set of individuals who were on publications.gc.ca for a reason other than one of finding a publication, and the remaining 58% were in fact looking for a publication. While both groups are generally satisfied with the quality of service they received, it is perhaps not surprising to note that it is those “publication seeking” individuals who report the higher level of satisfaction (71% satisfied, among which 31% were “very satisfied”.) Clearly, this should be expected given the expertise, mandate and resources available at PDSD. Non-publication seekers were only slightly less satisfied (67% satisfied, among which 21% were “very satisfied”.)
• The survey data with respect to satisfaction levels with the amount of time it took to receive service shows that the publication-seeking group was the more satisfied, although in this case by somewhat larger differentials than reported above: 67% of publication-seekers were satisfied (20% “very”), whereas 59% of non-publication seekers were satisfied (15% “very”).
• Much the same kind of pattern is seen with respect to the amount of time it took to receive a response to questions or comments: The data shows that the publication-seeking group was the more satisfied: 65% were satisfied (18% “very”), whereas 54% of non-publication seekers were satisfied (15% “very”).

The survey's data about the communication between the PDSD and its clientele suggest that it is doing a good job with both groups:

• Both groups are roughly similar in their estimation that communications with Publications Canada were effective (65% agree among publication seekers and 67% among others).
• Both groups were also roughly similar in their estimation that they obtained clear information (74% agree among publication seekers and 72% among others).
• Both groups are generally similar in their satisfaction with ease of access to the service (59% agree among publication seekers and 62% among others).
• Both groups deemed the service personnel as respectful (82% among publication seekers, 72% among those seeking other things), and knowledgeable (69% among publication seekers, 64% among those seeking other things), although for both measures, the publication seeking crowd was more likely to “strongly agree”, as we might expect given the expertise of PDSD staff, and the presumably more frustrating outcome of being told that one is on the wrong site.
• Finally, the survey shows that the PDSD's personnel seems to convey to both groups in near equal measure that they understand their needs (67% among publication seekers, 62% among those seeking other things), and in ultimately providing what the clients need (61% among publication seekers, 59% among those seeking other things), again with publication seekers being more likely to “strongly agree”.

In summary, then, this data illustrates that while the PDSD is dealing with requests for help and information across a breadth of requests far broader than its mandate, it is doing so with apparent intent to answer to all. The data also suggests that tangible gains in satisfaction and overall quality of service may be had if the “overload” of extraneous requests directed appropriately.

4.7 Profile of the Site Visitor Sample

Beyond the features of the sample profile already discussed in this report, there are additional and noteworthy things to note about who answered our survey as these provide some insight into the nature of the PDSD's web clientele:

4.7.1 Gender and Age

Respondents over 65 years of age are more prominent among general public respondents than other categories, constituting 25% of that group. That group is also composed of significantly more men (54%) than the total sample (47%). Not surprisingly, younger respondents between 18 and 34 years of age are more prominent among users from educational institutions (44%) than the total sample (20%), suggesting that this group of “educational users” may in fact comprise many students.

4.7.2 Education

The profile of respondents in terms of education levels is unremarkable except with respect to the differences observed between those looking for a publication and those looking for something else. The group of non-publication seekers has more than twice the proportion of people with an elementary or high school education (19%) than among publication seekers (7%). This suggests that the problems contributing to the large numbers of people ending up on the publications website for the wrong reasons may be in part due to deficiencies of the basic readability and clarity of the superseding information architecture.

4.7.3 Country of Residence

The distribution of respondents living in Canada is unremarkable. Canadian users come from all parts and provinces of the country in roughly the proportions we would expect. The largest group resides in Ontario (35%), and 22% reside in Quebec. The “West” (including Manitoba, Saskatchewan, Alberta and BC) constitutes some 24% of respondents. More interesting perhaps is the some 11% of the sample that resides outside of Canada, among which the largest group resides in (unsurprisingly) the United States (4%). Other regions of the world represented include Europe/Russia (3%), Central/South America (2%), and then countries in Africa (1%). No noteworthy distinctions were found relative to the difference between country of residence and country of workplace.

4.7.4 Types of Institutions Represented

Finally, our sample of people identifying themselves from “other government institutions” is composed primary of federal government employees (77%), followed by some 12% from provincial government and some 7% from municipal governments. Educational Institution respondents are primary from the university level (61%), followed by college (14%) and high schools (6%). The group identifying themselves as “other” kinds of educational organizations, presumably vocational training centers and the like represent a robust 17% of this group. As mentioned earlier, some of these proportions may be skewed by students ascribing themselves subjectively to educational institutions when in fact they would have been more accurately described as members of the general public.

4.8 Research With Library professionals

Given their role and their relationship with the PDSD, a substantial part of this study involved research with library professionals. This phase of the project began with an on-line bulletin board discussion followed by an email quantitative survey of generally similar structure and orientation as the site visitor pop-up survey. Although these two interventions covered common ground, each was designed to shed light on different aspects of the PDSD: the bulletin board focused on library professionals' perceptions of the role of the PDSD, the challenges and imperatives surrounding the archiving of government publications, and perceptions of the value of the PDSD's work in this regard. Additional attention was given to how they understood the PDSD's role in light of evolutions in technology and government website architecture, and aspects that they thought were particularly problematic or improvable. In sum, the bulletin board exercise was designed to approach the PDSD's role and performance from a qualitative perspective, providing room for exploration of themes and the identification of unknown issues that are impossible in quantitative surveys.

The follow-up quantitative email survey was intended to assess the PDSD's service delivery and the satisfaction of this important stakeholder community in generally the same manner as the site visitor pop-up exercise.

4.8.1 On-Line Bulletin Board

Participants in the bulletin board were recruited from lists provided by the PDSD and selected in an effort to ensure the participation of library professionals from a broad cross-section of the country. Participants came from all over the country and a participated in either a French or English board. Among the 18 library professionals that participated in this exercise, the biggest sub-group represented educational institutions (a little more than half), with the remaining from public libraries (4) and institutional organizations for the remaining three. As a group, participants in this exercise were all highly experienced, ranging from a minimum of 5 years to more than 30 years working the spectrum from more supporting to more leadership roles.

Importance of Government of Canada Publications

Among several themes to emerge from this discussion is how critical government publications are. Although participants occasionally reported that Government of Canada publications were not a high priority for their institutions (depending on the institution's clientele and mission), virtually all of them believe that this body of knowledge is highly important. Two broad notions underpin this view – one having to do with the nature of government-generated information itself, and a second that reposes on recognition of the rapidly changing, highly fluid nature of information in the modern world.

The importance seen in government-generated information was described as follows:

• Government publications constitute “a unique source of reliable, accurate, relevant primary information” simply because government has higher base credibility and competence than many if not most other sources. As another put it: “A government document collection is an information source that stands on its own, as unique in its coverage as books, journals, maps or DVD's.” And another: “an authoritative and primary source which supports teaching and research activities in our institution.” And finally, another: “Official publications are a unique and important source of information.”
• Government publications (carefully and consistently archived over time) provide a continuum of knowledge and insight that is rare and valuable. The record provided by maintained government publications helps shed historical light on current issues and enhances the capacity of citizens and governments alike to learn from the past. Again, in the words of one of our participants: “Government of Canada publications contain innumerable, reputable studies – [a] primary source of information – crucial for anyone (school or citizens) who needs to research how public policy came about and/or what is/what was the ‘state of this country' in any given time frame.”
• Government publications “help keep governments accountable and foster an engaged citizenry.” In the words of another, they constitute “a foundation of a functioning democracy” by centering debate and discourse around commonly accepted dimensions and facts.

Finally, government information was deemed important not only because of its quality and objectivity, but because it is threatened by broader changes in technology which tend to affect both user perceptions and its accessibility:

• From the perspective of library professionals, the Internet's expansion of access to information has not been accompanied with an equal increase in users' discernment about the quality of that information. This, in turn, has increased the public's tendency to favor sources of information that reflect their biases and presumptions. In this context, government publications – and their presumed impartiality are especially important, providing an increasingly rare source of accepted facts.
• Technology itself is driving constant evolution in how information is stored and rendered accessible. Yesterday's paper/printed publications gave way to PDF files and HTML documents and so on. Over-arching structures – particularly in government – are constantly evolving, instituting profound changes in how publications are made accessible. Such changes are apparent in the Government of Canada websites, where documents once stored and accessible via individual departmental websites are moving to more centralized portals, which in turn are evolving side by side with alternatives such as Open.Canada.ca.
• Cataloguing of information, which once was based on certain kinds of indices and made accessible in certain media is giving way to new cataloguing tools, available in different media and based on indices that themselves evolve over time. In this context of relentless evolution, the very accessibility and continuity of information – and especially of historical information is threatened. As one participant put it: “This information is a basic primary source collection, but being online has made it in some ways more difficult to find and use than it was when the print volumes were in the basement of the library.”

One library professional provided a summation of all of this in something of a lament:

Flash-forward, to the age of the InterWeb, and I find that now that everyone believes he or she is a "Master Googler" - AND - that it is all on the web - that anyone can find the info from present day and the past (HA!!!) and why keep any tangible collections OR gov info specialist??!

I would also add that the general impression I used to get from people who did not work with this material (i.e. when we received and maintained tangible collections), was a general lack of respect for the collection; it was treated lightly as something "not really worth cataloguing" and/or physically maintaining. I would get into "discussions" with the Head of Cataloguing about why I wanted items catalogued or kept, finding myself justifying the collection to the one person who never worked with researchers who needed the information within! What the belief generally was, though, was that "the only good gov doc is a current gov doc", which is just SO wrong!

As mentioned, keeping a collection back through the years, gives our researchers a sense (and proof) of what governments said and did, policy-wise. It's all there!!It is really very depressing for me... I LOVE this body of work, and so respect the huge quantity of knowledge and know-how I've had the opportunity to bump into over many years, within the practitioners who worked within these collections; most of those people have retired - or been retired - along with the physical gov doc collections they looked after. Now, I see the occasional forlorn call on gov info listservs in Canada and abroad, "Uh, does anyone have this title left in their collections, or know where we can find it???!"

Library Professionals' Diagnostic of the Current Situation

Participants in the bulletin board were generally similar in their assessment of the current situation regarding Government of Canada publications, highlighting generally similar issues and potential solutions. The issues emerge primarily as follows:

• A proliferation of sources, in different forms, formats and channels that are ultimately hindering accessibility. Government of Canada publications can be found via the PDSD's weekly acquisition list, via Library and Archives Canada (LAC), AMICUS (a free national catalogue listing the holdings held at Library and Archives Canada), GALLOP (the Government Legislative Libraries and Publications Portal), and various other tools. The sheer number of these is seen as an issue. One participant described these as a “patchwork of sources.” Another stated “The proliferation of online sources has both helped and hindered this process. It can be challenging to find stable sources for some publication; online publications on department websites frequently disappear, whether due to a URL change or because they have been removed from the site.”
• The instability and insufficiency of the above sources which threaten both immediate access and long term preservation: None of these sources are viewed as comprehensive, and all are subject to evolutions that result in unstable links and occasionally lost documents. One library professional discussing how and when they choose to include a publication in their collection put it this way: “If we add a government publication to our catalogue, we want to be sure that the link will still work in the future. We don't have time to constantly check and update broken links. We're also concerned that items sometimes disappear altogether from government websites. We want to be assured that these works will continue to be available for future users.”
• Threats to permanent and historical collections due to pressures on space, staff and evolving web structures that contribute to the potential loss of historical records: Libraries that were once “selected” to keep parts of the physical collections are under pressure to reduce these collections even though their long-term preservation isn't yet assured. One participant noted: “It is somewhat (very) depressing that in many academic libraries, many of us are feeling pressure from those who don't work with the material, to just "get rid of it". The beauty and breadth of information and history in these documents is going to be lost soon - along with people's knowledge of it.”

Imperatives as Identified by Library Professionals

From our bulletin board discussions, it is possible to synthesize the main imperatives as seen by library professionals under the broad themes of continuity, comprehensive visibility and stability, and finally planning.

Continuity

Given the importance of government publications and the evolving conditions in which they are preserved, continuity emerged as an important imperative. In this context, continuity implies ensuring that publication records, collections and catalogues encompass all previous forms of publications in taking steps to ensure that they can be identified in their proper context. Accordingly, this would imply a digitization of analog forms of publications, but also steps to ensure that collections or historical documents currently preserved on now defunct media (such as floppy disks) are not lost. This was deemed particularly important for publications of a statistical nature and serials.

Students and teachers are able to quickly find what they need on the Internet with a Google search. Unfortunately, this new way of searching has significant failings. In addition, several departments including Statistics Canada have stopped publishing in paper and now only produce electronic versions. In many cases, the digital versions no longer match the printed versions. This lack of continuity is deplorable.

The lack of continuity is mostly with Statistics Canada, where they provide access to electronic tables and serials without sufficiently establishing context. The printed versions made it easier to understand the statistics, which was often a starting point for better research and an easier path to locating reference material in a library.

Comprehensive Visibility and Stability

The library professional we exchanged with had a great deal of concern about stability in the various links, platforms and so on that make up the Government of Canada's digitally accessible records. Issues with stability arise when collections are moved from one platform to another, when policies and procedures evolve because of the emergence of new technologies (among other pressures), or when web architectures or information storage modes evolve. Concerns about comprehensive visibility emerge as collections evolve, are transformed and moved from one platform to another. There is concern that search engines, catalogues and any other sort of consultable record be complete in its capture of government publications.

Federal websites do not maintain access to publications, if they even get posted. They no longer provide publication lists as they did in the past. Where libraries have disappeared the staff in federal departments seem to know nothing about the publications their department has published. There seems to be no one keeping track and making sure publications get transferred to LAC or PDSD. I have often had to explain to federal government staff what these entities are and why older documents should not have been discarded but transferred to LAC. With online publications, it is even worse.

With its heterogeneous nature in format (microfiche, microfilm, print, Compact disc (CD), PDF, HTML and etc.), I have to search many places to find government publications and still not sure whether I have explored all options. Finding serial government publications over a period of time can be even more time consuming because they usually located in different places. With the switch to electronic publishing, secure access to government publications is a big concern.

Planning

Another tangible imperative to emerge from the bulletin board discussions was that the Government of Canada do a better job of planning so that future changes to web structures, departmental responsibilities, search engines and the like so that these other imperatives – stability, visibility, comprehensiveness and so on are more reliably achieved.

It would be nice, though, to have an annual report (or similar) to give us an idea of the scope of the current operations, who's who, and what the plans are for the next year. Some major changes to the platform and searchability have happened recently with very little communication through the INFODEP list, except in response to specific problems that have arisen.

Does anyone in the federal government have a handle on what has been (or is being) digitized within departments?

I don't think we have developed a coherent national plan, either within government or within the library community, for long-term collection and discovery of publications and other federal government information.

Solutions as Identified by Library Professionals

Finally, library professionals were queried about potential solutions to the various issues discussed, and whether there were examples from how other organizations managed access to digital publications that might serve as inspiration for the PDSD. The most prominent common themes to emerge were as follows:

• A “one-stop shop” – a single place where all of the government's publications can be found, hosted on one organization with the clearly mandated responsibility to ensure that the collection is comprehensive, easy to search, complete in terms of records and metadata, and stable over the long term. There was a general consensus that the PDSD was the government organization closest to achieving this centralization.
• Better search engine capability: Several library professionals mentioned that the current search engine can be frustrating to use. This could be in part due to varying needs from one library professional to another.
• More comprehensive digitization of existing physical publications, which was described as an increasingly important imperative in light of the moving of individual department collections off of the departmental websites and the advent of the Open Canada portal.

4.9 E-Mail Survey of Library professionals

4.9.1 Satisfaction with publications.gc.ca and Other Aspects of Publishing and Depository Services Directorate Service

The table above shows how library professionals assessed the adequacy of the publications.gc.ca website and other aspects and tools made specifically for them. Overall, what we see is that while the web-based self-serve capability is valued and appreciated, there remains room for improvement. More specifically…

• Overall satisfaction with publications.gc.ca website was good (3.9/5) for the total sample, but also indicating that there is room for modest improvement. This rating level is highly consistent across type of library, region and size of the population being served.
• A more general indicator of the website framed in the simple assertion “I'm able to find the Government of Canada publications I'm looking for” . The mean score on this question was 3.7 on our five-point scale, which also remains largely stable – increasing slightly among library professionals based in Quebec (4.1). Publishing and Depository Services Directorates' collection includes any publications received by authoring departments. In some cases, there may not be publications on a particular topic or it may be that PDSD has yet to receive a particular publication for inclusion in the collection. It is therefore difficult to gauge under what circumstance the client was unable to find the publication they were seeking.
• Generally, library professionals find the “weekly acquisition list useful”, with the total sample agreeing at 61% (mean score of 3.8), and again, a rating that remains generally stable despite slightly higher levels of endorsement among those representing special libraries (4.1).
• Most library professionals (76%) agree that “the links provided by the PDSD are reliable” (mean score 3.9) – although we note that the reliability of these links is slightly higher among library professionals that simply catalogue publications by pointing to the PDSD's links.
• Level of agreement with the statement “The bibliographic information in the catalogue meet my needs” is 3.7/5 (62% agreeing) for the total sample, but drops somewhat among library professionals representing academic institutions (3.5/5, 59% agreeing) suggesting that there are needs there that are perhaps not being addressed or that there are issues that are simply being encountered more often by library professionals from academic institutions. Our bulletin board discussions suggest that some of these issues may be related to gaps in coverage across certain years, coverage prior to 1990, the sense that few in the Government of Canada “have a handle on what has been (or is being) digitized within departments”, and the belief that “the PDSD catalogue doesn't include metadata from all of the past checklists, and that there isn't a Government of Canada initiative to systematically digitize all of its past published output.” A slightly higher level of appreciation is noted on “The quality of catalogue records is acceptable” (3.9/5, 70% agreeing). Again, these scores suggest cataloguing that is valued, is adequately constructed and maintained, but that could also be improved.

As was the case for members of the general public on the pop-up survey, the data suggest that the publications.gc.ca website's search engine functionality- presents room for improvement. “The basic search capabilities on the website are effective” gets a score of 3.6/5 (70% agreeing) from all library professionals, and 3.5/5 (56% agreeing) for “It was easy to find what I was looking for”. These ratings do not appear to vary across library type, region, size of community being served, language and so on. Finally, we see that for “In the end, I got what I needed” from the site was rated in kind: 3.7/5 with 72% agreeing, which suggests that improvements are possible.

4.9.2 Direct Contacts with PDSD and Related Quality of Service Ratings

As the following chart illustrates, a majority of library professionals surveyed have had no direct contact with PDSD, even though the vast majority (93%) use publications.gc.ca as a resource at least on occasion. The data shows that library professionals from academic institutions are proportionately the least likely to make contact, but the most likely to use email as the medium for doing so, followed by “through InfoDep” (8%) and by telephone (5%):

TABLE 11: DIRECT CONTACT WITH PDSD

	Total	Type of library
		Academic library	Public library	Special library
Q11) In the past 12 months, have you contacted Government of Canada Publications in any way?	n=115	n=53	n=33	n=29
I had no contact with Government of Canada Publications (publications.gc.ca)	64%	68%	58%	67%
By telephone	5%	5%	6%	6%
In-person	1%	0%	0%	3%
By e-mail	22%	28%	17%	16%
Through InfoDep	7%	8%	5%	6%
I do not remember	8%	2%	17%	8%

The quality of service ratings that library professionals provided are higher across the board than those provided by members of the general public who have contacted PDSD generally.

This likely reflects the more focused queries from library professionals that are also more likely to be answerable by the PDSD staff.

The results of this survey regarding the quality of service received by library professionals support the following observations and conclusions:

• Library professionals were generally satisfied with the quality of service they received, and at higher levels than was observed among general public users. The mean score on our five-point scale was 4.2 for the 33 library professionals in our sample who specifically recall having made some kind of direct entreaty to the Directorate.
• The survey data with respect to satisfaction levels with the amount of time it took to receive service shows a mean score of 4.1 (again, among the 33 library professionals who have specific recall of having made contact). In a similar vein, “I was satisfied with the amount of time it took to receive a response to my questions/comments” received a mean score of 4.0

The survey's questions about the communication between the PDSD and its clientele suggest that it is doing a good job with both sets of groups:

• Both groups are similar in their very positive estimation that communications with Publications Canada were effective (94% agree among those with specific recall of having made direct contact).
• Similarly, 85% agreed that they obtained clear information.
• 83% of these 33 library professionals were satisfied with the ease of access to the service they received.
• Generally, 88% of these library professionals agreed that the service personnel was knowledgeable, and exactly the same proportion agreed that personnel was respectful.
• Finally, the survey shows that the PDSD's personnel seems to convey to library professionals who made contact that they understand their needs (85% agreed).

These ratings show quite clearly that the quality of service provided by the PDSD is rated higher by library professionals than it is by their other clients, which likely reflects library professionals' greater familiarity with the subject matter (which we presume translates into clearer and certainly easier requests to fulfil than would the case for less specialized people).

4.9.3 Usage and Evaluation of Alternative Sources

TABLE 12: ALTERNATIVE SOURCES FOR PUBLICATIONS USED

	Total	Type of library
		Academic library	Public library	Special library
Q5) What resources do you use, at least on occasion, to find Government of Canada publications?	n=116	n=53	n=34	n=29
Government of Canada Publications website (publications.gc.ca)	93%	94%	89%	95%
GALLOP Portal website (gallopportal.ca)	15%	20%	5%	19%
Library and Archives Canada website (bac-lac.gc.ca)	68%	71%	60%	77%
Open Government website	26%	37%	8%	35%
Authoring department website	51%	53%	35%	74%
Google	5%	7%	5%	2%

Table 12 illustrates that the alternative to publications.gc.ca used by most library professionals is Library and Archives Canada (68%), followed by the authoring department website (51%), the Open Government portal (26%) and finally the Gallop portal (15%). Table 13 that follows show that each of these alternatives presents roughly similar levels of ease of use, albeit with authoring departmental websites rated as the hardest to use.

4.9.4 Profile of the Library Professionals Email Sample

Finally, the survey of library professionals provides information about the institutions they work for, the types of publication they collect, how they handle these, how often they search and so on.

TABLE 14: PROFILE OF SAMPLE CHARACTERISTICS

	Total	Type of library
		Academic library	Public library	Special library
Q2) Are Government of Canada publications integrated in your library’s general collection?	n=116	n=53	n=34	n=29
Yes	65%	58%	68%	75%
No, we have a separate section for Government of Canada publications	30%	40%	25%	12%
We do not collect Government of Canada publications	2%	0%	3%	5%
I don’t know	4%	2%	4%	8%
Q3) What types of Government of Canada publications does your library collect?	n=114	n=53	n=33	n=28
All	41%	57%	11%	61%
Corporate reports	7%	4%	6%	15%
Monographs of interest	33%	25%	46%	30%
Serials of interest	22%	19%	27%	18%
Print publications	34%	24%	52%	26%
Digital publications	31%	34%	37%	13%
I don’t know	4%	3%	7%	3%
Q4) Do you?	n=114	n=53	n=33	n=28
Download the publications to your server	12%	9%	9%	24%
Catalogue the publications and point to our URL	70%	83%	65%	46%
Download publications to server & catalogue publication pointing to URL	3%	2%	0%	13%
Catalogue paper copies	4%	3%	6%	0%
I don’t know	12%	4%	20%	17%

From this data, the survey illustrates that:

• Most libraries (65%) integrate government publications into their collections. Academic libraries, however, are less likely to do so, as 40% have a separate section for government publications.
• 41% of libraries carry all types of government publications, a proportion that increases to a majority (57%) of academic libraries, but drops to 11% among public libraries. 34% of libraries carry print publications, whereas 33% carry monographs of interest, 31% carry digital publications, 22% carry serials of interest, and finally only 7% carry corporate reports. On average, libraries carry 3.7 out of the possible 5 types of publications, with public libraries carrying on average 2.8 out of the 5 types.

The vast majority of libraries (70%) catalogue publications and point to the publications.gc.ca URL. This number highlights the importance of providing libraries with an easily accessible list of new acquisitions to add to their catalogue, as well as providing file hosting that is beyond their institutional capabilities. Moreover, it demonstrates that it is necessary for PDSD to maintain stable URLs.

4.10 In Conclusion

The perspective of library professionals is most useful for underscoring the major findings of this research:

• That it is nearly impossible to properly gauge the quality of service provided by the PDSD without a clear understanding the context in which it operates – a context dominated by relentless change in information technology, forms of publication and government information structures.
• In this context of change, it is essential that Government have resources dedicated to the identification and cataloguing of publications as they exist today and as they will exist in the future.
• For the most part, library professionals endorse the work of the PDSD and articulate suggestions for ensuring that it continue to do this critical work effectively in the future.

Appendix A: Discussion Guide

1 - Part 1 - Introductions & More

1.1 - Introductions

Hello and welcome.
My name is John and I am your moderator.
I live in Montreal with my wife, Madeleine, my 17 year old son Lucas and occasionally with his 27 year old sister who hasn't quite moved out yet. I should tell you: I'm not a librarian - my speciality is public opinion research, so you will have to bear with me if I ask questions that betray my lack of expertise on the subject of libraries....
We're here to talk about the services offered by the Publishing and Depository Services Directorate (PDSD). PDSD is responsible for the distribution of electronic Government of Canada publications through www.publications.gc.ca. The purpose of this focus group is to gather opinions from Librarians to help the Government evaluate and better orient its services in the future. Your input is important and very much appreciated.
Basically, this discussion will be divided into three parts: We will begin with introduction, how important government publications are to your institution (if at all) and why, and a discussion of how you go about finding them when the occasion arises. Part II will focus on quality of service issues: things that work, things that don't, and what other governments and documents sources have to offer that works better. Finally, in Part 3, we will talk some about how things will evolve in the future.

1.2 - A little bit about you

How about a short little bio? It's always good to know a few personal details about the people we are speaking with. So perhaps a little about yourselves, who lives at your house, etc.)

1.3 - Area

In what part of the country do you live and work?

• Maritimes
• Quebec
• Ontario
• Prairies
• Alberta
• BC
• North

1.4 - Experience

How long have you been a librarian, and how long have you been with your particular institution?

1.5 - Nature of your library

Can you provide us with some details about your library?

1.6 - Your role

Can you provide some details about your role in this library?

2 - Importance of GoC Publications

What they mean to your patrons and your library

2.1 - Importance

Let's begin by talking about Government of Canada publications and what they mean to your library and its patrons. We know that they are more important to some of you than for others, so we want to make sure we hear from everybody about this.

2.2 - Publications: Of specific or generic importance?

Please help me clarify something:
Are Government of Canada publications important to your institution because you have patrons who ask for and need specific documents, or simply because having them is part of your institution's mission, or both? Is there some other way in which these documents are important?

• We have clients/patrons who ask for specific documents
• It is part of our mission to have them
• Neither

3 - Finding and Getting Publications

3.1 - The process…

Can you describe the process of keeping track of Government publications? (We will ask you qualify how well that process goes later in the process.) There are a couple of sub-topics I would like you to cover here, assuming they are relevant…

• How often are you monitoring the inventory of Government of Canada publications?
• What tools do you use to monitor what's available?
• What government entities do you reach out to, and how do you do that?
• What other avenues (portals, websites, etc.) do you use?

3.2 - Changes over the last years

Several of you (by the way, we also have a bulletin board running in French) have mentioned larger, systemic or environmental changes to your discipline over the years. How have they affected how you acquire government publications?

3.3 - Do you know…

I would just like to be sure -- some of your have mentioned it and some not:
Does everyone recognize " PDSD"? What does it mean?

3.4 - Personal contacts?

Do you have any personal contacts within the directorate?

• Yes, and I use them regularly
• Yes, but I don't use them often
• I know a name or two but rarely reach out
• No, not really

4 - Part 1 ends

Thanks!

4.1 - That's a good start for today!

You've been fantastic! Thank you so much for your input. It's been both interesting and valuable.

We will pursue this conversation tomorrow with a focus on quality of service. In the meantime, however, if you feel you have forgotten something or simply want to add to what you have already posted, feel free.

Thanks again!

UPDATE​: We may need to keep this section of the board open for another 24 hours to allow your mates to catch up. So far, only half of you have shown up.

5 - Part 2 Starts

Quality of Service

5.1 - Let's evaluate the process

Welcome back and thanks once again for your input. So during this part of the conversation we are going to focus more on "quality of service" questions -- by which we mean how easy or difficult it is to actually find and get publications and whatever issues you encounter on the way. I have a few closed ended questions for you here, and of course we will keep the conversation going…

5.2 - Process related questions

Do you have any complaints or suggestions about the frequency of communications coming to you about government documents? Are they coming too frequently? Not frequently enough?

5.3 - How easy/difficult

So, how would you rate the process of keeping track of and finding government publications?

• Very easy
• Somewhat easy
• Somewhat hard
• Very Hard
• I can't answer this / Not sure / I would answer the question another way

5.4 - Problems and Issues?

What kinds of problems do you or have you experienced?

5.5 - Specific Issues

Do you mind -- I would like to catalogue which of the following kinds of issues / problems you have encountered... Please check all that apply.

• Missing documents
• Broken links
• Problems searching for or navigating to where the document is
• Cataloging issues
• Problems with uploading reference or cataloging information
• Anything else
• No problems or issues
• Not applicable to me

5.6 - Suggestions and Examples

What suggestions do you have to offer about how to improve the process?

5.7 - Examples from other Governments & Departments

Do you know of another government department or entity that provides access to its publications extremely well? If so, please describe.

5.8 - Part 2 Ends

You've reached the end of the questions we had in mind for day 2. Thank again for your valuable input.

We will pick up this conversation tomorrow and Friday and finish with a focus on the future.

Can I ask you again to check back in before day's end tomorrow to review everyone's answers and answer with follow up questions or comments in the appropriate space please? I would like each of you to provide at least two questions or comments to each other.

If you have done that, or when you have done that, feel free to proceed to the last section…

Thanks again!

6 - Part 3: Looking forward

6.1 - Coming changes?

Are you aware of anything on the horizon that will affect how the Government of Canada makes its publications available to you? Please elaborate and fill us in on how that will affect you and your institution…

6.2 - "Open Government"?

Are you aware of "Open Government" and what does this mean to you? Do you think it will affect your ability to get Government of Canada publications? How?

6.3 - Other suggestions looking forward

Do you have any suggestions to offer the Government about how to adapt moving forward? Do you have anything else to add? Please, try to think of a couple things... If you have none, just make sure you say so…

6.4 - A few wrap up questions

Are you aware of the Depository Services Program-Advisory Committee (DSP-AC)? Please use the open ended field to describe it if you said "yes".

• Yes
• Sort of, but I don't really know what it's about
• No

6.5 - Advisory Committee (Information only)

I've been asked to provide this link to this Advisory Committee as there is apparently good information relevant to some of the questions you have been posing and updates available. Depository Services Program Advisory Committee - Overview

6.6 - We're done!

Thank you so much for your participation! Your answers were thought-provoking and valuable.

Before you go, can you please make sure you provide me with the name and address of the recipient of the $100 we promised you.

Last, allow me to remind you that this is Government research, and that you are entitled to protection under the Privacy Act, and access to this research under the Access to Information Act. And of course, you can expect my report to be published and available to you soon as a Government of Canada publication! I welcome your feedback.

Thanks again!

Appendix B: Site Visitor Questionnaire

Final Version
January 2017

Government of Canada Publications
publications.gc.ca
Publishing and Depository Services Directorate
Public Services and Procurement Canada
Online Survey – General Population

Thank you for participating in this Government of Canada survey about your experience with the publications.gc.ca website.

Completing this survey should take approximately 10 minutes.

Your participation is voluntary and completely confidential, and your comments will be kept anonymous and administered in accordance with the Privacy Act. Responses will be reported only in aggregate form by the independent research firm, Ad hoc/Patterson Langlois consortium, hired for this study.

This survey is registered with the Marketing Research and Intelligence Association. You can verify that this survey is legitimate, by calling the registration system's toll-free number 1-800-554-9996 or by consulting their online Research Verification Service, using the registration number 20170123-495H.

Once you start the survey, you will no longer be on a Government of Canada website but rather a site managed by Ad hoc/Patterson Langlois consortium.

If you have any questions or technical issues regarding this survey, please contact: PDSD@adhoc-opinion.com

Thank you for your help.

Section 1—Accessing the publications.gc.ca website

Q1. Please indicate if you are from…

ACCEPT ONE ANSWER ONLY

• 1. The general public
• 2 .A government organization
• 3. A library (Skip to Library survey)
• 4. An educational institution
• 8. Other (Please specify)

Q2. What brought you to our website? (Click all that apply)

ACCEPT SEVERAL ANSWERS

• 1. Looking for general information on a subject
• 2. Searching for a publication
• 3. Looking for a government form
• 4. Looking for a Government of Canada program / service
• 5. Have been misdirected
• 996. Other (Please specify)

Programmer: if choices 1 or 2 were not chosen in Q2, go to section 5

Q3. How did you navigate to publications.gc.ca?

ACCEPT ONE ANSWER ONLY

• 1. By directly accessing [publications.gc.ca]
• 2. From another government webpage
• 3. By using a search engine
• 8. Other (please specify)
• 5. I don't know

Q4. How did you learn about publications.gc.ca?

ACCEPT ONE ANSWER ONLY

• 1. Search engine
• 2. Government program / service
• 3. Educational institution
• 4. Library
• 5. Colleague
• 8. Other (please specify)

Section 2—Government of Canada Publications

Q5. How often do you visit publications.gc.ca?

ACCEPT ONE ANSWER ONLY

• 1. Daily
• 2. Weekly
• 3. Monthly
• 4. A few times a year
• 5. Once a year or so
• 6. This is my first visit to publications.gc.ca
• 9. I don't know

Q6. How often do you look for Government of Canada publications?

ACCEPT ONE ANSWER ONLY

• 1. Daily
• 2. Weekly
• 3. Monthly
• 4. A few times a year
• 5. Once a year or so
• 6. This is my first time (Go to Q8)
• 9. I don't know

Q7. What resources do you use, at least on occasion, to find Government of Canada publications? (Click all that apply)

ACCEPT SEVERAL ANSWERS

• 1. Government of Canada Publications website (publications.gc.ca)
• 2. Library and Archives Canada website (bac-lac.gc.ca)
• 3. Government of Canada website (canada.ca)
• 4. Open Government website (open.canada.ca)
• 5. Website for a specific department
• 6. Library
• 7. Internet
• 8. Other (Please specify)
• 9. I don't know

Q8. Generally, why do you search for Government of Canada publications?

ACCEPT ONE ANSWER ONLY

• 1. Personal purposes
• 2. Professional purposes
• 3. Educational purposes
• 4. Legal purposes
• 8. Other (Please specify)

Q9. Using a scale from 1 (not important at all) to 5 (extremely important), please click the response that best describes how important each of the following aspects are to you. (Click one response per aspect)

Actual scale will appear: 1. Not important at all; 2. Not too important; 3. Somewhat important; 4. Very important; 5. Extremely important; or Don’t know or Does not apply

• a) Having the ability to self-serve on our website (searching, downloading, etc.)
• b) Having the ability to contact staff who can assist you in locating a publication
• c) Having a single access point for all Government of Canada publications
• d) Having long term access to Government of Canada publications
• e) Having the ability to access older publications in digital format
• f) Having access to information about publications not available in digital format
• g) Having a listing of publications published in a given week

Section 3—Satisfaction levels with Government of Canada Publications website (publications.gc.ca)

Q10. Using a scale from 1 (strongly disagree) to 5 (strongly agree), what is you level of agreement with each of the following aspects related to the publications.gc.ca website? (Click one response per aspect)

The entire scale will appear in the online questionnaire (Strongly disagree / Disagree / Neither agree nor disagree / Agree / Strongly agree or Don’t know or Does not apply)

• a) The basic search capabilities on the website are effective
• b) It was easy to find what I was looking for on the website
• c) In the end, I got what I needed

Q11A. Using a scale from 1 (very dissatisfied) to 5 (very satisfied), click the response that best describes your overall level of satisfaction with the publications.gc.ca website.

The entire scale will appear in the online questionnaire (Very dissatisfied / Dissatisfied / Neither satisfied nor dissatisfied / Satisfied / Very satisfied or Don’t know or Does not apply)

• My overall satisfaction with the website

Section 4—Communication with Government of Canada Publications Customer Service and Level of Satisfaction

Q11. In the past 12 months, have you contacted the Government of Canada Publications in any way? (Click all that apply)

ACCEPT SEVERAL ANSWERS

• 1. I had no contact with Government of Canada Publications (publications.gc.ca) (Skip to section 5)
• 2. By telephone
• 3. In-person
• 4. By e-mail
• 5. By fax
• 6. By letter or regular mail
• 9. I do not remember (Skip to section 5)

Q12. Using a scale from 1 (strongly disagree) to 5 (strongly agree), click the response that best describes your level of agreement with each of the following statements regarding the service you received from Government of Canada Publications. (Click one response per aspect)

The entire scale will appear in the online questionnaire (Strongly disagree / Disagree / Neither agree nor disagree / Agree / Strongly agree or Don’t know or Does not apply)

• a) I was satisfied with the amount of time it took to receive the service
• b) I was satisfied with the amount of time it took to receive a response to my questions/comments
• c) Communications with Government of Canada Publications were effective
• d) I obtained clear information
• e) I was satisfied with the ease of access to the service
• f) Personnel were knowledgeable
• g) Personnel were respectful
• h) Personnel understood my needs
• i) In the end, I got what I needed (Ask last)

Q13. Using a scale from 1 (very dissatisfied) to 5 (very satisfied), click the response that best describes your overall level of satisfaction with the quality of service you received from Government of Canada Publications (publications.gc.ca).

The entire scale will appear in the online questionnaire (Very dissatisfied / Dissatisfied / Neither satisfied nor dissatisfied / Satisfied / Very satisfied or Don’t know or Does not apply)

• Overall how satisfied were you with the quality of service (you received)?

Section 5—And finally, for statistical purposes only…

A few final questions that will help us analyze the results……

Q14. What is the highest level of formal education that you have completed?

ACCEPT ONE ANSWER ONLY

• 1. Grade 8 or less
• 2. Some high school
• 3. High School diploma or equivalent
• 4. Registered Apprenticeship or other trades certificate or diploma
• 5. College, CEGEP or other non-university certificate or diploma
• 6. University certificate or diploma below bachelor's level
• 7. Bachelor's degree
• 8. Post graduate degree above bachelor's level
• 9. Prefer not to answer

Q15. In which of the following age categories do you belong?

ACCEPT ONE ANSWER ONLY

• 1. 18 to 34
• 2. 35 to 49
• 3. 50 to 54
• 4. 55 to 64
• 5. 65 or older
• 9. I prefer not to answer

Q16. Please indicate your gender.

ACCEPT ONE ANSWER ONLY

• 1. Male
• 2. Female
• 3. Other
• 9. I prefer not to answer

Note to programmer:
if Q1=2 go to Q19
if Q1=4 go to Q18
if Q1=996 go to Q20

Q17. Where do you currently reside?

ACCEPT ONE ANSWER ONLY

• 01. Newfoundland and Labrador
• 02. Prince Edward Island
• 03. Nova Scotia
• 04. New Brunswick
• 05. Quebec
• 06. Ontario
• 07. Manitoba
• 08. Saskatchewan
• 09.Alberta
• 10. British Columbia
• 11. Yukon
• 12. Northwest Territories
• 13. Nunavut
• 90. Outside Canada (Please indicate in which country)
• 99. I prefer not to answer

Note to programmer : If General Public (Q1), end. Use only if Q1=4.

Q18. What type of educational institution do you work for?

ACCEPT ONE ANSWER ONLY

• 1. A university (Go to Q20)
• 2. A college (Go to Q20)
• 3. A high school (Go to Q20)
• 4. Other (Go to Q20)
• 9. prefer not to answer (Go to Q20)

Note to programmer : Use Q19 only if Q1=2.

Q19. What level of government do you work for?

ACCEPT ONE ANSWER ONLY

• 1. Federal
• 2. Provincial
• 3. Municipal
• 9. Prefer not to answer

Q20. In which province is your workplace located?

ACCEPT ONE ANSWER ONLY

• 01. Newfoundland and Labrador
• 02. Prince Edward Island
• 03. Nova Scotia
• 04. New Brunswick
• 05. Quebec
• 06. Ontario
• 07. Manitoba
• 08. Saskatchewan
• 09. Alberta
• 10. British Columbia
• 11. Yukon
• 12. Northwest Territories
• 13. Nunavut
• 90. Outside Canada (Please indicate in which country)

Thank you very much for participating in this survey. All information provided by you will be held in strictest confidence and will only be used for research purposes.

Thank you for your help.

Appendix C

Final Version
January 2017

Government of Canada Publications
publications.gc.ca
Publishing and Depository Services Directorate
Public Services and Procurement Canada
Online Survey - Libraries

Thank you for participating in our survey about your experience with the publications.gc.ca website.

This survey is addressed to library personnel who use publications.gc.ca. If you are this person, please click the “Continue” button and complete the survey. If not, please click the “Exit” button below and inform the most appropriate person in your library about this survey.

Completing this survey should take approximately 10 minutes, depending on your answers and the length of your text comments.

Your participation is voluntary and your comments will be kept anonymous and confidential. The survey will be administered in accordance with the Privacy Act. Responses will be reported only in aggregate form by the independent research firm, Ad hoc/Patterson Langlois consortium, hired for this study.

This survey is registered with the Marketing Research and Intelligence Association. You can verify that this survey is legitimate, by calling the registration system's toll-free number 1-800-554-9996 or by visiting the Marketing Research and Intelligence Association's Research Verification Service.

Once you start the survey, you will no longer be on a Government of Canada website but rather a site managed by Ad hoc/Patterson Langlois consortium.

If you have any questions or technical issues regarding this survey, please contact: PDSD@adhoc-opinion.com

Thank you for your help.

Section 1—Looking for and finding Government of Canada publications

Q41. Please indicate if you are from a…

ONE ANSWER

• 1. School library
• 2. Academic library
• 3. Public library
• 4. Special library
• 9. Other (please specify)

Q42. Are Government of Canada publications integrated in your library's general collection?

ONE ANSWER

• 1. Yes
• 2. No, we have a separate section for Government of Canada publications
• We do not collect Government of Canada publications (Skip to Q5)
• 9. I don't know

Q43. What types of Government of Canada publications does your library collect? (Select all that apply)

SEVERAL ANSWERS POSSIBLE

• 1. All
• 2. Corporate reports
• 3. Monographs of interest
• 4. Serials of interest
• 5. Print publications
• 6. Digital publications
• 9. I don't know

Q44. Do you…?

ONE ANSWER

• 1. Download the publications to your server
• 2. Catalogue the publications and point to our URL
• 8. Other (Please specify)
• 9. I don't know

Q45. What resources do you use, at least on occasion, to find Government of Canada publications? (Select all that apply)

SEVERAL ANSWERS POSSIBLE

• 01. Government of Canada Publications website (publications.gc.ca)
• 02. GALLOP Portal website (gallopportal.ca)
• 03. Library and Archives Canada website (bac-lac.gc.ca)
• 04. Open Government website (open.canada.ca)
• 05. Authoring department website
• 90. Other (specify)
• 99. None of the above (Skip to section 6)

Q46. Using a scale from 1 (very difficult) to 5 (very easy), click the response that best describes your experience in locating government publications using the following sources. (Click only one answer per aspect)

The entire scale will appear in the online questionnaire (Very difficult / Difficult / Neither difficult nor easy / Easy / Very easy or Don’t know or Does not apply)

Ask for each resource mentioned in Q5

• a) Government of Canada Publications website (publications.gc.ca)
• b) GALLOP Portal website (gallopportal.ca)
• c) Library and Archives Canada website (bac-lac.gc.ca)
• d) Open Government website (open.canada.ca)
• e) Authoring department website
• f) Other (specify)

Section 2—Awareness and usage of the Government of Canada Publications website (publications.gc.ca)

Q47. During the last 12 months, which of the following services have you used from Government of Canada Publications website (publications.gc.ca)? (Select all that apply)

SEVERAL MENTIONS

• 1. Visited the Government of Canada Publications website
• 2. Consulted the Weekly Acquisitions List
• 3. Searched for publications
• 4. Accessed Specialised Catalogues
• 5. Consulted the Services for Libraries information page
• 9. None of the above (Skip to Section 4)

Q48. Are you familiar with the Weekly Acquisitions List?

ONE MENTION

• 1. Yes
• 2. No (Skip to Q10)

Q49. How often do you consult the Weekly Acquisitions List?

ONE MENTION

• 1. Weekly
• 2. Every two weeks or so
• 3. Monthly
• 4. Occasionally
• 5. Never
• 9. I don't know

Section 3—Satisfaction levels with the Government of Canada Publications website (publications.gc.ca)

Programmer: If Q5 =1, ask Q10, else Go to Q11

Q50. Using a scale from 1 (strongly disagree) to 5 (strongly agree), please select your level of agreement with each of the following elements related to the Government of Canada Publications website (publications.gc.ca). (Click one response per aspect)

The entire scale will appear in the online questionnaire (Strongly agree / Disagree / Neither agree nor disagree / Agree / Strongly agree or Don’t know or Does not apply)

• a) I'm able to find the Government of Canada publications I'm looking for
• b) The Weekly Acquisitions List is useful
• c) The links are reliable
• d) The bibliographic information in the catalogue meet my needs
• e) The quality of catalogue records is acceptable
• f) The basic search capabilities on the website are effective
• g) It was easy to find what I was looking for on the website
• h) In the end, I got what I needed

Q51. Using a scale from 1 (very dissatisfied) to 5 (very satisfied), click the response that best describes your overall level of satisfaction with the publications.gc.ca website.

The entire scale will appear in the online questionnaire (Very dissatisfied / Dissatisfied / Neither satisfied nor dissatisfied / Satisfied / Very satisfied or Don’t know or Does not apply)

• My overall satisfaction with the website

Section 4—Communication with Customer Service and Satisfaction

Q52. In the past 12 months, have you contacted Government of Canada Publications in any way? (Click all that apply)

SEVERAL MENTIONS POSSIBLE

• 1. I had no contact with Government of Canada Publications (publications.gc.ca) (Skip to section 5)
• 2. By telephone
• 3. In-person
• 4. By e-mail
• 5. By fax
• 6. Through InfoDep
• 7. By letter or regular mail
• 9. I do not remember (Skip to Section 5)

Q53. Using a scale from 1 (strongly disagree) to 5 (strongly agree), click the response that best describes your level of agreement with each of the following aspects related to the service you received from Government of Canada Publications. (Click one response per aspect)

The entire scale will appear in the online questionnaire (Strongly disagree, / Disagree / Neither agree nor disagree / Agree /Strongly agree or Don’t know or Does not apply)

• a) I was satisfied with the amount of time it took to receive the service
• b) I was satisfied with the amount of time it took to receive a response to my questions/comments
• c) Communications with Government of Canada Publications were effective
• d) I obtained clear information
• e) I was satisfied with the ease of access to the service
• f) Personnel were knowledgeable
• g) Personnel were respectful
• h) Personnel understood my needs
• i) In the end, I got what I needed (Ask last)

Q54. Using a scale from 1 (very dissatisfied) to 5 (very satisfied), click the response that best describes your overall level of satisfaction with the quality of service you received from Government of Canada Publications (publications.gc.ca).

The entire scale will appear in the online questionnaire (Very dissatisfied / Dissatisfied / Neither satisfied nor dissatisfied / Satisfied / Very satisfied or Don’t know or Does not apply)

• Overall how satisfied were you with the quality of service (you received)?

Section 5—Environmental

Q55. Using a scale from 1 (not important at all) to 5 (extremely important), please indicate to what extent each of the following aspects are important to your institution. (Click one response per aspect)

Actual scale will appear: 1. Not important at all; 2 Not too important; 3 Somewhat important; 4 Very important; 5 Extremely important or Don’t know or Does not apply

• a) Having the ability to contact staff who can assist you in locating a publication
• b) Having a single access point for all Government of Canada publications
• c) Having the ability to self-serve on publications.gc.ca website (searching, downloading, etc.))
• d) Having long term access to Government of Canada publications
• e) Having stable links to Government of Canada publications (PURLs)

Q56. Using a scale from 1 (not at all valuable) to 5 (very valuable), click the response that best describes how valuable each of the following aspects are to you. (Click one response per aspect)

Actual scale will appear: 1. Not at all valuable; 2 Not very valuable; 3 Neutral; 4 Valuable; 5 Very valuable or Don’t know or Does not apply

• a) Providing bibliographic information about a Government of Canada publication not available in digital format
• b) Digitizing Government of Canada publications previously offered in print only format
• c) Providing control numbers such as ISBN/ISSN/GC Catalogue numbers in publications
• d) Providing a listing of publications published in a given week
• e) Having publications.gc.ca expose its metadata elements using Open Archives Initiative Protocol Harvesting (OAI-PMH)

Q57. Do you have additional comments that would help Government of Canada Publications serve you better?

Section 6—And finally, for statistical purposes only…

Q58. In which province is your workplace located?

ONE MENTION

• 01. Newfoundland and Labrador
• 02. Prince Edward Island
• 03. Nova Scotia
• 04. New Brunswick
• 05. Quebec
• 06. Ontario
• 07. Manitoba
• 08. Saskatchewan
• 09. Alberta
• 10. British Columbia
• 11. Yukon
• 12. Northwest Territories
• 13. Nunavut
• 90. Outside Canada (Please indicate in which country)

Q59. What is the size of the population served by your library

ONE MENTION

• 1. Less than 10,000
• 2. 10,000 – 50,000
• 3. 50,001 – 100,000
• 4. 100,001 – 250,000
• 5. More than 250,000
• 9. I don't know

Thank you very much for participating in this survey. All information provided by you will be held in strictest confidence and will only be used for research purposes.

Thank you for your help.

Footnotes

1

Some caution is warranted here as adherence to these categories likely reflects some subjective judgment: it is not clear, for example, if students consistently ascribe themselves to the general public or to their educational institution.

2

The reader will note that respondents were able to provide more than one answer to this question, which explains why the percentages do not necessarily add up to 100%. The small arrows in these documents are used to indicate a number that is statistically significant -- lower or higher -- depending on the case.