Advertising Public Opinion Research - Education and Skills Adult Campaign (Phase 2: Advertising Campaign Evaluation Tool ACET) Final Report

Executive Summary

Background

Budget 2018 announced a suite of measures to help Canadian workers get the skills they need to succeed in an evolving economy. The Government of Canada launched the Education and Skills - Helping Working Adults Upgrade Their Skills campaign in fall 2018. This campaign, with a budget of over $1 million, informed adults of the changing nature of work, encouraged this audience to pursue continuous learning, gain new professional skills and provided information about the Department’s skills training programs that support training and education of adult workers and unemployed people

The campaign targeted adults 35 to 55 years old.

Evaluating advertising campaigns that have a media buy of over $1 million is a requirement within the
Policy on Communications and Federal Identity.

Research Objectives

The purpose of the quantitative research is to evaluate the advertising campaign using the Government of
Canada’s Advertising Campaign Evaluation Tool (ACET), and as required, is conducted before and after the ad campaign runs in the media. Post-campaign data metrics are measured against the baseline data. Results will provide ESDC with data that assists in developing/adjusting future media strategies. In addition, results from the post-testing provide for future campaigns, valuable trend information for the Department. The specific campaign research objectives were to:

• Gauge the effectiveness of the campaign;
• Measure awareness and knowledge of the campaign and ESDC programs; and
• Detect if the campaign was effective in promoting the call to action.

Methodology

To fulfill these objectives, an online survey was undertaken in October 2018 to set a baseline for the campaign prior to the campaign launch, while a post-advertising campaign online survey was undertaken in March 2019. The pre-advertising campaign or ‘baseline’ survey required an average of 6.6 minutes to administer, while the post-advertising campaign survey required an average of 9.8 minutes to administer.

The participation rates in each wave were 22 percent, and 13 percent, respectively. The email contact records for each wave were drawn from a national panel administered by The Logit Group of Toronto, Ontario. There were a total of 2,000 surveys completed for the baseline pre-advertising campaign wave, while there as a total of 2,067 surveys completed for the post-advertising campaign wave.

The pre-advertising campaign wave was administered October 18 to October 25, 2018, while the post-advertising campaign wave was administered March 7 to March 13, 2019.

Research Usage

The survey data was collected to gauge the effectiveness of ESDC’s Education and Skills advertising campaign (adults). The surveys explored whether the target segments encountered the recent ad campaign, whether on television, through Facebook or other social media, or through some other medium.

As per the standard ACET methodology, the baseline survey conducted prior to ads being run in the media asked questions regarding the recall of Government of Canada advertising in general and more specifically on the upcoming campaign topic. Campaign specific attitudinal and behavioral questions were also measured.

Again as per the standard ACET methodology, the post-campaign survey collected information related to recall of the advertisement and its key messages, ad sponsorship, and what, if any, action has been taken as a result of seeing the ad. Post-campaign data metrics would subsequently be compared by Government of Canada representatives against the baseline findings.

Information obtained through this public opinion research would allow ESDC to determine the impact of its advertising campaign and provide direction on areas where campaigns can be adjusted. The findings provide useful information to improve future campaigns.

Given that this online survey methodology used a non-probability sample, the data collected cannot be extrapolated to the broader target population.

Expenditure

The quantitative survey entailed the expenditure of $47,113.40 including tax. This survey was part of a larger contract that also included qualitative research. The expenditure of the combined qualitative and quantitative research components was $76,668.24, including tax.

Consent

Corporate Research Associates offers this written consent allowing the Librarian and Archivist of Canada to post, in both official languages, this Methodological Report.

Political Neutrality Statement and Contact Information

I hereby certify as a Senior Officer of Corporate Research Associates that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications. Specifically, the deliverables do not contain any reference to electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leader.

Peter MacIntosh
Chief Research Officer & Partner
Corporate Research Associates
pmacintosh@cra.ca
902-493-3832

Appendix A: Study Methodology

This evaluation utilized the Government of Canada’s Advertising Campaign Evaluation Tool (ACET), and was administered to a sample of Canadians aged 35 to 55 years old. The online-based data collection regimen entailed a baseline or pre-advertising campaign data collection wave, as well as a post-advertising campaign data collection wave. This approach permits a comparison of awareness and opinions over time.

Questionnaire Design

The questions utilized in this study were based on the Government of Canada’s standard Advertising Campaign Evaluation Tool questionnaire (Appendix B). The primary difference between the baseline and the post-wave survey questionnaires was that in the latter wave, the survey respondents were aided in their ad recall by way of being shown advertisements from the recent advertising campaign. Specifically, the online respondents were shown a 50-second advertisement from the campaign, one static generic advertisement, as well as three randomly selected static program-specific advertisements. The respondents were subsequently asked a series of questions about the advertisements. This aiding of respondents by showing ads drawn from the advertising campaign is a process that is possible with an online survey methodology. 

CRA ensured that respondents were able to complete the survey on various platforms including computers, tablets or smartphones.

As required by Government of Canada standards, English and French pre-tests were conducted in each survey wave. In both surveys, a line of questioning was included at the end of the survey in which respondents were asked if they encountered any questions or survey wording that was difficult to understand. No comprehension or other difficulties regarding the pretest surveys were identified.

Sampling

The current baseline and post-campaign survey approaches were designed to be administered to separate, unique online panel samples of approximately 2,000 Canadians aged 35 to 55 years old. That is, different panelists were utilized in the post-campaign wave, as compared to the baseline wave.

The current baseline survey sampling approach was designed to closely represent the actual, true target population in terms of age, gender, language and region. To this end, quotas were implemented for age (35—44/45 – 55), gender (male/female), and region (Atlantic, Quebec, Ontario, West/Territories). In addition, to ensure representation from targeted sectors, sampling was designed to complete more than one-half of surveys within specific targeted sectors. This was based on whether individuals were currently employed or seeking work in one of the targeted sectors (namely, agriculture, construction, farming, fishing, forestry, hunting, logistics, manufacturing, mining and extraction, office and administration support, public administration, retail sales and related, retail or wholesale trade, supply chain, and transportation). 

Sample Records Source

CRA utilized the services of The Logit Group for this research. The Logit Group’s online general population panel is comprised of over 600,000 Canadian residents nationally, with sound representation across regions. Logit Group panelists are recruited from a large number of sources to maximize reach and representation.

The recruitment policies of The Logit Group’s partners (SSI, Toluna, Asking Canadians, and Research Now) are broad in scope. Survey data quality rests on many different factors, including sourcing of panelists who are vetted, using ongoing quality checks such as eliminating panelists who are no longer active, and so forth. The following are panel member sources for Logit Group studies:

• Email invitations: pre-authorized to opt in lists from associations and groups
• Social Media: advertising and social groups on leading social media platforms
• Media Platforms: advertising on online media platforms both niche and mainstream
• Use of major recruiting brands
• Loyalty programs
• Targeted audiences
• Web and social networking sites
• Targeted emails by The Logit Group’s online partners to their members or subscribers
• Referral programs

The Logit Group has established a variety of quality assurance processes to proactively identify invalid respondents. For example, the company has incorporated methods to quickly identify and flag straight-lining speedsters (i.e., respondents who give the same responses to all questions as a means of quickly finishing the survey), thereby monitoring whether panelists are able to provide thoughtful and accurate responses.

Panel members are monitored against Statistics Canada data to gauge statistical representation. Annual profile refreshing campaigns are conducted to incentivize panelists to remain active; these can also contain new questions in order to target specific niche audiences more precisely. Panelists’ participation is rewarded with their choice of HBC Rewards bonus points, Aeroplan Miles or Petro Points, as well as various prizes.

The sampling procedure reflected a computerized randomization of online panel members, with exclusions from the randomization process being based upon, for example, whether a panelist had received his/her monthly maximum number of survey invitations. As well, no baseline survey invitees were invited to complete a post-campaign survey.

Survey Administration

Survey Programming and Testing

The baseline and post-wave online surveys were programmed by CRA in both English and French, using Voxco Acuity programming. Respondents were given the option to complete the survey in the official language of their choice and were formally invited to the survey in the official language of their choice. As well, at any point when completing the questionnaire, respondents had the option to change the questionnaire language to the other official language. Assistance in completing the survey was available from bilingual CRA staff, as required. Respondents were able to verify the legitimacy of the survey via representatives from Corporate Research Associates or ESDC. The programmed surveys were tested to ensure question order and skip patterns were properly implemented. Testing included CRA researchers receiving the invitation via email just as a respondent would, to ensure accuracy of delivery, text, links, and so on. ESDC staff were also provided with the pre-test links.

In addition, a baseline wave pre-launch test was conducted among respondents, as was a post-campaign wave pre-test among respondents. In the baseline wave, a total of 19 English and 22 French completed pre-test surveys were completed, while in the post-campaign wave a total of 54 English and 11 French pre-tests were completed. These pre-test survey completions were conducted via a survey “soft launch” whereby a small number of panel respondents were invited to participate in the survey. As noted above, the pre-campaign baseline pre-test also included a survey question that sought respondent feedback on the survey questionnaire, in terms of qualities such as clarity and ease of understanding. The pre-testing of the surveys each wave allowed the collected data to be reviewed to ensure accuracy and to identify any programming aspects that should be modified. No substantive data quality issues arose as a result of either of the pre-tests, and thus the pre-test data was maintained in the final data sets.

Data Collection

Unlike telephone surveys which typically occur with new respondents being contacted throughout the specified data collection time period, in online surveys of the type implemented in the present case, the preponderance of respondents are notified within a day or two, for example, at the end of the advertising being assessed. Reminder notices are forwarded to these sampled respondents until such time as the target number of survey completions had been achieved. This data collection approach offers a timing advantage in contacting respondents as soon as possible after the campaign has ended.

As noted, this study consisted of two data collection phases, a pre-advertising campaign wave and a post-advertising campaign wave. The pre-advertising campaign wave was administered October 18–25, 2018, while the post-advertising campaign wave was administered March 7-13, 2019. The survey invitation as well as a reminder invitation were sent to panel members during the data collection periods (i.e., eight days for the baseline wave, and seven days for the post-campaign wave). Fieldwork was monitored and reviewed on an ongoing basis to ensure target quotas were being met. CRA provided regular reports (verbal and written) to Education and Skills Development Canada representatives regarding progress, as requested or pre-determined. Given that single use unique survey links were distributed to prospective respondents, no individual was able to complete the survey questionnaire more than once.
A total of 2,000 baseline wave surveys were collected, while a total of 2,067 post-wave wave surveys were collected.

It is important to note that because, for various reasons, a small percentage of submitted online panel surveys is often removed from study data sets after submission, CRA’s initial quota targets in each wave exceeded the overall final requirement of 2,000 per wave. Thus, given the unavoidable possibility of having to remove surveys, post collection, CRA as a precautionary measure collected more than the initially targeted number of surveys per wave. Reasons for removing surveys ultimately included respondent “speedsters” who are deemed to have moved too quickly through the questionnaire and consistently non-intelligible verbatim responses.

The baseline survey required a mean average of 6.6 minutes for respondents to complete, while the post-campaign survey required a mean average of 9.8 minutes for respondents to complete. The post-campaign survey aided respondents with a recording of a campaign advertisement, and accompanying questions added to the survey length. A non-probability sample approach was implemented given that the study was designed to be conducted among online Canadian general public panelists. All such panels are inherently non-probability in nature, given that panelists self-select to become members of such panels, and not all adult Canadians belong to such a panel. The tables below for the baseline wave and the post-wave surveys display regional, gender, and age data in terms of the actual distribution of adult Canadians as catalogued in the 2016 Statistics Canada Census.

The table for the baseline wave displays the actual distribution in terms of region, gender, and age as catalogued in the 2016 Statistics Canada Census, the quotas established for region, gender, age, and targeted sectors and the actual survey distribution by region, gender, age, and targeted sectors. 

Baseline Wave

(Percentages may not sum exactly to 100%, owing to rounding)

The table for the post-campaign wave displays the actual distribution in terms of region, gender, and age as catalogued in the 2016 Statistics Canada Census, the quotas established for region, gender, age, and targeted sectors and the actual survey distribution by region, gender, age, and targeted sectors.

Post-campaign Wave

(Percentages may not sum exactly to 100%, owing to rounding)

Data Tabulation: No weighting was applied to the survey results.

Participation Rates: The rates below were derived using the formula recommended by the Public Opinion Research Directorate of the Government of Canada:

Baseline wave

Baseline Wave Participation Rate = R/(U + IS + R) = 2,896/(10,051 + 431 + 2,896) = 22%

Post-wave

Post-Wave Participation Rate = R/(U + IS + R) = 3,280/(20,796+273 + 3,280) = 13%

Given that the online methodology utilized a non-probability sample, a margin of error cannot be applied to the results as per the Standards for the Conduct of Government of Canada Public Opinion Research for Online Surveys.  

Non-Response Bias Analysis

Any survey that is conducted is potentially subject to bias or error. When a survey is conducted with a sample of the population, there are two general classes of bias or error: sampling error, which is quantifiable, and non-sampling error, which is typically not quantifiable.  Sampling error arises from the fact that interviews are conducted with only a subset of the population, and thus is it possible that the results obtained from this group of respondents are not reflective of the population as a whole.  In contrast, non-sampling error encompasses a number of different types of errors including coverage error, measurement error, non-response error, and processing error.

No measurement of sampling error can be attributed to the current study, given that the contact records utilized in the data collection process were derived from an online panel of the general public, which is to say, a non-probability sample source. Having stated that, measures were taken in the implementation of the data collection to ensure sufficient completed surveys were obtained from demographic groups traditionally regarded as central in quantitative survey research, such as gender, age, and region/province.

With respect to non-sampling error, a number of steps were taken to minimize bias due to these sources. All surveys utilized online interviewing technology to ensure proper survey skip patterns were followed and to minimize errors due to data entry and data capture. The baseline, as well as post-campaign French and English survey instruments themselves, were pre-tested with a small sample of respondents to ensure the survey material was easily understood by respondents, and that the resultant data were being captured properly. In terms of coverage, the surveys were conducted with an online panel of the Canadian general public, based on a randomized sampling of panel records matching the target criteria drawn from a commercially available online panel.

Appendix B: Study Questionnaires

Online Survey

Advertising Campaign Evaluation Tool Baseline Survey

To be conducted before the ads are run in the media.

Introduction

Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians. Si vous préférez répondre au sondage en français, veuillez cliquer sur français [SWITCH TO FRENCH VERSION].

Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 5 to 7 minutes to complete. This survey is being administered according to the requirements of the Privacy Act, the Personal Information Protection and Electronic Documents Act, the Access to Information Act, and any other relevant legislation.

START SURVEY

To view our privacy policy, click here.

If you require any technical assistance, please contact XXX.

• a. Does anyone in your household work for any of the following organizations? SELECT ALL THAT APPLY
  • a marketing research firm
  • a magazine or newspaper
  • an advertising agency or graphic design firm
  • a political party
  • a radio or television station
  • a public relations company
  • the federal or provincial government
  • none of these organizations

IF “NONE OF THESE ORGANIZATIONS” CONTINUE, OTHERWISE THANK AND TERMINATE.

• b. Are you…
  • male 
  • female
  • Other
  • Prefer not to answer 
• c. In what year were you born?
  • [ ] YYYY

ADMISSIBLE RANGE 1962-1983

ASK D IF QUESTION C IS LEFT BLANK, OR IF 1962 OR 1983 IN Q.C

• d. In which of the following age categories do you belong? SELECT ONE ONLY
  • less than 18 years old
  • 18 to 24 
  • 25 to 34  
  • 35 to 44 
  • 45 to 55 
  • 56 to 64  
  • 65 or older

IF “LESS THAN 35 OR MORE THAN 55 YEARS OLD” OR “BLANK”, THANK AND TERMINATE

• e. In which province or territory do you live? SELECT ONE ONLY
  • Alberta
  • British Columbia
  • Manitoba
  • New Brunswick
  • Newfoundland and Labrador
  • Northwest Territories
  • Nova Scotia
  • Nunavut
  • Ontario
  • Prince Edward Island
  • Quebec
  • Saskatchewan
  • Yukon

IF NO PROVINCE OR TERRITORY IS SELECTED, THANK AND TERMINATE

• f. Do you identify yourself as Indigenous, that is, are you of First Nations, Inuit or Métis descent? PERMIT ONE RESPONSE ONLY
  • Yes, First Nations 1 
  • Yes, Inuit 2 
  • Yes, Métis 3 
  • No 4 
• i. [IF ‘YES, FIRST NATIONS’ IN Q.f (CODE 1), POSE] Do you live most of the time on-reserve or off-reserve? PERMIT ONE RESPONSE ONLY
  • On-reserve 1 
  • Off-reserve 2 
  • Prefer not to say 3 
• D1: Which of the following categories best describes your current employment status? Are you …: SELECT ALL THAT APPLY
  • working full-time (35 or more hours per week)
  • working part-time (less than 35 hours per week)
  • self-employed
  • unemployed, but looking for work
  • a student attending school full-time
  • a student attending school part-time
  • training in an apprenticeship program
  • retired
  • not in the workforce (Full-time homemaker or unemployed but not looking for work)
  • other employment status 
• D100: [POSE Q.D100 IF WORKING FULL-TIME, WORKING PART-TIME, OR SELF-EMPLOYED IN Q.D1] In which, if any, of the following industries or sectors do you currently work?  SELECT ALL THAT APPLY
  • Agriculture
  • Arts
  • Construction
  • Education
  • Engineering
  • Farming
  • Finance
  • Fishing
  • Forestry
  • Hunting
  • Logistics
  • Manufacturing
  • Medicine
  • Mining and Extraction
  • Office and administrative support
  • Public administration
  • Retail sales and related
  • Retail or wholesale trade
  • Science
  • Supply chain
  • Transportation
  • Some other sector or industry: (Please specify which industry or sector you currently work in: [ ]
• D101: [POSE Q.D101 IF UNEMPLOYED BUT LOOKING FOR WORK IN Q.D1] In which, if any, of the following industries or sectors do you currently seek work?  SELECT ALL THAT APPLY
  • Agriculture
  • Arts
  • Construction
  • Education
  • Engineering
  • Farming
  • Finance
  • Fishing
  • Forestry
  • Hunting
  • Logistics
  • Manufacturing
  • Medicine
  • Mining and Extraction
  • Office and administrative support
  • Public administration
  • Retail sales and related
  • Retail or wholesale trade
  • Science
  • Supply chain
  • Transportation
  • Some other sector or industry: (Please specify which industry or sector you currently seek work in: [ ]

Core Questions

ASK ALL RESPONDENTS

• Q1: Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
  • yes 
  • no GO TO T1A
• Q2: Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad? SELECT ALL THAT APPLY – ROTATE LIST ON SCREEN, BUT ALWAYS PRESENT ‘OTHER’ LAST
  • cinema
  • Facebook
  • Internet website
  • magazines
  • newspaper (daily)
  • newspaper (weekly, community, or minority language)
  • outdoor billboards
  • pamphlet or brochure in the mail
  • public transit (bus or subway)
  • radio
  • television
  • Twitter
  • YouTube
  • Instagram
  • LinkedIn
  • Snapchat
  • Spotify
  • Other, specify [ ]
• Q3: What do you remember about this ad? [ ]
• Q4: How did you know that it was an ad from the Government of Canada? [ ]

Campaign Specific Questions

ASK ALL RESPONDENTS

• T1A: Over the past three weeks, have you seen, read or heard any Government of Canada advertising about continuous learning, skills or job resources?
  • Yes
  • No GO TO T1D1
• T1B: Where have you seen, read or heard this Government of Canada ad about continuous learning, skills or job resources? SELECT ALL THAT APPLY – ROTATE LIST ON SCREEN, BUT ALWAYS PRESENT ‘OTHER’ LAST
  • cinema
  • Facebook
  • Internet website
  • magazines
  • newspaper (daily)
  • newspaper (weekly, community, or minority language)
  • outdoor billboards
  • pamphlet or brochure in the mail
  • public transit (bus or subway)
  • radio
  • television
  • Twitter
  • YouTube
  • Instagram
  • LinkedIn
  • Snapchat
  • Spotify
  • Other, specify
• T1C: What do you remember about this ad? [ ]

ASK ALL RESPONDENTS    

CAMPAIGN SPECIFIC ATTITUDINAL AND BEHAVIOURAL QUESTIONS MAY BE ADDED HERE

• T1D1: In the past ten years, have you considered the possibility of continuing your education or taking part in skills upgrading?
  • yes
  • no
• T1DA: Thinking specifically about Government of Canada programs and services related to continuous learning, skills or job resources, which programs and services are you aware of?  (UNAIDED) [ ]
• T1DB: Still thinking specifically about Government of Canada programs and services related to continuous learning, skills or job resources, which of the following programs and services are you aware of? SELECT ALL THAT APPLY [ROTATE RESPONSE OPTIONS]
  • Apprenticeship Completion Grant (ACG)
  • Apprenticeship Incentive Grant (AIG)
  • Apprenticeship Incentive Grant for Women
  • Canada Apprentice Loan
  • Canada Student Grants expanded eligibility (more financial support for adults studying)
  • Employment Insurance flexibility (Taking training while on EI)
  • Job Bank – Career tool, job alerts, job match
  • Skilled Trades and Apprenticeship (Red Seal Program)
  • Canada Student Loans
  • Other (specify)
  • None of the above
• T1E: Please rate how likely you would be to recommend, use, or visit the following sources for information on continuous learning, skills or job resources:  [ROTATE STATEMENTS]
  1. Government of Canada websites
  2. Provincial Government websites
  3. Career/education planning sites
  4. University and community college websites
  5. Recruitment sites
  6. Trade unions, certification or accreditation bodies, or industry experts
  • 1– Definitely would not recommend, use, or visit
  • 2
  • 3
  • 4
  • 5– Definitely would recommend, use, or visit
• T1F:[POSE T1F ONLY IF CODES 1-2 OR 4-5 IN T1Ea] [Why would you [codes 4-5 in T1Ea] / Why wouldn’t you [codes 1-2 in T1Ea] [recommend/consult] the Government of Canada’s websites for information on continuous learning, skills or job resources?  [OPEN-ENDED]
• T1FA-C:   All things considered, please rate how satisfied or dissatisfied you are in terms of the Government of Canada’s support for Canadians in the following areas: [ROTATE STATEMENTS]      
  • a. Education
  • b. Skills training
  • c. Employment programs/services
  • 1– Completely dissatisfied
  • 2
  • 3
  • 4
  • 5– Completely satisfied

Demographic Questions

• D2:What is the highest level of formal education that you have completed? SELECT ONE ONLY
  • grade 8 or less
  • some high school
  • high school diploma or equivalent
  • registered Apprenticeship or other trades certificate or diploma
  • college, CEGEP or other non-university certificate or diploma
  • university certificate or diploma below bachelor's level
  • bachelor's degree
  • postgraduate degree above bachelor's level
• D3: Are there any children under the age of 18 currently living in your household?
  • yes 
  • no 
• D4: Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted? SELECT ONE ONLY
  • under $20,000 
  • between $20,000 and $39,999 
  • between $40,000 and $59,999 
  • between $60,000 and $79,999 
  • between $80,000 and $99,999
  • between $100,000 and $149,999 
  • $150,000 and above  
  • prefer not to say
• D5: Where were you born?
  • born in Canada
  • born outside Canada
    • Specify the country: [ ]

ASK IF D5=BORN OUTSIDE CANADA

• D6: In what year did you first move to Canada? [ ] YYYY

ADMISSIBLE RANGE: 1900-2018

• D7: What is the language you first learned at home as a child and still understand? SELECT UP TO TWO
  • English
  • French
  • Other language, specify [ ]

PRE-TEST ONLY: TO BE REMOVED AFTER PRE-TEST [n = at least 15 in each official language]

• Are there any questions in this survey that you found difficult to understand?
  • Yes
  • No

IF YES: Which questions did you find difficult to understand? 

RECORD VERBATIM [ ]

That concludes the survey. This survey was conducted on behalf of EMPLOYMENT AND SOCIAL DEVELOPMENT CANADA. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.

Advertising Campaign Evaluation ToolPost-Campaign Survey

To be conducted after the ads have been run in the media.

Introduction

Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians. Si vous préférez répondre au sondage en français, veuillez cliquer sur français [SWITCH TO FRENCH VERSION].

Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 7 to 9 minutes to complete. This survey is being administered according to the requirements of the Privacy Act, the Personal Information Protection and Electronic Documents Act, the Access to Information Act, and any other relevant legislation.

START SURVEY

To view our privacy policy, click here.

If you require any technical assistance, please contact XXX.

• a. Does anyone in your household work for any of the following organizations? SELECT ALL THAT APPLY
  • a marketing research firm
  • a magazine or newspaper
  • an advertising agency or graphic design firm
  • a political party
  • a radio or television station
  • a public relations company
  • the federal or provincial government
  • none of these organizations

IF “NONE OF THESE ORGANIZATIONS” CONTINUE, OTHERWISE THANK AND TERMINATE.

• b. Are you?
  • Male gender
  • Female gender
  • Gender diverse 
• c. In what year were you born? [ ] YYYY

ADMISSIBLE RANGE 1963-1984

ASK D IF QUESTION C IS LEFT BLANK, OR IF 1963 OR 1984 IN Q.C

• d. In which of the following age categories do you belong? SELECT ONE ONLY
  • less than 18 years old
  • 18 to 24 
  • 25 to 34 
  • 35 to 44
  • 45 to 55
  • 56 to 64
  • 65 or older

[IF “LESS THAN 35 OR MORE THAN 55 YEARS OLD” OR “BLANK”, THANK AND TERMINATE]

• e. In which province or territory do you live? SELECT ONE ONLY
  • Alberta
  • British Columbia
  • Manitoba
  • New Brunswick
  • Newfoundland and Labrador
  • Northwest Territories
  • Nova Scotia
  • Nunavut
  • Ontario
  • Prince Edward Island
  • Quebec
  • Saskatchewan
  • Yukon

IF NO PROVINCE OR TERRITORY IS SELECTED, THANK AND TERMINATE

• f. Do you identify yourself as Indigenous, that is, are you of First Nations, Inuit or Métis descent? PERMIT ONE RESPONSE ONLY
  • Yes, First Nations 1
  • Yes, Inuit 2
  • Yes, Métis 3
  • No 4
• g. [IF ‘YES, FIRST NATIONS’ IN Q.f (CODE 1), POSE] Do you live most of the time on-reserve or off-reserve? PERMIT ONE RESPONSE ONLY
  • On-reserve 1
  • Off-reserve 2
  • Prefer not to say 3
• h. Which of the following categories best describes your current employment status? Are you …: SELECT ONE ONLY
  • working full-time (30 or more hours per week)
  • working part-time (less than 30 hours per week)
  • self-employed
  • unemployed, but looking for work
  • a student attending school full-time
  • a student attending school part-time
  • training in an apprenticeship program
  • retired
  • not in the workforce (Full-time homemaker or unemployed but not looking for work)
  • other employment status
• hh. And do any of these other options also apply to you? SELECT ALL THAT APPLY – DO NOT DISPLAY RESPONSE SELECTED IN Q.h
  • working full-time (30 or more hours per week)
  • working part-time (less than 30 hours per week)
  • self-employed
  • unemployed, but looking for work
  • a student attending school full-time
  • a student attending school part-time
  • training in an apprenticeship program
  • retired
  • not in the workforce (Full-time homemaker or unemployed but not looking for work)
  • other employment status
• i. [ASK IF WORKING FULL-TIME, WORKING PART-TIME, OR SELF-EMPLOYED IN Q.h OR Q.hh] In which, if any, of the following industries or sectors do you currently work?  SELECT ALL THAT APPLY
  • Agriculture
  • Arts
  • Construction
  • Education
  • Engineering
  • Farming
  • Finance
  • Fishing
  • Forestry
  • Hunting
  • Logistics
  • Manufacturing
  • Medicine
  • Mining and Extraction
  • Office and administrative support
  • Public administration
  • Retail sales and related
  • Retail or wholesale trade
  • Science
  • Supply chain
  • Transportation
  • Some other sector or industry: (Please specify which industry or sector you currently [ ]
•  j. [ASK IF UNEMPLOYED BUT LOOKING FOR WORK IN Q.h OR Q.hh] In which, if any, of the following industries or sectors are you currently seeking work?  SELECT ALL THAT APPLY
  • Agriculture
  • Arts
  • Construction
  • Education
  • Engineering
  • Farming
  • Finance
  • Fishing
  • Forestry
  • Hunting
  • Logistics
  • Manufacturing
  • Medicine
  • Mining and Extraction
  • Office and administrative support
  • Public administration
  • Retail sales and related
  • Retail or wholesale trade
  • Science
  • Supply chain
  • Transportation
  • Some other sector or industry: (Please specify which industry or sector you currently seek work in [ ]

Core Questions

ASK ALL RESPONDENTS

• Q1: Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
  • yes 
  • no GO TO T1A
• Q2: Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad? SELECT ALL THAT APPLY
  • cinema
  • Facebook
  • Internet website
  • magazines
  • newspaper (daily)
  • newspaper (weekly or community)
  • outdoor billboards
  • pamphlet or brochure in the mail
  • public transit (bus or subway)
  • radio
  • television
  • Twitter
  • YouTube
  • Instagram
  • LinkedIn
  • Snapchat
  • Spotify
  • Other, specify [ ]
• Q3: What do you remember about this ad? [ ]
• Q4: How did you know that it was an ad from the Government of Canada? [ ]

Campaign Specific Questions

ASK ALL RESPONDENTS

• T1A: Over the past three weeks, have you seen, read or heard any Government of Canada advertising about continuous learning, skills or job resources?
  • Yes 
  • No GO TO T1D1
• T1B: Where have you seen, read or heard this Government of Canada ad about continuous learning, skills or job resources? [SELECT ALL THAT APPLY – ROTATE LIST ON SCREEN, BUT ALWAYS PRESENT ‘OTHER’ LAST]
  • cinema
  • Facebook
  • Internet website
  • magazines
  • newspaper (daily)
  • newspaper (weekly or community)
  • outdoor billboards
  • pamphlet or brochure in the mail
  • public transit (bus or subway)
  • radio
  • television
  • Twitter
  • YouTube
  • Instagram
  • LinkedIn
  • Snapchat
  • Spotify
  • Other, specify [ ]
• T1C: What do you remember about this ad? [ ]

ASK ALL RESPONDENTS    

CAMPAIGN SPECIFIC ATTITUDINAL AND BEHAVIOURAL QUESTIONS MAY BE ADDED HERE

• T1D1: In the past ten years, have you considered the possibility of continuing your education or taking part in skills upgrading?
  • yes
  • no
• T1DA: Thinking specifically about Government of Canada programs and services related to continuous learning, skills or job resources, which programs and services are you aware of? (UNAIDED) [ ]
• T1DB: Still thinking specifically about Government of Canada programs and services related to continuous learning, skills or job resources, which of the following programs and services are you aware of?  SELECT ALL THAT APPLY [ROTATE RESPONSE OPTIONS]
  • Apprenticeship Completion Grant (ACG)
  • Apprenticeship Incentive Grant (AIG)
  • Apprenticeship Incentive Grant for Women
  • Canada Apprentice Loan
  • Canada Student Grants expanded eligibility (more financial support for adults studying)
  • Employment Insurance flexibility (Taking training while on EI)
  • Job Bank – Career tool, job alerts, job match
  • Skilled Trades and Apprenticeship (Red Seal Program)
  • Canada Student Loans
  • Other (specify)
  • None of the above
• T1E: Please rate how likely you would be to recommend, use, or visit the following sources for information on continuous learning, skills or job resources: [ROTATE STATEMENTS]
  • a. Government of Canada websites
  • b. Provincial Government websites
  • c. Career/education planning sites
  • d. University and community college websites
  • e. Recruitment sites
  • f. Trade unions, certification or accreditation bodies, or industry experts
  • 1– Definitely would not recommend, use, or visit
  • 2
  • 3
  • 4
  • 5– Definitely would recommend, use, or visit
• T1F: [POSE T1F ONLY IF CODES 1-2 OR 4-5 IN T1Ea] Why would you [codes 4-5 in T1Ea] / Why wouldn’t you [codes 1-2 in T1Ea] [recommend/consult] the Government of Canada’s websites for information on continuous learning, skills or job resources?  [OPEN-ENDED] [ ]
• T1FA-C:   All things considered, please rate how satisfied or dissatisfied you are in terms of the Government of Canada’s support for Canadians in the following areas: [ROTATE STATEMENTS]      
  • a. Education
  • b. Skills training
  • c. Employment programs/services
  • 1– Completely dissatisfied
  • 2
  • 3
  • 4
  • 5– Completely satisfied
• T1G: Here are some ads that have recently been broadcast on various media. Click here to watch.

[ADS INSERTED HERE]

[CLICK TO GO TO THE NEXT PAGE]

• Over the past few months, have you seen, read or heard these ads or any ads similar to these?
  • yes 
  • no GO TO T1I
• T1H: Where have you seen, read or heard these ads? SELECT ALL THAT APPLY
  • cinema
  • Facebook
  • Internet website
  • magazines
  • newspaper (daily)
  • newspaper (weekly or community)
  • outdoor billboards
  • pamphlet or brochure in the mail
  • public transit (bus or subway)
  • radio
  • television
  • Twitter
  • YouTube
  • Instagram
  • LinkedIn
  • Snapchat
  • Spotify
  • Other, specify [ ]
• T1I: What do you think is the main point these ads are trying to get across? [ ]
• T1J: Please indicate your level of agreement with the following statements about these ads? [ ]

RANDOMIZE STATEMENTS

• These ads catch my attention
  • 1 Strongly Disagree
  • 2
  • 3
  • 4
  • 5 Strongly Agree
• These ads are relevant to me
  • 1 Strongly Disagree
  • 2
  • 3
  • 4
  • 5 Strongly Agree
• These ads are difficult to follow
  • 1 Strongly Disagree
  • 2
  • 3
  • 4
  • 5 Strongly Agree
• These ads do not favour one political party over another
  • 1 Strongly Disagree
  • 2
  • 3
  • 4
  • 5 Strongly Agree
• These ads talk about an important topic
  • 1 Strongly Disagree
  • 2
  • 3
  • 4
  • 5 Strongly Agree
• These ads provide new information
  • 1 Strongly Disagree
  • 2
  • 3
  • 4
  • 5 Strongly Agree
• These ads clearly convey that the Government of Canada has programs and services related to continuous learning, skills or job resources. 
  • 1 Strongly Disagree
  • 2
  • 3
  • 4
  • 5 Strongly Agree

Demographic Questions

• D2: What is the highest level of formal education that you have completed? SELECT ONE ONLY
  • grade 8 or less
  • some high school
  • high school diploma or equivalent
  • registered Apprenticeship or other trades certificate or diploma
  • college, CEGEP or other non-university certificate or diploma
  • university certificate or diploma below bachelor's level
  • bachelor's degree
  • postgraduate degree above bachelor's level
• D3: Are there any children under the age of 18 currently living in your household
  • yes
  • no 
• D4: Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted? SELECT ONE ONLY
  • under $20,000 
  • between $20,000 and $39,999  
  • between $40,000 and $59,999 
  • between $60,000 and $79,999 
  • between $80,000 and $99,999
  • between $100,000 and $149,999  
  • $150,000 and above  
  • prefer not to say
• D5:Where were you born?
  • born in Canada
  • born outside Canada
    • Specify the country [ ]

ASK IF D5=BORN OUTSIDE CANADA

• D6: In what year did you first move to Canada? [ ] YYYY

ADMISSIBLE RANGE: 1900-2019

• D7: What is the language you first learned at home as a child and still understand? SELECT UP TO TWO
  • English
  • French
  • Other language, specify [ ]

Closing (PRE-TEST ONLY):

• D8: Are there any questions in this survey that you found difficult to understand?
  • Yes
  • No
• D9: IF YES IN D8: Which questions did you find difficult to understand? RECORD VERBATIM [ ]

That concludes the survey. This survey was conducted on behalf of EMPLOYMENT AND SOCIAL DEVELOPMENT CANADA. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.