Better Jobs Advertising Campaign Evaluation Survey: Final Methodology Report

Public Opinion Research Number: POR 033-11

Contract Award Date: 2011-01-09

Contract Call up Number: G9178-110001

Contracted cost of research (including HST): $65,028.11

February 13, 2012

Submitted to:

Human Resources and Skills Development Canada
Contact : nc-por-rop-gd@hrsdc-rhdcc.gc.ca

Prepared by : NRG Research Group

« Ce rapport est aussi disponible en français sur demande »

Political Neutrality Certification

I hereby certify as a Senior Officer of Company that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not contain any reference to electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leader.

Signature:

Russell G. Noseworthy
Senior Research Associate
NRG Research Group
Ottawa, Mobile Phone 613.292.0637

Table of Contents

1. Background

In support of the Government of Canada’s Budget 2011 and Throne Speech commitments to skills development, jobs and growth, and following the success of the 2010-11 Better Jobs advertising campaign, HRSDC recommended that the Government of Canada continue to inform Canadians of programs designed to help them make informed educational and career choices, thus strengthening Canada’s labour market inventory.

The 2011-12 campaign leveraged existing creative investments and sought to sustain the resonance of the messaging delivered in fiscal year (FY) 2010-11. Last year’s campaign offered working-age Canadians simple messaging regarding the importance of acquiring post-secondary education and upgrading their skills throughout their working life.

This year the campaign messaging focused on youth, the importance of post-secondary education, as well as financial assistance (loans and grants) to post-secondary students who demonstrate financial need.

1.1 Better Jobs Advertising Campaign

The media buy for the campaign is in excess of $1,000,000. As per the Communications Policy of the Government of Canada, post-campaign testing utilizing the standard Advertising Campaign Evaluation Tool (ACET) is required.

As per the statement of work, the Campaign was marked by two key overarching targets.

  1. The 2011-12 campaign leveraged existing creative investments and sought to sustain the resonance of the messaging delivered in fiscal year (FY) 2010-11. This year’s campaign, like last year’s, offered working-age Canadians simple messaging regarding the importance of acquiring post-secondary education and upgrading their skills throughout their working life.
  2. The campaign also sought, in a general sense, to inform key audiences of how the Government of Canada can help relative to jobs and finding employment.

The fall-winter 2012 campaign ran for six weeks from October 24 to December 4, 2011 and for two weeks from January 2 to January 15, 2012. It included television and Internet advertisements.

2. Research Objectives

The Better Jobs ACET is based on a survey of the general population plus samples of specific target populations as required - in this case, youth (N=300), and the general population (N=1,000) in Canada.

The research objective was to collect information related to:

3. Methodology and Approach

As per above, the proposed public opinion research study involved the conduct of 1,300 computer assisted telephone interviews (CATI) with Canadians 16 years of age or older. Methodological details for the general population (N=1,000) and youth population (N=300) are contained in the sections that follow.

3.1 General Population Survey (Sample Size - 1000)

NRG, on behalf of HRSDC, conducted 1,000 telephone interviews with a random sample of Canadian adults 18 years of age or older.  Households were selected at random and then one person was chosen or selected to participate in the survey using the last birthday respondent selection method1. The margin of error (MOE) for the overall sample is ± 3.1 percentage points2, 19 times out of 20 or at a 95 percent confidence interval (CI).

As per GoC telephone interviewing standards, the survey was pretested on Jan 6th, 2012 - 15 pretest interviews were conducted in English and French.  Revisions were made to the survey questionnaire and all interviews were conducted Jan. 10-19, 2012.

Table one (1) describes the final un-weighted sample distribution by region/province.
Strata Landline Sample Distribution (Un-weighted) Cell Only Sample Distribution(Un-weighted) Total Sample(Un-weighted) Margin of Error
Atlantic Canada 75 25 100 +/- 9.8
Quebec 160 51 211 +/- 6.8
Ontario 205 93 298 +/- 5.7
Manitoba/ Saskatchewan/Nunavut 95 35 130 +/- 8.6
Alberta/Northwest Territory 85 46 131 +/- 8.6
British Columbia/Yukon 80 50 130 +/- 8.6
Canada 700 300 1,000 +/- 3.1

For this component of study a disproportional sampling scheme was employed to provide for more accurate regional/provincial level results. Please note that within each of the above geographically defined strata NRG sampled proportionately to achieve a more or less representative sample across each of the six macro regions. For example, interviews in the Atlantic Provinces are proportionately spread across all four Atlantic Provinces to a total sample of 100. The same was done in the other five regions to ensure that the final sample reflected the range of opinion from across each region. Also, the three territories of Yukon, the Northwest Territories and Nunavut were sampled with their approximate adjacent provincial jurisdiction. The sample was sourced from all traditional landline3 telephone listings in Canada.

3.1.1 Sample Source – General Population Survey

NRG Research Group uses a product from ASDE Survey Sampler called Canada Survey Sampler for sampling the general population in Canada. For additional information on ASDE and this software product please visit www.surveysampler.com. This sampling software is updated every three months by ASDE and it allows NRG to develop random samples at various geography levels (e.g., Province, Economic Region, or Census Division) among landline households in Canada. These samples incorporate both listed and unlisted telephone numbers to ensure adequate coverage of all landline phone numbers in Canada.

For this survey, NRG randomly selected un-listed phone numbers with at least a 30 per cent chance of being live (i.e., ringing in a residential household) for inclusion in a sample. In so doing, NRG incorporated new and changed phone listings, as well as new neighborhoods in our sample. The unlisted numbers were, as is our normal practice, restricted to 20 per cent of the overall sample for this study. In total NRG research dialed 16,537 landline telephone numbers for this sample with just over 3,300 of these sourced from unlisted households.

All samples for a study were maintained in a custom database application. As phone numbers were added to a database, they were compared against a voluntary market research national do not call list, and all matches were removed from the sample prior to randomization/sample selection. The sample frame for the youth population survey was derived from a combination of Canada’s traditional landline and cellular telephone sample universes. All sample units used in the general population survey were excluded from the youth sample frame.

The general population survey data have been weighted according to the most recent Statistics Canada census information to ensure that the national level data reflects the views and perceptions of the respective target populations by sample type (landline versus CPO) gender and age within region/province. This weighting scheme is recommended for analysis of the national level results and not for within region level analysis. Regional level analysis should employ an alternate weighting scheme that takes into account the population characteristics of the region as opposed to the entire country.

3.2 Youth Survey (Sample Size - 300)

NRG, on behalf of HRSDC, conducted 300 12 minute computer assisted telephone interviews (CATI) with a randomly selected sample of Canadians between 16 and 24 years of age. On Jan. 6, 2012, 10 pretest interviews were conducted in English and French. Revisions were made to the survey questionnaire, and all interviews were conducted from Jan. 10 to Jan. 24, 2012. The margin of error4 (MOE) for the overall survey sample is +/- 5.65 percentage points, 19 times out of 20 or at a 95 percent confidence interval (CI). 

Table two below, describes the final regional/provincial sample distribution.
Strata Sample Size(Un-Weighted) Margin of Error
Atlantic Canada 26 +/-19.2
Quebec 60 +/-12.7
Ontario 90 +/10.3
Manitoba/ Saskatchewan/Nunavut 30 +/17.9
Alberta/Northwest Territory 44 +/14.8
British Columbia/Yukon 50 +/13.9
Canada 300 +/- 5.65

3.2.1 Sample Source –Youth Survey

As proposed, the sample for the youth population survey was derived from a combination of Canada’s traditional landline and cellular telephone sample universes.

NRG selected landline sample based on the population density of the target population with approximately 65 percent of the total youth sample dialed sourced from Dissemination Area (s) with an incidence greater that 12 per cent for our target group – youth 16-24 years of age. The remainder (35%) was sourced from a national random sample of active cell-phone listing.

Table three (3) shows the sample distribution by telephone service type – landline versus cellular/smart/mobile telephone listing5. A total of 106 cell phone interviews were conducted via cell-phone with youth 16-24 years of age. A little more than 5 in 10 cellular telephone respondents reside in households that also have landline telephone service while around 5 in 10 reside in cellular telephone only service households. Table three summarizes the sample breakdown by sample type.

Table 3
Sample Source Sample Frame Description Un-weighted Sample Weighted Sample
RDD Cell-Phone Sample Random Sample of Active Cell-phone Listings 106 118
Landline- Census Overlay Sample Dissemination Areas with the highest concentrations of the target population (top five percent per region) 194 182
TOTAL 300 300

3.3 Data Weighting and Tabulation

The survey data have been weighted according to the most recent Statistics Canada census information to ensure that the national level data reflects the views and perceptions of the respective target populations by age, gender within region/province. As is the case with the general population sample, this weighting scheme is recommended for analysis of the national level results and not for within region level analysis. Regional level analysis should employ an alternate weighting scheme that takes into account the population characteristics of the region as opposed to the entire country. Caution: given the relatively small sample size of the youth survey, the ability to conduct within region analysis is extremely limited. 

3.4 Quality Control

As per GoC standards and guidelines a minimum of 10 per cent of all completed interviews were monitored or otherwise verified – both audio and visual verification. A total of 19,332 landline (16,537) and cellular (2,795) telephone numbers were included in the general population survey sample. A total of 28,596 landline (20,468) and cell phone (8,128) listing were included in the youth survey sample frame. All English language interviewing was completed at NRG’s main data collection facility located in Winnipeg, Manitoba. All French language interviewing was conducted in Ottawa, Ontario by Elemental Research Group. The response rates for the both samples as well as their respective response rate calculations can be seen below.

Table 4
Universe Sample Dialled Unresolved (U) (Answering Machine, Busy Signal, No Answer) In-Scope Responsive (R) (Disqualified Due To Age, Completed Interviews, Quota Full, Other Disqualified) In-Scope Non Responsive (IS) (Respondent Refusals, Household Refusals, Unresolved Call-backs) Invalid (IV) (Business Listing, Not in Service, Fax modem, other non residential listing)
General Population 18+ 19,332 10,247 1,557 3,319 5,449
Youth 16-24 28,596 10,478 9,824 9,824 4,209
Table 5
Universe Response Rate*
General Population 18+ 10.3%
Youth 16-24* 42.7%

*Responding(R)/Unresolved (u) + In-Scope Non-Responding (IS) + Responding (R)

The youth survey’s response rate (42.7%), on the surface, is significantly higher than the rate achieved in the general population survey (10.3%). However, the youth sample response rate is being artificially inflated by a higher than usual number of household that were screened out of the survey due to age (9350). As illustrated in table six below, removing the “screened out” households from both sides of the empirical calculation formula has a lowering effect on the youth survey’s response rate. The reduced response rate of 3.9 per cent is on the low side of the preferred range for a survey of this nature. The general population response rate is also on the “low” side of the preferred response rate for a survey of that nature. Please note that these rates have improved since last year. Therefore, as per GoC Telephone Interviewing Guidelines for surveys with “low response rates”, we have provided an analysis of non-response, for both samples, in the section that follows.

Table 6
Youth Sample Response Rate Calculations
Unadjusted Response Rate = R/(U+IS+R) (9884/(10478+2785+9884))  42.7%
Adjusted Response Rate =AR/(U+IS+AR) (474/(10478+2785+474)) 3.9%

In essence then, we have two measures of response coming out of the youth sample.

  1. Response rate including screened out units due to age (42.7%)
  2. Response rate excluding screened out units due to age. (3.9%)

The adjusted response rate excluding the screened out units due to age is a far better indicator of the youth sample’s true response rate. A full summary of both surveys’ final calling results and response rates is located in Appendix A of this document.

3.5 Non-Response Bias - General Population and Youth Sample

According to the GoC’s standards and guidelines for the conduct of telephone surveys, non response bias is the systematic difference between true population values and the average result from all possible samples owing to non-response. The analysis of non-response bias will consist of a comparison of three survey variables - region, gender, age - in both samples with the equivalent parameters of the population. Table seven below compares the general population sample on a regional basis to the population parameters by gender and age.

Table 7
STRATA Gender
M
POP 		Gender
M
SAM 		Gender
F
POP 		Gender
F
SAM 		Age
18-34
POP 		Age
18-34
SAM 		Age
35-54
POP 		Age
35-54
SAM 		Age
55 plus
POP 		Age
55 plus
SAM
BC/ YK 48.4% 53.8% 51.6% 46.2% 26.7% 33.8% 39.5% 36.0% 33.8% 41.0%
AB/NWT 49.7% 48.1% 50.3% 51.9% 32.7% 32.8% 40.5% 35.1% 26.8% 31.8%
MB/SK/NV 48.4% 47.7% 51.6% 52.3% 29.1% 23.8% 38.3% 40.9% 32.6% 33.9%
ON 48.0% 48.0% 52.0% 52.0% 28.2% 24.8% 40.0% 38.5% 31.8% 36.9%
QB 48.3% 47.9% 51.7% 52.1% 26.9% 33.2% 39.2% 37.4% 33.9% 28.2%
AC 47.8% 48.0% 52.2% 52.0% 25.6% 22.0% 39.4% 31.5% 35.0% 33.1%

Table seven shows men and women across all age categories (18-34, 35-54, 55+) analyzed were sampled more or less to their proportions in each region. 

To correct for this minor sample imbalance relative to the target population’s parameters, the sample has been weighted by region, gender and age. The amount of bias this has introduced to the survey results can be measured and/or evaluated to some degree by examining the percentage change in the survey’s weighted and un-weighted findings. Our analysis reveals that the survey’s final un-weighted sample reflects the Canadian population by gender and age across all six regions. Our analysis reveals that there is very little to no difference between the survey’s weighted and un-weighted results.

Table eight and nine below compares the youth survey sample distribution on a regional basis to the population parameters for the target population of youth by gender and age.

Table 8: Gender by Region
Strata Male 16-24
POP 		Male 16-24
SAM		 Female 16-24
POP 		Female 16-24
SAM
BC/ YK 13.1% 14.2% 12.9% 19.1%
AB/NWT 12.0% 15.5% 11.6% 13.8%
MB/SK/NV 7.4% 10.8% 7.4% 9.2%
ON 39.2% 31.8% 39.4% 28.3%
QB 21.5% 20.3% 21.7% 19.7%
AC 6.9% 7.4% 7.0% 9.9%
Table 9: Age by Region
Strata 16-24
POP 		16-24
SAM 		16-17
POP 		16-17
SAM 		18-20
POP 		18-20
SAM 		21-24
POP 		21-24
SAM
BC/YK 13.0% 16.7% 13.0% 19.4% 13.1% 11.9% 12.9% 19.4%
AB/NWT 12.3% 14.7% 11.6% 17.2% 12.1% 19.3% 12.9% 7.1%
MB/SK/NV 7.0% 10.0% 7.2% 4.3% 7.1% 15.6% 6.8% 9.2%
ON 38.7% 30.0% 39.1% 33.3% 39.4% 25.7% 37.9% 31.6%
QC 22.0% 20.0% 21.7% 18.3% 21.2% 18.3% 22.7% 23.5%
AC 7.0% 8.7% 7.2% 7.5% 7.1% 9.2% 6.8% 9.2%

Table eight and nine show young men and young women were sampled more or less to their 2006 census proportions in each region.  Table nine shows the un-weighted national sample distribution by age is very close to population parameters for young Canadians aged 16-17, 18-20, and 21-24. To correct for the minor regional sample imbalances relative to the target population’s parameters, the sample has been weighted by gender and age across each geographic strata indicated above. As per above, the amount of bias this minor imbalance may introduce to the survey’s findings can be measured to some degree by examining the percentage change in the survey’s weighted and un-weighted findings. Our analysis reveals that the survey’s final un-weighted sample reflects the target population of youth by gender. In addition, very little to no difference is observed in the weighted and un-weighted survey results.

Appendix A: Call Dispositions

Final Calling Dispositions General Population Study
Final Call Disposition Totals % Landline % CPO % Code
No Answer 7977 41.3% 7093 42.9% 884 31.6% U
Line In Use/Busy Signal 325 1.7% 304 1.8% 21 0.8% U
Not in Service 3825 19.8% 3273 19.8% 552 19.7% IV*
Quota Full/Over Quota 182 0.9% 63 0.4% 119 4.3% R
Not Qualified (No Eligible Respondent in Household) 189 1.0% 94 0.6% 95 3.4% R
Refused to Respond 2514 13.0% 2066 12.5% 448 16.0% IS
Unresolved Callback 805 4.2% 638 3.9% 167 6.0% IS
Business Listing 80 0.4% 68 0.4% 12 0.4% IV
Fax/Modem 304 1.6% 303 1.8% 1 0.0% IV
Language DQ 186 1.0% 155 0.9% 31 1.1% R
Answering Machine 1945 10.1% 1780 10.8% 165 5.9% U
Completed Interview 1000 5.2% 700 4.2% 300 10.7% R
Total Numbers Attempted 19332 1 16537 1 2795 1 TA
Unresolved (U) (No Answer (7977) + Answering Machine (1945) + Busy Signal (325)) 10247
In-Scope Non-Responding (IS)(Refused to Respond (2514) + Unresolved Call-back (805) 3319
In-scope Responding (R) (Completions (1000) + Lang DQ (186) + Other DQ (189) +Quota Full (182) 1557
Response Rate (MRIA Empirical Method =R/(U+IS+R)) 1557/(10247+3319+1557) 10.3%

*Invalid

Analysis of non response has been provided for this sample.

Final Calling Dispositions Youth Population Survey
Final Call Disposition Total % Landline % Cell % Code
No Answer 7451 26.1% 5655 27.6% 1796 22.1% U
Line In Use/Busy Signal 172 0.6% 115 0.6% 57 0.7% U
Not in Service 5118 17.9% 2446 12.0% 2672 32.9%  IV*
Quota Full/Over Quota 60 0.2% 33 0.2% 27 0.3% R
Disqualified (DQ) (Due to age/not 16-24) 9350 32.7% 7516 36.7% 1834 22.6% R
Refused to Respond 2495 8.7% 1770 8.6% 725 8.9% IS
Unresolved Callback 290 1.0% 216 1.1% 74 0.9% IS
Business Listing 114 0.4% 59 0.3% 55 0.7% IV
Fax/Modem 217 0.8% 214 1.0% 3 0.0% IV
Language DG 174 0.6% 122 0.6% 52 0.6% R
Answering Machine 2855 10.0% 2128 10.4% 727 8.9% U
Completed Interview 300 1.0% 194 0.9% 106 1.3% R
Total Numbers Attempted 28596 100.0% 20468 100.0% 8128 100.0% TA
Unresolved (U) (No Answer (7451) + Answering Machine (2855) + Busy Signal (172)) 10478
In-Scope Non Responding (IS) (Refused to respond (2495) + Unresolved Call-back (290) 2785
In-scope Responding (R) (Completions (300) + DQ Due To Age (9350) + Lang DQ (174)+QF (60) 9824
Response Rate ( MRIA Empirical Method) R/(U+IS+R) (9884/(10478+2785+9884)) 42.7%
*Adjusted for Screening AR/(U+IS+AR) (474/(10478+2785+474) 3.9%

*Invalid

Incidence of contact for the youth sample frame is provided below.

Disposition Cell Sample Landline Sample Total
Complete 106 194 300
Not Qualified 1886 7632 9824
Quota full 27 33 60
Terminates 0 0 0
Incidence* 5.25% 2.5% 2.9%

*Incidence Calculation: Completes + Terminates/ Completes + Terminates + Not Qualified

+Quota Full

Appendix B: Questionnaire

ADVERTISING CAMPAIGN EVALUATION TOOL

INTRODUCTION: Landline Sample:

Hello, my name is _______________. I’m calling on behalf of NRG Research Group. I’d like to ask you a few questions on current issues of interest to Canadians. Your responses will be kept entirely confidential and anonymous. This survey is registered with the Marketing Research and Intelligence Association's (MRIA).

a) May I please speak with the person in your household who is 18 years of age or older and who has had the most recent birthday? Would that be you? [IF THAT PERSON IS NOT AVAILABLE ARRANGE CALLBACK]

b) Would you be willing to take part in this survey? We can do it now or at a time more convenient for you.

The survey will take about 7 minutes to complete.

CONTINUE TO Q1A

Introduction for the general population - Cellular phone only sample:

Hello, my name is _______________. I’m calling on behalf of NRG Research Group. We realize that we have contacted you on your cell-phone. I’d like to ask you a few questions on current issues of interest to Canadians.

a) Are you in a safe place to talk?

RESCHEDULE [IF INTRO 02]

b) We would like to conduct this interview with you when it is safe and convenient to do. During weekdays we can call anytime from XX:XX to XX:XX pm and on weekends from XXXX to XX:XX PM your time.

c) Have I reached you on a cellular phone?

d) In your home, do you have a phone line that is not a cell phone?

As I said, I’ll be asking you a few questions on current issues of interest to Canadians. Your responses will be kept entirely confidential and anonymous. This survey is registered with the Marketing Research and Intelligence Association's (MRIA).

[IF ASKED] The survey will take about 7 minutes to complete.

CONTINUE TO Q1A

Introduction for the youth – Landline Sample:

a) Hello, my name is _______________. I'm calling on behalf of NRG Research Group. Is there anyone in your household between the ages of 16 and 24 years old?

IF YES: Can I please speak with that person? [IF THAT PERSON IS NOT AVAILABLE ARRANGE CALLBACK]

WHEN CONTACT TARGET RESPONDENT

b) Hello, my name is _______________. I'm calling on behalf of NRG Research Group. We are currently conducting a study with young Canadians between the ages of 16 and 24 years old. Would you be willing to take part in this study?

We can do it now or at a time more convenient for you. Please note that your responses will be kept entirely confidential and anonymous, and this survey is registered with the National Survey Registration System. Your participation in this survey is voluntary, but would be extremely helpful.

May I proceed?

[IF ASKED] The survey will take about 7 minutes to complete.

Introduction for the youth - Cellular phone only sample:

Hello, my name is _______________. I'm calling on behalf of NRG Research Group. We realize that we have contacted you on your cell-phone. We are conducting surveys with Canadians 16-24 years old.

A). Are you in a safe place to talk?

b) Do you fall into that age group?

c) We would like to conduct this interview with you when it is safe and convenient to do. During weekdays we can call anytime from XX:XX to XX:XX pm and on weekends from XXXX to XX:XX PM your time.

d) Have I reached you on a cellular phone?

e) In your home, do you have a phone line that is not a cell phone?

As I said, I’ll be asking you a few questions on current issues of interest to Canadians. Your responses will be kept entirely confidential and anonymous. This survey is registered with the Marketing Research and Intelligence Association's (MRIA).

[IF ASKED] The survey will take about 7 minutes to complete.

CONTINUE TO Q1A

CORE QUESTIONS: ASK ALL RESPONDENTS

Q1A: Over the past few weeks or so, have you seen, heard or read any advertising about programs and services available to help Canadians obtain the right skills and training?

Q1B: What can you remember about this advertising? What words, sounds or images come to mind? [ACCEPT MULTIPLE RESPONSES]

(DO NOT RED LIST)

Q2: PROGRAMMING NOTE: RANDOMIZE Q2A, Q2B

Q2A: Do you remember seeing or hearing a [television] ad over the past few weeks or so [where a young woman morphs into an engineer, another young woman morphs into a construction worker and a young man morphs into a health care professional while a female voice talks about getting the right skills and training when thinking about your career?]?

Q2B: Do you remember seeing an Internet ad over the past few weeks or so showing a young female engineer, a young female construction worker and a young male health care professional? The ad talks about how you need the right tools to get the right skills and training for your career?

IF RESPONDENT RECALLS ANY OF THE ADS IN Q2 SERIES OF QUESTIONS, CONTINUE; OTHERWISE SKIP TO Q6.

Q3: Thinking about the advertising that you saw/heard, what do you think was the main point this advertising was trying to get across?

[OPEN-ENDED, ACCEPT ONE RESPONSE]

(DO NOT READ LIST)

Q4A: Did you do anything as a result of seeing/hearing this advertising?

Q4B: What did you do? Anything else?

[ACCEPT MULTIPLE RESPONSES]

(DO NOT READ LIST)

 Q4C: OPTIONAL QUESTION ON ATTITUDE / BEHAVIOUR CHANGE

[IF RESPONDENT SAID ‘YES’ AT Q4A, USE THIS INTRO]

Which, if any, of the following things did you do as a result of seeing/hearing this advertising?

[IF RESPONDENT SAID ‘NO’ ‘DON’T KNOW’ OR ‘REFUSED’ AT Q4A, USE THIS INTRO].

While you said you don’t recall doing anything as a result of the advertising, I’d like to briefly ask you about some specific things that you may have done. Did you…?

INSERT POSSIBLE ACTIONS SPECIFIC TO CAMPAIGN OBJECTIVES

RANDOMIZE a-d AND, IF APPLICABLE, EXCLUDE ITEMS MENTIONED BY RESPONDENT AT Q4B

Q5A: Thinking about the advertising you saw/heard, who do you think produced it? That is, who paid for it?

[DO NOT READ LIST, ACCEPT ONE RESPONSE]

Q5B: [ASK Q5B IF Q5A = ‘Government (General) OTHERWISE GO TO Q6]

Which level of government?

[READ LIST, ACCEPT ONE RESPONSE]

Q6: ASK ALL RESPONDENTS: The next three questions have to do with the performance of the government in general.

How would you rate the performance of the Government of Canada on [providing information about programs and services available to Canadians who wish to enhance their skills and training]? Please use a 7-point scale, where 1 means terrible, 7 means excellent and the mid-point, 4, means neither good nor bad.

Q7: And, using the same scale how would you rate the performance of the Government of Canada in providing information to the public in general? [REPEAT SCALE IF NECESSARY]

Q8: Generally speaking, how would you rate the overall performance of the Government of Canada? Once again, please use the same scale.

[REPEAT SCALE IF NECESSARY]

CAMPAIGN-SPECIFIC QUESTIONS

CS1. [ASK ONLY IF SAID “Got information about training and/or education” IN Q4C OTHERWISE GO TO DEMOGRAPHIC QUESTIONS]

Earlier, you mentioned that after seeing the advertising you got information about training and/or education. Where did you get this information?

[OPEN-ENDED- DO NOT READ. ACCEPT ALL]

CORE DEMOGRAPHIC QUESTIONS

Finally, I’d like to ask you some questions for statistical purposes. I'd like to remind you that all your answers are completely confidential.

[FOR CELLPHONE ONLY/RDD CELLPHPHONE SAMPLE]:

CD1. In which province or territory do you live? [RECORD ANSWER]

 D1: Which of the following categories best describes your current employment status? Are you...? [READ LIST, ACCEPT ONE RESPONSE]

D2: What is the highest level of formal education that you have completed to date? [READ LIST, ACCEPT ONE RESPONSE]

D3: In what year were you born?

D4: Are there any children under the age of 18 currently living in your household?

D5: Which of the following categories best describes your total household income? That is, the total income of all persons in your household, before taxes? [READ LIST, ACCEPT ONE RESPONSE]

D6: [Ask if Cell-Phone respondent]: What is your postal code?

MEDIA CONSUMPTION QUESTION

MEDIA: In the last two weeks, have you…

RANDOMIZE MEDIA CHANNELS

That Concludes the Survey. This survey was conducted on behalf of Human Resources and Skills Development Canada. In the coming months the report will be available from Library and Archives Canada. Thank you very much for taking part in this survey. It is appreciated.

INTERVIEWER RECORDS:

[IF RESPONDENT ASKS FOR MORE INFORMATION ABOUT THE SURVEY, INVITE THEM TO A) CALL THE NATIONAL SURVEY REGISTRATION SYSTEM AT 1-888-608-6742, EXT 8728, B) OFFER TO RECORD NAME AND NUMBER AND GIVE THIS INFORMATION TO RUSSELL NOSEWORTHY AT NRG, AND/OR 3) GIVE RESPONDENT NAME OF HRSDC CONTACTS: GUY SCHRYBURT AT 1-819-997-7353 OR ALEXANDRA HICKEY AT 1-819-994-2245 AND/OR COLLECT RESPONDENT CONTACT INFORMATION AND OFFER TO HAVE GUY OR ALEXANDRA CONTACT THE RESPONDENT

OUTIL D’ÉVALUATION DE CAMPAGNES PUBLICITAIRES

VEUILLEZ CONSULTER LES MODALITÉS D’UTILISATION.

INTRODUCTION

Introduction pour la population générale – échantillon aléatoire (RDD):

Bonjour, je m’appelle _______________. Je vous appelle au nom de la firme de recherche NRG. J’aimerais vous poser quelques questions au sujet d’enjeux actuels d’importance pour les Canadiens. Je tiens à signaler que vos réponses demeureront absolument confidentielles et que ce sondage est enregistré auprès de l'Association de la recherche et de l'intelligence marketing (ARIM).

a) Pourrais-je parler à un membre de votre foyer qui a 18 ans ou plus ayant le plus récemment célébré son anniversaire? Seriez-vous cette personne? [SI LA PERSONNE N’EST PAS DISPONIBLE, FIXER UN MOMENT POUR LE RAPPEL]

b) Seriez-vous disponible pour prendre part à ce sondage? Nous pouvons le faire dès maintenant ou à un moment plus propice pour vous.

[SI DEMANDÉ] Le sondage est d’une durée d’environ 12 minutes.

PASSER A Q1A

Introduction pour la population générale - échantillon des utilisateurs de téléphone cellulaire seulement:

Bonjour, je m’appelle _______________. Je vous appelle au nom de la firme de recherche NRG. J’aimerais vous poser quelques questions au sujet d’enjeux actuels d’importance pour les Canadiens.

a) Êtes-vous à un endroit sécuritaire pour répondre à un sondage?

b) Nous aimerions effectuer le sondage à un moment plus propice pour vous. Durant la semaine nous pouvons vous rappeler entre XX:XX et XX:XX pm et durant la fin de semaine entre XX:XX et XX:XX. Quel moment vous conviendrait le mieux?

c) Est-ce que je vous ai rejoint sur un téléphone cellulaire?

d) A la maison, avez-vous une ligne téléphonique traditionnelle autre qu’un cellulaire?

Tel que mentionné, je vais vous poser quelques questions au sujet d’enjeux actuels d’importance pour les Canadiens. Je tiens à signaler que vos réponses demeureront absolument confidentielles. Ce sondage est enregistré auprès de l'Association de la recherche et de l'intelligence marketing (ARIM).

[SI DEMANDÉ] Le sondage est d’une durée d’environ 12 minutes.

PASSER A Q1A

Introduction pour les jeunes 16-24 – échantillon aléatoire (RDD):

a) Bonjour, je m’appelle _______________. Je vous appelle au nom de la firme de recherche NRG. Y a-t-il une personne dans votre foyer âgée de 16 à 24 ans?

SI OUI: Est-ce que je pourrais parler à cette personne? [SI CETTE PERSONNE N'EST PAS DISPONIBLE, ARRANGER UN RAPPEL]

LORSQUE VOUS PARLEZ AU BON RÉPONDANT

b) Bonjour, je m’appelle _______________. Je vous appelle au nom de la firme de recherche NRG. Nous menons présentement un sondage auprès de jeunes Canadiens et Canadiennes âgés de 16 à 24 ans. Accepteriez-vous de participer à ce sondage?

Nous pourrions commencer maintenant, à moins que vous ne préfériez un autre moment. Je tiens à signaler que vos réponses demeureront absolument confidentielles et que ce sondage est inscrit auprès du système national d’enregistrement des sondages. Votre participation à ce sondage est volontaire, mais elle serait très utile.

Puis-je commencer?

[SI DEMANDÉ] Le sondage est d’une durée d’environ 12 minutes.

Introduction pour les jeunes - échantillon des utilisateurs de téléphone cellulaire seulement:

Bonjour, je m’appelle _______________. Je vous appelle au nom de la firme de recherche NRG. Nous réalisons que nous vous appelons sur votre téléphone cellulaire. Nous menons des sondages auprès de Canadiens et Canadiennes qui ont entre 16 et 24 ans.

1) Êtes-vous à un endroit sécuritaire pour répondre à un sondage?

b) Faites-vous parti de ce groupe d’âge (16-24 ans)?

c) Êtes-vous à un endroit sécuritaire pour répondre à un sondage?

d) Nous aimerions effectuer le sondage à un moment plus propice pour vous. Durant la semaine nous pouvons vous rappeler entre XX:XX et XX:XX pm et durant la fin de semaine entre XX:XX et XX:XX. Quel moment vous conviendrait le mieux?

e) Est-ce que je vous ai rejoint sur un téléphone cellulaire?

e) A la maison, avez-vous une ligne téléphonique traditionnelle autre qu’un cellulaire?

Tel que mentionné, je vais vous poser quelques questions au sujet d’enjeux actuels d’importance pour les Canadiens. Je tiens à signaler que vos réponses demeureront absolument confidentielles. Ce sondage est enregistré auprès de l'Association de la recherche et de l'intelligence marketing (ARIM).

[SI DEMANDÉ] Le sondage est d’une durée d’environ 7 minutes.

PASSER A Q1A

QUESTIONS GÉNÉRIQUES

DEMANDER À TOUS

Q1A: Au cours des dernières semaines, est-ce que vous avez vu, entendu ou lu des annonces sur les programmes et services disponibles pour aider les Canadiens et Canadiennes à acquérir les compétences et la formation dont ils ont besoin?

Q1B: De quoi vous souvenez-vous à propos de cette (ces) annonce(s)? Quels sont les mots, les sons ou les images qui vous viennent à l’esprit? [ACCEPTER TOUTES LES RÉPONSES PERTINENTES]

Q2: NOTE POUR LE PROGRAMMEUR: PRÉSENTER LES QUESTIONS Q2A ET Q2B ALÉATOIREMENT

Q2A: Vous rappelez-vous avoir vu ou entendu au cours des dernières semaines une annonce à la télévision présentant une jeune femme se transformant en ingénieure, une autre une jeune femme se transformant en ouvrière en construction et un jeune homme se transformant en professionnel de la santé pendant que la voix d'une femme parle d'acquérir les compétences et la formation nécessaires lorsqu'on pense à sa carrière?

Q2B: Vous rappelez-vous avoir vu ou entendu au cours des dernières semaines une annonce sur internet présentant une jeune ingénieure, une jeune ouvrière en construction et un jeune professionnel de la santé? L'annonce parle d'acquérir les compétences et la formation nécessaires pour votre carrière?

DÉCLARATION SPONTANÉE : Ne s’applique pas/n’utilise pas l’Internet

SI LE/LA RÉPONDANT(E) SE SOUVIENT DE L’UNE OU L’AUTRE DES ANNONCES SIGNALÉES DANS LA SÉRIE DE QUESTIONS Q2, CONTINUER; AUTREMENT, PASSER À LA Q6.

Q3: En ce qui concerne l’annonce que vous avez vue/entendue, à votre avis, quel était le message principal que l’annonce voulait communiquer? [QUESTION OUVERTE, ACCEPTER UNE SEULE RÉPONSE

Q4A: Avez-vous fait quelque chose en réponse à cette (ces) annonce(s)?

Q4B: Qu’avez-vous fait? Y a-t-il autre chose?

[NE PAS LIRE LA LISTE, CODER TOUTES LES RÉPONSES PERTINENTES]

Q4C: QUESTION OPTIONELLE CONCERNANT CHANGEMENT D’ATTITUDE /DE COMPORTEMENT

[SI LE/LA RÉPONDANT(E) A DIT 'OUI' À Q4A, LIRE CETTE INTRODUCTION]

Lesquelles de ces actions avez-vous prises en réponse à cette annonce?

[[SI LE/LA RÉPONDANT(E) A DIT ‘NON’, NSP/REF À LA QUESTION 4A, LIRE CETTE INTRODUCTION]

Bien que vous ayez dit que vous ne vous rappeliez pas avoir fait quoi que ce soit en réponse à cette annonce, j’aimerais vous demander brièvement certaines choses spécifiques que vous auriez pu faire. Avez-vous …

RÉPARTIR ALÉATOIREMENT a-d ET, S'IL Y A LIEU, EXCLURE LES ÉLÉMENTS MENTIONNÉS À Q4B

Q5A: En ce qui concerne la publicité que vous avez vue/entendue, à votre avis, qui en était l’auteur? Autrement dit, qui l’a financée?

[NE PAS LIRE, ACCEPTER UNE SEULE RÉPONSE]

Q5B: SI LE/LA RÉPONDANT(E) A RÉPONDU « GOUVERNEMENT – EN GÉNÉRAL » À Q5A, DEMANDER Q5B, AUTREMENT PASSER À Q6.

Quel ordre de gouvernement? [LIRE LA LISTE, ACCEPTER UNE SEULE RÉPONSE]

Q6: DEMANDER A TOUS: Les trois prochaines questions concernent le rendement du gouvernement du Canada en général.

Quelle serait votre évaluation du rendement du gouvernement du Canada pour ce qui a trait à [fournir des renseignements sur les programmes et services disponibles aux Canadiens et Canadiennes qui désirent perfectionner leur connaissances et leurs compétences?] Veuillez répondre selon une échelle de sept points, où « 1 » signifie affreux, « 7 », excellent et le point milieu, « 4 », ni bon, ni mauvais.

Q7: Et selon la même échelle [RÉPÉTER AU BESOIN], quelle serait votre évaluation du rendement du gouvernement du Canada pour ce qui est de communiquer de l’information à la population?

Q8: De façon générale, quelle serait votre évaluation du rendement global du gouvernement du Canada? Encore une fois, veuillez utiliser la même échelle [RÉPÉTER SI NÉCESSAIRE].

FIN DES QUESTIONS GÉNÉRIQUES [EXCLUANT QUESTIONS DÉMOGRAPHIQUES]

DES QUESTIONS SPÉCIFIQUES A LA CAMPAGNE PEUVENT ETRE AJOUTÉES ICI.

CS1. [DEMANDÉ SEULEMENT SI LE RÉPONDANT A “Obtenu des renseignements sur la formation et/ ou ”A LA QUESTION Q4C SINON PASSER AUX QUESTIONS DÉMOGRAPHIQUES]

Vous avez mentionné plus tôt qu'après avoir vu l'annonce, vous avez obtenu des renseignements sur la formation et/ou l'éducation. Où avez-vous obtenu ces renseignements?

[QUESTION OUVERTE - NE PAS LIRE. ACCEPTER TOUTES LES RÉPONSES]

QUESTIONS DÉMOGRAPHIQUES DE BASE

En terminant, j’aimerais vous poser certaines questions pour les besoins de la compilation de statistiques. Je tiens à vous rappeler que toutes vos réponses demeureront entièrement confidentielles.

[FOR CELLPHONE ONLY/RDD CELLPHPHONE SAMPLE]:

Dans quelle province ou territoire demeurez-vous? [ENTRER LA RÉPONSE]

D1: À l’heure actuelle, quelle est votre situation dans le contexte de l’emploi? Êtes-vous...? [LIRE LA LISTE, ACCEPTER UNE SEULE RÉPONSE]

D2: Quel est le plus haut niveau de scolarité que vous avez atteint? [LIRE LA LISTE, ACCEPTER UNE SEULE RÉPONSE]

D3: En quelle année êtes-vous né(e)?

D4: Est ce qu’il y a des enfants de moins de 18 ans qui vivent dans votre foyer ?

D5: Laquelle des catégories suivantes décrit le mieux la somme des revenus avant impôts de tous les membres de votre foyer?

[LIRE LA LISTE, ACCEPTER UNE SEULE RÉPONSE]

D6: [S'il s'agit d'un répondant sur téléphone cellulaire]: Quel est votre code postal?

QUESTIONS PORTANT SUR L’UTILISATION DES MEDIAS

MEDIA : Au cours des deux dernières semaines, avez-vous…

PRÉSENTER LES MÉDIAS ALÉATOIREMENT

Voilà qui met fin au sondage que nous avons effectué pour le compte de Ressources humaines et Développement des compétences Canada.

Dans les mois à venir, le rapport sera disponible de Bibliothèque et archives Canada. Nous vous remercions beaucoup d’avoir pris le temps d’y participer, nous vous en sommes reconnaissants.

POUR LES DOSSIERS DE L’ENQUÊTEUR :

SI LE RÉPONDANT DÉSIRE PLUS D’INFORMATION SUR CE SONDAGE, A) INVITEZ-LE A APPELER LE SYSTEME NATIONAL D’ENREGISTREMENT DES SONDAGES AU NUMÉRO 1-888-608-6742, POSTE 8728, B) DEMANDER-LUI SON NOM ET NUMÉRO DE TÉLÉPHONE ET DONNER CETTE INFORMATION A RUSSELL NOSEWORTHY CHEZ NRG, ET/OU 3) DONNER-LUI LE NOM D’UNE DES PERSONNES RESPONSABLES CHEZ RESSOURCES HUMAINES ET DÉVELOPPEMENT DES COMPÉTENCES CANADA: GUY SCHRYBURT AU NUMÉRO 1-819-997-7353 OU ALEXANDRA HICKEY AU NUMÉRO 1-819-994-2245 OU ENCORE PRENEZ EN NOTE LES COORDONNÉES DU RÉPONDANT ET DEMANDÉ A GUY OU ALEXANDRA DE LE RAPPELER

1 This method selects one respondent per household who celebrated their birth last or most recently to respond to the survey questionnaire.

2 Please note that the MOE is greater for sub-groups of the overall sample than that of the overall sample.

3 According to Statistics Canada’s bi-annual Residential Telecommunications Service Survey (RTSS) 2010 traditional landlines provide coverage for approximately 88-90 per cent of the households in Canada.

4 Please note that the MOE is greater for sub-groups of the overall sample than that of the overall sample.

5 Statistics Canada’s bi-annual Residential Telephone Service Survey (RTSS) 2010 estimated that around 78 per cent of households in Canada have at least one cell/smart/mobile phone service subscription. Some may have one subscription but multiple cell listings. This number also includes pay as you go account holders. The most recent RTSS estimates that almost thirteen (12.8) percent of Canadian households have only cellular telephone service. In 2010, 50% of households in the 18-to-34 age bracket were using only cell phones, up from 34% two years earlier..