Human Resources and Skills Development Canada Final Methodology Report Better Jobs/Jobs of Tomorrow Advertising Campaign Evaluation Survey:

Public Opinion Research Number: POR 092-10
Contract Award Date: 2011-01-09
Contract Call up Number: G9178-100001
Contact Amount: $74,351.29

March 23, 2011

Prepared for:

Communications Advisor, Public Opinion Research (POR)
Human Resources and Skills Development
140, promenade du Portage, Gatineau (Québec) K1A 0J9
nc-por-rop-gd@hrsdc-rhdcc.gc.ca

« Ce rapport est également disponible en française »

Table of Contents

1. Background

The Government of Canada (GoC) is looking to the future to ensure the ongoing and future prosperity of Canadians. More and more, this depends on a highly educated and skilled workforce able to acquire and upgrade its skills throughout their working life. As part of the initiative, the GoC launched a public awareness campaign. “The Better Jobs/Jobs of Tomorrow Advertising Campaign” was designed to raise awareness among Canadians’ working population of the need for higher education and skills upgrading throughout their working lives.

1.1 Jobs of Tomorrow Advertising Campaign

As per the statement of work, the Campaign was marked by two key overarching targets.

  1. It sought to generate awareness among the general public of the need to obtain the skills, post-secondary education (PSE) and experience for the jobs of today and the future.
  2. It sought to inform Canadians that the GoC has information, programs and services relating to skills acquisition, PSE and employment.

The Campaign ran on-line and on television from Jan 10th- Feb 17th, 2011.

More specifically, the main/overarching campaign included a broad awareness message targeting the Canadian working population 16 years of age or older about the importance of acquiring or upgrading their skills, pursuing PSE and gaining experience for the jobs of today and the future. The overarching campaign was designed to reach different segments of the Canadian working-age population from beginning to end of their work life cycle - from young adults to older workers, focusing on adults facing mid-career decisions.

Relative to youth, the Campaign sought to encourage young Canadians to stay in school, pursue a PSE, enroll in the skilled trade apprenticeships and to gain/upgrade the skills necessary to participate and succeed in the job market.

The campaign’s call to action is to drive the Canadian working population (including employers, youth and youth at risk) to the actionplan.gc.ca/betterjobs website or call 1 800 O-Canada for information on acquiring and upgrading skills, post-secondary education and work experience for the jobs of today and the future.

2. Research Objectives

In accordance with the GoC’s Communications Policy, departments and agencies of the GoC are required to conduct a post-campaign evaluation of all major advertising initiatives exceeding $1,000,000 in media buys.  This post-campaign testing was conducted using the Advertising Campaign Evaluation Tool (ACET), a series of standardized questions to be included at the beginning of a post-campaign survey.  The ACET telephone questionnaire including some additional campaign related questions took about 10 minutes on average to complete. 

For this evaluation, two samples of the Canadian population were surveyed.

  1. 1003 Canadians 18 plus (general population).
  2. 302 Young Canadians 16-20 years of age (youth population).

3. Methodology and Approach

As per above, the proposed public opinion research study involved the conduct of 1305 computer assisted telephone interviews with Canadians. Methodological details for the general population and youth population are contained in the sections that follow.

3.1 General Population Survey (Sample Size - 1003)

NRG, on behalf of HRSDC, conducted 1003 computer assisted telephone interviews with a random sample of Canadian adults 18 years of age or older. Households were selected at random and then one person was chosen or selected to participate in the survey using the last birthday respondent selection method1.  The margin of error (MOE) for the overall sample is +/-3.1 percentage points2, 19 times out of 20 or at a 95 percent confidence interval (CI).

As per GoC telephone interviewing standards, the survey was pretested on Feb 17th, 2011 - 153 pretest interviews were conducted in English and French. Revisions were made to the survey questionnaire and all interviews were conducted from Feb 18-27, 2011.

Table one (1) describes the final un-weighted sample distribution by region/province.
Strata Sample Size(Un-weighted) Margin of Error
Atlantic Canada 100 +/-9.80
Quebec 212 +/-6.76
Ontario 300 +/-5.66
Manitoba/ Saskatchewan/Nunavut 130 -+/-8.76
Alberta/Northwest Territory 131 -+/-8.76
British Columbia/Yukon 130 +/-8.76
Canada 1003 +/-3.10

For this component of study a disproportional sampling scheme was employed to provide for more accurate regional/provincial level results. Please note, that within each of the above geographically defined strata NRG sampled proportionately to achieve a more or less representative sample across each of the six macro regions. For example, interviews in the Atlantic Provinces are proportionately spread across all four Atlantic Provinces to a total sample of 100. The same was done in the other five regions to ensure that the final sample reflected the range of opinion from across each region.  Also, the three (3) northern territories of Yukon, Northwest Territory and Nunavut were sampled with their approximate adjacent provincial jurisdiction.  A total of 14 interviews were conducted in Canada’s North. The sample was sourced from all traditional landline4 telephone listings in Canada. 

3.1.1 Sample Source – General Population Survey

NRG Research Group uses a product from ASDE Survey Sampler called Canada Survey Sampler for sampling the general population in Canada. For additional information on ASDE and this software product please visit www.surveysampler.com. This sampling software is updated every 3 months by ASDE and it allows NRG to develop random samples at various geography levels (e.g., Province, Economic Region, or Census Division) among landline households in Canada. These samples incorporate both listed and unlisted telephone numbers to ensure adequate coverage of all landline phone numbers in Canada.

For this survey, NRG randomly selected un-listed phone numbers with at least a 30 per cent chance of being live (i.e., ringing in a residential household) for inclusion in a sample. In so doing, NRG incorporated new and changed phone listings as well as new neighborhoods in our sample. The unlisted numbers were, as is our normal practice, restricted to 20 per cent of the overall sample for this study. In total NRG research dialed 32 592 telephone numbers for this sample with just over 6000 of these sourced from directory not listed households. All samples for a study were maintained in a custom database application. As phone numbers were added to a database they were compared against a voluntary market research national do not call list and all matches were removed from the sample prior to randomization/sample selection. The sample frame for the youth population survey was derived from a combination of Canada’s traditional landline and cellular telephone sample universes. All sample units used in the general population survey were excluded from the youth sample frame.

3.2 Youth Survey (Sample Size - 302)

NRG, on behalf of HRSDC, conducted 302 12 minute computer assisted telephone interviews (CATI) with a randomly selected sample of Canadians between 16 and 20 years of age. On Feb 17, 2011, 10 pretest interviews were conducted in English and French.  Revisions were made to the survey questionnaire, and all interviews were conducted from Feb 18 to March 7, 2011. The margin of error5 (MOE) for the overall survey sample is +/-5.64 percentage points, 19 times out of 20 or at a 95 percent confidence interval (CI). 

Table two (2) below, describes the final regional/provincial sample distribution.
Strata Sample Size(Un-Weighted) Margin of Error
Atlantic Canada 25 +/-19.6
Quebec 60 +/-12.7
Ontario 101 +/-9.8
Manitoba/ Saskatchewan/Nunavut 26 +/-19.6
Alberta/Northwest Territory 42 +/-15.2
British Columbia/Yukon 48 +/-14.2
Canada 302 +/-5.64

3.2.1 Sample Source –Youth Survey

As proposed, the sample for the youth population survey was derived from a combination of Canada’s traditional landline and cellular telephone sample universes. 

NRG selected landline sample based on the population density of the target population with approximately 65 percent of the total youth sample dialed sourced from Dissemination Area (s) with an incidence greater that 12 per cent for our target group – youth 16-20 years of age. The remainder (35%) was sourced from a national random sample of active cell-phone listing. Table three (3) shows the sample distribution by telephone service type – landline versus cellular/smart/mobile telephone listing6. A total of 78 cell phone interviews were conducted via cell-phone with youth 16-20 years of age. A majority of those interviewed via cell phone (65%) reside in households that also have landline telephone service while 35 percent reside in cell phone only service households. Table three (3) summarizes the sample breakdown by sample type.

Table three (3) summarizes the sample breakdown by sample type.
Sample Source Sample Frame Description Un-weighted Sample Weighted Sample
Landline- Census Overlay Sample 30,060 Units (65%) 224 (74%) 220 (73%)
RDD Cell-Phone Sample 17,508 Units (35%) 78 (26%) 82 (27%)

3.3 Data Weighting and Tabulation

The survey data have been weighted according to the most recent Statistics Canada census information to ensure that the national level data reflects the views and perceptions of the respective target populations by age, gender within region/province.  As is the case with the general population sample, this weighting scheme is recommended for analysis of the national level results and not for within region level analysis.  Regional level analysis should employ an alternate weighting scheme that takes into account the population characteristics of the region as opposed to the entire country. Caution: given the relatively small sample size of the youth survey, the ability to conduct within region analysis is extremely limited. 

3.4 Quality Control

As per GoC standards and guidelines a minimum of 10 per cent of all completed interviews were monitored or otherwise verified – both audio and visual verification.  A total of 32 592 landline telephone numbers were included in the general population survey sample.  A total of 47 568 landline (30 060) and cell phone (17 508) listing were included in the youth survey sample frame. All English language interviewing was completed at NRG’s main data collection facility located in Winnipeg, Manitoba.  All French language interviewing was conducted in Ottawa, Ontario by Holinshed Research Group.  The response rates for the both samples as well as their respective response rate calculations can be seen below.

Table 4
Universe Sample Dialled Unresolved (U)
(Answering Machine, Busy Signal, No Answer)
In-Scope Responsive (R)
(Disqualified Due To Age, Completed Interviews, Quota Full, Other Disqualified)
In-Scope Non Responsive (IS) (Respondent Refusals, Household Refusals, Unresolved Call-backs) Invalid (IV)
(Business Listing, Not in Service, Fax modem, other non residential listing)
General Population 18+ 32,572 16,330 1,597 7,164 7,501
100% 50.14% 4.90% 21.99% 23.03%
Youth 16-20 47,586 17,583 17,850 5,353 6,782
100% 36.95% 37.51% 11.25% 14.25%
Table 5
Universe Response Rate*
General Population 18+ 1597/16330+7164+1597=6.36%
Youth 16-20 17850/5353+17853+17850=43.77%

*Responding(R)/Unresolved (u) + In-Scope Non-Responding (IS) + Responding (R)

The youth survey’s response rate (43.64%), on the surface, is significantly higher than the rate achieved in the general population survey (6.36%). However the youth sample response rate is being artificially inflated by a higher than usual number of household that were screened out of the survey due to age (17088). As illustrated in table 6 below, removing the “screened out” households from both sides of the empirical calculation formula has a lowering effect on the youth survey’s response rate. The reduced response rate of 3.22 per cent is on the low side of the preferred range for a survey of this nature. The general population response rate is also on the “low” side of the preferred response rate for a survey of that nature.  Therefore, as per GoC Telephone Interviewing Guidelines for surveys with “low response rates”, we have provided an analysis of non-response, for both samples, in the section that follows.

Table 6
Youth Sample Response Rate Calculations
Unadjusted Response Rate =17850/5353+17853+17850=43.64%
Adjusted Response Rate =762/5353+17853+762=3.22%

In essence then, we have two measures of response coming out of the youth sample.

  1. Response rate including screened out units due to age
  2. Response rate excluding screened out units due to age. 

The adjusted response rate excluding the screened out units due to age is a far better indicator of the youth sample’s true response rate.  A full summary of both surveys’ final calling results and response rates is located in Appendix A of this document.

3.5 Non-Response Bias - General Population and Youth Sample

According to the GoC’s standards and guidelines for the conduct of telephone surveys, non response bias is the systematic difference between true population values and the average result from all possible samples owing to non-response. The analysis of non-response bias will consist of a comparison of three survey variables - region, gender, age - in both samples with the equivalent parameters of the population. Table seven (7) below compares the general population sample on a regional basis to the population parameters by gender and age.

Table 7
STRATA Gender Age
M
POP
M
SAM
F
POP
F
SAM
18-34
POP
18-34
SAM
35-54
POP
35-54
SAM
55 plus
POP
55 plus
SAM
BC/ YK 48.4% 50.0% 51.6% 50.0% 26.7% 12.4% 39.5% 27.9% 33.8% 59.7%
AB/NWT 49.7% 49.6% 50.3% 50.4% 32.7% 13.7% 40.5% 38.9% 26.8% 47.3%
MB/SK/NV 48.4% 50.8% 51.6% 49.2% 29.1% 14.9% 38.3% 32.3% 32.6% 52.8%
ON 48.0% 50.0% 52.0% 50.0% 28.2% 16.9% 40.0% 42.9% 31.8% 40.2%
QB 48.3% 48.6% 51.7% 51.4% 26.9% 16.4% 39.2% 44.9% 33.9% 38.6%
AC 47.8% 50.0% 52.2% 50.0% 25.6% 8.1% 39.4% 40.4% 35.0% 51.5%

Table seven (7) shows Men and Women were sampled more or less to their proportions in each region. Across the three major age categories some sampling bias is present as Canadians in the oldest age category (55+) are over-represented while those in the youngest age grouping (18-34) are under-represented in this survey’s final sample. 

The amount of bias this has introduced to the survey can be measured to some degree by examining the percent change in the survey’s weighted and un-weighted findings. Our analysis reveals that the survey’s final un-weighted sample reflects the Canadian population by gender with some bias towards older Canadians.  This age bias has almost no impact on Canadians’ views of the performance of the Government of Canada and its performance providing information to Canadians.

Table eight (8) and nine (9) below compares the youth survey sample on a regional basis to the population by gender and age.

Table 8: Gender by Region
Strata Male 16-20
POP
Male 16-20
SAM
Female  16-20
POP
Female 16-20
SAM
BC/ YK 51.20% 54.17% 48.80% 45.83%
AB/NWT 51.12% 52.38% 48.88% 47.12%
MB/SK 51.01% 46.15% 48.99% 53.85%
ON 51.16% 48.51% 48.84% 51.49%
QC 50.93% 45.00% 49.07% 55.00%
AC 50.64% 56.00% 49.36% 44.00%
Table 9: Age by Region
Strata 16-18
POP
16-18
SAM
19-20
POP
19-20
SAM
BC/YK 5.04% 11.49% 7.88% 5.07%
AB/NWT 4.73% 10.14% 6.88% 4.06%
MB/SK 2.83% 6.08% 4.60% 2.70%
ON 15.14% 23.99% 23.77% 8.11%
QC 8.63% 13.85% 13.48% 6.08%
AC 2.68% 6.08% 4.34% 2.36%

Tables eight (8) and nine (9) show young Men and young Women were sampled more or less to their proportions in each region. At the regional level, across the four (4) major age categories some sampling bias is present as Canadians aged 19-20 are slightly under-represented while those in the youngest age grouping (16-18) are slightly overrepresented in this survey’s final sample. Weighting the sample provides some correction to this minor imbalance and the amount of bias this has introduced to the survey findings can be measured to some degree by examining the percentage change in the survey’s weighted and un-weighted findings. Our analysis reveals that the survey’s final un-weighted sample reflects the Canadian population by gender with very little to no difference in the surveys weighted and un-weighted survey results. For example, this age bias has almost no impact on Canadians’ views of the performance of the Government of Canada generally and in terms of the GoC’s ability to communicate with Canadians.

Appendix A: Call Dispositions

Table 7: Final Calling Dispositions General Population Survey
Final Call Disposition Totals % Code
No Answer 10925 33.52% U
Line In Use/Busy Signal 739 2.27% U
Not in Service 6647 20.39% IV*
Quota Full/Over Quota 241 0.74% R
Not Qualified (No Eligible Respondent in Household) 0 0.00% R
Refused to Respond 4520 13.87% IS
Unresolved Callback 2644 8.11% IS
Business Listing 189 0.58% IV
Fax/Modem 665 2.04% IV
Language DQ 353 1.08% R
Answering Machine 4666 14.32% U
Completed Interview 1003 3.08% R
Total Numbers Attempted 32592 100.00% TA
Unresolved (U) (No Answer (10925) + Answering Machine (4666) + Busy Signal (739)) 16330
In-Scope Non-Responding (IS) (Refused to Respond (4520) + Unresolved Call-back (2644) 7164
In-scope Responding (R) (Completions (1003) + Lang DQ (353) +Age DQ (0) +Quota Full (241) 1597
Response Rate (MRIA Empirical Method =R/(U+IS+R)) =1597/(16330+7164+1597)   6.36%

*Invalid

Analysis of non response has been provided for this sample.

Table 8: Final Calling Dispositions Youth Population Survey
Final Call Disposition Cell Landline Total % Code
No Answer 2392 3559 5951 12.45% U
Line In Use/Busy Signal 131 173 304 0.64% U
Not in Service 3830 1968 5798 12.13% IV*
Quota Full/Over Quota 0 41 41 0.09% R
Disqualified (DQ) (Due to age/not 16-20) 3871 13217 17088 35.76% R
Refused to Respond 1583 2614 4197 8.78% IS
Unresolved Callback 238 918 1156 2.42% IS
Business Listing 277 259 536 1.12% IV
Fax/Modem 10 438 448 0.94% IV
Language DG 148 271 419 0.88% R
Answering Machine 4950 6378 11328 23.70% U
Completed Interview 78 224 302 0.63% R
Total Numbers Attempted 17508 30060 47568 100.00% TA
Unresolved (U) (No Answer (5951) + Answering Machine (11328) + Busy Signal (304)) 17583
In-Scope Non Responding (IS) (Refused to respond (4197) + Unresolved Call-back (1156) 5353
In-scope Responding (R) (Completions (302) + DQ Due To Age (17088) + Lang DQ (419) 17850
Response Rate ( MRIA Empirical Method) R/(U+IS+R) (17850/(17583+5353+17850))  43.77%
*Adjusted for Screening AR/(U+IS+AR) 762/(17583+5353+762) 3.22%

*Invalid

Incidence of contact for the youth sample frame is provided below.
Disposition Cell Sample Landline Sample Total
Complete 78 224 302
Not Qualified 3871 13217 17088
Quota full 1 41 41
Terminates 9 7 16
Incidence7 .022 or 2.2% .017 or 1.7% .018 or 1.8%

Political Neutrality Certification

I hereby certify as a Senior Officer of Company that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not contain any reference to electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leader.

Signature:
Russell G. Noseworthy
Senior Research Associate
NRG Research Group
Ottawa
Mobile Phone 613.292.0637

Appendix B: English/French Questionnaire

BETTER JOBS (Jobs of Tomorrow)

ADVERTISING CAMPAIGN EVALUATION TOOL


INTRODUCTION: General Population Survey

INTRO:

Hello, my name is _______________. I’m calling on behalf of the government of Canada and I’d like to ask you some questions about current issues of interest to Canadians. Your responses will be kept entirely confidential and this survey is registered with the national survey registration system.

May I please speak to a member of the household who is 18 years of age or older and who has had the most recent birthday? Would that be you? [IF NOT, ASK TO SPEAK TO THAT PERSON AND START AGAIN]

Your participation in this survey is voluntary, but would be extremely helpful. Would you be willing to take part in this survey? We can do it now or at a time more convenient for you.

Introduction: Youth Survey

Land-line Introduction: YOUTH OVERSAMPLE

Hello, my name is _______________. I'm calling on behalf of NRG Research Group. Is there anyone in your household between the ages of 16 and 20 years old? 

IF YES: Can I please speak with that person? [IF THAT PERSON IS NOT AVAILABLE ARRANGE CALLBACK]

WHEN CONTACT target respondent

Hello, my name is _______________. I'm calling on behalf of NRG Research Group. We are currently conducting a study with young Canadians between the ages of 16 and 20 years old. Would you be willing to take part in this study?

We can do it now or at a time more convenient for you. Please note that your responses will be kept entirely confidential and this survey is registered with the National Survey Registration System. Your participation in this survey is voluntary, but would be extremely helpful.

Would you be willing to take part in this survey? May I proceed?

Cell Phone Introduction: YOUTH OVERSAMPLE

Hello, my name is _______________. I'm calling on behalf of NRG Research Group. We realize that we have contacted you on your cell-phone. We are conducting surveys with Canadians 16-20 years old.

SC1. Do you fall into that age group?

SC1A. Do you share this cell-phone with someone 16-20 years of age?

SC1B. Could I speak with person 16-20 years of age please? [IF THAT PERSON IS NOT AVAILABLE ARRANGE CALLBACK]

WHEN CONTACT target respondent

Hello, my name is _______________. I'm calling on behalf of NRG Research Group. We are currently conducting a study with young Canadians between the ages of 16 and 20 years old. Would you be willing to take part in this study?

We can do it now or at a time more convenient for you. Please note that your responses will be kept entirely confidential and this survey is registered with the National Survey Registration System. Your participation in this survey is voluntary, but would be extremely helpful.

Would you be willing to take part in this survey? May I proceed?

Please note, CON1 to CON2a are only asked of cell phone respondents.

CON1

To begin, we would like to confirm that you are not currently operating a motorized vehicle or machinery of any kind such as a car, truck, van, all terrain vehicle, snowmobile etc..?

SCH1

We would like to conduct this interview with you at a time that is more convenient - when you are not driving or operating machinery.  During weekdays we can call anytime from 4:00 to 9:00 pm and on weekends from noon to 8:00 PM your time.

CON2

In addition to your cellular telephone, does your household currently have a tradition telephone or landline?

CON2A

Thinking about the total number of personal telephone calls you receive as 100 per cent. What percentage out of 100 do you receive via your cell phone and what percentage do you receive using a traditional non wireless telephone or landline?

CON2A1

Cell phone per cent

CON2A2

Landline per cent

CORE ACET QUESTIONS

Programming Instructions: Questions from EAP1 to D6 administered to both the youth and general population sample frames, d7 only asked of cell phone respondents)

EAP1. Have you heard of Canada’s Economic Action Plan? 

EAP2. Why do you think Canada introduced the Economic Action Plan? 

[Do Not Read, ACCEPT UP TO 5 MENTIONS]

EAP3. Can you name any initiatives included in Canada’s Economic Action Plan? 

[Do Not Read, ACCEPT UP TO 5 MENTIONS]

Q1A:
Over the past few weeks or so, have you seen, heard or read any advertising about programs and services available to help Canadians obtain the right skills or upgrade their education and training?

Q1B: = > Q1A = YES

What can you remember about this advertising? What words, sounds or images come to mind? [DO NOT READ LIST. ACCEPT MULTIPLE MENTIONS]

PROGRAMMING NOTE:” Q2: RANDOMIZE 2A, 2B, 2C, 2D

2a. Do you remember seeing a television ad over the past few weeks or so where a young woman morphs into an engineer, another young woman morphs into a construction worker and a young man morphs into a health care professional while a female voice talks about getting the right skills and training when thinking about your career?

2b. Do you remember seeing a television ad over the past few weeks or so where a middle-aged man walks into a classroom, a female pharmacist assists a customer and an older man is being shown how to use a computer. A female voice talks about upgrading your education and training for a better job?

2c. Do you remember seeing an Internet ad over the past few weeks or so showing a young female engineer, a young female construction worker and a young male health care professional? The ad talks about getting the rights skills and training for your career?

2d. Do you remember seeing an Internet ad over the past few weeks or so showing a middle-aged man in a classroom, a female pharmacist and an older man in front of a computer? The ad talks about upgrading your education and training for a better job?

IF RESPONDENT RECALLS ANY OF THE ADS IN Q2 SERIES OF QUESTIONS, CONTINUE; OTHERWISE SKIP TO Q6.

Q3.Thinking about the advertising that you saw, what do you think was the main point this advertising was trying to get across? [OPEN-ENDED, ACCEPT ONE RESPONSE. DO NOT READ]

Q4A:

Did you do anything as a result of seeing this advertising?

Q4B:

What did you do? Anything else? [DO NOT READ LIST. CODE ALL THAT APPLY]

Q4C: OPTIONAL QUESTION ON ATTITUDE / BEHAVIOUR CHANGE

FOR EXAMPLE:

[IF RESPONDENT SAID ‘YES’ AT Q4A, USE THIS INTRO]

Which, if any, of the following things did you do as a result of seeing this advertising?

[IF RESPONDENT SAID ‘NO’, ‘DON’T KNOW’ OR ‘REFUSED’ AT Q4A, USE THIS INTRO]

While you said you don’t recall doing anything as a result of the advertising, I’d like to briefly ask you about some specific things that you may have done. Did you…?

RANDOMIZE a-d AND, IF APPLICABLE, EXCLUDE ITEMS MENTIONED BY RESPONDENT AT Q4B

Q5A: Thinking about the advertising you saw, who do you think produced it? That is, who paid for it? [DO NOT READ, ACCEPT ONE RESPONSE]

Q5B: Which level of government? [READ LIST, ACCEPT ONE RESPONSE]

Q6: ASK ALL RESPONDENTS: The next three questions have to do with the performance of the government in general.

How would you rate the performance of the Government of Canada in providing information about programs and services available to Canadians who wish to enhance their education and skills?  Please use a 7-point scale, where 1 means terrible, 7 means excellent and the mid-point, 4, means neither good nor bad.

Q7: And, using the same scale [REPEAT IF NECESSARY], how would you rate the performance of the Government of Canada in providing information to the public in general?

Q8: Generally speaking, how would you rate the overall performance of the Government of Canada? Once again, please use the same scale [REPEAT IF NECESSARY].

C2.  [ASK ONLY IF SAID “Got information about training and/or education” in Q4C OTHERWISE GO TO d1] 

Earlier, you mentioned that after seeing the advertising you got information about training and/or education. Where did you get this information? [OPEN-ENDED- DO NOT READ. ACCEPT ALL]

CORE DEMOGRAPHIC QUESTIONS

Finally, I’d like to ask you some questions for statistical purposes. I'd like to remind you that all your answers are completely confidential.

D1: What is your current employment status?

Are you...? [READ LIST IF HELPFUL, ACCEPT ONE ANSWER ONLY]

D2: What is the highest level of formal education that you have completed to date?

[READ LIST IF HELPFUL, ACCEPT ONE ANSWER ONLY]

D3: In what year were you born?

D4: Were you born in Canada?

D5: Are there any children under the age of 18 currently living in your household?

D6: Which of the following income ranges best describes the combined annual income of all persons in your household, before taxes?

[READ LIST]

Ask if Cell-Phone respondent D7:  What is your postal code?

Enter Postal Code: NAN-ANA

Refused: 999-999

That Concludes the Survey. This survey was conducted on behalf of Human Resources and Skills Development Canada. In the coming months the report will be available from Library and Archives Canada. Thank you very much for taking part in this survey. It is appreciated.

Interviewed to NOTE/RECORD:

ACET Questionnaire en français

OUTIL D’ÉVALUATION DE CAMPAGNES PUBLICITAIRES


INTRODUCTION : Population General

Bonjour, je m’appelle _______________. Je vous appelle au nom de NRG Research Group. J’aimerais vous poser certaines questions au sujet d’enjeux actuels d’importance pour les Canadiens et Canadiennes. Je tiens à signaler que vos réponses demeureront absolument confidentielles et que ce sondage est inscrit auprès du système national d’enregistrement des sondages.

Pourrais-je parler à un membre de votre foyer qui a 18 ans ou plus et dont l'anniversaire est le plus récent? Seriez-vous cette personne? [SI NON, DEMANDER DE PARLER AVEC CETTE PERSONNE ET RECOMMENCER]

Vous êtes libre de participer au présent sondage mais cela serait très utile. Seriez-vous d’accord pour prendre part à ce sondage? Nous pouvons le faire dès maintenant ou à un moment plus propice pour vous.

QUES_ Introduction pour les lignes terrestres: INTRO SURÉCHANTILLON JEUNESSE

Bonjour, je m’appelle _______________. Je vous appelle au nom de NRG Research Group. Y a-t-il une personne dans votre foyer âgée de 16 à 20 ans?

SI OUI: Est-ce que je pourrais parler à cette personne? [SI CETTE PERSONNE N'EST PAS DISPONIBLE, ARRANGER UN RAPPEL]

LORSQUE VOUS PARLEZ AU BON RÉPONDAN

Bonjour, je m’appelle _______________. Je vous appelle au nom de NRG Research Group. Nous menons présentement un sondage auprès de jeunes Canadiens et Canadiennes âgés de 16 à 20 ans. Acceptez-vous de participer à ce sondage?

Nous pourrions commencer maintenant, à moins que vous ne préfériez un autre moment. Je tiens à signaler que vos réponses demeureront absolument confidentielles et que ce sondage est inscrit auprès du système national d’enregistrement des sondages. Votre participation à ce sondage est volontaire, mais elle serait très utile.

Acceptez-vous de participer à ce sondage? Est-ce que je peux commencer?

Introduction pour les téléphones cellulaires: INTRO suréchantillon jeunesse téléphone cellulaire

Bonjour, je m’appelle _______________. Je vous appelle au nom de NRG Research Group. Nous réalisons que nous vous appelons sur votre  téléphone cellulaire. Nous menons des sondages auprès de Canadiens et Canadiennes qui ont entre 16 et 20 ans.

SC1.  Faites-vous partie de ce groupe d'âge?

SC1A. Partagez-vous un téléphone cellulaire avec une personne âgée de 16 à 20 ans?

SC1B. Est-ce que je pourrais parler à la personne âgée de 16 à 20 ans?  [SI CETTE PERSONNE N'EST PAS DISPONIBLE, ARRANGER UN RAPPEL]

LORSQUE VOUS PARLEZ AU BON RÉPONDANT

Bonjour, je m’appelle _______________. Je vous appelle au nom de NRG Research Group. Nous menons présentement un sondage auprès de jeunes Canadiens et Canadiennes âgés de 16 à 20 ans. Acceptez-vous de participer à ce sondage?

Nous pourrions commencer maintenant, à moins que vous ne préfériez un autre moment. Je tiens à signaler que vos réponses demeureront absolument confidentielles et que ce sondage est inscrit auprès du système national d’enregistrement des sondages. Votre participation à ce sondage est volontaire, mais elle serait très utile.

Acceptez-vous de participer à ce sondage? Est-ce que je peux commencer?

CON1

D’abord et avant tout, nous aimerions confirmer que vous n’êtes pas en train de conduire un véhicule ou de faire fonctionner de la machinerie, quels qu’ils soient. Êtes-vous présentement aux commandes d’une automobile, d’un camion, d’une fourgonnette, d’un véhicule tous-terrains, d’une motoneige, etc. ...?

SCH1

Nous aimerions effectuer l’entrevue à un moment qui s’y prêtera mieux – c.-à-d. lorsque vous ne serez pas en train de conduire ou de faire fonctionner de la machinerie. En semaine, nous sommes en mesure de vous appeler n’importe quand entre 16 h et 21 h, et la fin de semaine, entre midi et 20 h, selon le moment qui vous convient.

CON2

En outre de votre téléphone cellulaire, est-ce que votre ménage compte présentement un téléphone traditionnel, ou service conventionnel à fil?

CON2A [0,0]

J’aimerais que vous réfléchissiez au nombre total d’appels téléphoniques personnels que vous recevez, c.-à-d. 100% de vos appels personnels. Sur 100 appels, quelle proportion en pourcentage recevez-vous sur votre téléphone cellulaire, et quelle proportion en pourcentage recevez-vous au moyen d’un téléphone traditionnel qui n’est pas sans fil, c.-à-d. au moyen du service conventionnel à fil?

CON2A1

Pourcentage pour le téléphone cellulaire

CON2A2

Pourcentage pour le service conventionnel à fil

EAP1 : Avez-vous entendu parler du Plan d’action économique du Canada?

[SI OUI A LA QUESTION EAP1 CONTINUER, SI NON ALLER A LA QUESTION Q1A]

EAP2 : Selon vous, pourquoi le Canada a-t-il mis en place le Plan d’action économique?

[INSCRIRE TEXTUELLEMENT, ACCEPTER JUSQU’A 5 RÉPONSES, NE PAS LIRE LA LISTE]

EAP3 : Pouvez-vous nommer des initiatives qui font partie du Plan d’action économique du Canada?

[INSCRIRE TEXTUELLEMENT, ACCEPTER JUSQU’À 5 RÉPONSES, NE PAS LIRE LA LISTE]

QUESTIONS GÉNÉRIQUES

Q1A: Au cours des dernières semaines, est-ce que vous avez vu, entendu ou lu des annonces sur [les programmes et services disponibles pour aider les Canadiens et Canadiennes à acquérir les compétences dont ils ont besoin ou à perfectionner leur éducation et leur formation??]? [NE PAS LIRE. ÉNONCER LE SUJET EN QUELQUES MOTS, SANS TOUTEFOIS DÉCRIRE LE THÈME DE LA CAMPAGNE.]

Q1B: (Si, Q1A = OUI) De quoi vous souvenez-vous à propos de cette (ces) annonce(s)?

Quels sont les mots, les sons ou les images qui vous viennent à l’esprit?

[NE PAS LIRE LA LISTE, ACCEPTER TOUTES LES RÉPONSES PERTINENTES]

Q2: RÉPARTIR ALÉATOIREMENT LES MÉDIAS. RÉPÉTER LA Q2 POUR CHAQUE MÉDIA DE LA CAMPAGNE [C.-À-D. 2A, 2B, 2C…]. RÉVISER LA DESCRIPTION DE L’ANNONCE AU BESOIN.

2a. Vous rappelez-vous avoir vu ou entendu au cours des dernières semaines une annonce à la télévision présentant une jeune femme se transformant en ingénieure, une autre une jeune femme se transformant en ouvrière en construction et  un jeune homme se transformant en professionnel de la santé pendant que la voix d'une femme parle d'acquérir les compétences et la formation nécessaires lorsqu'on pense à sa carrière?

2b. Vous rappelez-vous avoir vu ou entendu au cours des dernières semaines une annonce à la télévision présentant [Un homme d'âge moyen entrant dans une salle de classe, une pharmacienne aidant un client et un homme âgé recevant une formation sur l'utilisation d'un ordinateur. La voix d'une femme parle de perfectionner votre éducation et votre formation pour profiter d'un meilleur emploi?

2c. Vous rappelez-vous avoir vu ou entendu au cours des dernières semaines une annonce sur internet présentant [une jeune ingénieure, une jeune ouvrière en construction et un jeune professionnel de la santé? L'annonce parle d'acquérir les compétences et la formation nécessaires pour votre carrière?

2d. Vous rappelez-vous avoir vu ou entendu au cours des dernières semaines une annonce sur internet présentant [Un homme d'âge moyen dans une salle de classe, une pharmacienne, et un homme âgé devant un ordinateur? L'annonce parle de perfectionner votre éducation et votre formation pour profiter d'un meilleur emploi?

SI LE/LA RÉPONDANT(E) SE SOUVIENT DE L’UNE OU L’AUTRE DES ANNONCES SIGNALÉES DANS LA SÉRIE DE QUESTIONS Q2, CONTINUER; AUTREMENT, PASSER À LA Q6.

Q3: En ce qui concerne l’annonce que vous avez vue/entendue, à votre avis, quel était le message principal que l’annonce voulait communiquer?  [QUESTION OUVERTE. ACCEPTER UNE SEULE RÉPONSE]

(QUESTION OUVERTE. ACCEPTER UNE SEULE RÉPONSE)  (SI LA RÉPONSE EST "ALLER AU SITE WEB", DEMANDER AU RÉPONDANTS DE PRÉCISER L'ADRESSE DU SITE WEB)

Q4A: Avez-vous fait quelque chose en réponse à cette (ces) annonce(s)?

Q4B: = >Q4A = OUI Qu’avez-vous fait? Y a-t-il autre chose?

[NE PAS LIRE LA LISTE, CODER TOUTES LES RÉPONSES PERTINENTES]

4B. Qu'avez-vous fait?   (PROBE) Y a-t-il autre chose?

(NE PAS LIRE LA LISTE, CODER TOUTES LES RÉPONSES PERTINENTES)

Q4C: QUESTION OPTIONELLE CONCERNANT CHANGEMENT D’ATTITUDE /DE COMPORTEMENT

PAR EXEMPLE :

[SI LE/LA RÉPONDANT(E) A DIT 'OUI' À Q4A, LIRE CETTE INTRODUCTION]

Lesquelles de ces actions avez-vous prises en réponse à cette annonce?

[SI LE/LA RÉPONDANT(E) A DIT ‘NON’, NSP/REF À LA QUESTION 4A, LIRE CETTE INTRODUCTION : Bien que vous ayez dit que vous ne vous rappeliez pas avoir fait quoi que ce soit en réponse à cette annonce, j’aimerais vous demander brièvement certaines choses spécifiques que vous auriez pu faire. Avez-vous …

[LIRE LA LISTE, ACCEPTER TOUTES LES RÉPONSES PERTINENTES]

RÉPARTIR ALÉATOIREMENT a-d ET, S'IL Y A LIEU, EXCLURE LES ÉLÉMENTS MENTIONNÉS À Q4B

*LISTE DEVRAIT EXCLURE LES ÉLÉMENTS IDENTIFIÉS PAR LE RÉPONDANT A Q4B.

Q5A: En ce qui concerne la publicité que vous avez vue/entendue, à votre avis, qui en était l’auteur? Autrement dit, qui l’a financée? [NE PAS LIRE, ACCEPTER UNE SEULE RÉPONSE]

Q5B: Quel ordre de gouvernement? [LIRE LA LISTE. ACCEPTER UNE SEULE RÉPONSE.]

Q6: DEMANDER À TOUS LES RÉPONDANTS: Les trois prochaines questions concernent le rendement du gouvernement du Canada en général.

POUR LES TROIS PROCHAINES QUESTIONS, INSCRIRE SÉPARÉMENT NE SAIS PAS ET REFUSÉ.

Quelle serait votre évaluation du rendement du gouvernement du Canada pour ce qui a trait à [fournir des renseignements sur les programmes et services disponibles aux Canadiens et Canadiennes qui désirent perfectionner leur connaissances et leurs compétences? ? Veuillez répondre selon une échelle de sept points, où « 1 » signifie affreux, « 7 », excellent et le point milieu, « 4 », ni bon, ni mauvais.

Q7: Et selon la même échelle [RÉPÉTER AU BESOIN], quelle serait votre évaluation du rendement du gouvernement du Canada pour ce qui est de communiquer de l’information à la population?

Q7: Et selon la même échelle [RÉPÉTER AU BESOIN], quelle serait votre évaluation du rendement du gouvernement du Canada pour ce qui est de communiquer de l’information à la population?

Q8: De façon générale, quelle serait votre évaluation du rendement global du gouvernement du Canada? Encore une fois, veuillez utiliser la même échelle [RÉPÉTER SI NÉCESSAIRE].

C2. [S'IL/ELLE A RÉPONDU “Obtenu des renseignements sur la formation et/ou l'éducation" à Q4C, else d1]

Vous avez mentionné plus tôt qu'après avoir vu l'annonce, vous avez obtenu des renseignements sur la formation et/ou l'éducation. Où avez-vous obtenu ces renseignements?

[QUESTION OUVERTE - NE PAS LIRE. ACCEPTER TOUTES LES RÉPONSES]

  1. Internet – Quels sites en particulier? Écrire le nom des sites Web _____________
  2. plandaction.gc.ca/meilleursemplois.gc.ca
  3. Le gouvernement du Canada (en général)
  4. Organisme/ ministère particulier du gouvernement du Canada – préciser l’organisme________________
  5. Le gouvernement provincial/territorial (en général)
  6. Orientation scolaire/conseiller d’orientation professionnelle
  7. Employeur
  8. Agences de recrutement/spécialistes des ressources humaines
  9. Parents/membres de la famille
  10. Amis
  11. Autre (préciser)
  12. NE SAIS PAS/NE SE SOUVIENT PAS/REFUS

QUESTIONS DÉMOGRAPHIQUES DE BASE

En terminant, j’aimerais vous poser certaines questions pour les besoins de la compilation de statistiques. Je tiens à vous rappeler que toutes vos réponses demeureront entièrement confidentielles.

D1: À l’heure actuelle, quelle est votre situation dans le contexte de l’emploi? Êtes-vous...? [LIRE LA LISTE SI UTILE, ACCEPTER UNE SEULE RÉPONSE]

D2: Quel est le plus haut niveau de scolarité que vous avez atteint?

D3: En quelle année êtes-vous né(e)?

_D4: Êtes-vous né(e) au Canada?

D5: Est ce qu’il y a des enfants de moins de 18 ans qui vivent dans votre foyer ?

D6: Laquelle des catégories suivantes décrit le mieux la somme des revenus avant impôts de tous les membres de votre foyer? [LIRE LA LISTE]

REFUS [DÉCLARATION SPONTANÉE]

S'il s'agit d'un répondant sur téléphone cellulaire à D7, demandé: Quel est votre code postal?

FIN DES QUESTIONS DÉMOGRAPHIQUES DE BASE

INSÉRER LES QUESTIONS DÉMOGRAPHIQUES OPTIONELLES ADDITIONELLES TEL QUE REQUIS._

Voilà qui met fin au sondage que nous avons effectué pour le compte de [AJOUTER LE NOM DU MINISTÈRE/DE L’ORGANISME]. Dans les mois à venir, le rapport sera disponible de Bibliothèque et archives Canada. Merci beaucoup d’y avoir participé. Nous vous en sommes très reconnaissants.

INTERVIEWER RECORDS:


1This method selects one respondent per household who celebrated their birth last or most recently to respond to the survey questionnaire.

2 Please note, the MOE is greater for sub-groups of the overall sample than that of the overall sample.

3 Pretest interview results are not included in this survey’s final sample (N) of 1003 interviews.

4 According to Statistics Canada’s bi-annual Residential Telecommunications Service Survey (RTSS) 2008 traditional landlines provide coverage for approximately 90-92 per cent of the households in Canada.

5 Please note that the MOE is greater for sub-groups of the overall sample than that of the overall sample.

6Statistics Canada’s bi-annual Residential Telephone Service Survey (RTSS) 2008 estimated that more than 74 per cent of households in Canada have at least one (1) cell/smart/mobile phone service subscription. Some may have one subscription but multiple cell listings. This number also includes pay as you go account listing holders. The most recent RTSS estimates that at least eight (8) percent of Canadian households have only cellular telephone service.  The RTSS estimates that approximately 35 per cent of households comprised only of 18-34 years old had cell only service. 

7 Incidence Calculation: Completes + Terminates/ Completes + Terminates + Not Qualified +Quota Full