PWGSC Contract # G9178-110001-006/CY
POR Registration 050-11
Contract Award Date: 04/01/2012

Post Testing of the 2012 Apprenticeship Grants Campaign

Methodology Report

Submitted to:

Human Resources and Skills Development Canada
140 Promenade du Portage, Phase IV
Gatineau, Québec
For more information on this study please contact: nc-por-rop-gd@hrsdc-rhdcc.gc.ca

EKOS RESEARCH ASSOCIATES INC.

March 14, 2012

Ce rapport est aussi disponible en français sur demande

EKOS RESEARCH ASSOCIATES

Ottawa Office
359 Kent Street, Suite 300
Ottawa, Ontario
K2P 0R6
Tel: (613) 235 7215
Fax: (613) 235 8498
E-mail: pobox@ekos.com

Toronto Office
181 Harbord Street
Toronto, Ontario
M5S 1H5
Tel: (416) 598 8002
Fax: (416) 533-4713
E-mail: toronto@ekos.com

www.ekos.com

Table of Contents

I hereby certify as Senior Officer of EKOS RESEARCH ASSOCIATES that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.

Susan Galley, Vice President, EKOS Research

1. Background

The Apprenticeship Incentive Grant (AIG) is a federal grant program that was implemented in 2007 that provides taxable cash grants of $1,000 (to a maximum of $2,000) to registered Red Seal apprentices who successfully complete their first and/or second year of training. The objectives of the AIG are to: increase access to apprenticeships in Red Seal Program trades; to encourage progression in and completion of apprenticeship training; and promote inter-provincial mobility. A new $2000 Apprenticeship Completion Grant (ACG) was introduced in 2009 to complement the AIG.

In 2009 an advertising campaign was launched to raise awareness of the AIG with apprentices and with young adults considering entering into the trades. This campaign was continued in 2010-2011 and 2011-2012. The primary target for this campaign is apprentices enrolled in Red Seal-designated trades to heighten awareness of the existence of these two supports available for those who continue in their program. The secondary audience in the first year of the campaign was young adults and was designed to increase awareness of the type of supports available to students who choose to enrol in an apprenticeship to pursue a career as a skilled tradesperson in a Red Seal trade. This secondary target group was not a focus of the campaign this year.

Public Opinion Research (POR) is a necessary next step to enable Human Resources and Skills Development Canada (HRSDC) to gauge the success of its outreach activities thus far, and to refine efforts where necessary to better reach the target audiences. Because the combined advertising and outreach campaign exceeds $1,000,000, this public opinion research will take the form of the mandatory Advertising Campaign Evaluation Tool, with additional validating and benchmarking questions following the ACET instrument. The current research is designed to assess the effectiveness of the campaign with the general public, as well as with the principle target group of students enrolled in apprenticeship programs.

Information obtained through this public opinion research will allow HRSDC to determine the impact of its advertising campaign and provide direction on areas where the campaign can be adjusted. The findings provide useful information to improve future campaigns which are better targeted, more informative and better geared to the target population and Canadians at large.

The survey entailed the expenditure of $46,360 plus HST. With HST this is $52,386.80.

2. Methodology

The survey instrument used the full set of ACET questions used by the Government of Canada in all of its advertising evaluation research, as well as a small subset of additional questions targeted specifically for apprentices related to application for and accessing of government grants including the Apprenticeship Incentive and Completion Grants (AIG and ACG). The length of the interview was 9.5 minutes for the general public and 13.5 minutes for apprentices. Pre-testing was conducted with both the general public and apprentices one week before the end of the campaign. A total of 24 cases were conducted in the general public and five cases were completed with apprentices. The cases completed with apprentices were retained in the final data file. Only minor adjustments to skip logic were implemented as a result of the pre-testing. The survey was conducted over 10 days starting the Monday after the campaign ended on February 20 (February 20-29).

Respondents to the telephone survey who were sampled from the general public were 18 years of age and older, and were randomly selected (through random digit dialling). The sample included all provinces and territories, and the survey was administered in both English and French. The sample distribution for this study was as follows:

Province/Territory

Sample Size

Margin of Error
(19 times out of 20)

Atlantic Provinces

103

± 11.2%

Quebec

227

± 6.6%

Ontario (incl. Nunavut)

393

± 4.8%

Manitoba/Saskatchewan

101

± 10.4%

Alberta (incl. NWT)

101

± 9.4%

British Columbia (incl. Yukon)

95

± 9.8%

Total

1010

± 3.1%

This sample of the general public was randomly selected from a wider sample frame of listed and unlisted landlines in Canada and is therefore considered to be a probability sample that is representative of the wider universe (exclusive of households with only cell phones). As such, a margin of error can be applied. The overall sample of 1010 cases has an associated margin of error of as wide as +/-3.1 per cent. The margin of error is wider for subgroups (e.g., 4.0 to 10.0 per cent in most cases).

During the study design, 45 post-secondary schools that offer apprenticeship programs were contacted and asked to participate in the study. A total of 271 schools agreed to participate representing all regions of the country and a variety of Red Seal trades. An online registry was set up on a secure website to enable apprentices to log on and sign up to register for the study. Apprentices were informed that they would be contacted by telephone and that the survey would take 13 minutes to complete. They were also offered an incentive of $20 to take part in the survey. Appendix A provides a copy of the letter sent (on HRSDC letterhead) to post-secondary institutions to inform them of the study. Appendix B provides a copy of the letter sent (on EKOS letterhead) that schools distributed to their students. Appendix C presents a copy of the online registration form that apprentices signed onto to register for the survey. Appendix D presents the survey instrument used for both the general public and apprentice samples.

Names of apprentices were collected over a period of five weeks, from January 24 to February 16. Following is a list of the 27 schools that participated in the study by distributing the letter to students directing them to our online registry.

Participating Schools

Registration was a slow and rather difficult process. Although it did not typically take long for schools to agree, interest among apprentices in spite of the incentive was low and registration was slow. Many calls were placed to schools to remind them to distribute the letter or to ask them to remind students about the study. Ultimately, we received a total of 320 apprentices that registered online. The following is a table of the top participation rates (i.e., schools where at least more than 10 apprentices signed up).

Algonquin College

12

3.75%

Conestoga College Institute of Technology and Advanced Learning

13

4.10%

Fanshawe College of Applied Arts and Technology

15

4.69%

George Brown College

16

5.00%

Lambton College

13

4.10%

Mohawak College

26

8.13%

New Brunswick Community College

22

6.86%

Northwest Community College

32

10.00%

Nova Scotia Community College

34

10.63%

Red River College

20

6.25%

Sheridan Institute of Technology and Advanced Learning

17

5.31

Vancouver Island University

24

7.50%

Other

76

23.75%

 

320

n=320

Prior to beginning the full fieldwork, the study objectives and sampling of the survey, as well as the meaning and intent of specific items in the interview, were thoroughly covered in training sessions with the full complement of interviewers who worked on the study. Test or practice interviews were conducted to familiarize interviewers with the questions, categories, flow and skip logic.

Survey supervisors continuously monitored interviewing during the data collection process, using a dual audio and visual monitoring system. Ten per cent of interviews from each survey were monitored to ensure consistency of questionnaire administration and interviewing techniques.

The general public sample for this study was derived from Survey Sample Software. Respondents were 18 years of age and older and were selected through random digit dialling. The person who answered the phone and was 18 years of age or older and last celebrated their birthday was asked to complete the survey. In the case of the apprentice sample all entries from the online registry were contacted looking for that individual, by name.

Eight call-backs (nine total calls) were made to each selected household or apprentice in the samples before retiring a case and substituting another household. Follow-up calls were made on subsequent days, at varying time periods to maximize the potential for reaching a given respondent and appointments were taken at the convenience of the respondent. All individuals were given the choice of conducting the interview in either official language. The survey was conducted over 10 days starting the Monday after the campaign ended on February 20th (February 20-29th). Call-backs were rotated over weekday evenings and weekends during that time.

The survey was registered with the National Survey Registration System. Survey data collection adhered to all applicable industry standards as set out by the Marketing Research and Intelligence Association (MRIA), of which EKOS is a Gold Seal member. EKOS informed respondents of their rights under the Privacy and Access to Information Acts and ensured that those rights were protected throughout the research process. This included: informing respondents of the purpose of the research; identifying both the sponsoring department and research supplier; informing respondents that their participation in the study is voluntary, and that the information provided would be administered according to the requirements of the Privacy Act.

Once the survey was collected, the database was reviewed for data quality. Coding was also completed (starting after the first 300 cases and completed a few days after the completion of the survey collection period. Survey results were weighted based on Statistics Canada data according to age, gender, and region to ensure the sample is representative of the general public aged 18 years and older.

Data tables were created to isolate results for major subgroups in the general public to be used in the analysis (age, gender, education, income, and also by those who recalled and did not recall the campaign). Among apprentices the sub-group array was not as wide given the smaller sample size. It should also be understood that the apprentice sample is a purposive and not a randomly generated sample, therefore it cannot be said to be representative of all apprentices studying in Canada. It merely provides illustration of recall rates and impact among some apprentices. No margin of error can be applied to this more targeted sample.

2.1 Response Rates2

The response rate for the telephone survey was 16.3 per cent for the general public portion of the sample, 40 per cent for the panel cell phone sample. Among the target group of apprentices, however, the response rate was 77 per cent. Details are presented below.

General Public Sample

Code

Count

AM_-_Callback_in_2_hrs

3433

AP_-_Callback_-_Specific_time/date

11

BC_-_Blocked_by_Bell

52

BU_-_Business/Fax_/Modem

190

Complete

878

DU_-_Duplicate_Number

2

FR_-_French_Household

22

IG_-_Ineligible

16

IR_-_Incomplete_Refusals

27

LN_-_Language_Barrier

148

NA_-_Callback_in_12_hrs

814

NF_-_Invalid_Number

55

QF_-_Quota_Filled

33

RF_-_Refusal

1201

UN_-_Unavailable_within_Project_Timeframe

14

X_-_Exit_without_Dialing

10

Total

6902

 

Response Rate Table

Method (MRIA formulas)

Outcome

Empirical Method

Response Rate

16.30%

Estimation Method

Eligibility Rate

91.50%

 

CPO Panel Sample

Code

Count

AM_-_Callback_in_2_hrs

175

AP_-_Callback_-_Specific_time/date

4

BU_-_Business/Fax_/Modem

2

Complete

132

IG_-_Ineligible

10

IR_-_Incomplete_Refusals

1

NA_-_Callback_in_12_hrs

17

NF_-_Invalid_Number

24

RF_-_Refusal

17

TOTALS

382

 

Response Rate Table

Method (MRIA formulas)

Outcome

Empirical Method

Response Rate

39.80%

Estimation Method

Eligibility Rate

93.80%

 

Apprentice Sample

Code

Count

AM_-_Callback_in_2_hrs

55

AP_-_Callback_-_Specific_time/date

1

Complete

230

IG_-_Ineligible

12

NA_-_Callback_in_12_hrs

10

NF_-_Invalid_Number

7

RF_-_Refusal

4

UN_-_Unavailable within Project Timeframe

1

TOTALS

320

 

Response Rate Table

Method (MRIA formulas)

Outcome

Empirical Method

Response Rate

77.10%

Estimation Method

Eligibility Rate

95.10%

The response rate for the telephone survey is calculated by dividing the cooperative call backs (i.e., those who completed the study, those who we spoke to but were ineligible, and the quota filled) by the functional sample. The functional sample is the sample remaining after numbers not in service, business/fax numbers, duplicate numbers and numbers blocked by the phone company are removed.

Appendix A: Letter to Institutions

Dear Training Institution,

On behalf of the Government of Canada, EKOS Research Associates Inc. has been hired to gather the input of apprentices in a Red Seal trade in order to evaluate the Government of Canada’s recent Apprenticeship Grants Advertising Campaign.

We are contacting your organization to assist in recruiting apprentices to participate in a short telephone survey. Participants that we would like to recruit are apprentices in a Red Seal trade who are currently training or have completed their Red Seal designation in the past two years. Apprentices would be asked to voluntarily register their interest online on a secure website.

Please be assured that the confidentiality of all participants will be respected and the names of participants will only be used for the purposes of the research and will not be revealed to the Government of Canada by the research firm. Furthermore, final results will contain only non-identifying aggregate information. The information provided by apprentices will be collected under the authority of the Department of Human Resources and Skills Development Act and will be administered in accordance with the provisions of the Privacy Act and other applicable privacy laws.

A representative from EKOS Research will be contacting you in the coming days to discuss the research project and the potential for your organization to assist us with this research. If you have any questions or concerns, please do not hesitate to contact me.

Thank you for your consideration.

Sincerely,

Guy Schryburt, Manager
Public Opinion Research
140,
promenade du Portage
Gatineau
Québec K1A 0J9
(819) 997-7353

Appendix B: Letters to Apprentices

January 2012

Dear Apprentice,

EKOS Research Associates is conducting a short telephone survey with students across Canada who are enrolled in an apprenticeship program. The purpose of the research is to evaluate recent communications aimed at apprentices.

If you are interested we would ask you to go to the link below. This leads to a secure website, for you to provide your name, telephone, and indicate a good time to reach you for the survey. The telephone interview will take only 8 to 10 minutes, and your name and phone number will not be used for any other purpose than this research. Your answers will remain completely confidential and no results from this research will ever be associated with individual names.

Each student enrolled in an apprenticeship program in Canada who completes the 8-10 minute telephone survey will be compensated $20 for their participation. The survey is expected to take place in the last two week so February.

http://www.ekos.com/apprentice

The information you provide will be administered in accordance with the Privacy Act and other applicable privacy laws. Note that the information you provide will be used solely for the purpose of the research and will be kept strictly confidential. Findings presented in the survey report will not be linked directly or indirectly to any individual respondent or organization.

If you have any questions about this upcoming survey, please contact EKOS Research at apprentice@ekos.com or 1-800-388-2873.

Thank you in advance for your participation,
EKOS Research Associates
(800) 388-2873
apprentice@ekos.com

Appendix C: Online Registry

INTRO

Thank you for visiting the Red Seal Apprenticeship Survey Registration site.

EKOS Research has been hired by Human Resources and Skills Development Canada to conduct a short telephone survey of apprentices in Red Seal trades. The survey will take 8-10 minutes to complete, and you will be paid $20 for your participation.

If you are interested in completing our survey, please fill in the information below, and if you are eligible one of our researchers will be in touch in March. Note that we will only be conducting 300 surveys across Canada with students in apprenticeship programs, so it is possible that we will not contact you.

Q1

Q2

Your college

Q2A

Q3

Q4

What is the best time of day for EKOS to phone you?

Q5

Do you prefer we call on a weekend or weekday?

THANK

Thank you very much for registering to complete this survey.

Appendix D: Questionnaire

INTRO

Hello, my name is _______________. I’m calling on behalf of EKOS Research Associates. I’d like to ask you some questions about current issues of interest to Canadians. Your responses will be kept entirely confidential and anonymous. This survey is registered with the national survey registration system.

[INDAPRNTS.NE.1] May I speak with the person in your household who is 18 years of age or older who has had the most recent birthday? Would that be you? (If that person is not available, arrange call back.)

Would you be willing to take part in this survey? We can do it now or at a time that is more convenient for you.

[INDAPRNTS.EQ.1]Your participation in this survey is voluntary, but would be extremely helpful. Would you be willing to take part in this survey? We can do it now or at a time more convenient for you.

[IF ASKED] The survey will take about 10 minutes to complete. NOTE: Reveal name of client department which commissioned survey at the end of the survey only.

SCRN1

If... INDAPRNTS.EQ.1

APPRENTICE Sample

Are you currently enrolled in an apprenticeship program – learning a trade?

SEX

Record gender of respondent

LANG

Record language of correspondence

SEXQUOTAS

PRIV

This call may be recorded for quality control or training purposes.

EAP1

Have you heard of Canada’s Economic Action Plan?

Q1A

Over the past few weeks or so, have you seen, heard or read any advertising about grants for apprentices?

Q1B [1,10]

If... Q1A.EQ.1

IF ADVERTISING SEEN

What can you remember about this advertising? What words, sounds or images come to mind?

DO NOT READ LIST - ACCEPT MULTIPLE RESPONSES

Q2A

Do you remember hearing an ad on the radio over the past few weeks that starts with sounds of hammering, welding and cooking, and talks about being able to get up to $4,000 in grants if you are an apprentice in one of the 53 Red Seal trades? The ad gives the examples of a welder, carpenter and a cook and says “that’s the kind of help every apprentice can use. Apply today".

Q2B

Do you remember seeing a poster ad in washrooms on campuses, resto-bars or at a construction site over the past few weeks that introduces the idea of $4,000 in grants for apprentices, if you are studying in one of the 53 Red Seal trades? The ad shows pictures of a carpenter, an electrician and a cook.

Q2C

Do you remember seeing any advertisements on the Internet over the past few weeks that introduces the idea of “$4,000 in grants” that is “there for you to learn a Red Seal trade”. The ad shows 3 pictures, a carpenter, an electrician and a cook. The ad talks about an Apprenticeship Grant of up to $4,000 and encourages apprentices to “Apply Today!”

Q3

If... Q2A.EQ.1.OR.Q2B.EQ.1.OR.Q2C.EQ.1

THOSE WHO RECALL ANY AD IN Q2 SERIES

Thinking about the advertising that you saw/heard, what do you think was the main point this advertising was trying to get across?

OPEN-ENDED. DO NOT READ. ACCEPT ONE RESPONSE

Q4A

If... Q2A.EQ.1.OR.Q2B.EQ.1.OR.Q2C.EQ.1

THOSE WHO RECALL ANY AD IN Q2 SERIES

Did you do anything as a result of hearing this advertising?

Q4B [1,15]

If... (Q2A.EQ.1.OR.Q2B.EQ.1.OR.Q2C.EQ.1).AND.Q4A.EQ.1

THOSE WHO RECALL ANY AD IN Q2 SERIES AND DID SOMETHING AS A RESULT OF SEEING THE AD

What did you do? Anything else?

DO NOT READ LIST - ACCEPT MULTIPLE RESPONSES

Q5A

If... Q2A.EQ.1.OR.Q2B.EQ.1.OR.Q2C.EQ.1

THOSE WHO RECALL ANY AD IN Q2 SERIES

Thinking about the advertising you heard, who do you think produced it? That is, who paid for it?

DO NOT READ LIST. ACCEPT ONE RESPONSE

Q5B

If... (Q2A.EQ.1.OR.Q2B.EQ.1.OR.Q2C.EQ.1).AND.Q5A.EQ.4

THOSE WHO RECALL THE AD IN Q2 AND SAID GOV GENERAL (Q5E)

Which level of government?

READ LIST. ACCEPT ONE RESPONSE

PREQ6 [0,0]

The next three questions have to do with the performance of the government in general.

Q6

How would you rate the performance of the Government of Canada when it comes to encouraging apprentices to complete training in a designated Red Seal trade? Please use a 7-point scale, where 1 means terrible, 7 means excellent and the mid-point, 4, means neither good nor bad.

Q7

And, using the same scale how would you rate the performance of the Government of Canada in providing information to the public in general?

REPEAT SCALE IF NECESSARY

Q8

Generally speaking, how would you rate the overall performance of the Government of Canada? Once again, please use the same scale.

REPEAT SCALE IF NECESSARY

Q9

If... INDAPRNTS.NE.1

GEN-POP Sample

Are you currently enrolled in an apprenticeship program – learning a trade?

Q10

If... INDAPRNTS.EQ.1.OR.Q9.EQ.1

APPRENTICE SAMPLE OR LEARNING A TRADE

What trade are you learning?

DO NOT READ LIST

Q11

APPRENTICE SAMPLE OR LEARNING A TRADE

If... INDAPRNTS.EQ.1.OR.Q9.EQ.1

Have you ever seen or read information about grants or other financial incentives for completing an apprenticeship?

Q11B

THOSE WHO HAVE LOOKED FOR GRANT/INCENTIVE INFORMATION Q11

If... Q11.EQ.1

Where did you go to look for this information?

Select all that apply

Q11B2

APPRENTICE SAMPLE OR LEARNING A TRADE

If... INDAPRNTS.EQ.1.OR.Q9.EQ.1

Do you feel you had enough information available to you about government grants and financial incentives?

Would you say that there was... (READ LIST)

Q11B3

OBTAINED INFO FROM GOV CAN OR SERV CAN IN Q11B

If... Q11B.EQ.1,2

How useful did you find the information available to you about government grants and financial incentives?

Would you say it was... (READ LIST)

Q11C

APPRENTICE SAMPLE OR LEARNING A TRADE

If... INDAPRNTS.EQ.1.OR.Q9.EQ.1

Have you ever applied for grants from a government for completing a portion or all of an apprenticeship program?

Q11D[1,2]

GRANT APPLICANTS (Q11C)

If... Q11C.EQ.1

What level of government did you apply to for grants or other financial incentives?

Select all that apply

Q11E

FEDERAL GOV GRANT APPLICANTS(Q11D)

If... Q11D.EQ.1

Do you remember the name of the grant that you applied for?

Q11E2

GRANT APPLICANTS (Q11C)

If... Q11C.EQ.1

What was your reason for applying for a government grant?

Select all that apply

Q11E3

GRANT APPLICANTS (Q11C)

If... Q11C.EQ.1

How do you find the application process?

PROMPT IF NECESSARY BUT DO NOT READ LIST

Q11F

AIG-ACG GRANT APPLICANTS (Q11E)

If... Q11E.EQ.1,2

Have you received money from a government grant for completing a portion or all of an apprenticeship program?

Q11F2

THOSE WHO RECEIVED MONEY FROM A GOVERNMENT GRANT (Q11F)

If... Q11F.EQ.1.AND.Q11D.EQ.1,2

Was this grant or financial incentive from the federal government or your provincial/territorial government?

Q11F3

THOSE WHO RECEIVED MONEY FROM A PROV/TERR GOVERNMENT GRANT (Q11F2)

If... Q11F2.EQ.2

What province or territory was this from?

Q11G

THOSE WHO RECEIVED MONEY FROM A GOVERNMENT GRANT (Q11F)

If... Q11F.EQ.1

When did you receive this grant?

MONTHQ11 [0,1]

Month grant was received

YESRQ11 [0,1]

Year Grant was received

Q12

APPRENTICE SAMPLE OR LEARNING A TRADE

If... INDAPRNTS.EQ.1.OR.Q9.EQ.1

What motivated you to enter an apprenticeship/learn a trade?

DEMIN [0,0]

Finally, I’d like to ask you some questions for statistical purposes. I'd like to remind you that all your answers are completely confidential.

D1

Which of the following categories best describes your current employment status? Are you...?

READ LIST. ACCEPT ONE ANSWER ONLY

D2

What is the highest level of formal education that you have completed to date?

READ LIST. ACCEPT ONE RESPONSE

D3

In what year were you born? NOTE: ANSWER THE FULL YEAR, I.E. 1977 as "1977"

D4

Are there any children under the age of 18 currently living in your household?

D5

Which of the following categories best describes your total household income? That is, the total income of all persons in your household, before taxes?

READ LIST

D6

Were you born in Canada?

PRED7

In the last two weeks, have you…

D7A

Read a daily newspaper?

D7B

Watched television?

D7C

Listened to the radio?

D7D

Used public transit?

D7E

Read a magazine?

D7F

Attended a fair or exhibition?

D7G

Used the Internet?

PRED7V2 [0,0]

If D7G.EQ.1

In the last two weeks, have you…

D7H

If... D7G.EQ.1

Used the Internet to access Facebook?

D7I

If... D7G.EQ.1

Used the Internet to access Youtube?

D7J

If... D7G.EQ.1

Used the Internet to access Twitter?

D7K

If... D7G.EQ.1

Used the Internet to access other social media sites

THNK

That concludes the survey. This survey was conducted on behalf of Human Resources and Skills Development Canada. In the coming months the report will be available from Library and Archives Canada. Thank you very much for taking part. It is appreciated.

End of Interview


1 Initially 27 schools agreed to participate in the study, however, students from only 21 schools ultimately registered at the website.

2 Response rates are calculated using the formula developed by the MRIA in conjunction with the Government of Canada.