PWGSC POR # 022-15
Employment and Social Development Canada
October 20, 2015
Prepared for:
Employment and Social Development Canada
Ce rapport est aussi disponible en français
Contract #: G9292-164589/001/CY
Contract award date: 2015-07-17
Project cost: $43,315.93
Prepared by:
Opinion Search Inc.
For further information:
For more information on this report, please contact:
nc-por-rop-gd@hrsdc-rhdcc.gc.ca
Any material or information provided by Employment and Social Development Canada and all data collected by Opinion Search will be treated as confidential by Opinion Search and will be stored securely while on Opinion Search’s premise (adhering to industry standards and applicable laws).
Employment and Social Development Canada (ESDC) commissioned Opinion Search to conduct quantitative research with Canadians on the topic of the Universal Child Care Benefit advertising campaign.
The research was performed to satisfy the Government of Canada’s requirement that all major ad campaigns be assessed using the Advertising Campaign Evaluation Tool. Specific objectives were:
It is expected that the results from this survey will be used to evaluate the effectiveness of the Universal Child Care Benefit ad campaign.
To meet the research objectives, a landline telephone survey was conducted with 2,015 Canadian adults, 18+ years. Surveys were conducted between July 22 to Saturday August 1, 2015 (in English and French) and took an average of 7.3 minutes to complete. A sample of this size yields a margin of error of +/-2.18%, 19 times out of 20. A detailed description of the survey methodology used to complete this research, including sample design, survey administration, and response rates (along with margin of error), is included in the Survey Methodology section. It contains all details necessary to replicate this study in the future.
The total contracted price of the research was $43,315.93 (including HST).
The Government of Canada aired the Enhanced Universal Child Care Benefit (UCCB) Advertising Campaign with the objective of raising awareness and visibility of the Family Tax Cuts and Benefits Plan, including the Enhanced UCCB.
As part of the Family Tax Cuts and Benefits Plan announced in October 2014, the Government of Canada has enhanced the Universal Child Care Benefit by increasing the benefit from $1,200 to $1,920 per year for each child under 6, and has introduced a new benefit of $720 per year for each child aged 6-17.
The planned media buy was in excess of $1,000,000. Per the Communications Policy of the Government of Canada, federal government advertising valued over $1 million must be assessed post-campaign. As such, the purpose of this research was to conduct post-campaign testing utilizing the standard Advertising Campaign Evaluation Tool (ACET). This research supported the Government of Canada’s commitment to Canadian families by helping to gauge levels of awareness and visibility of the Enhanced UCCB Enhanced Universal Child Care Benefit Advertising Campaign.
The objectives were to:
Opinion Search was not responsible for the survey design, development of the methodology or any analysis of the research.
The overall approach for this study was to conduct a telephone survey, using both landline and cell phone components, with Canadians 18 years of age and older, from across the country. A detailed discussion of the approach used to complete this research is presented below.
The questionnaire for this study was designed by Employment and Social Development Canada (ESDC) and was based on the Government of Canada’s Advertising Campaign Evaluation Tool (ACET), which is the standard tool for research on advertising recall. The survey was offered in both English and French depending on the needs of the respondent.
The sampling plan for the study that was provided by the ESDC team was designed to complete interviews with a random, representative sample of 2,000 Canadians aged 18 and over. The telephone surveys were completed using random-digit-dialing based on the Canada Survey Sampler, a proprietary selection engine specifically designed to generate a random sample of telephone numbers to be dialed based on exchanges. For this project, the survey completions were collected using both random landline and pre-screened cell phone only (CPO) numbers from ASDE. 80 percent of completions were with landline only or respondents with both a landline and cell phone and the remaining 20 percent were cell phone only.
In order to ensure that the survey was conducted with a random audience of Canadians, the survey instrument used the “most recent birthday” respondent selection method for the landline surveys (which means that in households with more than one eligible respondent, the person with the most recent birthday in the household is selected as the one to be interviewed). The use of this “most recent birthday” method was not applicable for the cell phone dialing.
While the sample database was generated and stratified using Canadian regional population numbers, the data was weighted in tabulation to replicate actual population distribution by region, gender and age, according to the most current Census data available.
The survey was conducted in English and French from Wednesday July 22nd to Saturday August 1st, 2015. Given the nature of the research (which includes advertising recall) a limited time frame was required in order to evaluate the recall and effectiveness of the campaign. The survey was conducted via telephone by Opinion Search Inc. using the Confirmit Horizons CATI technology in our data collection operation, by trained and fully supervised interviewers.
Quotas were used on the study to ensure that, prior to any additional weighting; we were working towards making the data as representative of the population by region as possible. Quotas were established on region, age and gender:
Region | Unweighted n Landline | Unweighted n Cell Phone | Weighted n Landline | Weighted n Cell Phone |
---|---|---|---|---|
Newfoundland | 25 | 6 | 23 | 7 |
Nova Scotia | 45 | 15 | 37 | 21 |
PEI | 3 | 1 | 2 | 1 |
New Brunswick | 42 | 4 | 41 | 5 |
Quebec | 397 | 67 | 388 | 87 |
Ontario | 596 | 175 | 538 | 240 |
Manitoba/Nunavut | 61 | 16 | 47 | 21 |
Saskatchewan | 51 | 12 | 42 | 16 |
Alberta/NWT | 174 | 61 | 152 | 68 |
British Columbia/Yukon | 212 | 52 | 198 | 79 |
Total | 1,606 | 409 | 1468 | 545 |
Gender | Unweighted n Landline | Unweighted n Cell Phone | Weighted n Landline | Weighted n Cell Phone |
---|---|---|---|---|
Male | 763 | 245 | 671 | 297 |
Female | 843 | 164 | 797 | 249 |
Total | 1,606 | 409 | 1468 | 546 |
Age | Unweighted n Landline | Unweighted n Cell Phone | Weighted n Landline | Weighted n Cell Phone |
---|---|---|---|---|
18-34 | 122 | 227 | 188 | 346 |
35-54 | 441 | 124 | 551 | 157 |
55+ | 961 | 42 | 648 | 27 |
Unknown | 82 | 16 | 82 | 16 |
Total | 1,606 | 409 | 1469 | 546 |
From there, the final data was weighted based on the 2011 census information. Weighting was applied based on region, gender and age statistics to ensure that the final dataset was in proportion to the Canadian population.
While the survey was ongoing, a minimum of 10 percent of all completed interviews were independently monitored and validated by on-site quality control monitors. To count towards the 10 percent total, 75 percent of each survey was monitored. Furthermore, the survey was registered with the Marketing Research and Intelligence Association’s (MRIA) Research Registration System.
A total of 128,313 Canadian phone numbers were dialed for this study, of which 2,015 qualified as eligible and completed the survey (adults 18 years and older). The overall response rate achieved for this study was 2.84%. The following report on sample disposition and response rate follows MRIA guidelines, which are set up to establish consistency in reporting across the market research industry.
A (1-14) Total Attempted | 128,313 |
---|---|
1 Not in service | 40,151 |
2 Fax | 416 |
3 Invalid #/Wrong# | 462 |
B (4-14) Total Eligible | 85,530 |
4 Busy | 1,931 |
5 Answering machine | 12,399 |
6 No answer | 47,006 |
7 Language barrier | 794 |
8 Ill/Incapable | 415 |
9 Eligible not available/Callback | 3,580 |
C (10-14) Total Asked | 19,405 |
10 Household/Company Refusal | 3,854 |
11 Respondent Refusal | 12,983 |
12 Qualified Termination | 139 |
D (13-14) Co-operative Contact | 2,429 |
13 Not Qualified | 413 |
14 Completed Interview | 2,015 |
REFUSAL RATE | 87.5% |
(10+11+12) / C | |
RESPONSE RATE | 2.84% |
D (13-14) / B (4-14) | |
INCIDENCE* | 83.9% |
[(14+12) / (13+14+12)]*100 | |
[(CI+QualTM)/(NQ+CI+QualTM)]*100 |
The approximate margin of error for a sample of this size is plus or minus 2.18% at the 95% confidence level. This margin of error will increase when analyzing smaller sub samples.
When conducting survey research by phone, there are certain demographic groups that are typically underrepresented due to non-response. A corresponding overrepresentation will naturally exist in the data as well. In this survey, individuals born outside of Canada, some age groups and some education levels all contain a bias when compared to the general Canadian population according to the 2011 Statistics Canada Census. Weighting by region, age and gender was conducted.
Demographic Group | Unweighted Data | 2011 Statistics Canada Census |
---|---|---|
Born Outside of Canada | 20% | 25% |
18-24 Years Old | 5% | 11% |
55 Years and Older | 50% | 33% |
Undergraduate University Program or Higher | 39% | 23% |
College, Vocational or Trade School | 26% | 36% |
The approximate margin of error for a sample of this size is plus or minus 2.18% at the 95% confidence level. This margin of error will increase when analyzing smaller sub samples.
When conducting survey research by phone, there are certain demographic groups that are typically underrepresented due to non-response. A corresponding overrepresentation will naturally exist in the data as well. In this survey, individuals born outside of Canada, some age groups and some education levels all contain a bias when compared to the general Canadian population according to the 2011 Statistics Canada Census. Weighting by region, age and gender was conducted.
I hereby certify as a Senior Officer of Opinion Search that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not contain any reference to electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leader.
Bruce Wilson
Senior Operations Field Manager
Opinion Search Inc.
Tel: 613-230-9109
The final cost for the data collection portion of the project was $43,315.93 (including HST).
Hello, my name is [Insert: your name]. I’m calling on behalf of Opinion Search, a public opinion research firm. I’d like to ask you a few questions on current issues of interest to Canadians. Before we begin, I would like to know if you prefer to do the survey in French or English.
Please note that your responses will be kept entirely confidential and anonymous and that this survey is registered with the Marketing Research and Intelligence Association (MRIA).
[IF ASKED] The survey will take about 7 minutes to complete.
CS1. Have I reached you on your cell phone?
CS2. Are you 18 years of age or older?
CS3. Are you in a place where you can safely talk on the phone and answer my questions?
CS4. When would it be more convenient for me to call back?
CS5. Would you be willing to take part in this survey?
C1. In addition to your cellular telephone, does your household currently have a traditional telephone or landline?
LS1. May I please speak with the person in your household who is 18 years of age or older and who has had the most recent birthday? Would that be you? [IF THAT PERSON IS NOT AVAILABLE ARRANGE CALLBACK]
LS2. Would you be willing to take part in this survey? We can do it now or at a time more convenient for you.
L1. In addition to your residential landline telephone, do you or someone else in your household also use one or more cell phone numbers?
Q1A:
Over the past few weeks or so, have you seen, heard or read any advertising about increasing benefits for families with children?
Q1B:
What can you remember about this advertising? What words, sounds or images come to mind? [ACCEPT MULTIPLE RESPONSES]
Q2A:
Do you remember seeing an ad on television over the past few weeks or so that showed parents and children celebrating a birthday? The ad mentioned increasing the Universal Child Care Benefit for every family with children under 18.
Q2B. And do you remember seeing this same ad as a video on the Internet over the past few weeks or so?
[READ AND ROTATE Q2C&D&E]
Q2C. And do you remember seeing an Internet banner ad over the past few weeks or so that mentioned increasing the Universal Child Care Benefit for every family with children under 18?
Q2D. And do you remember hearing an ad on the radio over the past few weeks that mentioned increasing the Universal Child Care Benefit for every family with children under 18?
Q2E. And do you remember seeing an ad in a newspaper that mentioned increasing the Universal Child Care Benefit for every family with children under 18? The ad showed a picture of two parents and children celebrating a birthday.
IF RESPONDENT RECALLS THE AD IN Q2A-E, CONTINUE; OTHERWISE SKIP TO D1.
Q3:
Thinking about the advertising that you saw or heard, what do you think was the main point these advertising were trying to get across? [OPEN-ENDED, ACCEPT ONE RESPONSE]
Q5A:
With regard to the ads you have seen or heard, in your opinion, who produced them? In other words, who funded them? [DO NOT READ LIST, ACCEPT ONE RESPONSE]
[ASK Q5B IF Q5A = ‘Government (General)’ OTHERWISE GO TO D1]
Q5B:
Which level of government? [READ LIST, ACCEPT ONE RESPONSE]
Finally, I’d like to ask you some questions for statistical purposes. I'd like to remind you that all your answers are completely confidential.
D1:
Which of the following categories best describes your current employment status? Are you...? [READ LIST, ACCEPT ONE RESPONSE]
D2:
What is the highest level of formal education that you have completed to date? [READ LIST, ACCEPT ONE RESPONSE]
D3:
In what year were you born?
D4:
Are there any children under the age of 18 currently living in your household?
D5:
Which of the following categories best describes your total household income? That is, the total income of all persons in your household, before taxes? [READ LIST, ACCEPT ONE RESPONSE]
D6:
Were you born in Canada?
D7: What is the language you first learned at home as a child and still understand? [DO NOT READ LIST, ACCEPT UP TO TWO RESPONSES]
D8 :
In the last two weeks, have you…
RANDOMIZE MEDIA CHANNELS
That concludes the survey. This survey was conducted on behalf of Employment and Social Development Canada. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to participate, it is greatly appreciated.
INTERVIEWER RECORDS: