PWGSC POR # 022-15
Employment and Social Development Canada
October 20, 2015
Prepared for:
Employment and Social Development Canada
Ce rapport est aussi disponible en français
Contract #: G9292-164589/001/CY
Contract award date: 2015-07-17
Project cost: $43,315.93
Prepared by:
Opinion Search Inc.
For further information:
For more information on this report, please contact:
nc-por-rop-gd@hrsdc-rhdcc.gc.ca
Any material or information provided by Employment and Social Development Canada and all data collected by Opinion Search will be treated as confidential by Opinion Search and will be stored securely while on Opinion Search’s premise (adhering to industry standards and applicable laws).
Employment and Social Development Canada (ESDC) commissioned Opinion Search to conduct quantitative research with Canadians on the topic of the Universal Child Care Benefit advertising campaign.
The research was performed to satisfy the Government of Canada’s requirement that all major ad campaigns be assessed using the Advertising Campaign Evaluation Tool. Specific objectives were:
It is expected that the results from this survey will be used to evaluate the effectiveness of the Universal Child Care Benefit ad campaign.
To meet the research objectives, a landline telephone survey was conducted with 2,015 Canadian adults, 18+ years. Surveys were conducted between July 22 to Saturday August 1, 2015 (in English and French) and took an average of 7.3 minutes to complete. A sample of this size yields a margin of error of +/-2.18%, 19 times out of 20. A detailed description of the survey methodology used to complete this research, including sample design, survey administration, and response rates (along with margin of error), is included in the Survey Methodology section. It contains all details necessary to replicate this study in the future.
The total contracted price of the research was $43,315.93 (including HST).
The Government of Canada aired the Enhanced Universal Child Care Benefit (UCCB) Advertising Campaign with the objective of raising awareness and visibility of the Family Tax Cuts and Benefits Plan, including the Enhanced UCCB.
As part of the Family Tax Cuts and Benefits Plan announced in October 2014, the Government of Canada has enhanced the Universal Child Care Benefit by increasing the benefit from $1,200 to $1,920 per year for each child under 6, and has introduced a new benefit of $720 per year for each child aged 6-17.
The planned media buy was in excess of $1,000,000. Per the Communications Policy of the Government of Canada, federal government advertising valued over $1 million must be assessed post-campaign. As such, the purpose of this research was to conduct post-campaign testing utilizing the standard Advertising Campaign Evaluation Tool (ACET). This research supported the Government of Canada’s commitment to Canadian families by helping to gauge levels of awareness and visibility of the Enhanced UCCB Enhanced Universal Child Care Benefit Advertising Campaign.
The objectives were to:
Opinion Search was not responsible for the survey design, development of the methodology or any analysis of the research.
The overall approach for this study was to conduct a telephone survey, using both landline and cell phone components, with Canadians 18 years of age and older, from across the country. A detailed discussion of the approach used to complete this research is presented below.
The questionnaire for this study was designed by Employment and Social Development Canada (ESDC) and was based on the Government of Canada’s Advertising Campaign Evaluation Tool (ACET), which is the standard tool for research on advertising recall. The survey was offered in both English and French depending on the needs of the respondent.
The sampling plan for the study that was provided by the ESDC team was designed to complete interviews with a random, representative sample of 2,000 Canadians aged 18 and over. The telephone surveys were completed using random-digit-dialing based on the Canada Survey Sampler, a proprietary selection engine specifically designed to generate a random sample of telephone numbers to be dialed based on exchanges. For this project, the survey completions were collected using both random landline and pre-screened cell phone only (CPO) numbers from ASDE. 80 percent of completions were with landline only or respondents with both a landline and cell phone and the remaining 20 percent were cell phone only.
In order to ensure that the survey was conducted with a random audience of Canadians, the survey instrument used the “most recent birthday” respondent selection method for the landline surveys (which means that in households with more than one eligible respondent, the person with the most recent birthday in the household is selected as the one to be interviewed). The use of this “most recent birthday” method was not applicable for the cell phone dialing.
While the sample database was generated and stratified using Canadian regional population numbers, the data was weighted in tabulation to replicate actual population distribution by region, gender and age, according to the most current Census data available.
The survey was conducted in English and French from Wednesday July 22nd to Saturday August 1st, 2015. Given the nature of the research (which includes advertising recall) a limited time frame was required in order to evaluate the recall and effectiveness of the campaign. The survey was conducted via telephone by Opinion Search Inc. using the Confirmit Horizons CATI technology in our data collection operation, by trained and fully supervised interviewers.
Quotas were used on the study to ensure that, prior to any additional weighting; we were working towards making the data as representative of the population by region as possible. Quotas were established on region, age and gender:
| Region | Unweighted n Landline | Unweighted n Cell Phone | Weighted n Landline | Weighted n Cell Phone |
|---|---|---|---|---|
| Newfoundland | 25 | 6 | 23 | 7 |
| Nova Scotia | 45 | 15 | 37 | 21 |
| PEI | 3 | 1 | 2 | 1 |
| New Brunswick | 42 | 4 | 41 | 5 |
| Quebec | 397 | 67 | 388 | 87 |
| Ontario | 596 | 175 | 538 | 240 |
| Manitoba/Nunavut | 61 | 16 | 47 | 21 |
| Saskatchewan | 51 | 12 | 42 | 16 |
| Alberta/NWT | 174 | 61 | 152 | 68 |
| British Columbia/Yukon | 212 | 52 | 198 | 79 |
| Total | 1,606 | 409 | 1468 | 545 |
| Gender | Unweighted n Landline | Unweighted n Cell Phone | Weighted n Landline | Weighted n Cell Phone |
|---|---|---|---|---|
| Male | 763 | 245 | 671 | 297 |
| Female | 843 | 164 | 797 | 249 |
| Total | 1,606 | 409 | 1468 | 546 |
| Age | Unweighted n Landline | Unweighted n Cell Phone | Weighted n Landline | Weighted n Cell Phone |
|---|---|---|---|---|
| 18-34 | 122 | 227 | 188 | 346 |
| 35-54 | 441 | 124 | 551 | 157 |
| 55+ | 961 | 42 | 648 | 27 |
| Unknown | 82 | 16 | 82 | 16 |
| Total | 1,606 | 409 | 1469 | 546 |
From there, the final data was weighted based on the 2011 census information. Weighting was applied based on region, gender and age statistics to ensure that the final dataset was in proportion to the Canadian population.
While the survey was ongoing, a minimum of 10 percent of all completed interviews were independently monitored and validated by on-site quality control monitors. To count towards the 10 percent total, 75 percent of each survey was monitored. Furthermore, the survey was registered with the Marketing Research and Intelligence Association’s (MRIA) Research Registration System.
A total of 128,313 Canadian phone numbers were dialed for this study, of which 2,015 qualified as eligible and completed the survey (adults 18 years and older). The overall response rate achieved for this study was 2.84%. The following report on sample disposition and response rate follows MRIA guidelines, which are set up to establish consistency in reporting across the market research industry.
| A (1-14) Total Attempted | 128,313 |
|---|---|
| 1 Not in service | 40,151 |
| 2 Fax | 416 |
| 3 Invalid #/Wrong# | 462 |
| B (4-14) Total Eligible | 85,530 |
| 4 Busy | 1,931 |
| 5 Answering machine | 12,399 |
| 6 No answer | 47,006 |
| 7 Language barrier | 794 |
| 8 Ill/Incapable | 415 |
| 9 Eligible not available/Callback | 3,580 |
| C (10-14) Total Asked | 19,405 |
| 10 Household/Company Refusal | 3,854 |
| 11 Respondent Refusal | 12,983 |
| 12 Qualified Termination | 139 |
| D (13-14) Co-operative Contact | 2,429 |
| 13 Not Qualified | 413 |
| 14 Completed Interview | 2,015 |
| REFUSAL RATE | 87.5% |
| (10+11+12) / C | |
| RESPONSE RATE | 2.84% |
| D (13-14) / B (4-14) | |
| INCIDENCE* | 83.9% |
| [(14+12) / (13+14+12)]*100 | |
| [(CI+QualTM)/(NQ+CI+QualTM)]*100 |
The approximate margin of error for a sample of this size is plus or minus 2.18% at the 95% confidence level. This margin of error will increase when analyzing smaller sub samples.
When conducting survey research by phone, there are certain demographic groups that are typically underrepresented due to non-response. A corresponding overrepresentation will naturally exist in the data as well. In this survey, individuals born outside of Canada, some age groups and some education levels all contain a bias when compared to the general Canadian population according to the 2011 Statistics Canada Census. Weighting by region, age and gender was conducted.
| Demographic Group | Unweighted Data | 2011 Statistics Canada Census |
|---|---|---|
| Born Outside of Canada | 20% | 25% |
| 18-24 Years Old | 5% | 11% |
| 55 Years and Older | 50% | 33% |
| Undergraduate University Program or Higher | 39% | 23% |
| College, Vocational or Trade School | 26% | 36% |
The average survey duration was 7.3 minutes.
I hereby certify as a Senior Officer of Opinion Search that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not contain any reference to electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leader.
Bruce Wilson
Senior Operations Field Manager
Opinion Search Inc.
Tel: 613-230-9109
The final cost for the data collection portion of the project was $43,315.93 (including HST).