Prepared for the Employment and Social Development Canada (ESDC)
For more information on this report, please contact nc-por-rop-gd@hrsdc-rhdcc.gc.ca
This public opinion research report presents the methodology for the conduct of two online surveys conducted by Ekos Research Associates Inc. on behalf of Employment and Social Development Canada (ESDC).
Cette publication est aussi disponible en français sous le titre : 2021-2022 L’évaluation de la campagne publicitaire Services pour les aînés : rapport méthodologique de l'OECP.
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Employment and Social Development Canada.
It is available upon request in multiple formats (large print, MP3, braille, e-text, DAISY), by contacting 1 800 O-Canada (1-800-622-6232). By teletypewriter (TTY), call 1-800-926-9105.
ESDC
© Her Majesty the Queen in Right of Canada, 2022
For information regarding reproduction rights: droitdauteur.copyright@HRSDC-RHDCC.gc.ca.
Employment and Social Development Canada (ESDC) supports the Minister of Seniors and is the Government of Canada's department responsible for developing, managing and delivering a wide range of social programs and services for seniors. The Services for Seniors campaign was tailored to support ESDC's mission to build a stronger and more competitive Canada, as well as to support Canadians in making choices that help them live productive and rewarding lives. This campaign was a continuation of the 2018-2019, 2019-2020, and 2020-2021 campaigns, led by ESDC in collaboration with the Anti-Fraud Centre, Canada Revenue Agency, Competition Bureau, the Financial Consumer Agency of Canada, Health Canada, and Veteran's Affairs Canada.
The Services for Seniors advertising campaign featured the Government of Canada's programs and services that either directly or indirectly benefit seniors, including changes to Canada Pension Plan (CPP) and Old Age Security (OAS). Canadians who are 55 or older and preparing for or living in retirement was the primary target audience for the campaign.
Because the advertising exceeded $1,000,000, public opinion research is required by Treasury Board using the Advertising Campaign Evaluation Tool (ACET), with additional validating and benchmarking questions following the ACET instrument. Data collection to evaluate the campaign supports the Government of Canada's goal of effectively gauging Canadians' awareness and knowledge of its initiatives and determining the effectiveness of the campaign.
The research included two components:
Two bilingual, national online surveys were conducted: one as a pre-campaign baseline, in advance of the campaign, and one at the end of the campaign[1] to measure change in awareness, and to assess the impact of the campaign. The baseline survey included 2,040 Canadians 55 years of age or older, and was collected between December 20th, 2021 and January 17th, 2022. The post campaign sample included 2,028 Canadians, also 55 years of age or older, and was collected between February 28th and March 14th, 2022. Each survey instrument included the Advertising Campaign Evaluation Tool (ACET) questions used by the Government of Canada in all of its advertising evaluation research, along with additional questions testing the impact on perceptions and behaviour targeted through the campaign. Each sample included randomly selected respondents from all provinces and territories.
Each survey sample relied on Ekos' Probit panel, which is assembled using a random digit dial process for sampling from a blended land-line cell-phone frame, and provides full coverage of Canadians with telephone access. The distribution of the recruitment process is meant to mirror the actual population in Canada (as defined by Statistics Canada). As such, our more than 120,000 member panel can be considered representative of the general public in Canada (meaning that the incidence of a given target population within our panel very closely resembles the public at large) and margins of error can be applied.
Appendix A provides the details of the methodology for these pre- and post-campaign surveys. The pre- and post-campaign questionnaires can be found in Appendix B and C.
I hereby certify as Senior Officer of Ekos Research Associates Inc. that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications.
Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.
Signed by: Susan Galley (Vice President)
The Advertising Campaign Evaluation Tool (ACET) consisted of a baseline and post-campaign survey, featuring questions used by the Government of Canada in all of its advertising evaluation research. The Services for Seniors advertising campaign included animated banners, print, Facebook, and search engine marketing ads. The duration of the campaign was 42 days from January 24 to March 6, 2022. The ACET focused on recall of the campaign, along with questions testing the impact of the ads on perceptions and behaviour. The average time it took respondents to complete the survey was seven minutes for the baseline, and 10 minutes for the post-campaign survey. The baseline survey was pre-tested with 50 respondents (30 in English, 20 in French), and the post-campaign survey was pre-tested with 48 respondents (25 in English, 23 in French). Since no significant changes were made these cases were retained in the final sample.
In each case, respondents to the online survey were 55 years of age and older and were randomly selected. The sample included all provinces and territories, and the survey was administered in English and French, as well as with an accessible link for those using a mobile phone or screen reading technology. The survey sample relied on Ekos' Probit panel, which is assembled using a random digit dial process for sampling from a blended land-line cell-phone frame and provides full coverage of Canadians with telephone access. The distribution of the recruitment process is meant to mirror the actual population in Canada (as defined by Statistics Canada). As such, our more than 120,000-member panel can be considered representative of the general public in Canada (meaning that the incidence of a given target population within our panel very closely resembles the public at large) and margins of error can be applied. All households/individuals in the Probit panel are contacted by telephone and the nature of the panel is explained in greater detail (as are Ekos' privacy policies) and demographic information is collected. At this time, the online/off-line as well as landline/cell phone status is ascertained to determine the method of completing surveys (i.e., online, telephone, or mail). This variable of ‘type of telephone service' (cell phone only, landline only or both) collected at the time of screening is used to determine cell phone only sample. As with any random digit dialing sample, Probit panel cases are considered to be a probability-based sample.
In the baseline survey, the total number of completed cases in the sample was 2,040 and was collected between December 20, 2021 and January 17, 2022. The post-campaign survey was collected between February 28 and March 14, 2022 including 2,028 cases[2]. The associated margin of error is up to plus or minus 2.2%, at a .95 confidence interval (i.e., 19 times out of 20) in each sample. The regional distribution is as follows:
| Province/Region | Baseline | Post-Campaign |
|---|---|---|
| British Columbia | 260 | 281 |
| Alberta | 224 | 239 |
| Manitoba & Saskatchewan | 145 | 152 |
| Ontario | 787 | 788 |
| Quebec | 481 | 437 |
| Atlantic | 143 | 131 |
| Total | 2,040 | 2,028 |
Survey data collection adhered to the Standards for the Conduct of Government of Canada Public Opinion Research—Online Surveys as well as all applicable industry standards. Ekos informed respondents of their rights under the Privacy Act and the Access to Information Act and ensured that those rights were protected throughout the research process. This included: informing respondents of the purpose of the research; identifying both the sponsoring department and the research supplier; informing respondents that their participation in the study is voluntary, and that the information provided would be administered according to the requirements of the Privacy Act.
Once the survey data was collected, each database was reviewed for data quality. Coding was also completed. Survey results were weighted based on Statistics Canada data according to age, gender and region to ensure the sample was representative of the general public aged 55 years and older.
Data tables were created for each survey to isolate results for major subgroups to be used in the analysis (e.g., results for each age segment, gender, region, education and household income). Other subgroups include mother tongue, respondents born in Canada, family with children, employment, recalled seeing an ad from the Government of Canada (in general and for older Canadians and/or seniors), disability identity, ethnicity and indigenous identity.
The response rate for the online pre-campaign baseline survey was 45.6%, using the formula recommended by the Government of Canada: Response Rate/Participation Rate = R/(U + IS + R).
A total of 4,918 email invitations were sent, of which 106 were returned as undeliverable, for a resulting valid sample of 4,812. A total of 2,040 valid cases were completed and combined with another 154 respondents who were screened out of the survey as ineligible[3], as the responding numerator in the calculation of the response rate. In the post-campaign, 5,563 invitations were sent, of which 586 were undeliverable for a valid sample of 4,977. In addition to the 2,028 valid cases that completed the survey, 176 respondents were screened out of the survey. The resulting response rate was 44.3%.
| Col1 | Pre-campaign Baseline | Post-Campaign |
|---|---|---|
| Total Invited | 4,918 | 5,563 |
| Invalid | 0 | 0 |
| Unresolved (U) | 106 | 586 |
| In-scope Responding (IS) | 2,618 | 2,773 |
| Responding Units (R) | 2,194 | 2,204 |
| Response Rate (R/(U+IS+R) | 45.6% | 44.3% |
A comparison of each unweighted sample with 2016 Census figures from Statistics Canada suggests that there are similar sources of systematic sample bias in the survey, following patterns typically found in most general public surveys. There is a somewhat more educated sample in each survey than found in the population with 26% reporting university degrees in the baseline and 27% in the post-campaign survey, compared with 23% in the population. There is also an under representation of Canadians born outside of Canada in each survey (13% in the baseline and 13% in the post-campaign survey versus 22%). As previously described, each sample was weighted by age, gender and region.
Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians. Si vous préférez répondre au sondage en français, veuillez cliquer sur français.
Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 9 minutes to complete.
Click here if you wish to verify the authenticity of this survey by entering project code 20220104-EK926. To view our privacy policy, click here. If you require any technical assistance, please contact online@ekos.comp>
Does anyone in your household work for any of the following organizations?
Select all that apply
Are you...
In what year were you born?
In which age category do you belong?
Select one only
In which province or territory do you live?
Select one only
Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad?
Select all that apply
What do you remember about this ad?
How did you know that it was an ad from the Government of Canada?
Over the past three weeks, have you seen, read or heard any Government of Canada advertising about programs and services for older Canadians and/or seniors?
Where have you seen, read or heard this Government of Canada ad about programs and services for older Canadians and/or Seniors?
Select all that apply
What do you remember about this ad?
Are you aware of the Government of Canada general information phone number, 1 800 O-Canada?
Have you called the 1 800 O-Canada number in the last 3 months?
Thinking specifically about Government of Canada programs and services related to support for older Canadians and/or seniors, which programs and services are you aware of?
Still thinking specifically about Government of Canada programs and services related to support for older Canadians and/or seniors, which of the following programs and services are you aware of?
Select all that apply
Are you aware of the Government of Canada's Facebook page, Seniors in Canada?
In the past three months, have you visited the Seniors in Canada Facebook page?
What do you like about the Seniors in Canada Facebook page?
Which of the following sources would you be likely to use for information on programs and services for older Canadians and/or seniors:
Select all that apply
Finally, a few questions for statistical purposes only. Please be assured that all of your answers will remain completely confidential.
Which of the following categories best describes your current employment status? Are you...
Select one only
What is the highest level of formal education that you have completed?
Select one only
Are there any children under the age of 18 currently living in your household?
Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
Select one only
Where were you born?
In what year did you first move to Canada?
What is the language you first learned at home as a child and still understand?
Select up to three
Do you identify yourself as Indigenous, that is, are you of First Nations, Inuit, or Métis descent?
Do you live most of the time on-reserve or off-reserve?
What is your ethnic background?
Do you identify as a person with a disability? A person with a disability is a person who has a long-term or recurring impairment (such as vision, hearing, mobility, flexibility, dexterity, pain, learning, developmental, memory or mental health-related) which limits their daily activities inside or outside the home (such as at school, work, or in the community in general).
What are the first three characters of your postal code?
That concludes the survey. This survey was conducted on behalf of Employment and Social Development Canada.
In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.
Screened-out
Unfortunately, based on your responses you are ineligible to participate in this survey. Thank you for your time!
Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians. Si vous préférez répondre au sondage en français, veuillez cliquer sur français.
Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 12 minutes to complete.
Click here if you wish to verify the authenticity of this survey by entering project code 20220104-EK926. To view our privacy policy, click here. If you require any technical assistance, please contact online@ekos.com.
Does anyone in your household work for any of the following organizations?
Select all that apply
Are you...
In what year were you born?
In which age category do you belong?
Select one only
In which province or territory do you live?
Select one only
Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad?
Select all that apply
What do you remember about this ad?
How did you know that it was an ad from the Government of Canada?
Over the past three weeks, have you seen, read or heard any Government of Canada advertising about programs and services for older Canadians and/or seniors?
Where have you seen, read or heard this Government of Canada ad about programs and services for older Canadians and/or Seniors?
Select all that apply
What do you remember about this ad?
Are you aware of the Government of Canada general information phone number, 1 800 O-Canada?
Have you called the 1 800 O-Canada number in the last 3 months?
Thinking specifically about Government of Canada programs and services related to support for older Canadians and/or seniors, which programs and services are you aware of?
Still thinking specifically about Government of Canada programs and services related to support for older Canadians and/or seniors, which of the following programs and services are you aware of?
Select all that apply
Are you aware of the Government of Canada's Facebook page, Seniors in Canada?
In the past three months, have you visited the Seniors in Canada Facebook page?
What do you like about the Seniors in Canada Facebook page?
Which of the following sources would you be likely to use for information on programs and services for older Canadians and/or seniors:
Select all that apply
Here are some ads that have recently been broadcast on various media.
[1 randomized banner ad among 4]
[1 randomized print ad among 2]
[1 randomized Facebook ad among 5]
Over the past three weeks, have you seen, read or heard these ads?
Where have you seen, read or heard these ads?
Select all that apply
What do you think is the main point these ads are trying to get across?
Please indicate your level of agreement with the following statements about these ads?
These ads catch my attention
These ads are relevant to me
These ads are difficult to follow
These ads do not favour one political party over another
These ads talk about an important topic
These ads provide new information
These ads clearly convey that the Government of Canada has programs and services for older Canadians and/or Seniors
Finally, a few questions for statistical purposes only. Please be assured that all of your answers will remain completely confidential.
Which of the following categories best describes your current employment status? Are you...
Select one only
What is the highest level of formal education that you have completed?
Select one only
Are there any children under the age of 18 currently living in your household?
Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
Select one only
Where were you born?
In what year did you first move to Canada?
What is the language you first learned at home as a child and still understand?
Select up to three
Do you identify yourself as Indigenous, that is, are you of First Nations, Inuit, or Métis descent?
Do you live most of the time on-reserve or off-reserve?
What is your ethnic background?
Do you identify as a person with a disability? A person with a disability is a person who has a long-term or recurring impairment (such as vision, hearing, mobility, flexibility, dexterity, pain, learning, developmental, memory or mental health-related) which limits their daily activities inside or outside the home (such as at school, work, or in the community in general).
What are the first three characters of your postal code?
That concludes the survey. This survey was conducted on behalf of Employment and Social Development Canada.
In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey, it is greatly appreciated.
Screened-out
Unfortunately, based on your responses you are ineligible to participate in this survey. Thank you for your time!
The following are examples of elements of the campaign that went to market and were included in the aided recall question of the Post-Campaign ACET at Question: T1HA.
Animated web banner showing a senior man in work clothes. Text reads, "Seniors know a lot, but there's always more to learn. Help ensure your later years are safe and secure. Learn about programs, services and information available for seniors."
Animated web banner showing a senior woman. Text reads, "Seniors know a lot, but there's always more to learn. Help ensure your later years are safe and secure. Learn about programs, services and information available for seniors."
Animated web banner showing a senior man reading on a tablet. Text reads, "Seniors know a lot, but there's always more to learn. Help ensure your later years are safe and secure. Learn about programs, services and information available for seniors."
Animated web banner showing two seniors exercising at home. Text reads, "Seniors know a lot, but there's always more to learn. Help ensure your later years are safe and secure. Learn about programs, services and information available for seniors."
Social media post of a senior man. Text reads, "Help ensure your later years are safe and secure by learning about programs, services and information available for seniors. Seniors know a lot. But there's more to learn."
Social media post of a senior woman. Text reads, "Help ensure your later years are safe and secure by learning about programs, services and information available for seniors. Seniors know a lot. But there's more to learn."
Social media post of two seniors exercising at home. Text reads, "Ensure your later years are healthy and safe. Learn about physical activity, nutrition, preventing slips and falls, and more. Seniors know a lot. But there's more to learn."
Social media post of a senior man in work clothes. Text reads, "Help ensure your later years are safe and secure. Learn how to make the most of the Canada Pension Plan and other programs. Seniors know a lot. But there's more to learn."
Social media post of a senior woman at work. Text reads, "Help ensure your later years are safe and secure. Learn how to make the most of the Canada Pension Plan and other programs. Seniors know a lot. But there's more to learn."
Print advertisement showing a senior woman. Text reads, "Seniors know a lot, but there's always more to learn. Help ensure your later years are safe and secure. Learn about programs and services for seniors, like how the Canada Pension Plan can work better for you, changes to the Guaranteed Income Supplement, fraud prevention information, and more."
Print advertisement showing a senior man in work clothes. Text reads, "Seniors know a lot, but there's always more to learn. Help ensure your later years are safe and secure. Learn about programs and services for seniors, like how the Canada Pension Plan can work better for you, changes to the Guaranteed Income Supplement, Old Age Security payments, and more."