Prepared for Veterans Affairs Canada
Supplier: Ekos Research Associates Inc.
PSPC Contract Number: # 51019-201018/001/CY
Contract Award Date: 10/21/2020
Delivery Date: 03/30/2021
POR Registration Number: POR 048-20
For more information on this report, please email: vac.marketing-marketing.acc@canada.ca
This report summarizes results from two surveys of 2000 Canadians each.
Cette publication est aussi disponible en français sous le titre : Évaluation de la campagne de publicité sur la commémoration de 2020 : rapport sur la méthodologie
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Veterans Affairs Canada. For more information on this report, please contact Veterans Affairs Canada at: vac.marketing-marketing.acc@canada.ca
Catalogue Number: V44-8/2020E-PDF
International Standard Book Number (ISBN): 978-0-660-36929-7
Related publications (registration number: POR 048-20):
© Her Majesty the Queen in Right of Canada
As part of its mandate, Veterans Affairs Canada (VAC) seeks to engage Canadians in remembrance. In 2020, the advertising campaign focused on the "Our Freedom" digital creative which illustrated the connection that past service has to our current freedom. The campaign was designed to encourage Canadians to remember and honour the accomplishments and sacrifices of our Veterans and invite Canadians' participation in remembrance activities, to visit VAC's website and to engage in social media acts of remembrance to honour them. The campaign's purpose was to honour the Canadians who served our country during times of war, military conflict and peace and to engage Canadians in remembering past and present sacrifices. Advertising concepts demonstrated Canadians' commitment to remember the sacrifices and achievements of Canada's traditional War Service Veterans and modern-day Veterans. The call to action of the campaign was to engage the pride of Canadians through VAC's website or through social media sites.
VAC's Remembrance advertising campaign has been marketed to an extensive demographic with solid post-campaign recall. The 2020 Remembrance Advertising Campaign had a comprehensive set of advertising products that honoured the sacrifices of Canada's Veterans through commemoration via online resources, social media channels and traditional media.
To remember Canada's Veterans, digital creatives were used for this campaign. As with previous campaigns, the 2020 campaign was supported by a marketing strategy to provide maximum reach through traditional channels, as well as online through mobile devices and social media sites, creating multiple opportunities for Canadians to engage in discussions and participate in remembrance. Specifically, this included a call to action to join the conversation on social media by using the hashtag "#CanadaRemembers" or visiting the veterans.gc.ca/CanadaRemembers website.
As the 2020 Remembrance Campaign media buy exceeded $1,000,000, public opinion research is required by Treasury Board using the Advertising Campaign Evaluation Tool (ACET), with additional validating and benchmarking questions following the ACET instrument. Data collection to evaluate the Remembrance Campaign supports the Government of Canada's goal of effectively gauging Canadians knowledge about the efforts of Veterans and those who died in service to Canada; and Canadian military milestones and the Veterans who participated in them are publicly recognized. Information obtained from this process will allow Veterans Affairs Canada to determine the impact of its advertising campaign and provide direction on areas where the campaign can be adjusted to reach a broader Canadian public. The findings will provide useful information to improve future campaigns which are more targeted, informative and geared to the target population.
Specifically, the purpose of the quantitative post-testing was to evaluate the 2020 Remembrance Campaign and to gauge:
Information obtained through this public opinion research will allow Veterans Affairs Canada to determine the impact of its advertising campaign and provide direction on areas where the campaign can be adjusted to reach a broader Canadian public. The findings will provide useful information to improve future campaigns which are better targeted, more informative and better geared to the target population and Canadians at large. It will also provide a comparison to previous measurement of success of the campaign.
Two bilingual, national, online surveys were conducted: one as a baseline, in advance of the campaign, and one following the end of the campaign to measure change in awareness and intent to visit over time, to assess impact of the campaign. The baseline included 2,000 Canadians 18 years of age or older, and the post campaign sample also included 2,000 Canadians, also 18 years of age or older. Each survey instrument was based on the Advertising Campaign Evaluation Tool questions used by the Government of Canada in all of its advertising evaluation research, adding a few campaign-specific questions testing the impact on perceptions and behaviour targeted though the campaign. Each survey instrument included a sample of randomly selected respondents from all provinces and territories.
Each survey sample relied on Ekos' Probit panel, which is assembled using a random digit dial process for sampling from a blended land-line cell-phone frame, and provides full coverage of Canadians with telephone access. The distribution of the recruitment process is meant to mirror the actual population in Canada (as defined by Statistics Canada). As such, the more than 100,000 member panel can be considered representative of the general public in Canada (meaning that the incidence of a given target population within the panel very closely resembles the public at large) and margins of error can be applied. In the two surveys, the number of total completed cases in the sample is 2,000 with an associated margin of error of up to plus or minus 2.2%, at a .05 confidence interval (i.e., 19 times out of 20).
The contract value for the POR project is $49,737.47 (including HST).
Supplier Name: Ekos Research Associates
PWGSC Contract Number: #51019– 201018/001/CY
Contract Award Date: October 21, 2020
To obtain more information on this study, please e-mail: vac.marketing-marketing.acc@canada.ca, or call 1-866-522-2122.
This certification is to be submitted with the final report submitted to the Project Authority.
I hereby certify as Senior Officer of Ekos Research Associates Inc. that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research.
Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.
Signed by:
Susan Galley (Vice President)
The survey was conducted in two phases: a benchmarking phase prior to the campaign, and a post-campaign phase. Each survey instrument consisted of the Advertising Campaign Evaluation Tool questions used by the Government of Canada for evaluating campaigns over $1,000,000. The post-campaign survey added questions testing recall of one of two videos and an internet banner ad key to the campaign, along with questions testing the impact of the creatives on perceptions and behaviour. The average time it took respondents to complete the questionnaire was 10.4 minutes for the baseline, and 13.3 minutes for the post-campaign survey. There was a pretesting phase conducted for each survey between October 24 and 25, 2020 (baseline) and between November 12 and 13, 2020 (post-campaign) to ensure each questionnaire was working well. This entailed completing cases with 14 English and 18 French (baseline), and 13 English and 10 French (post-campaign) with a suite of questions appended at the end to allow respondents a chance to describe their experience with the survey, in terms of comprehension, relevance, and technical issues. The results of the pretest surveys were reviewed to check for programming and logic errors.
In each case, respondents to the online survey were 18 years of age and older, and were randomly selected. The sample included all provinces and territories, and the survey was administered in English and French, as well as with an accessible link for those using a mobile phone or screen reading technology. The survey sample relied on an Ekos' Probit panel, which is assembled using a random digit dial process for sampling from a blended land-line cell-phone frame, and provides full coverage of Canadians with telephone access. The distribution of the recruitment process is meant to mirror the actual population in Canada (as defined by Statistics Canada). As such, the more than 100,000 member panel can be considered representative of the general public in Canada (meaning that the incidence of a given target population within the panel very closely resembles the public at large) and margins of error can be applied. All households/individuals in the Probit panel are contacted by telephone and the nature of the panel is explained in greater detail (as are privacy policies) and demographic information is collected. At this time the online/off-line as well as landline/cell phone status is ascertained in order to determine the method of completing surveys (i.e., online, telephone, or mail). This variable of ‘type of telephone service' (cell phone only, landline only or both) collected at the time of screening is used to determine cell phone only sample. As with any random digit dialling sample, Probit panel cases are considered to be a probability-based sample.
In each survey, respondents completed the survey online. For most, participation was prompted through an initial or one of several email reminders. A small percentage (two percent in the baseline and three percent the post-campaign) of online completions, however, were prompted by follow-up telephone calls reminding non-respondents to complete the survey online using the link in one of the emails sent. Calls were typically made with non-respondents in lower response segments including those under 35 years of age.
In the two surveys, the number of total completed cases in the sample is 2,000 with an associated margin of error of up to plus or minus 2.2%, at a .05 confidence interval (i.e., 19 times out of 20). Following the testing component, the baseline survey was collected between October 26 and 31, 2020. The post-campaign survey was collected between November 13 and 18, 2020.
The regional distribution of the sample is as follows:
| Province/Region | Baseline |
|---|---|
| British Columbia | 267 |
| Alberta | 235 |
| Manitoba & Saskatchewan | 132 |
| Ontario | 766 |
| Quebec | 449 |
| Atlantic | 148 |
| Territories | 3 |
| Total | 2,000 |
| Province/Region | Post-Campaign |
|---|---|
| British Columbia | 275 |
| Alberta | 233 |
| Manitoba & Saskatchewan | 137 |
| Ontario | 759 |
| Quebec | 449 |
| Atlantic | 141 |
| Territories | 6 |
| Total | 2,000 |
Survey data collection adhered to Government of Canada standards for public opinion research. Ekos informed respondents of their rights under the Privacy Act and the Access to Information Act, and ensured that those rights were protected throughout the research process. This included: informing respondents of the purpose of the research; identifying both the sponsoring department and the research supplier; informing respondents that their participation in the study is voluntary, and that the information provided would be administered according to the requirements of the Privacy Act.
Once the survey data was collected, each database was reviewed for data quality. Coding was also completed. Survey results were weighted based on Statistics Canada data according to age, gender and region to ensure the sample was representative of the general public aged 18 years and older.
Data tables were created for each survey to isolate results for major subgroups to be used in the analysis (e.g., results for each age segment, gender, education segment, employment, income segment, region, households with children under 18, those born outside of Canada, by mother tongue, awareness of ads, importance of advertisements to promote Canada's Veterans, importance of activities to remember and honour Canada's Veterans and importance of programs).
The response rate for the online baseline survey is 12.7%. This is calculated using the formula developed by the Marketing Research Intelligence Agency in conjunction with the Government of Canada. A total of 20,137 email invitations were sent, of which 139 were returned as undeliverable. A total of 2,000 valid cases were completed, and combined with another 541 sampled panel members who were screened out of the survey due to their responses or a filled quota as the numerator in the calculation of the response rate.
For the post-campaign online survey, the response rate is 8.3%, using the same formula. This includes 35,776 email invitations sent, of which 219 were returned as undeliverable. The 2,000 valid cases completed are combined with another 947 sampled panel members who were screened out of the survey due to their responses or a filled quota to form the numerator in the calculation. A total of 196 non-respondents in the baseline survey and 254 non-respondents in the post-campaign survey were reached by telephone and reminded to complete the survey online.
A comparison of each unweighted sample with 2016 Census figures from Statistics Canada suggests that there are similar sources of systematic sample bias in each survey, following patterns typically found in most general public surveys. These include a more educated sample in each survey than found in the population with 41% reporting university degrees in both the baseline and in the post-campaign surveys compared with 25% in the population. Each sample under represents Canadians with a high school level of education or below (22% in both the baseline and in the post-campaign compared with 41% in the population as per 2016 Census figures). There is also an under representation of Canadians born outside of Canada in each survey (14% in the baseline survey and 13% in the post-campaign versus 27% in the general population). This is reflected in an under representation of native speakers of non-official languages and an over representation of native English speakers. In both surveys, 8% of the sample reported having a mother tongue of something other than English or French compared to a 22% incidence in the population at-large. In both surveys, 73% of respondents reported English as their mother tongue, compared to 56% in the general population. As previously described, each sample was weighted by age, gender, and region.
Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians. Si vous préférez répondre au sondage en français, veuillez cliquer sur « Français » dans le coin supérieur droit.
Your participation is voluntary and your responses will be kept entirely confidential and anonymous. The survey takes about 10 minutes to complete. This survey is being directed by Ekos Research, on behalf of Veterans Affairs Canada, and is being administered according to the requirements of the Privacy Act and is registered with the Canadian Research Insights Council's (CRIC) Research Verification Service.
To view our privacy policy, click here.
If you require any technical assistance, please contact online@ekos.com.
Does anyone in your household work for any of the following organizations?
Select all that apply
Are you...
In which of the following age categories do you belong?
In which province or territory do you live?
Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad?
Select all that apply
What do you remember about this ad?
Over the past three weeks, have you seen, read or heard any Government of Canada advertising about Canadian Veterans?
Where have you seen, read or heard this Government of Canada ad about Canadian Veterans?
Select all that apply
What do you remember about this ad? What words, sounds or images come to mind?
Which sources would you rely on when looking for information about remembrance and the contributions of Canada's Veterans?
Select all that apply
Over the past three weeks, have you seen or read content related to Canadian Veterans on social media sites such as Facebook, YouTube, Twitter, Instagram or LinkedIn?
Thinking of commemorative programming that Veterans Affairs Canada is involved with, please rate the importance of the following:
Hosting commemorative events and ceremonies in Canada
Development of commemorative educational resources for schools
Funding for commemorative community initiatives
Hosting commemorative events and ceremonies in Europe
Maintenance of military memorials in Europe
Cemetery maintenance and grave marker repair
Please describe in your own words who you consider a Veteran?
Veterans and members of the Canadian Armed Forces have served in countries around the world. To your knowledge, in what countries have the Canadian military served over the years?
What do you think is the biggest issue facing Veterans today?
And in closing, a few questions that will help us to analyze the survey results.
Which of the following categories best describes your current employment status? Are you...?
What is the highest level of formal education that you have completed?
Are there any children under the age of 18 currently living in your household?
Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
Are you a Canadian Veteran, or do you personally know a Canadian Veteran? If so ... what is your relation to the Canadian Veteran?
Where were you born?
In what year did you first move to Canada?
What is the language you first learned at home as a child and still understand?
That concludes the survey. This survey was conducted on behalf of Veterans Affairs Canada. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey. Your help is greatly appreciated.
We regret that your responses have shown that you are ineligible to participate in this survey. Thank you for your time.
Thank you for taking the time to complete this survey dealing with current issues of interest to Canadians.
Si vous préférez répondre au sondage en français, veuillez cliquer sur « Français » dans le coin supérieur droit.
Your participation is voluntary and your responses will be kept entirely confidential and anonymous. The survey takes about 10 minutes to complete. This survey is being directed by Ekos Research, on behalf of Veterans Affairs Canada and is being administered according to the requirements of the Privacy Act and is registered with the Canadian Research Insights Council's (CRIC) Research Verification Service (20201022-EK923).
To view our privacy policy, click here.
If you require any technical assistance, please contact online@ekos.com.
For the visually impaired, please click Accessible version at the top of any screen.
Does anyone in your household work for any of the following organizations?
Select all that apply
Are you...
In which of the following age categories do you belong?
In which province or territory do you live?
Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
Think about the most recent ad from the Government of Canada that comes to mind. Where have you seen, read or heard this ad?
Select all that apply
What do you remember about this ad?
Over the past three weeks, have you seen, read or heard any Government of Canada advertising about Canadian Veterans?
Where have you seen, read or heard this Government of Canada ad about Canadian Veterans?
Select all that apply
What do you remember about this ad? What words, sounds or images come to mind?
Which sources would you rely on when looking for information about remembrance and the contributions of Canada's Veterans?
Select all that apply
In the past three weeks, have you seen or read content related to Canadian Veterans on social media sites such as Facebook, YouTube, Twitter, Instagram or LinkedIn?
In the past three weeks, what commemorative activities, if any, did you take part in to honour Canadian Veterans?
Wore a poppy
Watched a ceremony on TV or online
Took a moment of silence
Watched a talk online given by a Veteran
Had discussions with students, family, or friends
Shared posts on social media
Visited a cenotaph
Read about remembrance, military history, or Veterans
Watched a video about remembrance, military history, or Veterans
Other
Recently, various ads about Canadian Veterans aired. Click to watch.
Recently, various ads about Canadian Veterans aired. Click to watch.
And this internet banner ad was displayed
Over the past three weeks, have you seen these ads or any variation of these ads?
Where have you seen these ads?
Select all that apply
What do you think is the main point these ads are trying to get across?
Please indicate your level of agreement with the following statements about these ads.
These ads catch my attention
These ads are relevant to me
These ads are difficult to follow
These ads do not favour one political party over another
These ads talk about an important topic
These ads provide new information
These ads clearly convey that the Government of Canada wants you to remember and honour Canadian Veterans
Thinking of commemorative programming that Veterans Affairs Canada is involved with, please rate the importance of the following:
Hosting commemorative events and ceremonies in Canada
Development of commemorative educational resources for schools
Funding for commemorative community initiatives
Hosting commemorative events and ceremonies in Europe
Maintenance of military memorials in Europe
Cemetery maintenance and grave marker repair
How satisfied are you with Veterans Affairs Canada's communications about the services and benefits available to Veterans and their families?
Please describe in your own words who you consider a Veteran?
Veterans and members of the Canadian Armed Forces have served in countries around the world. To your knowledge, in what countries have the Canadian military served over the years?
What do you think is the biggest issue facing Veterans today?
And in closing, a few questions that will help us to analyze the survey results.
Which of the following categories best describes your current employment status? Are you...?
What is the highest level of formal education that you have completed?
Are there any children under the age of 18 currently living in your household?
Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
Are you a Canadian Veteran, or do you personally know a Canadian Veteran? If so ... what is your relation to the Canadian Veteran?
Where were you born?
In what year did you first move to Canada?
What is the language you first learned at home as a child and still understand?
That concludes the survey. This survey was conducted on behalf of Veterans Affairs Canada. In the coming months the report will be available from Library and Archives Canada. We thank you very much for taking the time to answer this survey. Your help is greatly appreciated.
We regret that your responses have shown that you are ineligible to participate in this survey. Thank you for your time.