Prepared for Environment and Climate Change Canada
Supplier: Elemental Data Collection
Contract Number: CW2265822
Contract Value (incl. HST): $41,781.75
Contract Award Date: December 15, 2022
Delivery Date: May 2023
Registration Number: POR 097-22
For more information, please contact Environment and Climate Change Canada at: POR-ROP@ec.gc.ca
Ce rapport est aussi disponible en français.
Nature Based Climate Solutions and Cleaner Environment Advertising Campaign 2023 (ACET)
Methodology Summary
Prepared for Environment and Climate Change Canada
Supplier: Elemental Data Collection
Registration number: POR 097-22
Aussi disponible en français sur le titre: Campagne publicitaire de 2023 sur les solutions climatiques fondées sur la nature et un environnement plus propre (OECP) - Résumé de la méthodologie
Cat. No.: En4-553/2023E-PDF
ISBN: 978-0-660-49562-0
EC22057
Unless otherwise specified, you may not reproduce materials in this publication, in whole or in part, for the purposes of commercial redistribution without prior written permission from Environment and Climate Change Canada’s copyright administrator. To obtain permission to reproduce Government of Canada materials for commercial purposes, apply for Crown Copyright Clearance by contacting:
Environment and Climate Change Canada
Public Information Centre
Place Vincent Massey building
351 St-Joseph boulevard
Gatineau Quebec K1A 0H3
Toll free: 1-800-668-6767
Email: enviroinfo@ec.gc.ca
© His Majesty the King in Right of Canada, as represented by the Minister of Environment and Climate Change Canada, 2023
Environment and Climate Change Canada (ECCC) were looking to support the Nature Based Climate Solutions (NBCS) and Cleaner Environment advertising campaign with public opinion research, using the Advertising Campaign Evaluation Tool (ACET). The following sections outline the methodology used to conduct the study, including the research objectives, sample design and data collection procedures.
Findings from 2021 Polling by Nature Canada and the David Suzuki Foundation found that 40% of Canadians hold an inaccurate definition of “nature-based climate solutions”, some believing “it relates to actions like using natural products and recycling.” Despite the confusion, 71% of Canadians believe NBCS are effective in addressing climate change (regardless of how they define the term). And 74% of Canadians support nature conservation efforts even if those measures limit their use of public/private lands – so we know support is out there (Nature Canada, 2021).
The campaign hopes to clarify this misinterpretation and encourage overall support for nature- based solutions as a means of addressing Canada’s climate crisis and biodiversity loss. One tool used in order to fulfil its mandate is public opinion research.
Overall, the ACET serves to measure the success of the advertisement campaign through a before/after measure of the campaign’s objectives and awareness/recall of the advertisements, amongst Canadians, particularly the campaign’s target audience. The goal of the campaign is to familiarize the term “nature-based climate solutions” among Canadians and inform them of the role it plays in addressing the twin crisis of climate change and biodiversity loss.
Online interviews were conducted from February 8th to February 16th, 2023 (Baseline survey) and April 1st to April 11th, 2023 (Post-campaign survey) with a total sample of 3,000 Canadians, 18 years of age and older. Given the non-probabilistic nature of the sample, the data collected cannot be extrapolated to the overall population of adult Canadians. However, for a probability sample of this size, the overall results would have a margin of error of ±1.79%, 19 times out of 20.
An ACET is mandatory, as per Directives for the TBS Policy on Communication and Federal Identity, for campaigns with a media component of $1,000,000 or more. The ACET pre-wave establishes audience awareness baselines, while the post wave measures campaign recall, understanding and response to the advertisement.
The total contract value of this research was $41,781.75, including HST.
I hereby certify as a Representative of Elemental Data Collection that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.
Colin Kiviaho
Partner
Elemental Data Collection Inc.
ckiviaho@elementaldci.com
May 8, 2023.
A pre and post campaign online survey was conducted with a geographical proportionate stratified sample of 3,000 Canadians, 18 years of age and older. Given the non-probabilistic nature of the sample, there is no margin of error associated with the sample. However, for a probability sample of this size, the overall results would have a margin of error of ±1.79%, 19 times out of 20 (adjusted to take into account sample stratification). Details regarding the methodology are outlined below.
Quotas were set to ensure that each wave of the study would target approximately 50/50 splits of male and female completes proportionate to the stratified regions. The sample frame was geographically proportionate to improve the accuracy on the national results.
| Strata | Completed Interviews | Margin of Error (%) |
|---|---|---|
| Atlantic | 200 | ±6.93 |
| Quebec | 700 | ±3.70 |
| Ontario | 1150 | ±2.89 |
| MB/SK/NU | 200 | ±6.93 |
| AB/NWT | 350 | ±5.24 |
| BC/Yukon | 400 | ±4.90 |
| Canada | 3,000 | ±1.79 |
Screening questions were used to ensure that all respondents were eligible in terms of age, region and industry bias.
The online survey was administered in two waves to 3,000 (1500/wave) total respondents, from February 8th, 2023 to April 11th, 2023, using computer-assisted web interviewing (CAWI) technology. The data collection was carried out by Elemental Data Collection in Ottawa, Ontario. Online surveys were conducted in the respondent’s official language of choice. Quotas were set to ensure that each wave of the study would target approximately 50/50 splits of male and female respondents proportionate to the stratified regions. The average length of time required to complete the pre campaign survey was 6.4 minutes, and the average length of time required to complete the post campaign survey was 10.4 minutes.
The sponsorship of the study was revealed during the introduction as Government of Canada. All survey respondents were informed that participation was voluntary and that information collected was protected under the authority of privacy legislation.
In total, 44,875 Canadian survey respondents were contacted. The overall response rate for this survey was 15.18%. This is consistent with typical response rates for online panel surveys of the general public conducted over this length of field period. The table below presents the detailed information.
| Online Disposition Table | |||
|---|---|---|---|
| Total | Pre Wave | Post Wave | |
| Total Emails | 44,875 | 26,300 | 18,575 |
| Out-of-scope - Invalid | 223 | 114 | 109 |
| In-scope - Non-responding (IS) | 37,874 | 22,038 | 15,678 |
| No response | 37,380 | 22,038 | 15,342 |
| Termination | 494 | 158 | 336 |
| In-scope - Responding units (R) | 6,778 | 3,990 | 2,788 |
| Not Qualified/Screened out | 783 | 467 | 316 |
| Quota full | 2,995 | 2,023 | 972 |
| Completed Interview | 3,000 | 1,500 | 1,500 |
| Response Rate | 15.15% | 15.24% | 15.10% |
The response rates were calculated using the formula: R=R/(IS+R).
Weighting adjustments were applied to the survey data to ensure that the results were representative of the Canadian population aged 18 years of age and older. Specifically, the survey results were weighted by region, gender, and age according to the most recent Statistics Canada census of the population.
The following table presents a breakdown of actual and weighted completions by regional strata.
| Strata | Unweighted Sample Size | Weighted Sample Size |
|---|---|---|
| Atlantic | 200 | 201 |
| Quebec | 700 | 690 |
| Ontario | 1,150 | 1,160 |
| MB/SK/NU | 200 | 195 |
| AB/NWT | 350 | 334 |
| BC/Yukon | 400 | 420 |
| Total | 3,000 | 3,000 |
A non-response analysis was conducted to assess the potential for non-response bias. Non-response is the result of a unit of the sample not participating in the survey—either refusing to take part in the survey (a refusal) or not being reached during the data collection period (non-contact). Non-response results in biases in the survey sample when there are differences between respondents and non-respondents.
To undertake the analysis for this survey, the unweighted sample distribution by gender, age, household income, employment status and level of education was compared to the actual population (based on 2021 Census figures from Statistics Canada).
| Survey Sample (Unweighted) | Population (Census 2021) | % diff (+/-) | Survey Sample (weighted) | |
|---|---|---|---|---|
| 18-34 | 26.17% | 26.69% | -6.47% | 26.79% |
| 35-54 | 34.63% | 32.19% | 2.67% | 32.17% |
| 55+ | 39.20% | 41.11% | 3.82% | 41.04% |
| Male | 49.83% | 48.76% | 1.24% | 48.76% |
| Female | 50.17% | 51.24% | -1.24% | 51.24% |
| Under $20,000 | 6.93% | 4.74% | 1.98% | 6.73% |
| $20,000 to under $40,000 | 17.87% | 14.64% | -1.00% | 18.62% |
| $40,000 to under $80,000 | 33.52% | 28.41% | 0.25% | 33.54% |
| $80,000 and over | 41.68% | 52.21% | -1.22% | 41.11% |
| Employed | 58.43% | 57.10% | 4.64% | 57.70% |
| Unemployed | 5.70% | 6.56% | -2.46% | 5.75% |
| Not in labour force | 35.87% | 36.34% | -2.18% | 36.55% |
| No certificate, degree, or diploma | 3.87% | 16.15% | -9.20% | 4.00% |
| High school certificate or equivalent | 20.13% | 26.69% | -9.28% | 21.00% |
| Apprenticeship or trades certificate or diploma | 7.27% | 8.71% | -3.27% | 7.09% |
| College, CEGEP or other non-university certificate or diploma | 22.93% | 18.84% | 4.85% | 22.77% |
| University degree, certificate, or diploma | 45.80% | 29.61% | 16.90% | 45.14% |
As is typically found with online surveys in Canada, the final sample over-represents those with higher levels of education. The survey results were weighted to address these variations, as well as the sample design for the survey, which was regionally proportionate. Weighting serves to reduce bias should it be present, but not to eliminate it completely. It is very unlikely that this small sampling bias introduced any meaningful bias to the survey results.
ADVERTISING CAMPAIGN EVALUATION TOOL
POST-CAMPAIGN SURVEY
INTRODUCTION
Thank you for taking a few minutes to complete this survey on current issues that matter to Canadians. Si vous préférez répondre à ce sondage en français, veuillez cliquer sur français.
Your participation is voluntary and your responses will be kept entirely confidential. The survey takes about 12 minutes to complete.
QA. Does anyone in your household work for any of the following organizations?
a marketing research firm
a magazine or newspaper
an advertising agency or graphic design firm
a political party
a radio or television station
a public relations company
the federal or provincial government
none of these organizations
IF “NONE OF THESE ORGANIZATIONS” CONTINUE, OTHERWISE THANK AND TERMINATE.
QB. What is your gender?
Male
Female
Other
Prefer not to answer
QC. In what year were you born?
ADMISSIBLE RANGE 1900-2005
IF > 2005, THANK AND TERMINATE
ASK D IF QUESTION C IS LEFT BLANK
QD. In which of the following age categories do you belong?
less than 18 years old
18 to 24
25 to 34
35 to 44
45 to 54
55 to 64
65 or older
IF “LESS THAN 18 YEARS OLD” OR “BLANK”, THANK AND TERMINATE
QE. In which province or territory do you live?
Alberta
British Columbia
Manitoba
New Brunswick
Newfoundland and Labrador
Northwest Territories
Nova Scotia
Nunavut
Ontario
Prince Edward Island
Quebec
Saskatchewan
Yukon
IF NO PROVINCE OR TERRITORY IS SELECTED, THANK AND TERMINATE
CORE QUESTIONS
Q1. Over the past three weeks, have you seen, read or heard any advertising from the Government of Canada?
yes
no =>GO TO T1A
Q3. Think about the most recent Government of Canada ad that comes to mind. What do you remember about this ad?
[OPEN ENDED]
CAMPAIGN-SPECIFIC QUESTIONS
T1A. Over the past three weeks, have you seen, read or heard any Government of Canada advertising about climate change and the environment?
yes
no =>GO TO T1D
T1B. Where have you seen, read or heard this ad about climate change and the environment?
SELECT ALL THAT APPLY
Flyer/postcard/brochure (by mail)
Cinema
Digital billboard
Radio
Television
Digital/Streaming TV (e.g. Netflix, Disney+)
In a mobile app
Internet website
Online news sites
Snapchat
Digital/Streaming radio (e.g. Spotify, Podcast)
TikTok
Twitch
Video game
Web search (e.g. Google, Bing)
YouTube
Print newspaper
Print magazine
T1C. What do you remember about this ad?
[OPEN ENDED]
T1D. On a scale of 1 to 5, where 1 means not at all familiar and 5 means very familiar, how familiar would you say you are with the term “nature-based climate solutions”?
SELECT ONE ONLY
1 - Not at all familiar
2
3
4
5 - Very familiar
FOR THOSE WHO ARE FAMILIAR [4,5]
T1E. What is one example of a nature-based solution?
[OPEN ENDED]
T1F. Please rate your level of agreement or disagreement with the following statements. Protecting nature’s elements such as trees, wetlands, grasslands, and coastlines helps to:
[RANDOMIZE STATEMENTS]
a) reduce greenhouse gas emissions
b) reduce the impacts of climate change
c) advance reconciliation with Indigenous peoples
d) support biodiversity and habitats for wildlife
e) create green jobs in communities across the country
Strongly disagree
Somewhat disagree
Neither agree nor disagree
Somewhat agree
Strongly agree
Prefer not to say
T1G. How worried do you currently feel about the issue of climate change?
Not at all
Slightly
Moderately
Very
Extremely
Prefer not to say
T1H. Please rate your level of agreement or disagreement with the following statements
[RANDOMIZE STATEMENTS]
a) I believe that climate change is real.
b) My local area will be influenced by climate change.
c) It will be a long time before the consequences of climate change are felt.
Strongly disagree
Somewhat disagree
Neither agree nor disagree
Somewhat agree
Strongly agree
Prefer not to say
T1I. Please rate your level of agreement or disagreement with the following statements
[RANDOMIZE STATEMENTS]
a) I am willing to make substantial changes in my life to help limit climate change.
b) People who are close to me generally expect that I do my part to help limit climate change.
c) Climate change can’t be limited without governments helping.
d) Climate change can’t be limited without big corporations and industry leaders helping.
e) Governments in Canada are willing to make substantial changes to help limit climate change.
f) Big corporations and industry leaders in Canada are willing to make substantial changes to help limit climate change.
g) I feel pressure from friends and family members to do my part to help limit climate change.
Strongly disagree
Somewhat disagree
Neither agree nor disagree
Somewhat agree
Strongly agree
Prefer not to say
I don’t think the climate is changing
AD RECALL QUESTIONS
T1J. Here are some ads that have recently been broadcast on various media. Click here to watch.
[INSERT VIDEO AND SOCIAL MEDIA ADS]
[CLICK TO GO TO THE NEXT PAGE]
Over the past three weeks, have you seen, read or heard any of these ads?
yes
no =>GO TO T1L
T1K. Where have you seen, read or heard these ads?
SELECT ALL THAT APPLY
Cinema
Digital billboard
Television
Digital/Streaming video
Online banners
Digital/Streaming audio (e.g., podcasts)
Web search (e.g., Google, Bing)
YouTube
T1L. What do you think is the main point these ads are trying to get across?
[OPEN ENDED]
T1M. Please indicate your level of agreement with the following statements about these ads?
[RANDOMIZE STATEMENTS]
a) These ads catch my attention
b) These ads are relevant to me
c) These ads are difficult to follow
d) These ads do not favour one political party over another
e) These ads talk about an important topic
f) These ads provide new information
g) These ads motivate me to visit a Government of Canada website to learn more about nature-based climate solutions
h) These ads have improved my understanding about the role that nature and nature-based climate solutions play in addressing climate change and biodiversity loss
i) These ads motivate me to support conservation efforts (e.g., of wetlands, grasslands, coastlines, and forests)
Strongly disagree
Somewhat disagree
Neither agree nor disagree
Somewhat agree
Strongly agree
DEMOGRAPHIC QUESTIONS
D1. Which of the following categories best describes your current employment status? Are you...
SELECT ONE ONLY
working full-time (30 or more hours per week)
working part-time (less than 30 hours per week)
self-employed
unemployed, but looking for work
a student attending school full-time
retired
not in the workforce (Full-time homemaker or unemployed but not looking for work)
other employment status
D2. What is the highest level of formal education that you have completed?
SELECT ONE ONLY
grade 8, elementary school or less
some high school
high school diploma or equivalent
registered Apprenticeship or other trades certificate or diploma
college, CEGEP or other non-university certificate or diploma
university certificate or diploma below bachelor's level
bachelor's degree
postgraduate degree above bachelor's level
D3. Are there any children under the age of 18 currently living in your household?
yes
no
D4. Which of the following categories best describes your total annual household income, including income from all household members, before taxes are deducted?
SELECT ONE ONLY
under $20,000
between $20,000 and $40,000
between $40,000 and $60,000
between $60,000 and $80,000
between $80,000 and $100,000
between $100,000 and $150,000
between $150,000 and $200,000
$200,000 and above
prefer not to say
D5. Where were you born?
born in Canada
born outside Canada, specify the country:
ASK IF D5=BORN OUTSIDE CANADA
D6. In what year did you first move to Canada?
ADMISSIBLE RANGE: 1900-2023
D7. What is the language you first learned at home as a child and still understand?
SELECT UP TO TWO
English
French
Other language, specify:
That concludes the survey. This survey was conducted on behalf of Environment and Climate Change Canada. In the coming months the report will be available from Library and Archives Canada.
We thank you very much for taking the time to answer this survey, it is greatly appreciated.