Pre-testing and Evaluation of the “Make Change that
Counts: Managing Your Money in a Changing World” Ad Campaign
Summary
Prepared for the Financial Consumer Agency of Canada (FCAC)
Supplier: Leger Marketing Inc.
Contract Number: CW2240026
Contract Value: $126,206.31 (including HST)
Award Date: September 12, 2022
Delivery Date: March 29, 2023
Registration Number: POR 045-22
For more information on this report, please contact the Financial
Consumer Agency of Canada at: info@fcac-acfc.gc.ca
Ce rapport est aussi disponible en français
Pre-testing and Evaluation of the “Make Change that Counts: Managing Your Money in a Changing World” Ad Campaign
Executive Summary
Prepared for the Financial Consumer
Agency of Canada
Supplier
Name: Leger Marketing Inc.
March 2023
This public opinion research report presents the executive summary of a series of focus groups conducted pre-campaign, as well as pre- and post-campaign online surveys conducted by Leger Marketing Inc. on behalf of the Financial Consumer Agency of Canada. The focus group research was conducted with 106 Canadians between October 3rd and October 26th, 2022. The ACET pre-campaign survey was conducted with 2,000 respondents from the Canadian general public aged 18 years and older, between October 19 and October 27, 2022. The ACET post-campaign survey was conducted with 2,005 respondents from the Canadian general population aged 18 years and older, between February 6 and 16, 2023.
Cette publication
est aussi disponible en français sous le titre : Essai préliminaire et évaluation de la campagne publicitaire « Faisons des
changements qui comptent : Gérer votre argent dans un monde en évolution »
Groupe de discussion et rapport méthodologique de l’OECP
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from the Financial Consumer Agency of Canada. For more information on this report, please contact the Financial Consumer Agency of Canada at: info@fcac-acfc.gc.ca
Financial Consumer Agency of
Canada
427 Laurier Avenue West, 6th Floor
Ottawa ON K1R 1B9
Catalogue Number: FC5-83/2023E-PDF
International Standard Book Number (ISBN): 978-0-660-48170-8
Related publications (registration number: POR 045-22):
· Catalogue Number: FC5-83/2023F-PDF (Executive Summary, French)
· ISBN: 978-0-660-48172-2
© His Majesty the King in Right of Canada, as represented by the
Commissioner of the Financial Consumer Agency of Canada, 2023
Executive
Summary
Supplier: Leger Marketing Inc.
Contract Number: CW2240026
Contract Value: $126,206.31
(including HST)
Award Date: September 12, 2022
Registration
Number: POR 045-22
Introduction
Leger was contracted by the Financial
Consumer Agency of Canada (FCAC) to
conduct concept testing and campaign
evaluation using the Advertising Campaign Evaluation Tool (ACET) for the ad
campaign “Make Change that Counts: Managing Your
Money in a Changing World”. A total of twelve (12) online focus groups were
conducted from October 3rd to October 25th, 2022, with
Canadian residents 18 to 54 years old. Participants were recruited from across
Canada to take part in a two (2) hour focus group, held virtually. A total of eight
(8) groups were conducted in English and four (4) groups were conducted in
French. In addition, the
pre-campaign survey was conducted with 2,000 respondents from the Canadian
general population aged 18 years and older, between October 19 and October 27,
2022. The post-campaign survey was conducted with 2,005 respondents from the
Canadian general population aged 18 years and older, between February 6 and 16,
2023. The following summarizes the findings from the
qualitative research study and the methodological report
on findings from the ACET surveys designed to determine
the effectiveness of the advertising campaign in shifting attitudes, beliefs,
and behaviours by measuring awareness of the subject
matter with the target audiences.
Background
The
Financial Consumer Agency of Canada (FCAC) is an independent federal government
agency mandated to protect Canadian financial consumers. FCAC supervises
federally regulated financial entities, ensuring that they comply with consumer
protection measures set out in legislation, public commitments, and codes of
conduct. It also promotes financial education to strengthen the knowledge,
skills and confidence of Canadians and raise awareness of consumer rights and
responsibilities.
Strengthening
the financial literacy of Canadians is a key pillar of FCACs’ consumer
protection mandate. Canadians face an increasingly complex and digital
financial marketplace. For this reason, it is more important than ever that
Canadians have the knowledge, skills, and confidence to make informed decisions
about their personal finances. Understanding the basics about money is as
essential today as numeracy and basic literacy. Financial literacy is important
not only for the financial well-being of individuals, but also for the economy.
Canadians
have faced many economic challenges over the past couple of years - from the
unprecedented and uneven economic effects of the pandemic to the recent surge
in inflation and rising interest rates. Inflation globally and in Canada
continued to rise, largely driven by higher prices for energy and food. As of
June 2022, inflation in Canada reached 7.6% year over year (the largest
increase in decades) and, at the time, it was expected that interest rates would
continue to rise, making it more expensive for Canadians to borrow money. As
the cost-of-living increased, Canadians may have needed
to adjust their household spending to deal with rising prices and interest
rates.
Inflation
and rising interest rates left many Canadians worried about their financial
futures, particularly those on low or fixed incomes and those with high debt
burdens. According to the Bank of Canada’s Financial System Review in June 2022,
more households were carrying large mortgage debt and households with high debt
burdens are especially vulnerable if their incomes decline. They will also feel
the impact of higher interest rates on their budgets more than others will.
This is especially true for those who have a variable-rate mortgage or other
variable-rate loans, like a home equity line of credit (HELOC), and those with
an upcoming mortgage renewal.
A
mortgage is the most common and significant type of debt held by Canadians.
FCAC’s COVID-19 Financial Well-Being Survey found that most Canadian households
(55%) own a home and two-thirds of Canadian homeowners (60%) have a mortgage
and about 12% have an outstanding balance on a HELOC. Other common types of
debt include balances owing on credit cards (held by 39% of Canadians), vehicle
loans or leases (27%), personal lines of credit (30%) and student loans (10%)[1].
Given
the potential of further interest rate hikes, it was important that indebted
Canadians have access to trustworthy and unbiased financial information and
tools to help them make informed decisions that will lead to positive financial
outcomes. Strengthening the financial literacy of Canadians would help decrease
their risk of vulnerability and increase their resilience in these challenging
and changing times. Financial literacy is a key contributor to financial
resilience and building the financial resilience of Canadians is the vision of
FCAC’s Make Change that Counts: National Financial Literacy Strategy 2021-2026.
On
top of personal financial challenges, Canadians faced a daunting financial
landscape. The marketplace for financial products, services, and advice is
complex, constantly changing and increasingly digital. There is a wide range of
product choices, an overwhelming amount of information available through financial
websites and financial apps, and more people and companies providing advice on
money and finances. As a result, it is more difficult to navigate the financial
marketplace, know who to trust, and how to make the best decisions for one’s
circumstances. This is especially problematic for people with limited financial
knowledge.
National
advertising campaign
FCAC’s
national multimedia campaign “Make Change that Counts: Managing Your Money in a
Changing World” aimed to strengthen the financial literacy of Canadians to help
build their financial resilience in a continually shifting economic landscape.
With Canada’s annual inflation at its highest point in over 30 years and
interest rates rising, the campaign focused on equipping indebted Canadians
with practical tips and tools to manage their money and debt in a challenging
economy and increasingly complex financial marketplace.
The campaign was in market for two flights:
1) a Search Engine Marketing campaign launched on November 1 and ran until December
15, 2022 (just over 6 weeks), and 2) a multimedia campaign, including display
banners, social media ads, and Search Engine Marketing launched January 9 and
ran until February 5, 2023 (4 weeks).
Research Purpose and Objectives
The purpose of
the qualitative research was to conduct a pre-test and disaster check of the
creative concepts/ads to guide decisions related to the selection of the most
effective creative concept and assist with the finalization of creative
elements for the ads. The purpose of the quantitative research was to conduct the
campaign evaluation using the standardized Advertising Campaign Evaluation Tool
(ACET). Pre- and post-campaign ACETs are used for all major Government of
Canada advertising campaigns to consistently evaluate campaigns and comply with
key requirements in the Government of Canada Communications Policy.
The campaign’s research objectives included four components:
1.
Qualitative pre-testing
of creative concepts: The goal was to
choose the creative concept that resonated the most with the target audiences and
performed the best in terms of meeting the campaign objectives.
2.
Qualitative disaster
check of selected creative concept: The goal was to test
changes to the selected creative concept with the target audiences and choose
the ads that resonated the most with them and performed the best in terms of
meeting the campaign objectives.
3. Quantitative pre-campaign
evaluation using the standardized ACET baseline online survey: The goal was to assess pre-campaign
awareness of the subject matter, including:
·
Aided and unaided awareness of general Government of
Canada advertisements or other advertisements related to the campaign
topic;
·
Aided and unaided awareness of the subject matter.
4. Quantitative
post-campaign evaluation using the standardized ACET post-campaign online
survey: The goal was to measure the
post-campaign awareness of the subject matter, including:
·
Aided and unaided awareness of FCAC’s ads and general Government of
Canada ads;
·
Recall of key campaign messages;
·
Whether the ads were attention grabbing, relevant, difficult to follow,
about an important topic, and provided new information;
·
Whether any action was taken as a result of seeing or hearing FCAC’s
ads;
·
Awareness of who was responsible for creating the ads.
Methodology
Pre-testing and disaster check of creative concepts (qualitative
research)
A qualitative (focus group) research approach was used to pre-test the creative concepts for final feedback and disaster check
before being released in the media. The target audience for the research included
working-aged Canadians (18 to 54) with participants recruited from the
following targets:
·
Homeowners with a mortgage
·
Individuals with any type of loan
·
Low-income (less than $40,000)
·
Youth (18-24)
A total of twelve (12) online focus groups were conducted from October 3rd
to October 25th, 2022, with Canadian residents 18 to 54 years old.
Participants were recruited from across Canada to take part in a two (2) hour
focus group, held virtually. A total of eight (8) groups were conducted in
English and four (4) groups were conducted in French. A total of one hundred and six
(106) participants took part in the focus groups and all participants were
provided with $100 as an honorarium. All participants were recruited per
established industry standards and per the recruitment specifications for the
Government of Canada. The participants were screened using online panel and
then validated and recruited by phone.
It is important to note that the focus groups were conducted in two
rounds. The first round tested concepts one (1), two (2), and three (3) (titled
Shaping Balance, Push & Pull, and Voice of Reason
respectively) and consisted of eight (8) groups, six (6) conducted in English
and two (2) conducted in French. A total of sixty-six (66) participants took
part in the first round. The second round (disaster check) tested ads four (4),
five (5), and six (6) from the selected concept (titled Financial Balance,
Financial Future, and Abstract respectively) and consisted of
four (4) groups in total, two (2) conducted in English and two (2) conducted in
French. A total of forty (40) participants took part in the second round.
This report presents the findings from these focus groups. Caution must
be exercised when interpreting the results from this research, as qualitative
research is directional only. Results cannot be attributed to the overall
population under study, with any degree of confidence.
Pre- and post-campaign ACET (quantitative research)
For the ACET, the target audience was Canadians
18 years of age and older. Data for both the pre- and post-campaign surveys was
gathered from Leger’s panel LEO. Leger ensured that the sample reflected the
key demographics of Canadians by age, gender, and region.
The pre-campaign ACET baseline survey of
Canadians aged 18+ years of age was undertaken online from October 19 to 27,
2022. It required an average of approximately four minutes to administer for
both the French and English versions. There was a total of 2,000 useable
surveys completed in the pre-campaign evaluation.
The post-campaign ACET survey of Canadians aged
18+ years of age was undertaken online from February 6 to 16, 2023. It required
an average of approximately six minutes to administer for both the French and
English versions. There was a total of 2,005 useable surveys completed in the
post-campaign evaluation.
Interpretation of ACET Research Findings
Given that the pre-
and post-campaign ACET was conducted using online survey methodology and the
sample drawn from an internet panel is a non-probability sample, the data
collected cannot be extrapolated to the Canadian general public adult
population. Respondents for this survey were selected from among those who had volunteered
or registered to participate in online surveys. The results of such surveys
cannot be described as statistically projectable to the target population. The
data has been weighted to reflect the demographic composition of the target
population.
Key Findings and Conclusions (Qualitative Research)
The pre-testing and
disaster check findings indicated that the creative concepts/ads did not
contain any hidden disasters and the overall reactions to them were positive.
Among the first three
concepts shown (Concepts 1-3), Voice of Reason (Concept 3) was the preferred
concept. Among the last three ads (Ads 4-6), Abstract (Ad 6) was the preferred
ad. However, these preferences were fairly minor, with all concepts/ads having
several positive and negative notes. And across the different demographic
segments, some differences of opinion were present among young adults (18 to
24) and French and English-speaking participants.
Overall, Canadians
want to see the Government of Canada wordmark, especially on a social media ad,
so they can know the information is coming from a trusted source. However,
while many look to the government as a trusted source of information, many also
fault the government for the rising cost of living. It is important to keep
this conundrum in mind when choosing words to express debt responsibility and
financial management. Emotions are high when it comes to discussions around
finances, even more so in today’s economic climate. Words such as habits,
resilience, responsibility are not well received by Canadians.
Keeping the ads short,
vibrant, and easy to follow will increase the likelihood that Canadians will
take action and click on the link for more information. And, if possible,
providing some information on the type of support (tips and tools) that are
being offered would be a valuable aspect to include. Financial Literacy Month
is not well known so this information does not add to the consumers knowledge
of what supports may be available.
Shaping Balance
(Concept 1)
The concept Shaping
Balance was preferred by and resonated with participants between 25 to 40
but was believed to speak to all Canadians. Younger participants (18-24) were
more likely to indicate that this concept was targeted towards older Canadians
(40+). Participants who are currently facing financial challenges, such as debt,
said they would most likely click the link to get more information. And, while
some participants felt the simplicity of the ads worked for the overall
messaging, many also felt that not enough information was provided.
When it comes to the
wording for the resilience version, the use of the term resilience
was not understood by all participants. Many indicated they did not understand
what financial resilience meant. In Quebec, the term resilience
was not appreciated because it is a word that was overused during the pandemic
and has lost its impact, i.e., people don’t want to see it anymore.
The version with the
wording balance was preferred when it came to the copy used. Balance
resonated with many participants, as they felt this was a practical and
obtainable way to manage finances and debt. However,
while participants understood the
phrase, “You don’t need to be an expert to find your financial balance”, the
accompanying imagery was not easily recognized.
Push and Pull (Concept 2)
Among the first set of concepts (Concepts 1-3), the concept Push
and Pull was the most effective in getting participants’ attention and
communicating the message. It was especially effective for those in the 18-24
age range. Participants who are currently facing financial challenges, such as
lower incomes, said they would most likely click the link to get more
information. Overall, the bright colors, text bubbles, and use of emojis was
thought to be attention grabbing. However, it was noted that this concept might
be too long to hold someone's attention, so it would need to be executed as quickly
as possible.
Some participants, especially in Quebec, said they did not understand
the link between the pictures and the message. For example, participants
mentioned that they would never go into debt for some cheese and therefore the
images and messaging was not relatable. Additionally, in Quebec, the words Tiraillements et bras de fer were seen as negative and not a good phrase
for the message being shared.
The grocery version of
the concept was felt by many to be the most relatable, with almost all indicating that they had felt the strain of
increasing grocery costs, and many indicating that cutting back on grocery
spending was an important way to save money. The movie version was perceived to
be a bit confusing, as many weren’t sure if the person in the ad ended up going
to the movies or not, while the leaking roof version was too depressing. Some
participants mentioned that they expected the ad to end on something more positive.
Voice of Reason (Concept 3)
The concept Voice of Reason had the highest
rating among the first three concepts when it came to the message being
important. The concept was specifically preferred by and resonated most with
those in the 41-54 age range. In addition, many participants in their late 30s
also found this concept most relevant to them. However, this concept did not
resonate with the younger participants (18-24), particularly those in British
Columbia. Demographically, participants with a higher household income ($100k+)
and a mortgage were more likely to find this concept appealing. Those in the
41-54 age range who were managing larger debts, such as mortgages and car loans,
indicated this concept (out of Concepts 1-3) was their most preferred.
Notably, having the
Government of Canada wordmark more prominently visible was seen as necessary by
participants. They wanted to be assured that the information was coming from a
reliable source (the government) and not a scam ad, a banking ad, or a debt
rescue ad. The personification of the credit card in the spending habits
version was somewhat lost on participants and the concept felt a little
disjointed. The manage your money version was preferred, with the
phrasing came across as empowering.
Financial
Balance (Ad 4)
The ad Financial Balance resonated with those having large debt (either a mortgage or student loan). Many participants felt that the ad was targeted towards millennials because this age group is looking for “balance” in life and are often carrying a lot of debt with minimal experience. Only a few participants believed that the message was not important, while most believed that the message was very important. Those who thought the message was unimportant indicated they did not think they needed financial literacy to manage their finances.
Quebec participants liked the combination of the messaging and the image because it clearly illustrated the struggle with finding financial balance.
Many participants in the age group 41-54 believed that they didn’t need help managing debt. Participants with higher household incomes ($80,000+) were more likely to indicate this ad resonated with them. Quebec participants generally agreed the ad was targeted towards an older audience, those who were 30+ because they were likely to be managing more financial responsibilities, such as loans and mortgages.
Overall, participants didn’t find anything negative or disturbing about the ad. However, the ad was not perceived as overly effective. Participants highlighted that they would need more details on the help available before clicking the link. Some participants also highlighted that it should be made clear at the outset that the ad is from the Government of Canada to increase its credibility.
Financial Future
(Ad 5)
Most participants felt that the ad Financial Future was more relevant for a 25+ age group. Those with large debt were more likely to react positively to this creative. Younger participants (aged 18-24) didn’t relate with the language and thought it was unappealing. Almost all participants felt that it conveyed an important message. This ad was effective and believable and specifically resonated with those who were late millennials and those in the 41-54 age group. Participants understood the messaging and appreciated the clear and simple text and images. In Quebec, Remboursez vos dettes was off-putting and did not resonate with participants. In addition, Prenez votre avenir financier en main was thought to be somewhat judgmental and indicated that there was something that needed fixing.
While younger participants felt shaping the financial future made sense in the ad, paying down debt wasn’t something that would make them click an ad to find out more. When it comes to the wording in the two different versions, participants believed that build would have made more sense if they had just started out and had no debt. Those aged 18-24 liked build more than shape for this reason. In Quebec, Bâtissez votre avenir financier was preferred. Shape was clearly more liked by English language participants. Many reasoned that shape made more sense since they already have debt, and they would need to shape it for a better future.
Abstract (Ad 6)
Overall, the message
of the ad Abstract was seen as more supportive and empathizing in
comparison to the other two ads in the set (Ads 4-6). Respondents appreciated
that this message showed that the government understands that their financial
situation is not their fault. While some Quebec participants thought the ad was
the best of the three, some thought the message was vague and as a whole, did
not make sense.
This message resonated the most with young respondents, especially 18-24 and early 30s. Those in eastern Canada (comparatively lower mortgage/debt as compared to BC) found it most appealing, especially when compared to the other ads. Among British Columbians, youth aged 18-24, who are new to managing their own finances were drawn towards this ad. Quebec participants were fairly divided when it came to the Abstract ad, with the first group expressing dislike for the ad and the second group providing much more positive feedback.
The ad was seen as having wider appeal, and most liked the term abstract. They liked the tone of the message and found it more empathizing than others. Additionally, it was felt to have wider appeal, in that you don’t have to be debt ridden to manage your finances. However, some found the ad a touch vague and felt that there may be a need for more details before expecting a reaction to the call-to-action.
Expenditure
The total project
expenditure was $126,206.31 including HST, for qualitative concept testing and the
quantitative pre- and post-campaign ACET surveys.
Political
Neutrality Certificate
I hereby certify as a representative of Leger Marketing Inc. that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications.
Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Signed
Llisa Morrow, Associate Vice-President
Leger Marketing Inc.
March 10, 2023