Prepared by Narrative Research
PSPC Contract Number: 46G85-213786/001/CY
Contracted Value: $55,720.30
Contract Date: August 12, 2020
Delivery Date: October 1, 2020

POR number: 022-20
For more information, please contact: media.relations@cra-arc.gc.ca

Prepared for Canada Revenue Agency
Supplier Name: Narrative Research
October 2020

This public opinion research report presents the results of focus groups conducted by Narrative Research on behalf of the Canada Revenue Agency. The research study was done using qualitative focus groups, specifically 12 online Netfocus discussion groups. Eight groups were conducted in English, namely four groups with participants from each of Western (BC, AB, SK, MB) and Eastern (ON, NB, NS, PE, NL) Canada. Four groups were conducted in French with residents of the province of Quebec. In each location (Western, Eastern and Quebec), one group was conducted with each of four audiences: newcomers, youth 18-35 years old, seniors 60 years or older, and members from the general population. Each group included a mix of gender, age (within specifications) education, employment status and household income. Quebec sessions were conducted in French while other focus groups were conducted in English. Across the groups, 108 individuals were recruited, while 99 people participated in the sessions. Each group discussion lasted approximately two hours and participants received a cash incentive of $100 in appreciation for their time. The focus groups were conducted between September 14 and 16, 2020.

Cette publication est aussi disponible en français sous le titre:
Mise à l'essai du concept de la campagne publicitaire sur les arnaques menée par l’Agence du revenu du Canada de 2020 à 2022

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Canada Revenue Agency. For more information on this report, please contact Canada Revenue Agency at: media.relations@cra-arc.gc.ca or at:

Catalogue Number:
Rv4-143/2021E-PDF
International Standard Book Number (ISBN):
978-0-660-36709-5

Related publications (registration number: POR-022-20):
Catalogue Number: Rv4-143/2021F-PDF (Final Report, French)
ISBN: 978-0-660-36710-1

Executive Summary

Narrative Research Inc.
Contract Number: 46G85-213786/001/CY
POR Registration Number: 022-20
Contract Award Date: August 12, 2020
Contracted Cost: $55,720.30

Background and Research Methodology

There has been an increase in recent years in government-related scams, culminating this year, following the introduction of COVID-19 related emergency programs to help Canadians through the pandemic. As a result, a total of $1.8 million was lost to COVID-19 fraud over a single three-month period earlier this spring.

There has been historical collaboration between public sector organizations to increase the public’s awareness of government-related scams. Despite these initiatives, losing money to scams continues to be a reality for many Canadian residents and the type of scams continues to evolve. With this in mind, the Government of Canada is looking to develop a targeted advertising campaign, as an additional tool to raise awareness of these scams in an effort to reduce the public’s susceptibility to them. This multimedia campaign will be led by the Canada Revenue Agency (CRA) in collaboration with other departments, as a two-year initiative that will be launching in the Fall of 2020.

Building on knowledge gathered from previous years’ scam awareness campaigns, the main objective of the upcoming campaign is to increase awareness of government-related scams among three primary audiences: newcomers, seniors, and youth, as well as members of the general population. More specifically, the ad campaign will aim to:

Increase Canadians’ awareness of ways that government-related scams occur (i.e. via telephone, letter, email, text message, online refund forms and gift cards)
Increase traffic to the campaign webpage for more scam information

The media tactics, will include a mix of television, social media, Search Engine Marketing (SEM), and web banner ads. The campaign is expected to run from Fall 2020 to Spring 2022.

At this stage in the campaign development, the CRA is interested in evaluating the relative effectiveness of proposed ad concepts with the four target audiences, to ensure that the concepts will draw the attention of those vulnerable populations. The research findings will guide the final choice and development of a campaign creative concept. More specifically, research objectives include:

Determine which of four creative concepts is most effective at:
- Increasing Canadians’ awareness of government-related scams and the way they occur (e.g., via telephone, letter, email, text message, online refund forms and gift cards); and
- Eliciting interest to visit the campaign webpage for more scam information.
Assess each concept to determine if the content is clearly understood, credible and memorable; relevant and of value; appealing; and attention-grabbing.
Evaluate proposed taglines and messaging in terms of comprehension and relevance.

To achieve these objectives, a qualitative research approach was undertaken. This entailed a total of 12 online, real-time focus groups conducted from September 14 to 16, 2020 with Canadian residents from four specific audiences: youth 18-35 years old, seniors 60 years and older, newcomers, and individuals from the general population. Within each of three regions, one group was conducted with each of the four audiences. Regions included Western Canada (BC, AB, SK and MB), Eastern Canada (ON, NB, NS, PE and NL) and Quebec. Each group included a mix of gender, age (within specifications), education, employment status, and household income. The Quebec focus groups were conducted in French while other sessions were conducted in English. Group discussions each lasted approximately two hours with participants each receiving $100 in appreciation of their time. A total of 108 individuals were recruited across all 12 groups; 9 individuals withdrew resulting in 99 final participants.

All participants were recruited per the specifications for the Government of Canada. Recruitment was conducted through qualitative panels stored on Canadian servers, with follow-up calls to confirm the details provided and to ensure quotas were met.

This report presents the findings from the study. Caution must be exercised when interpreting the results from this study, as qualitative research is directional only. Results cannot be attributed to the overall population under study, with any degree of confidence.

Political Neutrality Certification

I hereby certify as a Representative of Narrative Research that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Directive on the Management of Communications. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.

Signed
Margaret Brigley, CEO & Partner | Narrative Research
Date: October 1, 2020

Key Findings and Conclusions

Findings from the 2020-2020 CRA Scams Advertising Campaign Concept Testing (POR-022-20) reveal that government-related scams are considered more and more prevalent and complex, and that members from the general public are interested in finding out more about the ways to identify scams and how properly react to those kinds of requests.

Of the four concepts tested for a public awareness campaign regarding government-related scams, ‘Question It’ and ‘Ask Yourself’ shared the strongest and widest appeal and either should be considered for further development. Both concepts present a diversity of scamming methods (e.g., text and phone) and approaches (e.g., gift card payment, sharing of personal information), although it was felt that any final concept should provide an even greater variety of methods to achieve wider appeal.

Showing realistic, yet lesser known, scam scenarios was considered informative and engaging. The more a situation was realistic and introduced lesser known elements, the more it was seen as effectively reaching those who felt they were already aware of government-related scams. That being said, the situations should be common and familiar to people to be viewed as realistic. For example, referring to a request payment by credit card or e-transfer, rather than by bitcoin or gift card, would grab attention and elicit interest of younger people and those who consider themselves technologically savvy.

The scenario presented in the ‘Question It’ concept was found to be attention-grabbing, relevant, credible, engaging and memorable. That said, the simplicity of the situations portrayed, in addition to the age of the individuals featured in the TV ads, suggest that the target audience is limited to seniors and those who are less familiar with government-related scams. As such, it was recommended to add more diversity in age and background of individuals featured to broaden the campaign’s target audience.

The concept’s message invited the public to question government-related requests, thus being empowering, despite some of the missing information. While the idea of someone’s internal voice was liked and viewed as realistic, the presence of an alter ego in the TV ad concept caused some confusion, suggesting that the approach to featuring the ‘questioning self’ should be reconsidered or adjusted to ease comprehension. The scenario should also imply that the victim’s response was informed and not only a result of their instincts. The online ad was seen as text heavy and providing limited information, while the lie detector scale should be reversed and made more vivid.

The inclusion of facts and statistics in the ‘Ask Yourself’ concept made this concept informative, credible and impactful. While it implies that the situation is pervasive, it fails to provide direction on how to address those kinds of requests. It does not imply personal responsibility but rather, establishes the scope of the issue and places the onus on government to address the situation. As such, the target audience is unclear. There are also issues with the clarity of the scenarios in the TV ad, especially difficulties identifying the individuals featured as being a scammer or victim. Showing facts and statistics on the screen was recommended for those with hearing issues or who might have the sound turned low on their television. In the online ad, consideration should be given to increase the claim’s legitimacy, so as to avoid the perception that the ad itself is a scam.

Both the ‘Question It’ and ‘Ask Yourself’ concepts failed to provide clear direction on the desirable behaviours in responding to scams, or what to expect from government communications. The use of simplistic language and imagery was also considered important to reach a wide audience, notably newcomers and those for whom English or French is not their mother tongue.

By contrast, ‘Scam-Q’ elicited mixed opinions. Although for some it stood out for the originality of the game show concept, many felt that the approach was too light-hearted to lend credibility to such a serious topic as scams. Issues were also raised with the clarity of messaging, as it is normally expected that questions on game shows typically provide a correct and incorrect choice of response and not two correct or incorrect choices as featured in the TV ad. There were also issues with the clarity of the online ad with some perceiving that the featured URLs were actual scams to watch for, rather than asking the public to avoid clicking on links received by text messages. There was an expectation that the ad would provide insight to identify fraudulent URLs. The target audience is unclear, with the online ad seen as targeting those with a smart watch, while the seventies game show featured in the TV ad was felt to have greater appeal for an older audience.

The concept referred to as ‘Confessions of a Scam Artist’ was considered the least effective and posed serious concerns. Of greatest concern, it positioned young people as scammers and the message lacked clarity. The intent of the campaign was perceived as warning scammers that they are being investigated or warning the public that scammers could be someone they know or interact with. It was also believed that the campaign could inadvertently promote scamming among young people, as a lucrative activity. Finally, the campaign focused on text and online scamming only, thus limiting its appeal.

Despite ‘Question It’ and ‘Ask Yourself’ being considered most effective to increase awareness of government-related scams and the way they occur, none of the four concepts tested effectively elicited interest in visiting the campaign webpage for more information. Knowing that the government is working to protect the public from scams is already expected, thus not providing a strong argument to visit the website. Rather, participants want to learn how they can be better prepared to identify scams and react to them. The call to action was felt to need to be more directed, establishing personal relevance and motivating action; in a sense, letting people know they are able to find information on how to be prepared. In addition to a URL, there is an expectation that a toll-free number would be provided for those who are less familiar with technology.

It was also believed that establishing the message’s credibility is paramount given the topic, and the Government of Canada identifier should be prominently featured on all campaign materials. At the same time, identifying the campaign sponsor as the Government of Canada can broaden the campaign’s appeal, and naming specific departments that are targeted by scams in the concept scenarios makes the situation more tangible and more ‘real’.

Finally, of the four URLs tested, ‘Stop-scams’ (‘Stopper-la-fraude’) and ‘Be-scam-smart’ (‘Évitez-les-arnaques’) are the preferred options and participants felt either of them should be considered for the campaign.

Introduction

There has been historical collaboration between public sector organizations to increase the public’s awareness of government-related scams. Despite these initiatives, losing money to scams continues to be a reality for a number of Canadian residents and the type of scams continues to evolve. With this in mind, the Government of Canada is looking to develop a targeted advertising campaign, as an additional tool to raise awareness of these scams in an effort to reduce the public’s susceptibility to them. This multimedia campaign will be led by the Canada Revenue Agency (CRA) in collaboration with other departments, as a two-year initiative that will be launching in the Fall of 2020.

Building on knowledge gathered from previous years’ scam awareness campaigns, the main objective of the upcoming campaign is to increase awareness of government-related scams among three primary audiences: newcomers, seniors and youth, as well as members from the general population. More specifically, the ad campaign will aim to:

The media tactics, will include a mix of television, social media, Search Engine Marketing (SEM), and web banner ads.The campaign is expected to run from fall 2020 to spring 2022.

Objectives

This report presents the findings of the research. It includes a high-level executive summary, the description of the detailed methodology used, the detailed findings of the focus group discussions, and considerations derived from the analysis of research findings. The working documents are appended to the report, including the recruitment screener (Appendix A), the moderator’s guide (Appendix B), and a description of the materials tested (Appendix C).

Research Methodology

Target Audiences

Research Approach

The study included a total of 12 online, real-time group discussions, equally divided to include three regions. All groups were conducted from September 14 to 16, 2020. The following provides a breakdown of groups based on location, language, and audience:

Each group included a mix of gender, age (within range), household income, employment status, and education level. The recruitment also considered the market’s cultural composition, with Indigenous people and non-Caucasians represented in focus groups. As the groups were conducted online and via conference call simultaneously, participants required access to a laptop or desktop computer connected to high speed Internet, as well as a dedicated telephone line.

All participants were recruited per the recruitment specifications for the Government of Canada. Recruitment was conducted through qualitative panels stored on Canadian servers, with follow-up calls to confirm the details provided, and to ensure quotas were met. Those with current or past employment in sensitive occupations were excluded from the research, in addition to those living in the household. These sectors included marketing, marketing research, public relations, advertising, media, graphic design, provincial and federal government departments or agencies, tax preparation companies and accounting firms. Individuals who participated in at least five qualitative sessions in the past five years, those who have participated in a session in the past six months, and those who have participated in group discussions or interviews on the Census, advertising or creative testing were excluded from the research.

Group discussions were held in English, with the exception of those in Quebec which were conducted in French. Each session lasted approximately two hours with participants each receiving $100 in appreciation of their time. A total of 108 participants were recruited across all 12 groups and attendance reached 99 individuals.

Presentation of Concepts

Three campaigns were presented during each session for participants’ comments. Each campaign included a web banner and a mock-up of a TV ad (in a storyboard format). To avoid any presentation bias, the campaigns were presented in different orders across groups, as follows:

Online, Real-Time Focus Groups
Region	Language	General Population	Newcomers	Seniors 60+	Youth 18-35	Total
Western	English	1	1	1	1	4
Eastern	English	1	1	1	1	4
Quebec	French	1	1	1	1	4
TOTAL # SESSIONS:	3	3	3	3	12

CAMPAIGN A – Scam-Q (TV1; TV2)
CAMPAIGN B –Confessions of a Scam Artist (TV1; TV2)
CAMPAIGN C –Question it (TV1; TV2)
CAMPAIGN D – Ask Yourself (TV1; TV2)

Context of Qualitative Research

Qualitative discussions are intended as moderator-directed, informal, non-threatening discussions with participants whose characteristics, habits, and attitudes are considered relevant to the topic of discussion. The primary benefits of individual or group qualitative discussions are that they allow for in-depth probing with qualifying participants on behavioural habits, usage patterns, perceptions, and attitudes related to the subject matter. This type of discussion allows for flexibility in exploring other areas that may be pertinent to the investigation. Qualitative research allows for more complete understanding of the segment in that the thoughts or feelings are expressed in the participants’ “own language” and at their “own levels of passion.” Qualitative techniques are used in marketing research as a means of developing insight and direction, rather than collecting quantitatively precise data or absolute measures. As such, results are directional only and cannot be projected to the overall population under study.

Detailed Analysis

Government Scam Awareness and Experience

The focus group began with a very short discussion on awareness of recent government-related scams. Familiarity with these types of scams is prevalent across audiences and locations. With few exceptions, participants consistently had either personally experienced or knew someone who had experienced calls or text messages of government-related scams.

Most spoke of regular and ongoing calls, in some instances citing that individuals were telling them that their Social Insurance Number (SIN) had been compromised, or that they owed funds to Service Canada or the Canada Revenue Agency and that charges would be laid if they did not pay those dues. Mention of RCMP arrests, gift card purchase requests, and in more rare occasions, bitcoin requests, were shared, as were an increased number of calls pertaining to Canada Emergency Response Benefit (CERB) payment / banking account requests from ‘Revenue Canada’ during the pandemic. Many spoke of phone numbers originating from outside of Canada.

Participants consistently reported that scammers were relentless in their outreach, both in terms of the frequency of their calls or texts and in the aggressive nature or tone in their communication. In addition, it was felt that many appeared to be legitimate in their approach, while always expressing a sense of urgency.

Campaign Testing

The bulk of the discussion was spent on reviewing and discussing ideas for an advertising campaign being considered by the Government of Canada. Participants were shown four different advertising campaign concepts, each including an online advertisement and mock-ups of two tv ads shown as storyboards (still images with the description of the scenario read out loud by the moderator).

The presentation of campaign concepts and storyboards within each concept were rotated across groups to avoid any order bias. Each campaign was presented and discussed one at a time. The campaign presentation was followed by a short individual exercise asking participants to rate each concept on key aspects (ability to grab attention; message clarity; memorability; trustworthiness; and ability to motivate action) and subsequently a group discussion.

Overall Reactions Across Concepts

Rotation Schedule – Campaign Presentation Order
Audience	Eastern (EN)	Western (EN)	Quebec (EN)
Newcomers	G1 – A-B-D-C (TV: 1-2)	G3 – B-C-A-D (TV: 2-1)	G2 – D-A-C-B (TV: 1-2)
Young People	G6 – C-D-B-A (TV: 2-1)	G4 – D-A-C-B (TV: 1-2)	G7 – B-C-A-D (TV: 2-1)
Seniors	G9 – B-C-A-D (TV: 2-1)	G8 – C-D-B-A (TV: 2-1)	G5 – A-B-D-C (TV: 1-2)
Gen Pop	G11 – D-A-C-B (TV: 1-2)	G12 – A-B-D-C (TV: 1-2)	G10 – C-D-B-A (TV: 2-1)

Participants felt any final campaign should clearly demonstrate the desired behaviours or actions to address scams, use simplistic language and visuals, present new information, feature diversity of scamming methods and approaches, and include a strong call to action.

While a detailed discussion on each concept is presented later in this report, a few reactions were consistent across all four concepts and are worth mentioning.

Strongest Campaign

Of the four concepts, Confessions of a Scam Artist is considered the most problematic and viewed as the least likely to be effective in raising the public’s awareness about government-related scams. Although the three other concepts were generally well received, and deemed effective overall, Ask Yourself and Question It shared the strongest and widest appeal. Results showed that modifications would be needed in each concept to enhance effectiveness.

The following sections present the detailed findings related to each of the four campaign concepts included in the testing.

Campaign Concept: Question It (Ça ne coûte rien de se questionner)

The ‘Question It’ concept is attention-grabbing and credible, but fails to engage the target audience and provide a strong call to action.

Overall Reactions, Attention and Memorability

This concept was generally well received, deemed attention-getting by some, and effectively showed the main ways individuals can get scammed (i.e., by text or telephone) and various scamming approaches (i.e., offering a refund by text or asking for gift card payment). It appeared more effective at eliciting interest among seniors and newcomers. By contrast, young people felt the ads did not stand out, presented known information and were targeted towards an older, more vulnerable, audience. As a result, the campaign appeared less memorable to younger people than to other audiences.

“If I saw the online ad, I would not glance at it. Same for the TV. I would not pay attention to it. Seems a little bit simple and plain.” – Young Adult, West

“I see myself in the pictures, but it’s just focussing on seniors. Youth get scammed too. It does not speak to young people.” – Senior, Western Canada

“RCMP asking for gift cards seems a bit farfetched. But this speaks to a wider audience than the others.” – Young Adult, Eastern Canada

In general, the TV ads were far more appealing and considered relevant and informative, than the online ad. The use of a scenario was considered engaging and realistic, while being informative. By contrast, mixed appeal was offered with the online ad, although the concept of the lie detector was considered light, original, and somewhat attention grabbing. In addition, the two-frame approach with the claim stated on the first one (‘The CRA needs to know my passport info to deposit my benefit payment’) combined with the lie detector image, positions the concept as a quiz, thus being engaging and attention-getting to some who were interested in ‘testing their skills’. That said, the online ad was generally considered text heavy and lacking vibrancy, which affected its overall appeal.

“J’aime la pub en ligne. J’aime le message. Au début c’est comme s’ils nous posaient une question à nous [dans la première image] : Est-ce que tu crois ou non? Et après, ça me dit, ben non il ne faut pas croire ce que tu vois et c’est correct d’aller te renseigner.” (I like the online ad. I like the message. At first, it is as though they are asking us a question (in the first frame): Do you believe it or not? And then, it says, well, no, don’t believe what you see and it’s ok to get information about it.) – Young Adult, Quebec

“The online ad could better grab my attention. A moving needle, and bolder colours would be better.” – Newcomer, Eastern Canada

Message, Confidence and Credibility

The campaign’s message was considered simple and clear, communicating that people should not fall for scams and encouraging them to question government-related requests for money or information. The campaign provided a good reminder for people to cast doubt on requests and to take time before responding – in essence to think twice about the ask before following through with it.

“Ça dit qu’il faut réfléchir avant de répondre à un courriel. De toujours penser : ‘est-ce que c’est vraiment le gouvernement du Canada qui va envoyer ça ou est-ce que c’est une arnaque’?” (It says to always think before answering an email. Always think, ‘Is this really the Government of Canada sending this, or is this a scam?’) – Senior, Quebec

“In the TV ad, they are telling you ‘don’t jump in with both feet and thing something over; think it over’. Trying to get people to think before they do something. Is this right? Is this wrong?” –General Public, Western Canada

“I love this campaign. It makes you question yourself, and makes you think. It gives you a chance to think.” – Newcomer, Eastern Canada

Participants appreciated that two distinct scam scenarios (by phone or by text message) were shown. They also appreciated the diversity of scamming approaches shown (a refund offer and a payment by gift cards). That being said, a few participants felt that referring to a lesser-known method (for example asking for payment using a credit card or e-transfer) may be more effective at grabbing their attention and in some instances, would be more informative for younger people and those who consider themselves aware of scamming approaches.

Although it lacks interest for some, the familiarity of the situations illustrated in the TV ad scenarios lends credibility to the message. Indeed, many shared stories of having second-guessed scamming calls or text messages they received. At the same time, clearly identifying the Government of Canada as the TV ad sponsor helps create a sense of trust in the information that is conveyed.

Despite the TV ad being well received, a few participants identified issues affecting the scenario’s credibility. Specifically, it was believed that the older woman’s calm reaction after she hung up was considered unrealistic and there was an expectation that this person would be more visibly ‘shaken’ as a result of having received this call. At the same time, some felt that the message may be perceived as inviting people to trust their instincts (i.e., listening to their internal voice) rather than rely on sound advice.

“On sait que les personnes âgées et leurs alter égos peuvent faire des erreurs. [La campagne dit que si] vous recevez un appel ou un texto, posez-vous des questions et j’espère que vous trouverez les bonnes réponses. Il manque une séquence pour leur dire rappeler-vous le site Web ou la brochure que vous avez reçu.” (We know that older people and their alter egos can make mistakes. The campaign says, if you get a call or text message, ask yourself questions and we hope that you will find the right answers. There is something missing to remind them of the website or the brochure they may have received.) – Newcomer, Quebec

“Tell me what the CRA WILL do. Show me real emotion and expression. This doesn’t show the anxiety people go through.” – Senior, Eastern Canada

For the online ad, the lack of clear government identification (i.e., logo) affected the message’s credibility, casting doubts on the trustworthiness of the source. A suggestion was also made to expand the list of personal information that should not be shared in these kinds of circumstances (e.g., SIN, bank account number) to make the online ad more informative.

Relevance and Appeal

Many felt the campaign implied that seniors are most at risk and most often targeted by scams. While there was an appreciation that seniors are especially vulnerable, all age groups were deemed impacted by such scams and a few could see themselves in the imagery. Many, however, felt that the simplicity and familiarity of the scamming scenarios and the people shown in the TV ads pointed exclusively to an older audience, or one who is unaware of common government-related scams.

“It’s for people who are less tech savvy or those at risk of not knowing what kinds of signs to look for. Individuals who may not understand the medium or devices used by scammers.” – Young Adult, Western Canada

“Ça ne m’attire pas vraiment. De recevoir des courriels et des messages textes, tout le monde le sait que le gouvernement n’envoie pas de message texte.” (It does not appeal to me. Everyone knows that the government does not send text messages.) – Young Adult, Quebec

“They did not apply to me. I am very aware of things like this and I would hope that I would be able to spot a scam. I am aware of the ones out there.” –General Public, Western Canada

“It’s targeted towards seniors but also younger who think of what could happen to their grandma.” –General Public, Western Canada

Clarity

Mixed opinions were offered regarding the clarity of the message. While some felt that the TV ad clearly conveyed the need to question ourselves when receiving government-related calls or text message requests, others felt that the scenario involving the alter ego took away from the message. Some found the TV idea (namely the two personalities) difficult to grasp in the storyboard format – in some instances they could be perceived as twins rather than an illustration of someone’s internal voice. Others, however, felt that it reflected a realistic situation by showing people’s internal dialogue that is often happening in difficult decision-making situations.

“J’ai aimé qu’ils font référence à notre petite voix intérieure; souvent dans la vie on fait référence à notre petite voix intérieure.” (I liked that they referenced our internal voice. In life, we often refer to our internal voice.) – Young Adult, Quebec

“Having the ‘alternate’ you (like a good angel, bad angel) works, but showing the two people could be a bit confusing.” – Newcomer, Western Canada

While the online ad was generally considered clear and easy to understand, it was mentioned a few times that the lie detector scale should show the negative statement on the left and the positive on the right, as this is the direction people are used to (i.e., from negative on the left to positive on the right).

In addition, a few participants felt strongly that the use of the term ‘go ahead’ in the online ad could be problematic for multicultural audiences, given that the phrase could be misunderstood and taken literally in its translation (i.e. move forward). Accordingly, it was suggested that a more affirmative or direct command – or simply inclusion of ‘Question It’ would avoid confusion.

Intended Call to Action

The use of the two personas (the person and their alter ego) was generally considered effective at encouraging people, notably seniors, to think before responding. While participants found the approach friendly and engaging, some felt it did not go far enough, in that clear action should be provided on what to do when receiving these kinds of text messages or phone calls. Although the TV ad scenario asks people to question the situation, it does not clearly articulate what to expect of government communications (e.g., ‘the government will never send you text messages’).

At the same time, the scenario demonstrated a conclusive action (i.e., hang up or don’t click on the link in the text message and do not fall for the scam), thus lacking a compelling reason to follow-up for more information online. Further, the concept does not convey a sense of urgency in learning more.

“…je sais c’est quoi déjà [les arnaques] et je ne m’attends pas à en apprendre plus [sur le site Web].” (I already know about scams and I don’t expect to learn more about it on the website.) – Newcomer, Quebec

Finally, a suggestion was made to further highlight that the individual in the TV ads took the right action by showing the alter ego giving a ‘high five’ or a ‘thumbs up’ to the main character. It was felt that this would provide a stronger positive conclusion that clearly states that the right action was taken in response to the scam requests. In addition, some felt that adding visual cues on screen would help to articulate the intended conclusion.

“The online ad is more straightforward and uses strong verbiage. You need to have visual cues on the screen for the TV – seniors can’t hear well.” – General Public, Eastern Canada

Campaign Concept: Ask Yourself (Interrogez-vous)

The ‘Ask Yourself’ concept is informative and credible and successfully positions government-related scams as prevalent, though it fails to effectively engage the target audience and provides a weak call to action.

Overall Reactions, Attention and Memorability

This concept was considered simple, straightforward, serious and direct. Participants appreciated the informational content in this idea and the inclusion of both phone and text examples. In addition, they felt that the provision of facts and statistics helps to grab attention and position the severity of the problem, as well as making it more tangible. Others, however, felt the concept was too simplistic and did not provide a conclusion to the presented problem (i.e., how to respond to requests for payment or information by phone or text message).

“This approach depends on the individual to present some type of response – it’s not as direct.” – Newcomer, Western Canada

The statistics initially grabbed attention primarily because they demonstrated the severity of a problem. That said, participants consistently suggested that including some statistics written on the screen, in addition to being mentioned in the voiceover, would enhance memorability, and ensure that the ad would still have an impact with those with hearing issues, or with the sound turned low. Others felt that including statistics of more personal relevance would create a more compelling concept.

“It’s more attention getting (than other concepts), but you need to show stats that people can relate to. Like the number of seniors that are hit by scams...” – General Public, Eastern Canada

Message, Confidence and Credibility

It was believed that this idea clearly educates on the seriousness of scams and encourages the viewer to question situations that may appear to be a scam. It was also felt that the concept clearly demonstrates the prevalence of the issue, thus suggesting that it can hit closer to home than one might think, with the campaign acting as a warning or reminder to be on the lookout. To some, the concept focuses more on the government’s ability to protect people from scams, than to stress the importance for people to be careful in those situations.

“D’après ces publicités ils me disent d’aller vérifier. Qu’il y a une ressource ou apprendre à reconnaitre les arnaques. C’est comme ça que je perçois le message. Le gouvernement est là pour vous assister pour identifier les arnaques.” (These ads tell me to check it out. That there is a resource where to learn to identify scams. That is how I see the message. The government is there to help you identify scams.) – Newcomer, Quebec

While some appreciated the simplistic design of the TV ads with minimal visuals, others felt it lacked personal relevance. Further, a few older participants did not see the visuals as personally relevant to them given that they were not accustomed to smart phones.

Relevance and Appeal

The provision of statistics was considered powerful and impactful, but participants consistently felt it lacked personal relevance, and direct action. Rather than leaving the viewer to ‘figure it out’, participants believed a clear conclusion would be helpful, as mentioned below. In general, it was believed that the ads do not clearly identify a target audience and thus the message could be appealing to a broad diversity of people. At the same time, many did not see themselves personally in the ad.

“I like things that are more direct, that I don’t need to figure out. This is saying you need to figure it out.” – Senior, Eastern Canada

That said, the simplicity and commonness of the scenarios (i.e., ‘if you get a call requesting money’ and ‘if you get a text requesting personal information’) did not provide a compelling argument to those who are already aware that government-related scams exist. As a result, it was often unclear to participants who the campaign was speaking to.

Clarity

Some confusion was evident in the first TV concept, whereby a number of participants questioned if the black male was the scammer or the victim. In a few instances, this resulted in discussion on racial discrimination and stereotyping, with participants underscoring the importance of removing any such potential confusion, particularly given current racial tensions across the country. There was also some confusion with the significance of showing a victim answering the call, but not showing this person’s reactions to the scam.

“You need to show words on the screen – flashing ‘beware of scams’ or ‘scam alert’. That keeps it clear and simple.” – Senior, Western Canada

“The TV ads both end with the person confused. That left me confused. You need an action at the end that is clear.” – Young Adult, Eastern Canada

The online ad was clear as presented, however some suggested the text could be further simplified and made more concise. It also lacked clarity in what someone would learn online and why they should be visiting the website. The invitation to ‘learn more’ left some questioning the need to go online (‘learn more about what?’).

“I’m confused. This is making me do extra work. I’m looking for guidance. Why put out a question is this a scam?’ Just tell me – yes it’s a scam!” – Newcomer, Eastern Canada

Intended Call to Action

To some the concept lacked clarity by not providing direction on what someone should do when they receive such a request. Participants want clear guidance on what direction they should take and felt the concept lacked a strong call to action. There needs to be a clear and compelling reason provided for people to go on the website.

Many felt the online ad was vague in its intended outcome and felt it was oddly ironic that the ad indicates ‘before you click, ask yourself if it’s a scam…’ and then asks the viewer to click on a link / go to a URL. Participants spoke openly of their hesitancy in clicking links for information and generally felt they would not click on a link or visit a site for more information. The absence of a Government of Canada logo also weakened the source’s trustworthiness.

“I would hesitate to go on that link. How do I know it’s not a scam!?” – Senior, Eastern Canada

Campaign Concept: SCAM-Q (Déjouer les fraudeurs)

Despite standing out for its original approach, the ‘Scam-Q’ concept poses serious issues in terms of message credibility and clarity.

Overall Reactions, Attention and Memorability

Mixed reactions were offered regarding this concept although it presents some significant challenges. Positives included that the scenario was built to tell a story, thus being engaging to some and sustaining interest. This concept was considered entertaining and not typical of a government ad which might make it more memorable in the minds of the audience. To some, particularly young people, the game show approach stood out and grabbed their attention. Others, however, believed it lacked sufficient originality to be memorable or felt that the scenario’s light-hearted tone did not align with the seriousness of the situation. Numerous respondents felt that given the gravity of the topic, and the significant financial loss to those impacted, this approach made light of a very serious topic.

“Je m’interroge à savoir pourquoi on ferait un jeu quand ce n’est pas un jeu alors que les gens se font arnaquer.” (I question why we are making it a game when it is not a game and people are getting scammed.) – General Public, Quebec

“It is too cheesy. It is not memorable if the answer is ‘neither’.” – General Public, Western Canada.

“This sounds like entertainment and looks like a joke. Showing it in a fun way is wrong.” – Newcomer, Eastern Canada

Message, Confidence and Credibility

While the general intent of the campaign was considered to be warning the public about government-related scams and the possible scamming methods, the message lacked credibility in large part due to the use of a game show theme in the TV ad which does not align with the seriousness of the topic. Some felt that in a sense, it appears to make fun of the issue.

“Les seules personnes qui peuvent nous donner l’heure juste c’est les gens du gouvernement. Pas un panel de gens, de monsieur et madame tout le monde. Ce n’est pas crédible le jeu genre Bob Barker.” (The only people who can provide us with the right information are people from the government. Not a panel of everyday people. The Bob Barker type of game is not credible.) – Senior, Quebec

“The premise of this is wrong. How does one person know more than the other people? This trivializes the issue.” – Newcomer, Western Canada.

“There is no credibility in this approach, when you try to guess what’s right you just pick one. It should be clear- this is wrong, and show what is right.” – Senior, Eastern Canada

It was also believed that the scenario referred to (i.e., a request for gift card or bitcoin payment) is a well-recognized scam thus the message lacking impact, especially among those who consider themselves familiar with government-related scams. As such, it was suggested to use the example of e-transfer rather than bitcoin.

"… de dire que les cartes cadeaux ou les bitcoins ne sont pas des modes de paiement pour payer des impôts, il faudrait… ils devraient utiliser un exemple plus commun, plus fréquent." (Saying that gift cards or bitcoins are not payment methods for taxes… they should use a more common or frequent example.) – Newcomer, Quebec

It was also believed that the TV ad did not provide sufficient information, specifically in specifying what communication or payment methods were used by the Government (in essence, what is the ‘correct’ response to the game show questions).

“Ils donnent la réponse mais sans explication. Quelle est la bonne réponse? Et il faut expliquer aux gens ce qu’ils doivent faire ou pas et ne pas renvoyer les gens sur un site. Il faut donner un minimum d’explications dans la publicité.” (They give the response but they do not provide an explanation. What is the correct answer? And they need to explain what people should or shouldn’t do and not direct people on a website. They need to provide a minimum of information in the advertisement.) – Senior, Quebec

While participants appreciated that the online ad specifically identified fraudulent URLs (especially with the inclusion of a ‘gc.ca’ URL), some took the message literally. More specifically, it was felt that showing two URLs implies that these are the two most common fraudulent URLs and thus people should be on the lookout for those specific website addresses.

At the same time, while the online ad invited people to question the legitimacy of URLs that appear to be government-related, it did not provide clues to help them identify how to spot a fake one. Many would not trust the information given the lack of clear ad sponsor identification.

“Dans la publicité en ligne, ils devraient indiquer pourquoi les hyperliens sont frauduleux – comment les identifier.” (In the online ad, they should indicate why the URLs are fraudulent; how to identify that.) – Senior, Quebec

When considering the online ad design, a number of participants suggested that greater colour contrast was needed between the two images to clearly highlight that the content is different and warrants attention. It was also suggested to put a large red ‘X’ on both watches in the second frame, to further illustrate the message.

Relevance and Appeal

In terms of personal relevance, those who felt well aware of government-related scams considered the information conveyed in the campaign too simplistic to effectively capture their attention. Indeed, those people indicated being already aware that the government does not communicate via email or text messaging and that it would not request gift cards or bitcoins as payment methods.

“A game show removes the seriousness and the urgency of the message. This is silly.” – Newcomer, Eastern Canada.

“I don’t like the government using a game show to get a message out. People are trying to win money (on a game show) – not the place for government messaging.” – General Public, Eastern Canada.

As such, the target audience was generally identified as people who are not technologically savvy or who are less familiar with the occurrence of such scams. At the same time, many also felt that most seniors, who they believed are most at risk of being scammed, would not know what bitcoin was. It was believed that this audience would also be less familiar with new technology, such as a smart watch shown in the online ad, thus suggesting a disconnect between the intended target audience and the campaign message.

“I thought it was interesting. But directed at people who are less tech savvy. But engaging with the game show genres and follows script that makes sense.” – Young Adult, Western Canada

The game show approach was also considered more relevant to an older audience, given the lack of popularity for the genre within the past decade. Nonetheless, a few participants still felt that the concept could appeal to everyone, regardless of age. Similarly, some newcomers were unfamiliar with the game show concept.

It should be noted that while reactions to the uniqueness and fun creative concept were generally more positive among young adults, they did not consider themselves to be the campaign’s target audience primarily due to the oversimplified message.

“Je trouve ça intéressant le jeu questionnaire. C’est original par rapport à quelque chose de comme d’habitude.” (I find the game show approach interesting; it is original compared to what we usually see.) – Young Adult, Quebec

Clarity

Issues were noted regarding clarity of messaging. While the TV concept identifies the scamming methods (i.e., text, email, gift card, bitcoin), it does not clearly inform the public about what to expect of government communications. At the same time, there was an expectation that if offered two response choices in any game show, at least one of the two is correct. As such, there is a fear that viewers would spontaneously try to identify the correct response when watching the ads, thus causing confusion or lessening the impact of the message when being told that both responses are either correct or incorrect.

“Si j’écoute la question de l’animateur ça laisse croire que l’une des deux réponses est vraie. Donc si je ne vois pas la fin de l’annonce, je pars avec l’idée qu’il y a une des deux réponses qui est correcte. Ça porte à confusion.” (Listening to the show host, I am left with the impression that one of the two responses is true. So, if I don’t see the end of the ad, I leave with the impression that one of the two is correct.) – General Public, Quebec

“It tells you the message will not come from CRA, but it doesn’t show the correct method.” – General Public, Eastern Canada.

“Saying ‘which is real’ suggests that one is right. If they both are wrong it should add a big red X on both, so that it is clear.” – Young Adult, Eastern Canada.

The online advertisement failed to effectively communicate that the government never sends text communications and participants were left with the impression that the intent of this ad was to invite them to be vigilant when accessing a government website through a link. In many instances, participants were left with more questions as a result of seeing the online ad without clear direction about where to find the answers.

Intended Call to Action

Despite the information in the advertisements, the concept lacked a strong call to action, with very few participants across groups expressing a desire to visit the website for more information. For some, the game show approach in the TV ads provided a response to the questions asked and thus the concept did not motivate viewers to find out more online. It was also perceived as lacking a sense of urgency.

“La publicité nous dit qu’est-ce qui est frauduleux et qu’est-ce qui n’est pas frauduleux. On apprend tout de suite et on n’a pas besoin d’aller voir plus loin sur le site Web. Ça n’a rien d’urgent quand on voit quelque chose de ludique.” (The advertisement tells us what is fraudulent and what is not. We learn about it right away and we don’t need to go on the website for more. There is nothing urgent when we see something that is playful.) – Newcomer, Quebec

Young adults and those who believe they are well aware of government-related scams again failed to see what they would gain by visiting the website. A number of participants disliked the URL mentioned in the TV ads and felt that it was meaningless, did not provide a strong call to action and thus would not be memorable.

“I would not do too much after seeing this. For myself I would know this is a scam more than I would need to check it out. And the website I would forget it. The actual URL is not memorable.” – Young Adult, Western Canada

At the same time, the online ad lacked a clear invitation to find out more and did not provide direction as to where to find more information. Very few participants expressed an interest to go online to find out how to spot fake URLs after seeing the online ad.

Campaign Concept: Confessions of a Scam Artist (Confessions d’un fraudeur)

The concept ‘Confessions of a Scam Artist’ was seen as posing serious issues in stereotyping young people as scammers, as well as lacking clarity of messaging.

Overall Reactions, Attention and Memorability

This concept was the least preferred and most unclear, in addition to eliciting strong negative reactions in some instances. The concept was seen as causing potential issues, notably positioning young people as being scammers, or even encouraging them to scam or learn how to scam, given its apparent lucrative nature.

For most, the intent of the campaign is unclear, with most not understanding the benefit of letting the public know that scammers can be ordinary people. The message confusion contributed to the concept lacking attention and memorability. Some, however, considered the concept to be attention-getting.

“It’s says anyone could be sending the message. It is simple and very direct.” – Newcomer, Western Canada

Message, Confidence and Credibility

The message generally caused confusion and participants did not believe that seeing the scammers in action contributed to effectively warning the public about the risks of being scammed. Consistently, one message that came through was that there are lots of scammers out there or that scammers can be ordinary Canadians and not necessarily large organized groups or from overseas. While in some instances this implied to participants that the issue is ‘closer to home’ than one might think, it did not provide insight on the need for protection. In a few instances, participants also felt that the campaign was aimed at warning fraudsters that the government is on the hunt for them.

“Dans le fond [ils disent que] n’importe qui peut faire ça. L’habit ne fait pas le moine; fait attention à ton voisin.” (In essence they say that anyone can do it. Don’t judge a book by its cover. Be weary of your neighbour.) – Newcomer, Quebec

“Les fraudeurs savent que le gouvernement les tient à l’œil. Donc c’est un ‘warning’ que les fraudeurs doivent faire attention. Et aussi aviser que n’importe quelle personne peut se transformer en fraudeur.” (Scammers know the government is watching them so this is a warning to scammers to be careful. And also, to let people know that anyone can become a scammer.) – Newcomer, Quebec

“This message is almost deceiving. It says it’s easy to run a scam – I think it might encourage people to learn how to scam people – they’d google to see how you would be able to do that.” – General Public, Eastern Canada

At the same time, the concept appeared to position young people as scammers, thus potentially stigmatizing this generation. This was seen as offensive to a number of younger participants. In a few instances, it was even believed that the underlying message was that it is easy to scam people, thus perhaps indirectly encouraging such behaviours. This concept evoked a strong emotional response for some – namely anger towards a younger, techno-savvy generation, or being offended for stereotyping younger people.

“The concept for this ad is a bit risky. No matter if you change the character – who you are painting out to be the scammer can be reinforcing stereotypes. The point is not how to protect yourself from scams so much as anyone can be a scammer.” – Young Adult, Western Canada

“It’s eye opening to see who’s on the other end.” – General Public, Eastern Canada

“Cette campagne-là me choque. Ça cible des gens. Je ne pense pas que le but est de botter ceux qui font ça. On leur donne la première image. Il y a déjà assez de préjugés dans la société. Je serais contre ces publicités-là. Je vois juste que c’est eux qu’on accuse.” (This campaign is shocking. It targets people. I don’t think the goal is to kick those who do that. To put them front and centre. There are enough prejudices in society. I would be against those advertisements. I can only see those people being accused.) – Member from the General Public, Quebec

A few participants criticized this concept for implying that scams are conducted by single individuals rather than organized groups. In fact, most indicated that their experience has been that calls are from overseas call centres, thus implying more structured scheming. As a result, the campaign lacked credibility.

“Ça ne me semble pas être le genre ou les arnaques typiques donc ça manque de crédibilité. Ce n’est pas le profil typique des arnaqueurs.” (It does not look like a typical scam so it lacks credibility. It is not the profile of typical scammers.) – Member from the General Public, Quebec

“You just can’t trust people like you used to.” – Senior, Eastern Canada

“This idea is not at all realistic. More often a scammer is a robust, big organization – more organized crime and a call centre setting. It needs to look darker and be more reflective of reality.” – Newcomer, Western Canada

Some also felt that the online ad lacked credibility, as the scammed amount reported (i.e., $700,000) appears unlikely and too large to reflect a realistic situation. Indeed, it appeared unlikely that a young person would achieve this kind of results on their own.

“C’est infantiliser les gens avec ce message-là. Qu’il ait fraudé pour 700 000 $ ce n’est pas crédible. Qui il a fraudé? Ce n’est pas crédible.” (The message is infantilizing people. It is not credible that he scammed from $700,000. Who did he scam? It’s not credible.) – Newcomer, Quebec

It was also mentioned that in the online ad, the young man appears to be bragging about having scammed many people, thus presenting scamming as a lucrative activity, and one that is encouraged.

“On est en train de faire la promotion des hackers. Pour encourager les geeks, les jeunes à prendre des cours de programmation parce que vous allez réussir à faire ces fraudes-là. Avec le sourire mesquin du jeune ça porte à confusion.” (They are promoting hackers. Encouraging geeks, young people, to learn how to program in order to achieve this type of scam. With the young man smirking, it is confusing.) – General Public, Quebec

“This is suggesting to youth that someone like them could do it!” – Young Adult, Eastern Canada

Finally, the concept was limiting in the type of actions shown, as it only illustrated scams via text rather than by phone.

Relevance and Appeal

The target audience was considered to be exclusively people who own a mobile phone and those who use text messaging, thus most often pointing to younger people. That said, the message itself did not provide clues about the target audience, other than perhaps speaking to people who are generally unaware that text messaging scamming exists. Given the media attention to this issue in the past few years, this was believed to be an extremely small portion of the population.

“I don’t feel it is directed at me but more to people that don’t realize that people scam people. For people who don’t know there are scam text messages.” – Young Adult, Western Canada

A few, however, were unclear regarding the intent of the campaign, and thus were less certain who the target audience might be.

“Est-ce qu’on s’adresse aux arnaqueurs? Est-ce qu’on s’adresse aux enfants? Ce n’est pas clair à qui ça s’adresse.” (Are they targeting scammers? Are they targeting kids? It’s unclear who they are speaking to.) – Newcomer, Quebec

Clarity

With the intent of the campaign clearly lacking clarity, as discussed above, there were few other areas of the concept that were identified as being problematic. It should be noted, however, that a few seniors were unsure if the young man/young woman in the TV ads portrayed the scammer or the victim.

Intended Call to Action

Perhaps given the message confusion, the campaign was viewed as lacking a clear call to action and the scenarios did not create a desire to obtain more information despite the invitation to visit the website.

It should be noted that the message conveyed in the online advertisement also instilled a feeling of uneasiness and uncertainty among some people who as a result would not visit the website or click on the link for fear that it would be fraudulent.

“Moi personnellement en lisant ça je ne cliquerais pas là-dessus. Ça a l’air trop frauduleux et déjà que c’est en ligne je serais du genre à ne pas cliquer dessus.” (I would not click on the link after seeing this. It looks fraudulent and since it is online, I would not click on it.) – Young Adult, Quebec

“I am not one to click on anything – and I certainly wouldn’t click where this ad suggested.” – General Public, Eastern Canada

Advertisement Sponsor

Identifying the Government of Canada was considered important to establish credibility and broaden the campaign’s appeal, however naming specific departments would make the message more tangible.

The sponsor for each campaign shown was identified as either the Government of Canada or a specific department that is impacted by the scam (e.g., CRA and RCMP). Participants were asked to indicate who they believed should be listed as the campaign sponsor, understanding that the goal of the campaign is to warn the public about these specific types of scams.

By far, participants across groups and locations believed that the Government of Canada should be identified in advertisements to clearly state that the scams could involve multiple departments. It was also believed that there is greater awareness of the Government of Canada than of specific departments, thus helping to establish the message’s credibility.

“The public is more familiar with the CRA scams but they need to make it clear that it could be other departments.” – General Public, Western Canada

“Government of Canada is simple, comprehensive and covers everything – it’s most credible.” – General Public, Eastern Canada

“The Government of Canada is the larger entity. The message needs to be coming from the most powerful body.” – Senior, Eastern Canada

“I’ve become used to hearing ‘message from the Government of Canada’. It’s reassuring to hear that.” – Newcomer, Western Canada

That being said, a few participants suggested mentioning both the specific department and the Government of Canada in campaign materials, for added clarity. This was seen as a good idea since some people may associate scams with specific departments (e.g., CRA) while not realizing that it can happen elsewhere within the federal government. At the same time, naming a department makes the message more specific and tangible thus implying that the threat is ‘more real’.

By contrast, a few participants felt that it would be short sighted to position a specific department as the campaign sponsor, given that some – most notably the CRA – have been clearly positioned as having been hacked and victimized by scams recently. Further, among newcomers, it was felt that there may be a lack of familiarity with various government departments, thus causing confusion if they are mentioned.

“The scammers use CRA. There are more opportunities for fraud with that, so I think people would tune out if they heard from that specific department.” – General Public, Eastern Canada

“It needs to be the Government of Canada. Scams can come from anywhere, and a lot of scams are happening to newcomers. They may not know who Canada Revenue is, but they will know Government of Canada.” – Newcomer, Western Canada

URLs

‘Stop-scams’ (‘Stopper-la-fraude) and ‘Be-scam-smart’ (‘Évitez-les-arnaques’) are the preferred URLs.

Following the presentation of all campaigns, participants were asked to select which of four URLs would best capture the public’s attention regarding government-related scams. Participants were informed that the URLs could be interchangeable across concepts, and thus the four options were presented on a single screen, independent of any campaign.

The URL ‘Stop-scams’ (Stopper-la-fraude) was among the preferred version across audiences and participant language, followed by ‘Be scam smart’ (Évitez-les-arnaques) and to a lesser extent, ‘Is this a scam’ (Est-ce-une-arnaque). ‘Slam-the-scam’ (À-bas-l’arnaque) trailed far behind as the least preferred option, with none of the French-speaking participants, and only a few English-speaking participants, expressing a preference for this option. Participants generally indicated that their preference was based on brevity, memorability, and clarity.

French-speaking participants also look for a URL that does not include punctuations or special characters as it is easier to remember. Many also preferred the use of the term “fraudes” rather than “arnaques” for being more familiar.

“L’arnaque, ça a une tournure française, tandis que fraude ça peut être n’importe quoi, plus ce qu’on vit ici.” (The word ‘arnaque’ is more of a French expression while ‘fraude’ can be anything. It’s more what we live here.) – Young Adult, Quebec

“C’est la plus simple. ‘À bas l’arnaque’, il faut l’apostrophe; ‘Évitez’, il y a l’accent. Quand on rentre un URL c’est plus simple de mettre des lettres simples.” (It is the simplest. ‘À bas l’arnaque’, you need the apostrophe; ‘Évitez’, there is an accent. When we type in a URL, it is simpler to just have letters.) – Young Adult, Quebec

Due to time constraints, extensive discussion was not held on the URLs, although some comments were captured in select groups, as outlined below:

Canada.ca/Stop-Scams (Stopper-la-fraude)

This URL was considered short, simple and easy to remember. The word ‘stop’ was considered a strong reminder of the campaign’s intent and participants appreciated its clarity in messaging. Some in the English-speaking groups, however, felt that this tagline suggested an unrealistic goal – namely stopping scams, since they did not believe that scams will likely ever be completely stopped. Further, while it was appreciated that efforts could be made to reduce the prevalence or impact of scams across the general public, it was not deemed realistic that the government could actually stop scams.

“This is straightforward and most powerful. It reinforces a sense of pause.” – General Public, Eastern Canada

“This is too generalized. You can’t stop scams. It’s unrealistic and too much of a dream world!” – Senior, Eastern Canada

Rather, it was felt that the primary objective of the campaign was public education on the presence and threat of scams. While the intent was clearly perceived to be alert and aware that scams exist, it was not necessarily considered to be how to stop scams.

“You can’t stop scams, but you can avoid them. Maybe it should say -avoid scams?” – Newcomer, Eastern Canada

Canada.ca/Be-Scam-Smart (Évitez-les-arnaques)

This URL was generally considered clear, concise and highlighted the need to be aware of scams and consider your actions. The tagline was seen as educational, suggesting alertness awareness, and strongly aligning with the campaign’s perceived intent. It was also seen as an invitation to seek advice online, to be better prepared to face those kinds of scams.

“Ça donne l’idée que [le site Web] va me donner des conseils pour déjouer les arnaques.” (It suggests that the website will provide advice on how to address scams.) – Member from the General Public, Quebec

“This gives power to the people and is more educational.” – Newcomer, Western Canada

“This is plain, simple and to the point. It’s calling on us to be responsible and to educate ourselves!” – Senior, Eastern Canada

One participant criticized the suggestion that people needed to be smart with scams.

“There should be nothing smart about scams. This makes it sound like those running scams are smart.” – Newcomer, Eastern Canada

Canada.ca/Is-This-A-Scam (Est-ce-une-arnaque)

This tagline was considered straightforward, perky and asking people to question whether or not a circumstance is a scam. This tagline was personal in its approach and implied that the URL would lead to a resource that would provide clear information on what is or is not a scam, as well as how to react. A number of participants felt the length of the URL could be problematic given that dashes or a question mark can not be included in a URL.

“This suggests to me that if I’m going to the website, there is a way that I will be able to see what is a scam or not.” – Newcomer, Eastern Canada

Canada.ca/Slam-the-Scam (À-bas-l’arnaque)

Although catchy and memorable, this URL was considered retaliatory in nature, unprofessional in tone, and atypical of a government tagline. The URL caused confusion, as many were unsure of the significance of the word ‘slam’ in this context. Of note, the word slam was considered problematic by a few newcomers or to those whose first language is not English.

“Slam the scam, what […]is that supposed to mean? When I read that I did not know what they were talking about. I don’t really understand.” – General Public, Western Canada

“It’s indirect and unclear – what does it want me to do? It might appeal more to a younger audience.” General Public – Eastern Canada

Conclusions and Direction

The following provides conclusions derived from the detailed analysis of findings and the resulting recommendations to address the study objectives.

Appendix A: Recruitment Screener

Name:______________________________________________________________________________
Home phone: ________________ Work phone:___________________ Cell: ____________________
Email: ______________________________________________________________________________
Community: __________________________________ Province: ______________________________

Date	Group	ADT	EDT	Participant Time	Audience	Language	Moderator
Monday Sept. 14, 2020	1	7:00pm	6:00pm	7:30pm NDT 7:00pm ADT 6:00pm EDT	Eastern Canada: Newcomers	EN	MB
Monday Sept. 14, 2020	2	7:00pm	6:00pm	6:00pm EDT	Quebec: Newcomers	FR	CP
Monday Sept. 14, 2020	3	9:30pm	8:30pm	7:30pm CDT 6:30pm MDT 5:30pm PDT	Western Canada: Newcomers	EN	MB
Monday Sept. 14, 2020	4	9:30pm	8:30pm	7:30pm CDT 6:30pm MDT 5:30pm PDT	Western Canada: Young Adults (aged 18-35)	EN	CP
Tuesday Sept. 15, 2020	5	4:00pm	3:00pm	3:00pm EDT	Quebec: Seniors	FR	CP
Tuesday Sept. 15, 2020	6	4:00pm	3:00pm	2:00pm CDT 1:00pm MDT noon PDT	Western Canada: Seniors	EN	MB
Tuesday Sept. 15, 2020	7	7:00pm	6:00pm	6:00pm EDT	Quebec: Young Adults (aged 18-35)	FR	CP
Tuesday Sept. 15, 2020	8	7:00pm	6:00pm	7:30pm NDT 7:00pm ADT 6:00pm EDT	Eastern Canada: Young Adults (aged 18-35)	EN	MB
Wednesday Sept. 16, 2020	9	3:00pm	2:00pm	3:30pm NDT 3:00pm ADT 2:00pm EDT	Eastern Canada: Seniors	EN	MB
Wednesday Sept. 16, 2020	10	6:30pm	5:30pm	7:00pm NDT 6:30pm ADT 5:30pm EDT	Eastern Canada: Gen Pop	EN	MB
Wednesday Sept. 16, 2020	11	7:00pm	6:00pm	6:00pm EDT	Quebec: Gen Pop	FR	CP
Wednesday Sept. 16, 2020	12	9:30pm	8:30pm	7:30pm CDT 6:30pm MDT 5:30pm PDT	Western Canada: Gen Pop	EN	CP

Specification Summary
Twelve (12) focus groups in total: Newcomers – 3 groups Seniors (aged 60+) 3 groups Young Adults (aged 18-35) – 3 groups Gen Pop – 3 groups 4 English groups in each of Eastern Canada (NL, NS, PE, NB & ON) & Western Canada (MB, SK, AB & BC) 4 French groups in Quebec	Mix of: Gender Age Education Employment status Household income Incentive: $100 per participant 9 participants recruited per group Group discussion lasts up to 2 hours

Hello, my name is _______________ and I am calling on behalf of Narrative Research, a national market research company. Let me assure you that we are not trying to sell you anything. As part of a research project we are conducting on behalf of the Government of Canada, we are looking for people to take part in a small online group discussion on <INSERT DATE> that will last two hours. Those who qualify and take part in the group discussion will receive a $100 financial incentive. Is this something you might be interested in?

Would you prefer that I continue in English or French? / Préférez-vous continuer en français ou anglais?

RECRUITER NOTE - FOR ENGLISH GROUPS, IF PARTICIPANT WOULD PREFER TO CONTINUE IN FRENCH, PLEASE RESPOND WITH: "Nous recherchons des gens qui parlent anglais pour participer à ce groupe de discussion. Êtes-vous intéressé à participer à un autre groupe de discussion qui se tiendra en français? Si oui, quelqu’un fera le suivi avec vous dans les prochains jours.
Puis-je avoir votre nom afin que quelqu’un communique avec vous ? [RECORD NAME]
J’aimerais confirmer votre numéro de téléphone :[CONFIRM PHONE NUMBER]"

The purpose of this group discussion is to hear people’s views on an advertising campaign being considered. Participation in this research is voluntary and completely anonymous and confidential. No attempt will be made to sell you anything or change your point of view. The format of the focus group is an informal small online group discussion led by a professional moderator. May I ask you a few quick questions to see if you are the type of participant we are looking for? This should take about 5 or 6 minutes.

To begin, do you or any member of your household work in or has retired from…? [READ LIST – ROTATE ORDER]

[IF NO TO QP3, ASK] How many years have you been living in Canada? [RECORD # of Years: ____]

In which community and province do you currently live?
Record name of city/village and province: ______________________

IF EMPLOYED, ASK: In what sector do you work and what is your current occupation?
Sector: ____________________________________
Occupation: ________________________________

IF RETIRED, ASK: In what sector were you last employed before retirement and what was your occupation?
Sector: ____________________________________
Occupation: ________________________________

What was your household’s total income last year? That is, the total income of all persons in your household combined, before taxes? [READ LIST IN ORDER]

To make sure that we speak to a diversity of people, could you tell me what is your ethnic background? DO NOT READ – CODE ALL THAT APPLY

INSTRUCTIONS: For groups 4-12, recruit mix in each group to represent the general composition of each market

The discussion groups for this project will be conducted over the phone and online simultaneously and will require the use of a laptop or desktop computer and a telephone line (landline or cell phone) for the duration of the focus group session. Note that you cannot use a computer tablet or smartphone to access the online platform/screen.

Do you have access to a laptop or desktop computer with high-speed Internet to take part in this focus group?

Once you are online for the session you will also be required to join a telephone conference call to be connected to the discussion with the rest of the group. You will need to use a telephone line and NOT your computer audio. Will you have access to a dedicated telephone (either landline or cellular) located near your computer? Note that you will not incur long-distance charges.

INSTRUCTIONS FOR NF1-NF3 THANK & TERMINATE: Based on your responses, we are unable to invite you to take part in this online focus group, as you do not meet the technical requirements. We thank you for your interest in this research.

Have you ever attended a group discussion or interview for which you received a sum of money?

When was the last time you attended a group discussion or interview? _____________

How many groups or interviews have you attended in the past 5 years? ____________ MAX 4

What was the subject(s) of the focus group(s) or interview? __________________________

NETFOCUS INVITE:
Based on your responses so far, we would like to invite you to participate in a small group discussion that will be conducted simultaneously over the telephone and online at <INSERT TIME> on <INSERT DATE>. The session will bring together 6 to 8 people and it will last about two hours. The discussion will be about an advertising campaign being considered and you will receive $100 in appreciation for your time. To take part, you would simply log on to a secure website from your computer and at the same time join in a group discussion on the phone via conference call.

The discussion in which you will be participating will be audio recorded for research purposes only. Be assured that your comments and responses are strictly confidential and that your name will not be included in the research report. Are you comfortable with the discussion being audio recorded?

There may also be employees from the organizations who is sponsoring the research, who will be listening in on the discussion. They will not be given the last names of participants. Are you comfortable with having observers?

The group discussion will be held [GROUPS 1, 3, 4, 6, 8, 10 & 12: in English] [GROUPS 2, 5, 7 & 11: in French]. Participants may also be asked to read text, write responses and/or review images during the session. Are you able to take part in these activities [GROUPS 1, 3, 4, 6, 8, 10 & 12: in English] [GROUPS 2, 5, 7 & 11: in French] on your own, without assistance?

Could we please confirm the email address where we can send you the detailed conference call instructions for logging in to the group?

Record email address (and verify): ________________________________________________.

We will send you the instructions by email at least 1 day in advance of the group. The group discussion will begin promptly at <TIME> and will end at <TIME>, lasting up to 2 hours. Please log in on time to ensure that the session is not delayed. If you arrive late, we will not be able to include you in the discussion, and will not provide you with the incentive.

As mentioned, we will be pleased to provide everyone who participates with $100, provided by e-Transfer or cheque, as you’d prefer. It takes approximately 3 business days to receive an incentive by e-Transfer or approximately 2-3 weeks following your participation to receive an incentive by cheque.

IF PREFER TO RECEIVE INCENTIVE BY E-TRANSFER:
Could you please confirm the e-mail address where you would like the e-transfer sent after the focus groups?

Email address:________________________________________________________________
And please confirm the spelling of your name: _______________________________________

IF PREFER TO RECEIVE INCENTIVE BY CHEQUE:
Could I have the mailing address where you would like the cheque mailed after the focus groups?

Mailing address:_______________________________________________________________
City: ________________________________________________________________________
Province: ____________________________________Postal Code: _____________________
And please confirm the spelling of your name: _______________________________________

ALL:
As these are very small groups and with even one person missing, the overall success of the group may be affected, I would ask that once you have decided to attend that you make every effort to do so. If you are unable to take part in the study, please call_____ (collect) at ________as soon as possible so a replacement may be found. Please do not arrange for your own replacement.

So that we can call you to remind you about the focus group or contact you should there be any changes, can you please confirm your name and contact information for me? [CONFIRM INFORMATION ALREADY COLLECTED AND CHANGE/COMPLETE AS NECESSARY]

First Name: ____________________________________________
Last Name: ____________________________________________
Email: ________________________________________________
Cell Phone: ____________________________________________
Home Phone: __________________________________________
Work Phone: ___________________________________________

Thank you for your interest in our study. We look forward to hearing your thoughts and opinions!

Attention Recruiters
Recruit 9 per group
CHECK QUOTAS
Ensure participant has a good speaking (overall responses) ability-If in doubt, DO NOT INVITE
Do not put names on profile sheet unless you have a firm commitment.
Repeat the date, time and verify email before hanging up.

Appendix B: Moderator’s Guide

CANADA REVENUE AGENCY
2020-2022 CRA Scams Advertising Campaign Concept Testing
Moderator’s Guide – V5

Study Goals (Confidential – For reference only - Not read to participants)

[SLIDE 3] The advertising ideas we will look at today are about scams involving the government. Essentially, people or organizations pretending to be the Government of Canada.

[SLIDE 4] Tonight we are going to look at different ideas for an advertisement campaign that is being considered by the Government of Canada. A campaign is a family of ads that share the same theme. We will look at four different advertising concepts, each including a web banner and mock-ups of two tv ads. As we look at each campaign, keep in mind these are just ideas at this point; this means the ads you will see are not final. You’ll have to use your imagination a little to envision what the ads would be like in their final form. For example, for the tv ads, I will show you a storyboard which is a series of still images that will give you an idea of the scenario and help you imagine what the video might look like once produced. The final ad would be a fully-produced commercial like any other you would see on TV.

Given that what I’m showing you are in the draft form only, please focus on what the campaign is saying and the manner in which the message is communicated, rather than the quality of the design. For now, I’m mostly interested in your views on the ideas underlying each concept.

[SLIDES 5 TO 10] We will look at and discuss each campaign one at a time. Please hold your thoughts as I show you the advertisements that are part of each campaign. Once I am done, I will ask for your individual opinion before we discuss the campaign as a group. MODERATOR PRESENTS EACH CAMPAIGN ONE AT A TIME – PRESENT BOTH WEB BANNER AND STORYBOARD TOGETHER - ROTATE ORDER OF CAMPAIGN ACROSS GROUPS

CAMPAIGN A – Scam-Q (TV1: TBD / TV2: TBD)
CAMPAIGN B –Confessions of a Scam Artist (TV1: TBD / TV2: TBD)
CAMPAIGN C –Question it (TV1: TBD / TV2: TBD)
CAMPAIGN D – Ask Yourself (TV1: TBD / TV2: TBD)

Rotation Schedule – Campaign Presentation Order
Audience	Eastern (EN)	Western (EN)	Quebec (EN)
Newcomers	G1 – A-B-D-C (TV: 1-2)	G3 – B-C-A-D (TV: 2-1)	G2 – D-A-C-B (TV: 1-2)
Young People	G6 – C-D-B-A (TV: 2-1)	G4 – D-A-C-B (TV: 1-2)	G7 – B-C-A-D (TV: 2-1)
Seniors	G9 – B-C-A-D (TV: 2-1)	G8 – C-D-B-A (TV: 2-1)	G5 – A-B-D-C (TV: 1-2)
Gen Pop	G11 – D-A-C-B (TV: 1-2)	G12 – A-B-D-C (TV: 1-2)	G10 – C-D-B-A (TV: 2-1)

AFTER THE PRESENTATION OF THE CAMPAIGN: Before we talk about this idea, I’d like you to complete an individual exercise. Take a moment to answer the questions you see on the screen. Keep in mind, I can see your responses, but other participants will not; and I will not ask you to share your responses with others.

Indicate on a 1-10 scale the extent to which the statement aligns with what you think (where 1 means “not at all” and 10 means “absolutely”):

AFTER THE EXERCISE: Now let’s talk about this idea together…
Overall Reactions:

Now let’s have a look at the next campaign idea.
REPEAT THE EXERCISE AND QUESTIONS FOR THE SECOND, THIRD AND FOURTH CAMPAIGNS

[SLIDES 11-12] Now that we’ve seen all four ideas, I’d like to know which campaign you think would be strongest at making people think about government-related scams. Take a moment to share your response individually:

The campaigns we saw identify the sponsor as either the Government of Canada or a specific department that is impacted by the scam (e.g., CRA). The goal of the campaign is to warn the public about those types of scams.

[SLIDES 13-14] I’d like to get your thoughts on the taglines we’ve seen in the campaigns. Any of the taglines could work with the ideas we’ve seen so don’t worry about the context in which they would be used. SHOW ALL TAGLINES ON THE SCREEN ONE AT A TIME – ROTATE ORDER ACROSS GROUPS FOLLOWING THE CAMPAIGN PRESENTATIONS SCHEDULE ABOVE

CAMPAIGN A: Canada.ca/Stop-Scams
CAMPAIGN B: Canada.ca/Slam-the-Scam
CAMPAIGN C: Canada.ca/Be-Scam-Smart
CAMPAIGN D: Canada.ca/Is-This-A-Scam

[SLIDE 15] That’s all my questions; on behalf of the Government of Canada, thank you for taking part in our discussion. We will follow-up with each of you regarding the distribution of your incentive.

Marketing/Market Research	1
Public relations	2
Advertising	3
Media (TV, Radio, Newspaper)	4
Graphic Design	5
Provincial government	6
Federal government	7
A tax preparation company	8
Accounting	9

Less than 5 years	1
5-9 years	2
10-19 years	3
20-29 years	4
30 or more years	5
Don't know / No response	6

Less than 18	1
18-25	2
26-35	3
36-45	4
46-59	5
60-65	6
More than 65 years old	7

Some high school only	1
Completed high school	2
Some college/university	3
Completed college/university	4
Post-graduate studies	5
DK/NR	6

Working full-time (at least 30 hours per week)	1
Working part-time (less than 30 hours per week)	2
Self-employed	3
Retired	4
Unemployed	5
Student	6
Other (Specify: _____)	7
DK/NR	8

Under $20,000	1
$20,000 to under $40,000	2
$40,000 to under $60,000	3
$60,000 to under $80,000	4
$80,000 to under $100,000	5
$100,000 to under $150,000	6
$150,000 or more	7
DK/NR	9

White/European (for example, German, Irish, English, Italian, French, Polish, etc.)	1
Hispanic, Latino, Spanish (for example, Mexican, Cuban, Salvadoran, Columbian, etc.)	2
Black or African Canadian (for example, African Canadian, Jamaican, Haitian, Nigerian, Ethiopian, etc.)	3
East Asian (for example, Chinese, Filipino, Vietnamese, Korean, etc.)	4
South Asian (for example, East Indian, Pakistani, etc.)	5
Middle Eastern or North African (for example, Lebanese, Iranian, Syrian, Moroccan, Algerian, etc.)	6
Indigenous (e.g. First Nations, Métis, Inuit)	7
Other (Specify: ___)	8
Don't know / No response	8

Very comfortable	1
Somewhat comfortable	2
Somewhat uncomfortable	2	THANK & TERMINATE
Very uncomfortable	2	THANK & TERMINATE

Yes	1	CONTINUE – Max of 5 recruits per group
No	2	Go To SECTION I: Invitation

Male	1
Female	2
Gender diverse	3

e-Transfer	1
Cheque	2

2020-2022 CRA Scams Advertising Campaign Concept Testing - Final Report

Table of Contents

Executive Summary

Background and Research Methodology

Political Neutrality Certification

Key Findings and Conclusions

Introduction

Objectives

Research Methodology

Target Audiences

Research Approach

Presentation of Concepts

Context of Qualitative Research

Detailed Analysis

Government Scam Awareness and Experience

Campaign Testing

Overall Reactions Across Concepts

Strongest Campaign

Campaign Concept: Question It (Ça ne coûte rien de se questionner)

Overall Reactions, Attention and Memorability

Message, Confidence and Credibility

Relevance and Appeal

Clarity

Intended Call to Action

Campaign Concept: Ask Yourself (Interrogez-vous)

Overall Reactions, Attention and Memorability

Message, Confidence and Credibility

Relevance and Appeal

Clarity

Intended Call to Action

Campaign Concept: SCAM-Q (Déjouer les fraudeurs)

Overall Reactions, Attention and Memorability

Message, Confidence and Credibility

Relevance and Appeal

Clarity

Intended Call to Action

Campaign Concept: Confessions of a Scam Artist (Confessions d’un fraudeur)

Overall Reactions, Attention and Memorability

Message, Confidence and Credibility

Relevance and Appeal

Clarity

Intended Call to Action

Advertisement Sponsor

URLs

Canada.ca/Stop-Scams (Stopper-la-fraude)

Canada.ca/Be-Scam-Smart (Évitez-les-arnaques)

Canada.ca/Is-This-A-Scam (Est-ce-une-arnaque)

Canada.ca/Slam-the-Scam (À-bas-l’arnaque)

Conclusions and Direction

Appendix A: Recruitment Screener

Appendix B: Moderator’s Guide

Appendix C - Materials Tested