Study on Online Disinformation Advertising Creative Testing - Focus Group Report

Prepared for the Communications Security Establishment (CSE)

Supplier:
Ekos Research Associates Inc.
Contract Number:
CW2246079
Contract Value:
$47,369.60
Award Date:
October 26, 2022
Delivery Date:
November 30, 2022
Registration Number:
POR 073-22

For more information on this report, please contact Communications Security Establishment at: media@cse-cst.gc.ca

This public opinion research report presents the results of focus groups conducted by Ekos Research Associates on behalf of Communications Security Establishment. The research was conducted in November 2022.

Cette publication est aussi disponible en français sous le titre: ???

This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Communications Security Establishment. For more information on this report, please contact Communications Security Establishment at: media@cse-cst.gc.ca or at:

Communications Security Establishment

P.O. Box 9703

Terminal

Ottawa, Ontario K1G 3Z4

Canada

Catalogue Number:
D96-97/2022E-PDF
International Standard Book Number (ISBN):
978-0-660-46802-0

Related publications (registration number: POR 073-22):

Évaluation de concepts publicitaires sur la désinformation en-ligne (Final Report, French)

Catalogue Number:
D96-97/2022F-PDF (French)
ISBN:
978-0-660-46803-7

© His Majesty the King in Right of Canada, as represented by the Minister of National Defence, 2022

Table of Contents

Summary

A. Campaign Background

Budget 2022 states that "issues ranging from COVID-19 to climate change and increasingly confrontational authoritarian regimes demand the attention of Canada and our allies. The spread of misinformation and disinformation is directly challenging the stability of even the most long-standing democracies. Budget 2022 recognizes those challenges and proposes new action to respond to them...It commits to reinforcing global democracy, to combating illicit financing, and to pushing back against the forces of disinformation and misinformation that threaten public institutions around the world."

While both misinformation and disinformation involve the sharing of false information, disinformation is maliciously and intentionally spread. The Canadian Centre for Cyber Security also recognizes malinformation and defines the terms as follows:[1]

The same publication from the Canadian Centre for Cyber Security also states that: "The effects of misinformation, disinformation, and malinformation (MDM) cost the global economy billions of dollars each year. Often known colloquially as "fake news", MDM are damaging to public trust in institutions and, during elections, may even pose a threat to democracy itself. MDM has become a serious concern for consumers and organizations of all sizes."

Social media plays a key role in the spread of MDM. Social media algorithms favour engagement (engagement is an action on a social media post such as a like, comment or share). The more engagement a piece of content gets, the more likely it is to be rewarded by the algorithm and shown to more users on the platform, leading to further engagement in a snowball effect.[2]

Misinformation and fake news appear to spread comparatively quickly through social media channels due to the content's novelty and capacity to illicit emotional responses. Once a user's preference for misinformation or fake news is established, recommendation algorithms may direct people to similar content and encourage people to travel down a 'rabbit hole' of misinformation and fake news.[3]

The spread of disinformation, specifically, reduces the government's effectiveness in providing Canadians with programs and services. When disinformation centres around vaccine safety, the COVID-19 pandemic, the Canadian democratic process or climate change, it undermines government efforts to protect its citizens with respect to those issues.

The Communications Security Establishment (CSE) will be launching an advertising campaign that will focus on disinformation using what is known about misinformation and its spread. Through this campaign, Canadians will be invited to learn more about disinformation by visiting canada.ca/online-disinformation. This series of focus groups were conducted to test reactions to two different proposed concepts for intended use in the campaign and to assess which concepts and approaches are most appropriate for the target audiences. The information gained through this public opinion research will be shared throughout the CSE to assist it when developing communications products and strategies.

Specifically, the research was used to evaluate how the target audience responds to creative concepts for the Online Disinformation advertising campaign, including:

B. Methodology

Concept Testing

Two concepts were tested in eight online focus groups, which included participants from five different regions across the country. Participants logged onto a Zoom video meeting to generally discuss their comfort with distinguishing between information, news and announcements that are "real" or "fake". Participants subsequently viewed a series of panels proposed to create a 15-second video as well as associated Facebook post, for two separate proposed concepts. They were asked to react to each concept in terms of overall impressions, clarity of message and appropriateness of the approach in encouraging Canadians to visit the CSE's Disinformation online resource. They were also asked to provide a series of ratings for each concept along similar dimensions.

Participants were recruited to represent a region, rather than one specific city, which was a cost-effective means of obtaining feedback. Eight participants were recruited to attend each discussion, using the Probit online panel, targeting Canadians 18 or older, keeping in mind a mix of participants in terms of gender, age, socioeconomic status and parents vs. non-parents (recruitment screener is provided in Appendix A). Probit panel members were selected from those who had previously completed one of several waves of the Risk Monitor, where four items were used to calculate a "Disinformation Score"[4]. Only panel members with a Disinformation Score ranging between 3 and 8 were included for recruitment. In total, 46 individuals participated in the concept testing discussions, of the 64 recruited (see details in following table). Discussions occurred on November 8th, 9th and 10th, 2022.

Table 1: Number of Participants per Region – Concept Testing Discussions
Region # of Groups Total # of Recruits Total # of Participants
Atlantic Ontario, Anglophones in Quebec 3 24 16
Manitoba, Saskatchewan, Alberta, British Columbia 3 24 18
Francophones in Quebec and New Brunswick 2 16 12

Focus group guides (provided in Appendix B) were developed by Ekos in consultation with CSE, along with a rating sheet used to rate each concept and an overall rating and ranking of the top concepts to quantify the results and obtain an initial reaction from each participant prior to discussion. Six of the eight groups were conducted in English, while the two groups held with residents of Quebec were conducted in French. Each focus group took between 65 to 90 minutes to conduct. Participants received an incentive of $120 for their participation. Video recordings, researchers' notes and observations from the focus groups formed the basis for analysis and reporting of results.

C. Key Findings

Concept Testing

Overall, there were a few overarching themes expressed by participants when discussing the two concepts:

Both concepts are described in the Detailed Findings section. Below are a brief summaries of participant reactions to the concepts.

What is Dis?

The What is Dis concept was described by most participants as bland or not appealing. Some participants related to the action of scrolling through their phone, as well as with the phrase "Social media platforms favour popularity over fact". The "What is Dis" question, however, did not resonate with most participants, many of whom expressed a particular dislike for the term "Dis" viewing it as slang. Further, the use of the term "Dis" resulted in the perception that the concept was intended for younger audiences.

Don't share it?

Some participants favoured the Don't Share It concept for its quirky humour, making it more attention-grabbing or noticeable approach. Many participants, however, saw the humour as juvenile or "cheesy". The opening frames of the concept, displaying a blog and scrolling through the site, was seen as relevant to participants, however, when the bear popped onto the screen, many participants felt the attempt to garner attention became "silly". Most participants also did not like the jingle sung by the bear, in part because of the double negative of the line "If you're not sure if it's not true, dooooooon't share it".

D. Note to Readers

It should be kept in mind when reading this report that findings from the focus groups are qualitative in nature, designed to provide a richer context rather than to measure percentages of the target population. These results are not intended to be used to estimate the numeric proportion or number of individuals in the population who hold a particular opinion as they are not statistically projectable.

E. Contract Value

The contract value for the POR project is $47,369.60 (including HST).

Supplier Name:
Ekos Research Associates
PWGSC Contract Number:
CW2246079
Contract Award Date:
October 26, 2022

To obtain more information on this study, please e-mail CSE at: media@cse-cst.gc.ca

F. Political Neutrality Certification

I hereby certify as Senior Officer of Ekos Research Associates Inc. that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the on Communications and Federal Identity and the Directive on the Management of Communications. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.

Signed by Susan Galley (Vice President)

Concept Testing – Detailed Findings

A. Confidence Identifying Disinformation

Most participants said they feel confident they can tell the difference between real and false information that you see on the Internet, at least most of the time. Some said it is difficult to tell the difference because some information seems very real, and there is so much information available online. Some said they try to stay alert to any clues they believe might indicate the information is misinformation, such as mis-spelled words, or posted from unrecognized domains. Many participants described their process to verify questionable information through other sources, with many pointing to Google to find multiple sources of information. A few also said they read the comments section of a post. A few participants noted it is harder for them to identify misinformation when it is something they want to believe.

B. Concept Testing

What is Dis

The What is Dis concept opens with an image of a social media feed, with typical sounding posts until the scrolling slows at a specific post. The image zooms in on the post reading "This post is a lie, but something about it is believable, so you should share it". This post is subsequently surrounded by a large highlighted square with the question "What is dis". The square animates and reassembles to reveal the message, "This is disinformation". The next frame reads "Social media platforms favour popularity over fact" followed in smaller font, by "Learn how to spot disinformation, fact check it, and stop it's spread at Canada.ca/ disinformation". A Facebook post is included in the concept material, with the words, on a yellow square on top of blue background, "Can you be sure this post is legit?" and the opportunity to click to learn more.

The What is Dis concept was viewed by most participants as "basic", "bland", or not appealing. The concept was not seen as having the ability to attract attention and is unlikely to stand out among the mass of online content, according to many participants. Some participants said they related to the idea of scrolling through their phone. A few, however, felt the sentence "This post is a lie, but something about it is believable, so you should share it" is confusing or that the ad should include a specific post.

The message of the concept was clear to many. The phrase "Social media platforms favour popularity over fact" resonated with some participants. This is something they know to be true, but a few said it is good to be reminded of this. A few also said the phrase did not seem judgmental and was not seen as making fun of people using or sharing posts on social media.

Most participants reacted negatively to the use of the word "Dis" in the concept. They felt strongly that the government of Canada should not be using slang, and noted that mis-spelling words, or words out of context, is an attribute of online scams and phishing communication, or that "Dis" might be "Disrespect". Overall, the use of "Dis" was also interpreted as used by a younger demographic, and therefore, may only reach a narrow target. A few appreciated the use of "Dis" as a reference to "Disinformation" and understood it as a play on "This".

A few said the Facebook post with this concept is more appealing than the 15 second ad, perhaps because it does not use the term "Dis". A few liked the use of the yellow as a pop of colour and perhaps resemblance to a sticky note. Interestingly, no participants noted the use of the slang term "Legit" in the Facebook post for What is Dis. Many participants said they would scroll past this post.

Don't Share It

The Don't Share It begins with a mock-up of a Newsblog. A mouse pointer scrolls the page and comes to rest on a post with the headline "New bear-like creature spotted in suburban neighbourhood". An overly-large share button subsequently appears and the mouse pointer hovers above the share button. The action is interrupted by a large bear taking up much of the frame. The bear then launches into a jingle, "If you're not sure if it's not true, dooooooooooon't share it!". A page subsequently appears with the caption "Real disinformation doesn't come with a timely interruption or a jingle" and then, in smaller font, "Learn how to spot disinformation, fact check it, and stop its spread at Canada.ca/disinformation". A coordinating Facebook post is also presented, with the text "If you're not sure if it's true, don't share it", displaying a tuft of fur from the bear-like creature. Two versions were presented to participants, one with the furry bear, another with a finger dressed as a bear.

The Don't Share It concept was appreciated by some participants for its quirky humour, as an ad that would be noticeable. Many participants, however, felt the humour is juvenile or "cheesy". Some of these individuals felt this approach must be targeting pre-teens or teens, and therefore, they would ignore it although they might pass it on to their child.

A few participants felt the reference to a "bear-like creature" in the post sounds believable, as they live in areas with bears and a sighting would not be unusual. A few perceived that the post could have been real, equivalent to reported sightings of sasquatch or a newly discovered creature; for these participants, a post would have to be obvious in its disinformation.

Most participants understood the message of this concept to be: think before you share information as it may be misinformation. The exaggerated share button helps to communicate this message along with the bear actively warning not to share it.

The Don't Share It concept was presented to participants using two versions: one with a furry bear character, and one with a finger dressed as a bear. There appeared to be a slight preference for the furry bear character. Conversely, some preferred the finger as the bear was "too in your face". A few, however, said that the finger bear gave them the sense that the government was "giving me the finger" or that they are wrong or being lectured to in the idea of "finger wagging". While the connection between the furry bear character and the story was clear to most, some participants did not appear to make the connection of the finger as the instrument for sharing the story.

A jingle was included in the Don't Share It concept. While the draft audio of the jingle was not clear when it was played in the discussions, few participants appreciated the jingle. Specifically, many participants felt it added to their distain of the concept and the silly humour. Further, the "double negative" of the jingle ("If you're not sure if it's not true, dooooooooon't share it") added confusion for some participants. A few liked the jingle, seeing it as catchy, helping to capture attention for the concept.

C. Concept Ratings

It should be kept in mind when reading this report that findings from the focus groups are qualitative in nature, designed to provide a richer context rather than to measure percentages of the target population. These results are not intended to be used to estimate the numeric proportion or number of individuals in the population who hold a particular opinion as they are not statistically projectable.

In advance of any discussion, participants were asked to rate each concept along four dimensions for top-of-mind reactions. In terms of process, a concept was introduced (i.e., read to participants, with visual materials shown), and participants were subsequently sent to a ratings page to capture their initial reaction, in advance of the in-depth discussion. Once a concept was described, rated and discussed, the second concept was presented. The order of presentation of the concepts was also rotated from discussion to discussion.

Following are the results of the ratings provided by focus group participants for the concepts tested. In each table below, results are collapsed according to ratings indicating whether a specific concept is weak to moderate (rating it a 1, 2 or 3 out of 5), or strong (rating it a 4 or a 5). Results show a split in opinion, but with a lean to Don't Share it because of its ability to garner attention and stand out.

Clarity of Message (n=38)
Rating What Is Dis? Don't Share
Strong (4-5) 20 (53%) 22 (58%)
Weak – Moderate (1-3) 18 (47%) 16 (42%)
Grabs Attention (n=38)
Rating What Is Dis? Don't Share
Strong (4-5) 13 (32%) 17 (45%)
Weak – Moderate (1-3) 25 (66%) 21 (55%)
Relevant (n=38)
Rating What Is Dis?[5] Don't Share
Strong (4-5) 8 (22%) 13 (34%)
Weak – Moderate (1-3) 29 (78%) 25 (66%)
Overall (n =38)
Rating What Is Dis? Don't Share
Strong (4-5) 11 (29%) 16 (42%)
Weak – Moderate (1-3) 27 (71%) 22 (58%)

Once the two concepts were introduced and discussed, participants were asked to again go to a ratings page to individually provide their overall preference for a concept, in advance of further discussion about their preferences. Based on these results, Don't Share It was chosen as the preferred concept by nearly half of participants. One in four participants preferred What is Dis, and an equal proportion indicated that they did not like either concept.

Preference (n=36[6])
Concept What Is Dis?
What is Dis? 9 (25%)
Don't Share it? 16 (44%)
Both equally (2 (6%) 2 (6%)
Neither 9 (25%)

D. Website Mock-up

A mock up of an "Online disinformation" website was presented to participants. This image had a Government of Canada header, indicated that "Disinformation spreads online, causing harm to Canada and Canadians. Learn how to identify and fact-check disinformation". Fourteen categories were bulleted on the page, with the implied ability to click on a category to learn more, along with a video that could add further explanation.

Most participants responded positively to the website mock up. Although the mock-up was often described as "pretty standard" or "typical Government of Canada website", many felt that the website looks official. Many said the site looks clean, well organized, and easy to read and navigate.

Many participants said the 14 categories presented in the mock-up seem relevant and they would click on many of them for further information. Some participants, however, felt a few of the categories were subjective and may risk being political or expression of an opinion about what is true and what is false information, or to be self-serving. Participants tended to focus on the headings "How disinformation undermines democracy", "Who is creating disinformation and why?" and "Government of Canada efforts to fight disinformation". A few participants felt there are too many topics overall and the list could be reduced.

Most participants said any website would have to contain neutral and apolitical information to be effective. Some participants, particularly those most distrustful of the government, said they would be cautious when reading the information on the site, or were concerned the government is trying to be the arbiter of what is true or wants "all information to come from them and no one else".

Appendices

Appendix A: Recruitment Script

PREENG

Government of Canada Online Group Discussions

Dear [name],

We are holding a series of small, online discussions with Canadians to gather feedback about a proposed approach to an upcoming public awareness campaign. All participants will receive a $120 incentive for their participation. If you are interested, please click on the following link to complete a few qualifying questions and to get a few more details:

Click [here]

If the link does not work for you, please copy the following one into your browser: [link]

Details:

We are holding these small, online discussions for the Communications Security Establishment (CSE) to gather feedback about proposed creative approaches to an upcoming awareness campaign on how Canadians get their information. It's important that the CSE is able to get this kind of feedback so that they can design the best possible awareness campaign.

Participation is easy, and of course, voluntary! You will be asked to log into a video conference website, using a password protected link, to discuss and review proposed approaches to the campaign. The discussion will last between 75 and 90 minutes and will be attended by only 6 participants. The discussion will be led by an experienced moderator and observed by officials responsible for planning the awareness campaign.

This online discussion will take place in a secure and confidential environment. All personal identities will be protected. You will only log in, and be referred to by, your first name. No one else in the discussion will know your identity. Anonymous excerpts or quotes from the discussion may be included in the final aggregate report to illustrate the findings, but they will not be linked to any participant.

Once you register, you will receive a confirmation email right away and, if you are selected to participate, you will receive a short confirmation call within a day or two. We can answer any questions you may have at that time. You will also be sent an email reminder the day before the discussion which will include the video conference link, meeting ID and password to gain access to the discussion.

If you have any questions, contact Ekos Research: 1-800-388-2873 or focusgroups@ekos.com. This research is also registered with the Research Verification Service maintained by the Canadian Research Insights Council and can be verified at https://canadianresearchinsightscouncil.ca/rvs/home/ by entering project code [code].

Thank you in advance for your interest. We hope you are able to join us!

Susan Galley

Project Manager

Ekos Research Associates, Inc.

www.ekos.com

Introduction

Thank you for visiting the registration site for the online focus groups with Canadians for the Communications Security Establishment (CSE). As explained in the email invitation, Ekos Research Associates is leading a series of discussions to get feedback on several proposed approaches to an awareness campaign to be launched in early 2023.

The 75 to 90-minute discussions will take place online on November 8th, 9th and 10th in the evening. In each session, a small group of 6 participants will be asked to provide constructive feedback to help the CSE by reviewing and discussing proposed approaches to the upcoming campaign. The sessions will be observed by representatives responsible for planning the campaign so that they can hear first-hand what Canadians think. Confidentiality and anonymity will be maintained throughout the research process. Participants will receive a $120 incentive for their participation.

Once you have registered, you will receive an email confirmation right away. We will randomly select 6 participants for each session, among those who register. If you are selected, you will receive a telephone confirmation within a few days. You will also receive an email reminder with the link to the secure video conference meeting a day or two before the session.

QGENDR

Are you...

QAGEX

May we have your year of birth, please?

QAGEY

Hesitant

If... QAGEX = 99

Would you be willing to tell us in which of the following age categories you belong?

Q2

Are you or is any member of your household or immediate family employed in:

Q2A

Government of Canada

Q2B

An advertising agency

Q2C

A market research company

Q2D

The media (Print, Radio, TV, Internet)

CALCQ2

If yes, Screen out

QINCOME

Which of the following categories best describes your total household income? That is, the total income of all persons in your household, before taxes?

D2

Which of the following categories best describes your current employment status? Are you...?

Q3

Participants in these discussions will be asked to voice their opinions and thoughts in the discussion. How comfortable are you in voicing your opinions in front of others, in [QC] French [Else] English ? Are you...

Q3B

If you are selected to participate in one of the discussions, you will be asked to log onto a video conference website and if you use separate audio, also dial into a teleconference number, with a Canadian number provided. You will participate through a general discussion, as well as by viewing some materials shown to you online throughout the discussion. Sessions will be recorded for research purposes only, but confidentiality and anonymity will be maintained.

Would you be comfortable clicking on a link that we provide in an email to log onto the website to participate and see these materials?

Q3C

Participants may also be asked to read a few paragraphs of materials during the discussion. Is there any reason why you could not participate, including reading some material on screen?

Q4

Have you ever attended a focus group or one on one discussion for which you have received a sum of money?

Q5

Yes, Q4

If... Q4 =: 1

When did you last attend one of these discussions that was sponsored by the Government of Canada?

CALCQ5

If... Q4 = 1 and Q5 not = 999

Calculation

Q5B

Yes, Q4 and not never, Q5

If... Q4 = 1 and Q5 not = 999

Have you attended 5 or more of these discussions that were sponsored by the Government of Canada?

QINFO

Would you be interested in participating in one of these online discussions?

QLANG

Would you prefer to participate in a discussion in English or in French?

QFOCUS

Replacements are not permitted. If you usually use reading glasses you should make sure to have them with you as there will be some viewing of some images and reading of materials throughout the discussion.

Are you able to participate in the 75 to 90 minute discussion on ...?

Note that times are listed as eastern standard time and may not necessarily be in your own time zone.

Note:

QFOCUSB

We are asking that all participants log in 5 minutes prior to the start time of the session. If you are selected, would you be able to log on and dial in 5 minutes prior to the session time? Late arrivals may not be admitted to the discussion, nor would an incentive be received.

QTELE

Text/phone

If you are selected to participate in one of the discussions, we will be giving you a reminder telephone call and sending an email a day or two prior to your group discussion.

Is [telephone] the best telephone number at which to reach you?

QEMAIL

Email

[EMAIL is not empty] Is [Email] the best email address at which to send you an invitation to the discussion, with the secure link and 1-800 number? [Else] What is the best email address at which to send you an invitation to the discussion, with the secure link and 1-800 number?

PFNAME

Please provide your first and last names. Only your first name will be used in the discussion.

FNAME
LNAME

PSMAIL

If you participate, your $120 honorarium will be sent by e-transfer, or if this is not possible, then by regular mail following the discussion. Would e-transfer to the email address [insert email address confirmed earlier] work for you?

If not, correct/change email address

or

By mail

If by mail - May we have your mailing address?

ECONFIRM

Confirmation of online registration for group discussion

(La version française du message suit)

Dear [name],

This is to confirm that you are registered to attend an online discussion taking place on [date, time]. The discussion will be in [language] .

Thank you for expressing your interest. If you are selected to participate in one of the discussions we will call you to confirm by telephone and provide a few more details about the discussion, as well as answer any questions you may have. We will then send you an email reminder the day before the discussions, including the login instructions as well as the password protected link for the discussion. We must first have the confirmation telephone call completed or we will not be able to send you these details for you to participate in the discussion.

If you want to contact us about this group, please call Ekos Research, at 1-800-388-2873 or email focusgroups@ekos.com. You may also verify this research with the Research Verification Service operated by the Canadian Research Insights Council (CRIC) at https://canadianresearchinsightscouncil.ca/rvs/home/ by entering (project code).

Sessions will be recorded for research purposes only. Ekos Research Associates Inc. is committed to maintaining the security and privacy of the information we collect from the public, and we protect your personal information through appropriate physical, organizational and technological measures. For more information about our privacy practices, please read our Privacy Policy. For questions regarding access to personal information held by Ekos, the accuracy of this personal information, or complaints related to Ekos' privacy practices, please contact our Privacy Officer at pobox@ekos.com.

Thank you for your registration.

Susan Galley

Project Manager

Ekos Research Associates, Inc.

www.ekos.com

THNK

If you have any questions, please let us know by calling us toll-free at 1-800-388-2873 or by sending an e-mail to focusgroups@ekos.com. Thank you for your cooperation and time.

THNK2

Screened out

[QFOCUS = 99] If you are able to select one of the other available focus groups, please use the Back button to return to the previous screen and change your selection. Otherwise, thank you for your time. [Else] Thank you for your cooperation! Based on the information you have provided, unfortunately you are not eligible to participate in this survey.

Appendix B: Concept Testing Discussion Guide

1. Introduction (5 Minutes)

2. Introductions (5 Minutes)

Let's start by going around the group. Please introduce yourself and tell us something about yourself (work, children, city you live in, etc).

3. Warm up (15 Minutes)

Would you say that you feel confident that you can tell the difference between real and false information that you see on the Internet?

  1. How do you usually judge how information you see online is real or fake?
  2. Are you more likely to trust information from certain sources?

[Probe: family/friends, certain people online, governments, banking institutions, books]

Do you feel generally confident that you know where to go to obtain information from reliable sources when you are faced with decisions?

  1. Do you feel you need more information or guidance about your options?
  2. Do you look for ways to double check the information?

4. Concept Testing (40 Minutes, 20 Per Concept)

We are going to look at two different concepts or approaches that the Government of Canada is considering and get everyone's reaction to them. Your feedback is important and will feed into developing some new communication material to help people get informed about harmful disinformation and see why it is important to fact check.

Again, remember that there are no right or wrong answers here. Everybody has an equally valid opinion.

Moderator: Show and go through each concept individually. Rotate order each time.

Each of these concepts or approaches is currently at the development stage, so what we will look at is rough mock-up. First, we will look at a rough story line for a 15 second video to be used in an online ad. As you will see, it isn't acted out, but is shown in a single drawing, with a brief description of the audio portion. So, it's not really what the video of the ad would look like, but it's meant to give us an idea of what they are thinking about. We will also go to a ratings page for the set so open your chat now and that is where you will find the links after each concept.

Moderator: show the 15 second animatic of sample concept

Before we start our discussion, take a minute to fill in a few ratings on the sheet and write down a few words on your initial reaction on the comments line on the ratings sheet.

What do you think of it? What is your first impression?

  1. What do you like/not like about it?
  2. Is the message or main point clear (e.g., the value of knowing how to recognize reliable and verified information sources, and what is likely disinformation/fake news, what the Government of Canada can offer to support Canadians when it comes to recognizing how to look for reliable information on which to base decision-making)?
  3. Is the tone of the material appropriate? Likes/dislikes and why?
  4. What do you think of the images or characterization used? Does it work? Is there anything that you don't like?
  5. Would it have you looking for information or going to the website?

If not:

  1. Where would you go to find further info?
  2. Why wouldn't you go the site?

Moderator: show the Facebook post

What is your impression of this Facebook post?

  1. What do you like/not like about it?
  2. Is the message or main point clear (e.g., the value of knowing how to recognize reliable and verified information sources, and what is likely disinformation/fake news, what the Government of Canada can offer to support Canadians when it comes to recognizing how to look for reliable information on which to base decision-making)?
  3. Would it have you looking for information or going to the website?
After presenting all concepts:

Thinking about these two different approaches, which one do you like the best and why?

  1. Which approach and tone appeals to you more?
  2. Which one more clearly tells the story of the objectives: emphasizing the importance of getting informed about harmful disinformation, telling you what to look for to decide if a source is trustworthy or not and seeing why it is important to fact check?
  3. Which approach would you be more apt to pay attention to or is generally more compelling?
  4. Which one would be more likely to have you go to the website or generally look for more information?

Is there anything that you would change about the one you prefer the best that we have not already discussed that would make it better or clearer or more impactful as far as you are concerned?

What kind of approach would be more likely to spark your interest to find out more about identifying disinformation?

  1. A focus on the harms of disinformation?
  2. A focus on helpful tips and hints; Here's how to spot it
  3. A message that empowers Canadians to stand up for the truth?

A message that encourages Canadians to stop before they share something?

5. Website Discussion – 20 Minutes

Moderator: show the website mock-up page

Now let's look at a mock-up of an intended website on Disinformation. Notice the headings:

What is your impression of this website and the headings shown here?

  1. How do you feel about the look of the website? What works and what doesn't?
  2. Do you think that this would be the type of website that might have useful information for Canadians?
  3. Even though we don't have the content here, does it seem this content would help Canadians increase their understanding of disinformation and why it matters?
  4. Could it help Canadians learn how to identify disinformation and emphasize the need to fact-check?
  5. What is the most important element of this site?
  6. Would you like to see any part of this emphasized more, or is there anything that you think is missing?

6. Overall Impression of Campaign (10 Minutes)

What do you think generally about this campaign and its objectives?

  1. a. Does it seem to make sense to you to inform Canadians about harmful disinformation and seeing why it is important to fact check?
  2. b. Would the campaign prompt you to pause before sharing something online? Does one of the approaches we looked at do a better job of this than the other?

What do you think of the feel of the campaign? Is CSE striking the right chord with Canadians, particularly in light of the Covid-19 pandemic? Is the campaign on the right track / taking the right approach in how it is informing Canadians about fact checking?

7. Wrap Up (2 Minutes)

Is there anything that we haven't talked about or that you would like to add before we go?

Thank you.

Appendix C: Ratings Sheet

What is Dis?/Don't share it

Tone:

Clarity of the message:

Grabs my attention:

Is relevant to me:

Comments:

Which of these two concepts do you prefer?

Appendix D: Concepts

Concept 1: Don't Share it

15 Second Storyboard, Frame 1

15 Second Storyboard, Frame 1. Text version below.

15 Second Storyboard, Frame 1 - Text Version

This page shows a blue box on the left side with text inside.

We open on a typical looking blog homepage. A mouse pointer moves in and scrolls the page upwards.

On right side, the image shows a page called Newsblog with different headlines, articles, and options, an image of a suburban neighbourhood with the headline "New bear-like creature spotted in suburban neighborhood".

15 Second Storyboard, Frame 2

15 Second Storyboard, Frame 2. Text version below.

15 Second Storyboard, Frame 2 - Text Version

This page shows a blue box on the left side with text inside.

The mouse pointer scans around the page and then hovers over a post with a ridiculous sound headline: New bear-like creature spotted in suburban neighborhood.

On right side, the image shows a page called Newsblog with the headline "New bear-like creature spotted in suburban neighbourhood", and an image of a suburban neighbourhood.

15 Second Storyboard, Frame 3

15 Second Storyboard, Frame 3. Text version below.

15 Second Storyboard, Frame 3 - Text Version

This page shows a blue box on the left side with text inside.

An absurdly-large share button suddenly appears over the story. It glows and pulses as the mouse pointer hovers above it.

On right side, the image shows a page called Newsblog with the headline "New bear-like creature spotted in suburban neighbourhood", a big blue button with the word "share" in white in it, and an image of a suburban neighbourhood.

15 Second Storyboard, Frame 4

15 Second Storyboard, Frame 4. Text version below.

15 Second Storyboard, Frame 4 - Text Version

This page shows a blue box on the left side with text inside.

Then, from out-of-nowhere, a bear pops up into the frame – interrupting the video.

With the sound of a mouse-click, the bear nods directly at the viewer... then...

On right side, the image shows a page called Newsblog with the headline "New bear-like creature spotted in suburban neighbourhood" partially hidden by a big bear, and an image of a suburban neighbourhood.

15 Second Storyboard, Frame 5

15 Second Storyboard, Frame 5. Text version below.

15 Second Storyboard, Frame 5 - Text Version

This page shows a blue box on the left side with text inside.

...bursts into song.

If you're not sure

if it's not true

Doooooooooooooooon't

Share it!

And the screen wipes to the next frame.

On right side, the image shows a page called Newsblog with the headline "New bear-like creature spotted in suburban neighbourhood" partially hidden by a big bear singing "If you're not sure if it's not true, doooooooooooon't share it!", and an image of a suburban neighbourhood.

Below the image, another image showing bars as a sound track says: click here to hear the jingle.

15 Second Storyboard, Frame 6

15 Second Storyboard, Frame 6. Text version below.

15 Second Storyboard, Frame 6 - Text Version

This page shows a blue box on the left side with text inside.

A very clear endline animates on screen: Real disinformation doesn't come with a timely interruption or a jungle. Followed by the call-to-action: Learn how to spot disinformation, fact check it, and stop its spread at Canada.ca/disinformation.

On right side, the image shows a white page with the text: Real disinformation doesn't come with a timely interruption or a jungle. Learn how to spot disinformation, fact check it, and stop its spread at Canada.ca/disinformation.

15 Second Storyboard, Frame 7

15 Second Storyboard, Frame 7. Text version below.

15 Second Storyboard, Frame 7 - Text Version

This page shows a blue box on the left side with text inside.

The official Canada wordmark resolves on the screen. On right side, the image shows a white page with the Canada wordmark.

Static Facebook Post

Static Facebook Post. Text version below.

Static Facebook Post - Text Version

This page shows a blue box on the left side with text inside.

Image copy: If you're not sure if it's true, don't share it. Fact-check before you click.

Post copy (above): Learn how to spot disinformation, fact-check it, and stop its spread.

Headline & description (below): You can identify disinformation. Free tips and tools.

On right side, the image shows a telephone with the heading and text: Learn how to spot disinformation, fact-check it, and stop its spread. "If you're not sure if it's true, don't share it." Fact-check before you click.

The Canada wordmark image appears below.

And then the text: "You can identify disinformation. Free tips and tools" with the image of a button saying: "learn how".

On the middle-right side, an image shows part of the top head (hair) of a bear.

Alternate 15 Second Video Images (Finger)
15 Second Storyboard, Frame 4

15 Second Storyboard, Frame 4. Text version below.

15 Second Storyboard, Frame 4 - Text Version

This page shows a blue box on the left side with text inside.

Then, from out-of-nowhere, a finger dressed as a bear pops up into the frame, interrupting the video.

With the sound of a mouse-click, the bear nods directly at the viewer... then...

On right side, the image shows a page called Newsblog with the headline "New bear-like creature spotted in suburban neighbourhood" partially hidden by a finger dressed as a bear, and an image of a suburban neighbourhood.

15 Second Storyboard, Frame 5

15 Second Storyboard, Frame 5. Text version below.

15 Second Storyboard, Frame 5 - Text Version

This page shows a blue box on the left side with text inside.

...bursts into song.

If you're not sure

if it's not true

Doooooooooooooooon't

Share it!

And the screen wipes to the next frame.

On right side, the image shows a page called Newsblog with the headline "New bear-like creature spotted in suburban neighbourhood" partially hidden by a finger dressed as a bear singing "If you're not sure if it's not true, doooooooooooon't share it!", and an image of a suburban neighbourhood.

Below the image, another image showing bars as a sound track says: click here to hear the jingle.

Alternate Facebook Post (Finger)
Static Facebook Post

Static Facebook Post. Text version below.

Static Facebook Post - Text Version

This page shows a blue box on the left side with text inside.

Image copy: If you're not sure if it's true, don't share it. Fact-check before you click.

Post copy (above): Learn how to spot disinformation, fact-check it, and stop its spread.

Headline & description (below): You can identify disinformation. Free tips and tools.

On right side, the image shows a telephone with the heading and text: Learn how to spot disinformation, fact-check it, and stop its spread. "If you're not sure if it's true, don't share it." Fact-check before you click.

The Canada wordmark image appears below.

And then the text: "You can identify disinformation. Free tips and tools" with the image of a button saying: "learn how".

On the middle-right side, an image shows a finger dressed as a bear.

Concept 2: What is Dis?

15 Second Storyboard, Frame 2

15 Second Storyboard, Frame 2. Text version below.

15 Second Storyboard, Frame 2 - Text Version

This page shows a blue box on the left side with text inside.

We open on a generic social feed as a few typical sounding social posts scroll past. The scrolling slows as this post appears: "This post is a lie, but something about it is believable, so you should share it."

The animation looks like it would if you were scrolling through a feed on your phone.

On right side, the image shows a post on a screen saying: This post is a lie, but something about it is believable, so you should share it.

Below, two posts from two different people but the text is not legible.

15 Second Storyboard, Frame 3

15 Second Storyboard, Frame 3. Text version below.

15 Second Storyboard, Frame 3 - Text Version

This page shows a blue box on the left side with text inside.

The post is pulled back and centers up on the screen. On right side, the image shows a post on a screen saying: This post is a lie, but something about it is believable, so you should share it.

15 Second Storyboard, Frame 4

15 Second Storyboard, Frame 4. Text version below.

15 Second Storyboard, Frame 4 - Text Version

This page shows a blue box on the left side with text inside.

The post is surrounded by a large highlighted square, as this question animates on screen: What is dis?

On right side, the image shows a post on a screen with a large highlighted yellowish square saying: What Is Dis?

15 Second Storyboard, Frame 5

15 Second Storyboard, Frame 5. Text version below.

15 Second Storyboard, Frame 5 - Text Version

This page shows a blue box on the left side with text inside.

The text animates, slides over and re-assembles to reveal our primary message: This is disinformation.

On right side, the image shows a post on a screen with a large highlighted yellowish square saying: This is disinformation.

15 Second Storyboard, Frame 6

15 Second Storyboard, Frame 6. Text version below.

15 Second Storyboard, Frame 6 - Text Version

This page shows a blue box on the left side with text inside. A very clear endline animates on screen: Social media platforms favour popularity over fact. Followed by the call-to-action: Learn how to spot disinformation, fact check it, and stop its spread at Canada.ca/disinformation.

On right side, the image shows a white page with the text: Social media platforms favour popularity over fact. Learn how to spot disinformation, fact check it, and stop its spread at Canada.ca/disinformation.

15 Second Storyboard, Frame 7

15 Second Storyboard, Frame 7. Text version below.

15 Second Storyboard, Frame 7 - Text Version

This page shows a blue box on the left side with text inside.

The official Canada wordmark resolves on the screen. On right side, the image shows a white page with the Canada wordmark.

Static Facebook Post

Static Facebook Post. Text version below.

Static Facebook Post - Text Version

This page shows a blue box on the left side with text inside.

Image copy: Can you be sure this post is legit?

Post copy (above): Disinformation hides in plain sight, grabs attention, and hopes that you won't fact-check before sharing. But you can stop it.

Headline & description (below): What is dis? This is disinformation.

On right side, the image shows a telephone with the heading and text: Disinformation hides in plain sight, grabs attention, and hopes that you won't fact-check before sharing. But you can stop it. "Can you be sure this post is legit?" And then the text: "What is dis? This is disinformation." with the image of a button saying: "learn how".