Supplier Name: Leger Marketing Inc
Contract Number: B8815-23-0100
Contract Value: $107,143.21 (including HST)
Award Date: 2022-08-08
Delivery Date: 2023-03-29
Registration Number: POR 026-22
For more information on this report, please contact IRCC at:
IRCC.COMMPOR-ROPCOMM.IRCC@cic.gc.ca
Ce rapport est aussi disponible en français.
This public opinion research report presents the results of two studies conducted by Léger Marketing Inc. on behalf of Immigration, Refugees and Citizenship Canada (IRCC). The first study consisted in a qualitative research with nine online focus groups and was conducted with 55 Canadians between October 18 and October 20, 2022. The second study was a quantitative study consisting in a baseline survey and a post-campaign survey conducted with 2,502 respondents each.
Cette publication est aussi disponible en français sous le titre : Évaluation des exécutions publicitaires de la campagne Communautés inclusives & Outil d’évaluation de campagnes publicitaires (OECP).
This publication may be reproduced for non-commercial purposes only. Prior written permission must be obtained from Immigration, Refugees and Citizenship Canada. For more information on this report, please contact Immigration, Refugees and Citizenship Canada at IRCC.COMMPOR-ROPCOMM.IRCC@cic.gc.ca or at:
Communications Branch
Immigration, Refugees and Citizenship Canada
Jean Edmonds Tower South
365 Laurier Ave W
Ottawa ON K1A 1L1
Catalogue Number: Ci4-248/2023E-PDF
International Standard Book Number (ISBN): 978-0-660-67843-6
Related publications (registration number: POR 026-22):
Catalogue Number: Ci4-248/2023F-PDF (Final Report, French)
ISBN: 978-0-660-67844-3
© His Majesty the King in Right of Canada, as represented by the Minister of Immigration, Refugees and Citizenship Canada, 2023
Leger is pleased to present IRCC with this report on findings from virtual discussion groups which aimed to collect the participants' impressions and comments on three advertising concepts. The report also details the methodology of the quantitative survey designed to evaluate IRCC’s Inclusive Communities campaign using the Advertising Campaign Evaluation Tool (ACET). This report was prepared by Léger Marketing Inc. who was contracted by IRCC (contract number B8815-23-0100 awarded August 8, 2022).
The Inclusive Communities campaign first launched in 2020-21. This year’s advertising campaign aimed to encourage Canadians to engage with immigrants, understand their contributions to the Canadian economy and local communities and get involved in welcoming newcomers into their communities for everyone’s benefit. The tactics used included digital, search engine marketing (SEM), audio ads (e.g., radio and Spotify), print ads, and out-of-home ads.
By being inclusive, communities can help newcomers (immigrants and refugees) feel welcomed. Demonstrating the benefits of immigration at the local, community level, dispelling common myths about immigration and refugees, and promoting positive engagement between Canadians and newcomers is vital to ensuring that newcomers are welcomed, and communities are inclusive.
The main objective of the qualitative phase of this research was to determine which creatives (and elements of those creatives) most positively resonate with the target audience. The feedback from this research provided guidance to IRCC on how best to construct effective communications and marketing material which will increase awareness of the advertising campaign and increased intention or action to welcome newcomers. Specifically, the creative testing aimed to:
The main objective of the quantitative study was to measure awareness, recall, and impact of the ads among the target audience. The surveys used the Government of Canada’s Advertising Campaign Evaluation Tool and consisted of two waves: a baseline wave conducted before the campaign launched and a wave after the campaign had been running for at least three weeks. The ACET assessed:
Leger conducted a series of nine virtual discussion group sessions with French-speaking and English-speaking Canadians recruited from all the regions in Canada aside from the province of Quebec. The groups were conducted among two targets: Canadians from 30 to 75 years of age, who are likely to engage in community improvement, and established immigrants. Eight participants were recruited by our professional recruiters for each discussion group session (see Appendix 2 under separate cover for screening guide).
The main purpose of the focus groups was to test three different ad concepts and to collect comments and impressions (see Appendix 2 under separate cover for tested materiel). Each group saw the three concepts in a different order to reduce fatigue bias. A total of 55 recruits participated in the virtual discussion groups. All participants received an honorarium of $100.
| Group Detail | Date | Recruits | Participants | Language |
|---|---|---|---|---|
| Group 1 – Group with Canadians aged 30-75 who are likely to engage in community improvement, GTA | October 18, 2022 (6 PM EST) | 8 | 7 | English |
| Group 2 – Group with Canadians aged 30-75 who are likely to engage in community improvement, Prairies urban centres | October 18, 2022 (7:15PM EST) | 8 | 6 | English |
| Group 3 – Group with Canadians aged 30-75 who are likely to engage in community improvement, Vancouver | October 18, 2022 (8:30 PM EST) | 8 | 6 | English |
| Group 4 – Group with Canadians aged 30-75 who are likely to engage in community improvement, Atlantic | October 19, 2022 (6:00 PM EST) | 8 | 5 | English |
| Group 5 – Group with Canadians aged 30-75 who are likely to engage in community improvement, Ontario (smaller centers) | October 19, 2022 (7:15 PM EST) | 8 | 7 | English |
| Group 6 – Group with Canadians aged 30-75 who are likely to engage in community improvement, Western Canada smaller centers | October 19, 2022 (8:30PM EST) | 8 | 8 | English |
| Group 7 – Group with Canadians aged 30-75 who are likely to engage in community improvement, Francophones outside of Quebec | October 20, 2022 (6PM EST) | 8 | 5 | French |
| Group 8 – Group with established immigrants, small centers | October 20, 2022 (7:15PM EST) | 8 | 6 | English |
| Group 9 – Group with established immigrants, larger centers | October 20, 2022 (8.30PM EST) | 8 | 5 | English |
The virtual discussion group sessions were approximately 60 minutes in duration and were conducted by a moderator using the CMNTY online platform. The choice of the platform helped to facilitate the moderation, ensure an optimal interface between moderator and participants, and enable interaction as the discussion unfolded. The platform also allowed the moderator to share advertising concepts with participants to lead each part of the discussion. A polling question on the preferred advertising concept was also integrated. The online platform also allowed for remote viewing of each session by Leger and IRCC observers.
All focus group sessions were moderated and supervised by a senior Leger researcher assisted by a research analyst. The discussion guide (see Appendix 2 under separate cover) consisted of a semi-structured discussion guide. It allowed the moderator to follow the thread of the discussion while leaving sufficient room for the participants to express themselves and develop in detail their ideas, opinions and perceptions.
Quantitative research was conducted through online surveys, using Computer Aided Web Interviewing (CAWI) technology. The questionnaires used for both phases of the study are the Government of Canada ACET questionnaires. Leger Marketing Inc. used these questionnaires without making any modifications to their format nor to their content, except for the modifications necessary to adapt the questions to this present study on the Inclusive Communities ad campaign requested by Immigrations, Refugees and Citizenship Canada.
The targeted audience of this study included a national sample reflecting the 30-75 year old Canadian population outside the Province of Quebec. Both the baseline and post-campaign surveys were conducted with a sample of around 2,500 Canadian adults. The respondents were recruited via Leger’s web panel using a random selection stratified by region. The general adult population sample was distributed as follows:
| Region | Number of respondents – baseline study | Number of respondents – post test study |
|---|---|---|
| Ontario | 1,265 | 1,233 |
| British Columbia + Territories | 441 | 426 |
| Alberta | 352 | 347 |
| Atlantic region | 222 | 227 |
| Manitoba | 119 | 121 |
| Saskatchewan | 103 | 148 |
| Total | 2,502 | 2,502 |
This public opinion research was conducted via online surveys, using Computer Aided Web Interviewing (CAWI) technology. Fieldwork for the survey was carried out from December 12 to 23, 2022. A total of 2,502 Canadians aged between 30 and 75 living outside the province of Quebec were surveyed. The sample was drawn randomly from the Leo panel and the overall response rate for the survey was 18.5% (see Table 2 in the Appendix for the calculation details).
Using data from the most recent Canadian census, the weighting was done within each region by gender, age, language, presence of children in the household and education to ensure the best possible representativeness of the sample within each region and overall. The weight of each region was adjusted to be equivalent to its actual weight in relation to the distribution of the Canadian population. The weighting factors are presented in detail in the appendix of this report.
A pre-test of 49 interviews was completed before launching data collection to validate the programming of the questionnaire in both English and French. An average of two minutes and thirty one seconds was required for the respondents to complete the survey.
Leger adheres to the most stringent guidelines for quantitative research. The survey instrument was compliant with the Standards of Conduct of Government of Canada Public Opinion Research.
A complete methodological description is provided in the Appendices section of this document (please see Appendix A).
This public opinion research was conducted via online surveys, using Computer Aided Web Interviewing (CAWI) technology. Fieldwork for the survey was carried out from March 6th to 24th, 2023. A total of 2,502 Canadians aged between 30 and 75 living outside the province of Quebec were surveyed. The sample was drawn randomly from the Leo panel and the overall response rate for the survey was 17.8% (see Table 11 in the Appendix for the calculation details).
Using data from the most recent Canadian census, the weighting was done within each region by gender, age, language, presence of children in the household and education to ensure the best possible representativeness of the sample within each region and overall. The weight of each region was adjusted to be equivalent to its actual weight in relation to the distribution of the Canadian population. The weighting factors are presented in detail in the appendix of this report.
A pre-test of 33 interviews was completed before launching data collection to validate the programming of the questionnaire in both English and French. An average of four minutes and fifty-five seconds was required for the respondents to complete the survey.
Leger adheres to the most stringent guidelines for quantitative research. The survey instrument was compliant with the Standards of Conduct of Government of Canada Public Opinion Research.
A complete methodological description is provided in the Appendices section of this document (please see Appendix A).
Leger recommends moving forwards with the concept: Canadian Welcome. This was the concept that was most appreciated by the participants and that was the easiest to understand. However, some changes should be made to improve the concept. The scene in the airport should be changed, because it did not convince the participants and they did not seem to understand what the government was asking them to do. The images used for social media were also criticized for looking old and unappealing. Nevertheless, the concept was the one that spoke most to the Canadian-born and immigrant participants.
Based on the discussions, it seems that the key to the appreciation of the campaign is:
More details on the different concepts are presented below:
Warmest welcome
Canadian Welcome
Everyday Heroes
Website Landing Page
The contract value was $107,143.21 (HST included).
The views and observations expressed in this document do not reflect those of IRCC. This report was compiled by Leger based on the research conducted specifically for this project.
Qualitative research is designed to reveal a rich range of participants’ opinions, perceptions and interpretations. It does not and can not measure what percentage of the target population holds a given opinion or perception. Findings are qualitative in nature and cannot be used quantitatively to estimate the numeric proportion or number of individuals in the population who hold a particular opinion.
The results of the quantitative research use a sample drawn from an internet panel which is not probabilistic in nature. As a result, the margin of error cannot be calculated for this survey and the results cannot be described as statistically projectable to the target population.
Leger certifies that the final deliverables fully comply with the Government of Canada’s political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications.
Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.
Signed:
Christian Bourque, Senior Researcher
Léger
Two versions of the Warmest place concept were presented. Participants saw only one version of the concept. The two versions were randomly distributed across the groups. Since the differences between the two versions were minor, no specific comments regarding each version were raised by the participants.
Most participants agreed that the main message of this concept was to be more welcoming of others coming to Canada, to ask Canadians to welcome newcomers, to help out, to support each other or that there are many different ways to help new Canadians.
This concept was mainly appreciated by participants who agreed that Canadians are welcoming and warm. They globally liked the concept and the fact that it showed different ways to get involved and help out newcomers. One participant said that “it might get the wheels moving if you were not already thinking about [getting involved]”. Many established immigrants recognized their experience with this concept, and some mentioned that “it gives a lot of pride for people living here” or that “it makes you want to be here [in Canada]”. They also said that it showed that regardless of where you come from, there are always going to be people ready to help you throughout the settling process.
On the other hand, several less appreciated elements that need to be improved were discussed throughout the groups. The main point brought up by both Canadians and established immigrants was that the message could be confusing. Some were wondering whether the goal of the concept was to ask Canadians to be more welcoming towards newcomers or to advertise to newcomers to move to Canada. Following the same idea, many mentioned not knowing what the government wanted from them with that concept: “What is the goal? Whatever the reason is, it is not stated.” The part about donating was seen as out of touch and frustrating for some participants: “The donate part confuses me. Especially at a time when Canadians are struggling with money, is the government really asking us for more money? I find that troubling.” Many said that the mention of donation should either be taken out of the advertisement, or further detailed with types of donations (time, clothes, food, etc.). They feel like the government can ask Canadians to volunteer but asking them for money is too much.
The emphasis on the weather also received mixed responses from participants. Although some participants seemed to enjoy the transition from winter to summer within the concept, stating that it accurately represented one of the main characteristics of Canada, many others expressed their discontent toward this element. Two main criticisms were raised: 1) Canada is already known for being cold, so there is no need for advertising it and 2) it shows a negative image of Canada, as if the winter is the same as “in Antarctica” and that Canadians are not social during that time of the year. According to some participants, winter should be presented in a positive light rather than just the other dull side of the coin to summer. Some participants felt that the Canadian winter should be valued. Also, the focus on the cold weather could discourage people from moving to Canada. This concept only shows winter and summer, and some participants felt that not showing fall and spring did not show the full range of weather we have in Canada: “newcomers are going to feel like it is only a two-season country.” Some other participants also criticized the weather transition aspect as being somewhat “tacky” or “cliché”.
The wording of this concept also raised some concerns both in English and in French, especially for the radio script. Many participants from the established immigrants group mentioned that some newcomers might not understand the play on words used with warm versus cold or that the play on words made them feel that the government was trying to sell them something. Others mentioned that they felt this concept was a bit wordy and long. It was suggested to use simpler language. Regarding the social media ads, many participants enjoyed the warmth and welcoming image they gave off. However, some participants felt like the pictures used were not natural and suggested that real images from newcomers' experiences should be used. The third picture was seen as somewhat confusing or unnecessary among the participants who mostly regarded it as taken from an office party rather than a moment of welcoming newcomers.
Regarding the radio script and the video concept, some mentioned the fact that they would not search for more info since there was no clickable link. It was suggested to add more concrete ways to help directly within the ad.
Out of the three concepts, Canadian Welcome was the most appreciated, especially the radio script. Participants liked that the message was “direct”, “inviting”, “clear”, “kind”, “respectful” and “serious” at the same time. The message was well understood by participants who mostly agreed that this concept was targeted at Canadians to encourage them to volunteer to help newcomers from the moment they arrive, but also targeted newcomers to show them that Canadians are welcoming and are ready to help.
Many participants mentioned that some parts in the radio script really caught their attention and made them want to take some actions, notably the sentences: “newcomers come for many reasons” and “every newcomer deserves a real Canadian welcome.” Some established immigrants also added that this type of message appealed to them since it reminded them of their own story and made them want to give back to others. This type of sentence also resonated with Canadians who felt they now have a better understanding of the situation of newcomers: “hearing it on the radio would make me feel empathy toward the immigrants and their long journey to come here.” A few Francophone participants mentioned that they would like to know what a true Canadian welcome was (véritable accueil canadien in French). They felt like it was repeated many times in the ads but were unsure of what it implied.
Regarding the video script, many visual aspects were appreciated by respondents. The colours and the fact that the light goes from dark to light and sunny when the arrivals door opens was liked by most participants. Some also mentioned that the facial expressions of the members of the family in the storyboard made them feel empathy and would make them want to help more. The fact that concrete actions were presented in this concept was appreciated by the participants who felt that this was a missing element in the other concepts presented. However, a majority of respondents were unsure of the scene at the airport. They were wondering if welcoming newcomers directly at the airport was the best way to help them: “the whole airport cheer thing just seems out of place and strange. It is more important to help out your neighbours feel welcome, and everyday things rather than welcoming people at the airport.” Many believed that this is not a feasible action, especially those who do not live near an airport. Many participants suggested that adding other scenes showcasing Canadians welcoming newcomers in a community or neighbourly setting would make the concept more appealing, realistic and relatable. Participants from the established immigrant groups felt compelled by the reference to the fact that the journey to Canada can be long. However, many noted that it did not represent the welcome they received upon arrival in Canada: “it conveys this idea that the welcome is at the airport and that’s where it ends, but that’s where it begins. The important part of the volunteering and welcoming is after they arrive. Refugees might get welcomed at the airport but others probably not.”
Across all groups, many participants felt like the radio script, the video storyboard and the social media banners were not coherent in terms of visuals and messages: “to me, they’re two different stories, the script and the social media banners.” Others mentioned that it was the best script, but the worst video among all concepts. Regarding the social media banners, many pointed out that the pictures used in this concept seemed outdated and of low quality. One participant mentioned that depending on the type of platform, specifically if it is a visual app like Instagram, the ad will look like a typical government ad in contrast with the other content available on social media. Another mentioned that the imagery was “bland” and “boring” and that it did not catch his attention.
This last concept was appreciated by some participants who found it warm and appealing. A majority felt that the message was clear and that the concept invited Canadians to volunteer to help newcomers. However, some participants felt that the message was confusing. Most participants understood the message as being: “Canadians can be heroes if they helped welcome newcomers,” “helping newcomers is a superhero quality everyday Canadians can have”, or simply to be welcoming to newcomers. A participant from the established immigrant group mentioned that it will make newcomers feel good about coming to Canada. The fact that the message was short and to the point was highly appreciated by participants. However, some were not sure to whom this advertisement was addressed. The superhero theme felt like it was aimed more at children, younger people or at families more than at individuals. For that reason, many did not feel personally engaged and would not have clicked on a link to find out more.
The sense of community and friendliness that emerges from the video script was much appreciated by participants who felt that it brought light to the fact that Canadians are welcoming: “It makes it more community-based. I think Canadians feel proud of their openness and this ad hits that mark.” On the other hand, the concept of superheroes was criticized across all groups, mainly because participants felt like this expression was overused during the COVID-19 pandemic. This term was regularly used to refer to health care workers and since the pandemic has negatively impacted the lives of these workers, reusing this term did not sit well with some participants. A majority mentioned that they are not paying attention to that kind of message anymore. One participant from Ontario even said that the theme of the COVID-19 childhood vaccination campaign in her province was also superheroes, so she thought they were the same ads. Others were unsure of the use of superheroes for different reasons. Some mentioned that: “immigrants don’t need saving or heroes, they just need support” and others mentioned that helping someone does not make them a superhero.
Regarding the text in the radio script and in the video storyboard, the phrase Now that’s super! (ça, c’est super! in French) was judged as tacky by participants in both English and French groups. The call to action: Donate, volunteer, learn, celebrate received a variety of responses. For some, it was better than the other concepts because it implied that you could give some of your time instead of just giving money. One participant said that this message would “kick him in the butt” to check out the website. For others, it felt like the government was giving them orders and telling them how to be good neighbours, which felt inappropriate to them. With regard to the sentence: “Canadians who donate and volunteer are making a huge difference,” some felt like it should be changed to “could be making a huge difference” to make it more encouraging for Canadians.
Most participants felt like the social media banners were not coherent. A majority mentioned that the third one (the one with the child wearing a superhero costume) was too different from the first two. Once again, participants felt like the ad was aimed at children or parents of young children and they would have preferred to see an adult dressed as a superhero. Many also mentioned that the pictures seem staged, did not feel like it was representing Canada and that it was hard to tell what was going on: “It seems very generalized and vague.” Only a few respondents, mainly in the established immigrants’ group, said that the social media banner ads would make them click on the link to learn more.
In general, the website content matches with what participants would have expected by clicking on the link. Some participants mentioned that it has a very “governmental feel” to it, mostly meaning that it was clean, basic, simple and user-friendly, but not very creative nor exciting. One participant stated that the landing page felt very cold and unengaging, contrasting with the concepts’ messages which were largely based on warmth and conviviality. They suggested that the landing page’s feel should be consistent with the concepts, which they thought was not at all the case currently. Most participants would have liked to read more about ways to volunteer and donate directly on the page and not to have to click on another link to go to another website. It was suggested by some to add easy actions that Canadians could take to help directly on the landing page.
The fact that Volunteer and Donate are at the top was liked by most participants because it makes a direct link to the ads they just saw. One participant particularly liked the Match with a newcomer link because it made her feel like they could really make a difference.
The main critique expressed by participants across all groups was that the website showed only two specific groups of immigrants: Afghans and Ukrainians. Even if the participants acknowledged the fact that these groups are the ones who need the most help at the moment, some found it strange that the government put the focus exclusively on them on their website.
Quantitative research was conducted through online surveys, using Computer Aided Web Interviewing (CAWI) technology.
As a CRIC Member, Leger adheres to the most stringent guidelines for quantitative research. The survey was conducted in accordance with Government of Canada requirements for quantitative research, including the Standards of the Conduct of Government of Canada Public Opinion Research - Online Surveys .
Respondents were assured of the voluntary, confidential and anonymous nature of this research. As with all research conducted by Leger, all information that could allow for the identification of participants was removed from the data, in accordance with the Privacy Act.
The questionnaire is available in Appendix A2.
Computer Aided Web Interviewing (CAWI)
Leger conducted a panel-based Internet survey with a sample of adult Canadians. The same sampling procedure was used for both the baseline and the post-campaign phases of the study. The exact distribution for each of the two phases of the study is presented in the following section. Participant selection was done randomly from Leo’s online panel.
Leger owns and operates an Internet panel of more than 400,000 Canadians from coast to coast. An Internet panel is made up of Web users profiled on different sociodemographic variables. The majority of Leger’s panel members (61%) have been recruited randomly over the phone over the past decade, making it highly similar to the actual Canadian population on many demographic characteristics.
Fieldwork for the survey was conducted from December 12 to 23, 2022. The participation rate for the survey was 18.5%. A pre-test of 49 interviews was completed on December 12, 2022.
To achieve data reliability in all subgroups, a total sample of 2,502 Canadians aged 30-75 living outside the province of Quebec were surveyed.
Since a sample drawn from an Internet panel is not probabilistic in nature, the margin of error cannot be calculated for this survey. Respondents for this survey were selected from among those who have volunteered to participate/registered to participate in online surveys. The results of such surveys cannot be described as statistically projectable to the target population. The data have been weighted to reflect the demographic composition of the target population. Because the sample is based on those who initially self-selected for participation, no estimates of sampling error can be calculated.
Based on data from Statistics Canada’s 2021 national census, Leger weighted the results of this survey by age, gender, language, education and presence of children in the household within each region of the country.
The following table details the regional distribution of respondents. The baseline sample attempted to replicate as closely as possible the actual distribution of the Canadian population.
| Region | Number of respondents |
|---|---|
| Ontario | 1,265 |
| British Columbia + Territories | 441 |
| Alberta | 352 |
| Atlantic region | 222 |
| Manitoba | 119 |
| Saskatchewan | 103 |
| Total | 2,502 |
The overall participation rate for this study is 18.5%.
Below is the calculation of the Web survey’s participation rate. The participation rate is calculated using the following formula: Participation rate / response rate = R ÷ (U + IS + R). The table below provides details of the calculation.
| Invalid cases | |
|---|---|
| Invitations mistakenly sent to people who did not qualify for the study | 203 |
| Incomplete or missing email addresses | - |
| Unresolved (U) | 10,944 |
| Email invitations bounce back | 13 |
| Email invitations unanswered | 10,931 |
| In-scope non-responding units (IS) | 115 |
| Non-response from eligible respondents | - |
| Respondent refusals | 51 |
| Language problem | - |
| Selected respondent not available (illness; leave of absence; vacation; other) | - |
| Early breakoffs | 64 |
| Responding units (R) | 2,504 |
| Surveys disqualified – quota filled | 2 |
| Completed surveys disqualified for other reasons | - |
| Completed interviews | 2,502 |
| Potentially eligible (U+IS+R) | 13,563 |
| Participation rate= R/(U + IS + R) | 18.5% |
Typical participation rates for web surveys are between 20% and 30%. A response rate of 18.5% may seem a bit low, but due to time restraints, we had to spread the invitations more widely in the panel to achieve our objectives, which has an impact on the participation rate.
A basic comparison of the unweighted and weighted sample sizes was conducted to identify any potential non-response bias that could be introduced by lower response rates among specific demographic subgroups (see tables below).
The table below presents the geographic distribution of respondents, before and after weighting. The weighting adjusted for some minor discrepancies so that each province has a representative importance in the results.
| Region | Weighted | Unweighted |
|---|---|---|
| ON | 1,250 | 1,265 |
| BC + Territories | 455 | 441 |
| AB | 371 | 352 |
| ATL | 224 | 222 |
| MB | 109 | 119 |
| SK | 93 | 103 |
| Total | 2,502 | 2,502 |
The following tables present the demographic distribution of respondents, according to gender and age.
First, regarding gender, we can see that weighting has adjusted slightly the proportion of male and female. The adjustments made by weighting are minor, and in no way can we believe that the small differences observed in the effective samples could have introduced a non-response bias for either of these two sample subgroups.
| Gender | Weighted | Unweighted |
|---|---|---|
| Male | 1,205 | 1,267 |
| Female | 1,286 | 1,225 |
| Total | 2,502 | 2,502 |
Note: Gender-diverse people and answer refusals make up the rest of the sample.
Regarding age distribution, the weighting process has corrected some minor discrepancies. The actual distribution of the sample generally follows the distribution of age groups in the actual population. In this case, it is unlikely that the observed distributions introduce a non-response bias for a particular age group. Because the differences were so small, weighting allowed the weights to be corrected without further manipulation.
| Age | Weighted | Unweighted |
|---|---|---|
| 30-44 | 864 | 772 |
| 45-54 | 547 | 602 |
| 55-64 | 599 | 531 |
| 65-75 | 493 | 597 |
| Total | 2,502 | 2,502 |
There is no evidence from the data that having achieved a different age or gender distribution prior to weighting would have significantly changed the results for this study.
The following tables present the demographic distribution of respondents according to native language, the presence of children in the household, and education.
| Native language | Weighted | Unweighted |
|---|---|---|
| English | 2,241 | 2,187 |
| French | 82 | 139 |
| Other | 268 | 300 |
| Total | 2,502 | 2,502 |
Note: As multiple answers were possible, total may exceed the total sample size.
| Presence of children in the household | Weighted | Unweighted |
|---|---|---|
| Yes | 737 | 707 |
| No | 1,765 | 1,795 |
| Total | 2,502 | 2,502 |
| Education level | Weighted | Unweighted |
|---|---|---|
| High school or less | 678 | 514 |
| College | 1,029 | 794 |
| University | 795 | 1,194 |
| Total | 2,502 | 2,502 |
Again, the corrections were minor and there is no evidence that they would have had an impact on the results.
The relatively small weight factors and differences in responses between various subgroups suggest that data quality was not affected. The weight that was applied corrected the initial imbalance for data analysis purposes and no further manipulations were necessary.
Fieldwork for the survey was conducted from March 6th to 24th, 2023. The participation rate for the survey was 17.8%. A pre-test of 33 interviews was completed on March 6th, 2023.
To achieve data reliability in all subgroups, a total sample of 2,502 Canadians aged 30-75 living outside the province of Quebec were surveyed.
Since a sample drawn from an Internet panel is not probabilistic in nature, the margin of error cannot be calculated for this survey. Respondents for this survey were selected from among those who have volunteered to participate/registered to participate in online surveys. The results of such surveys cannot be described as statistically projectable to the target population. The data have been weighted to reflect the demographic composition of the target population. Because the sample is based on those who initially self-selected for participation, no estimates of sampling error can be calculated.
Based on data from Statistics Canada’s 2021 national census, Leger weighted the results of this survey by age, gender, language, education and presence of children in the household within each region of the country.
The following table details the regional distribution of respondents. The baseline sample attempted to replicate as closely as possible the actual distribution of the Canadian population.
| Region | Number of respondents |
|---|---|
| Ontario | 1,233 |
| British Columbia + Territories | 426 |
| Alberta | 347 |
| Atlantic region | 227 |
| Manitoba | 121 |
| Saskatchewan | 148 |
| Total | 2,502 |
The overall participation rate for this study is 17.8%.
Below is the calculation of the Web survey’s participation rate. The participation rate is calculated using the following formula: Participation rate / response rate = R ÷ (U + IS + R). The table below provides details of the calculation.
| Invalid cases | 354 |
|---|---|
| Invitations mistakenly sent to people who did not qualify for the study | 354 |
| Incomplete or missing email addresses | - |
| Unresolved (U) | 11,660 |
| Email invitations bounce back | 17 |
| Email invitations unanswered | 11,643 |
| In-scope non-responding units (IS) | 117 |
| Non-response from eligible respondents | - |
| Respondent refusals | - |
| Language problem | - |
| Selected respondent not available (illness; leave of absence; vacation; other) | - |
| Early breakoffs | 117 |
| Responding units (R) | 2,549 |
| Surveys disqualified – quota filled | 47 |
| Completed surveys disqualified for other reasons | 0 |
| Completed interviews | 2,502 |
| Potentially eligible (U+IS+R) | 14,326 |
| Participation rate= R/(U + IS + R) | 17.8% |
Typical participation rates for web surveys are between 20% and 30%. A response rate of 17.8% may seem a bit low, but due to time restraints, we had to spread the invitations more widely in the panel to achieve our objectives, which has an impact on the participation rate.
A basic comparison of the unweighted and weighted sample sizes was conducted to identify any potential non-response bias that could be introduced by lower response rates among specific demographic subgroups (see tables below).
The table below presents the geographic distribution of respondents, before and after weighting. The weighting adjusted for some minor discrepancies so that each province has a representative importance in the results.
| Region | Weighted | Unweighted |
|---|---|---|
| ON | 1,248 | 1,233 |
| BC + Territories | 455 | 426 |
| AB | 370 | 347 |
| ATL | 223 | 227 |
| MB | 112 | 121 |
| SK | 95 | 148 |
| Total | 2,502 | 2,502 |
The following tables present the demographic distribution of respondents, according to gender and age.
First, regarding gender, we can see that weighting has adjusted slightly the proportion of male and female. The adjustments made by weighting are minor, and in no way can we believe that the small differences observed in the effective samples could have introduced a non-response bias for either of these two sample subgroups.
| Gender | Weighted | Unweighted |
|---|---|---|
| Male | 1,218 | 1,266 |
| Female | 1,278 | 1,230 |
| Total | 2,502 | 2,502 |
Note: Gender-diverse people and answer refusals make up the rest of the sample.
Regarding age distribution, the weighting process has corrected some minor discrepancies. The actual distribution of the sample generally follows the distribution of age groups in the actual population. In this case, it is unlikely that the observed distributions introduce a non-response bias for a particular age group. Because the differences were so small, weighting allowed the weights to be corrected without further manipulation.
| Age | Weighted | Unweighted |
|---|---|---|
| 30-44 | 868 | 770 |
| 45-54 | 547 | 487 |
| 55-64 | 599 | 561 |
| 65-75 | 489 | 684 |
| Total | 2,502 | 2,502 |
There is no evidence from the data that having achieved a different age or gender distribution prior to weighting would have significantly changed the results for this study.
The following tables present the demographic distribution of respondents according to native language, the presence of children in the household, and education.
| Native language | Weighted | Unweighted |
|---|---|---|
| English | 2,196 | 2,163 |
| French | 121 | 141 |
| Other | 285 | 300 |
| Total | 2,502 | 2,502 |
Note: As multiple answers were possible, total may exceed the total sample size.
| Presence of children in the household | Weighted | Unweighted |
|---|---|---|
| Yes | 753 | 616 |
| No | 1,749 | 1,886 |
| Total | 2,502 | 2,502 |
| Education level | Weighted | Unweighted |
|---|---|---|
| High school or less | 650 | 519 |
| College | 1,052 | 837 |
| University | 800 | 1,146 |
| Total | 2,502 | 2,502 |
Again, the corrections were minor and there is no evidence that they would have had an impact on the results.
The relatively small weight factors and differences in responses between various subgroups suggest that data quality was not affected. The weight that was applied corrected the initial imbalance for data analysis purposes and no further manipulations were necessary.
English and French quantitative and qualitative instruments are provided under separate cover.