Public Opinion Research Study on Electoral Matters - Wave 1 - Appendix A: Methodology

Printable PDF version

Quantitative Methodology

Quantitative research was conducted using online surveys and computer-assisted web interviewing technology.

As a member of the Canadian Research Insights Council, Léger adheres to the most stringent guidelines for quantitative research. The survey was conducted in accordance with Government of Canada requirements for quantitative research, including the Standards for the Conduct of Government of Canada Public Opinion Research—Series D—Quantitative Research.

Respondents were assured of the voluntary, confidential and anonymous nature of this research. As with all research conducted by Léger, all information that could allow for the identification of participants was removed from the data, in accordance with the Privacy Act.

The questionnaire is available in Appendix B.

Sampling Procedure

Léger conducted a panel-based Internet survey with a sample of adult Canadians. A total of 2,582 respondents participated in the survey. The exact distribution is presented in the following section. Participants were selected randomly from the Leger Opinion online panel.

Léger operates a proprietary Internet panel of more than 400,000 Canadians from coast to coast. An Internet panel is made up of web users profiled based on certain socio-demographic variables. The majority of Léger's panel members (61%) have been recruited randomly over the phone over the past decade, making it highly similar to the actual Canadian population on many demographic characteristics.

Data Collection

Fieldwork for the survey was conducted from April 1, 2021, to April 11, 2021. The participation rate for the survey was 16%. A pretest of 65 interviews was completed on April 1, 2021.

To achieve data reliability in all subgroups, a total sample of 2,582 Canadians who are eligible voters were surveyed, in all regions of the country.

Since a sample drawn from an Internet panel is not probabilistic in nature, the margin of error for this survey cannot be calculated. Respondents were selected from among those who have volunteered to participate or registered to participate in online surveys. The results of such surveys cannot be described as being statistically projectable to the target population. The data were weighted to reflect the demographic composition of the target population. Because the sample was based on those who initially self-selected for participation, no estimates of sampling error could be calculated.

Based on data from Statistics Canada's 2016 national census, Léger weighted the results of this survey by age and gender in each region of the country.

The following table details the regional distribution of respondents. The baseline sample attempted to replicate as closely as possible the actual distribution of the Canadian population.

Participation Rate

Table A2 breaks down the calculation of the web survey's participation rate. The overall response rate for this study was 16%. The participation rate is calculated using the following formula: participation rate / response rate = R ÷ (U + IS + R).

Typical participation rates for web surveys are between 20% and 30%. A response rate of 16% may seem a bit low, but given the limited amount of time for fieldwork, we had to spread the invitations more widely among panel members to achieve our objectives; this had an impact on the participation rate.

Unweighted and Weighted Samples

A basic comparison of the unweighted and weighted sample sizes was conducted to identify any potential non-response bias that could be introduced by lower response rates among specific demographic subgroups (see tables below).

Table A3 presents the geographic distribution of respondents before and after weighting. The weighting adjusted for some discrepancies due to quotas that had been placed on certain regions, such as the Atlantic region and the Prairies, to obtain a sufficient sample. Therefore, the weighting minimized the weight of those regions and slightly increased the weight of Quebec and Ontario.

Tables A4 and A5 present the demographic distribution of respondents according to gender and age.

First, regarding gender, we can see that weighting slightly adjusted the proportion of men and women. These adjustments are minor, and there is no reason to believe that the small differences observed in the effective samples could have introduced a non-response bias for either of these two sample subgroups.

Regarding age, the weighting process corrected some minor discrepancies. The distribution of the sample generally follows the distribution of age groups in the actual population. In this case, it is unlikely that the observed distributions introduced a non-response bias for a particular age group. Because the differences were so small, weighting allowed the weights to be corrected without further manipulation.

There is no evidence from the data that having achieved a different age or gender distribution before weighting would have significantly changed the results for this study. The relatively small weighting factors and differences in responses among the various subgroups suggest that data quality was not affected. The weighting that was applied corrected the initial imbalance for data analysis purposes, and no further manipulations were necessary.

Tables A6 and A7 present the weighting factors applied to the database according to the different respondent profiles.

• Previous
• Table of Contents
• Next