POR Number: POR 086-17
Contract Number: 5R000-171980/001/CY
Award Date: 2018-01-26
Delivery Date: 2018-04-11
Department Contact: info@fcac-acfc.gc.ca
Contract Value: $33,448.00
Financial Consumer Agency of Canada
Final Methodological Report
Prepared by: Ipsos Public Affairs
1 Nicholas Street, Suite 1400
Ottawa ON K1N 7B7
Tel: 613.241.5802
Fax: 613.248.7981
www.ipsos.ca
Ce sommaire est aussi disponible en français.
I hereby certify as Senior Officer of Ipsos that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.
Mike Colledge
President
Ipsos Public Affairs
The research consisted of a quantitative online survey with a national sample of Canadians aged 18 years and older. In total, n=2,034 online surveys were completed taking place between February 12th and February 26th, 2018. The survey was conducted in English and French.
For this survey, rim weighting[1] was employed to balance demographics by three variables (Gender, Age and Region) to ensure that the survey sample’s composition reflects that of the adult population according to 2016 Census data and to provide results intended to approximate the sample universe.
Ipsos programmed, hosted and provided sample management services only, while the Financial Consumer Agency of Canada provided the online survey. Ipsos was responsible for data collection and data storage in Canada, data processing, and data weighting. The survey was registered with the National Survey Registration System (20180201-924G). The total contract value of this research was $33,448.00 including HST.
Fielding this online survey is an opportunity to advance and deepen understanding of financial literacy and financial wellbeing in Canada and prepare for better policy initiatives to improve financial security for households and individuals. It will help to identify the types of financial knowledge, financial behaviour, and psychological factors most closely tied to financial wellbeing outcomes in order to better target financial literacy initiatives.
Respondents for this survey were selected from among those who have volunteered to participate in online surveys. Because the sample is based on those who initially self-selected to participate, no estimates of sampling error can be calculated. The results of such surveys cannot be described as statistically projectable to the target population.
The purpose of this research is to establish quantitative evidence about financial literacy and financial wellbeing in Canada.
Specific research objectives include collecting information on:
Data on these factors can be compared with results from other countries which have conducted, or will conduct, this international survey. Originally developed by Elaine Kempson (Director of the Personal Finance Research Centre and Emeritus Professor of Personal Finance and Social Policy Research at the University of Bristol) and her colleagues at Oslo and Akershus University College of Applied Sciences, the survey was first conducted in Norway in 2017. At the time of writing the survey had also been carried out in Ireland, Australia, and New Zealand. While in each case some modification of questions was necessary to align with the country context, the results are largely comparable.
The Ipsos i-Say panel is the primary source for recruiting respondents. Where necessary, a supplemental sample from third party vendor panels is also used as a source. The Ipsos i-Say online panel is a diversely-sourced and actively maintained panel of over 200,000 Canadian adults. Utilizing this panel allows us to confidently build online samples based on known population characteristics like the ones available through Census data. Invitations to participate in each survey are adjusted to reflect the most recent Current Population Data provided by Statistics Canada.
Online panel participants are recruited and screened using rigorous double and triple opt-in checks[2] to ensure that respondents want to receive contact such as invitations to participate in research surveys. Ipsos i-Say panel Terms and Conditions of Membership details the respondents’ rights to, at any time, for any reason ask to be removed from the panel and stop receiving emails from Ipsos and its clients.
Panel recruitment consists of the following steps:
“wide net”methodologies (e.g., email campaigns, affiliate networks, banner ads, text ads, search engine, co-registration, offline-to-online, specialized websites); and,
It is important to note that any panel – no matter how well recruited – does not enable random selection in the sense that not all members of the public are apt to join a panel. This applies to panels recruited through online approaches as well as panels recruited via traditional telephone Random Digit Dialing.
Rigorous panel maintenance procedures mean only those participants who actively participate in online surveys, at least once every six months, are included. Participants can choose whether to enter a survey or not. While participating in a survey, the respondent can simply exit the survey by closing the survey Internet browser window at any time without submitting their responses if they no longer want to participate.
To ensure a high participation rate from our panelists, the i-Say panel provides a number of innovative incentive programs. Ipsos’ i-Say online panel participants are incentivized through a points program whereby respondents collect points that can be redeemed for rewards of their choice from a collection of products from online retailers such as gift cards and prizes. Each incentive is tailored to the specific requirements of each survey, depending on the length of the survey, the subject matter of the study, and the time required to complete a minimum number of interviews.
A sampling frame was devised to outline a quota structure representative of Canadian adults 18 years and older. All Canadian adults aged 18 years and older were eligible to participate.
The Ipsos i-Say online panel consists of over 200,000 Canadian adults recruited through diverse sources and is constantly refreshed. Background data gathered when respondents join the panel allows us to finely target respondents on the basis of key criteria such as region (national samples are typically divided into over 40 submarkets nationwide ensuring accurate distributions within provinces), age, and gender. The results of quota based sampling against these criteria allow us to construct samples that are reflective of Canadian population characteristics. In this instance, sampling quotas were set according to the respondents’ region, gender, and age to match a target profile based on Census 2016 data. This serves to underpin the representativeness of the survey results.
All sample surveys and polls may be subject to sources of error, including, but not limited to coverage error and measurement error. Traditional margins of error don’t apply to online panel-based samples of general public audiences as the traditional margin of error calculation assumes that the sample measured is a probability sample. As there is no way to measure the probability that any given member of the public may join or be a part of an online panel, these samples are inherently non-probabilistic.
The precision of online polls is measured using a credibility interval. Bayesian Credibility Intervals measure the degree of certainty one has in the results based on one’s experience, understanding and knowledge of the population, tempered by the data that has been observed. The credibility interval for this survey is +/-2.5%.
Non-Response Analysis: There exists within the current sample the possibility of non-response bias. In particular, this survey would not include members of the population who do not have access to the Internet or who are not capable of responding to a survey in either English or French. In addition, some groups within the population are systemically less likely to answer surveys.
Extrapolation: The table below compares the unweighted sample to the 2016 Census results by region, age, and gender, as well as the unweighted geographical and demographic distribution of the sample.
| Canadian adults aged 18+ |
Unweighted Sample Size |
Sample Proportions | Census 2016 Proportions |
|---|---|---|---|
| Canada | 2,034 | 100% | - |
| Region | |||
| Newfoundland and Labrador | 22 | 1.10% | 1.53% |
| Prince Edward Island | 5 | 0.20% | 0.41% |
| Nova Scotia | 39 | 1.90% | 2.71% |
| New Brunswick | 30 | 1.50% | 2.18% |
| Québec | 516 | 25.40% | 23.47% |
| Ontario | 675 | 33.20% | 38.41% |
| Manitoba | 93 | 4.60% | 3.51% |
| Saskatchewan | 65 | 3.20% | 3.00% |
| Alberta/North | 285 | 13.90% | 11.21% |
| British Columbia | 304 | 14.90% | 13.57% |
| Gender | |||
| Male | 905 | 44.50% | 48.57% |
| Female | 1129 | 55.50% | 51.43% |
| Age | |||
| 18 -24 | 208 | 10.20% | 10.94% |
| 25 - 29 | 155 | 7.60% | 8.12% |
| 30 - 34 | 178 | 8.80% | 8.28% |
| 35 - 44 | 334 | 16.40% | 16.14% |
| 45 - 54 | 376 | 18.50% | 17.91% |
| 55 - 64 | 355 | 17.50% | 17.47% |
| 65 - 100 | 428 | 21.00% | 21.14% |
The following table provides the response dispositions and response rate calculation, as per the MRIA’s empirical method of calculating response rates for online surveys.
| Calculation for Data Collection[3] | Panel (iSay) | Other Panel | Total |
|---|---|---|---|
| Total Email Invitations Issued | 28363 | 2055 | 30418 |
| Unresolved (U) (no response) | 22891 | 0 | 22891 |
| In-scope - non-responding (IS) | 360 | 56 | 416 |
| Qualified respondent break-off (incomplete) | 360 | 56 | 416 |
| In-scope - Responding units (R) | 5112 | 1999 | 7111 |
| Over quota | 1425 | 421 | 1846 |
| Other disqualified | 1905 | 1326 | 3231 |
| Completed Interviews | 1782 | 252 | 2034 |
| Response Rate = R/(U+IS+R) | 18% | 97% | 23% |
Sampling is carried out via a proprietary sampling application that allows us to construct complex samples based on the target and screening requirements. To develop the survey sample and minimize non-response, we use a large and diversely sourced panel. In addition, we have the capability of broadening the sample frame further through the inclusion of other actively managed panel resources as needed to ensure the sample requirements are met among the hardest-to-reach audiences (such as younger males).
[1] Rim (Random Iterative Method) weighting is used to adjust the sample to ensure it is representative of the target population on two or more characteristics at the same time (i.e. gender, age and region).
[2] The Ipsos online panel is recruited utilizing a double and triple opt-in screening process to ensure maximum return from an engaged and representative audience. After the intital opt-in to the Ipsos online panel, respondents are sent a second and third email to confirm they want to join our panel.
[3] ‘Unresolved (U)’ indicates no response received to email invitation; ‘In-scope - non-responding (IS)’ indicates qualified respondents who started but did not complete the survey (incomplete); ‘In-scope – responding units (R)’ indicates qualified respondents who started the survey but were disqualified as over-quota or disqualified for other reasons. Valid completes are counted as ‘Completed interviews’.