Leger is pleased to present Veterans Affairs Canada with this report on findings from the qualitative and quantitative surveys on pre-testing VAC’s 2018 services advertising concepts.

This report was prepared by Leger who was contracted by Veterans Affairs Canada (contract number 51019-172010/001/CY awarded February 6, 2018).

1.1 Background and Objectives

Veterans Affairs Canada (VAC) has the mandate to provide support and services to Veterans and their families. Veterans can be eligible for a range of benefits and services, including disability awards and pensions, health care, mental health support, as well as compensation for hardships arising from disabilities and lost economic opportunities.

The Government of Canada is committed to supporting those who have served Canada. In Budget 2017, it announced programs focusing on support for Veterans, families and caregivers. VAC will begin providing these services in April 2018. It is a comprehensive package that recognizes the important role of caregivers, helps more families, supports mental health, and funds the education and training Veterans may need to find the work they want in their post-military lives.

VAC has had a successful Remembrance advertising campaign for a number of years and is committed to implementing an innovative and successful advertising campaign in 2018 to promote these services for Veterans and those who care for them. To that end, they have engaged in testing concepts and approaches to services advertising.

The purpose of the 2018 advertising campaign is to improve awareness among the Veteran community (Veterans, family members, CAF members) of benefits and services available to them and how they can be easily accessed. Emphasis will be placed on increasing the awareness of the diversity and breadth of support and services available from Veterans Affairs Canada. The campaign will also encourage target audiences to seek information about the support and services and to apply for benefits they may be entitled to.

Objectives

The purpose of this survey is to:

· provide direction for the creative concepts insofar as the approaches resonate with the target audiences;

· ensure the creative concepts meet the advertising campaign’s objectives;

· offer a foundation for future campaigns; and,

· provide a creative platform for short- and long-term advertising development.

1.2 Methodology - Hybrid Research

To achieve the study objectives, a research plan based on a hybrid method, qualitative and quantitative, was developed. First, a qualitative methodology consisting of focus groups with the main targets of the advertising campaigns (Veterans and their family members) was set up and secondly, a quantitative methodology consisting of a Web survey with the general population in Canada has been done. Both methodologies were used to test the advertising concepts with actual targets of the advertising campaign and to validate the research insights.

1.2.1 Qualitative Methodology

Leger conducted a series of six focus groups with Veterans and six with close family members or caregivers of Veterans. There were two of each group type in each of three cities across Canada: Halifax and Winnipeg in English, and Quebec City in French. All sessions were held in professional focus group facilities. Leger used a mixed method of recruitment using its internal database of focus group participants, telephone recruitment, and contacts supplied by VAC’s stakeholders in each city where groups were to be conducted. The definition of Veteran used for recruitment was “an individual who has completed basic training and has been honourably discharged.” For families and caregivers, recruitment targeted family members like spouse, child or grandchild of a Veteran.

Each group session lasted approximately 100 minutes. Every session was recorded for analysis purpose. All groups used streaming methodology to allow for remote viewing by Leger and Government of Canada observers.

Leger was responsible for recruiting participants, preparing the recruitment and moderation guides, managing the logistical and technical aspects of the project (facilities, audio, video recording), moderating the groups, and reporting verbally in English. The moderator’s guide was developed in consultation with VAC’s project authority.

All participants in the focus group received an honorarium of $100. Participant incentive payment signature sheets were used and a summary outlining the total number of participants and corresponding incentives paid were completed and signed by each focus group host. These signed sheets, with family names removed to protect participants’ confidentiality, were submitted to VAC along with relevant invoices. The recruitment screener informed participants of all their rights under Canada’s Privacy Act and the Standards for the Conduct of Government of Canada Public Opinion Research. Specifically their confidentiality was guaranteed, and their participation was considered voluntary.

Locations and dates

Groups were held in the following cities on the dates specified.

Table 1. Detailed Recruitment

City	Recruits	Participants	Target	Time	Language	Date
Halifax, NS	12	10	Veterans	5:30 p.m.	EN	March 7, 2018
Halifax, NS	12	9	Veterans’ families and caregivers	7:30 p.m.	EN	March 7, 2018
Winnipeg, Man	12	10	Veterans	5:30 p.m.	EN	February 28, 2018
Winnipeg, Man	12	9	Veterans’ families and caregivers	7:30 p.m.	EN	February 28, 2018
Quebec City, QC	12	8	Veterans	5:30 p.m.	FR	March 5, 2018
Quebec City, QC	12	8	Veterans’ families and caregivers	7:30 p.m.	FR	March 5, 2018
Total	72	54

In each of the focus groups, four creative ad concepts were tested in rotation. The four concepts were the following:

Table 2: List of Concepts Tested

Concept A	Man in uniform holding child
Concept B	The transition concept
Concept C	The family in the helping hand
Concept D	The family

1.2.2 Quantitative Methodology

Quantitative research was conducted through online surveys, using Computer Aided Web Interviewing (CAWI) technology.

Fieldwork for the survey was conducted from March 7, 2018 to March 10, 2018. The national participation rate for the survey was 12.97%. Calculation of the Web survey’s participation rate is presented in Appendix A. A pre-test of 61 interviews, in both official languages, was completed on March 7, 2018. More specifically, 30 interviews were conducted in French and 31 were conducted in English. Survey interviews lasted 10 minutes on average.

A total sample of 1,005 Canadian adults, were surveyed in all regions of the country.

Special attention was given to ensure a distribution of respondents that is representative of the general population, while ensuring a minimum number in every region of the country. The following table shows the target distribution of respondents across the regions and the effective sample collected by Leger:

Table 3. Quotas by Region

	Target sample	Effective sample
Regions	n	n
Atlantic	50	56
Québec	250	378
Ontario / Nunavut	350	238
Prairies (including Alberta) / NWT	200	203
British Columbia, Yukon	150	130
TOTAL	1,000	1,005

Based on data from Statistics Canada’s 2016 national census, Leger weighted the results of this survey by age, gender, region, language (mother tongue), education level and the Veterans population of the armed forces.

Since a sample drawn from an Internet panel is not probabilistic in nature, the margin of error cannot be calculated for this survey. Survey data are weighted by age, gender, region, language (mother tongue), education level and the Canadian Veteran population to ensure that they replicate what the latest census would indicate is the composition of the adult population of Canada. Details regarding the weighting procedures can be found in Appendix A.

Below is the calculation of the survey’s participation rate. It should be noted that this is not a classic calculation of the response rate, given the difference in methodology between a phone and an online survey.

Table 4. Participation Rate

Items	n=
Total number of invitations sent	7,747
Total number of clicks on the link	1,129

Number of completed questionnaires	1,005
Number of screened out	7
Number of incompletes	117
Participation rate	12.97%

As a Certified Gold-Seal MRIA Member, Leger adheres to the most stringent guidelines for quantitative research. The survey has been registered with the MRIA in accordance with Government of Canada requirements for quantitative research, including the MRIA Code of Conduct and Standards of the Conduct of Government of Canada Public Opinion Research – Series E – Qualitative and Quantitative Research.

The details of the methodology and more information on Leger’s quality control mechanisms are presented in Appendix A.

The questionnaire is available in Appendix D.

Note to the reader

The cross-tabulations in this report present the detailed responses obtained for the question(s) in the Advertising Pre-Testing of VAC’s 2018 Support and Services Campaign.

1.3 Overview of Qualitative Study Findings

· Generally speaking, Veterans’ level of knowledge of the services offered by VAC varied between those who believed they had a very good understanding of services and benefits for Veterans, to those who say they are largely unaware of VAC’s service offering. The former group tended to be composed of Veterans who said they were actively involved with VAC (related to a specific program or because of a medical condition) and very likely to have a My VAC Account or those who are in close contact with their local Veterans’ organizations or stakeholder groups. At the opposite end, the latter group tended to be made up of somewhat younger Veterans who had not kept in close contact with VAC or any local organization.

· When it comes to families and/or caregivers of Veterans, most claimed to have very limited knowledge when it comes to the services and benefits available for Veterans. The only exceptions tended to be the caregivers of Veterans who say they are directly involved with VAC because of the care they provide to that Veteran. These caregivers tended to be involved in the care of an elderly or a physically disabled Veteran.

· Participants in both the Veteran and family/caregiver groups were then asked to comment and evaluate four creative concepts (shown in random order across all groups) to help guide VAC’s decisions when it comes to creative direction. While the qualitative research is not designed to “elect a winning concept” and indeed the research results would not be able to do just that, two concepts tended to be favoured: Concept B and Concept A.

· Regardless of the concept shown, participants provided us a series of insights on what they ‘want’ to see in a VAC creative concept to increase their awareness of new or modified services and benefits available to Veterans or their families. We believe the following six elements are important to consider:

1) For participants to pay attention to, and appreciate, the ad, it needs to show a clear and evident symbol representing Veterans. In all groups, the presence of the uniform was seen as the best possible symbol, and most point directly at the “CAD PAT” (a uniform common to all branches of the army and worn by every member of the Canadian Armed Forces at some point in their military career). Several also pointed either to the Tri-service crest or the poppy as easily recognizable symbol for them.

2) There was a very strong feeling that the ad concept should include a clear element of patriotism and several felt the ad lacked the red Canadian flag and red maple leaf.

3) The usage of the colour green was also referred to as a military colour which would act as a visual trigger for Veterans.

4) There is a strong sense that the usage of illustration in advertising would be an issue for several Veterans and family members. Only younger Veterans (below the age of 40 roughly) and younger family members said they liked the use of illustration. For most Veterans, they believed using “cartoon” images could be belittling or not adequately represent them as a group.

5) Veterans tended to react somewhat negatively to the usage of actual dollar figures to inform them and their families of some of the services and benefits available to them. Many Veterans expressed concern over the reaction of the public to these figures and believe the public may feel Veterans are “getting too much” or are somewhat “spoiled” by the government.

6) Using only images of people in civilian dress tended to confuse Veterans and their families who believed they may not visually associate the image with services for Veterans; it might lead them to believe it was government advertising for any department and simply skip the message. Although they are now back in civilian society, they want an advertising directed at them to graphically remind them of their military service.

· Concept A (man in uniform holding child): Positive elements from this concept were the usage of the highlight colour green (at the top and bottom), the presence of the uniform and the presence of the poppy of the military man’s lapel. Family members also liked the fact that the text clearly outlined specific information about services and benefits. The phrase “Starting April 1, 2018” also drew a lot of attention due to its immediacy. The usage of “New” in the headline also caught the attention of many participants. Negative perceptions tended to focus on the choice of uniform. Veterans and some family members said the image was clearly a Navy uniform, and some believed Veterans from other branches of the military may not feel represented in the ad. Some also felt that the concept was “too wordy” and did not need to provide this amount of detail.

· Concept B (the transition concept): Positive comments for this concept were for the narrative behind the graphic used. Veterans in particular applauded the concept of the visual, which portrays a person transitioning from a military to a civilian career. They believed this concept was a very adequate representation of their own transition and also liked the fact that the use of the “CAD PAT” uniform was a direct reference to their military service. The use of the “yellow stamp”¾the colour tab on which appears the call to action to contact VAC¾was seen as a plus, and a visual reference they have seen from VAC in the past. This stamp “drew” them in and would likely better trigger a visit. The use of the two text bubbles was also eye-catching. However, the content of the bubbles tended to be a negative among Veterans but a positive among families/caregivers. Negative comments on concept B tended to focus on the graphic itself. Again, most rejected the usage of an illustration and several believe that the features of the figure were not very representative of Veterans (too young-looking). Many Veterans rejected being depicted as “faceless”. While they liked the storyline from the military medic to the civilian paramedic, they rejected the “cartoon” style that was adopted.

· Concept C (family in the helping hand): Main positives for Concept C were its appealing headlines (“New and Enhanced”) and the mention of “to help you after your military service”, which many saw as a key reference to what VAC actually does. However, this concept tended to draw several negative comments. Several felt that the family depicted does not correspond to what they believe is the typical family of a Veteran, and some enquired about the use of a small child and the mention of “caregiver” in the body of the text. This tended to create confusion as to “does it cover daycare for the child?” The main element which tended to be rejected was the use of the “big hand” holding the family. Veterans, in particular, tended to refer to it as the “benevolent hand of government” supporting them and several felt it was “talking down” to them. Most simply said they “did not get it” when it came time to analyzing the hand.

· Concept D (the family): On the positive side, many Veterans liked it because it was short, not too wordy, and the call to action (to call VAC or visit the website) was clear. On the negative side, they could not reconcile the use of the family image with VAC services. Again, it lacked a clear symbol or signal that the ad was addressed to or was about Veterans. Some suggested that the man (the father figure) or the female character in yellow (the mother figure) should be in uniform. Some also felt the ad lacked any patriotic symbol or clearer identification that this was from the Government of Canada.

· When it comes to media placement and how best to reach Veterans and their families, responses were varied. Veterans that are currently engaged with VAC, suggested that the newsletter should be used to show the ad, along with a posting next time they open their VAC Account. Some suggested that the ad be made available through their local Veterans organizations. When it comes to media, Facebook advertising was seen as a very efficient way to reach them, along with public transit advertising. Since the groups were held in major urban areas (Winnipeg, Halifax and Quebec City), advertising on bus shelters and either the interior or exterior of the bus was seen as a great way to ensure they see it.

1.4 Overview of Quantitative Study Findings

· Generally, half of respondents (54%) chose Concept A as their favourite. Four respondents out of ten (40%) also chose its headline “New Benefits and Services for Veterans and their Families,” as the one they prefer. This concept is also the one respondents liked the best (88%). On every evaluated aspect of the concepts, Concept A scores the highest, both on informative and visual criteria.

· The main reason for liking Concept A more than other concepts is because of the information it contains. More specifically, one-quarter of respondents (26%) liked that it is informative and two out of ten (19%) liked that it is clear or straight to the point. Other respondents liked its family theme (19%) or that it is visually appealing (16%). It should also be mentioned that 13% appreciated that it is realistic or shows a real person.

· Concept A is perceived as the best on all evaluated aspects. For a vast majority of respondents it has a clear message that is easy to understand (93%), and is effective (86%) and credible (85%). A majority also agree that it is appropriate for VAC (89%), that it is a good way of showing what VAC is doing for Veterans and their families (89%) and that it gives a positive image of VAC (88%).

· On most aspects, Concept B ranks second. The concept or scenario, describing a transition from soldier to worker (20%) is the primary reason for preferring Concept B more than the other concepts. However, Concept C ranks second when it comes to the headline. Its headline “New and Enhanced Services for Veterans and their Families” was the favourite for 30% of respondents, compared to 18% who preferred “New Education and Training Benefit for Veterans.”

· Respondents who had ideas about visual elements or symbols that could be added to these concepts to indicate that the ads are for Canadian Veterans and their families mainly mentioned a Canadian flag (11%), a maple leaf (3%), people wearing uniforms (6%), a Canadian armed forces logo, or a coat of arms (3%). Other ideas, such as a visual of a Veteran with his or her family (2%) or a poppy (1%) were mentioned by a few respondents only.

· There were no significant differences between general population's opinion and the ones of Veterans, their families, or caregivers.

1.5 How the Results Will Be Used

Since this is a new campaign, it is critical that VAC ensure that the ads resonate with the intended audiences. The results of this research will allow VAC to assess the effectiveness of the creative concepts and make adjustments as required based on feedback from the survey data and analysis. The project sought direction and insight into the clarity, comprehension, content, and formatting of the proposed creative concepts.

1.6 Statement of Limitations

Qualitative research provides insight into the opinions of a population, rather than providing a measure in percent of the opinions held, as would be measured in a quantitative study. The results of this type of research should be viewed as directional only. No inference to the general population can be done with the results of this research.

1.7 Notes on Interpretation of Research Findings

The views and observations expressed in this document do not reflect those of Veterans Affairs Canada. This report was compiled by Leger based on the research conducted specifically for this project. This research is not probabilistic; the results cannot be inferred to the general population of Canada.

1.8 Political Neutrality Statement and Contact Information

I hereby certify as Senior Officer of Leger that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Policy on Communications and Federal Identity and the Directive on the Management of Communications- Appendix C (Appendix C: Mandatory Procedures for Public Opinion Research).

Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leaders.

Signed:

Christian Bourque

Executive Vice-President and Associate

Leger

507 Place d’Armes, bur 700

Montréal, Québec

H2Y 2W8

cbourque@leger360.com

Additional information

Supplier name: Leger

Contract Number: 51019-172010/001/CY

Contract Award Date: 2018-02-06

The cost of this research was $92,900.00 (HST included).

To obtain more information on this study, please email: vac.information.acc@canada.ca

Appendix A – Detailed Research Methodology

A.1 Qualitative Methodology

Leger conducted a series of six focus groups with Veterans and with the close family or caregivers of Veterans in three cities across Canada. There were two of each group type in each of three cities across Canada: Halifax and Winnipeg in English, and Quebec City in French. All six sessions were held in professional focus group facilities. Leger used a mixed method of recruitment using both its internal database of focus group participants using telephone recruitment and using contacts supplied by local stakeholders of VAC in each city visited. The definition of “Veteran” used for recruitment was an individual “who has completed basic training and has been honourably discharged.” For families and caregivers, recruitment ensured that the term family be include only the spouse, child or grandchild of a Veteran (all participants were over 18 years of age).

The groups lasted approximately 100 minutes. Every group were recorded for analysis purpose and all groups used streaming methodology to allow for remote viewing by Leger and Government of Canada observers.

Leger was responsible for recruiting the participants, reviewing the recruitment and moderation guides, managing the technical aspect of the project (facilities and audio, video recording), moderating the groups and reporting in English. The interviewer’s guide was developed in consultation with VAC’s project authority.

The participants in the focus group received an honorarium of $100.

Participant incentive payment signature sheets were used and a summary outlining the total number of participants and corresponding incentives paid were completed and signed by each focus group host. These signed sheets with family names removed to protect participants’ confidentiality, were submitted to VAC along with relevant invoices. The recruitment screener informed participants of all their rights under Canada’s Privacy Act and the Standards for the Conduct of Government of Canada Public Opinion Research. Specifically their confidentiality was guaranteed, and their participation was voluntary.

Locations and dates

Groups were held in the following cities on the dates specified.

Table 1. Detailed Recruitment

City	Recruits	Participants	Target	Time	Language	Date
Halifax, NS	12	10	Veterans	5:30 p.m.	EN	March 7, 2018
Halifax, NS	12	9	Veterans’ families and caregivers	7:30 p.m.	EN	March 7, 2018
Winnipeg, Man	12	10	Veterans	5:30 p.m.	EN	February 28, 2018
Winnipeg, Man	12	9	Veterans’ families and caregivers	7:30 p.m.	EN	February 28, 2018
Quebec City, QC	12	8	Veterans	5:30 p.m.	FR	March 5, 2018
Quebec City, QC	12	8	Veterans’ families and caregivers	7:30 p.m.	FR	March 5, 2018
Total	72	54

In each of the focus groups, four creative ad concepts were tested in rotation. The four concepts were the following:

Table 2: List of Concepts Tested

Concept A	Man in uniform holding child
Concept B	The transition concept
Concept C	The family in the helping hand
Concept D	The family

A.2 Quantitative Methodology

A.2.1 Methods

Quantitative research was conducted through online surveys, using Computer Aided Web Interviewing (CAWI) technology.

As a Certified Gold-Seal MRIA Member, Leger adheres to the most stringent guidelines for quantitative research. The survey was registered with the MRIA in accordance with Government of Canada requirements for quantitative research, including the MRIA Code of Conduct and Standards of the Conduct of Government of Canada Public Opinion Research – Series E – Qualitative and Quantitative Research.

Respondents were assured of the voluntary, confidential and anonymous nature of this research. As with all research conducted by Leger, all information that could allow for the identification of participants was removed from the data, in accordance with the Privacy Act.

The questionnaire is available in Appendix B.

A.2.2 Sampling Procedures

Computer Aided Web Interviewing (CAWI)

Leger conducted a panel-based Internet survey with a sample of adult Canadians. A total of 1,005 respondents participated in the survey. Participant selection was done randomly from LegerWeb’s online panel.

Leger owns and operates an Internet panel of more than 400,000 Canadians from coast to coast. An Internet panel is made up of Web users profiled on different sociodemographic variables. The majority of Leger’s panel members (61%) have been recruited randomly over the phone over the past decade, making it highly similar to the actual Canadian population on many demographic characteristics.

Since an Internet sample (from the panel) is non-probabilistic in nature, the margin of error does not apply.

LegerWeb’s Online Panel

For several years now, Leger had been conducting surveys via the Internet and this method is increasingly preferred because of the speed of operation, the flexibility it allows, especially when presenting visuals, and the reliability of results.

Our testing reveals that results obtained in Web surveys statistically match the results obtained in phone surveys. The validity of online surveys rests on three determining factors: 1) the validity of respondents, 2) the validity of responses and 3) the expertise of a team dedicated to online surveys.

The Validity of Respondents

Our online surveys are conducted with a panel of Internet users. Our panel has 400,000 Canadians and grows each month by 10,000 new panellists. This impressive panel allows us to conduct surveys with the public, but also with specific clienteles. For example, some segments of the population are more difficult to reach, such as people who travel a great deal, workers, young people, consumers of specific products, and so on. In addition, this volume provides close sample control and strict quality control measures.

To ensure the most representative respondents for our online surveys, we have developed a proven sampling method. By cross tabulating data related to gender, age, language and region, we can establish a target group, minimize margins of error, and provide a better respect of quotas or eligibility criteria.

Internet users who make up our panel were recruited through complementary methods: telephone recruitment, e-mail invitations, links between sites and invitations using a reference system.

Data from our panel is secured though a private system accessed with a password that panellists receive when invited to answer an online survey.

To foster participation and increase response rates, each panellist invited to answer a questionnaire online has a chance to win a prize drawn each month.

Moreover, to guarantee respondent quality, the LegerWeb team manages the panel database on a continual basis.

The Validity of Responses

By answering an online survey, participants feel they represent an anonymous portion of a targeted group, providing reassurance regarding response confidentiality and validity. Also, the questionnaire must be designed with a minimum number of clicks and a reasonable number of questions because experience has shown that after a certain amount of time, respondents lose interest and no longer concern themselves with answering properly.

In addition, a pre-test with thirty Internet users is essential to catch any skip errors between questions, to respect screening criteria, and to check navigation fluidity from one question to the next, as well as interview duration. During the pre-test, panellists are invited to provide comments after having answered the questionnaire. This information is not only very useful, but may also be used to improve the questionnaire. Client representatives involved in the project participate in this validation process by testing the questionnaire online.

A process of elimination is applied to avoid keeping a questionnaire in which question fatigue ratios are detected at the end of the questionnaire.

The Expertise of a Team Dedicated to Online Surveys

The LegerWeb team includes about ten professionals and technicians who specialize in information technology and e-marketing and who ensure follow-up from 8h00 AM to 7h00 PM every day of the week. Most members of our team come from the fields of telephone polling or face-to-face interviews, and draw on their mastery of the art of the interview.

With its customer support, the LegerWeb team provides panellists with all the technical information and assistance they may need when answering an online survey.

The LegerWeb team works in close collaboration with the research and statistics teams and shares all pertinent information concerning a polling project. Constant communication allows us to proceed very quickly, to detect errors or problems as soon as they arise and to resolve them in record time.

Software developed by the LegerWeb team to conduct online surveys was designed to allow maximum flexibility, efficiency and security when administering the questionnaire. Furthermore, the team performs continuous monitoring of each online survey.

Leger uses the most recent innovations in the field of online surveys and acts in full knowledge of the limits and possibilities of research performed through online surveys.

Finally, it should be specified that Leger strives to develop its expertise in online polling by implementing the same quality standards and criteria from its renowned telephone surveys.

Also, for all our quantitative research projects, Leger has established quality control measures that are identical to the ISO process, in which all stages are verified, allowing us to verify previous stages as well. In practical terms, the quality assurance process is based on the following elements:

· Designating a project manager responsible for final product quality to avoid diluting responsibility internally;

· Scrupulously verifying how well client objectives match the final questionnaire, making sure that each dimension is found in the questionnaire;

· Verifying how each question is formulated, from the perspective of simplicity of expression and the unequivocal meaning of the syntax according to the specific idea to be covered during the interview;

· Verifying the effects of contamination a priori, i.e., that the location of each question in the survey overall does not have undue effects on the following responses (generally by providing information indirectly to respondents, making the sample un-representative);

· Closely verifying the computerized version of the questionnaire with the reference questionnaire approved by the client;

· Before the pre-test, verifying programmed skips in the computerized system;

· Conducting a pre-test, which allows to verify comprehension of questions and concepts, possible ambiguities, and logical question skips, etc.;

· The highly vigilant LegerWeb team detects all questions with problems while on field;

· The use of software prevents data entry errors, non-established skips, etc. Logical validation is therefore done beforehand and not after the fact;

· Coding of open-ended questions is done with a first sample selection of responses on file and by setting up the codes, which are submitted to the client for approval.

We consider that the implementation of all these procedures is a guarantee of optimal quality when conducting online surveys.

Data Cleaning

Upon completion of data collection, Leger’s data analysts and data processing department cleaned the data thoroughly, ensuring that:

· all closed-ended questions were within the allowable or logical range (allowable ranges would be confirmed with the client under all circumstances, whether or not it is obvious from the questionnaire);

· outliers were verified and, if necessary, excluded from the data;

· all skip patterns had been followed correctly;

· the data was complete (except where it is intentional and within client expectations); and

· information was consistent and logical across questions, with no contradictions in the data.

The data was checked and cleaned after the first night of field and at project completion. During analysis, all numbers were double-checked and any outliers are double-checked to ensure the data has been entered accurately in the first place

A.2.3 Data Collection

Fieldwork for the survey was conducted from March 7, 2018 to March 10, 2018. The national participation rate for the survey was 12.97%. A pre-test of 61 interviews was completed on March 7, 2018. More specifically, 30 interviews were conducted in French and 31 were conducted in English. Survey interviews lasted 10 minutes on average.

To achieve data reliability in all subgroups, a total sample of 1,005 Canadians were surveyed, in all regions of the country. Since a sample drawn from an Internet panel is not probabilistic in nature, the margin of error cannot be calculated for this survey.

Based on data from Statistics Canada’s 2016 national census, Leger weighted the results of this survey by age, gender, region, language (mother tongue), education level and the Canadian Veteran.

A.2.4 Participation Rate for the Web Survey

The overall participation rate for this study is 12.97%.

Below is the calculation of the Web survey’s participation rate. It should be noted that this is not a classic calculation of the response rate, given the difference in methodology.

Table 5. Participation Rate

ITEMS	n=
Total number of invitations sent	7,747
Total number of clicks on the link	1,129

Number of completed questionnaires	1,005
Number of screened out	7
Number of incompletes	117
Participation rate	12.97%

A.2.5 Non-Response Bias and Additional Socio-Demographic Analysis

An effective response rate of 12.97% may seem low for a national Web survey of 1,005 respondents. However, it was conducted in a short period (less than a week) making it similar to typical rates for studies usually conducted in two weeks. This response rate minimizes the risks of a significant non-response bias in the survey as it decreases the likelihood of an unrepresentative sample.

Nevertheless, a basic comparison of the unweighted and weighted sample sizes was conducted to identify any potential non-response bias that could be introduced by lower response rates among specific demographic subgroups (see tables below).

A.2.6 Unweighted and Weighted Samples

The table below presents the geographic distribution of respondents, before and after weighting. There were almost no imbalances in geographical distribution in the unweighted sample. The weighting process has mainly adjusted the weight of Quebec, which had been slightly over-represented in the sample because the Veterans, their families and caregivers were target in cities such as Val-Cartier and St-Jean, in order to reach a minimum sample.

Table 6. Unweighted and Weighted Sample Distribution by Province

Province / Territory	Unweighted	Weighted
Atlantic	56	69
Québec	378	236
Ontario / Nunavut	238	386
Prairies (including Alberta) / NWT	203	178
British Columbia, Yukon	130	136

The following tables present the demographic distribution of respondents, according to gender, age, language (mother tongue), education and household income. First, regarding gender, we can see that weighting has adjusted slightly the proportion of male and female. Male were a little bit overrepresented in this survey. But we can see, in the following table, the required adjustment was minor.

Table 7. Unweighted and Weighted Sample Distribution by Gender

GENDER	Unweighted	Weighted
Male	517	488
Female	488	517

Regarding age distribution, the weighting process has mainly adjusted the weight of young Canadians younger than 35 years old, which had been slightly under-represented in the sample.

Table 8. Unweighted and Weighted Sample Distribution by Age Group

AGE	Unweighted	Weighted
Between 18 and 34	191	274
Between 35 and 55	334	343
55 years old and over	480	388

Minor imbalances with language distribution were corrected with weighting, as presented below.

Table 9. Unweighted and Weighted Sample Distribution by First Language

LANGUAGE (MOTHER TONGUE)	Unweighted	Weighted
French	324	209
English	561	566
Other	119	229

The table below presents the distribution of respondents identifying as Veterans. A normally distributed sample would include about 24 Veterans (2.35%), but would not allow for an adequate analysis of the data with a high level of precision. Since VAC wanted to have greater accuracy when analyzing the answers of this target sample, Leger overrepresented them in the final sample. Because this oversample creates an imbalance in the final sample, the weighting has adjusted the proportion of Veterans and non-Veterans targets.

Table 10. Unweighted and Weighted Sample Distribution by Veteran

TARGET SAMPLE	Unweighted	Weighted
Veteran	50	24
Other	955	982

The last tables present the distribution based on education, although this variable was not included in the weighting procedures.

Table 11. Unweighted and Weighted Sample Distribution by Education Level

EDUCATION	Unweighted	Weighted
Elementary / High school	239	337
College	333	291
University	427	369

There is no evidence from the data that having achieved a different age or gender distribution prior to weighting would have significantly changed the results for this study. The relatively small weight sizes and differences in responses between various subgroups suggest that data quality was not affected. The weight that was applied corrected the initial imbalance for data analysis purposes and no further manipulations were necessary.

As with all research conducted by Leger, contact information was kept entirely confidential and all information that could allow for the identification of participants was removed from the data, in accordance with Canada’s Privacy Act.

A.2.7 Weighting Factors

Some subgroups are sometimes under- or overrepresented in a sample compared to the general population. The weighting of a sample makes it possible to correct the differences that exist in the representation of the various subgroups of this sample compared to what is usually observed in the entire population under study. The weighting factors are therefore the weight given to each respondent corresponding to a subgroup of the sample.

The following tables present the weight accorded to each target of the sample.

Table 12. Weight by Gender and Age

GENDER BY AGE	Weight
Male AND Between 18 and 24	0.0555
Male AND Between 25 and 34	0.0816
Male AND Between 35 and 44	0.0791
Male AND Between 45 and 54	0.0881
Male AND Between 55 and 64	0.0854
Male AND 65 or older	0.0959
Female AND Between 18 and 24	0.0531
Female AND Between 25 and 34	0.0827
Female AND Between 35 and 44	0.0826
Female AND Between 45 and 54	0.0912
Female AND Between 55 and 64	0.0894
Female AND 65 or older	0.1154
Total	1.000

Table 13. Weight by Gender and Region

GENDER BY REGION	Weight
...A man AND NL/PE	0.0094
...A man AND NS	0.0129
...A man AND NB	0.0106
...A man AND QC	0.1143
...A man AND ON	0.1848
...A man AND MB	0.0172
...A man AND SK	0.0148
...A man AND AB	0.0559
...A man AND BC	0.0658
...A woman AND NL/PE	0.0101
...A woman AND NS	0.0141
...A woman AND NB	0.0113
...A woman AND QC	0.1203
...A woman AND ON	0.1990
...A woman AND MB	0.0180
...A woman AND SK	0.0153
...A woman AND AB	0.0563
...A woman AND BC	0.0699
Total	1.000

Table 14. Weight by Region and Language

REGION BY LANGUAGE	Weight
QC AND French	0.1809
QC AND English	0.0196
QC AND Other	0.0341
ROC* AND French	0.0271
ROC AND English	0.5434
ROC AND Other	0.1949
Total	1.000

*ROC stands for Rest of Canada (in this table, Quebec and ROC)

Table 15. Weight by Education Level

EDUCATION LEVEL	Weight
Elem. / Soc.	0.343
College	0.29
University	0.367
Total	1.000

Table 16. Weight by Region and Veteran

REGION BY VETERAN	Weight
ATLANTIC AND Yes	0.0033
ATLANTIC AND No	0.06504
QC AND Yes	0.00433
QC AND No	0.23036
ON AND Yes	0.00855
ON AND No	0.37542
MB/SK AND Yes	0.00136
MB/SK AND No	0.06379
AB AND Yes	0.0025
AB AND No	0.10964
BC AND Yes	0.00335
BC AND No	0.13236
ATLANTIC AND Yes	0.0033
Total	1.000

Appendix B – Screening Guide

ENGLISH FOCUS GROUP SCREENER

Government of Canada Study with Veterans, Family Members, and Caregivers

Hello/Bonjour

Would you like to continue this discussion in English? / Voulez-vous continuer cette conversation en français?

[SI le langage de préférence est le français : une personne parlant français vous recontactera pour poursuivre cette discussion]

I'm ___________ of Leger, a marketing research company. We are organizing a research project on behalf of the Government of Canada. This research project is about some topics relevant to the federal government.

We will be holding a few research sessions with people like yourself. In these sessions, individuals are asked to sit down with several others and give their opinions and share their ideas. We are organizing several of these group discussions, and would be interested in possibly having you participate.

Your participation is voluntary. All information collected, used and/or disclosed will be used for research purposes only and the research is entirely confidential. The names of participants will not be provided to the government or any third party. May I continue?

Yes 1

No 2 Thank and terminate

Thank and terminate

I need to ask you a few questions to see if you fit the profile of people we are looking for in this research. This will just take about 5 minutes of your times.

Screening criteria

We are holding focus groups to show creative concepts to Veterans of the Canadian Armed Forces, their immediate family (spouse, children over 18) and their caregivers and listen to their opinions. By Veterans we mean any person who has completed basic training and been honourably discharged. Would you qualify to participate into this study?

In which category do you qualify? Is it...

Profile Groups Number of participants

CAF Veterans Qualify for Group 1 -3 -5 10 per groups

Veterans' family members Qualify for Group 2 -4 -6 10 per groups

Caregivers Qualify for Group 2 -4 -6

THANK AND TERMINATE IF THE PERSON DOES NOT QUALIFY IN ANY PROFILE

Note to recruiter: When terminating a call with someone, say: Thank you for your cooperation. We already have enough participants who have a similar profile to yours, so we are unable to invite you to participate.

1) Record gender:

Male	1	Ensure a good mix
Female	2	Ensure a good mix

2) Do you, or does anyone in your household, work in any of the following areas? (Read list)

	No	Yes
A marketing research firm	( )	( )
A magazine or newspaper	( )	( )
An advertising agency or graphic design firm	( )	( )
A political party	( )	( )
A radio or television station	( )	( )
A public relations company	( )	( )
Federal government (Crown Corporations count as NO)	( )	( )

If ‘yes’ to any, thank and terminate.

3-a) Sometimes participants are asked to watch videos, read a document and/or write out their answers to a questionnaire during the discussion. Is there any reason why you could not participate?

Yes 1 THANK & TERMINATE

No 2

3-b) The discussion groups that we organize will gather around 10 people. Would you say that you feel comfortable sharing your opinion with a group of about ten people?

Yes

No THANK AND TERMINATE

TERMINATE IF RESPONDENT OFFERS ANY REASON WHICH SUGGESTS THEY MAY BE UNABLE TO LOOK AT AND/OR PROVIDE THEIR FEEDBACK ON CREATIVE CONCEPTS SHOWN AND DESCRIBED TO THEM.

4) We would like to talk to people in different age groups. Into which one of the following groups should I place you? (Read List - Good distribution of age groups)

Under 18 1 Thank and terminate

18-24 2

25-34 3 Veterans: maximum of participants between 35 and 54

35-44 4

45-54 5

55-64 6

65 or over 7

5) What is the highest level of education you have attained?

(Do not read list – Good distribution of education levels)

Some high school or less	1
Completed high school	2
Post-secondary technical training	3
Basic military training	4
Some college/university	5
Completed college/university	6
Other training (i.e. officer training)	7
Post-graduate studies	8

6) What is your current employment status?

(Recruit a good distribution of occupation)

Working full time 1

Working part time 2

Self-employed / freelance work 3

In between careers 4

Student 5

Homemaker 6

Unemployed 7

Retired 8

I prefer not to answer 9

7) What was your household’s income for 2017? Was it…

(Recruit a good distribution of household income)

$19,999 or less	1
Between $20,001 and $39,999	2
Between $40,000 and $59,999	3
Between $60,00 and $79,999	4
Between $80,00 and $99,999	5
$100,000 and above	6

8) As I mentioned to you earlier, we are organizing some discussion groups of people like yourself. Have you ever taken part in such market research discussion groups?

Yes	1
No	2	Go to Q.11

9) And when was the last time you attended a market research discussion group?

6 months ago or less

Thank and terminate

OR more than 6 months ago

10) How many times did you attend a market research discussion group or in-depth interview in the last five years?

Less than 5 times	1
OR 5 times or more	2	Thank and terminate

11) Thank you. We would like to invite you to participate in one of our group discussions. The discussion group will last 2 hours. Refreshments will be provided at the meeting, and you will be paid $100 for your participation.

Would you be willing to participate in one of these sessions?

Yes	1	Go to invitation
No	2	Thank and terminate

GR	City	Recruits	Participants	Target	Time	Language	Tentative Date
1	Halifax, NS	12	8-10	Veterans	5:30 p.m.	EN	March 7, 2018
2	Halifax, NS	12	8-10	Veterans’ families and caregivers	7:30 p.m.	EN	March 7, 2018
3	Winnipeg, Man	12	8-10	Veterans	5:30 p.m.	EN	February 28, 2018
4	Winnipeg, Man	12	8-10	Veterans’ families and caregivers	7:30 p.m.	EN	February 28, 2018
5	Quebec City	12	8-10	Veterans	5:30 p.m.	FR	March 5, 2018
6	Quebec City	12	8-10	Veterans’ families and caregivers	7:30 p.m.	FR	March 5, 2018
	Total	92	48-60

“Contact” Section

Someone from our office will be calling you back to confirm these arrangements. Could I please have your phone number where we can reach you during the evening and during the day?

Name:________________________________________________________________________

Evening phone:________________________ Work phone:___________________________

Thank you very much!

Recruited by:__________________________________________________________________

Confirmed by:_________________________________________________________________

As we are only inviting a small number of people to take part, your participation is very important to us. If for some reason you are unable to participate, please call so that we can get someone to replace you. You can reach us at ____ at our office. Please ask for ____.

Privacy Questions

Now I have a few questions that relate to privacy, your personal information and the research process. We will need your consent on a few issues that enable us to conduct our research. As I run through these questions, please feel free to ask me any questions you would like clarified.

P1) First, we will be providing the hosting facility and session moderator with a list of respondents’ names and profiles (screener responses) so that they can sign you into the group. This information will not be shared with the Government of Canada department organizing this research. Do we have your permission to do this? I assure you it will be kept strictly confidential.

Yes 1 GO TO P2

No 2 READ RESPONDENT INFO BELOW

We need to provide the facility hosting the session and the moderator with the names and background of the people attending the focus group because only the individuals invited are allowed in the session and the facility and moderator must have this information for verification purposes. Please be assured that this information will be kept strictly confidential. GO TO P1A

P1a) Now that I’ve explained this, do I have your permission to provide your name and profile to the facility?

Yes 1 GO TO P2

No 2 THANK & TERMINATE

P2) An audio and/or video tape of the group session will be produced for research purposes. The tapes will be used by the research professional to assist in preparing a report on the research findings.

Do you agree to be audio and/or videotaped for research purposes only?

Yes 1 THANK & GO TO P3

No 2 READ RESPONDENT INFO BELOW

It is necessary for the research process for us to audio/video tape the session as the researcher needs this material to complete the report.

P2a) Now that I’ve explained this, do I have your permission for audio/video taping?

Yes 1 THANK & GO TO P3

No 2 THANK AND TERMINATE

P3) Each month we submit the names of individuals that have participated in our focus groups to the Marketing Research and Intelligence Association Qualitative Central system (www.mria-arim.ca). Qualitative Central serves as a centralized database to review participation in qualitative research and focus groups. You will not be contacted for any reason whatsoever as a result of being on this list.

Do we have your permission to submit your name and phone number to MRIA’s Qualitative Central system?

Yes 1 THANK & GO TO INVITATION

No 2 GO TO P3A

P3a) To participate in this focus group we must have your permission to add your name to the Qualitative Central system as it is the only way for us to ensure the integrity of the research process and track participation in qualitative research. The system is maintained by the industry body, the Marketing Research and Intelligence Association, and is solely used to track your participation in qualitative research (such as focus groups). You will not be contacted for any reason whatsoever as a result of being on this list.

Now that I've explained this do I have your permission to add your name to our qualitative central list?

Yes 1 THANK & GO TO INVITATION

No 2 THANK & TERMINATE

AS REQUIRED, ADDITIONAL INFO FOR THE INTERVIEWER:

Please be assured that this information is kept confidential and is strictly accessed and used by professional market research firms to review participation and prevent “professional respondents” from attending sessions. Research firms participating in MRIA’s Qualitative Central require your consent to be eligible to participate in the focus group - the system helps ensure the integrity of the research process.

AS REQUIRED, NOTE ABOUT MRIA:

The Marketing Research and Intelligence Association is a non-profit organization for marketing research professionals engaged in marketing, advertising, social, and political research. The Association's mission is to be the leader in promoting excellence in the practice of marketing and social research and in the value of market information.

Invitation:

Do you have a pen handy so that I can give you the address where the group will be held? It will be held at:

We ask that you arrive fifteen minutes early to be sure you find parking, locate the facility and have time to check-in with the hosts. The hosts will be checking respondents’ identification prior to the group, so please be sure to bring some personal identification with you (for example, a driver’s license). If you require glasses for reading make sure you bring them with you as well.

Group specifications:

- Groupe 1-3-5:

CAF Veterans

Good mix of demo profile

- Groupe 2-4-6

Veterans' family members

Caregivers

Good mix of demo profile

Appendix C – Moderation Guide

ENGLISH FOCUS GROUP DICUSSION GUIDE

Introduction: General Presentation

Duration: 10 minutes

PRESENTATION

· Introduce moderator and welcome participants to the focus group.

· Present the research firm.

· As we indicated during the recruiting process, we are conducting focus group discussions on behalf of the Government of Canada (Veterans Affairs Canada).

· The focus of tonight’s discussion is to evaluate a creative campaign that is being planned by the government. Your feedback on the advertising concepts, although they are not in their final format, will guide the Department in producing the final ads.

· The discussion will last approximately 90-120 minutes.

PURPOSE OF THE MEETING

· Video and audio-taping – The session is being video and audio-taped for research purposes, in case we need to double-check the discussion against our notes. These audio-tapes remain in our possession and will not be released to the Department or anyone else without written consent from all participants.

· Another thing that I would like to point out is one aspect of the room design – the mirror. This aspect of the room has been developed to allow those who have worked on a project or advertising to hear what participants have to say, in a way that won’t disrupt a group discussion. So, behind this mirror there are some people who have worked on the project. They are very interested in what you have to say.

· It is also important for you to know that your responses today will in no way affect your dealings with the Government of Canada and your name is not shared with any organization.

· Confidentiality – Please note that anything you say during these groups will be held in the strictest confidence. In our reporting, we do not attribute comments to specific people. Our report summarizes the findings from the groups but does not mention anyone by name. The report will be available through Library and Archives Canada (up to 6 months after).

· Please see the hostess at the end of the focus group for your incentive.

Description of a discussion group:

· Discussion groups are designed to encourage an open and honest discussion. My role as a moderator is to guide the discussion and encourage everyone to participate. Another function of the moderator is to ensure that the discussion stays on topic and on time.

· Your role is to answer questions and voice your opinions. We are looking for all opinions in a focus group, so don't hold back if you have a comment even if you feel your opinion may be different from others in the group. There may or may not be others who share your point of view. Everyone's opinion is important and will be respected.

· I would also like to stress that there are no wrong answers. We are simply looking for your opinions and viewpoints/thoughts/feedback. This is not a test of your knowledge. We did not expect you to do anything in preparation for this group.

· It is also important that you talk loudly enough for everyone to hear and that you talk one at a time so I can follow the discussion.

Please note that I am not an employee of the Government of Canada and may not be able to answer all of your questions. If you have some important questions, we could ask Veterans Affairs Canada and let you know by email about the answer.

· Moderator introduces herself/himself. Instructs participants to introduce themselves, using their first names only.

· What is your main occupation?

Today we would like to get your reactions to some advertising concepts. All the elements in these concepts could be used to create ads in a variety of media, including making videos.

Today’s groups are meant to assist the government in understanding how you perceive their advertising in terms of content,its look and feel, and its impact on you.

PART 1: Media Habits

Duration: 15 minutes ( )

To start the conversation, I would like to hear about your media habits. Please describe your media habits in a typical day. Let’s split this into morning, afternoon and evening. Tell us about the medias (TV, radio, Internet, newspaper, etc), the devices you use (Mobile or not), the social media and the websites you visit, apps you use, etc.

Wait for spontaneous answers, and ask about:

Morning

Media (TV, radio, Newspapers, Internet, etc.)
Devices (e.g., smartphone, tablet)? Which and why?
Social Media? Which and why?
Websites? Most visited and why?
Mobile apps? Most used and why?
Others?

· Afternoon

Medias (TV, radio, Newspapers, Internet, etc.)
Devices (e.g., smartphone, tablet)? Which and why?
Social Media? Which and why?
Websites? Most visited and why?
Mobile apps? Most used and why?
Others?

· Evening

Medias (TV, radio, Newspapers, Internet, etc.)
Devices (e.g., smartphone, tablet)? Which and why?
Social Media? Which and why?
Websites? Most visited and why?
Mobile apps? Most used and why?
Others?

PART 2: Awareness of Veterans Affairs Canada and its services

Duration: 10 minutes ( )

[BEFORE GOING ANY FURTHER INTO THE DISCUSSION : Tell participants that any relationship they may have with VAC in terms of receiving services is confidential. No information will be gathered or retained on that point.]

Now, I would like you to tell me to what extent you are familiar with Veterans Affairs Canada.

What would be your preference about the way Veterans Affairs communicates with you?

If Veterans Affairs Canada wanted to let you know about benefits, services and programs that are offered, what would be the best way to reach you?

Do you know the website of Veterans Affairs Canada? Have you ever been on this website before? Why? (Veterans.gc.ca)

Do you know the section of the website which is about services? (Veterans.gc.ca/services)

Do you know about “My VAC Account” on the VAC website?

If you have to communicate with Veterans Affairs Canada, what would be your preference? Would you call them , email them, use social medias; etc...

If not spontaneously, probe for the following items:

· Phone

· Email

· Facebook pages of VAC

· Other social medias

· My VAC Account on the website

· Any other means to communicates with VAC

PART 3:

Duration: UP TO 75 minutes

We will talk about video concepts that Veterans Affairs Canada is now considering. The department wishes to launch an advertising campaign and wants your input to help them make the best decisions.

MODERATOR WILL SHOW EACH CONCEPT ON THE TV SCREEN. A DISCUSSION OCCURS AFTER EACH CONCEPT VIEW.

AFTER EACH VIEW – ASK PARTICIPANTS TO EVALUATE THE CONCEPT WITHOUT COMPARING CONCEPTS

Individual Exercise #1-2-3-4 (ONE FOR EACH CONCEPT)

ROTATE ORDER BETWEEN CONCEPTS FOR EACH GROUP FOLLOWING THE CHART BELOW.

LOCATION	GROUP 1	GROUP 2
Winnipeg
Winnipeg	Concept A Concept B Concept C-D	Concept D-C Concept B Concept A
Quebec
Quebec	Concept B Concept D-C Concept A	Concept A Concept B Concept C-D
Halifax
Halifax	Concept C-D Concept B Concept A	Concept B Concept A Concept D-C

Since concept C and concept D are very similar, they are always going to be presented one after one as part of the randomization.

CONCEPT A

Participants observe concept A (1-2 minutes)

Participants evaluate image A (2-3 minutes)

Discussion of Concept A (10 minutes)

1. Overall, what do you think of the concept? What are your first impressions? What makes you say that?

2. What is the main message? What makes you think that?

3. Is it clear to you that this is from Veterans Affairs Canada? Why or why not?

4. Is there anything in the concept that you find inappropriate or confusing? Please explain.

5. What do you see as the key strengths of the concept? Explain

6. And what do you see as the key weaknesses of the concept? Explain

7. Does the concept provide you with new information that is relevant to you? Please explain.

8. Is this ad likely to make you seek more information about Vetrans Affairs Canada services? Does it raise your level of awareness about the services offered by Veterans Affairs Canada?

9. Would you change anything in the concept? What could be done to improve it?

10. Does this concept change your view about Veterans Affairs Canada? Please explain why.

11. Were any services shown that you did not think were available?

12. Moderator asks participant to talk about their evaluation of the concept (Exercise on paper first – then a discussion about the evaluation)

Each criterion on a scale of 1 to 10

The concept…

… Caught my attention

... Is credible

... Differs from other concepts that I'm used to seeing

... Contains a clear message that is easy to understand

... Could make me think about sharing with a Veteran I know, or finding out more for myself

… Changed my perception of VAC

CONCEPT B

Participants observe concept B (1-2 minutes)

Participants evaluate image B (2-3 minutes)

Discussion of Concept B (10 minutes)

13. Overall, what do you think of the concept? What are your first impressions? What makes you say that?

14. What is the main message? What makes you think that?

15. Is it clear to you that this is from Veterans Affairs Canada? Why or why not?

16. Is there anything in the concept that you find inappropriate or confusing? Please explain.

17. What do you see as the key strengths of the concept? Explain

18. And what do you see as the key weaknesses of the concept? Explain

19. Does the concept provide you with new information that is relevant to you? Please explain.

20. Is this ad likely to make you seek more information about Vetrans Affairs Canada services? Does it raise your level of awareness about the services offered by Veterans Affairs Canada?

21. Would you change anything in the concept? What could be done to improve it?

22. Does this concept change your view about Veterans Affairs Canada? Please explain why.

23. Were any services shown that you did not think were available?

24. Moderator asks participant to talk about their evaluation of the concept (Exercise on paper first – then a discussion about the evaluation)

Each criterion on a scale of 1 to 10

The concept…

… Caught my attention

... Is credible

... Differs from other concepts that I'm used to seeing

... Contains a clear message that is easy to understand

... Could make me think about sharing with a Veteran I know, or finding out more for myself

… Changed my perception of VAC

CONCEPT C

Participants observe concept C (1-2 minutes)

Participants evaluate image C (2-3 minutes)

Discussion of Concept C (10 minutes)

25. Overall, what do you think of the concept? What are your first impressions? What makes you say that?

26. What is the main message? What makes you think that?

27. Is it clear to you that this is from Veterans Affairs Canada? Why or why not?

28. Is there anything in the concept that you find inappropriate or confusing? Please explain.

29. What do you see as the key strengths of the concept? Explain

30. And what do you see as the key weaknesses of the concept? Explain

31. Does the concept provide you with new information that is relevant to you? Please explain.

32. Is this ad likely to make you seek more information about Vetrans Affairs Canada services? Does it raise your level of awareness about the services offered by Veterans Affairs Canada?

33. Would you change anything in the concept? What could be done to improve it?

34. Does this concept change your view about Veterans Affairs Canada? Please explain why.

35. Were any services shown that you did not think were available?

36. Moderator asks participant to talk about their evaluation of the concept (Exercise on paper first – then a discussion about the evaluation)

Each criterion on a scale of 1 to 10

The concept…

… Caught my attention

... Is credible

... Differs from other concepts that I'm used to seeing

... Contains a clear message that is easy to understand

... Could make me think about sharing with a Veteran I know, or finding out more for myself

… Changed my perception of VAC

CONCEPT D

Participants observe concept D (1-2 minutes)

Participants evaluate image D (2-3 minutes)

Discussion of Concept D (10 minutes)

37. Overall, what do you think of the concept? What are your first impressions? What makes you say that?

38. What is the main message? What makes you think that?

39. Is it clear to you that this is from Veterans Affairs Canada? Why or why not?

40. Is there anything in the concept that you find inappropriate or confusing? Please explain.

41. What do you see as the key strengths of the concept? Explain

42. And what do you see as the key weaknesses of the concept? Explain

43. Does the concept provide you with new information that is relevant to you? Please explain.

44. Is this ad likely to make you seek more information about Veterans Affairs Canada services? Does it raise your level of awareness about the services offered by Veterans Affairs Canada?

45. Would you change anything in the concept? What could be done to improve it?

46. Does this concept change your view about Veterans Affairs Canada? Please explain why.

47. Were any services shown that you did not think were available?

48. Moderator asks participant to talk about their evaluation of the concept (Exercise on paper first – then a discussion about the evaluation)

Each criterion on a scale of 1 to 10

The concept…

… Caught my attention

... Is credible

... Differs from other concepts that I'm used to seeing

... Contains a clear message that is easy to understand

... Could make me think about sharing with a Veteran I know, or finding out more for myself

… Changed my perception of VAC

Discussion and Comparison of Different Concepts

49. Is there an element in any of the concepts you saw that particularly interested you? What made this stand out?

Probe for :

Headlines (Which one did you prefer? Why?)

1. Headline 1

2. Headline 2

3. Headline 3

4. Headline 4

Images (Are there any images that you liked or disliked in any of the concept we have seen before?)

1) Veteran illustration

2) Uniform

3) Family

The call to action (Do you find it interesting or not? Can it work for you? Would you be enticed to act?

50. Which concept will you remember seeing the most, say in a week from now? Why that one?

51. Among the four concepts that we have seen, which one appeals to you the most? Why

52. Which of the concepts do you identify most with? Why

53. Is there a piece of information that surprised or that was new to you?

54. Would these ads prompt you to visit Veterans Affairs’ website?

55. What are the services presented in these ads that interested you the most?

56. Thinking about these ads, is there something missing?

57. Would you add something to these ads?

PART 4:

Duration: 5 minutes

Thinking about those ads, what would be the best way for Veterans Affairs Canada to communicate these with you.

How should Veterans Affairs communicate this information?

Where would you like to see these ads?

When would you like to see these ads?

Conclusion

Duration: 5 minutes

Go around the table and conclude.

Finally, if you had a chance to speak directly with the creator of the creative ads we have seen earlier, what would you tell him or her? What should be the most important thing you would tell him/her? What would you most like him/her to understand?

Do you have final comments that you would like to address to Veterans Affairs Canada?

THANK YOU VERY MUCH FOR YOUR VALUABLE PARTICIPATION!

END OF GROUP

Appendix D – Survey Questionnaire

ENGLISH QUESTIONNAIRE

GENERAL PROGRAMMING INSTRUCTIONS:

[TYPE OF PROJECT: Web]

[LANGUAGES: FR/EN]

[TRACKING: No]

[PROJECT DETAILS: Study with 1,000 Canadians respondents. Sample profiling similar to the general population)]

NOTE TO READERS: Text in green and in grey = Programming instructions.

INFORMATION

Page info

[ASK ALL]

QINF#

Leger is currently conducting a study on the behalf of the Government of Canada. We would greatly appreciate your observations and opinions.

Your participation is voluntary and completely confidential. Your answers will remain anonymous and will not in any way affect your dealings with the Government of Canada.

It will take no more than 10 minutes. Please take time to read each question and answer them carefully; it will not be possible to go back and edit your answers once they have been saved.

This survey is registered in the national survey registration system of the MRIA. You may visit the MRIA website to verify this survey at www.surveyverification.ca (English) or www.verificationsondage.ca (French). MRIA Project Registration Number: xxxxxxxxxxxx-xxx

Section 1: Sociodemographic Profile

[ASK LANG TO ALL]

[SINGLE MENTION]

LANG

Préféreriez-vous répondre à ce questionnaire en anglais ou en français
Would you prefer to complete the survey in English or French?

English FR

Français EN

[ASK PROV TO ALL]

[SINGLE MENTION]

[Prog: Insert the standard regional questions for Ontario and Quebec]

PROV

In which province or territory do you live?

British Columbia BC

Alberta AB

Saskatchewan SK

Manitoba MB

Ontario ON

Quebec QC

New Brunswick NB

Nova Scotia NS

Prince Edward Island PE

Newfoundland NF

Northwest Territories NT

Yukon YK

Nunavut NU

[ASK SEXE TO ALL]

[SINGLE MENTION]

SEXE

You are…?

... A man 1

... A woman 2

Note: As indicated by Statistics Canada, transgender, transsexual, and intersex Canadians should indicate the sex (male or female) with which they most associate themselves.

[ASK AGE TO ALL]

[SINGLE MENTION]

AGE

How old are you?

INTERVIEWER INSTRUCTIONS: (READ LIST)

Under 18 1 TERMINATE

Between 18 and 24 2

Between 25 and 34 3

Between 35 and 44 4

Between 45 and 54 5

Between 55 and 64 6

Between 65 and 74 7

75 or older 8

I prefer not to answer 9 TERMINATE

[ASK LANGU TO ALL]

[SINGLE MENTION]

LANGU

What is the language you first learned at home in your childhood and that you still understand?

French 1

English 2

Other 3

English and French 7

French and other 4

English and other 5

Other and other 6

I prefer not to answer 9

[ASK ENFAN TO ALL]

[SINGLE MENTION]

ENFAN

Are there any children who are UNDER 18 YEARS OF AGE living in your household?

(IF SO:) Are any of them 12 YEARS OLD AND OLDER or LESS THAN 12 years old?

Yes: 12 years of age and OLDER ONLY 1

Yes: 12 years of age and older AND younger than 12 years old 2

Yes: YOUNGER than 12 years old ONLY 3

No children under 18 years old at all in the household 4

I prefer not to answer 9

[ASK SCOL TO ALL]

[SINGLE MENTION]

SCOL

What level of education have you completed?

Elementary (7 years or less) 1

High school, general or vocational (8 to 12 years) 2

College (pre-university, technical training, certificate, accreditation or advanced diploma) 3

University certificates and diplomas 4

University Bachelor (including classical studies) 5

University Master's degree 6

University Doctorate (PhD) 7

I prefer not to answer 9

Section 2: Evaluation of Concepts

Veterans Affairs Canada (VAC) is currently working on different concepts for an advertising campaign. We would like to get your opinion on four different concepts that could be used in the future. These concepts will be intended primarily for Veterans and their family members.

We first ask you to evaluate each concept individually, and later ask you to choose the one you prefer.

Please take time to look at and read over each concept carefully.

[NOTE TO PROGRAMMING: QUESTIONS RELATED TO CONCEPT A, CONCEPT B, CONCEPT C AND CONCEPT D NEED TO BE RANDOMIZED]

[ASK ALL]

[SINGLE MENTION]

[LIST ORDER: In order]

Q2A

Concept-A

Visual

To what extent do you like this concept?

Like it a lot 1

Like it somewhat 2

Dislike it somewhat 3

Dislike it a lot 4

I prefer not to answer 9

[ASK ALL]

[SINGLE MENTION GRID TABLE]

[LIST ORDER: In order]

[STATEMENT LIST ORDER: Randomized]

[PROGRAMMER NOTES: Insert the visual of Concept A as a reminder in case respondents feel the need to read it one more time.]

Q2B

To what extent do you agree or disagree with the following statements?

In my opinion, this concept …

Answer List

Totally agree 1

Somewhat agree 2

Somewhat disagree 3

Totally disagree I 4

I prefer not to answer 9

Statements List

A. …is memorable.

B. …caught my attention

C. …is unique and stands out from the ones I usually see.

D. …gives me a positive image of Veterans Affairs Canada (VAC).

E. ...has a clear message that is easy to understand.

F. …is appropriate for Veterans Affairs Canada (VAC).

G. ...is credible.

H. ...is a good way of showing what Veterans Affairs Canada (VAC) is doing for Veterans and their families.

I. …is effective.

J. ...would make me want to visit their website if I were a Veteran or a Veteran family member.

[ASK ALL]

[SINGLE MENTION]

[LIST ORDER: In order]

Concept-B

Visual

To what extent do you like this concept?

Like it a lot 1

Like it somewhat 2

Dislike it somewhat 3

Dislike it a lot 4

I prefer not to answer 9

[ASK ALL]

[SINGLE MENTION GRID TABLE]

[LIST ORDER: In order]

[STATEMENT LIST ORDER: Randomized]

[PROGRAMMER NOTES: Insert the visual of Concept B as a reminder in case respondents feel the need to read it one more time.]

To what extent do you agree or disagree with the following statements?

In my opinion, this concept …

Answer List

Totally agree 1

Somewhat agree 2

Somewhat disagree 3

Totally disagree I 4

I prefer not to answer 9

Statements List

A. …is memorable.

B. …caught my attention

C. …is unique and stands out from the ones I usually see.

D. …gives me a positive image of Veterans Affairs Canada (VAC).

E. ...has a clear message that is easy to understand.

F. …is appropriate for Veterans Affairs Canada (VAC).

G. ...is credible.

H. ...is a good way of showing what Veterans Affairs Canada (VAC) is doing for Veterans and their families.

I. …is effective.

J. ...would make me want to visit their website if I were a Veteran or a Veteran family member.

[ASK ALL]

[SINGLE MENTION]

[LIST ORDER: In order]

Concept-C

Visual

To what extent do you like this concept?

Like it a lot 1

Like it somewhat 2

Dislike it somewhat 3

Dislike it a lot 4

I prefer not to answer 9

[ASK ALL]

[SINGLE MENTION GRID TABLE]

[LIST ORDER: In order]

[STATEMENT LIST ORDER: Randomized]

[PROGRAMMER NOTES: Insert the visual of Concept C as a reminder in case respondents feel the need to read it one more time.]

To what extent do you agree or disagree with the following statements?

In my opinion, this concept …

Answer List

Totally agree 1

Somewhat agree 2

Somewhat disagree 3

Totally disagree I 4

I prefer not to answer 9

Statements List

A. …is memorable.

B. …caught my attention

C. …is unique and stands out from the ones I usually see.

D. …gives me a positive image of Veterans Affairs Canada (VAC).

E. ...has a clear message that is easy to understand.

F. …is appropriate for Veterans Affairs Canada (VAC).

G. ...is credible.

H. ...is a good way of showing what Veterans Affairs Canada (VAC) is doing for Veterans and their families.

I. …is effective.

J. ...would make me want to visit their website if I were a Veteran or a Veteran family member.

[ASK ALL]

[SINGLE MENTION]

[LIST ORDER: In order]

Concept-D

Visual

To what extent do you like this concept?

Like it a lot 1

Like it somewhat 2

Dislike it somewhat 3

Dislike it a lot 4

I prefer not to answer 9

[ASK ALL]

[SINGLE MENTION GRID TABLE]

[LIST ORDER: In order]

[STATEMENT LIST ORDER: Randomized]

[PROGRAMMER NOTES: Insert the visual of Concept D as a reminder in case respondents feel the need to read it one more time.]

To what extent do you agree or disagree with the following statements?

In my opinion, this concept …

Answer List

Totally agree 1

Somewhat agree 2

Somewhat disagree 3

Totally disagree I 4

I prefer not to answer 9

Statements List

A. …is memorable.

B. …caught my attention

C. …is unique and stands out from the ones I usually see.

D. …gives me a positive image of Veterans Affairs Canada (VAC).

E. ...has a clear message that is easy to understand.

F. …is appropriate for Veterans Affairs Canada (VAC).

G. ...is credible.

H. ...is a good way of showing what Veterans Affairs Canada (VAC) is doing for Veterans and their families.

I. …is effective.

J. ...would make me want to visit their website if I were a Veteran or a Veteran family member.

[ASK ALL]

[RANKING QUESTION]

[LIST ORDER: Randomized]

[PROGRAMMER NOTES: Show all four images of the four concepts]

Please rank the four concepts from 1 to 4, 1 being the concept you prefer the most and 4 the concept you prefer the least.

CONCEPT: A

CONCEPT: B

CONCEPT: C

CONCEPT: D

[ASK ALL]

[OPEN QUESTION]

[Please show image of the concept ranked 1 at the previous question]

Q10

You said you liked this concept more than the other ones. In one sentence, please say why you choose this one.

Please specify 96 Open box

[ASK ALL]

[SIMPLE MENTION]

[PLEASE SHOW CONCEPT A-D GROUPED AND CONCEPT B-C GROUPED]

Q11- A

Between these two sets of concepts, would you prefer the concepts with the photos (Concept A-D) or the concepts with the illustrations (Concepts B-C)?

CONCEPT: A-D (PHOTOS) 1

CONCEPT: B-C (ILLUSTRATIONS) 2

[ASK IF Q11A = 1]

[SIMPLE MENTION]

[PLEASE SHOW CONCEPT A AND CONCEPT D]

Q11- B

Between these two photos, which one do you prefer?

PHOTO A 1

PHOTO D 2

[ASK IF Q11A = 2]

[SIMPLE MENTION]

[PLEASE SHOW CONCEPT B AND CONCEPT C]

Q11- C

Between these two illustrations, which one do you prefer?

ILLUSTRATION B 1

ILLUSTRATION C 2

[ASK ALL]

[RANKING QUESTION]

[LIST ORDER: Randomized]

Q12

lease rank the four headlines from 1 to 4, 1 being the headline you prefer the most and 4 the headline you prefer the least.

New Benefits and Services for Veterans and their Families 1

New Education and Training Benefit for Veterans 2

New and Enhanced Services for Veterans and their Families 3

Enhanced Services for Veterans and their Families 4

[ASK ALL]

[OPEN-END]

[PROGRAMMER NOTES:]

[VALIDATION:]

Q13.

In your opinion, what visual elements or symbols could be added to these concepts to indicate that the ads are for Canadian Veterans and their families?

Please specify 96 O

I prefer not to answer 99

QSCTDEMO

The following questions are for statistical purposes only.

[ASK ALL]

[SINGLE MENTION GRID TABLE]

[LIST ORDER: In order]

[STATEMENT LIST ORDER: In order]

[PROGRAMMER NOTES:]

[VALIDATION:]

Q14.

Are you…

Answer List

Yes 1

No 2

Statements List

… a Canadian Veteran* 1

… a member of the immediate family of a Veteran (spouse, parent, or child of a Veteran) 2

… a caregiver for a Veteran 3

* Note: Any former member of the Canadian Armed Forces who successfully underwent basic training and is honourably released.

[ASK ALL]

[SINGLE MENTION]

EMPLO

What is your current employment status?

Working full time 1

Working part-time 2

Self-employed / freelance work 3

Student 4

Homemaker 5

Unemployed / between jobs 6

Retired 7

I prefer not to answer 9

[ASK ALL]

[SINGLE MENTION]

REVEN

Among the following categories, which one best reflects the total INCOME, before taxes, of all the members of your household in 2017?

$19,999 or less 1

Between $20,000 and $39,999 2

Between $40,000 and $59,999 3

Between $60,000 and $79,999 4

Between $80,000 and $99,999 5

$100,000 or more 6

I prefer not to answer 9

Thank you for your participation. You have completed the survey.

Landing page (EN): http://www.Veterans.gc.ca/eng

Landing page (FR): http://www.Veterans.gc.ca/fra