I. Detailed findings – quantitative phase

A. Familiarity with veterans.gc.ca

Fewer than half of Veteran community members describe themselves as even somewhat familiar with the Department’s services and benefits; around half knew VAC provides services to people besides CAF Veterans. Four in ten have visited veterans.gc.ca before, and three in ten of these visit the site at least once a month.

Familiarity with VAC services and benefits

Veterans and family members were asked to indicate how familiar they personally are with the services and benefits offered by VAC. Fewer than half in either wave (46%) say they are very or somewhat familiar, while just over half (54%) are unfamiliar to some extent.

Familiarity with VAC services and benefits

Familiarity with VAC benefits and services	Very familiar	Somewhat familiar	Not very familiar	Not at all familiar
Wave 1 (n=518)	8%	37%	32%	22%
Wave 2 (n=522)	9%	37%	38%	16%

Q7 How familiar would you say you are with the services and benefits offered by Veterans Affairs Canada?

In both waves, Canadian Armed Forces (CAF) Veterans are around twice as likely as reservists to say they are familiar with VAC services to some extent (the small number of RCMP Veterans are the most likely to say they are familiar). Family members are about as likely as Veterans to say they are at least somewhat familiar with VAC offerings.

Familiarity is generally similar across the country but is lower in Quebec in both waves (around three in ten) and among Francophones (24% in Wave 1, 33% in Wave 2). There is no difference by age of respondent, but in Wave 2, familiarity is somewhat higher among women. There are no clear patterns by household income but, as is often the case with knowledge and awareness questions in public opinion surveys, familiarity increases as level of education increases.

Awareness of VAC services to people other than CAF Veterans

Around half in either wave of research say they were aware before the survey that VAC offers some services to people other than CAF Veterans, including family members, Regular or Reserve members, or members of the RCMP. Just over four in ten say they were not aware.

Awareness that VAC offers services to people other than CAF Veterans – by type

Aware that VAC offers services to people other than CAF Veterans

Total aware

(n=518 w1,
n=522 w2)

CAF Veterans (n=161 w1
n=141 w2)

RCMP Veterans (n=36* w1
n=35* w2)

Reserve/
rangers (n=72* w1, n=93* w2)

Family
(n=271 w1, n=271 w2)

Wave 1

48%

66%

63%

38%

Wave 2

51%

61%

82%

43%

46%

Q8 Before today, were you aware Veterans Affairs Canada offers some services to people other than CAF Veterans, for example family members of Veterans, to Regular or Reserve members of the Canadian Armed Forces, or members of the RCMP? *Note: Small base (<100); caution is advised in interpreting results.

Awareness VAC offers some services to non-CAF Veterans is notably higher among CAF Veterans than among reservists or family members of Veterans. Awareness is higher among men than women (Veterans being more likely to be men), especially in Wave 1. There are no clear patterns of response by region, language of interview, education or household income. Higher awareness is linked to being more familiar with VAC services and benefits, and to identifying as a frequent veterans.gc.ca website user.

Reasons for visiting veterans.gc.ca

Members of the Veterans community were asked if they have ever visited the VAC website to obtain information, to access services, or both. Around four in ten indicate they have visited veterans.gc.ca. One-quarter have used the site to obtain information, and a further one in seven have visited both to obtain information and to access services. Just over half in either wave say they have not visited the website. Small proportions have visited just to access services, or are not sure. The high level of comparability indicates any observed differences between the two waves in terms of impressions of the page designs are not due to differences in familiarity with the site in general.

Reasons for visiting veterans.gc.ca

Use of veterans.gc.ca	To obtain information	To access services	Both	Have not visited	Not sure
Wave 1 (n=518)	24%	3%	14%	55%	4%
Wave 2 (n=522)	24%	2%	15%	57%	3%

Q10 Have you ever visited the Veterans Affairs Canada website, veterans.gc.ca, for the following reasons?

Of the different types of respondents, having visited the site previously is highest among CAF and RCMP Veterans; around seven in ten reservists/rangers and six in ten family members say they have not visited veterans.gc.ca. Having visited the site is fairly similar by region and by age. There are no notable demographic or socioeconomic differences to note in having visited the site, but having visited is linked to greater familiarity with VAC services and benefits.

NOTE: Respondents to the surveys were not asked directly if they are VAC clients, for privacy reasons. Instead, respondents were identified as being potential VAC clients by extrapolating from their responses to two questions. Those who said they are very or somewhat familiar with VAC services and benefits and also indicated they have visited the VAC website to access services (n=88 or 17% in each of Wave 1 and Wave 2) have been classified as “VAC clients” for the purposes of survey analysis. Those who did not give these responses to both of these questions are referred to as “non-clients” in this report. Being a VAC client is higher among CAF and RCMP Veterans than among reservists/rangers or family members.

Frequency of visiting veterans.gc.ca

Those who report having previously gone to veterans.gc.ca were asked how frequently they visit the site. Three in ten report visiting the site at least once a month, in either wave, and a similar three-in-ten proportion says they visit every few months. A plurality of just under four in ten visit once a year or less often. Reported use is similar by region.

Frequency of visiting veterans.gc.ca

Frequency of use of veterans.gc.ca	1 x week or more	Several times a month	Once a month	Every few months	Yearly or less often	Not sure
Wave 1 (n=215)	5%	13%	12%	30%	37%	3%
Wave 2 (n=211)	5%	10%	15%	31%	38%	1%

Q11 Approximately how often do you visit the Veterans Affairs Canada website, veterans.gc.ca?
BASE: THOSE WHO HAVE VISITED THE SITE

Members of the Veterans community were categorized by website visit frequency for the purposes of this report. Those reporting visiting veterans.gc.ca at least once a month are described as “frequent visitors,” those visiting every few months to yearly or less often are “infrequent visitors,” and those who have not visited the site or were not sure if they have are classified as “non-users.” There is good comparability between the two waves.

Frequency of visiting veterans.gc.ca – by type

Frequency of use of veterans.gc.ca	Wave 1					Wave 2
Frequency of use of veterans.gc.ca	Total (n=518)	CAF Veteran (n=161)	*RCMP (n=36)**	*Reserve/ranger (n=72)**	Family member (n=271)	Total (n=522)	CAF Veteran (n=141)	RCMP (n=35%)	*Reserve/ranger (n=93)**	Family member (n=271)
Frequent user	12%	21%	26%	13%	6%	12%	15%	30%	6%	11%
Infrequent user	28%	33%	31%	17%	28%	28%	36%	30%	16%	29%
Non-user	60%	46%	43%	69%	66%	60%	49%	40%	79%	60%

Q11 Approximately how often do you visit the Veterans Affairs Canada website, veterans.gc.ca?
BASE: THOSE WHO HAVE VISITED THE SITE

*Note: Small base (<100); caution is advised in interpreting results

In both waves, frequent use is reported more among Veterans than family members; frequent use tends to decrease as the age of the respondent increases. Being a non-user is higher among those who report household incomes of under $40,000.

B. Overall site ratings

Nine in ten or more agree the veterans.gc.ca website is credible, has useful resources, and that they would be comfortable using it and that they are confident it will provide accurate information. Wave 2 viewers are more likely than those viewing Wave 1 to say the site is at least somewhat better than other government sites; this is largely attributed to a good design that facilitates navigation.

Agreement with statements about veterans.gc.ca

At the end of the survey, after viewing the home page and two of four other pages (chosen at random), members of the Veterans community were asked to indicate their level of agreement with four general statements about the veterans.gc.ca site, based on what they had seen during the survey or on previous visits. Strong majorities of nine in ten or more agree to some extent with each of the statements, in both waves. Because overall agreement is very high, analysis will focus on the proportions saying they agree strongly.

In Wave 1, strong agreement is highest that the site is credible. Strong agreement that the site has useful resources and that they would be comfortable using it is higher in Wave 2 than Wave 1, on par with the proportion strongly agreeing it is credible. Strongly agreeing they would be very confident the site will provide accurate information is lower, but still at majority levels in both waves, and notably higher in Wave 2 than in Wave 1.

Strong agreement with statements about veterans.gc.ca
(bold indicates proportions that are statistically higher than in the other wave)

Strong agreement with statements about veterans.gc.ca	Wave 1 (n=518)		Wave 2 (n=522)
Strong agreement with statements about veterans.gc.ca	Net agree	Strongly	Net agree	Strongly
veterans.gc.ca is a credible website	94%	67%	93%	68%
veterans.gc.ca has useful resources	93%	60%	94%	66%
I would be comfortable using the veterans.gc.ca website	92%	59%	95%	67%
I am very confident the veterans.gc.ca website will provide me with accurate information	90%	54%	92%	63%

Q32 Based on what you have seen of the veterans.gc.ca website (during this survey/either during this survey or on previous visits), please indicate your level of agreement with each of the following statements. (By this we mean the main Veterans Affairs website, not My VAC Account)

In all cases, family members are the most likely to strongly agree with these statements, and CAF Veterans the least likely – although it should be noted that half or more Veterans do strongly agree with each statement, in both waves.

Strong agreement with statements about veterans.gc.ca – by type

Strong agreement with statements about veterans.gc.ca	Wave 1					Wave 2
Strong agreement with statements about veterans.gc.ca	Total (n=518)	CAF Veteran (n=161)	*RCMP (n=36)**	*Reserve/ranger (n=72)**	Family member (n=271)	Total (n=522)	CAF Veteran (n=141)	*RCMP (n=35)**	*Reserve/ranger (n=93)**	Family member (n=271)
veterans.gc.ca is a credible website	67%	60%	61%	64%	72%	68%	59%	67%	67%	73%
veterans.gc.ca has useful resources	60%	53%	52%	50%	67%	66%	57%	70%	68%	69%
I would be comfortable using the veterans.gc.ca website	59%	55%	52%	53%	63%	67%	59%	75%	64%	72%
I am very confident the veterans.gc.ca website will provide me with accurate information	54%	50%	40%	46%	62%	63%	53%	71%	67%	65%

*Note: Small base (<100); caution is advised in interpreting results

Strong agreement with these statements is quite consistent by region in both waves, although Quebec residents are less likely than others to have strongly agreed in Wave 1 that they are very confident the site would provide accurate information. In both waves, strong agreement is somewhat higher among women than men, but the gender difference is more marked in Wave 2. There are no clear patterns by age, employment type or education level, but in both waves those in the lowest household income bracket are the least likely to strongly agree, while those in middle income brackets ($60,000 to under $100,000) tend to be the most likely. In both waves, there are no notable differences in agreement by whether someone is a frequent, infrequent or non-user of the website, and no clear pattern by either level of self-assessed familiarity with VAC services, or by whether someone is identified as a VAC client or not.

Opinion of VAC site compared to other government service sites

Veterans and family members were asked how they would rate veterans.gc.ca compared to other websites they might use to a research or access government services. In Wave 1, just under six in ten (58%) say it is at least a little better, compared to two-thirds (66%) in Wave 2. One-third (35%) in Wave 1 say it is neither better nor worse, versus nearly three in ten (28%) in Wave 2. Few in either wave say it is worse.

Opinion of VAC site compared to other government service sites
(bold indicates proportions that are statistically higher than in the other wave)

veterans.gc.ca compared to other government service websites	A lot better	A little better	Neither better nor worse	Worse (a lot/a little)	Not sure
Wave 1 (n=518)	29%	30%	35%	3%	4%
Wave 2 (n=522)	33%	33%	28%	1%	4%

Q33 Compared to other websites you may use to research or access government services, how would you rate veterans.gc.ca?

Saying the veterans.gc.ca website is better than others is quite consistent by region and by type of respondent (CAF Veteran, other types of Veterans, and family members) in both surveys. In Wave 2, women are notably more likely (73%) than men (62%) to say it is at least a little better. Saying it is at least a little better is similar by respondent age, but saying it is a lot better increases as age increases; younger respondents (age 18 to 34) are the most likely to say it is a little better. There are no other demographic differences to note.

Saying it is at least a little better is similar by level of familiarity with VAC services, frequency of website use, and whether or not someone is considered a VAC client for survey purposes.

Why veterans.gc.ca is better than other sites

Those who indicated the veterans.gc.ca site is better than other government information sites were asked why they think this. Their verbatim responses were reviewed and then coded to allow quantification.

Both waves see numerous mentions of good navigation or organization as reasons the VAC site is better than others they have used. Top responses include it being easy to navigate, that information is easily found without having to search the whole site (mentioned in Wave 2), that it has a good layout/design (more mentions in Wave 2), that it is not as confusing as other government sites, or that it is straightforward and easy to follow (more mentions in Wave 1 than Wave 2). Wave 2 viewers are more likely than those in Wave 1 to mention the site being clean and uncluttered, well-organized, or that options and information are clear. Wave 1 visitors were more likely than those in Wave 2 to refer to the links as being clearly defined, or having links or options available to provide more information.

Reasons for saying veterans.gc.ca is better than other sites
(3% or more in Wave 2)
(bold indicates proportions that are statistically higher than in the other wave)

Why veterans.gc.ca is better than other sites with government information	Wave 1 (n=300)	Wave 2 (n=345)
Easy to navigate/user-friendly	22%	25%
Direct/info easily found without need to search whole site	-	20%
Good layout/design/interesting to look at/looks modern	12%	17%
Better than other government websites/not as confusing	15%	16%
Simple/straightforward/concise/easy to follow	21%	15%
Informative/detailed with relevant info/comprehensive	11%	13%
Clean/uncluttered/no ads	6%	12%
Links clearly defined to find required info easily	30%	11%
Well organized/all information is well grouped	6%	11%
Clarity of information/options	-	11%
Easy to understand/does not use technical terms/plain English	7%	9%
Big fonts/prints helpful to read easily	6%	9%
Clearly states programs/services available for Veterans	6%	5%
Offers help a Veteran would need	1%	4%
Links, options available to provide more info when needed	12%	3%

Q34 Could you please briefly describe why you said the veterans.gc.ca website is (a lot better/somewhat better) than other websites for researching or accessing government services?
BASE: THOSE SAYING VETERANS.GC.CA IS BETTER THAN OTHER SITES

Comments are generally similar across subgroups. Mentioning the layout and design being good are most common among younger viewers (under age 35) in either wave, as well as those with higher household incomes ($60,000 and over) and post-graduate education in Wave 2. Those with higher levels of education (university degrees) are the most likely to mention it being not as confusing as other sites, in both waves. Anglophones are more likely than Francophones to mention the links being clearly defined in Wave 1, or the layout being good or it being less confusing than other sites in Wave 2.

Why veterans.gc.ca is neither better nor worse than other sites

Those who are more ambivalent about the veterans.gc.ca site compared to other sites providing government information were asked why they feel this way. More than half in both waves say it is because all government sites are the same or use similar formats. A somewhat larger proportion in Wave 2 mention it being a comprehensive site than those in Wave 1. Seven percent in either wave say they would need to use the site more to be able to judge how well it performs.

Subgroup bases are small and deeper analysis is not recommended.

Reasons for saying veterans.gc.ca is neither better nor worse than other sites
(3% or more in Wave 2)
(bold indicates proportions that are statistically higher than in the other wave)

Why veterans.gc.ca is neither better nor worse than other sites with government information	Wave 1 (n=181)	Wave 2 (n=147)
All gov't websites are the same/use similar format and layout	56%	56%
Informative/detailed with relevant information/ comprehensive	3%	9%
Need to use the site to judge how easy/helpful it is	7%	7%
Not easy to navigate/not user-friendly	4%	6%
Some tabs/links don't take one to the right information	2%	5%
All government websites are bad	5%	4%
Easy to navigate/user-friendly	4%	4%
Good layout/design/interesting to look at/looks modern	2%	4%
Haven't seen any other sites to compare	-	4%
Good/like it	2%	3%
Lacks images/colour to make it more interesting	3%	3%
Links are clearly defined to find required information easily	-	3%

Q34 Could you please briefly describe why you said the veterans.gc.ca website is neither better nor worse than other websites for researching or accessing government services?
BASE: THOSE SAYING VETERANS.GC.CA IS NEITHER BETTER NOR WORSE THAN OTHER SITES

Why veterans.gc.ca is worse than other sites

Very small numbers of respondents in either survey say the veterans.gc.ca site is worse than others they have used to obtain government information or services (n=18 in Wave 1, n=7 in Wave 2). In Wave 1, comments about this include it not being appealing to look at, that there is too much clutter on the pages (“too busy”), or that too many clicks are required to take you through to information. In Wave 2, a very small number feel it was not easy to navigate, and only one person each mentioned it being too wordy, lacking images or colour, not being organized, or that contact information was not prominent. The bases of those saying the site is worse are too small for subgroup analysis.

C. Web page testing – summary

Veterans Affairs identified four specific pages, in addition to the home page, to be shown to respondents in either wave and assessed in terms of clarity of wording, usefulness, design appeal and other attributes. Each respondent was shown the home page and then two of four additional pages, randomly selected to reduce burden, since the same questions were asked of each page; therefore each of the four pages were viewed by at least 250 people in each wave. Respondents were shown static “sandboxed” versions of the pages, with most links made inactive, and asked to review the page carefully prior to answering questions about it. Using these pages instead of images of the pages facilitated the review, and allowed for the use of accessibility features. The pages were selected because they were scheduled to be updated within the field period of the surveys, so pre- and post-change reactions could be assessed. The pages shown were:

· Home page

· Disability benefits

· Rehabilitation services

· Veterans Independence Program (VIP)

· Health-related travel information

Viewers were asked to indicate their level of agreement with several statements about the page they just viewed. Two of the statements had negative phrasing (“there is too much content on this page” and “this page is confusing or disorganized”), while the balance were positive. The statements appeared in a grid format, but the order in which the statements were listed was randomized for each page viewed.

Majorities of two-thirds or more agree at least somewhat with each positive statement about each web page, in both waves. This is an expected result, given the amount of research and effort that went into the Government of Canada standardized web page design (viewed in Wave 1) and also the amount of design research around the new VAC site design shown in Wave 2.

The home pages shown in Wave 1 and Wave 2 received statistically similar levels of agreement for equivalent positive and negative statements, except for the wording being clear, where strong agreement is higher in Wave 2. As well, the Veterans Independent Page (VIP) got similar results between the two waves.

Considerably more variation in response is observed for the other three pages. Most notably, the Wave 2 viewers of the health-related travel information page had statistically higher levels of overall and strong agreement for the positive statements, and statistically lower levels of overall agreement with the negative statements, than expressed by Wave 1 viewers.

See Sections D-H for more detailed information on the results of the agreement statement questions by pages viewed.

Strong agreement with page attributes
Base: Those seeing each page
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Percent who strongly agree	Home page		Disability		Rehabilitation		VIP		Health-related travel
Percent who strongly agree	W1 n=518	W2 n=522	W1 n=259	W2 n=261	W1 n=254	W2 n=261	W1 n=264	W2 n=261	W1 n=259	W2 n=261
The wording used on this page is very clear	56%	63%^¥	50%	61%	52%	61%^¥	51%	54%	36%	62%^¥
I would be able to find helpful info by using this page	53%	56%	55%	59%	50%	58%	53%	52%	43%	62%^¥
It is clear where to go to get additional information	47%	53%	52%	59%	43%	59%^¥	50%	52%	44%	61%^¥
The design (look and feel) of this page is appealing	42%	48%	30%	46%^¥	38%	44%	30%	38%	22%	46%^¥
There is too much content on this page	6%	7%	12%	9%	5%	11%^¥	6%	8%	13%^¥	7%
This page is confusing or disorganized	3%	4%	5%	3%	6%	7%	4%	5%	5%	4%

Please indicate your level of agreement with the following questions about the page you just saw.

In Wave 2, an additional statement was presented for agreement: “This looks like an authentic Government of Canada web page.” There is little confusion in the minds of viewers on this score: at least 85 percent agree to some extent with this statement, for all of the web pages shown, and at least half strongly agree. Results are comparable for each of the pages viewed.

Viewers were asked to indicate if the organization of each page they viewed made it easy or difficult to find information. Results are quite similar between the two waves, except that viewers of the health-related travel page in Wave 2 are notably more likely to feel the organization makes it very easy to locate information.

Those viewing each page were given an opportunity to indicate what they liked and didn’t like about each page, in their own words. The verbatim responses were coded and quantified. While responses differ between the different pages, two measures the pages have in common are the proportions saying they liked nothing about the page, or who disliked nothing about the page. Again, viewers of the health-related travel pages stand out.

See Sections D-H for more detailed information on the results of these questions by the pages viewed.

Impressions of pages
Base: Those seeing each page
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Percent responding	Home page		Disability		Rehabilitation		VIP		Health-related travel
Percent responding	W1 n=518	P2 n=522	W1 n=259	W2 n=261	W1 n=254	W2 n=261	W1 n=264	W2 n=261	W1 n=259	W2 n=261
The organization of the page makes it very easy to find information	37%	42%	37%	45%	39%	46%	40%	44%	25%	48%^¥
Nothing liked about this page	4%	7%	9%	5%	7%	9%	9%	8%	13%^¥	6%
Nothing disliked about page	60%	65%	57%	64%	62%	63%	57%	60%	48%	62%^¥

Does the way this page is organized make it easy or difficult to find information?
What, if anything, did you like about the page you just saw, overall?
And what, if anything, did you dislike about the page?

D. Web page testing – home page

Things liked about home page

All Veterans community members were shown the home page (original in Wave 1, redesigned in Wave 2) and asked what they liked about it. Their verbatim comments were reviewed and coded. Viewers in both waves make a wide range of mentions, notably that the page is well-designed and professional, informative and easy to navigate, and use good/readable fonts. Very small proportions indicate there is nothing they like about the page. The comments made are quite similar between the two waves.

Things liked about home page
(mentioned by 3% or more in Wave 2)
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Things liked about home page	Wave 1 (n=518)	Wave 2 (n=522)
Well-designed/professional layout/visually appealing	16%	19%
Informative/provides needed info/details on services	17%	15%
Easy to navigate/user-friendly	14%	14%
Good/large fonts choice/easy to read	13%	14%
Links to services/broad categories for all interests, needs	9%	13%^¥
Simple and straightforward/easy to follow/steps laid out	10%	13%
Bold/clear to see choices/subjects	10%	10%
Organized/easy to find what I am looking for/clear menu	12%	10%
Information organized in categorized links for easy searching	9%	9%
Clean/uncluttered/not busy/just the right amount of info	4%	7%^¥
Clear language/easy to understand	3%	6%^¥
Like all of it/everything/good/what I need	5%	4%
Accessible to people less familiar with online	2%	3%
Bright/good colours	3%	3%
Easy to find contact information/listed at the bottom of page	<1%	3%
Helpful webpage/Veterans can relate to it	-	3%
Provides good services/benefits to Veterans and their families	7%^¥	3%
Nothing/did not like anything	4%	7%^¥

Q12 What, if anything, did you like about the page you just saw, overall?

In general, responses are fairly consistent by subgroup. Mentioning the fonts are good or readable is higher in both waves among family members and women. That there are links to lots of services and categories for all needs is higher among older people (age 65 and over) in Wave 2.

Things disliked about home page

When asked about things they might dislike about the home page, majorities of at least six in ten in either wave say there is nothing about it they dislike. Only very small proportions indicate anything specific, with the colours being the most mentioned, but by only five percent in Wave 2.

Things disliked about home page
(mentioned by 2% or more in Wave 2)

Things disliked about home page	Wave 1 (n=518)	Wave 2 (n=522)
Dislike nothing/all is good	60%	65%
Dislike colours used/could be difficult to see/read	2%	5%
A lot of information on one page to scroll through	3%	2%
Dislike appearance/design/layout of page/not visually attractive	3%	2%
Does not provide contact info/not easy to find/not prominent	-	2%
Inability to find benefits/services for different categories of vets	-	2%
Looks boring/bland/nothing attracts attention	1%	2%
Not informative/have to link to other pages to get info	1%	2%

Q13 And what, if anything, did you dislike about the page?

Mentions are quite consistent by subgroup. The groups most likely to say there is nothing they dislike about the home page are women, Atlantic residents, those with high school or less education, and those who like the veterabns.gc.ca website a lot better than other government sites, and those categorized as non-clients of VAC.

Agreement with statements about home page

Members of the Veterans community were asked to indicate their level of agreement with several statements about the version of the home page they were shown during the survey. From eight to nine in ten agree at least somewhat with the positive statements, in both survey waves. In Wave 2, just under nine in ten (85%) agree to some extent the page looks like an authentic Government of Canada (GC) web page. In contrast, around one-quarter agree to some extent there is too much content on the page, and around one in seven agree the page is confusing or disorganized.

In both waves, strong agreement is highest that the wording used is very clear, with Wave 2 respondents being somewhat more likely to indicate this. Those in Wave 2 are also somewhat more likely than those in Wave 1 to strongly agree it is clear where to get additional information, or that the design is appealing. Strong agreement with other statements is fairly similar by wave.

Strong agreement with statements about home page
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Strong agreement with statements about home page	Wave 1 (n=518)	Wave 2 (n=522)
The wording used on this page is very clear	56%	63%^¥
I would be able to find helpful information by using this page	53%	56%
It is clear where to go to get additional information	47%	53%
Wave 2 only: This looks like an authentic Government of Canada web page	-	50%
The design (look and feel) of this page is appealing	42%	48%
I like the images on this page	35%	36%
There is too much content on this page	6%	7%
This page is confusing or disorganized	3%	4%

Q14 Please indicate your level of agreement with the following questions about the page you just saw.

Overall agreement with these statements is generally quite similar by type of respondent (Veteran or family member), although in Wave 2 agreement is somewhat higher among family members than Veterans that they like the images on the page, that the wording is clear, that it looks like an authentic GC page, and that it is clear where to get additional information. Responses are quite similar by region and gender, and there are no clear response patterns by age.

In both waves, strongly agreeing with positive statements about the home page is generally higher among family members and (relatedly) women than among Veterans themselves and men and is higher among those who indicate (later in the survey) that the veterans.gc.ca site is better than other government websites. Strong agreement that the wording is clear and that they would be able to find useful information by using the page is somewhat higher among those age 55 and over; strong agreement that the wording used is clear is in both waves linked to being an infrequent or non-user of veterans.gc.ca.

Ease of finding information on the home page

Around three-quarters in each wave say the organization of the home page they viewed makes it at least somewhat easy to find information; around four in ten in each wave say it is very easy. Around one in six in either wave say it is made neither easy nor difficult.

Ease of finding information on the home page

If organization of page makes finding information easy or difficult	Very easy	Somewhat easy	Neither easy nor difficult	Difficult (somewhat/very)	Not sure
Wave 1 (n=518)	37%	36%	17%	8%	2%
Wave 2 (n=522)	42%	34%	16%	6%	2%

Q15 Does the way this page is organized make it easy or difficult to find information?

Saying the organization of the home page makes it very easy to find information is higher among immediate family members of Veterans, women, and those age 65 and over, and is linked to thinking the veterans.gc.ca website is a lot better than other government sites.

E. Web page testing – disability page

Things liked about disability page

Around half of survey respondents in each wave were randomly selected to view the disability benefits page, and were asked after viewing it to indicate, top-of-mind without prompting, what they liked about it. As with the home page, top mentions in both waves include the page being informative, well-designed and professional, organized, straightforward and using good/readable fonts.

The spontaneous comments made are generally quite similar between the two waves. Wave 2 viewers are somewhat more likely than those seeing the Wave 1 page to say it is well-designed, organized or has good fonts; Wave 1 viewers are somewhat more likely to mention it being uncluttered or having just the right amount of information, or liking the information categorization.

Things liked about disability page
(mentioned by 3% or more in Wave 2)
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Things liked about disability page	Wave 1 (n=259)	Wave 2 (n=261)
Informative/provides needed info/details on each service	16%	21%
Well-designed/professional layout/visually appealing	11%	18%^¥
Organized/easy to find what I am looking for/clear menu	10%	16%^¥
Simple and straightforward/easy to follow/steps laid out	16%	12%
Good/large fonts choice/easy to read	3%	10%^¥
Lot of links to services/categories for all interests, needs	5%	9%
Clear language/easy to understand	6%	8%
Easy to navigate/user-friendly	6%	8%
Bold/clear to see choices/subjects	3%	8%
Like all of it/everything/good/what I need	5%	7%
Clean/uncluttered/not busy/just the right amount of info	15%^¥	4%
Info organized in categorized links for ease of searching/finding	8%^¥	3%
Lists most relevant information and services/answers FAQs	-	3%
The purpose for each link was clearly indicated	-	3%
Nothing/did not like anything	9%	5%

Q16 What, if anything, did you like about the page you just saw, overall?
BASE: THOSE VIEWING THIS PAGE

Responses are fairly consistent, with no clear patterns by subgroup that are common to both waves (seeing distinct patterns might have suggested key groups were looking for specific features). In Wave 2, mentioning the fonts is higher among women than men, and among Anglophones, who are more likely than Francophones to mention it being well-designed/visually appealing. Francophones are more likely than Anglophones to say they like everything about it. Saying it is organized is higher among those who are not previous users of the website.

Things disliked about disability page

Those viewing the disability benefits page were asked what they dislike about the page they saw. As with the home page, around six in ten in either wave say there is nothing about it they especially dislike. Fewer than one in ten make any specific mention, with the most common ones being that there is a lot of information to scroll through, too much text (related to lots of information), or not particularly liking the appearance of the page. Disliked mentions are similar by wave.

Things disliked about disability page
(mentioned by 2% or more in Wave 2)
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Things disliked about disability page	Wave 1 (n=259)	Wave 2 (n=261)
Dislike nothing/all is good	57%	64%
A lot of information on one page to scroll through	3%	7%^¥
Too much text/too many details	5%	5%
Dislike appearance/design/layout of page/not visually attractive	5%	4%
Dislike colours used/could be difficult to see/read	2%	3%
Looks boring/bland/nothing attracts attention	5%	2%
Lack of graphics/pictures associated with topics	3%	2%
Confusing/not clear enough to use	2%	2%
Some tabs are not easy to find/should be highlighted	-	2%^¥
Hard to find items/information looking for/not well-organized	1%	2%

Q17 And what, if anything, did you dislike about the page?
BASE: THOSE VIEWING THIS PAGE

Mentions are quite similar by subgroup, with only small proportions indicating specific dislikes. That there is nothing to dislike about the page is the dominant response, in both waves. In Wave 2, indicating they like everything is higher among CAF Veterans than reservists/rangers, highest in Quebec, and higher among those age 65 and over and those extrapolated to be VAC clients. Saying there is nothing to dislike is higher among those with high school or less education, and decreases as education increases, but is still over half among those with a post-graduate education.

Agreement with statements about disability page

Those seeing the disability page were asked about their level of agreement with statements about it. Majorities agree at least somewhat with the positive statements in both waves, but Wave 2 respondents are more likely than Wave 1 to agree to some extent that it is clear where to go to find additional information, or to think the design is appealing. Wave 1 viewers are more likely than those in Wave 2 to agree to some extent that the page is confusing or disorganized; their views on whether there is too much content are similar.

Level of strong agreement is fairly similar between the two waves, as shown in the table below, although Wave 2 viewers are more likely than Wave 1 to strongly agree that the wording used is very clear, or that the design is appealing.

In Wave 2, just under nine in ten (88%) agree to some extent that the page looks like an authentic Government of Canada web page, and half agree strongly with this.

Strong agreement with statements about disability page
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Strong agreement with statements about disability page	Wave 1 (n=259)	Wave 2 (n=261)
I would be able to find helpful information by using this page	55%	59%
It is clear where to go to get additional information	52%	59%
The wording used on this page is very clear	50%	61%^¥
The design (look and feel) of this page is appealing	30%	46%^¥
There is too much content on this page	12%	9%
This page is confusing or disorganized	5%	3%
Wave 2 only: This looks like an authentic Government of Canada web page	-	52%
I would be able to find helpful information by using this page	55%	59%

Q18 Please indicate your level of agreement with the following questions about the page you just saw.
BASE: THOSE VIEWING THIS PAGE

In both waves, strong agreement with all positive statements tends to be higher among women than men. Also in both waves, family members of Veterans are the most likely to strong agree they would be able to find useful information by using the page, or that the design is appealing. Strong agreement with positive statements is also linked to saying the VAC website is better than other government sites.

Ease of finding information on the disability page

Seven in ten in Wave 1 and three-quarters in Wave 2 (statistically similar proportions) indicate the way the disability benefits page they viewed is organized makes it at least somewhat easy to find information. Around one in in six in either wave say the organization does not impact the ease of locating information. One in ten in Wave 1 and seven percent in Wave 2 say finding information is made somewhat or very difficult due to the page organization.

Ease of finding information on the disability page

If organization of page makes finding information easy or difficult	Very easy	Somewhat easy	Neither easy nor difficult	Difficult (somewhat/very)	Not sure
Wave 1 (n=259)	37%	34%	18%	10%	2%
Wave 2 (n=261)	45%	31%	16%	7%	1%

Q19 Does the way this page is organized make it easy or difficult to find information?
BASE: THOSE VIEWING THIS PAGE

Saying the page’s organization makes it at least somewhat easy to find information is the dominant position in both waves. That it is made very easy is highest in both waves among immediate family members and those who think the VAC sites is better than other government sites, and among those age 65 and older, Anglophones and non-VAC clients in Wave 2.

F. Web page testing – rehabilitation page

Things liked about rehabilitation page

Around half of survey respondents in each wave were randomly selected to view the rehabilitation page, and were asked to indicate what they liked about it, recording their comments verbatim. Top mentions in both waves include the page being organized (higher in Wave 2), informative, simple and straightforward, and being well-designed.

Things liked about rehabilitation page
(mentioned by 3% or more in Wave 2)
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Things liked about rehabilitation page	Wave 1 (n=254)	Wave 2 (n=261)
Organized/easy to find what I am looking for/clear menu	14%	26%^¥
Informative/provides needed info/details on each service	17%	19%
Simple and straightforward/easy to follow/steps laid out	19%	17%
Well-designed/professional layout/visually appealing	11%	13%
Links to services/categories for all interests, needs of vets	1%	10%^¥
Good/large fonts choice/easy to read	8%	9%
Easy to navigate/user-friendly	7%	7%
Clear language/easy to understand	6%	7%
Like all of it/everything/good/what I need	5%	7%
Clean/uncluttered/not busy/just the right amount of info	8%	4%
Seems comprehensive/thorough	-	4%^¥
Easy to find contact information/listed at the bottom of page	1%	3%
Nothing/did not like anything	7%	9%

Q20 What, if anything, did you like about the page you just saw, overall?
BASE: THOSE VIEWING THIS PAGE

There are no clear response patterns by subgroup that are common to both waves. In Wave 2, mentioning it is organized/has a clear menu, or that there are a lot of links to services for all interests and needs of a Veteran, is somewhat higher among family members than Veterans, and among those who think the VAC site is better than other government sites. That it is simple and straightforward is cited more by women than men, and by those in the Western provinces and Ontarians than by Quebecers and Atlantic region residents.

Things disliked about rehabilitation page

Viewers of the rehabilitation benefits page were asked about things they might dislike about it. Again, majorities of at six in ten in either wave say there is nothing they dislike. Specific dislikes are generally similar by wave, with somewhat more in Wave 2 mentioning there being too much detail/text or a lot of information to scroll through.

Things disliked about rehabilitation page
(mentioned by 2% or more in Wave 2)
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Things disliked about rehabilitation page	Wave 1 (n=254)	Wave 2 (n=261)
Nothing I don’t like/all is good	62%	63%
Too much text/too many details	3%	8%^¥
A lot of info on one page to scroll through	1%	5%^¥
Dislike colours used/could be difficult to see/read	3%	4%
Looks boring/bland/nothing attracts attention	2%	4%
Dislike appearance/design/layout/not visually attractive	3%	3%
Lack of graphics/pictures associated with topics	<1%	3%^¥
Size of print/fonts too small for an old vet to read	<1%	2%

Q21 And what, if anything, did you dislike about the page?
BASE: THOSE VIEWING THIS PAGE

Mentions are generally consistent by subgroup, with only small proportions mentioning specific dislikes. That there is nothing to dislike about the page is the dominant response, in both waves. In Wave 2, indicating they like everything is higher among CAF Veterans than reservists/rangers. There are no regional differences to note in saying everything is good, and no clear pattern by age. As with the disability page, saying there is nothing to dislike is highest among those with high school or less education, and decreases as education increases, but is still a six in ten majority of those with a post-graduate degree.

Agreement with statements about rehabilitation page

Those seeing the rehabilitation page were asked to indicate how much they agree or disagree with statements about it. From three-quarters up to just under nine in ten in Wave 1 agree at least somewhat with positive statements about the page. Majorities of at least eight in ten also agree with all positive statements in Wave 2, and over nine in ten agree at least somewhat they would be able to find helpful information by using the page (94%) or that it is clear where to go to get more information (93%).

When it comes to two negative statements, only minorities agree to some extent there is too much content on the page (14% in Wave 1 and 31% in Wave 2, statistically higher), or that the page is confusing or disorganized (17% in Wave 1, 18% in Wave 2).

In both waves, strong agreement is highest that the wording used is very clear, with Wave 2 respondents being more likely to indicate this. Those in Wave 2 are also more likely than those in Wave 1 to strongly agree it is clear where to get additional information, and to say there is too much content. Strong agreement with other statements is fairly similar by wave.

In Wave 2, a strong majority of just under nine in ten agree to some extent (and half agree strongly) that the page looks like an authentic Government of Canada web page.

Strong agreement with statements about rehabilitation page
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Strong agreement with statements about rehabilitation page	Wave 1 (n=254)	Wave 2 (n=261)
The wording used on this page is very clear	52%	61%^¥
I would be able to find helpful information by using this page	50%	58%
It is clear where to go to get additional information	43%	59%^¥
The design (look and feel) of this page is appealing	38%	44%
Wave 1 only: I like the images on this page	38%	-
This page is confusing or disorganized	6%	7%
There is too much content on this page	5%	11%^¥
Wave 2 only: This looks like an authentic Government of Canada web page	-	52%

Q22 Please indicate your level of agreement with the following questions about the page you just saw.
BASE: THOSE VIEWING THIS PAGE

There are no clear response patterns common to both waves, In Wave 2, strong agreement with all positive statements is higher among women and those who believe the VAC website is a lot better than that of other government entities.

Ease of finding information on the rehabilitation page

Just under three-quarters in Wave 1 and about eight in ten in Wave 2 say the organization of the rehabilitation page they viewed makes it at least somewhat easy to find information; these proportions are statistically similar. Four in ten in Wave 1 and just under half in Wave 2 say it is made very easy. Similar proportions of around one in ten each say it is made neither easy nor difficult, or made difficult to some extent.

Ease of finding information on the rehabilitation page

If organization of page makes finding information easy or difficult	Very easy	Somewhat easy	Neither easy nor difficult	Difficult (somewhat/very)	Not sure
Wave 1 (n=254)	39%	34%	15%	10%	2%
Wave 2 (n=261)	46%	33%	10%	10%	<1%

Q23 Does the way this page is organized make it easy or difficult to find information?
BASE: THOSE VIEWING THIS PAGE

Saying the page’s organization makes it at least somewhat easy to find information is the dominant position in both waves. Saying the organization makes finding things very easy is highest in Wave 2 among Atlantic Canadians, women, those age 18 to 34, those in households with incomes of $150,000 or more, and those who believe the VAC website to be lot better than other government information sites.

G. Web page testing – Veterans Independence Program (VIP) page

Things liked about VIP page

About half of survey respondents in each wave were randomly selected to view the Veterans Independence Program (VIP) page, and were asked after viewing it to indicate, top-of-mind without prompting, what they liked about it. Top mentions in both waves include the page being organized (higher in Wave 2), informative, simple and straightforward, and using good/readable fonts (higher in Wave 2). One in ten in Wave 2 say they like all of it.

Things liked about VIP page
(mentioned by 3% or more in Wave 2)
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Things liked about VIP page	Wave 1 (n=264)	Wave 2 (n=261)
Organized/easy to find what I am looking for/clear menu	16%	23%^¥
Informative/provides needed info/details on each service	18%	22%
Simple and straightforward/easy to follow/steps laid out	15%	12%
Good/large fonts choice/easy to read	6%	12%^¥
Like all of it/everything/good/what I need	3%	10%^¥
Well-designed/professional layout/visually appealing	11%	9%
Lot of links to services/categories for all interests, needs	6%	9%
Clear language/easy to understand	3%	7%^¥
Easy to navigate/user-friendly	6%	6%
Seems comprehensive/thorough	-	4%
Nothing/did not like anything	9%	8%

Q24 What, if anything, did you like about the page you just saw, overall?
BASE: THOSE VIEWING THIS PAGE

Responses are generally similar by subgroup in both waves, with no overarching patterns shared. In Wave 2, saying the VIP page is organized is highest among Quebec residents and Francophones; Atlantic Canadians are the most likely to mention it being informative. Mentioning it is simple and straightforward is highest among seniors. Saying it is informative or simple/straightforward is linked to thinking the VAC site is a lot better than that of other government organizations, and to non-VAC clients.

Things disliked about VIP page

When asked about things they might dislike about the VIP page, as with other pages, majorities of around six in ten in either wave have no dislikes to mention. Only very small proportions indicate anything specific, with the colours being the most mentioned, but by only five percent in Wave 2. The Wave 2 new design showed some signs of still being a work in progress, with a small number viewing the French version of the page mentioning a translation-related issue, although some of these mentions may be related to links being disabled in the survey environment’s version of the page.

Things disliked about VIP page
(mentioned by 2% or more in Wave 2)
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Things disliked about VIP page	Wave 1 (n=264)	Wave 2 (n=261)
Nothing I don’t like/all is good	57%	60%
Dislike colours used/difficult to see/read/should use cheerful colours (e.g., red)	1%	5%^¥
A lot of information on one page to scroll through	4%	4%
Looks boring/bland/nothing attracts attention	2%	4%
French language issues (e.g., missing translations)	-	3%^¥
Dislike appearance/design/layout of page/not visually attractive	5%	2%
Lack of graphics/pictures associated with topics	1%	2%
Does not provide contact information/not easy to find/not prominent	-	2%

Q25 And what, if anything, did you dislike about the page?
BASE: THOSE VIEWING THIS PAGE

Responses are fairly similar across subgroups, with small proportions mentioning individual negative impressions. Saying all is good is the dominant response across subgroups, in both waves. In Wave 1, saying they like everything increases as age increases, and decreases as level of education increases; and is higher among non-clients, non-users of the VAC website, and those who think the website is a lot better than other government sites. In Wave 2, liking everything about the page is highest among CAF Veterans, those age 50 to 54, and those with high school or less education.

Agreement with statements about VIP page

Those seeing the VIP page were asked to indicate level of agreement with statements about it. From three-quarters to nine in ten agree at least somewhat with the positive statements in Wave 1; and from eight to nine in ten in do so in Wave 2. In Wave 2, close to nine in ten (88%) agree to some extent the page looks like an authentic Government of Canada web page, and half agree strongly. Around one-quarter agree to some extent there is too much content on the page in either wave, with around one in seven agreeing the page is confusing or disorganized.

In both waves, strong agreement is quite comparable for three statements: that they would be able to find helpful information by using the page, that the wording is very clear, and that it is clear where to for to get more information. Strong agreement with other statements is also statistically similar by wave.

Strong agreement with statements about VIP page

Strong agreement with statements about VIP page	Wave 1 (n=264)	Wave 2 (n=261)
I would be able to find helpful information by using this page	53%	52%
The wording used on this page is very clear	51%	54%
It is clear where to go to get additional information	50%	52%
The design (look and feel) of this page is appealing	30%	38%
There is too much content on this page	6%	8%
This page is confusing or disorganized	4%	5%
Wave 2 only: This looks like an authentic Government of Canada web page	-	52%

Q26 Please indicate your level of agreement with the following questions about the page you just saw.
BASE: THOSE VIEWING THIS PAGE

There are no clear strong agreement patterns common to both waves, and responses are fairly similar by subgroup, with a few exceptions. In Wave 2, strong agreement with all positive statements is higher among those who believe the VAC website is a lot better than that of other government organizations. Strong agreement that they would be able to find helpful information, or that it is an authentic Government of Canada website, is higher among women than men. Seniors are among the most likely to agree strongly with several positive statements, including they would be able to find helpful information, the wording used is clear, and the design is appealing. Family members of Veterans and non-clients are the most likely to strongly agree the wording used is clear. Strong agreement that this page is an authentic Government of Canada site is somewhat higher among Atlantic region residents, women, those not at all familiar with VA’s offerings, and non-clients.

Ease of finding information on the VIP page

The proportions in each wave saying the organization of the VIP page they viewed makes it at least somewhat easy to find information is statistically similar (70% Wave 1, 76% Wave 2); around four in ten in each wave say it is made very easy. Two in ten in Wave 1 and one in six in Wave 2 say it is made neither easy nor difficult; small proportions in either wave say finding information is made more difficult because of the page’s organization.

Ease of finding information on the VIP page

If organization of page makes finding information easy or difficult	Very easy	Somewhat easy	Neither easy nor difficult	Difficult (somewhat/very)	Not sure
Wave 1 (n=264)	40%	30%	22%	5%	2%
Wave 2 (n=261)	44%	33%	16%	7%	1%

Q27 Does the way this page is organized make it easy or difficult to find information?
BASE: THOSE VIEWING THIS PAGE

Saying the page’s organization makes it at least somewhat easy to find information is the dominant position in both waves. That it is made very easy is highest in both waves among those who think the VAC site is better than other government sites. In Wave 2, it is also highest among women, those not at all familiar with VAC’s services, and non-clients.

H. Web page testing – health-related travel information page

Things liked about health-related travel information page

About half of survey respondents in each wave were randomly selected to view the health-related travel information page. After viewing the page they were asked to indicate, without prompting, what they liked about it. Of all the pages tested in the surveys, this is the one with the most extreme differences of opinion between the two waves, with the Wave 2 version being clearly preferred over the original design. It should be noted that this page was fully redesigned.

A number of spontaneous comments are common to the two waves, but Wave 2 respondents seeing the newly designed health-related travel page are notably more likely than those in Wave 1 to say the page is organized, has good fonts, and is well-designed or easy to navigate. More Wave 1 respondents than in Wave 2 spontaneously reveal there is nothing they especially like about the page.

Things liked about health-related travel information page
(mentioned by 3% or more in Wave 2)
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Things liked about health-related travel information page	Wave 1 (n=259)	Wave 2 (n=261)
Organized/easy to find what I am looking for/clear menu	9%	25%^¥
Informative/provides needed info/details on each service	21%	18%
Simple and straightforward/easy to follow/steps laid out	11%	15%
Good/large fonts choice/easy to read	6%	14%^¥
Well-designed/professional layout/visually appealing	7%	13%^¥
Easy to navigate/user-friendly	3%	9%^¥
Clear language/easy to understand	4%	7%
Lot of links to services/categories for all interests, needs	6%	6%
Clean/uncluttered/not busy/just the right amount of info	2%	6%^¥
Like all of it/everything/good/what I need	6%	5%
Provides information on reimbursement of travel expenses	-	4%^¥
Nothing/did not like anything	13%^¥	6%

Q28 What, if anything, did you like about the page you just saw, overall?
BASE: THOSE VIEWING THIS PAGE

There are no clear patterns by subgroup that are common to both waves. In Wave 1, women are more likely than men to think it is organized; and Quebec residents and those with education below the university degree level are the most likely to say they like all of it. Saying there is nothing they like about it is higher among Ontarians and those age 35 to 49. Wave 2 responses are fairly consistent by most subgroups. There are no notable differences by how familiar someone is with the VAC website, but mentions of the page being organized, having good fonts or using clear language are higher among those who later in the survey indicate the VAC website is a lot better than other government sites.

Things disliked about health-related travel information page

Those viewing the health-related travel information page were also asked what they dislike about the page. While six in ten in Wave 2 say there is nothing they dislike, similar to the other pages, just under half say this in Wave 1, a notable difference.

For Wave 1, the main dislikes are that there is too much text, a lot of information to scroll through, and that it reads like a book/lacks grouping of information (mentioned by 6%, compared to only 1% in Wave 2).

Things disliked about health-related travel information page
(mentioned by 3% or more in Wave 2)
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Things disliked about health-related travel information page	Wave 1 (n=259)	Wave 2 (n=261)
Nothing/all is good	48%	62%^¥
Dislike colours used/difficult to see/read/should use cheerful colours (e.g., red)	1%	6%^¥
Looks boring/bland/nothing attracts attention	2%	5%
A lot of information on one page to scroll through	6%	3%
Too much text/too many details	10%^¥	2%
Services don't apply to me/access conditions too strict	1%	2%
Generic/same look and feel of all government websites	4%	2%
Dislike appearance/design/layout of page/not visually attractive	4%	2%
Does not provide contact information/not easy to find/not prominent	-	2%
Too busy/looks crowded/not easy to read	4%	2%
Confusing/not clear enough to use	4%	2%

Q29 And what, if anything, did you dislike about the page?
BASE: THOSE VIEWING THIS PAGE

Mentions are generally consistent by subgroup, with only small proportions mentioning the specific dislikes. That there is nothing to dislike about the page is the dominant response, in both waves. In Wave 2, indicating they like everything about the page/nothing disliked is higher among Quebec residents and Francophones, those who are very familiar with VAC services, and those who think the VAC site is a lot better than other government sites.

Agreement with statements about health-related travel information page

Viewers of the health-related travel information page were asked their level of agreement with statements about it. There is a strong difference in results between the two waves. In Wave 1, from two-thirds to just under nine in ten agree to some extent with the positive statements, with the lowest agreement being that the design is appealing. Overall agreement in Wave 2 ranges from over eight to over nine in ten for the positive statements, including over eight in ten approving of the design. As with other pages viewed in Wave 2, just under nine in ten (85%) agree to some extent the page looks like an authentic Government of Canada web page. That there is too much content on the page is the opinion of four in ten in Wave 1, but only two in ten in Wave 2, a statistically significant decrease. As well, the proportion agreeing to some extent that the page is confusing or disorganized decreases from one-quarter in Wave 1 to just over one in ten in Wave 2.

Strong agreement is notably higher for all positive statements in Wave 2 compared to Wave 1. In Wave 2, three statements are equally high, with the lowest (but still close to half) being for the statement about the design being appealing.

Strong agreement with statements about health-related travel information page
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

Strong agreement with statements about health-related travel information page	Wave 1 (n=259)	Wave 2 (n=261)
It is clear where to go to get additional information	44%	61%^¥
I would be able to find helpful information by using this page	43%	62%^¥
The wording used on this page is very clear	36%	62%^¥
The design (look and feel) of this page is appealing	22%	46%^¥
There is too much content on this page	13%^¥	7%
This page is confusing or disorganized	5%	4%
Wave 2 only: This looks like an authentic Government of Canada web page	-	57%

Q30 Please indicate your level of agreement with the following questions about the page you just saw.
BASE: THOSE VIEWING THIS PAGE

In Wave 2, CAF Veterans are less likely than other types to strongly agree with positive statements about the health-related travel information page; however, it should be noted CAF Veterans are as likely to agree overall as others. They are no less likely to strongly agree than CAF Veterans did in Wave 1 on most statements, and are statistically less likely to strongly agree there is too much content on the page than was the case in Wave 1.

In Wave 2, strong agreement with all positive statements tends to be higher among women than men, and among seniors compared to their younger counterparts. Strong agreement with positive statements is also higher among those with household incomes from $40,000 to under $60,000, those familiar with VAC service offerings, and those who think the VAC website is a lot better than other government sites.

Ease of finding information on the health-related travel information page

Opinions regarding the impact of the organization of the health-related travel page on finding information vary notably between the two waves. Six in ten viewing the Wave 1 version say the organization makes it easy to some extent, compared to nearly eight in ten saying this in Wave 2. In Wave 1, one-quarter say it is made very easy, compared to close to half saying this in Wave 2; these results are statistically distinct between the two waves.

Similar proportions in each wave say the page’s organization makes it neither easy nor difficult to find information, but those viewing the page in Wave 1 are notably more likely to feel it makes finding information difficult to some extent (16%, compared to only 5% in Wave 2).

Ease of finding information on the health-related travel information page
(bold/¥ symbol indicates proportions that are statistically higher than in the other wave)

If organization of page makes finding information easy or difficult	Very easy	Somewhat easy	Neither easy nor difficult	Difficult (somewhat/very)	Not sure
Wave 1 (n=259)	25%	36%	21%	16%^¥	1%
Wave 2 (n=261)	48%^¥	30%	16%	5%	1%

Q31 Does the way this page is organized make it easy or difficult to find information?
BASE: THOSE VIEWING THIS PAGE

As with other pages, saying the health-related travel information page’s organization makes it at least somewhat easy to find information is the dominant position in both waves. That it is made very easy is highest in both waves among those who think the VAC site is better than other government sites. In Wave 2, it is highest among seniors and those very familiar with VAC’s services. It is lower among CAF Veterans than other types, but somewhat higher than was the case in Wave 1.

I. Profile of respondents

Type of respondent

The survey samples were designed to obtain feedback from Veterans (50%) and family members of Veterans (50%). Breakdowns of Veteran and family member types are very similar between the two surveys.

Type of respondent	Wave 1 (n=518)	Wave 2 (n=522)
Net: Veteran	51%	50%
CAF Veteran	32%	27%
Reserve/ranger	14%	18%
RCMP	7%	7%
Net: Family member	50%	52%
Immediate family	40%	41%
Other family	10%	11%

Demographics

The demographic distributions of Veteran and family member types are consistent for the two waves.

Demographics	Wave 1 (n=518)	Wave 2 (n=522)
Age of respondent
18-34	17%	12%
35-49	22%	23%
50-54	15%	16%
55-64	22%	18%
65-74	20%	23%
75+	3%	7%
Gender
Male	65%	62%
Female	34%	38%
Prefer not to answer	1%	1%
Language spoken at home
English	78%	78%
French	22%	25%
Other	2%	1%

Socioeconomic factors

The socioeconomic breakdowns of the samples for the two waves are very comparable.

Socioeconomic factors	Wave 1 (n=518)	Wave 2 (n=522)
Household income
Under $20,000	2%	1%
$20,000 to just under $40,000	6%	7%
$40,000 to just under $60,000	13%	11%
$60,000 to just under $80,000	15%	16%
$80,000 to just under $100,000	17%	18%
$100,000 to just under $150,000	19%	21%
$150,000 and above	14%	13%
Prefer not to answer	14%	13%
Education
Up to high school	1%	1%
Some high school	1%	2%
High school diploma or equivalent	15%	13%
Registered Apprenticeship/trades certificate/diploma	6%	5%
College/CEGEP/non-university certificate/diploma	24%	23%
University certificate or diploma below bachelor’s level	8%	9%
Bachelor’s degree	27%	29%
Post-graduate degree above bachelor’s level	18%	18%
Prefer not to answer	1%	1%
Employment status
Working full-time, that is, 35 or more hours per week	46%	46%
Working part-time, that is, less than 35 hours per week	7%	7%
Self-employed	7%	7%
Unemployed, but looking for work	2%	<1%
A student attending school full-time	3%	2%
Retired	30%	33%
Not in workforce (full-time homemaker/unemployed/not looking for work)	1%	2%
Other (not specified)	2%	2%
Prefer not to answer	2%	1%

II. Detailed findings – qualitative phase

The two qualitative components of this research were designed to gather more in-depth information from members of the Veteran Community on VAC’s digital communications products as well as on the redesigned website. The qualitative research findings described in the first five sub-sections (Sections A through F) were obtained via the 12 focus groups. These focus groups examined the content, clarity, effectiveness and appeal of VAC communications materials, as well as how the products themselves fare as communications tools to motivate Veterans and their families to seek VAC services. Products for testing included communication products such as social media posts, infographics, facts sheets and storyboard ad concepts.

The findings for Section G (Website Evaluation) were derived from the series of 20 one-on-one online interviews. Once new sections of the VAC website had been prepared, Environics conducted a deeper dive into the website sections, through individual in-depth interviews. The interviewers followed participants as they navigated through specific redesigned services pages and sections, providing their impressions and comments.

A. Media and social media consumption

Veterans are active media consumers, and many have joined Veterans-specific groups on Facebook to stay in touch with others, share resources, and receive or provide support to other Veterans. The VAC website is not typically used as a general resource and awareness of VAC’s social media presence is low.

Media consumption. Focus group participants are active media consumers. Many follow the news on a regular basis and through a variety of sources and formats. While radio consumption is rather rare, television, Internet and newspapers are fairly common. Younger participants seem to be more active consumers of online news, although some of the older Veterans also use online resources. The trend is reversed when it comes to newspapers, where we see older Veterans more likely to follow the news using this medium compared to younger participants – but, again, not exclusively as younger participants did mention using newspapers, although these tend to be free dailies (e.g., Metro).

Participants who have experienced, or who are still experiencing, mental health problems – in particular Post-traumatic Stress Disorder (PTSD) – explained that they actively avoid the media and the news since it could activate an emotional or psychological trigger. Many of these participants also find it difficult to read articles of any sort, whether online or otherwise, since they cannot concentrate or focus enough to make it through the article.

Social media consumption. Use of social media is fairly mixed across focus group participants. Some do not use it at all, whereas among users, there is a mix of active users (i.e., they actively search and share information) and passive users (i.e., they occasionally use social media and when they do, it is more to read what is sent to them rather than to engage, share or post anything).

Social media activity is limited almost exclusively to Facebook, where participants tend to either correspond with friends and family in general; but, more importantly, they are part of groups that have some connection to their past service, either in the military or the RCMP. For example, several people reported being a member of a Facebook group for Veterans of a specific regiment or for Veterans with a particular area of interest. While many feel these groups help them stay in touch with others with whom they had served, the groups are also an important vehicle through which various “tips,” advice and resources are shared among Veterans. For some, it is a vehicle through which they seek and provide support.

Participants do not tend to resort to many other resources when they want to find out about things that relate to their experience as Veterans. This is mostly due to a lack of awareness of any other resources rather than a lack of desire to seek out or further explore resources at their disposal.

VAC website and social media presence. Use of the VAC website is limited to those who have a VAC Account, or those who have sought information or a specific form related to a benefit they were exploring or wanting to access. A few have been redirected to the website via social media. Otherwise, most participants did not tend to report casually browsing the website to see what it could offer or if there was anything new. In fact, many don’t even think to visit the VAC website, mostly because they either don’t consider themselves a “Veteran” or they aren’t aware of what VAC can provide in terms of resources and support.

Those familiar with the website are generally quite pleased with it. They consider it user-friendly and informative. There were some participants who feel it’s like “all the other Government websites” in terms of being dense, hard to navigate and challenging to use when seeking specific information or a specific form. It was difficult to ascertain, however, how recent these experiences were – so caution is warranted in considering this feedback.

In terms of VAC’s social media presence, very few were aware that VAC had a Facebook page or a YouTube channel or was active on Twitter. While many were interested in finding out more about the Facebook page, very few were interested in VAC on Twitter.

B. Top-of-mind issues for Veterans

Participants believe Veterans face a number of challenges, including adjusting to civilian life, mental health, accessing benefits, and lack of recognition. Many are not informed about VAC services beyond mental/physical health care. Awareness could be increased via improved hand-off processes, direct outreach, and an increased presence in Veterans’ spaces (e.g., the Legion, social media).

Focus group participants were asked to list what they considered to be the biggest challenges or problems that Veterans in Canada have to deal with. The key themes that were raised included the following:

· Many explained that there was no effective hand-off between serving in the Canadian Forces or the RCMP and VAC, especially in terms of introducing almost-discharged individuals to VAC, helping them understand what is available to them in the way of support and benefits after they are discharged, and how to access those benefits. It appears RCMP Veterans are provided some orientation before being discharged, although some would argue that it was a very short meeting and, overall, insufficient.

· Not only do participants feel there is no hand-off, but that there is no follow-up with recently discharged Veterans to “check in” to make sure that they are on a positive path and have landed on their feet.

· Many also explained that it is difficult to access benefits (e.g., disability pension). While nobody had issues with the application process (such as finding the right form), many did have issues with the timeliness or how long it takes for a decision. The challenges related to receiving benefits to which they believe they are rightfully entitled leave some feeling like they are not appreciated as Veterans by their own Government.

· Somewhat related to the first issue (that there is no “hand-off” to VAC), some explained that Veterans often struggle with social reintegration – in other words, the transition to civilian life. This, in part, explains the level of isolation that some feel, that there is no sense of belonging, and that they don’t understand civilians and civilians don’t understand them.

· Challenges with mental health were specifically noted by some participants.

· A few participants struggle with the lack of recognition that Veterans get from both the general public and the federal government.

· A few participants also believe that some Veterans face important financial difficulties, due in some part to difficulty finding “civilian work” and to challenges related to their mental and/or physical health.

VAC outreach. Other than perhaps support for mental and physical health, participants did not seem informed about any other service or support provided by VAC to address the main issues raised. Even regarding support for mental and physical health issues, a good number of Veterans were not aware of any services offered to them.

Those most likely to be aware of the services provided by VAC were also most likely to be “clients,” many of whom have a VAC Account, and have accessed some of the services or benefits directly themselves and recommended them to other Veterans. Nearly all these individuals had taken the initiative to contact VAC themselves rather than having been contacted by VAC.

There was widespread agreement that VAC offers a lot of services to Veterans that many Veterans do not know about. The main solutions offered by participants to rectify this situation would involve the following:

· A greater effort to inform individuals about VAC and the services it offers to everyone just prior to their discharge.

· For those already discharged, many were under the impression that the Government has their contact information and should be more proactive in contacting Veterans to inform them about VAC and the services it offers.

· The Government should work through the channels that Veterans tend to use or visit (e.g., the Legion, Facebook groups, etc.) to promote VAC and its services.

C. Evaluation of social media posts

Participants were fairly critical of six social media posts for not effectively conveying relevance or standing out from the clutter. The posts were broadly criticized for lacking explicit references to Veterans in imagery/graphics, and not always having a “hook”. If visuals were more effective, many would click through, as the programs seemed relevant.

Six existing social media postings from VAC on Facebook or other social media platforms were presented to participants. Participants were asked to write down what they thought about each one and to focus on whether they spark their curiosity; whether they would click on them; and if they are useful, clear and easy to understand. Participants were instructed not to evaluate the actual programs that were being referenced.

Once participants had written down their thoughts on each of the six posts, the moderator led a discussion on each one.

My VAC Account FB Post

The image below shows a Facebook post for My VAC Account.

[ALT TEXT: My VAC Account Facebook post – Facebook post with an image of a file folder on the left-hand side, with an image of a hand holding a smartphone with a checkmark in the middle. Below the hand are the words “My VAC Account” in large font and, below this, the words “Applying is faster and easier.” To the right, below an image of a Canadian flag and the words “Veterans Affairs Canada,” the text reads: “My VAC Account makes applying for programs and benefits faster and easier. Register today:” followed by a link that brings the visitor to the VAC website.]

There was some awareness of My VAC Account among a minority of participants in all the groups.

This concept was widely criticized by participants. An important weakness noted was that it did not suggest that it pertains to Veterans. Some would think it was an advertisement to apply for a bank loan. Some also felt the ad was for a service one would access on a smartphone. Some wondered if the image was of someone holding a can of beer. From a design perspective, some liked the clean look of the image, while others felt it was too simplistic and not sufficiently engaging.

Those most likely to appreciate the post were those already familiar with My VAC Account. As well, some felt that the only way they could determine that this ad was relevant for Veterans was by looking at the text on the right-hand side of the post, but many also argued that they would not even read that if the image did not first get their attention.

LifeSpeak Post

The image below shows a post for LifeSpeak.

[ALT TEXT: LifeSpeak Post: The image features the word “LifeSpeak” in large font with overlapping, multi-coloured rings to the right of this word, made to look like speech bubbles. Below this is written in smaller font: “Now available for Veterans, former RCMP members and their families.” To the right, below an image of a Canadian flag and the words “Veterans Affairs Canada,” the text reads: “Veterans, former RCMP members and their families can access a new, free, online self-help resource called LifeSpeak. This health and wellness library provides information and support from leading experts on a range of topics. Confidentiality is guaranteed in the comfort of your home 24/7.” Below this text, the text reads: “More information:” followed by a link that brings the visitor to the VAC website.]

There was no awareness of LifeSpeak among participants, and the text on the right-hand side received mixed reviews in terms of explaining what the service was. The text and the program name seemed to provide conflicting information. Some guessed it was a collection of resources one could access online; others wondered if it was a way to speak with someone, given the name of the service and that it promises confidentiality (which is something helplines typically promote). Notwithstanding the ambiguity of the text, some just felt the text was too long and that this alone would dissuade them from reading it. Francophones noted the program did not have a French name and therefore they felt excluded by the name “LifeSpeak”.

Participants felt that this post was more relevant to them, given the explicit reference to “Veterans, former RCMP members and their families.” Nonetheless, participants still believed that there needs to be some sort of image to “hook” them, especially since the image used (the overlapping speech bubbles) did nothing to get their attention or suggest that it was a service for Veterans.

RCMP members were quick to appreciate the fact that they were called out specifically in the text, since some do not necessarily refer to themselves as “Veterans.”

OSISS (Family) Post

The image below shows a post for a program called Operational Stress Injury Social Support (OSISS).

[ALT TEXT: Operational Stress Injury Social Support (OSISS) Post: The image features the shadow of a family (father, mother, baby and young child with their dog) with a clear blue sky at sunset in the background. The word “Family” in large font features prominently across the top. Below this is written in smaller font: “a Veteran’s most important ally on the path to recovery.” To the right, below an image of a Canadian flag and the words “Veterans Affairs Canada,” the text reads: “As a family member, you have an important role in your loved one's recovery. Seek support from a family peer support coordinator through Operational Stress Injury Social Support (OSISS).” Below this text, it says: “More information:” followed by a link that brings the visitor to the VAC website.

There was limited awareness of OSISS among participants, and the text on the right-hand side helped participants understand a little bit what the program was about. Participants generally agreed that a family is a Veteran’s most important ally on the path to recovery.

Visually, this was one of the more popular posts. The word “family” was noteworthy for many and, while the image of a young family also struck a chord with many with similar families, it did not prove appealing to Veterans who do not have a family (e.g., single, young couple, etc.).

The main improvement participants suggested for this concept was to more explicitly show a Veteran in the image.

Veteran Family Program

The image below shows a post for the Veteran Family Program.

[ALT TEXT: Veteran Family Program Post: The image features a blue background and in the centre is a cut-out of a family holding hands (father, mother and two young children). The words “Veteran Family Program:” in large font features prominently across the middle. Below this is written in smaller font: “Supporting medically-releasing CAF members, medically-released Veterans and their families.” To the right, below an image of a Canadian flag and the words “Veterans Affairs Canada,” the text reads: “#DYK The Veteran Family Program can be accessed online? Access these resources anywhere, anytime here,” followed by a link that brings the visitor to the VAC website.]

There was limited awareness of the Veteran Family Program among participants. The text on the right-hand side did little to help Anglophone participants understand what the program was about. Participants looking at the English concept felt the text was a little confusing, since it started with a question. The confusion was compounded for those who did not know what “DYK” meant. Francophones were more appreciative of the text in the margin, since it actually describes what the program is about.

From a design perspective, participants disliked the entire approach. They felt it lacked anything that would grab their attention, except perhaps the use of the word “Veteran.” A good number did not even notice the cut-out of a family in the background. Furthermore, the font style of the text embedded into the image, along with the amount of text used, did not seem to focus the reader on any given theme or key word – it stuck some as just a block of text. This was accentuated in the French sessions, where the block of text was larger.

It is also worth noting that a few Francophone participants did not know what “FAC” meant, since they are more accustomed to seeing “CAF”.

Balance

The image below shows a post that generally encourages Veterans to strive for “balance.” The English concept features a woman striking a yoga pose near water at sunrise/sunset, while the French concept features four stones balanced one on top of the other.

[ALT TEXT: Balance Post: The image features the shadow of a woman striking a yoga pose (balancing stick pose). She is on the shore with a sunset or sunrise in the background. The word “Balance” in large font features prominently across the top. Along the bottom of the image is written in smaller font: “Take care of yourself and keep your life in balance.” To the right, below an image of a Canadian flag and the words “Veterans Affairs Canada,” the text reads: “Functioning well physically, mentally, socially and spiritually = Good well-being. VAC can help you reach that balance. More information:” followed by a link that brings the visitor to the VAC website.]

Even though the image was different for Francophones and Anglophones, the image used for each was appreciated, although more so among Francophones than Anglophones. Some Anglophones were inclined to believe it was a post for a yoga studio, which was not of particular interest to them. Participants also liked the underlying message that the post was communicating – it was both relevant and simple.

The challenge with this concept remains that there is no direct connection to “Veterans” except for the text in the right-hand margin. So, other than feeling it was a nice image and a nice message, participants didn’t see the post specifically engaging them as Veterans, or compelling them to any specific call to action. That being said, many felt they would click on this post if it more directly referenced Veterans.

Centre of Excellence on PTSD

The image below shows a post for the Centre of Excellence on PTSD.

[ALT TEXT: Centre of Excellence on PTSD Post: The image features the words “Centre of Excellence on PTSD and Mental Health Conditions” in large font across the top. Below this is written, in smaller font: “A virtual centre for information and resources on Veteran-specific mental health treatment.” In the background, we see the profile of two heads facing each other with cogs in and above the heads. To the right, below an image of a Canadian flag and the words “Veterans Affairs Canada,” the text reads: “The new Centre of Excellence (CoE) on PTSD will be the virtual centre for information and resources on Veteran-specific mental health treatment. More information:” followed by a link that brings the visitor to the VAC website.]

Participants did not like the visual treatment for this post. Similar to feedback provided on the other posts, participants did not feel there was any imagery related to Veterans that would get their attention or suggest the post was relevant for them. In fact, some did not even notice the profiles and cogs used in the background of this post.

Furthermore, the text with the largest font, which is what most read first, did not compel them to read more or suggest that the post was relevant to them. Some did not even read the sub-text on the left-hand side.

Those who read the entire post seemed to appreciate the importance of the CoE – but, presented out of context and blended in with their other social media posts, few believe they would have clicked on this post to find out more.

General observations regarding VAC social media posts

Irrespective of the concept being considered, a few observations were made by participants that could apply to nearly all, if not all, posts. They included:

· Participants are looking for more explicit references to Veterans in the images used.

· Some questioned why military logos, badges or similar symbols were not used. They argued that Veterans had spent their entire service surrounded by similar imagery and symbols, and would quickly notice these and gravitate toward them if they were used in a social media post. A few even suggested that VAC should come up with its own “brand” or badge that could be used on all its marketing material.

· Participants are not paying much attention to the websites, the FIP or any other text used in the footer, below the image on the left-hand side of each concept.

· Seeing as these are social media posts that will be blended in with all the other posts that they see, participants are looking for an image or a key word that will grab their attention. The simplicity of the imagery and the minimalistic approach to the main text used with the imagery were seen as critical to the success of any post – so that it not only cuts through the clutter, but could be clearly seen on smaller screens like on their smartphones.

D. Evaluation of infographics

The infographics evaluation underscored how good design, combined with effective interactive features, can lead to a successful product. Feedback stressed how participants were prepared to forego detail in exchange for headline-type information if there are opportunities to click on specific elements to access more detailed information.

Two infographic concepts were presented to participants. Participants were asked to write down what they thought about each one. Similar to the social media post evaluation exercise, participants were instructed not to evaluate the actual programs that were being referenced, but rather to focus on the visual elements of each concept. Once participants had written down their thoughts on each of the six posts, the moderator led a discussion on each one.

Infographic: The expansion of the Veteran Family Program

[ALT TEXT: Infographic: The expansion of the Veteran Family Program: The top of the image features white text on black background at the top, featuring the following title: “Expansion of the Veteran Family Program,” then a sub-title: “Supporting families who serve, where they serve” followed by this text: “We are expanding the Veteran Family Program. Effective April 1, 2018, medically-releasing CAF members, medically-released Veterans and their families can take advantage of VFP Transition Programs at all 32 Military Family Resource Centres across Canada, through the Family Information Line 1-800-866-4546 or CAFconnection.ca.” In the centre of the infographic, in large font is featured “Types of Services Available” with the following listed: Transition Programs, Referral Services, Advocacy, Intervention Support. Below these is a map of Canada in dark grey with the location of each of the 32 centres identified with a number. In the background, behind the map, is a cut-out of a family holding hands (father, mother and two young children). The bottom of the image, white text on black background, features VAC social media channels, their website and the Canada wordmark.]

The first infographic shown to participants was one that described the expansion of the Veteran Family Program.

Participants felt the design was interesting and well laid out. The map was considered visually appealing, although it should be noted that interest in the concept largely depended on the infographic being interactive. In other words, if this were on a piece of paper, participants would value it much less than if it were on a website and fully interactive. In particular, participants would want to click on the four types of services shown above the map, and they would want to click on the map to select the centre closest to their location. If the map is not interactive, it is seen as useless.

Many participants did not notice the cut out of the family in the background, and many did not see it as useful.

The text at the top of the infographic was seen as dense and hard to read, whereas many liked the succinct list of services presented above the map. In fact, some would just want to read the large title, the sub-title and then click into the list of four services available.

Infographic: Service Dog Pilot Study

[ALT TEXT: This infographic is titled “Service Dog Pilot Study Results” and sub-titled “Evaluating the effectiveness of service dogs for Veterans with PTSD.” It then reads “Results: The reported impacts for 18 Veterans with PTSD after acquiring a service dog, include:” Six impacts are listed using large numbers 1 to 6, each a different colour with three on the left of an image of a man with a service dog and three to the right of the image. The impacts listed are: 1) Decreased nightmares, improved sleep and a slight increase in physical activity; 2) No reduction in medication use; 3) Moderate, long lasting reduction in depressive symptoms; 4) Improved quality of life, more social integration in the community; 5) Decreased PTSD symptoms; 6) No reduction in reliance on a caregiver. Below the image of the man we can read: “For more information on the Service Dog Pilot Study, visit: veterans.gc.ca.” The bottom of the infographic features the VAC and the Government of Canada wordmarks.]

The second infographic shown to participants was one listing the results from a study that evaluated the effectiveness of service dogs for Veterans with PTSD.

This was one of the more popular visual concepts tested in the focus groups. Participants were particularly drawn to the layout of the information, the succinct nature of the information capsules, and the use of colour and a simple graphic. Participants indicated the infographic provides just enough information about the study, and clearly indicates where readers can go to access more information.

E. Evaluation of fact sheets

Most participants were comfortable with the language used on the fact sheets, the quantity of information provided and the general layout. This approach would seem to work for situations in which the client is keen on finding out more information about a product or service.

Three different fact sheets available on the VAC website were evaluated, although any given group only saw one fact sheet. The three fact sheets (available in the Appendix) were:

· Caregiver Recognition Benefits

· Career Transition Services

· Veterans Education and Training Benefit

Irrespective of their interest in the specific program, participants were asked to evaluate the fact sheet concept, the amount of information provided, the clarity of the language used, the use of images, and the use of scenarios and examples.

This exercise revealed that fact sheets are not for everyone. Those who struggle reading long-form copy couldn’t even read the entire fact sheet provided to them (which is not uncommon with Veterans, among whom mental health issues are more likely to exist). This prompted some participants to suggest a greater use of bullet points to break up blocks of text, making the text more inviting and easier to read. A few participants suggested that any text dedicated to explaining the Minister’s role or connection to the program was not necessary. The focus for them is understanding what the program offers and if it applies to them.

By and large, most participants were comfortable with the language used, the quantity of information provided, and the general layout. They appreciated the one-page format, which provided them just enough information to understand the main purpose of the program/benefit and assess the extent to which it could be relevant to them.

The use of examples and scenarios was generally appreciated. It gave readers a concrete application of how the program or benefit could work or apply. The main criticism of the examples was that they did not reflect the reader’s reality and until that would happen, examples were seen as having limited value for certain participants.

The images received mixed reviews; while some felt they didn’t add much value, others liked how they broke up the text and offered some visual variety.

F. Assessment of mental health concepts

Of the four ad concepts relating to mental health (A-D) tested, Anglophone participants tended to prefer Concept D, while Francophones preferred Concept C.

At the request of VAC, Environics conducted two online focus groups on December 10 (English) and 11 (French). Participants were members of the Veteran community across Canada; each received a cash incentive of $100 for participating in the research. The purpose of these groups was to test reaction to four different ad concepts addressing mental health issues. These were shown in storyboard form.

Concept A – “Parting clouds”

Anglophone participants generally did not this concept. It was described as dark, distant and formal. Francophones had more mixed views, but still saw it as dark and sombre, and not attention-grabbing. Some liked the metaphor of a clearing sky, but others saw it as dull. Some found the taglines were somewhat of a cliché and felt it was implying that “You did your service and now you are finished”. Other comments included the following:

“Campy, cliché, and dark.”

“Covers everything quick but comes across as a helpline number only.”

« Toute pareille et pas vraiment intéressant. »

Concept B – “You’re not alone”

Participants did not have a strong response to this concept. While some liked that the ad showed non-traditional Veterans, others felt it did not have a strong connection to Veterans. Some Francophones liked the link between mental and physical health, but others felt it was more like an ad for a fitness club. The images seemed disconnected from the words. Everyone noted that the images are all of women. Some said it made them nostalgic for being in the Forces, and they recognized the army boots.

All liked the image of someone alone with a dog, and the line “vous ne baisser pas les bras” went over well in French. It was noted in French that “ACC est la pour vous et votre famille” was a good line at the end, but should finish with “et vos proches”. Other comments included:

“I do like that it shows other types of Veterans. There’s a lot of young people that aren’t those old grizzled sergeants.”

“I did like the line that VAC is here for you and your loved ones.”

“I like the picture with the dog, but it could also show a family. Where are the men?”

Concept C – “Help”

Participants had a more positive reaction to this concept. The message that Veterans should make their well-being a priority was well-received, but a couple of people felt that the ad should use the term “support” rather than “help”. Again, in French, “vos proches” is more inclusive than “votre famille”. Everyone liked the overarching message that PTSD affects not just the Veteran but everyone around them.

Francophones appreciated how this concept showed emotion and symbolized someone who may actually be suffering, and so it was seen as more on point. They just wished it showed an actual person, rather than drawings. Some noted the need to include a phone number and action words, like “call us”. Other comments included:

“These people look real and are a variety of real people. You also want to get things early, before it’s a crisis. That’s why I say they should use the word ‘support’, rather than ‘help’.”

“I don’t like the drawings. As far as the message goes, I didn’t mind that. The message, that you should look after yourself, it’s inclusive.”

“It’s not an overall dark or depressing theme—it’s more warm. I like it more than some of the others.”

Concept D – “Part of the Family”

Anglophone participants reacted the most positively to this concept, liking the images and the message. Some said they would have liked to see a single person depicted as well. In French, it was seen as too idealistic and more like a travel brochure. Many who do not have a traditional family around them said it made them feel excluded, and maybe even more depressed. Participants all really liked the final image of a hand on a phone, as it showed action; it was mentioned that this image could be used with any of the concepts. Some said that, while it’s great that VAC provides services, it’s not a replacement for having an actual family. “Unité de soutiens” is good. Other comments included:

“I think maybe friends and family are a support unit. Not just family. I think this one speaks more militarily.”

“By saying services for you and your family, if the Veteran doesn’t jump at it, the family might.”

“I don’t have a family or children; I would change the channel or switch away. It’s good if you have a spouse, because it affects the whole family, I get that, but if you include the single men and women that don’t have a family, that would be better.”

Overall reactions to mental health ad concepts

All in all, participants responded most positively to Concept D in English, but in French they warmed more to concept C, as it showed an image of some in distress and had more emotion. Other points raised included:

· None of the concepts showed people in uniform, which some felt was a mistake.

· The inclusion of pictures of pets was appreciated.

· Participants reacted positively to non-traditional images of Veterans and showing people by themselves.

· Participants liked the use of the term “support” rather than “help”. They perceive “help” to refer to more urgent or crisis situations.

· Some felt the concepts needed to show more emotion – and that if the ad is to be aimed at someone who feels distressed, the concept should show that.

G. Website evaluation

The VAC website is largely seen as user-friendly, attractive and well laid out. Those familiar with the older site believe the current version is vastly improved. The home page and main dropdown menu offer the right topics and navigation flexibility.

Sources of information used. Participants in the one-on-one interviews – focused exclusively on evaluating the VAC website – were very likely to report using the VAC website to keep informed about issues related to Veterans. Other popular resources included general mass media, especially television and newspapers. There was also some use of social media, particularly Facebook. Only a few indicated that they do not access or seek out information related to Veterans.

Use of the VAC website. Many participants visit the VAC website and a good number were also familiar with the previous version of the website. The current website is especially used to access My VAC Account, and to seek out information on and apply for benefits and services. The consensus seemed to be that the current website is significantly improved compared to the previous version, especially in terms of being more user-friendly overall, providing a better user interface from the home page, and having an improved look and feel:

“It’s better than the last website because you don’t need glasses to read it. Not hard to find things.”

“It is very different, much more user-friendly, not too much on the page, good size font – readable, not cluttered, straightforward. I am not second-guessing where I am going – the first page has different buckets, you are focused right away, […], much easier to navigate.”

“It’s an improvement from the last website, which was far more busy and complicated than this one.”

General reactions to the current website. Irrespective of any experience with the previous website, participants generally liked the current VAC website. The key highlights for participants included:

· They feel like they can find what they’re looking for easily.

· The main links and the language used are easy to understand.

· The default font sizes are good.

There were not many overall concerns with the site, although there were a few concerns about the website being a bit dense in terms of text. One participant felt that the home page was difficult to navigate, although she was accessing the site from a tablet, which involved much more scrolling compared to how the site would be viewed/used on a computer.

Home page evaluation. Participants considered the home page easy to use, well laid out and “clean,” and found it allowed the visitor to quickly figure out which direction to take based on their information needs. The links featured on the home page were easy to find, easy to decipher and relevant:

"It appears to have what I expect to be there – how to find out more information about things or what is available to me as a Veteran."

Most liked the general manner in which information is organized on the home page. Everyone noticed the eight themes featured directly on the home page, and most appreciated the short descriptions provided for each one. Very few felt there was redundancy between the eight themes, and most felt they were all relevant.

· A few questioned the order in which the themes were presented on the home page – they did not see any particular logic to how themes were sorted. A few would have liked them presented in alphabetical order.

· One participant was not certain why the RCMP had its own link – it seemed to suggest that an RCMP Veteran should only be clicking there and nowhere else; and that, if that is the case, perhaps that link needs to be pulled out of the list and featured in a different way on the home page.

Very few participants could think of a key theme missing from the home page. A few suggestions included:

· Add a chat line.

· Links to non-profit groups that are supporting Veterans (e.g., homeless Veterans) to increase awareness and to donate/ support these causes.

· Getting ready to retire/transitioning to civilian life: "I would think a lot of the ones who are still serving are looking for stuff related to retirement, which would be a reason to go to this website."

· Survivor benefits should be a separate theme.

There was also a sense that the language used on the home page was easy to understand and devoid of any “lingo,” which some believe is often found on Government of Canada websites in general. The home page was also commended for not being overloaded with text, which again is seen as a common challenge on Government of Canada websites.

Choice of colours on the home page. In terms of general look and feel, most liked the colours selected, and the balance of images and text. Those who liked the colours felt that they made the site seem “crisp” and eye-catching, and reminded them of military tones:

“[…] you don’t want to make it pink or purple, or make it too crazy. It’s more traditional and more conservative; you still have more greys and blacks – still very military.”

A few would however like to see a bit more colour on the home page to make it more inviting and eye-catching:

“It is very blue, looks very sterile, there are some colours at the bottom but on the top it is very clinical.”

“It’s is not aesthetically something that grabs my attention.”

The current colours distinguish the VAC website from other Government of Canada websites, in a good way. Nonetheless, some would like to see a bit more of the colour red, especially in the flag located in the home page header.

Drop-down menu. Most participants didn’t notice the drop-down menu located in the upper left-hand side of the home page. Some felt this was a bit of a shame, since they really liked the layout and functionality of that menu once they became aware of it. Participants especially liked how they could quickly glance at the second layer of content without having to click into any given theme – in other words they could access more information while staying on the same page.

Those using a tablet also felt this menu was very practical, since it did not require them to scroll down on the home page to access the key themes.

Participants did not feel that much was missing from the menu (other than the same themes identified earlier). A few also questioned why this list was different from the main themes featured on the main home page, prompting a few to suggest that maybe themes like “Veterans’ Rights” and “Forms” should be more prominently listed on the main home page, while the drop-down menu should have a quick link to My VAC Account.

Evaluating the lower section of the home page. The bottom section of the home page was also well-received by participants, especially the ease with which they can find a phone number. They were hard pressed to find anything that could be missing; adding a chat line was noted. It was suggested that the link to the VAC Assistance Service be placed higher up on the webpage, given that those wanting to use it will want that information as readily as possible – and putting it at the bottom of the home page may not be ideal.

"I find it a bit strange that the help is at the bottom and not at the top, which makes it looks like an afterthought."

As well, a few participants were not entirely clear on how they would contact VAC by email.

Locating the social media information in this section seemed intuitive to most. If they were to follow VAC on social media, most would use Facebook, with LinkedIn a distant second.

Evaluating the second layer of information. Participants were asked to explore a particular theme in order to obtain their feedback on the next layer of web pages. Similar to the feedback obtained for the home page, participants generally liked the next page they got after clicking through the home page. They valued the easy layout of the information, the succinct headers and descriptors and felt confident they could assess whether they were headed in the right direction. They also liked that they did not need to scroll much, if at all, to access the content of the page.

· The landing page for My VAC Account was discussed with a few participants. This page was quite popular, especially its layout, the graphics used, and the amount of text on the page (which was appreciated for being fairly limited). Experienced users and those less familiar with the service both felt the landing page provided them with the information they needed. Participants recognized the icons for videos, and liked that those were available.

· A few who were somewhat more critical of the colours used on the home page looked at the My VAC Account as an example of what they would like to see elsewhere on the site.

Search tasks

Five different search tasks were created to understand how users would use the website to find the information identified in each task. Each participant was asked to complete two or three tasks at most. The moderator read out the task and then observed how the participant navigated the website. They would only interfere if they felt the participant was far off course or very frustrated, which did not happen very often.

Task 1

Your father was a WWII Veteran and your mother, his widow, now needs home care assistance. Can Veterans Affairs Canada help? How would you apply?

Typical outcome: Moderately successful task completion

Participants were moderately successful at finding this information by following a single path from the home page. Nearly all started their search by accessing “Families and caregivers” (rather than “Housing and home life”). From the next page, they would most likely click on “Housing and home life,” followed by “VIP for survivors,” which resulted in a successful task completion. Participants were more inclined to select “VIP for survivors” rather than “VIP for primary caregivers.”

· After clicking on “Families and caregivers,” a few opted for “Death and bereavement,” which tended to lead them down the wrong path.

Task 2

A Veteran you know is struggling to make ends meet. After some unforeseen expenses, her family has fallen behind two months in paying their electricity bill. Can Veterans Affairs Canada help this Veteran?

Typical outcome: Successful task completion

Participants were generally successful at finding this information by following a single path from the home page. All participants started their search by accessing “Financial programs and services” and then selecting “Emergency funds.” The few who followed a less direct path tended to select “Income support” on the “Financial programs and services” page, mostly because it seemed intuitive and it was located above “Emergency funds.”

Task 3

You are a Veteran and you have to travel to another city to obtain cancer treatment from a specialist. Can VAC help cover the costs of getting and staying in that city?

Typical outcome: Successful task completion

Participants were generally successful at finding this information by following one of two particular paths from the home page.

· Some participants started from “Financial programs and services,” and successfully completed the task by then selecting “Medical costs” and then “Health-related travel expenses.”

· Some participants started from “Health programs and services,” and successfully completed the task by then selecting “Physical health and wellness,” and then “Medical costs” and finally “Health-related travel expenses.” However, it was noted that the first place one would start the search would be under “Health programs” – and if you start there, you have to do more clicks to get to the answer than if you start with “Financial programs.”

Task 4

You are the spouse of (OR: You are) a CAF member who is leaving the Forces and talks (OR: you are thinking) about going back to school. Can VAC offer financial help?

Typical outcome: Moderately successful task completion

All participants started their search by clicking on “Education and jobs” from the home page. Those who easily completed the task would then select “Go back to school.” Those who were less successful or not successful at all would have selected “Prepare to release” instead. Those having taken the wrong path initially attempted to complete the task by selecting “Financial programs and services” from the home page, which would also lead them down the wrong path.

Task 5

You are the niece or nephew of a Veteran for whom you have been providing daily care and support for a few years. Someone told you that you can be compensated for providing this care. How would you go about applying?

(OR: You are a Veteran who has been receiving daily care and support from a niece or nephew for a few years. Someone told you that your niece/nephew can be compensated for the care they have provided to you. How would you go about applying?)

Typical outcome: Moderately successful task completion

Participants would start their search by following one of the following entry points from the home page:

· “Families and caregivers,” which would then be followed by either “Health programs and services” or “Financial programs and services”. Irrespective of the last theme selected, participants would then find the link to the “Caregiver Recognition Benefit.”

· A few would access “Financial programs and services” from the home page, which did not always lead to a successful outcome.

· A few participants also ended up in the Caregiver Zone, which again, did not lead to a successful outcome for this task.

Appendix C: Qualitative research instruments

December 18, 2018

Environics Research Group

Focus Groups on Communications

Veterans Affairs Canada

PN10177

Recruitment Screener

Respondent Name:

Home #:

Business #:

Group #:

Recruiter:

GROUP 1

Online group (English)

Monday, December 10

7:30 pm EST / 5:30pm MST

GROUP 2

Online group (French)

Tuesday, Dec. 11

5:30 pm EST

GROUP 3

Toronto

Monday, February 11

5:30 pm

GROUP 4

Toronto

Monday, February 11

7:30 pm

GROUP 5

St. John’s, NL

Tuesday, February 12

5:30 pm

GROUP 6

St. John’s, NL

Tuesday, February 12

7:30 pm

GROUP 7

Ottawa/Gatineau (English)

Wednesday, Feb. 20

5:30 pm

GROUP 8

Ottawa/Gatineau (French)

Wednesday, Feb. 20

7:30 pm

GROUP 9

Montreal (French)

Monday, February 18

5:30 pm

GROUP 10

Montreal (French)

Monday, February 18

7:30 pm

GROUP 11

Victoria

Tuesday, February 19

5:30 pm

GROUP 12

Victoria

Tuesday, February 19

5:30 pm

Ten recruits per in-person session/seven recruits per online session. All must be Veterans or immediate family of Veterans (3-4 maximum per group). $100 incentives.

Hello/Bonjour, my name is _________ from Trend Research, calling on behalf of Environics Research. Would you like to continue this discussion in English? / Voulez-vous continuer cette conversation en français?

We are calling today to invite participants to attend a (focus group discussion/ a telephone conference call focus group discussion) we are conducting on behalf of Veterans Affairs Canada exploring issues of interest to members of the Veteran Community. This study is a research project, not an attempt to sell or market anything. Your participation in the research is completely voluntary, confidential and your decision to participate or not will not affect any dealings you may have with Veterans Affairs Canada or any other part of the government.

IN PERSON: The session will last a maximum of 1:45 hours and you will receive a cash gift as a thank you for attending the session.

ONLINE: The format will be a conference call discussion led by a research professional that will involve you and some other members of the Veteran Community. There will also be an online component of the session where you will be given a log-in to view some materials. The session will last a maximum of 1.5 hours and you will receive a cash gift of $100 as a thanks for attending the session.

May we have your permission to ask you or someone else in your household some further question to see if you/they fit in our study? This will take about 5 minutes.

NB: An audio-recording of the session will be produced for research purposes. The recording will be used only by the research professional to assist in preparing a report on the research findings and will be destroyed once the report is completed. All information collected, used and/or disclosed will be used for research purposes only and administered as per the requirements of the Privacy Act.

NB: If a participant asks for information on the research company conducting the research they can be told: Environics Research is located at 33 Bloor Street East, Suite 900, Toronto Ontario and can be reached at 416-920-9010.

For the purposes of this study a Veteran is anyone who has served in the Canadian Forces (CAF) or the RCMP, completed basic training and who has been honourably discharged.

1. Are you a Veteran of the Canadian Forces or the RCMP?

01 – Yes, Canadian Forces Veteran SKIP TO Q. 3

02 – Yes, RCMP veteran SKIP TO Q. 3

03 – No ASK Q. 2

2. Are you an immediate family member (i.e., a spouse or adult child) of a Veteran of the Canadian Forces or the RCMP?

01 – Yes, spouse of a Veteran (SPECIFY IF CF OR RCMP)

02 – Yes, child of a Veteran (SPECIFY IF CF OR RCMP)

03 – No THANK AND TERMINATE

NB: MAXIMUM 4 OUT OF 10 IN PERSON / 3 OUT OF 7 ONLINE CAN BE FAMILY OF VETERANS

3. Are you or is any member of your household or your immediate family employed in:

Type	No	Yes
A market research, communications or public relations firm, or an advertising agency	1	2
Media (Radio, Television, Newspapers, Magazines, etc.)	1	2
Veterans Affairs Canada	1	2
Department of National Defence	1	2
A political party	1	2

IF YES TO ANY OF THE ABOVE – THANK AND TERMINATE

ASK ALL

4. How long ago were [you/the veteran in your family] discharged from the CF/RCMP?

_____YEARS AGO

FOR OTTAWA/GATINEAU/ONLINE GROUPS ASK:

5. Which language do you speak most often at home?

English 1 GROUP 5 (1 ONLINE)

French 2 GROUP 6 (2 ONLINE)

6. INDICATE:

Male 1

Female 2 Try to get at least a couple of women in each group

7. We have been asked to speak to participants from all different ages. So that we may do this accurately, may I have your exact age please? (GET MIX)

_________. WRITE IN

Under 18 0 TERMINATE

18-24 years of age 1

25-34 years of age 2

35-44 years of age 3

45-54 years of age 4

55-64 years of age 5

65-74 years of age 6

75 years or more 7 TERMINATE

8. Could you please tell me what is the last level of education that you completed? (GET MIX)

Some High School only 1

Completed High School 2

Trade School certificate 3

Some Post-secondary 4

Completed Post-secondary 5

Graduate degree 6

9. Participants in group discussions are asked to voice their opinions and thoughts, how comfortable are you in voicing your opinions in front of others? Are you... (read list)

Very comfortable 1- MIN 5 PER GROUP

Fairly comfortable 2

Not very comfortable 3 - TERMINATE

Very uncomfortable 4 - TERMINATE

10. Have you ever attended a focus group or a one-to-one discussion for which you have received a sum of money, here or elsewhere?

Yes 1 MAXIMUM 4 PER GROUP

No 2 -> (SKIP TO Q.13)

IF Q10 YES ASK:

11. When did you last attend one of these discussions?

(TERMINATE IF IN THE PAST 6 MONTHS)

12. How many focus groups or one-to-one discussions have you attended in the past 5 years?

(SPECIFY)

IF 5 OR MORE, TERMINATE

ASK ALL

13. Sometimes participants in the focus group are also asked to write out their responses on paper. Is there any reason why you could not participate? If you need glasses to read or a hearing aid, please remember to bring them.

Yes 1 TERMINATE

No 2

NOTE: TERMINATE IF RESPONDENT OFFERS ANY REASON SUCH AS SIGHT OR HEARING PROBLEM, A WRITTEN OR VERBAL LANGUAGE PROBLEM, A CONCERN WITH NOT BEING ABLE TO COMMUNICATE EFFECTIVELY.

14. IN PERSON: I would like to invite you to attend the focus group session where you will exchange your opinions in a moderated discussion with other Canadians from your community. The session will be videotaped and observed but your participation will be confidential. Do you consent to take part in the focus group? By agreeing to participate you are giving your consent to these procedures. You will be asked to sign a release to this effect when you arrive at the session.

Yes 1

No 2 TERMINATE

ONLINE: In addition to dialling into a telephone conference call, this focus group will require participants to go online to an online meeting using a desktop or laptop computer. You will need internet access in a private and quiet location to take part in the study. We cannot provide this technology for you. Will you be able to access the Internet for a 1.5-hour discussion using a desktop or laptop computer?

Yes 1

No 2 TERMINATE

NOTE: PARTICIPANTS CANNOT USE A TABLET OR MOBILE PHONE FOR THIS TASK

Are you able to use high-speed internet connection to take part in the session?

Yes 1

No 2 TERMINATE

I would like to invite you to attend the focus group session where you will exchange your opinions in a moderated discussion with other people from the Veteran Community. The session will be audio recorded and some other members of the research team may also listen in on the session, but your participation will be confidential. Do you consent to take part in the focus group? By agreeing to participate you are giving your consent to these procedures

Yes 1

No 2 TERMINATE

15. We will contact you again before the date of the session to confirm your attendance. Do you consent to this?

Yes 1

No 2 TERMINATE

16. IN PERSON: Please bring along some form of identification as you will be asked to show it. Only you may attend, you cannot send a substitute. The session is 2 hours in length, but we are asking that all participants arrive 15 minutes prior to the start time of the session. Are you able to be at the research facility 15 minutes prior to the session time?

Yes 1

No 2 TERMINATE

ONLINE: The session is about an hour and a half (i.e., 90-minutes), but we are asking that all participants call into the toll-free number and log into the Zoom online meeting 5 minutes prior to the start of the session. Are you able to call in and log-in about 5 minutes prior to the start time?

Yes 1

No 2 TERMINATE

IN PERSON:

I would like to invite you to a group discussion on (date).The session will last 2 hours in total and you will receive $100 to thank you for your participation.

Toronto – Monday, February 11^th (5:30-7:15 and 7:30-9:15pm)

St. John’s – Tuesday, February 12^th (5:30-7:15 and 7:30-9:15pm)

Montreal – Monday, February 18^th (5:30-7:15 and 7:30-9:15pm)

Victoria – Tuesday, February 19^th (5:30-7:15 and 7:30-9:15pm)

Ottawa/Gatineau – Wednesday, February 20^th (5:30-7:15 and 7:30-9:15pm)

ONLINE:

Can you please provide me with your email address, so I can send you login details for the Zoom web conference application, as well as dial-in details for the conference line.

E-mail address:

PLEASE ENSURE PARTICIPANTS ARE TOLD THE TIME OF SESSION

Please call 1-855-703-8985 at that time. You will then be prompted for a “meeting ID” [which we will send you later]. You will then be placed in conference with the moderator and the other participants.

INTERVIEWERS: Tell respondent that it is a small group and anyone who does not show or cancels at the last minute will compromise the project. Make sure they know we feel their opinions are valuable and we are serious about finding out what they have to offer.

NOTE: PLEASE TELL ALL RESPONDENTS THAT THEY WILL RECEIVE A CONFIRMATION CALL THE DAY PRIOR TO THE SESSION. IF FOR SOME REASON THEY HAVE NOT HEARD FROM US THEY SHOULD CONTACT US AT __________. IF THEIR NAME IS NOT ON THE ATTENDANCE FORM THEY WILL NOT BE ADMITTED TO THE GROUP. IF A RESPONDENT HAS ANY OTHER QUESTIONS ABOUT THE RESEARCH, THEY SHOULD ALSO CONTACT US AT THIS NUMBER.

Environics Research

Focus Groups on VAC Communications Materials

Veterans Affairs Canada – Discussion Agenda

PN10177

1. Introduction to Procedures

Welcome to this focus group. My name is [INSERT NAME] and I work for Environics Research, which is an independent market research company. We are conducting this research project on behalf of Veterans Affairs Canada.

We want to hear your opinions. Feel free to agree or disagree. Even if you are just one person among ten that takes a certain point of view, you could represent a lot of people who feel the same way as you do. You don’t have to direct all your comments to me; you can exchange ideas and arguments with each other too.

There are some observers from the research team and from Veterans Affairs Canada who are watching the session on the other side of the two-way mirror. We are also video-taping this session to help me write my report. The video will only be used internally to analyse the research and will not be released to anyone else. I may also take some notes during the group to remind myself of things.

Your identity and anything you say here will remain confidential and anonymous. Nothing you say here will be linked to you by name in any reporting we do on this project.

The session should last about an hour and a half and the host/hostess will pay you your incentives at the end of the session. Please turn off your cell phones.

Let’s go around the table so you can each introduce yourselves to the group. Tell us your name and a bit about yourself such as where you are calling from and who lives with you in your house. Also, since we invited you because you are all part of the Veteran Community – meaning that you are either a Veteran yourself or you are the family member of a Veteran - could you also tell us about your background as a Veteran (or that of the Veteran in your family) – i.e., when you served and in what capacity.

2. Media/social media consumption

Our main focus in this discussion is going to be to evaluate some communications materials. Before we start looking at any materials, I’d like to go around the table again to hear about your media habits or sources of information. Could you each tell us about the media you look at (TV, radio, internet, newspaper, etc.), the devices you use (Mobile or not), the social media and the websites you visit, apps you use, etc.

PROBE IF NOT MENTIONED: Facebook? Twitter? Instagram?

What about when you specifically want to find out about things that relate to your experience as Veterans or for the sake of your family member who is a Veteran? Where do you look? What are your main sources of information?

PROBE: VAC website? Other sites that focus on Veterans’ issues?

Do any of your follow VAC on social media? (e.g., Facebook or Twitter) IF YES, what do you think of VAC’s social media presence? Do you post useful information or links?

What do you think of the VAC website? Do you find it user-friendly? Informative? How so?

3. Top-of-mind issues for Veterans

I’d like to start by going around the room discussing what you each think are the biggest challenges or problems that Veterans in Canada have to deal with. Could you each jot down on paper one or two things that come to mind? Let’s hear what each of you come up with.

PROBE IF NOT MENTIONED: Mental health issues, PTSB? Pensions? Transition to civilian life? Family issues?

Why do you say that?

To what extent does VAC offer any services to address these issues?

How many of you have dealt with VAC at all since [you/your family member who is a veteran] were discharged?

How did you get in touch with them after you were discharged? What was the process by which you became a client of VAC?

Do you feel that VAC offers a lot of services to Veterans that many Veterans do not know about?

We know that only about a third of Canada’s Veterans are currently clients of VAC (i.e., they take advantage of any of VAC’s services and programs). How could VAC make more Veterans aware of what they can offer?

4. Social media posts (discussed in in-person sessions only)

I’m going to show you all a series of six social media postings from Veterans Affairs Canada. There are all things someone could see if they follow VAC on Facebook or other social media platforms. I’d like you to flip through them and feel free to underline or circle anything that stands out and if you want to write down any comments about what you think of each one you can do so. You will note that each page has a large image on the left and some text on the right so make sure you read the text as well as looking at the image. I will also had out a separate piece of paper where you can jot down your thoughts about each posting.

Please note that we are not here to evaluate the actual programs that are being referenced. We just want to see what you think of the materials themselves in terms of whether they spark your curiosity, whether you would click on them, if they are useful, clear, easy to understand etc. HAND OUT SIX PAGES OF SOCIAL MEDIA POSTS AS WELL AS WRITTEN EXERCISE SHEET

My VAC Account FB post: Have any of you ever heard of “My VAC Account”? IF YES: What exactly is it?

“My VAC Account” is something like the “My Account” portal that some of you may use with the CRA when you do your taxes. You register online and then you can use it to correspond with VAC, upload documents or receipts, apply for various benefits and see what benefits you’re are entitled to etc.”

What did you think of the My VAC Account post? PROBE: Is it clear? Easy to understand? Useful?

Would you click on (or recommend it to another Veteran)? Why? Why not?

What do you think of the design or look of it?

What about the text? Was there anything that stood out?

LifeSpeak: Had any of you ever heard of LifeSpeak?

What did you think of the LifeSpeak page? PROBE: Is it clear? Easy to understand? Useful?

Would you click on it (or recommend it to another Veteran)? Why? Why not?

What do you think of the design or look of it?

What about the text? Was there anything that stood out?

OSISS (Family) post: Had any of you ever heard of OSISS (Operational Stress Injury Social Support)?

What did you think of this FB about OSISS that says “Family”? PROBE: Is it clear? Easy to understand? Useful?

Would you click on it (or recommend it to another Veteran)? Why? Why not?

What do you think of the design or look of it?

What about the text? Was there anything that stood out?

Veteran Family Program: Had any of you ever heard of the Veteran Family Program?

What did you think of the Veteran Family Program post? PROBE: Is it clear? Easy to understand? Useful?

Would you click on it (or recommend it to another Veteran)? Why? Why not?

What do you think of the design or look of it?

What about the text? Was there anything that stood out?

Balance: What did you think of this page that says “Balance”? PROBE: Is it clear? Easy to understand? Useful?

Would you click on it (or recommend it to another Veteran)? Why? Why not?

What do you think of the design or look of it?

What about the text? Was there anything that stood out?

Centre of Excellence on PTSD: Had any of you ever heard of the Centre of Excellence on PTSD?

What did you think of the Centre of Excellence on PTSD post? PROBE: Is it clear? Easy to understand? Useful?

Would you click on it (or recommend it to another Veteran)? Why? Why not?

What do you think of the design or look of it?

What about the text? Was there anything that stood out?

All in all, what do you think of all these pages? Which ones would you be most likely to click on? Which would you recommend to another Veteran or family member?

5. Infographics (discussed in in-person sessions only)

Please turn over the written exercise I gave you and on the flip side there is space for you to jot down what you think of a few more things I want to show you. Here is an “infographic” from VAC about the expansion of the Veteran Family Program. Please jot down your comments about it on the exercise sheet.

HAND OUT INFOGRAPHIC

What did you think of this Infographic?

PROBE: Is it clear? Easy to understand? Useful?

Would you click on it (or recommend it to another Veteran)? Why? Why not?

What do you think of the design or look of it?

Was there anything in the text that stood out for you?

Here is another infographic about the Service Dog Pilot Study results. Please jot down your comments about it on the exercise sheet.

HAND OUT INFOGRAPHIC.

What did you think of this Infographic?

PROBE: Is it clear? Easy to understand? Useful?

Would you click on it (or recommend it to another Veteran)? Why? Why not?

What do you think of the design or look of it?

Was there anything in the text that stood out for you?

6. Fact sheets (discussed in in-person sessions only)

NB: There are three fact sheets and we will rotate which one we distribute:

Caregiver Recognition Benefits

Career Transition Services

Veterans Education and Training Benefit

The last thing I want to give you all to look at is an example of a “fact sheet” that you could download and print off from the VAC website. I’d like you to take your time and read through it and underline anything that stands out. You may or may not be personally interested in the program described here, but you can still have an opinion of what you think of the fact sheet. Please jot down your comments about it on the exercise sheet.

HAND OUT FACT SHEET

What did you think of this Fact Sheet?

PROBE: Is it clear? Easy to understand? Useful? Interesting?

Was there anything in the text that stood out for you?

Would you recommend it to another Veteran? Why? Why not?

What do you think of the design or look of it? Are the images appealing?

7. Mental health ad concepts (discussed in online groups only)

We are going to be looking at some ad concepts in this session that Veterans Affairs Canada is considering running. There will be four of them in total and in each case, I will share the concept with you on your screen. These are not finished ads. They are what we call “storyboards”. They will look like a series of images with text. If we decide to move forward with one or two of these ad concepts, they would be professionally produced with actors etc. So, when you look at them you will have to use your imagination.

CONCEPT A – “Parting Clouds”

CONCEPT B – “You’re not Alone”

CONCEPT C – “Help”

CONCEPT D – “Part of the Family”

MODERATOR WILL SHOW EACH CONCEPT IN THE ZOOM CONFERENCE ONE AFTER THE OTHER – THEN COME BACK TO SHOWING EACH INDIVIDUALLY FOLLOWED BY EVALUATION AND DISCUSSION

CONCEPT A – “Parting Clouds”

Let’s start with the first ad concept that is Concept A. I am going to play it twice and then I will ask some questions in the chat for you to respond to:

MODERATOR WILL TYPE IN: “What is the main message you get from Concept A?”

Let’s discuss what you each thought was the main message in this concept.

Overall, what do you each think of this concept? What are your first impressions?

What are the strengths of this concept?

What are the weaknesses? Is there anything in the concept that you found inappropriate or confusing?

Could you see yourself sharing this ad on social media or forwarding it to anyone else in the Veteran Community?

To what extent would this ad encourage you to seek more information about Veterans Affairs Canada services around mental health?

CONCEPT B – “You’re not Alone”

Let’s continue with the second ad concept that is Concept B. I am going to play it twice and then I will ask some questions in the chat for you to respond to:

MODERATOR WILL TYPE IN: “What is the main message you get from Concept B?”

Let’s discuss what you each thought was the main message in this concept.

Overall, what do you each think of this concept? What are your first impressions?

What are the strengths of this concept?

What are the weaknesses? Is there anything in the concept that you found inappropriate or confusing?

Could you see yourself sharing this ad on social media or forwarding it to anyone else in the Veteran Community?

To what extent would this ad encourage you to seek more information about Veterans Affairs Canada services around mental health?

CONCEPT C – “Help”

Let’s continue with the third ad concept that is Concept C. I am going to play it twice and then I will ask some questions in the chat for you to respond to:

MODERATOR WILL TYPE IN: “What is the main message you get from Concept C?”

Let’s discuss what you each thought was the main message in this concept.

Overall, what do you each think of this concept? What are your first impressions?

What are the strengths of this concept?

What are the weaknesses? Is there anything in the concept that you found inappropriate or confusing?

Could you see yourself sharing this ad on social media or forwarding it to anyone else in the Veteran Community?

To what extent would this ad encourage you to seek more information about Veterans Affairs Canada services around mental health?

CONCEPT D – “Part of the Family”

Let’s continue with the fourth ad concept that is Concept D. I am going to play it twice and then I will ask some questions in the chat for you to respond to:

MODERATOR WILL TYPE IN: “What is the main message you get from Concept D?”

Let’s discuss what you each thought was the main message in this concept.

Overall, what do you each think of this concept? What are your first impressions?

What are the strengths of this concept?

What are the weaknesses? Is there anything in the concept that you found inappropriate or confusing?

Could you see yourself sharing this ad on social media or forwarding it to anyone else in the Veteran Community?

To what extent would this ad encourage you to seek more information about Veterans Affairs Canada services around mental health?

Now I want you to think about the four ad concepts we have been talking about and I will show them all to you one more time. Then I want to have you each pick which one you personally find most effective.

MODERATOR TO PLAY ALL FOUR AD CONCEPTS

MODERATOR TO TYPE IN CHAT ROOM: “Which one of these ad concepts does the best job of making you want to find out more about Veterans Affairs Canada’s mental health services?”

Could you each tell us which concept you picked and why?

Are there elements you would take from another concept that you would add to the one you liked best?

8. Wrap-up

We have covered a lot of topics today and really appreciate you taking the time and energy to come down here and give your opinion. Your input is very important and insightful. Before we leave today, I wanted to ask you whether you have any last thoughts that you want to give Veterans Affairs Canada about today’s topics.

THANK YOU FOR PARTICIPATING!

Location	Dates	Time and composition
Online	December 10 and 11, 2018	December 10 – English; December 11 – French
Toronto	February 11, 2019	5:30 pm – Group #1 English; 7:30 pm – Group #2 English
St. John’s	February 12	5:30 pm – Group #1 English; 7:30 pm – Group #2 English
Montreal	February 18	5:30 pm – Group #1 French; 7:30 pm – Group #2 French
Victoria	February 19	5:30 pm – Group #1 English; 7:30 pm – Group #2 English
Ottawa/Gatineau	February 20	5:30 pm – Group #1 English; 7:30 pm – Group #2 French

Disposition	Wave 1\| (N)	Wave 2\| (N)
Total invitations (c)	63,776	54,134
Total completes (d)	518	522
Qualified break-offs (e)	1,113	1,306
Disqualified (f)	10,195	10,798
Not responded (g)	51,501	40,935
Quota filled (h)	449	573
Contact rate = (d+e+f+h)/c	19%	24%
Participation rate = (d+f+h)/c	18%	22%

Location	Dates	Time and composition
Online	December 10 and 11, 2018	December 10 – English; December 11 – French
Toronto	February 11, 2019	5:30 pm – Group #1 English; 7:30 pm – Group #2 English
St. John’s	February 12	5:30 pm – Group #1 English; 7:30 pm – Group #2 English
Montreal	February 18	5:30 pm – Group #1 French; 7:30 pm – Group #2 French
Victoria	February 19	5:30 pm – Group #1 English; 7:30 pm – Group #2 English
Ottawa/Gatineau	February 20	5:30 pm – Group #1 English; 7:30 pm – Group #2 French

Region	Actual share of population (Census 2016)	Unweighted Sample Wave 1	Unweighted Sample Wave 2
Atlantic Canada	7%	40	50
Quebec	23%	98	98
Ontario	38%	185	181
Prairies/NWT/Nunavut	19%	35	38
B.C. /Yukon	13%	83	80
CANADA	100%	518	522

Sample type	Wave 1*	Wave 2*	Canada (2016 Census)
Gender (18+)
Male	36%	38%	49%
Female	64%	64%	51%
Age
18-34	17%	17%	29%
35-49	21%	21%	24%
50-64	38%	38%	26%
65+	24%	24%	21%
Education level^α
High school diploma or less	17%	17%	35%
Trades/college/post-sec no degree	38%	38%	36%
University degree	46%	46%	29%

Phase 1: Quantitative research phase

Phase 2: Qualitative research phase

1) Focus group testing

2) In-depth interviews

A. Quantitative research

Web page testing

B. Qualitative research

Familiarity with VAC services and benefits

Awareness of VAC services to people other than CAF Veterans

Reasons for visiting veterans.gc.ca

Frequency of visiting veterans.gc.ca

Agreement with statements about veterans.gc.ca

Opinion of VAC site compared to other government service sites

Why veterans.gc.ca is better than other sites

Why veterans.gc.ca is neither better nor worse than other sites

Why veterans.gc.ca is worse than other sites

Things liked about home page

Things disliked about home page

Agreement with statements about home page

Ease of finding information on the home page

Things liked about disability page

Things disliked about disability page

Agreement with statements about disability page

Ease of finding information on the disability page

Things liked about rehabilitation page

Things disliked about rehabilitation page

Agreement with statements about rehabilitation page

Ease of finding information on the rehabilitation page

Things liked about VIP page

Things disliked about VIP page

Agreement with statements about VIP page

Ease of finding information on the VIP page

Things liked about health-related travel information page

Things disliked about health-related travel information page

Agreement with statements about health-related travel information page

Ease of finding information on the health-related travel information page

Type of respondent

Demographics

Socioeconomic factors

My VAC Account FB Post

LifeSpeak Post

OSISS (Family) Post

Veteran Family Program

Balance

Centre of Excellence on PTSD

General observations regarding VAC social media posts

Infographic: The expansion of the Veteran Family Program

Infographic: Service Dog Pilot Study

F. Assessment of mental health concepts

Concept A – “Parting clouds”

Concept B – “You’re not alone”

Concept C – “Help”

Concept D – “Part of the Family”

Overall reactions to mental health ad concepts

Search tasks

Appendix A: Quantitative methodology

Appendix B: Qualitative methodology (focus groups and in-depth interviews)

Group composition

Moderation

Appendix B: Qualitative methodology
(focus groups and in-depth interviews)