POR-085-11
Contract award date: 03-26-2012

Pre-testing of Creative Elements for a Government of Canada Communication Campaign – Executive Summary

April 2012

Submitted to: Human Resources and Skills Development Canada
Prepared by: Ipsos Reid Corporation

Contract #:G9178-110001/007/CY
Fieldwork completed: April 10^th, 2012
Contracted cost of research, including HST: $41,894.75

Contact: nc-por-rop-gd@hrsdc-rhdcc.gc.ca

«Ce rapport est aussi disponible en français sur demande»

Table of Contents

• Executive Summary
  • Background and Objectives
  • Methodology
  • Key Findings
• Appendix I – Screener
• Appendix II – Moderator’s Guide
• Appendix III - Creatives

Political Neutrality Statement

I hereby certify as Senior Officer of Ipsos Reid that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not include information on electoral voting intentions, political party preferences, standings with the electorate or ratings of the performance of a political party or its leaders.

Marc Beaudoin
Vice President
Ipsos Reid Public Affairs

Executive Summary

Background and Objectives

Human Resources and Skills Development Canada is planning to conduct a communications campaign on the possible changes to the Old Age Security (OAS) program as mentioned in Budget 2012. The proposed communications’ campaign will include several pieces: direct mail (insert and letter), an OAS brochure as well as a national weekly newspaper print advertising buy. Therefore, the department has identified the need for pre-testing of potential print creative elements.

Pre-testing the print creative elements of the communications campaign is critical to the success of the overall campaign. The research will be used to help determine the most effective communications materials for the target audience. HRSDC has identified the need to test creative elements among those 40 years of age or more, broken into two specific audience segments (those defined as high income and those defined as low income, as per recruitment screener specs in Appendix 1).

Accordingly, qualitative research was intended to obtain feedback from Canadians on a series of print weekly advertising concepts intended to:

• Inform current OAS/GIS recipients and Canadians who are aged 54+ as of March 31, 2012 that the proposed changes to OAS will not affect them;
• Inform those 40 years of age or more of the proposed changes to OAS announced in Budget 2012; and,
• Generate traffic to http://www.servicecanada.gc.ca/retirement and the related program-specific web pages and 1 800 O-Canada.

Methodology

Ipsos Reid conducted a total of six focus groups from April 4^th to April 10^th, 2012; two groups were each held in Kelowna, Mississauga and Quebec City. In each city, we conducted one group among participants with high household income levels and one with those with lower household income levels. All focus groups participants were 40 years of age or more.

Participants received an $85 honourarium as a token of appreciation for their time. Research instruments were designed to gather feedback and the participants’ reactions to the four print concepts presented, as well as to identify potential areas of improvement for each.

Note to Reader

Qualitative data from focus group discussions are reviewed by the researchers, who draw out continuities and synthesize the main themes, critical "hot buttons", "typical" responses and general levels of agreement/disagreement voiced during the sessions. The analysis of qualitative findings yields tendencies and indications of how the target audience perceives the study issues. However, results are not representative of the general population. Qualitative research involves a limited number of people who are specially selected according to pre-determined attributes. Only quantitative surveys that have a robust number of respondents who have been selected randomly from the target population can be generalized as to represent the broader population.

Key Findings

• Participants’ recollection of specific initiatives included in Budget 2012 was limited. When prompted for specifics, those who did have some recollection most often mentioned changes to Old Age Security specifically, as well as changes to the retirement age (not part of Budget 2012 initiatives).
• Quebec City participants did express some confusion and were in some cases generally unclear as to the difference between OAS and RRQ programs.
• All felt that changes to OAS was an important issue and that the Government of Canada should indeed be making efforts to ensure that Canadians were aware of proposed changes to the program. In addition, when prompted on their preference for a full page versus a half page advertisement, there was a clear preference among participants in all sessions for the full page option, with most saying that this issue was of sufficient importance to warrant a full page advertisement.
• There was consistency in feedback collected from participants in all locations as to the message being conveyed in each of the proposed concepts. Virtually all participants clearly understood that these ads were intended to:
  • Inform people that changes to OAS were coming, and;
  • Communicate that those 54 years of age or more would not be affected by these changes.
• There was a clear preference among participants in all sessions for concept options which included both text and graphed presentation of information. A number of participants commented that the addition of the graphed information effectively communicated to the reader why changes to the OAS program were necessary.
• Participants appreciated the table titled ‘what does this mean for you’ appearing in both the full and half page versions of the Making Old Age Security Sustainable concept. This tabled approach was considered unambiguous and helpful in quickly determining who is and is not affected by the proposed changes to OAS.
• Sponsorship of the advertisement was clear to all; likewise, participants felt that there would be no confusion as to how to obtain additional information – simply visit the web site Service Canada - Retirement Planning or call the 1-800 line.
• Reactions to the proposed titles were mixed. Those who preferred ‘Changes to Old Age Security’ most often said that they felt it was eye-catching, and left little doubt as to what the advertisement would be about.  Conversely, those who preferred the alternative ‘Making Old Age Security Sustainable’ felt that it was more positive in tone and that it was more effective in conveying the notion that changes were needed so that the program could be there for future generations.
• The use of a street sign as a visual was a point of contention among some participants in the Quebec City group, who struggled to understand its meaning; Some felt that it was an unnecessary distraction.

Appendix I – Screeners (English)

English

FOCUS GROUPS - HRSDC PRE-TESTING OF A GOVERNMENT OF CANADA COMMUNICATIONS CAMPAIGN– April 2012

Hello, my name is ______________. I'm calling from Ipsos Reid, a national public opinion research firm. On behalf of the Government of Canada we’re organizing a series of discussion groups to explore various issues of importance to the country.

EXPLAIN FOCUS GROUPS. About ten people like you will be taking part, all of them randomly recruited just like you. For their time, participants will receive an honorarium of $85.  But before we invite you to attend, we need to ask you a few questions to ensure that we get a good mix and variety of people. May I ask you a few questions?

• Yes CONTINUE
• No THANK AND TERMINATE

Your participation is voluntary. We are interested in hearing your opinions, no attempt will be made to sell you anything or change your point of view.  The format is a “round table” discussion led by a research professional. All opinions expressed will remain anonymous and views will be grouped together to ensure no particular individual can be identified.

1. Do you or any member of your household work in or has retired from:

Market Research or Marketing

• YES
• NO

Public Relations or Media (TV, Print)

• YES
• NO

Advertising and communications

• YES
• NO

An employee of a political party

• YES
• NO

An employee of a government department or agency, whether federal or provincial

• YES
• NO

IF “YES” TO ANY OF THE ABOVE, THANK AND TERMINATE

2. Are you a Canadian citizen at least 40 years old who normally resides in the [XX] area?

• Yes [CONTINUE to Q2b]
• No [Go to Q3]

2b. How long have you lived in [INSERT LOCATION]?_______

3. Have you worked in Canada for at least ten years?

• Yes [CONTINUE]
• No [THANK AND TERMINATE]

4. Are you the head or co-head of your household?

• Yes [CONTINUE]
• No [THANK AND TERMINATE]

5. Have you ever attended a consumer group discussion, an interview or survey which was arranged in advance and for which you received a sum of money?

• Yes [MAX - 3 PER GROUP]
• No [GO TO Q 8]

6. How long ago was it? ___________________[TERMINATE IF IN THE PAST 6 MONTHS]

7. How many consumer discussion groups have you attended in the past 5 years? __________________[TERMINATE IF MORE THAN 4 GROUPS]

8. Could you please tell me what age category you fall in to? Are you... [ENSURE GOOD MIX PER GROUP]

• Under 40 [THANK AND TERMINATE]
• 40- 54
• 55-64
• 65-74
• 75+ [MAX 1 PER GROUP]
• Refuse [THANK AND TERMINATE]

9. Could you please tell me what is the highest level of education that you have completed? [ENSURE GOOD MIX PER GROUP]

• Some high school only
• Completed high school
• Some College/University
• Completed College/University
• RF/DK

10. What is your current employment status?

• Working full-time
• Working part-time
• Self-employed
• Retired
• Currently not working [SKIP TO Q12 – MAX 2 PER GROUP]
• Student [THANK AND TERMINATE]
• Other
• DK/RF

11. [IF EMPLOYED/RETIRED] What is/was your current/past occupation? __________________________ (PLEASE SPECIFY)

12. How many people above the age of 18 are there in your household?

• One CONTINUE
• More than One SKIP TO Q 14

13. Was your household’s income for 2011 greater or less than 75 thousand dollars?

• 1. $75K OR GREATER CONTINUE FOR POSSIBLE RECRUIT TO GROUP 2
• 2. LESS THAN $75K CONTINUE FOR POSSIBLE RECRUIT TO GROUP 1
• Refused THANK AND TERMINATE
• IF Q12=1, SKIP TO Q15

14. Was your household’s income for 2011 greater or less than 100 thousand dollars?

• 1. $100K OR GREATER CONTINUE FOR POSSIBLE RECRUIT TO GROUP 2
• 2. LESS THAN $100K CONTINUE FOR POSSIBLE RECRUIT TO GROUP 1
• Refused THANK AND TERMINATE
• IF Q14=1, SKIP TO Q16

15. IF Q13 OR Q14 =2 ASK: And would that be? [ENSURE A GOOD MIX]

• Under $20,000
• Between $20,000 and $34,999
• Between $35,000 and $44,999
• Between $45,000 and $59,999
• Between $60,000 and $74,999
• [IF Q14=2] Between $75,000 and $99,999

16. DO NOT ASK – NOTE GENDER [ENSURE 50-50 SPLIT]

• Male
• Female

17. If you could be a superhero, what superpower would you want to have? Why? (MUST BE ABLE TO PARTICIPATE, I.E. PROVIDE A RESPONSE AND EXPLANATION. TERMINATE IF FLIPPANT, COMBATIVE OR EXHIBITS DIFFICULTY IN RESPONDING)

TERMINATE IF RESPONDENT OFFERS ANY REASON SUCH AS SIGHT OR HEARING PROBLEM, A WRITTEN OR VERBAL LANGUAGE PROBLEM, A CONCERN WITH NOT BEING ABLE TO COMMUNICATE EFFECTIVELY OR IF YOU HAVE A CONCERN.

Wonderful, you qualify to participate in one of our discussion sessions. Is there anything in particular that we can do to accommodate your participation in the session? [Prompt with: do you have any special needs? i.e. accessibility issues] The session will be 2 hrs in length. We are reserving this discussion time for you. So if for any reason you cannot attend, please call: XXX-XXXX. We may be reviewing certain print materials so if need be please bring along your glasses.

Appendix II – Moderator’s Guide

English

HRSDC OAS PRINT CONCEPT TESTING FOCUS GROUPS APRIL 3, 2012

INTRODUCTION (5 Minutes)

• Explain to participants:
• Ipsos Group conducting research on behalf of Government of Canada (if asked Human Resources and Skills Development Canada is sponsoring the research)
• the length of session (2 hours)
• taping of the discussion
• one-way mirror and colleagues viewing in back room
• results are confidential and reported in aggregate/individuals are not identified/participation is voluntary
• the role of moderator is to ask questions, timekeeper, objective/no vested interest
• role of participants: not expected to be experts, no need to reach consensus, speak openly and frankly about opinions, no wrong answers
• get participants to introduce themselves and their occupation/hobbies, etc.

WARM UP and CONTEXT (35 Minutes)

• Has anyone read/seen/heard anything about the federal government’s most recent budget which was tabled in the House of Commons last Thursday? What have you heard? What are some of the measures that are being put forth? [MODERATOR LIST ON FLIP CHART AND PROMPT AS NEEDED] Why do you think some of these measures were introduced? What were the goals of the budget (broad question, not each specific measure)?
• Some of you mentioned OAS [PROMPT IF NECESSARY] What did you hear? What do you think of these changes? Why do you think they did this? When will changes come into effect? Who is likely to be most affected by these changes?
• Are changes limited to this (OAS) program only? Are there any other programs that are likely to be affected by these changes? Which ones? [MODERATOR LISTEN FOR REFRENCES TO GUARANTEED INCOME SUPPLEMENT (GIS), SURVIVORS ALLOWANCE, VETERANS AND ABORIGINAL PROGRAMS. NO CHANGES TO CPP.]

Concept Testing (75 Minutes) – OF 3 CONCEPTS TO BE TESTED

[NOTE TO MODERATOR – IN ALL MARKETS ENSURE INDIVIDUAL CONCEPTS ARE PRESENTED IN RANDOMIZED ORDER FROM ONE FOCUS GROUP SESSION TO THE NEXT. LINE OF QUESTIONING BELOW TO BE REPEATED FOR EACH CONCEPT PRESENTED. ]

PARTICIPANTS ARE TO BE REMINDED THAT CREATIVE BEING PRESENTED IS NOT FINISHED PRODUCT RATHER CONCEPTS FOR AN EVENTUAL PRINT AD. WE ARE NOT LOOKING FOR ART CRITICS, WHAT IS KEY IS THEIR FIRST IMPRESSIONS OF THE CONCEPT, THE MESSAGES BEING COMMUNICATED AND IF THEY LEARNED ANYTHING NEW OR IMPORTANT ABOUT CHANGES TO THE OAS PROGRAM AS A RESULT OF SEEING THIS AD.

HAND OUT QUESTIONNAIRE, HAVE PARTICIPANTS COMPLETE PRIOR TO OPENING THE FLOOR TO DISCUSSION.

CONCEPT TESTING

• To start with I would like to show you a concept for an eventual print advertisement you might see in the coming weeks/months in your local newspaper. After having shown you each concept I would like you to complete a short questionnaire based on this concept. I’ll be handing out a short questionnaire for each concept that I’m going to present before we open up the floor in order to discuss each treatment individually.

[PRESENT CONCEPT BOARDS] – MODERATOR ALLOW A FEW MINUTES TO COMPLETE QUESTIONNAIRE FOR EACH CONCEPT PRESENTED].

Ok now that you’ve had an opportunity to complete your questionnaire, let’s go through your answers [FLIP CHART]

1. What is the main message? That is, what is this concept trying to say?

2. Did you learn anything new or important about changes to the OAS program as a result of seeing/reading this ad? What specifically?

Follow-up questions for discussion:

• Would you do anything as a result of seeing it?
• What would you do? What makes you say that?
• Is there anything confusing here? If yes, what is confusing?
• Does the ad clearly indicate how to obtain additional information?
• Is it clear to you who is sponsoring this ad, i.e., who is this ad from? Who produced it?
• If you saw an ad like this in your local newspaper, would you pay attention to it? What makes you say that?
• What’s the first thing that jumped out at you, when you look at this concept? What is your eye drawn to?
• Are there specific words or images that are particularly effective in terms of getting your attention?

OK, NOW I’M GOING TO SHOW YOU ANOTHER POSSIBLE CONCEPT AND WE’LL COMPLETE THIS SAME EXERCISE AGAIN.  [MODERATOR REPEAT FOR ALL THREE PRINT CONCEPTS BEING TESTED]

MODERATOR – [ONCE CONCEPTS PRESENTED]

Conclusion (5 Minutes)

• Do you have any other comments or thoughts about the advertising concepts you saw today/this evening?

Appendix III - Creatives

Changes to Old Age Security - Full Page Concept

Text Alternative for Graphic: Changes to Old Age Security - Full Page Concept

Changes to Old Age Security - Half Page Concept

Text Alternative for Graphic: Changes to Old Age Security - Half Page Concept

Making Old Age Security Sustainable – Full Page Concept

Text Alternative for Graphic: Making Old Age Security Sustainable – Full Page Concept

Making Old Age Security Sustainable – Half Page Concept

Text Alternative for Graphic: Making Old Age Security Sustainable – Half Page Concept