TD Summer Reading Club 2018
National Program Statistics Final Report

Acknowledgements

Environics would like to extend its thanks to the many people who assisted with the TD Summer Reading Club (TDSRC) Statistics and Evaluation project. In particular, we would like to thank Lianne Fortin, Program Manager at Library and Archives Canada, for her guidance and commentary throughout the project. We would also like to extend special thanks to Ashley-Ann Brooks, Project Assistant for the TD Summer Reading Club, who took the lead in contacting libraries and helping to collect information which was vital to the data collection and analysis.

We would also like to thank the provincial and regional library coordinators for their assistance in communicating with their library systems, as well as the library systems that took the time to compile the statistics from their branches and affiliates. Finally, we cannot overlook the contribution of the individual libraries that devoted their time to providing their program statistics and the employees and volunteers who collected the data that made this report possible.

The TDSRC is made possible thanks to the joint initiative between TD Bank Group (TDBG), Library and Archives Canada (LAC) and the Toronto Public Library (TPL).

Executive summary

1. Objectives

The TD Summer Reading Club (TDSRC) is a national bilingual program that engages kids in the joy of reading while championing Canadian writers and illustrators. Kids who register at their local library receive free physical program materials and can access the program materials through the website. Participating libraries are provided with thematic booklists, access to illustrations, programming and activity ideas and free outreach materials for promoting the program in their community.

The study population is the library systems, branches or independent libraries that signed the Letter of Agreement, received free program materials and participated in the TDSRC during the summer of 2018. This year, the TDSRC was offered in eleven provinces and territories across the country through the support of TD Bank Group.

Library and Archives Canada's (LAC) interest in conducting this study is to provide information about the reach and success of the program to the TDSRC Partners, TD Bank Group, Toronto Public Library, and Library and Archives Canada as well as to participating libraries. In addition, feedback gathered will inform improvements to the TDSRC program. LAC's specific research objectives were to:

• Measure the number of programs, activities and attendance attributed to, organized by, or otherwise associated with, the TD Summer Reading Club (i.e., those offered by participating libraries to children in the library or in the community that were part of the TDSRC).
• Determine what the various library systems, library branches or independent libraries considered to be their program reading outcomes or indicator of reading successes resulting from the TDSRC.
• Compare the measures of success, when possible, with at least the previous two years with a view to advising on developing appropriate performance measurement parameters.
• Provide recommendations for how the TDSRC program may be improved.

2. Methodology

Since libraries are organized differently in each province and territory, "service points" were chosen as the common denominator to respond to the program evaluation. A service point may be an individual library or a library which is part of a library system with many branches. An individual library represents one service point while a system with five separate branches would represent five service points. Evaluations were either completed by individual libraries themselves using the online Statistics and Evaluation Form or individual branches provided the necessary information regarding their program to their library system administrator who filled in the form for their entire system.

Individual libraries and library systems were invited to complete their evaluation online via an email invitation with a unique link. This unique link pre-identified individual libraries and library systems and brought them directly to the survey where they entered their data. To accommodate reporting for both individual libraries and systems with multiple service points, two separate survey instruments were programmed. Individual libraries were sent a link to the library survey, enabling them to enter data for their library only, whereas those pre-identified to be reporting for multiple service points (library coordinators) were sent a link that led them to the library systems survey, enabling them to enter data for multiple service points.

In 2018, the project start-up was delayed due to contracting and therefore the online evaluation form was made available to libraries on September 27th. Environics sent out 893 email invitations in both official languages to participating public libraries or systems within eleven participating provinces/territories. In total, 701 evaluations were collected between September 27 and November 27, 2018. The completed evaluations contained data for 1,738 of the 2,052 total service points that participated in the TDSRC in 2018, resulting in a response rate of 85 percent.

3. Statement of limitations

This research used a census approach wherein an attempt was made to contact every participating library/system. Environics sent a minimum of three separate reminder emails and regional coordinators also followed up to encourage completion of the online evaluation form. The data reported herein includes responses from 85 percent of all eligible service points and weighting factors are used to extrapolate to population-wide totals within each region. This weighting process assumes that there are no structural or statistically important differences between libraries who did respond and those that did not.

4. Cost of research

The cost of this research was $59,332.91 (HST included).

5. Key findings

Registration, Activities and Attendance

An estimated 346,604 children registered for the TDSRC program in 2018, representing roughly six percent of all Canadian children under 18. This represents an increase of almost 20,000 registrations over 2017, continuing the pattern of annual growth evident since program statistics began being collected (starting with an estimated total of 216,312 in 2005). This year's increase was driven primarily by registration growth in Reseau BIBLIO in Quebec, the OLS-North libraries in Northern Ontario and the participating British Columbia libraries.

The age distribution among participating children remains similar to previous years, with 6- to 8-year-olds being the most likely age group to participate (39% of registrations).

2018 registrations were comprised of 60 percent past participants and 40 percent new participants. The overall pattern over the past two years suggests that new registrations have been declining, while the number of previous participants has been increasing.

Libraries ran a total of 45,522 programs and activities organized around the theme in 2018 (excluding promotional visits), which is a substantial jump over 2017 (up 25% from 37,044 activities). Since the average number of children attending these activities has remained stable (roughly 18 children per activity), there has been a corresponding increase in overall attendance (up 25% since 2017 to 831,546).

In terms of visits specifically to promote the TDSRC to children in the community, the majority (76%) of all libraries made at least one such visit. Promotional visits to schools were most common (visited by 60% of libraries), but also included childcare centres (25% of libraries) and day camps (19% of libraries). In total, 11,065 visits were made to promote the program at various locations, reaching 659,411 children in the process – roughly 60 children per visit.

Indicators of Success & Satisfaction

Overall satisfaction with TDSRC in 2018 is strong (70% rated it between 8 and 10 out of 10). Satisfaction with the website and web content for library staff (66%) and the website usability (67%) was similarly good. All three measures of satisfaction are lower than in 2017, the latter two returning closer to levels previously observed in 2016 (overall satisfaction was not asked in 2016). These slight declines in satisfaction occurred outside Quebec; as a result, Quebec libraries now report the highest satisfaction with the TDSRC.

Libraries using the English materials provided by the TDSRC are most likely to be using, and to be most satisfied with, the images. Satisfaction with the promotional templates is also relatively high and their use (perhaps accordingly) has steadily increased.

Similarly, libraries using the French materials are most widely using, and most satisfied with, the illustrations. However, there also appears to be declining use of many of the French materials dating back to 2016.

Suggestions for future improvements to the website for library staff include: better usability (e.g., navigation, organization); more content; better relevance for different age groups and library sizes; and improved sharing of information (including timeliness of when the program materials are made available).

Satisfaction with the evaluation process

There is good satisfaction with the evaluation process (64% rate it 8-10 out of 10). This level of satisfaction is also down from 2017 (consistent with the trend in overall program satisfaction) and has returned to 2016 levels.

Libraries also give positive ratings for the ease of navigating the system (70%) and that the evaluation asks relevant questions (64%). Notably, however, both of these ratings are up from 2017, in direct contrast to other satisfaction measures. It is unclear why this is but does suggest that usability and relevance may not be the primary drivers of overall satisfaction with the evaluation process.

Suggestions for future improvements to the website include: making is easier to collect and accurately enter the desired information; improving timeliness (i.e., when questions/data requirements are made available and when the online form is ready at the end of the program); usability issues; and, the opportunity to provide more in-depth contextual information around the statistics.

6. Political neutrality statement and contact information

I hereby certify as a Senior Officer of Environics Research that the deliverables fully comply with the Government of Canada political neutrality requirements outlined in the Communications Policy of the Government of Canada and Procedures for Planning and Contracting Public Opinion Research. Specifically, the deliverables do not contain any reference to electoral voting intentions, political party preferences, standings with the electorate, or ratings of the performance of a political party or its leader.

Introduction

1. Background & research objectives

The TD Summer Reading Club (TDSRC) focuses on young Canadians, promoting the fun of reading and encouraging them to visit their local library over the summer months. The program gives children a structured reading environment and rewards personal achievements. Beginning in Toronto in 1994, the program expanded across Ontario in 2001 and has been widely available across Canada under the auspices of Library and Archives Canada as of summer 2004.

In collaboration with the Toronto Public Library and Library and Archives Canada, teams of librarians from Winnipeg, Toronto, Ottawa, Montreal and Quebec City created the bilingual material for this program.

The TD Summer Reading Club is a national bilingual program that engages kids in the joy of reading while championing Canadian writers and illustrators. Kids who register at their local library receive free physical program materials and can access the program materials through the website. Participating libraries are provided with thematic booklists, access to illustrations, programming and activity ideas and free outreach materials for promoting the program in their community.

The study population is the library systems, branches or independent libraries that signed the Letter of Agreement, received free program materials and participated in the TD SRC during the summer of 2018. This year, the TDSRC was offered in eleven provinces and territories across the country through the support of TD Bank Group.

Library and Archives Canada's (LAC) interest in conducting this study is to provide information about the reach and success of the program to the TDSRC Partners, TD Bank Group, Toronto Public Library, and Library and Archives Canada as well as to participating libraries. In addition, feedback gathered will inform improvements to the TDSRC program. LAC's specific research objectives were to:

• Measure the number of programs, activities and attendance attributed to organized by, or otherwise associated with, the TD Summer Reading Club (i.e., those offered by participating libraries to children in the library or in the community that were part of the TDSRC).
• Determine what the various library systems, library branches or independent libraries considered to be their program reading outcomes or indicator of reading successes resulting from the TDSRC.
• Compare the measures of success, when possible, with at least the previous two years with a view to advising on developing appropriate performance measurement parameters.
• Provide recommendations for how the TDSRC program may be improved.

2. Methodology

As a provision for receiving the free program materials, each library was asked to collect and evaluate statistical information related to the program. So that libraries would know what information to collect over the summer, a version of the evaluation form was included on the Reading Club website.

Library and Archives Canada teamed with Environics to collect the requisite information and perform the analysis. As libraries are organized differently in each province and territory, a common denominator to respond to the evaluation was identified – service points. A service point may be an individual library or a library which is part of a library system with many branches. An individual library represents one service point while a system with five separate branches would represent five service points. Evaluations were either completed by individual libraries themselves using the online Statistics and Evaluation Form or individual branches provided the necessary information regarding their program to their library system administrator who filled in the form for their entire system. An example of the former is the Toronto Public Library, whose system includes roughly 100 library branches/service points. Each service point within the Toronto Public Library completed their own evaluation survey and entered information for their branch only.

LAC provided Environics with a database containing the most recent contact information available for all participating libraries/systems from each regional system coordinator. A regional system is a larger grouping of libraries/systems, usually at a provincial level (although within Ontario and Quebec, a total of five separate regional systems exist). All individual libraries and library systems were invited to complete their evaluation online via a bilingual email invitation with a unique link embedded in the email text. This approach ensured that no duplicate entries would be possible and allowed for a means to track which libraries/systems had submitted their results.

The unique link pre-identified individual libraries and library systems and brought them directly to the survey where they entered their data. To accommodate reporting for both individual libraries and systems with multiple service points, two separate survey instruments were programmed. Individual libraries were sent a link to the library survey, enabling them to enter data for their library only, whereas those pre-identified to be reporting for multiple service points (library coordinators) were sent a link that led them to the library systems survey, enabling them to enter data for multiple service points.

In 2018, the opening of the form was delayed due to contracting issues and was only made available to libraries on September 27th^{Footnote 1}. Environics sent out 893 email invitations in both official languages to participating public libraries or systems within eleven participating regional systems. In total, 701 evaluations were collected between September 27th and November 27th, 2018. The completed evaluations contained data for 1,738 of the 2,052 total service points that participated in the TDSRC in 2018, resulting in a response rate of 85 percent.

During the data collection period, Environics provided LAC with updates on which systems had not yet accessed their online file. LAC relayed this information to provincial/territorial coordinators, who then followed up with these systems to encourage participation. Additionally, Environics sent a minimum of three reminder emails over the course of the project.

3. Response rate and weighting

Response rate. Participating libraries or systems were asked to tally the evaluation results for their own library and any subsidiary service points. The response rate is calculated as the proportion of all service points participating in the TDSRC for whom evaluation data was received. Evaluation data was submitted for 1,738 service points from a total of 2,052 libraries that participated in the program, for a response rate of 85 percent. The table on the next page presents the response rate by region.

Weighting. The weighting scheme was designed to account for all of the individual libraries within a given regional system or province, including those who responded to the evaluation and those who did not. A weighting factor was applied to libraries who responded to the evaluation within a specific regional system, so that the regional results are in their correct proportion within the total of all Canadian libraries who participated in the program. For example, in Alberta, a total of 246 service points participated in the Summer Reading Club in 2018, but statistics were available for only 208 of them (85%). The responses from those 208 libraries were multiplied by a factor of 1.18 to account for the 38 missing service points and ensure that the totals for Alberta libraries accurately reflect the whole province and represent 12% of all libraries who participated in the program nationwide (246 service points of the 2,052). This approach assumes that the libraries who provided evaluation data are similar to the libraries who did not (i.e., there is no meaningful difference between the two groups).

Detailed Statistical Findings

Section 1: Program registration

Registration tracking

More than 346,000 children were registered in 2018, an increase in keeping with the trend over the past ten years and higher in almost every region than in 2017.

Overall, the number of children registered for the TD Summer Reading Club in 2018 was 346,604. This is an increase over the previous two years of almost 20,000 and continues a longer trend of steady increase in registration.

The largest relative increase was among independent libraries (up 53% from 2017), although this fluctuation is largely due to the small number of independent libraries that participate in the program each year (7 in 2017 and 11 in 2018). There was also a substantial registration increase in Quebec (up 18%), driven by growth in the Reseau BIBLIO system; this brings Quebec's registration back in line with 2016. The most notable decrease was in Atlantic Canada (a nine percent decrease in registration).

Registration figures going back to 2014 are shown below for comparison purposes.

Children registered by age

Children between the ages of six and eight were the most likely to participate in the TDSRC.

In 2018, registrations were most likely to include children aged 6-8 (39%), followed by children in the 0-5 (29%) and 9-12 (29%) age groups; only three percent of registered children were 13 years or older. The distribution of registered children by age group is roughly similar across regions, with the exception that libraries in the Territories and independent libraries tend to report greater participation among pre-teens (9-12). The overall age distribution has remained consistent from year to year.

The table below summarizes the number of children within each age group that participated in the 2018 program by province/network.

Proportion of the population registered

Roughly six percent of children across Canada registered in the 2018 TD Summer Reading Club

The table below presents the number of children registered in the program as a proportion of all Canadian children under 18, per the 2016 Census. Nationally, six percent of all children in parts of Canada with the TD Summer Reading Club register for the program, which is generally consistent with previous years (five percent in both 2016 and 2017).

The areas with the highest proportion of children registering in the program were Saskatchewan (11.4%) and Nova Scotia (6.9%) with Newfoundland (2.3%) and the Territories (1.1%) having the lowest.

Program language

The program was run largely along provincial language lines, although there were also a fair proportion of bilingual programs in Quebec, OLS-North and among independent libraries.

Across the country, most libraries ran the program in the primary language of their province: unilingual English programs outside Quebec and unilingual French programs inside Quebec. However, one in five libraries in Quebec (22%) ran a bilingual program; there was also substantial representation of bilingual programs from independent libraries (33%), libraries in OLS-North (29%) and Alberta (11%).

Previous program participation

Six in ten registrants in the 2018 program had participated in previous years. The results indicate a decline in new registrations over the past two years, and a corresponding increase in retention of previous participants.

Nationally, six in ten children (60%) had participated in a TDSRC in a previous year, while four in ten registered for the TDSRC for the first time in 2018. The proportion of previous registrants increased for the second consecutive year, where prior to 2017, new and previous registrations were about equally balanced. This trend suggests that although there are fewer new registrants each year, retention among previous participants has improved, leading to the overall increase in program registration.

However, this shift toward greater retention of previous participants is driven almost entirely by Manitoba (from 52% to 85%) and Alberta (from 59% to 85%), while the proportion of previous participants has remained relatively stable in most other regions. Thus, it remains to be seen how the source of registrations (new vs. previous participants) will evolve in future years.

Section 2: Program participation statistics

Programs & activities organized around the club theme

More than 45,000 TDSRC-related activities were held in 2018 attended by more than 830,000 children. Eight in ten of these activities were held in libraries and more than half of them happened in Ontario.

Libraries held program-related activities at their libraries or in their communities which did not necessarily require registration in the TD Summer Reading Club (and were not events for promotional purposes). A total of 45,522 activities were held across the country, attended by 831,546 children.

Libraries were four times more likely to hold these events in their library than elsewhere in their community and an average of 18.3 children attended each activity nation-wide. The region with the most total activities and attendance was Ontario (around half of total activities and attendance) with Western Canada accounting for roughly 30 percent of each.

The number of activities has increased dramatically since 2008, resulting in a corresponding increase in attendance.

There was a sizeable increase in the number of activities and attendance between 2017 and 2018 with both jumping by around 25 percent. The size of the groups for each activity has remained stable over the past year (average of 18 children per activity), but the larger number of activities taking place has resulted in an overall increase in attendance. The largest year-over-year regional change was in Alberta which increased from 6,093 activities with an attendance of 112,670 in 2017 to 9,588 activities with an attendance of 176,602 in 2018^{Footnote 2}.

Promotion of programs & activities organized around the club theme

Three-quarters of libraries made at least one promotional visit in 2018, most commonly to schools. The proportion making visits to schools or childcare centres is lower than in 2017, but on par with the 5-year trend.

Librarians were asked if they promoted the program at specific locations (schools, day camps, childcare centres and other locations) and, if so, how many visits they made and how many children they estimate reaching. Most libraries made visits to schools (60%) while a smaller proportion visited childcare centres (25%), day camps (19%) or other locations (21%). Overall, three in four libraries (76%) across the country reported making at least one promotional visit to any location.

In total, librarians estimated that almost 659,410 children were reached as a result of these promotional efforts over the course of 11,065 separate promotional visits. More promotional visits were made to schools than to all other locations combined and these visits accounted for the vast majority of children contacted since an average of 74 children were reached per school visit.

Regionally, libraries were most likely to have made promotional visits to schools in the West (67% of libraries) and Ontario (63%). Visits to childcare centres were more common amongst Independent LAC libraries (33%) while visits to day camps were more widespread amongst libraries in the Territories (50% of libraries).

The proportion who report visiting schools and childcare centres is down from 2017, but in line with the broader five-year trend.

Section 3: Program satisfaction

Overall program satisfaction

Libraries were asked about their overall satisfaction^{Footnote 3} with the program and with specific aspects (e.g., web content, navigation and resources). Trend data is available in most cases only for 2016 and 2017, since the Statistics and Evaluation Form changed substantially (streamlined) following the 2015 study.

Overall satisfaction with the 2018 TD Summer Reading Club is high but has slipped a few points since last year.

Satisfaction with the TD Summer Reading Program is high overall, with seven in ten (70%) who are satisfied (a rating between 8 and 10), one-quarter (23%) who give moderate ratings (6-7) and eight percent who are less satisfied (0-5). The overall satisfaction rating is slightly lower than 2017 (down from 76%). This question was re-introduced in 2017 so data before then is unavailable.

Overall program satisfaction is highest in Quebec and this level has not changed substantially since 2017. The slight decline in overall satisfaction is evident in the three other regions, Atlantic Canada (down 11 points), Ontario (down 9) and the West (down 6), where it is now lower than in Quebec.

Staff website satisfaction

Satisfaction with the website/content for librarians and with ease of navigation remain strong. Although not as high as in 2017, satisfaction is in line with 2016 results.

Two thirds of librarians are satisfied with the website and its content (66% rating 8-10) and with the ease with which they can navigate the website (67%). In both cases, most of the remainder give a moderate rating (6-7), with only one in ten who are less satisfied (0-5 rating). Satisfaction with both aspects of the website has returned to 2016 levels, after slightly higher ratings in 2017.

As with overall program satisfaction, satisfaction with these two aspects of the website remains highest in Quebec (where ratings are stable), and has declined in Atlantic Canada, Ontario and the West.

Use of and satisfaction with English website resources

Use of English materials was stable or trending upward and satisfaction remains high.

Librarians were asked about their use of, and satisfaction with, the various resources provided for the language in which they ran their program. Given that the resources differ by language, librarians were asked about the English resources if their program was run in English and the French resources if it was run in French, regardless of their location^{Footnote 4}. Although all results are included in the totals, regional data is only reported for regions where at least ten libraries responded.

Use. Among libraries that used English materials, the most widely used resource continues to be the Images (83%), followed by the Recommended Reads (74%) and the Activities (70%). Use of the promotional templates (70%, up from 55% in 2016) and the brand guidelines (62%, up from 53%) have both increased steadily over the past two years. More than half used the Programs (56%) and the 'How to Run a Successful Program' section (55%). The least used English resource continues to be the staff newsfeed (39%), although this is level has been increasing (from 24% in 2016).

Regionally, libraries in Atlantic Canada were generally more likely to use each of the available resources than libraries in the West or Ontario.

Satisfaction. Satisfaction is high for each of the English resources, particularly the promotional templates (74% with a score from 8 to 10) and the program images (72%). Two thirds were satisfied with the other resources except for the staff news feed, which received slightly lower satisfaction scores (57%).

Regionally, although libraries in Atlantic Canada were most likely to use these resources, libraries in the West were more likely to be satisfied with them (except for the brand guidelines).

Use of and satisfaction with French website resources

Use of French materials is trending downward in many cases, but satisfaction remains very high.

Libraries that ran their programs in French were asked about their use of and satisfaction with the French resources available to them. The Quebec results are broken out by system; due to small sample size, results are combined for libraries who used French materials outside Quebec.

Use. The illustrations were the most commonly used French resource (83%), followed by suggestions de livres (78%), both at levels similar to previous years. Most of the other French resources have seen a steady decline in reported use since 2016, including modèles et directives (53%, down 15 points), the activités express (52%, down 9 points) and the bricolages (51%, down 9 points). As before, the activités longues (35%) remain the least widely used.

Libraries outside of Quebec were more likely than those in Quebec to use most of the resources with the exceptions being the activités longues, the nouvelles du Club and the conseils pour la mise sur pied d'un club de lecture d'été réussi. This difference may be due to the relative lack of access to French materials for libraries outside Quebec, who therefore make good use of the French program materials.

Satisfaction. Users of the French web resources were then asked to rate their satisfaction with those resources. Satisfaction is highest for the illustrations (83% rating of 8-10), followed by the modèles et directives (77%) and the guide sur l'image de marque (75%). There was also strong satisfaction with other French resources, including two-thirds or more of users expressing satisfaction with the suggestions de livres (72%), the bricolages (70%) and the activités express (69%). Reseau BIBLIO libraries were more satisfied with the available resources than the ABPQ libraries, although it is unclear why this is.

Staff web resource suggestions

The most common single suggestion was a simpler, more user-friendly website, but libraries also expressed interest in better content, targeting and communications.

Librarians were asked for suggestions on how to improve the librarians' website for future years. The most common suggestions centred on the content, particularly a desire for more ideas or specific outreach examples. Other suggestions included: better targeting of materials (either by age of the child or library size), improved usability (e.g., navigation, search functions), and better communications (especially making the materials available sooner). Three in ten (30%) said that they were satisfied or had nothing to suggest.

Section 4: Program evaluation satisfaction

Overall satisfaction with the program statistics & evaluation process

Since libraries are required to report program statistics as a condition of their participation, the TDSRC partners have an interest in making the evaluation process as easy as possible. Each year, librarians are asked for feedback on the process to highlight any weaknesses or areas of concern and to provide suggestions for how it could be improved.

A majority of libraries are satisfied with the program evaluation process, although this is lower than in 2017.

Satisfaction with the overall evaluation process was high with two in three (64%) librarians providing a top three box satisfaction score (rating of 8-10). This represents a return to 2016 levels (67%) after slightly higher satisfaction in 2017.

Quebec libraries were the most satisfied with the program evaluation process (69%). The decline in satisfaction was most acute in Atlantic Canada (falling 14 percentage points) and Ontario (a fall of 12 percentage points), both of which are back in line with 2016 levels.

Ease and Relevancy of the Program Statistics & Evaluation System

Satisfaction with the relevancy of the questions asked and the ease of using the system were both higher than in 2017.

Although overall satisfaction with the program evaluation process fell in 2018, librarian ratings of two specific elements of the process were higher than the previous year. Seven in ten (70%) gave a top three box score for the ease of using the system (up from 63 percent in 2017); more than six in ten (64%) were satisfied that the evaluation asks about relevant concerns (up from 43% in 2017). The reason for these changes is unclear, since the evaluation questions were virtually identical. However, given the decline in overall satisfaction with the evaluation process, it suggests that ease and relevancy of the system may not be the primary drivers of overall satisfaction.

Regionally, there were no consistent trends beyond Atlantic Canada providing the lowest scores on both questions in 2018.

Program statistics & evaluation process suggestions

Suggestions for improving the evaluation process commonly revolved around clarifying data requirements to ensure data is entered accurately, but also improving timeliness, usability and the depth of information collected

Librarians were asked for their suggestions on how the program evaluation and statistical collection process could be improved. The most common suggestion was ensuring data requirements are clear and data can be accurately recorded (for example, how to record children who weren't officially registered; being forced to enter a number for which they did not collect data).

Improved timeliness was also a common suggestion, with librarians requesting the questions sooner and having the online evaluation form ready closer to the end of the program.

Other suggestions included improving usability (e.g., simplify the process) and improving the depth of the information (particularly the opportunity to provide contextual information that explains the statistics). More than one in three librarians (35%) had no suggestions for improvement.

Testimonials Indicating an Increased Love of Reading

Testimonials about the success of the program included more tangible observations about increased reading skills as well as more qualitative results about excitement to read.

Libraries were asked to indicate whether they had any indicators of children's increased enjoyment of reading, reading successes or changes in attitudes toward reading. Many libraries reported concrete and quantifiable outcomes such as improved reading levels, exceeding the goals of the club, more children coming to the library and children being more willing to read at home. However, in a more qualitative way, librarians report that parents and their kids love the program, that the program excites kids to read over the summer, that children enjoy the activities and that they were motivated to read more.

Contributions from TD employees

Libraries were asked whether TD employees played any role in their TDSRC programming in 2018. A selection of responses from this question are given below:

"The entire staff at TD Canada Trust Renfrew branch played a key role in the TD Summer Reading Club. They handed out library TDSRC information, put display in bank, hung SRC posters, as well as providing all the prizes for our TDSRC culmination party!"

"TD in Midland always supports us with really great products to give out to the participants i.e. water bottles, beach toys, pens, notebooks etc. They are such a good community partner and also support us for other events such as our 'Battle of the Books' program for children. They always say, "what can we do for you" and mention that literacy especially for school age children is very important to them."

"Our local TD branch provides further financial support for our summer reading program, which allows us to purchase books as prizes for our young readers."

"Our local branch has a fabulous Community Liaison named Bev. She is always willing to supply us with what resources she has - allows us to come and set up a manned display to promote the program."

Section 5: Conclusions and recommendations

The TDSRC continues to be a highly successful program. Roughly six percent of Canadian children were enrolled in the program last summer, and the number of registrations continues to grow; an even larger number of children were reached through theme-related activities and promotional visits. Librarians' open-ended feedback testifies to the benefits for children and their families by promoting a love of books and reading.

Among the libraries themselves, there is widespread use of, and satisfaction with, the resource materials provided, as well as good levels of satisfaction with program overall. There is lower satisfaction with the evaluation process than in 2017 (closer to 2016 levels), although the data in not conclusive about why this might be.

Based on the findings, there are two key recommendations for future evaluations of the TDSRC:

Changes to process – Generally speaking, it would be beneficial to move up the planning of the evaluation data collection activities to earlier in the larger TDSRC process – to have the program evaluation be more a part of the TDSRC program design, rather than done after the program is concluded. This would likely require contracting the evaluation research much sooner (i.e. before the program materials are sent to librarians) so that a research firm can advise on the questionnaire itself.

Librarians have pointed out the importance of knowing what information they will need to collect before they run their program so the form would ideally be finalized earlier in the process. This would also allow the evaluation to be launched earlier (preferably immediately after the TDSRC ends). Although the launch date of the form has varied over the years, late August is the ideal time to make it available to libraries since the TDSRC statistics are often collected by summer students who leave the library around this time.

Changes to content – While the statistical evaluation form has been largely consistent for several years, feedback suggests areas where improvements could be made to the evaluation questions:

• Additional/expanded questions: The wording of some of the questions around satisfaction could be revisited to ensure that they capture all relevant areas of concern. In certain cases, it is hard to determine why numbers (e.g. satisfaction with the evaluation process) have changed and additional questions could be added to explore this more fully. The section on satisfaction with the statistical evaluation only asks about two aspects of the process and could be expanded to seek variables which could also be affecting satisfaction with the process.
• Double-barrelled question wording: Similarly, we recommend a review of the wording used in the section about satisfaction with the staff website and content. The first question in this section merges together the website itself and the content for librarians. While these are similar concepts, this question would be more effective if it separated out the two and asked about each separately.
• Unclear question wording: The final question of the questionnaire asked librarians about the role played by TD employees in their TDSRC. In reviewing the results of this question, many librarians did not understand what this question was asking and instead provided information about the number of hours that library staff spent on the program. If this question is to be retained in future years, different wording will need to be used to make clear what information is being sought.
• Indicators of success: The objectives of this research include determining what libraries consider to be their program reading successes from the TDSRC. Although information on this is obtained through the key metrics and the open-ended questions about testimonials of project success (and has been the focus of qualitative research in earlier years), we feel that this could be more thoroughly explored through additional open- or closed-ended questions that focus on this topic.
• Forced responses: Based on the open-ended feedback, a review of the questions where librarians should be forced to enter a response should be done. Currently, these questions are the main metrics of program success (registration, activities/attendance, program promotion), so either it could be made possible for librarians to select 'no data' and bypass them or additional communication with libraries could be provided to stress that these metrics must be provided in the evaluation form. It may also be helpful to allow libraries to provide open-ended feedback/context for individual questions if they have issues providing data.

We feel that the benefits of these changes in content would include a better understanding of what libraries like/don't like about the program as well as how the program could be further improved.