Final Report
November 2011
This document is also available in PDF (447 KB) [help with PDF files]
The Canada Border Services Agency's (CBSA) mandate is twofold: to facilitate the movement of legitimate travellers and goods and intercept those travellers and goods that pose a threat to Canada. The CBSA fulfills this mandate by providing integrated border services that support national security, public safety and economic prosperity priorities. A cadre of well-trained and effective border services officers (BSOs) are the foundation of the CBSA's frontline service delivery.
The CBSA ensures that each Port of Entry (POE) has adequately prepared BSOs to deliver this frontline service through a recruitment process and training program. BSO recruitment is a centrally managed process with responsibility shared between Headquarters (HQ) and regional offices. While HQ manages the recruitment campaign, the regional offices assess the candidates and ensure that the required supporting documents are included in the candidate's file. Once recruits have successfully completed the application process and have met the initial conditions of employment, they are eligible to be selected by a region to attend Port Of Entry Recruit Training (POERT). Since April 2007, POERT has consisted of three components: an online orientation, classroom training at the CBSA Learning Centre in Rigaud, Quebec, and an in-service component that is delivered once the BSOs have been deployed in the regions.
Over the period of fiscal year 2006-2007 to 2010-2011, CBSA expenditures[ 1 ] on BSO recruitment and POERT [ 2 ] are estimated to have been $106.6M or an average annual cost of $21.3M. As an example, total expenditures for fiscal year 2010-2011 are estimated to have been $18.4M. This figure includes an estimated $5M for recruitment, $11.4M for Online and Core POERT, and $2M for In-service training. However, after factoring in additional internal services[ 3 ] costs, total recruitment and POERT expenditures are estimated to have been $30.2M[ 4 ] in 2010-2011.
The evaluation incorporated multiple lines of evidence, including analysis of CBSA statistics and performance data in relation to BSO recruitment, POERT, budget and expenditure data; review of documents, including design, delivery and performance documents, as well as planning and governance documents/reports outlining roles and responsibilities, processes and procedures, Memorandums of Understanding and others related to CBSA priorities. An online survey was administered to BSOs who graduated from POERT between the periods of January 2010 to February 2011 in order to gather the BSO perspective on the recruitment process and POERT. In addition, one-on-one and group interviews were conducted with key stakeholders including management in CBSA HQ and in the regions, as well as superintendents, chiefs and BSOs at the POEs. Site visits were conducted in the Pacific and Greater Toronto Area regions and the CBSA's Learning Centre in Rigaud.
Overall, POERT is addressing an ongoing need to provide recruits with the specialized knowledge and skills needed to work in the unique CBSA operational environment. BSO recruitment and POERT are consistent with and contribute to the realization of the CBSA's goal of ensuring Canada's security and prosperity, and are aligned with the Agency's Change Agenda.[ 5 ]
The Agency's recruitment process has been successful in attracting candidates who want to establish a career within the CBSA. When participants in the BSO survey were asked where they saw themselves in five years, 84% indicated that they saw themselves working at the CBSA. Overall, the national recruitment processes have been able to supply the recruits needed to meet regional requirements for new BSOs. However, some challenges continue to exist in terms of staffing bilingual positions and positions in remote areas.
The Agency held two national recruitment processes in 2007 and 2008. One was to staff BSO positions across Canada and the other to meet the specific needs of the Pacific Region. These processes were conducted to meet an increased demand for BSOs that in part had resulted from initiatives like Doubling-up and Arming. While most of the recruitment needs of the Agency were met by the above national processes, there was a shortage of bilingual recruits and recruits that were prepared to work in remote POEs in the Pacific and Prairie regions. To address these issues, the CBSA conducted two more targeted recruitment processes in 2009. Despite these initiatives, not all regional needs were met and two further recruitment processes were initiated in 2011: one process targeted English speaking candidates for specific locations, while the second process was to fill bilingual positions in specific locations.[ 6 ]
The lack of a defined job performance standard[ 7 ] for BSOs on completion of POERT has resulted in differing expectations and understanding across the Agency with respect to the new BSOs' job performance abilities. While there are clearly articulated training objectives for Core and In-service POERT, there is no defined job performance standard that details the competencies, skills, knowledge and abilities that a new BSO will have on completion of both Core and In-service POERT. As a result, there is a lack of understanding about what jobs and tasks a BSO is capable of performing within a POE on completion of Core POERT versus In-service POERT.
The Training and Learning Directorate within the Human Resources (HR) Branch is regularly called upon to respond to requests from program areas to include additional content in the Core POERT curriculum. Without a job performance standard it is difficult to fully assess whether or not the request directly impacts the job performance requirements of a new BSO and should, therefore, be included in the curriculum as opposed to the development and dissemination of a job aid, for example. If there is a lack of consensus on what this job performance requirement is, and if the training in its entirety is not assessed on a regular basis, it is difficult to ensure that the training provided is actually meeting said job performance requirement. The lack of such a standard also serves to underscore the importance of In-service POERT. It is In-service POERT that provides the new BSO with the on-the-job experience, combined with formal classroom instruction, needed to be operationally effective at a POE.
The POERT curriculum states that the In-service component of POERT is mandatory and must be completed within 12 months of a recruits' arrival at the POE. However, nearly 17% of new BSOs surveyed did not complete In-service POERT within the 12-month period. Furthermore, a significant number did not receive the number of In-service classroom training days specified in the POERT curriculum learning path. In addition, nearly half (43.5%) of that group indicated that they were either dissatisfied or very dissatisfied with the In-service training they received at the POE.[ 8 ]
While regions indicated that they have some form of tracking in place to record whether In-service training has been completed, there is no standardized monitoring and reporting throughout In-service POERT and, consequently, there is no national data available to indicate how many BSOs did not complete any In-service training.[ 9 ] As a result, the Agency does not have an accurate picture of the In-service POERT training activity and the degree to which POERT has been fully implemented. Nor does the Agency have the assurance that all new BSOs have in fact completed all required recruit training.
In response to the 2007 Report of the Auditor General of Canada,[ 10 ] the Evaluation and Monitoring Division within the HR Branch conducted a limited level 1 and level 2 training evaluation of POERT in 2010-2011 and is scheduled to conduct a limited level 3 training evaluation of POERT in November 2011.[ 11 ] While the HR Branch proposes to conduct these as a means to assess whether or not employees have the appropriate knowledge and skills to perform their jobs,[ 12 ] there are no plans in place to conduct further level 3 training evaluations on a regular basis beyond November 2011. It is level 3 training evaluation that determines whether what is learned is being applied on the job and whether that application of learning is meeting the job performance requirements of the organization. Consequently, it will be difficult to assess the alignment of POERT, or any program that replaces POERT, with the job performance requirements for a new BSO.
In light of these findings, the evaluation recommends:
The HR Branch work with Operations Branch, in consultation with Programs Branch, to develop and implement a job performance standard that details the POE job performance requirement for BSO recruits on completion of POERT.
The HR Branch develop and implement a monitoring and training evaluation strategy for POERT, or any program that replaces POERT, to ensure that the delivery of such a program is consistent and compliant with the curriculum requirements and that the training delivered matches the job performance standard of BSO recruits.
The requirement to complete In-service POERT within the 12-month post Core POERT period coincides with the probationary period of the new BSO. However, there is no formal assessment against a standard set of competencies required of the new BSO during or at the completion of In-service POERT. Of the BSOs surveyed for this evaluation, 52.5% indicated that their performance had not been assessed against identified goals and objectives. Furthermore, nearly half (49.8%) of BSOs surveyed indicated that they had not discussed their In-service training and the results they were expected to achieve with their immediate supervisor. The evaluation observed that it is possible that the probationary period is not being used as effectively as possible to ensure that those new BSOs who are poor performers are released during that probationary period. Once the probationary period has ended, it becomes much more difficult to terminate the employment of an individual for poor performance or for not meeting the job performance standard. Placing new BSOs, who have completed Core POERT, in an FB developmental position would recognize that they do not fully meet the requirements of the position until In-service training has been completed, and the employee has been assessed as part of the probationary requirements and fully meets the job performance standard of the position. It would also serve to underscore the importance of an In-service component of any new training program for new BSOs.
In light of these findings, the evaluation recommends:
The HR Branch develop an assessment tool that is used to assess all BSO recruits' performance prior to the end of the 12-month probationary period to ensure that they complete the required training and meet identified performance objectives.
The evaluation noted that the cost per POERT graduate has increased significantly, due mainly to declining intake levels. Between 2006-2007 and 2010-2011, the number of POERT recruits dropped by 44% while expenditures increased by 13% during the same time period. As a result, the cost of Online and Core POERT per recruit has risen from $14,100 in 2006-2007 to $28,290 in 2010-2011. The cost per successful Core POERT graduate[ 13 ] increased from $15,900 to $32,100 over the same period. While recent declines in the number of POERT participants have driven costs up per recruit, the proportion of recruits that are successful in completing Core POERT has increased, resulting in a more efficient use of resources invested by the CBSA. It is also significant to note that the CBSA Learning Centre was successful in accommodating a surge in demand between 2006-2007 and 2007-2008. Over this period the number of POERT participants doubled while the increase in expenditures was held to 38%.
The evaluation found that the Agency incurs costs due to overestimation of POERT seat requirements by regions. Regions estimate seat requirements late in the third quarter of the fiscal year, prior to receiving budget forecasts. Regional managers interviewed indicated that seat allocations are often reduced once budget forecasts are available. The CBSA Learning Centre plans for and incurs fixed costs based on the annual forecast number of recruits as provided by the regions. These costs are not easily reduced in response to last-minute cancellations. For example, in 2010-2011, the CBSA Learning Centre planned to train 432 recruits, but due to such cancellations, only trained 359 recruits. As a result, the forecast cost per recruit, estimated at $23,509, increased by about 20% to $28,290. The CBSA Learning Centre has attempted to mitigate the impact of cancellations by offering released seats to other regions. This can be a successful strategy provided there is adequate lead time, since all conditions must be met and security screening completed before a recruit can proceed to Core POERT. Managers interviewed indicated that lead time is particularly important with respect to the security screening requirement, since it is only conducted once a recruit has been selected to attend POERT. The declining intake levels in the foreseeable future will require precise management of the seat allocation process to ensure maximum efficiencies in training delivery and use of the CBSA Learning Centre.
In light of these findings, the evaluation recommends:
The HR Branch work with Operations Branch to develop a process for seat allocation on POERT that better aligns with the budget allocation process.
The Canada Border Services Agency's (CBSA) mandate is twofold: to facilitate the movement of legitimate travellers and goods and to intercept those travellers and goods that pose a threat to Canada. The CBSA fulfills this mandate by providing integrated border services that support national security, public safety and economic prosperity priorities. The CBSA was created on December 12, 2003 by an order-in-council amalgamating Canada Customs (from the-then Canada Customs and Revenue Agency) with border and enforcement personnel from the Department of Citizenship and Immigration Canada and the Canadian Food Inspection Agency.
The newly created CBSA resulted in an organization with a broad range of responsibilities, impacting especially frontline staff. This in turn necessitated the development of a comprehensive training program, called Port of Entry Recruit Training (POERT), for border services officers (BSOs) that would address the new areas of responsibility. Major initiatives such as the arming of BSOs (2007), doubling up to address work-alone situations (2008) and the elimination of summer student BSOs at armed ports of entry (POEs) increased recruitment and training requirements. Since 2009, the CBSA has relied on a workforce that includes about 5,300 BSOs (Exhibit 1).
Year[ 1 ] | 2006 | 2007 | 2008 | 2009 | 2010 | 2011 |
---|---|---|---|---|---|---|
Number of BSOs | 3,963 | 4,319 | 4,936 | 5,377 | 5,399 | 5,307 |
Source: CBSA Demographics and Workforce Analysis Unit, August 16, 2011
Over the period of fiscal year 2006-2007 to 2010-2011, CBSA expenditures[ 2 ] on BSO recruitment and POERT[ 3 ] are estimated to have been $106.6M or an average annual cost of $21.3M. As an example, total expenditures for fiscal year 2010-2011 are estimated to have been $18.4M. This figure includes an estimated $5M for recruitment, $11.4M for Online and Core POERT, and $2M for In-service POERT. However, after factoring in additional internal services[ 4 ] costs, total recruitment and POERT expenditures are estimated to have been $30.2M[ 5 ] in 2010-2011.
BSO recruitment is a centrally managed process with the responsibility shared between Headquarters (HQ) and regional offices. While HQ manages the recruitment campaign, the regional offices assess the candidates and ensure that the required supporting documents are included in the candidate's file. Regional assessors use a standard rating guide designed by HQ for the recruitment and assessment of candidates.
Prior to March 2007, BSOs were recruited via regional selection processes and many new recruits were student BSOs that were bridged into indeterminate positions. As of March 2007, a national approach to BSO recruitment was implemented. The new approach was in response to the Public Service Employment Act's requirement for a national area of selection for officer-level jobs, and a reduction in the number of student BSO positions as a result of the Arming Initiative. National processes were intended to create a national pool of candidates that would allow for better coordination of BSO staffing between regions, facilitate staffing in geographic areas where challenges existed, to enhance linguistic and employment equity representation and to speed up the staffing process.[ 6 ]
Two national BSO recruitment campaigns were carried out in 2007 and 2008. The ensuing pools of candidates continue to be drawn from, and will also be used to fill the September 2011 intake requirements. In 2009 and 2011, the CBSA conducted four (two for each year) targeted recruitment processes in an effort to meet the need for bilingual BSOs and to staff positions in some remote locations.
Once recruits have successfully completed the application process and have met the initial conditions of employment, they are eligible to be selected by a region to attend POERT. The 2004 Vision for Ports of Entry called for the creation of an integrated training program for new recruits that would include all program lines. While this concept was further developed, the CBSA launched an interim POERT program in April 2005 to address the new responsibilities of the BSOs. The POERT curriculum has been modified several times since then, as have the length and the components of the program.[ 7 ] Since April 2007, POERT has consisted of the three components: an online orientation, classroom training at the CBSA Learning Centre in Rigaud, Quebec and an In-service component that is delivered once the BSOs have been deployed in the regions. (Exhibit 2)
Exhibit 2: BSO Training Process
Source: CBSA National Institute of Learning and Excellence (NILE)[ 8 ], July 2010
In 2010-2011, the Curriculum Advisory Board (CAB) reviewed and assessed the POERT curriculum to determine whether integrating the BSO training program with other similar Agency training needs (e.g., training for inland enforcement officers) in an attempt to create a common Induction training program was feasible. The Training and Learning Directorate (TLD) is currently establishing a model for the Induction training program and will present the model to the Executive Committee at a future date.
The Training and Learning Directorate of the Human Resources Branch has overall responsibility for POERT. Advice on curriculum is provided by CAB. The board is comprised of one representative per region, two representatives from TLD (including the co-chair), two members from Operations Branch (including the co-chair) and four members from the Programs Branch. CAB participants are typically at the manager, chief and director levels.
The National BSO Recruitment Division of the HR branch is responsible for managing the National Border Services Officer Recruitment Initiative, developing future initiatives, modernizing recruitment approaches and identifying best practices to streamline the recruitment process within the Agency.
The Operations Branch is responsible for POE operations and is the Headquarters' link with the regions. The regions under the direction of a regional director general play a significant role in the assessment of BSOs during the recruitment process and in the delivery of the In-service phase of POERT.
The evaluation of the CBSA BSO recruitment and POERT activities was identified as a priority for completion in fiscal year 2011-2012 in the 2009-2011 CBSA Risk-Based Multi-Year Evaluation Plan, approved by the Executive Evaluation Committee in July 2010. The evaluation study covers BSO recruitment and POERT performance between 2006-2007 and 2011-2012. The purpose of this evaluation was to examine the relevance and performance of the CBSA recruitment and POERT activities, with a focus on the ongoing need for these activities, as well as their effectiveness and efficiency. As the POERT curriculum is currently being revised, the evaluation did not evaluate the POERT curriculum itself.
The CBSA's Program Evaluation Division carried out the evaluation research between January and August 2011. In preparation for the evaluation, the evaluation team consulted with key stakeholders to develop a logic model for BSO recruitment and POERT activities, around which a full evaluation project plan was elaborated. Similarly, research tools and instruments were developed on the basis of the logic model and the evaluation project plan. The following immediate and intermediate program outcomes were identified:
Specific evaluation questions are outlined in Exhibit 3.
Evaluation Issue | Evaluation Questions |
---|---|
Relevance | Do recruitment activities and POERT address an actual and ongoing need? |
Are POERT program objectives consistent with CBSA and federal government priorities? | |
Performance/Achievement of Expected Outcomes | Is BSO recruitment effective in meeting Agency needs? |
Does POERT provide graduates with the necessary skills and knowledge to effectively carry out BSO duties and responsibilities? | |
Are new recruits able to effectively balance the facilitation and enforcement roles? | |
How effective is the monitoring of trainee performance and oversight to ensure that standards have been adhered to across all phases of POERT? | |
Are recruitment activities, documents and training provided to recruits in both official languages? | |
Efficiency and Economy | Are BSO recruitment activities and POERT delivered efficiently? |
Are there more efficient and effective delivery models that would achieve the expected results? |
The following research methodologies were used:
To understand the context, management and delivery of CBSA recruitment and recruit training, the evaluation team reviewed a variety of documents, including design, delivery and performance documents, as well as planning and governance documents/reports outlining roles and responsibilities, processes and procedures, Memorandums of Understanding and other relevant documents related to CBSA priorities.
In order to answer the evaluation questions on performance and efficiency, the evaluation team collected and analyzed CBSA statistics, performance and financial data in relation to BSO recruitment and POERT. These included training and learning data, and budget and expenditure data. In addition, files and data relating to public complaints on BSO POE service delivery, correspondence and information related to BSO recruit complaints (e.g., Human Rights complaints) were reviewed and analyzed.
To help assess the performance of CBSA recruitment and POERT activities, the evaluation team reviewed documents on the design, delivery and performance of recruit programs in other government departments and other countries. The evaluation team connected with representatives from selected organizations including Parks Canada, Royal Canadian Mounted Police (RCMP), Department of Fisheries and Oceans, Correctional Service Canada, the US Customs and Border Protection and New Zealand Customs Service to facilitate comparisons.
The evaluation team conducted 37 one-on-one and 9 group interviews with a total of 66 individuals in the CBSA. These interviews were used to gather information pertaining to the design and performance of CBSA recruitment and POERT activities from a variety of perspectives. In addition to face-to-face interviews, telephone interviews were conducted with CBSA management in regions that were not included in site visits. Exhibit 4 provides a summary of the number and category of interviews that were conducted.
Interview Category | Number of Interviewees |
---|---|
CBSA HQ management | 20 |
CBSA Regional management | 19 |
CBSA BSOs | 13 |
CBSA Chiefs and Superintendents | 14 |
Total | 66 |
An online survey of all BSOs who graduated from Core POERT between January 2010 and February 2011 was developed and administered to gather the BSO perspective on the recruitment process and POERT. This group was selected in order to ensure that the survey results were obtained from a group for which the Core and In-service POERT experience was the most recent. The survey was administered between May 12, 2011 and May 25, 2011. The survey respondents represented all CBSA regions and the survey achieved a 71% response rate (359 surveys were sent out and 255 responses received).
To develop a better understanding of regional perspectives on POERT, including the In-service component that is delivered by the regions, the evaluation team conducted site visits at CBSA offices and POEs located in the Pacific and Greater Toronto Area regions. These regions were chosen due to the fact that they had received the majority of new BSOs, and because these BSOs had been deployed to positions in different modes within these regions. Site visits also provided an opportunity to conduct group interviews with new BSOs to identify strengths and weaknesses of the recruitment process and POERT from their perspective, and to clarify and confirm BSO survey results.
The evaluation team also made a site visit to the CBSA's Learning Centre. This provided an opportunity to conduct interviews with centre management and inspect the learning facilities.
This evaluation was conducted at the time when the CBSA Training and Learning Directorate was in the process of re-designing the POERT training program in an attempt to evolve the current program into an induction program. The evaluation team did not compare the existing POERT curriculum and structure to the proposed induction program as it was too early in the development process to be within the scope of this evaluation.
The evaluation team was not able to access financial information for the delivery of the In-service training in the regions. As a result, it was not possible to make an accurate estimate of the cost to deliver In-service training. Similarly, costing information for the recruitment process was only available for fiscal year 2008-2009 and, therefore, it was not possible to observe recruitment cost trends.
To successfully exercise its responsibilities, the CBSA must provide new recruits with the knowledge and skills needed to exercise their duties.
The CBSA is responsible for the administration of over 90 acts, regulations and international agreements. The Agency carries out these responsibilities with a workforce that includes 5,300 uniformed BSOs at 119 land-border crossings, 13 international airports, as well as major marine ports and marinas across the country. As is the case for other federal government departments and agencies such as the RCMP, Department of Fisheries and Oceans, Parks Canada and Correctional Services Canada, the CBSA must ensure that officers receive the specialized training and knowledge needed to conduct their duties. There is no external training available to prepare new recruits for the duties of a BSO, nor are there other law enforcement agencies or training programs that address the specific responsibilities and unique work environment typical of CBSA operations.
The CBSA has an ongoing need to recruit and train BSOs due to employee movement within the Agency and attrition.
In 2010-2011, approximately 26% of CBSA employees moved positions. This includes assuming acting positions, moving within the Agency, leave without pay and permanent departures and retirements. At the border, where the majority of BSOs are assigned, this figure is slightly lower at 18.6% or over 1,000 employee movements. The lower movement rate in border positions is partly explained by the fact that BSOs are, on average, younger than the overall CBSA workforce.[ 9 ] To meet operational requirements, there is a need to replace departing employees and to backfill positions that are temporarily left vacant. The recruitment of officers and the POERT program address this need.
% Acting position | % Transfer or Deployment | % Leave without Pay | % Departures (Quits, Retirements) | % of Total Employee Movement | # involved in Employee Movement | |
---|---|---|---|---|---|---|
At-Border | 3.6 | 7.0 | 4.4 | 3.7 | 18.6 | 1,095 |
Total CBSA | 6.5 | 10.2 | 4.2 | 5.1 | 26.0 | 3,546 |
Source: Workforce Development Analysis & Strategy, June 2011
Changing operational requirements and new initiatives have resulted in the need for ongoing recruitment activities and the training of new recruits.
BSOs operate in a work environment that includes the use of detection technologies (e.g. X-radiation, Vehicle and Cargo Inspection System) that are continuously evolving and improving. As well, they must be able to detect illegal activity at the border in the face of changing and more sophisticated concealment techniques. Initiatives such as doubling–up to address work-alone situations[ 10 ] have resulted in the increased need for BSOs. The introduction of the Arming Initiative also increased the Agency's training requirements. At the time of the evaluation, the POERT program was under review by a Curriculum Advisory Board and may include an Arming component in the future.
BSO recruitment and training are consistent with and contribute to the realization of the CBSA's goal of ensuring Canada's security and prosperity. These activities are aligned with the Agency's Change Agenda.[ 11 ]
Border services are integral to Canada's security and to the trade that underpins so much of our economy.[ 12 ] In May 2011, the President of the CBSA reinforced this message in the statement that ‘Frontline service delivery is what the CBSA does'.[ 13 ] A sufficient number of well-trained and effective BSOs are the foundation of CBSA's frontline service.
The People Management Pillar of the CBSA Change Agenda emphasizes the importance of recruiting the best, training them, developing and nurturing their talent and thus enhancing the leadership capacity within the CBSA.[ 14 ] BSO recruitment and training delivered through POERT are clearly aligned with these objectives. The POERT program focuses on service delivery and facilitation and is also aligned with and supports the goal of the Front Line Service Delivery pillar which is to entrench a culture of service excellence in the Agency.
National recruitment processes have been able to supply the recruits needed to meet regional requirements for new BSOs. However, staffing bilingual positions and positions in remote areas remains a challenge.
There were two national recruitment processes run in 2007 and 2008. These processes were, in part, in response to an increased demand for BSOs as a result of initiatives such as Doubling-up and Arming. The 2007-2008 recruitment process resulted in the creation of a national pool of partially assessed POERT-ready candidates. The pool enabled the Agency to respond to sudden increased demands for BSOs with a “just in time” staffing[ 15 ] approach. This recruitment approach worked well when there was an increased need to train new BSOs during the years 2007-2009. As a result of the above and some earlier recruitment processes, the Agency received over 56,000 applications for employment.
Most of the Agency's recruitment needs were met through these national processes and at the time of the evaluation, over 900 candidates remained in the partially assessed pool created as a result of these national processes. However, there are ongoing and persistent shortages of bilingual recruits and recruits prepared to work in remote POEs. Some regional managers interviewed noted that it was a challenge to attract and retain new recruits in some remote POEs because even the BSOs that accepted positions in remote POEs often applied for a transfer or deployment soon after being appointed. As a result, targeted recruitment processes were required in 2009 and again in 2011 to meet these demands.
Recruitment processes have been successful in attracting candidates who want to make a career at the CBSA.
The majority of BSOs interviewed during regional site visits indicated that their intent was to remain with the CBSA. Several BSOs said that they would like to gain additional regional experience and work in different parts of the organization to develop their careers and to progress within the Agency.
Furthermore, when participants in the BSO survey were asked where they saw themselves in five years, 84% of the respondents indicated that they saw themselves working for the CBSA. Although some managers expressed concern that the BSO job was seen by some recruits as a stepping stone to other law enforcement careers, only 6% of the survey respondents indicated that they saw themselves working in law enforcement outside the CBSA within the next five years.
BSOs surveyed were asked for the most important factor that influenced their decision to join and stay at the CBSA. Survey results indicate that job security was the top reason for joining and wanting to stay at the CBSA. While the law enforcement aspect of the BSO job attracted 11% of survey respondents, only 5% said that it would be a factor in their decision to stay (Exhibit 6).
Reasons for joining the CBSA | Reasons for staying at the CBSA |
---|---|
33% Job security | 33% Job security |
13% Good work/life balance | 11% Good work/life balance |
17% Good match between the responsibilities of a BSO and my skills & interests | 11% Good match between the responsibilities of a BSO and my skills & interests |
11% Law enforcement aspect of the job | 5% Law enforcement aspect of the job |
Source: BSO online survey
The BSO position is classified by the CBSA as a high-integrity position.[ 16 ] An integrity questionnaire for BSO applicants is in process of being developed as a strategy for risk mitigation.
Regional managers interviewed were unanimous in stressing the importance of BSO honesty and integrity. Although rare, they cited instances where BSO integrity had potentially been compromised through association with known criminals or where a BSO had engaged in unlawful activities. The risk of this occurring is partially mitigated through security checks conducted during the screening process that reveal known criminal activity. As well, the monitoring of recruits by instructors during Online POERT and Core POERT has resulted in the release of a small number of recruits due to inappropriate behaviour.
The Security and Professional Standard Directorate in the Comptrollership Branch has developed an integrity questionnaire for BSO applicants in order to further mitigate the risk. Similar to the approach used by the RCMP as part of its recruit screening process, the integrity questionnaire will include questions related to past and current behaviour that may not be consistent with CBSA values and ethics, and with required BSO integrity. The CBSA is currently in the process of seeking Treasury Board Secretariat approval for the questionnaire and a Privacy Impact Assessment will be completed.
The lack of a defined knowledge or job performance standard for BSOs on completion of POERT has resulted in differing expectations and understanding across the Agency with respect to the new BSOs' job performance abilities.[ 17 ]
While there are clearly articulated training objectives for Core and In-service POERT, there is no defined knowledge or job performance standard[ 18 ] which details the competencies, skills, knowledge and abilities that a new BSO will have on completion of both Core and In-service POERT. As a result, there is a lack of understanding about what jobs and tasks within a POE a BSO is capable of performing on completion of Core POERT versus In-service POERT. In addition, the evaluation found that there is a lack of consensus on what the job performance abilities of a BSO should be on completion of Core POERT. This results in challenges in several areas.
The Training and Learning Directorate within the HR Branch is regularly called upon to respond to requests from program areas to include additional content in the Core POERT curriculum. Without a knowledge or job performance standard it is difficult to fully assess whether or not the request directly impacts the job performance requirements of a new BSO and should, therefore, be included in the curriculum as opposed to the development and dissemination of a job aid, for example. A curriculum change should occur in response to a change in the job performance requirement. If there is a lack of consensus on what this job performance requirement is, it is difficult to ensure that the training provided is actually meeting said job performance requirement.[ 19 ]
Furthermore, the lack of such a standard also serves to underscore the importance of In-service POERT in preparing new BSOs for the range of jobs and tasks to be performed by BSOs at the POE. Core POERT provides common training, whereas In-service POERT provides the new BSO with the opportunity to gain the job performance abilities needed to be operationally effective at a POE within a given mode or modes.
Finally, chiefs and superintendents interviewed stated that they expected new Core POERT graduates to be better prepared for work at their designated POE. In addition, this same group observed that new BSOs are often frustrated when they find out that Core POERT has not fully prepared them for work at the POE. However, until a BSO completes the In-service component of POERT, their job performance abilities are somewhat limited. For example, while recruits on Core POERT are prepared to work at the Primary Inspection Line (PIL), they have no experience in the use of the Integrated Primary Inspection Line (IPIL).[ 20 ] In the current design of Core POERT, experience in and knowledge of IPIL is expected to be gained through mode specific In-service POERT.
The GTA region has taken steps to help new BSOs at the POE by developing a Web-based help guide that provides links, forms, step-by-step procedures and news available to BSOs all the time. This help guide provides guidance to new officers on applying penalties and taxes, making seizures and steps to follow after an enforcement action. The region is presently piloting an immigration help guide and is in the process of designing a similar guide for customs related activities.
While survey results indicate that Core POERT has been reasonably successful at preparing BSOs for POE duties, the evaluation found several areas for improvement.
A majority of BSOs surveyed reported that Core POERT was effective in preparing them to make release and refer decisions, in recognizing indicators and conducting inspections at PIL. However, respondents indicated that Core POERT was less useful in training them to be able to exercise duties related to the collection of taxes and duties and use of the IPIL. (Exhibit 7)
Exhibit 7: Effectiveness of Core POERT
Use of IPIL | 15% |
---|---|
Collect Taxes and Duties | 47% |
Make Timely Decisions | 66% |
Conduct Inspections | 69% |
Recognize Indicators | 75% |
Making Release & Refer Decisions | 76% |
Source: BSO Online Survey
Regional and program managers across the Agency also indicated that poor note taking, inadequate interviewing skills and poor evidence handling by BSOs have resulted in the CBSA being unsuccessful in admissibility hearings and criminal investigations as the evidence provided was not able to withstand legal challenges as the cases moved forward.
While over 82% of the BSOs surveyed indicated that they received In-service POERT in their primary mode, the evaluation found that In-service POERT is not being delivered in the manner prescribed in the curriculum learning paths.
The POERT curriculum[ 21 ] states that the In-service component of POERT is mandatory and must be completed within 12 months of a recruit's arrival at the POE, a time frame that corresponds to the new officer's probationary period of employment. However, about 17% stated that they had not received this training within 12 months of graduating Core POERT. BSOs are not considered to have finished POERT until the In-service component is completed. Therefore, those BSOs that did not receive In-service training within the prescribed 12 months had completed their employment probation period prior to completing the POERT program.
Nearly half (43.5%) of BSOs surveyed indicated that they were either dissatisfied or very dissatisfied[ 22 ] with the In-service training they received in the region. A significant number did not receive the number of In-service classroom training days specified in the POERT curriculum learning path. (Exhibit 8) It was not possible for this evaluation to determine the full extent of this issue or the consequence of not providing new BSOs with the full In-service POERT.
Air Mode In-service Learning Paths – also includes an additional 7.5 hrs online modules | Air Mode Survey Responses Number of respondents in air mode: 42 |
---|---|
In-Class Days: 12 days in class | 45% received less than 8 days of in-class training |
On–the- job learning: 5-15 days | 21% received no on-the-job training 52% received less than 8 days of on-the-job training |
Land Mode In-service Learning Paths – also includes 7 hrs of online modules | Land Mode Survey Responses Number of respondents in Land (Highway) mode: 193 |
In-Class Days: 11 days in class | 6% received no in-class training 44% received less than 8 days of in-class training |
On-the-job learning: 5-15 days | 28% received no on-the-job training 51% received less than 8 days of on-the-job training |
Marine Mode In-service Learning Paths – also includes 5 hrs of online modules | Marine Mode Survey Responses Number of respondents in Marine mode: 20 |
In-Class Days: 11 days in class | 10% received no in-class training 40% received less than 8 days of in-class training |
On- the- job learning: 5-15 days | 5% received no on-the-job training 95% received less than 8 days of on-the-job training |
Source: BSO online survey
Regional management interviewed by the evaluation team stated that the smaller regions often found it difficult to organize the In-service training due to a lack of available trainers. Also, regions mentioned that it was not cost effective to organize In-service training for a small number of recruits. Two regions, the Prairies and the Pacific region have combined the In-service training delivery and this practice has worked well for both the regions involved.
No detailed or regular training evaluation[ 25 ] has been conducted to determine whether or not the training provided throughout the whole of POERT meets the job performance requirement at the POE.
In response to the 2007 Report of the Auditor General of Canada,[ 26 ] the Evaluation and Monitoring Division within the HR Branch of the CBSA conducted a limited level 1 and level 2 training evaluation of POERT in 2010-2011 and is scheduled to conduct a limited level 3 training evaluation of POERT in November 2011.[ 27 ] While the HR Branch proposes the conduct of these as a means to assess whether or not employees have the appropriate knowledge and skills to perform their jobs,[ 28 ] there are no plans in place to conduct further level 3 training evaluations on a regular basis beyond November 2011. It is level 3 training evaluation that determines whether what is learned is being applied on the job and whether that application of learning is meeting the job performance requirements of the organization. Consequently, it will be difficult to assess the alignment of POERT, or any program that replaces POERT, with the job performance requirements for a new BSO at the POE.
There is effective monitoring and assessment of trainee performance and oversight to ensure that standards are met during Online and Core POERT. However, the evaluation found that there is no standardized monitoring or assessment of either a recruit's progress through In-service POERT or of the performance of the instructors who deliver the In-service training.
Instructors who deliver training during Core POERT are routinely and regularly monitored. While instructors who deliver In-service POERT are supposed to receive a Train-the-Trainer course, there is no monitoring at the HQ level to ensure that this is done consistently, nor is instructor performance monitored during the delivery of In-service POERT.
Online POERT participants are monitored throughout this portion of their training to ensure the completion of modules and assignments within four weeks. Candidates are measured and assessed against established criteria. Candidates that do not complete assignments or who exhibit behaviour that is not consistent with CBSA values may be removed from POERT at this stage.
Recruits attending Core POERT are assessed against established competencies through open and closed book exams and at two determination point examinations (D1 & D2) that use simulation scenarios. Three independent trained assessors use standardized assessment criteria to assess recruit performance. Each of the three assessors assesses candidates against core competencies and decisions to pass or fail candidates are made jointly.
While regions indicated that they have some form of tracking in place to record whether In-service training has been completed, there is no standardized monitoring and reporting throughout In-service POERT and consequently there is no national data available to indicate how many BSOs did not complete any In-service training.[ 29 ] As a result, the Agency does not have an accurate picture of the In-service POERT training activity and the degree to which POERT has been fully implemented. Nor does the Agency have the assurance that all new BSOs have in fact completed all required recruit training.
The requirement to complete In-service POERT within the 12-month post-Core POERT period coincides with the probationary period of the new BSO. However, there is no formal assessment required of the new BSO during or at the completion of In-service POERT against a standard set of competencies. [ 30 ] Of the BSOs surveyed for this evaluation, 52.5% indicated that their performance had not been assessed against identified goals and objectives. Furthermore, nearly half (49.8%) of BSOs surveyed indicated that they had not discussed their In-service training and the results they were expected to achieve with their immediate supervisor. The evaluation observed that it is possible that the probationary period is not being used as effectively as possible to ensure that those new BSOs who are poor performers are released during that probationary period. Once the probationary period has ended, it becomes much more difficult to terminate the employment of an individual for poor performance or for not meeting the job performance standard. Placing new BSOs who have completed Core POERT in an FB developmental position would recognize that they do not fully meet the requirements of the position until In-service training has been completed, and the employee has been assessed as part of the probationary requirements and fully meets the job performance standard of the position. It would also serve to underscore the importance of an In-service component of any new training program for new BSOs.
Regional managers interviewed for the evaluation expressed concern that complaints received by the travelling public reflected a lack of service and facilitation focus on the part of some BSOs. They attributed this to the focus of the 2007-2008 recruitment processes on attracting individuals with an interest in law enforcement and security.
Consultations with management and employees conducted to support the CBSA Branding Initiative found that there were “competing perceptions and images of the Agency which is causing confusion internally and externally. Employees and stakeholders primarily see the Agency as being enforcement-focussed. This sentiment has been solidified with the arming of border services officers. At the same time, senior managers perceive the Agency as a ‘service' organization as reflected in the Agency's name”[ 31 ].
Survey results indicate that a law enforcement orientation is reflected somewhat in the profile of BSOs recently recruited. Approximately, 18% of respondents had a law enforcement, security or military background prior to becoming a BSO. When asked what had attracted them to the job of a BSO, about 25% of survey respondents pointed to the law enforcement aspect of the BSO job as amongst the top three reasons for applying.
Steps taken by the Agency to address concerns over the need to balance facilitation and enforcement include Core POERT content that stresses the importance of both aspects of BSO duties. In addition, while the 2007 and 2008 national recruitment posters specifically targeted individuals with a law enforcement background by using references to ‘enforcement orientation'[ 32 ], this wording was modified in the 2010-2011 posters to include ‘effective service delivery' as an experience requirement.
The Agency receives complaints related to BSO conduct. While important, they represent only a very small portion of BSO interactions with travellers.
Data gathered through the Enhanced Complaints Mechanism[ 33 ] for the period January to March 2011 reveal that the Agency received 484 written complaints, of which over half (257) related to employee conduct. However, over this time period, BSOs processed over 19 million travellers and proportionate to the volume of travellers, the percentage of complaints is low at 0.0025%.[ 34 ] Ministerial correspondences reviewed[ 35 ] indicated that approximately 27% (250) of complaints about the CBSA received by the Minister over the period January 2009 to January 2011 were related to BSO rudeness and poor customer service.
In 2009-2010, the Niagara–Fort Erie region conducted an analysis of complaints received from the public. The analysis revealed that alleged negative officer conduct was the most frequent complaint from the travelling public and many complaints were related to an enforcement action. The region received a total of 101 complaints in 2009-2010 of which 71 were related to officer conduct and approximately 63% of those complaints were enforcement related complaints.[ 36 ]
Overall, recruitment activities, documents and training are provided to recruits in both official languages.
Virtually all survey respondents (99%) indicated that they had received recruitment materials and were interviewed in their official language of choice. Core POERT instruction is provided in both official languages and 96% of survey respondents indicated that training materials at Rigaud were also available in their official languages of choice.
Senior managers in the regions shared in interviews that although French instruction classes are available at Rigaud, on various occasions these classes have been cancelled due to insufficient demand. As a result, Francophone bilingual recruits have sometimes opted to attend the English instruction classes.
In fiscal 2008-2009, the Agency spent about $5 million on BSO recruitment.
Annual expenditure data specific to the BSO recruitment process are not available. As a result, the evaluation could not determine whether recruitment costs per application processed have increased or decreased over recent years.
The Agency has saved recruitment time and resources by establishing a large, partially qualified pool of candidates from which regions can draw new recruits.
The recruitment assessment process is complex and involves many steps. As a result, it takes a considerable amount of time from initial application to the point that a recruit has met all conditions necessary to be accepted into POERT. Drawing from an existing pool eliminates the need to run a competitive process each time new recruits are required. A competitive process typically requires over nine months to complete.[ 37 ]
The CBSA received over 56,000 applicants in response to the 2007-2008 national recruitment processes. The application assessment process included an initial screening, Border Services Officer Testing (BSOT), interviews and psychological testing[ 38 ] amongst other pre-requisites[ 39 ]. Following the assessment process, 1,191 applicants (approximately 2.2% of the total applicants) were selected to attend POERT. By contrast, the RCMP 2009-2010 recruitment process resulted in 6.2% of total applicants attending the RCMP cadet training program[ 40 ] and the 2009 Department of Fisheries and Oceans (DFO) recruitment process[ 41 ] resulted in approximately 10% attending the DFO officer training program.
The ratio of applications per seat available at POERT is high. In the future, the introduction of an integrity questionnaire that applicants could use as a self-screening tool may result in a decrease in unsuitable applications received and the failure rate at points along the process.
Recruitment costs are born by the regions, with some regions incurring recruitment costs that are disproportionate to their actual recruitment needs.
Regions are allocated applications to process based on applicant home address. Managers interviewed at HQ and in some regions stated that this can result in some regions processing more applications than their regional recruitment needs. For example, for the 2007-2008 national recruitment process the Northern Ontario region processed 16% of the national applications. (Exhibit 9) Since regions are required to bear the cost of processing applications themselves, the Northern Ontario region incurred approximately 11% of the total regional recruitment costs and only required about 5% of the national POERT BSO seats. In contrast, the Prairie region incurred 7% of the total regional recruitment costs and used over 14% of the national BSO seats (Exhibit 10).
Region | # of Applications Processed by Region | % of Applications |
---|---|---|
SOR – Niagara–Ft. Erie &[ 42 ] Windsor–St. Clair | 6,707 | 11.9 |
NOR – Northern Ontario Region | 8,934 | 15.9 |
QUE – Quebec Region | 11,571 | 20.6 |
PRA – Prairie Region | 5,727 | 10.2 |
PAC – Pacific Region | 8,948 | 15.9 |
GTA – Greater Toronto Region | 8,733 | 15.5 |
ATL – Atlantic Region | 5,672 | 10.0 |
Total | 56,292 | 100 |
Source: CBSA HR Branch
Region | Recruitment costs per region $ |
% of National Regional Recruitment Cost | Regional POERT Seat Allocation (2008-2009) | % of National BSO Seats Used Regionally |
---|---|---|---|---|
SOR - Southern Ontario | 480,000 | 12.5 | 181 | 15.2 |
NOR – Northern Ontario | 430,000 | 11.2 | 64 | 5.4 |
QUE – Quebec | 670,000 | 17.4 | 216 | 18.1 |
PRA – Prairie | 280,000 | 7.3 | 174 | 14.6 |
PAC – Pacific | 800,000 | 21 | 207 | 17.4 |
GTA – Greater Toronto | 820,000 | 21.3 | 238 | 20 |
ATL – Atlantic Region | 360,000 | 9.4 | 111 | 9.3 |
Total | 3,840,000 | 100.0 | 1,191 | 100.0 |
Source: HQ Client Services, Human Resources Branch & the CBSA Learning Centre
The cost per recruit that has successfully completed Online and Core POERT has increased significantly, mainly due to declining intake levels.
Overall, approximately $56M[ 43 ] has been spent on Online and Core POERT during the five-year period from 2006-2007 to 2010-2011. While total expenditures per fiscal year have fluctuated significantly on a year-to-year basis, these changes have not matched changes in the number of recruits being trained.
As illustrated in Exhibit 11, the number of POERT recruits dropped by 44% between 2006-2007 and 2010-2011 while expenditures increased by 13%. As a result, the cost of Online and Core POERT per recruit has increased substantially, rising from $14,100 in 2006-2007 to $28,290 in 2010-2011. The cost per successful Core POERT graduate increased from $15,900 to $32,100 over the same period.[ 44 ]
The CBSA Learning Centre incurs fixed costs in the form of administrative, program and facility costs and to maintain a significant number of indeterminate and determinate training staff at the main campus.[ 45 ]
2006-2007 | 2007-2008 | 2008-2009 | 2009-2010 | 2010-2011 | |
---|---|---|---|---|---|
Salaries[ 46 ] $ | 3,963,719 | 5,568,035 | 7,620,792 | 6,545,143 | 6,368,235 |
Operating Costs[ 47 ] $ | 5,041,309 | 6,860,977 | 6,846,252 | 3,098,508 | 3,747,717 |
Total Expenditure $ | 9,005,028 | 12,429,013 | 14,467,044 | 9,643,651 | 10,155,952 |
Recruits | 639 | 1,245 | 1,191 | 351 | 359 |
Graduates[ 48 ] | 566 | 1,037 | 923 | 304 | 316 |
Cost per Recruit[ 49 ] $ | 14,092 | 9,983 | 12,147 | 27,475 | 28,290 |
Cost per Graduate $ | 15,910 | 11,986 | 15,674 | 31,723 | 32,139 |
Source: Comptrollership Branch, CBSA Learning Centre
As illustrated in Exhibit 12, over the period of 2006 to 2011, the total expenditures per fiscal year have fluctuated in response to changes in the number of recruits. However, changes in expenditures have not matched the magnitude of change in number of recruits.
Exhibit 12: Change in POERT Expenditures and Number of Recruits and Graduates[ 50 ]
2006-2007 – 2010-2011
2006/2007 | 2007/2008 | 2008/2009 | 2009/2010 | 2010/2011 | |
---|---|---|---|---|---|
POERT Expenditures | 100 | 138.0230356 | 160.6551806 | 107.0918491 | 112.7809042 |
Recruits | 100 | 194.8356808 | 186.3849765 | 54.92957746 | 56.18153365 |
Graduates | 100 | 183.2155477 | 163.0742049 | 53.71024735 | 55.83038869 |
Source: Comptrollership Branch & the CBSA Learning Centre
While recent declines in the number of POERT participants have driven costs per recruit up, it is also significant to note that the CBSA Learning Centre was successful in accommodating a surge in demand between 2006-2007 and 2007-2008. Over this period, the number of POERT participants doubled, while the increase in expenditures was held at 38%.
The Agency incurs costs due to overestimation of POERT seat requirements by regions. Cancellations result in an increased cost per recruit.
The CBSA Learning Centre plans for and incurs fixed costs based on the annual forecasted number of recruits as provided by the regions. It is not always possible to reduce costs in response to last-minute cancellations. For example, in 2010-2011 the CBSA Learning Centre planned to train 432 recruits, but due to cancellations only trained 359 recruits. As a result, the forecast cost per recruit that was estimated at $23,509, increased by about 20% to $28,290 (Exhibit 13).
2006-2007 | 2007-2008 | 2008-2009 | 2009-2010 | 2010-2011 | |
---|---|---|---|---|---|
Forecasted number of recruits | 639 | 1299 | 1333 | 468 | 432 |
Actual number of recruits[ 51 ] | 639 | 1245 | 1191 | 351 | 359 |
Difference (%) | 0% | 4% | 11% | 25% | 17% |
Source: CBSA Learning Centre
Regions estimate seat requirements late in the third quarter of the fiscal year, prior to receiving budgets for the next fiscal year. Regional managers interviewed indicated that seat requirements are often reduced once budget forecasts are available. In an effort to assist regions in forecasting their training needs, the Human Resources Branch provides regions with information on attrition. As well, the CBSA Learning Centre has attempted to mitigate the impact of cancellations by offering released seats to other regions. This can be a successful strategy provided there is adequate lead time, since all conditions for employment, including security screening, must be completed before a recruit can attend Core POERT.
The proportion of recruits that are successful in completing Core POERT has increased resulting in a more efficient use of resources invested by the CBSA.
The Core POERT pass rate has increased from the 2007-2008 level resulting in a significant decrease in the unrealized investment by the Agency. The Core POERT pass rate in 2010-2011 was 88% [ 52 ]. Based on the cost per recruit for that year of $28,290, this translates into an unrealized investment of $462,070.[ 53 ] While still significant, these costs would have been higher had the pass rate remained at the 2007-2008 level of 83%. This lower pass rate translates into an unrealized investment of approximately $716,680 [ 54 ] (Exhibit 14).
Exhibit 14: Core POERT Pass Rates
2006/2007 | 2007/2008 | 2008/2009 | 2009/2010 | 2010/2011 |
---|---|---|---|---|
86 | 83 | 77 | 87 | 88 |
Source: The CBSA Learning Centre
In 2009, the CBSA Learning Centre piloted an online learning tool in an effort to improve recruit success and reduce costs associated with failure rates. Performance statistics gatherered after the pilot of the Web-based learning tool ‘Second Life' indicate that 7.7% of recruits that had used the ‘Second Life' failed the D1 exam, compared to 10.9% of those that had not used the tool. At the D2 determination point, 6.9% of recruits that had used the learning tool failed compared to a 13.3% failure rate for those that had not used ‘Second Life'.[ 55 ]
Regional senior management stated that it may be desirable to offer some recruits a second chance when they failed the D1 or D2 exams based on performance assessments prior to the determination point. A number of BSOs interviewed indicated that, based on their observations, good recruits sometimes fail at the D1 and D2 points. Both regional senior managers and BSOs that were interviewed said that the CBSA should adopt a system similar to the RCMP that allows recruits up to three attempts to successfully complete exams.
The CBSA POERT program is similar to other government department training programs in terms of cost per recruit and duration of training.
The duration of recruit training provided by other government department and agencies are comparable to POERT, with the exception of the RCMP where the recruit training program takes 24 weeks (Exhibit 15).
CBSA | Fisheries & Oceans Canada | Parks Canada Agency | RCMP | ||
---|---|---|---|---|---|
Approximate cost per recruit[ 56 ] | $34,600[ 57 ] | $28,800 | $37,900 | $50,000 | |
Total duration of training | 17-20 Weeks | 17 Weeks | 12 Weeks | 24 Weeks | |
Recruit Allowance/ Salary | $125/week (Core POERT) | $460/week for 8 weeks | $530/week for 9 weeks | $1,225/week | $500/week |
Source: The CBSA Learning Centre, Fisheries & Oceans, Parks Canada Agency & RCMP, 2010-2011 estimates.
Notable differences in similar programs across agencies and departments include facilities and the approach to recruit remuneration. While the CBSA has its own Learning Centre, the Department of Fisheries and Oceans and Parks Canada contract with the RCMP for a portion of their training. Both the CBSA and Department of Fisheries and Oceans provide recruits with a training allowance, while Parks Canada offers a conditional employment contract to new recruits who are paid a salary during their training.
Some managers interviewed for the evaluation expressed concern that the relatively small amount of the CBSA stipend would dissuade potentially good candidates from applying for a position with the Agency. Examples cited included individuals with families and financial obligations, and those who would have to leave existing employment. While these factors may influence the decision to apply for a BSO position in some instances, based on the large number of applications received in response to national recruitment processes, they do not appear to be significantly restricting the CBSA's access to sufficient numbers of candidates. As well, the BSO online survey revealed that 75% of survey respondents were employed immediately prior to attending Core POERT.
The structure and length of POERT is comparable to border officer training programs offered by the U.S. Customs and Border Protection (CBP) and the New Zealand (N.Z.) Customs Service.
As illustrated in Exhibit 16, U.S. CBP Border Protection Officer training takes from 21 to 23 weeks. Although it does not include an immigration component, N.Z. customs officers receive 20 weeks of training. While all three agencies use a combination of classroom and on-the-job training, the N.Z. Customs Service places the greatest emphasis on the latter, which accounts for 75% of the allocated training time.
In comparing these data it should be noted that U.S. officer's training includes the use of firearms. Duty firearm training is not a component of POERT at this time and CBSA officers that are required to carry a duty firearm must take an additional two weeks of training.
CBSA - Border Services Officer | U.S. CBP - Customs and Border Protection Officer | N.Z. Customs Service - Customs Officer |
|
Orientation | 4 weeks online | 1 month training at duty station | - |
Classroom training | 10 weeks (Core POERT) 2 weeks In-service |
17-19 weeks | 5 weeks |
On-the-job training/coaching | 1-3 weeks | Curriculum includes on-the- job and cross-training[ 58 ] | 15 weeks |
Total duration | 17-20 weeks | 21-23 weeks[ 59 ] | 20 weeks |
Approximate cost per recruit | $34,600[ 60 ] | $ 33, 200 to 37,800 US[ 61 ] | $20,334- $24,400 (CAN)[ 62 ] |
Officer training costs are comparable between the CBSA and U.S. CBP. N.Z. Customs Service costs are somewhat lower, which reflects the fact that the classroom training component is of a shorter duration and does not include an immigration component.
The CBSA's timing of an offer of employment is different to the U.S. CBP and the N.Z. Customs Service in that CBSA recruits do not receive an offer of indeterminate employment until after the successful completion of Core POERT[ 63 ]. Both the U.S. CBP and N.Z. Customs Service appoint candidates to positions in their respective organizations prior to the commencement of training. However, the offer of employment in both cases is conditional to recruits successfully completing their training. In the case of the CBSA, indeterminate employment is conditional to the successful completion of Online and Core POERT training.
There are no established training programs available through public learning institutions, such as regional colleges, that can replace POERT.
Several regional colleges across Canada offer certificate programs that relate to border management. Both Fleming College and Loyalist College offer a Customs Border Services Program. While these programs include curricula that are related to the BSO position, including the role and legislative jurisdiction of customs, immigration and Canadian food inspection[ 64 ], they do not cover the specialized and operational skills that BSOs require to function at a POE. Perhaps more importantly, they do not fulfill the role of instilling the CBSA culture and values amongst students.
As illustrated in Exhibit 17, there are also numerous institutions across Canada that offer college level programs focussed on law enforcement and that are intended to provide a foundation for a career in policing or law enforcement. These programs include criminology, sociology, the criminal code, police powers, criminal law and civil law. While these programs are intended as a foundation for a law enforcement career, police departments do not consider these programs to be a replacement for the basic officer training that is provided through various police academies and police colleges across the country.
Institution | Location | Program |
---|---|---|
Fleming College | Peterborough | Law & Security Administration – Customs Border Services Program |
Humber College | Toronto | Police Foundations |
Atlantic Police Academy & Holland College | Prince Edward Island | Firearms Training & Learning Academy Requirements |
Loyalist College | Belleville | CUSB – Customs Border Services Programs |
Algonquin College | Ottawa | Police Foundations |
Cité Collégiale | Ottawa | Techniques de services policiers |
Lethbridge College | Lethbridge | Criminal Justice - Policing |
Grant MacEwan College | Edmonton | Police Studies |
Department of Justice | Vancouver | Police Academy |
Source: Literature review
There is precedent for the use of public training institutions to supply elements of BSO training. For example, the CBSA has a Memorandum of Understanding with the Atlantic Police Academy and Holland College to deliver firearms training for CBSA officers. However, outsourcing all or part of the Core POERT curriculum would be challenging. Given the decline in CBSA recruit requirements over recent years, there may not be sufficient demand to warrant a college or colleges establishing a program targeted specifically to CBSA needs. Typical college programs have intakes of about 30-60 students per class and total intake per year varies from 30 to 120. The CBSA training requirement for 2012-2013 is forecast to be only about 160 recruits and this figure is unlikely to increase significantly over the next several years due to relatively low anticipated retirement rates for BSOs and ongoing budget constraints. The Agency would also have to consider the issue of access, a factor that could disadvantage individuals that do not reside in areas where a program is offered.
Agency management interviewed for the evaluation generally supported the idea of providing a common training curriculum to other new officers as well as BSOs.
Examples of common elements that might apply to all CBSA officer trainees include orientation to the Agency, an overview of CBSA programs and operations, values and ethics and pertinent legislation.[ 65 ] As such, an induction program would support the CBSA Branding Initiative by helping to instil an organizational culture amongst employees and breaking down program silos. In fact, the Branding Initiative report on employee consultations recommended that the Agency consider “cross-training as a means to break down barriers between Headquarters and the regions, facilitation and enforcement; and the various legacy organizations and branches.”[ 66 ] While management interviewed agreed that this approach could have value, they stressed the need for training tailored to the roles and responsibilities for each type of officer in addition to any common induction training.
The CBSA ensures that each port of entry has adequately prepared BSOs to deliver frontline service through a recruitment process and training program. Overall, POERT is addressing an ongoing need to provide recruits with the specialized knowledge and skills needed to work in the unique CBSA operational environment. BSO recruitment and POERT are consistent with and contribute to the realization of the CBSA's goal of ensuring Canada's security and prosperity and are aligned with the Agency's Change Agenda.[ 67 ]
The Agency's recruitment process has been successful in attracting candidates who want to establish a career within the CBSA. When participants in the BSO survey were asked where they see themselves in five years, 84% indicated that they saw themselves working at the CBSA. Overall, the national recruitment processes have been able to supply the recruits needed to meet regional requirements for new BSOs. However, some challenges continue to exist in terms of staffing bilingual positions and positions in remote areas.
The Agency held two national recruitment processes in 2007 and 2008. One was to staff BSO positions across Canada and the other to meet the specific needs of the Pacific Region. The CBSA conducted these processes to meet an increased demand for BSOs that in part had resulted from initiatives like Doubling-up and Arming. While most of the recruitment needs of the Agency were met by the above national processes, there was a shortage of bilingual recruits and recruits that were prepared to work in remote POEs in the Pacific and Prairie regions. To address these issues, two more targeted recruitment processes were conducted in 2009. Despite these initiatives, not all regional needs were met and two further recruitment processes were initiated in 2011. One process targeted English speaking candidates for specific locations, while the second process was to fill bilingual positions in specific locations. [ 68 ]
The lack of a defined job performance standard[ 69 ] for BSOs on completion of POERT has resulted in differing expectations and understanding across the Agency with respect to the new BSOs' job performance abilities. While there are clearly articulated training objectives for Core and In-service POERT, there is no defined job performance standard that details specific performance expectations including the knowledge, skills and duties of the job broken down by major duties and the key responsibilities that a new BSO will have on completion of both Core and In-service POERT. As a result, there is a lack of understanding about what jobs and tasks a BSO is capable of performing within a POE on completion of Core POERT versus In-service POERT.
The Training and Learning Directorate within the Human Resources (HR) Branch is regularly called upon to respond to requests from program areas to include additional content in the Core POERT curriculum. Without a job performance standard it is difficult to fully assess whether or not the request directly impacts the job performance requirements of a new BSO and should, therefore, be included in the curriculum as opposed to the development and dissemination of a job aid, for example. If there is a lack of consensus on what this job performance requirement is, and if the training in its entirety is not assessed on a regular basis, it is difficult to ensure that the training provided is actually meeting said job performance requirement. The lack of such a standard also serves to underscore the importance of In-service POERT. It is In-service POERT that provides the new BSO with the on-the-job experience, combined with formal classroom instruction, needed to be operationally effective at a POE.
The POERT curriculum states that the In-service component of POERT is mandatory and must be completed within 12 months of a recruits' arrival at the POE. However, nearly 17% of new BSOs surveyed did not complete In-service POERT within the 12-month period. Furthermore, a significant number did not receive the number of In-service classroom training days specified in the POERT curriculum learning path. In addition, nearly half (43.5%) of that group indicated that they were either dissatisfied or very dissatisfied with the In-service training they received in the regions.
While regions indicated that they have some form of tracking in place to record whether In-service training has been completed, there is no standardized monitoring and reporting throughout In-service POERT and consequently there is no national data available to indicate how many BSOs did not complete any In-service training.[ 70 ] As a result, the Agency does not have an accurate picture of the In-service POERT training activity and the degree to which POERT has been fully implemented. Nor does the Agency have the assurance that all new BSOs have in fact completed all required recruit training.
In response to the 2007 Report of the Auditor General of Canada[ 71 ], the Evaluation and Monitoring Division within the HR Branch conducted a limited level 1 and level 2 training evaluation of POERT in 2010-2011 and is scheduled to conduct a limited level 3 training evaluation of POERT in November 2011.[ 72 ] While the HR Branch proposes the conduct of these as a means to assess whether or not employees have the knowledge and skills to perform their jobs,[ 73 ] there are no plans in place to conduct further level 3 training evaluations on a regular basis beyond November 2011. It is level 3 training evaluation that determines whether what is learned is being applied on the job and whether that application of learning is meeting the job performance requirements of the organization. Consequently, it will be difficult to assess the alignment of POERT, or any program that replaces POERT, with the job performance requirements for a new BSO.
In light of these findings, the evaluation recommends:
The HR Branch work with Operations Branch, in consultation with Programs Branch, to develop and implement a job performance standard that details the POE job performance requirement for BSO recruits on completion of POERT.
Management Response: The HR Branch concurs with this recommendation. The Training and Learning Directorate (TLD) of the HR Branch will work with the Operations Branch and the Programs Branch to develop a job performance standard for BSO recruits that is based on defined competencies. The new job performance standard will be applied through a 12-month competency-based probationary period wherein new BSO recruits will be assessed and measured against the BSO competencies as part of their probationary period. The design of the new Officer Induction Training Program will be based on the job performance standard and will produce job-ready, multi-mode armed officers. |
|
Key Milestones | Expected completion: |
---|---|
In consultation with the Operations and Programs Branches, develop a job performance standard based on the suite of competencies (behavioural and technical) and knowledge standards required/expected during the recruits' 12-month probationary period. These competencies and job performance standards will also form the foundation for BSO recruitment activities and the design of the new Officer Induction Training Program. | May 2012 |
Align the job performance standard/expectations of the BSO recruits with the CBSA's Performance Management Process. Superintendents and chiefs will be required to assess recruits against these standards. | June 2012 |
The HR Branch develop and implement a monitoring and training evaluation strategy for POERT, or any program that replaces POERT, to ensure that the delivery of such a program is consistent and compliant with the curriculum requirements and that the training delivered matches the job performance standard of BSO recruits.
Management Response: The HR Branch's Training and Learning Directorate concurs with this recommendation. The TLD of the HR Branch has developed a Learning Evaluation Framework that outlines the methodology for evaluating training and learning products and programs. This methodology is currently in use and will continue to be used to evaluate the current POERT program and the new Officer Induction Training Program. A Learning Monitoring Framework will also be developed to guide the monitoring of the Officer Induction Training Program against the newly defined job performance standard. A monitoring and evaluation strategy will be developed and implemented for the new Officer Induction Training Program that will ensure that the training curriculum is consistently delivered, addresses all curriculum requirements, and is appropriately aligned with the new job performance standards. |
|
Key Milestones | Expected completion: |
---|---|
Finalize the development of an electronic questionnaire to gather participant and instructor reaction (Level 1 evaluation) specifically for the recruit training program. The data from this questionnaire will be gathered from the current version of POERT and throughout the pilot session of the new Officer Induction Training Program in order to make changes/improvements to the new training program. | January 31, 2012 |
Finalize the development of the action plans to respond to the findings of the Core-POERT Level 3 Evaluation Report (evaluation of the transfer of learning) and align them with the recommendations of this POERT Program Evaluation. | January 31, 2012 |
Implement the review and updates that are required to current POERT lessons and subsequent Officer Induction Training Program lessons. | February 2012 – November 2012 |
Finalize the development of a Learning Monitoring Framework, in consultation with key stakeholders, to guide the development and implementation of a monitoring strategy for the new Officer Induction Training Program. | March 2012 |
Develop and implement a monitoring strategy and a learning evaluation strategy for the new Officer Induction Training Program to ensure that the training curriculum is consistently delivered and is appropriately aligned with the new job performance standards (see recommendation #1). | July 2012 |
Use approved evaluation and monitoring methodologies during the pilot of the new Officer Induction Training Program. | September 2012 |
The requirement to complete In-service POERT within the 12-month post-Core POERT period coincides with the probationary period of the new BSO. However, there is no assessment against a standard set of competencies required of the new BSO during or at the completion of In-service POERT. Of the BSOs surveyed for this evaluation, 52.5% indicated that their performance had not been assessed against identified goals and objectives. Furthermore, nearly half (49.8%) of BSOs surveyed indicated that they had not discussed their In-service training and the results they were expected to achieve with their immediate supervisor. The evaluation observed that it is possible that the probationary period is not being used as effectively as possible to ensure that those new BSOs who are poor performers are released during that probationary period. Once the probationary period has ended, it becomes much more difficult to terminate the employment of an individual for poor performance or for not meeting the job performance standard. Placing new BSOs who have completed Core POERT in an FB developmental position would recognize that they do not fully meet the requirements of the position until In-service training has been completed, and the employee has been assessed as part of the probationary requirements and fully meets the job performance standard of the position. It would also serve to underscore the importance of an In-service component of any new training program for new BSOs.
In light of these findings, the evaluation recommends:
The HR Branch develop an assessment tool that is used to assess all BSO recruits' performance prior to the end of the 12-month probationary period to ensure that they complete the required training and meet identified performance objectives.
Management Response: The HR Branch concurs with this recommendation. The TLD of the HR Branch will work with the Operations Branch and the Programs Branch to develop an assessment tool to effectively measure BSO recruits' performance against the job performance standard identified in recommendation 1. This tool will be mode-specific and applied during the 12-month probationary period and as part of a field coaching program. A monitoring mechanism will also be put in place to ensure that the assessment tool is being applied consistently across regions. |
|
Key Milestones | Expected completion: |
---|---|
Develop an integrated, mode-specific assessment tool for the new Officer Induction Training Program that will include a suite of assessments from recruitment to the end of the 12-month probation period. It will include written exams with multiple choice, assignments/projects, simulations, feedback from peers as well as feedback from instructors and coaches. | March 2012 |
Develop a field coaching program that will support both the recruit and the supervisor during the probation period and will provide a thorough set of assessments that will measure the capacity of recruits to apply knowledge, skills and attitudes in their daily job performance.
|
November 2012 |
The evaluation noted that the cost per POERT graduate has increased significantly, due mainly to declining intake levels. Between 2006-2007 and 2010-2011, the number of POERT recruits dropped by 44% while expenditures increased by 13% during the same time period. As a result, the cost of Online and Core POERT per recruit has risen from $14,100 in 2006-2007 to $28,290 in 2010-2011. The cost per successful Core POERT graduate[ 74 ] increased from $15,900 to $32,100 over the same period. While recent declines in the number of POERT participants have driven costs up per recruit, the proportion of recruits that are successful in completing Core POERT has increased, resulting in a more efficient use of resources invested by the CBSA. It is also significant to note that the CBSA Learning Centre was successful in accommodating a surge in demand between 2006-2007 and 2007-2008. Over this period the number of POERT participants doubled while the increase in expenditures was held to 38%.
The evaluation found that the Agency incurs costs due to overestimation of POERT seat requirements by regions. Regions estimate seat requirements late in the third quarter of the fiscal year, prior to receiving budget forecasts. Regional managers interviewed indicated that seat allocations are often reduced once budget forecasts are available. The CBSA Learning Centre plans for and incurs fixed costs based on the annual forecast number of recruits as provided by the regions. These costs are not easily reduced in response to last-minute cancellations. For example, in 2010-2011, the CBSA Learning Centre planned to train 432 recruits, but due to such cancellations, only trained 359 recruits. As a result, the forecast cost per recruit, estimated at $23,509, increased by about 20% to $28,290. The CBSA Learning Centre has attempted to mitigate the impact of cancellations by offering released seats to other regions. This can be a successful strategy provided there is adequate lead time, since all conditions must be met and security screening completed before a recruit can proceed to Core POERT. Managers interviewed indicated that lead time is particularly important with respect to the security screening requirement, since it is only conducted once a recruit has been selected to attend POERT. The declining intake levels in the foreseeable future will require precise management of the seat allocation process to ensure maximum efficiencies in training delivery and use of the CBSA Learning Centre.
In light of these findings, the evaluation recommends:
The HR Branch work with Operations Branch to develop a process for seat allocation for POERT that better aligns with the budget allocation process.
Management Response: The HR Branch concurs with this recommendation. The Training and Learning Directorate (TLD) of the HR Branch will work with the Operations Branch to develop a multi-year BSO seat allocation plan that is aligned with National and Regional Integrated Business Plans. This plan will optimize the use of training and learning human and financial resources, including training facilities. |
|
Key Milestones | Expected completion: |
---|---|
Develop a new seating allocation process that is aligned with Agency Integrated Business Plans, approved by the Operations Branch and supported by corresponding funding and that is also aligned with the new recruitment process. | February 2012 |
Pilot the new seating allocation process in fiscal year 2012-13 to test the process, identify opportunities for improvement and find efficiencies, and to assess alignment with budget and planning cycles. | April 2012 |
Abbreviation/Acronym | Definition and Description |
---|---|
BSO | Border Services Officer |
BSOT | Border Services Officer Test |
CAB | Curriculum Advisory Board |
CBP | Customs and Border Protection |
CBSA | Canada Border Services Agency |
CDT | Control of Defence Tactics |
DFO | Department of Fisheries and Oceans |
ECM | Enhanced Complaint Mechanism |
FOSS | Field Operational Support System |
HQ | Headquarters |
HR | Human Resources |
ICES | Integrated Customs Enforcement System |
IPIL | Integrated Primary Inspection Line |
NILE | National Institute of Learning and Excellence |
NZ | New Zealand |
PIL | Primary Inspection Line |
POE | Port of Entry |
POERT | Port of Entry Recruit Training |
TLD | Training and Learning Directorate |