Review of the Acquisition Process to Replace Canada's CF-18 Fleet

PDF Version (1.2Mb)
Help with alternative formats

Final report

November 14, 2013

Introduction
Major military acquisition
The joint strike fighter program
Lessons learned and recommendations
Shortcomings identified by the auditor general of canada
Conclusion
Appendix A: Stakeholders Interviewed
Appendix B: Overview of Process and Key Players
Appendix C: Policy Framework
Appendix D: Department of National Defence Organizational Structure
Appendix E: Summary of Recommendations

Introduction

Objective

The objective of this engagement was to review the steps taken in the acquisition process for the replacement of the Department of National Defence (DND)'s CF-18 fleet up to June 2012; to provide lessons learned; and to propose recommendations for improvements to the current practices and policies to increase the level of due diligence for any future acquisitions of similar nature, risk, and/or complexity.

Furthermore, reviewers assessed whether the acquisition process shortcomings identified by the Auditor General of Canada have been addressed.

Scope

The review included an examination of the steps taken by Public Works and Government Services Canada (PWGSC) and DND in the acquisition process for the replacement of Canada's CF-18 fleet up to June 2012. We also examined the project activities undertaken by DND in the planning stages of the procurement life cycle and its participation in the Joint Strike Fighter (JSF) Program.

Our review specifically excluded revisiting the work conducted and the facts established by the Auditor General up to June 2010 and any work related to the assessment of the full life cycle costs of the F-35.

Approach / Methodology

Key stakeholders in DND and PWGSC involved in the CF-18 replacement program were interviewed. In addition, interviews were conducted with officials from the Treasury Board of Canada Secretariat (TBS), the Privy Council Office (PCO), and Industry Canada (IC). A list of interviewees is presented in Appendix A.

Available documentation was reviewed to understand the processes undertaken by DND and PWGSC, Industry Canada, as well as by the relevant central agencies. We examined what policies were in place at the time throughout the process as well as current Treasury Board policies relating to Crown procurement contracting, investment planning and project management. In addition, we examined the associated DND and PWGSC policies and practices for complex, high risk military acquisitions.

We also reviewed the 2012 Spring Report of the Auditor General of Canada: Chapter 2 – Replacing Canada's Fighter Jets to identify the underlying causes that lead to the Auditor General's findings.

The culmination of the above provided the basis for the development of the lessons learned for future similar acquisitions.

Review Limitations

The review team's ability to fully assess the actions undertaken by the government during the process was hampered by specific limitations:

Cabinet Documents: Key documents, such as the 2010 Memorandum to Cabinet and relevant DND Treasury Board Submissions from 2001 and 2006, are Confidences of the Queen's Privy Council for Canada, and could not be provided. As such, reviewers were not able to examine the advice given to the applicable Ministers or the decisions emanating from Cabinet and Treasury Board deliberations.
Other documents: Documents such as DND's Strategic Capability Roadmap and a classified annex to the Statement of Operational Requirements were not provided due to their sensitive content.
Interviews: Interviews with key stakeholders involved in the process were not always possible as their employment with the federal government had ceased or they were otherwise unavailable. Individuals currently involved in the process did not necessarily have a complete knowledge or understanding of earlier activities that had been undertaken with respect to the CF-18 fleet replacement.

Through our interviews we noted that various participating stakeholders (within and among departments) in the process have differing views about the steps that were undertaken and why they were undertaken.

Our lessons learned and recommendations are based upon the information provided and individuals interviewed throughout the review process.

Major military acquisition

Building a first-class military that is well-trained, well-equipped and ready to take on the challenges of the 21^st century represents its own unique risks and opportunities.

Maintaining a military that is prepared to face uncertain future security challenges often requires the acquisition and procurement of new and technologically advanced equipment, which is a major expense.

In doing so, the Government of Canada's procurement system endeavours to balance the need for the Canadian Armed Forces to be equipped with a modern military that can carry out its missions while maximizing value for money and leveraging investments in military equipment to the benefit of the Canadian industry and workers.

Overview of the Acquisition Process

The federal government acquisition process is part of the broader asset management life cycle^{Footnote 1}, as depicted below:

The four phases of the asset management life cycle – Image description below.

Image Description

This image shows the four phases of the asset management life cycle, in the order of: Planning, Acquisition, Use and Maintenance and Disposal. The CF-18 fleet replacement activities are depicted in the planning and acquisition phases.

This section addresses both the planning and the acquisition phases of the asset management life cycle.

Planning Phase

This initial phase of the asset management life cycle involves, among other things, establishing the operational need, producing the Statement of Operational Requirements, conducting an options analysis for the entire investment options over their whole life, establishing an appropriate management control framework, and conducting an initial assessment of project risk.^{Footnote 2}

The work undertaken during this phase is the responsibility of the department identifying the need.^{Footnote 3}

Acquisition Phase

This next phase involves all of the activities associated with planning for and acquiring goods and services in a manner that “… enhances access, competition, and fairness and results in best value or, if appropriate, the optimal balance of overall benefits to the Crown and Canadians.”^{Footnote 4} While respective departments are responsible to identify their requirements, the Minister of PWGSC has the exclusive authority to buy or otherwise acquire defence supplies and construct defence projects.

The acquisition phase is comprised of four sub-phases:

Pre-contractual: The activities associated with defining a requirement and planning for its procurement.
Contract award: The activities associated with soliciting and evaluating bids and negotiating and awarding a contract.
Contract administration: The activities associated with monitoring time, cost and contractor performance and administering a contract.
Contract close-out: The activities associated with closing out the procurement, including final acceptance, final payment, and audits.

DND has defined its standard project life cycle into five phases as part of the planning and the acquisition steps:

The five phases of DND's standard project life cycle – Image description below.

Image Description

This image shows the five phases of the Department of National Defence's standard project life cycle, in the order of: Identification, Options Analysis, Definition, Implementation and Close Out.

Appendix B presents a more detailed description of the first four phases.

Policy Framework

DND, like any other department is subject to a comprehensive set of laws, regulations and policies to be referred to in the conduct of its project planning and acquisitions activities. Treasury Board has instituted a number of key policies which departments must respect in conducting their necessary activities. Treasury Board also stipulates the requirement of appropriate governance and control processes to be adopted based on various risk considerations.

History of Treasury Board Policy Suite

In 2007, Treasury Board moved to less prescriptive, more principles-based policy direction that holds deputy heads accountable for achieving specific high-level management objectives. A summary table is presented in Appendix C with the applicable Treasury Board policy instruments in effect until 2009, and the policies in effect since that time. As it relates to the project management suite of policies, TBS adopted a phased approach to implementation of the Policy on the Management of Projects, starting in 2007 with a pilot with a group of departments^{Footnote 5} and culminating in full implementation on April 1, 2012. We note that DND signed the Production, Sustainment, and Follow-on Development Memorandum of Understanding (MOU) in 2006, at which point the old suite of Treasury Board policies would still have been in effect.

Public Works and Government Services Canada Policy Framework

The Department of Public Works and Government Services Act establishes PWGSC as a common service organization and gives the Minister of PWGSC the sole statutory authority for the procurement of goods. The Defence Production Act establishes the Minister of PWGSC as the exclusive authority to buy or otherwise acquire defence supplies and construct defence projects required by DND.

The PWGSC Supply Manual clarifies PWGSC's role and responsibilities in the acquisition process. At the definition of technical requirements and procurement planning stages, PWGSC is responsible for, among other things:

Participating in the review and refinement of the Statement of Work and the development of the technical bid evaluation plan;
Assessing industrial capability;
Identifying the major contracting policies that must be considered in accomplishing the procurement; and
Developing the contracting approach.

Department of National Defence Policy Framework

Over the years, DND has developed policy guidance on the project life cycle from the identification of the need to the implementation of the procurement. Most of this guidance is described in the Project Approval Directive^{Footnote 6}. The Project Approval Directive is intended to reflect the standard practices resulting from changes to the Treasury Board project management policy suite and to reflect the Project Management Body of Knowledge^{Footnote 7}.

The Project Approval Directive assigns responsibility to the Project Sponsor for identifying the requirements and performing the option analysis.

CF-18 Fleet Replacement Acquisition Project

Canada currently operates a fleet of CF-18 Hornet fighter jets purchased in the 1980s with an original life expectancy until 2003. The fleet's life has since been extended to between 2017 and 2020 following a modernization program. National Defence has determined that a suitable replacement for the CF-18 is required, otherwise, it will lose its ability to carry out domestic and international missions as mandated by the Government of Canada.

In May 2008, through the Canada First Defence Strategy, the federal government announced its intent to replace the CF-18 fleet with 65 “next generation” fighter aircrafts. In July 2010, the federal government announced its decision to purchase the F-35 Lightning II (henceforth, the F-35), at a cost of approximately $9 billion. The purchase of the current fleet's replacement was to be done on a sole source basis, without following a competitive procurement process.

At the time of the announcement in 2010, the CF-18 replacement acquisition process had not reached the Implementation phase. The CF-18 replacement was designated as a project in January 2010 when it entered into the Options Analysis phase. In October 2010, a project management office was established within Assistant Deputy Minister (Materiel) and work was initiated to prepare a Treasury Board Submission for project approval, following the finalization of the Statement of Operational Requirements.

Governance Structure

A project is only designated as such when it reaches the Options Analysis phase.

From 1997, when the first MOU for the JSF Program was signed, up until January 2010, the project was in Identification phase. TBS was involved in JSF Program activities through their review of the Treasury Board submissions for the 2002 and 2006 MOU. PWGSC was not involved in the activities leading up to the signing of the MOUs since it was understood, within DND and TBS, that there was no commitment to purchase at that time.

During that period, Assistant Deputy Minister (Materiel) managed Canada's participation in the JSF Program.

In January 2010, the CF-18 replacement project entered the Options Analysis phase. In the policy context at that time, high risk projects such as the CF-18 replacement were required to have appropriate governance structure, project management capacity, controls, and reporting in place, as well as appropriate project approvals sought from Treasury Board. For the CF-18 replacement, the Department of National Defence Senior Review Board met in January 2010, the project went before the Major Crown Project Interdepartmental Oversight Committee for the first time in 2010 and a Senior Project Advisory Committee meeting was convened in October 2011.

Projects undertaken today would proceed similarly once entering the Options Analysis phase.

Project and Investment Planning

The 2001 CF-18 modernization program extended the estimated life expectancy of the fleet to 2017-2020. Ideally, a project to replace the modernized fleet should have been initiated in the years 2000-2002, given that projects of the magnitude of the CF-18 replacement typically take in excess of fifteen years to plan, implement and close out following the Identification phase. The CF-18 replacement project only entered the Options Analysis phase in January 2010 and it never reached the Implementation phase.

Interrelation of the Joint Strike Fighter Program and CF-18 Replacement Activities

Canadian participation in the JSF Program was an initial step taken by the DND in preparing for the eventual replacement of the CF-18. Participation in the JSF Program was intended to allow Canada to obtain industrial benefits, provide access to data about next generation fighter aircraft and cost avoidance should Canada chose to purchase the F-35.

Other than Canada's participation in the JSF Program, minimal steps were undertaken for the planning or acquisition to replace the CF-18 fleet before 2005, when an Operational Requirement and Concept Document was prepared. This work was initiated at the request of Assistant Deputy Minister (Materiel) to determine whether Canada should continue to participate in the JSF program. Following the signing of the 2006 MOU, a small project office was established within the Royal Canadian Air Force in 2007 to support the future replacement of the CF-18.

Current Investment Planning Process

The process for identification of capability deficiency has improved within DND over the recent years. The investment planning cycle is led by the Vice Chief of Defence Staff and Assistant Deputy Minister (Finance and Corporate Services). In the past, the identification of capability requirements was initiated by the various Sponsors (Air Force, Navy and Army) with little senior level strategic coordination, oversight or review. The creation of the Chief Force Development within Vice Chief of Defence Staff should enable the Canadian Armed Forces to better coordinate the needs, requirements and available funding. Furthermore, it allows better long term planning of investment through early identification of needs.

Additionally, when high risk and complex acquisition projects are identified during investment planning activities, departments are now required to provide the following information for Treasury Board approval with their resulting Investment Plan:

Procurement strategy;
Estimated costs^{Footnote 8};
Project Complexity and Risk Assessment– with relevant and appropriate governance structure; and
Related risk mitigating strategies.

Following the identification of a project, DND is expected to involve necessary central agencies and PWGSC.

Recent changes in the process for identifying capability deficiencies and the new investment planning process should improve governance and oversight, including timely PWGSC participation. However, as indicated by the Auditor General and many stakeholders, several aspects surrounding the participation in the JSF Program through the use of MOU contributed to the misalignment of steps taken related to the JSF program with steps taken in the CF-18 fleet replacement project planning process.

Acquisition Project Management Process

Identification Phase

The Capability Based Planning exercise established in 2010 by Chief of Force Development is critical in establishing the best approach - including value for money – to meet the policy direction established by the Government, and informing the development of DND's Investment Plan. This new process has yet to complete its first three-year cycle. However, based on our review, it appears to be well structured, includes a challenge role and involves key stakeholders within the Canadian Armed Forces.

The Capability Based Planning exercise is led by Chief Force Development, with the involvement and cooperation of the Canadian Armed Forces (Royal Canadian Air Force, Royal Canadian Navy and Canadian Army) and other key players such as Chief of Programme. The output of this exercise is the Force Capability Plan, which is reviewed by senior leaders of the Canadian Armed Forces and submitted to the Chief of Defence Staff for approval. The Force Capability Plan details the military's future requirements and is a new deliverable produced by Chief Force Development for the Chief of Defence Staff. The first version of this plan, which is updated every three years, will be available in 2013. The Capability Based Planning exercise includes a review by senior military personnel to ensure the requirements presented by the Canadian Armed Forces are appropriately defined and supported.

Option Analysis Phase

Chief of Programme coordinates the development of the necessary Treasury Board submissions for project approval. Chief of Programme also initiates discussion with Assistant Deputy Minister (Materiel) on how to proceed with the procurement and to determine when the leadership of the project should be transferred from the Project Sponsor (Royal Canadian Air Force, Royal Canadian Navy, or Canadian Army) to a project manager in Assistant Deputy Minister (Materiel) and when a formal Project Management Office should be established within Assistant Deputy Minister (Materiel).

A Project Management Office is typically established following Cabinet approval of a Memorandum to Cabinet and when project approval has been received from Treasury Board. While a formal Project Management Office is typically established at the Definition phase of a project, the Project Sponsor usually has established a small project office at the time work is initiated at the Identification phase. For the CF-18 replacement, this was done in 2007, following the signing of the Production, Sustainment, and Follow-on Development 2006 MOU.

The work conducted in the Identification and Options Analysis phase is led by the Project Sponsor. For the CF-18 replacement, there was limited involvement of the project management personnel working under the auspices of Assistant Deputy Minister (Materiel). Although project management expertise from Assistant Deputy Minister (Materiel) is often involved in the earlier phases of a project with the Project Sponsors, it is not always the case. The Project Office that prepared the necessary supporting documentation for the 2010 Memorandum to Cabinet was relatively small for a project this size.

The joint strike fighter program

Canada's military enters into various international collaboration arrangements through the use of Memorandum of Understanding (MOU). In most cases, these MOU allow DND to participate in collaborative research and development projects and provide opportunities for Canadian industry participation.

However, the JSF Program was conceived to be a new model for development and acquisition of military equipment and considered unique at that time.

In 1997, Canada (through DND) signed an international MOU with the United States Government, along with eight other country partners, to undertake concurrent design, development and manufacture of the F-35 Lightning II aircraft via the JSF Program. For the Government of Canada, in particular Industry Canada and DND, the purpose of participating in this joint collaboration initiative was to have access to and transfer of classified information, to participate in an international program to design, develop and manufacture the aircraft, to benefit from the waiving of U.S. Foreign Military Sales surcharge, and to gain access to industrial opportunities for Canadian companies. Canada signed a second MOU in 2002 for the system development and demonstration of the F-35, and a third, in 2006, for the production, sustainment, and follow-on development of the F-35.

The third MOU signed in 2006 introduced the terms for the production of the aircraft and contained a chapter on procurement, which defined the process for country partners to acquire the F-35, if desired. The cost of participating in the 2006 MOU was based on the number of aircraft estimated to be purchased by each country. At the time the 2006 MOU was signed, Canada estimated it would purchase 80 aircraft; this number was later amended to 65.

Assistant Deputy Minister (Materiel) managed Canada's participation in the JSF Program, until a Project Office was created in October 2010. Prior to the signing of the 2006 MOU, Assistant Deputy Minister (Materiel), (DND) requested that the Royal Canadian Air Force complete a preliminary analysis of options by reviewing five candidate aircraft that could potentially replace the CF-18.

1997, 2002 and 2006 Memoranda of Understandings

The JSF Program was led by the United States Department of Defence. Canada was invited to participate in the Program through the signing of a series of MOUs. Three MOU were entered into for the JSF Program. They are:

1997 CD MOU: The Concept Demonstration (CD) MOU was signed in December 1997. With a US$10.6 million contribution, this allowed Canada to work with allies in developing a new fighter jet as well as providing opportunities for Canadian companies to be part of the design phase of the program. During the period covered by this MOU, the United States collaborated with the United Kingdom in establishing the requirements that led to a design competition between Boeing and Lockheed Martin for the selection of the prime contractor to build the F-35. Lockheed Martin was selected as a result of that competition.
2002 SDD MOU: The System Development and Demonstration (SDD) MOU was signed in February 2002. Through this MOU, Canada committed to contribute US$150 million (US$100 million in cash and US$50 million through Industry Canada programs).
2006 PSFD MOU: The Production, Sustainment, and Follow-on Development (PSFD) MOU was signed in December 2006. Through this MOU, Canada committed US$551 million over 40 years.

While the F-35 could become an alternative for the replacement of the CF-18, and the 2006 MOU had one chapter dedicated to the procurement process with an estimated number of planes Canada would acquire, there was no commitment to purchase for the signatories to these MOUs. However, the JSF Program established the acquisition process through these MOUs if Canada was to purchase the F-35.

TBS was involved with DND prior to signing the 2006 MOU in the review of the related Treasury Board submission. At that time, the MOU was not considered part of the CF-18 replacement project. PWGSC was not involved in any of the MOUs.

Prior to signing the 2006 MOU, although no CF-18 replacement project was in place, the Royal Canadian Air Force worked on a number of Options Analysis phase related activities. Although the documentation produced was limited, some of the activities conducted related to the following:

Established operational requirements (Operational Requirement Concept Document and Statement of Operational Requirements);
Conducted Preliminary Options Analysis;
Established Preliminary Project Costs;
Performed a Risk Assessment; and
Prepared a Business Case.

In 2010, to replace the CF-18 with what was considered the only valid Next Generation Fighter Aircraft and to ensure maximum Canadian industry participation in the supply chain supporting the JSF Program, a Memorandum to Cabinet (MC) was presented to Cabinet to obtain policy approval on the sole source purchase of the F-35.

Lessons learned and recommendations

Alignment of the CF-18 Replacement Project and the Joint Strike Fighter Program

Observation

The JSF Program was conceived to be a new model for development and acquisition of military equipment and considered unique. As described previously and presented in the figure below, Canada's participation in the JSF Program was under way before a decision was made on how to replace the CF-18 fleet. This reflects one of the unique aspects of the JSF program, which is a U.S.-led multinational effort.

JSF Program

JSF Program MOUs – Image description below.

Image Description

This graph represents the years of the MOUs (1997, 2002 and 2006) for the JSF Program.

CF-18 Replacement Project

2010 CF-18 Replacement Project Options Analysis – Image description below.

Image Description

This graph represents the identification phase followed by the Options Analylsis (2010) for the CF-18 Replacement Project.

The JSF Program requirement was that Canada had to sign an MOU in 2006 to support production related activities. However, the CF-18 replacement had not yet been defined as a project within DND and was still in Identification phase, and a thorough and documented options analysis had yet to be completed. PWGSC was not involved in the replacement of the CF-18 until January 2010.

It is our opinion that the parallel tracks of the JSF Program and CF-18 replacement project were misaligned. The options analysis for the CF-18 replacement project should have been completed to provide the necessary support for signing the Production, Sustainment, and Follow-on Development MOU in 2006.

Applicability of the Policy Framework

The current policy framework is applicable to all acquisitions. In our opinion, the current policy framework is adequate for all major military acquisitions that follow DND's standard project life cycle noted earlier. It provides for the early engagement of PWGSC in the planning of the acquisition and aims at ensuring that departments put the appropriate governance and project management processes and controls in place.

However, up until 2010, the JSF Program was not considered an acquisition process; the CF-18 replacement project was only entering in Options Analysis phase. There is little policy direction covering participation in an international initiative such as the JSF Program that is not yet part of an acquisition process, but could lead to a potential acquisition. As such, we found that the current policy framework does not provide sufficient direction to DND personnel regarding the necessary steps leading to participation in such a program.

Impact

PWGSC, as contracting authority, was not engaged in the participation to the JSF Program. These steps included the signing of the three MOUs but also the supporting analysis and activities such as the Operational Requirement and Concept Document. The signing of the production MOU for participation in the JSF Program was not synchronized with the approval process for the acquisition of a replacement for the CF-18. Canada was already involved with the F-35 aircraft (through the JSF Program) prior to conducting the CF-18 replacement Options Analysis phase in January 2010.

Lesson-learned

DND engages in a range of MOUs to support the development of new capabilities, but not all such arrangements lead to acquisitions. MOUs that can lead to potential acquisitions need to align with the underlying capital related project in order that program related decisions are adequately supported by well documented analysis. Key aspects of the procurement process should be completed and key stakeholders involved before entering a production MOU.

Recommendation #1

In order for the Government to make an early and informed decision on the available options and procurement strategy, we recommend that:

The Department of National Defence develop necessary policy to ensure that the signing of production Memoranda of Understanding is done in conjunction with its project planning and acquisition cycle.
The Treasury Board Secretariat formally clarifies that Memoranda of Understanding which are contract-like should be treated as contracts and follow appropriate related processes.
Public Works and Government Services Canada always be involved in the development and signing of production related Memoranda of Understanding. Participation in the related program at the production phase should only be considered when a formal project has been designated, the Options Analysis phase has been completed and a procurement strategy has been defined.

Project Management Capability

Observation

From the mid-late 1990's to January 2010, the Royal Canadian Air Force had been working on the CF-18 replacement project with limited involvement of Assistant Deputy Minister (Materiel).

From the mid-late 1990's to January 2010, the Royal Canadian Air Force had been working on the CF-18 replacement project in what is considered the Identification phase. Following the signing of the 2006 MOU, a small project office was established within the Royal Canadian Air Force in 2007 to support the future replacement of the CF-18. The same Project Office was involved in the preparation of the necessary supporting documentation for the 2010 Memorandum to Cabinet. It was recognized that this Project Office was relatively small for a project this size.

Although project planning is the foundation of effective project management and delivery, the Project Approval Directive does not require that any specific project management capability be established until the Definition Phase, when a Project Management Office within Assistant Deputy Minister (Materiel) takes the project lead. In the case of the F-35, this office was not established until October 2010.

While project management expertise from Assistant Deputy Minister (Materiel) is often involved in the earlier phases of a project with the Project Sponsors, it is not always the case.

Impact

Where there is limited professional project management capability assigned to an acquisition project there is increased risk that supporting documentation for key decisions is inadequate, lacks precision, cannot be produced in a timely manner and fails to consider all aspects of the eventual procurement requirements.

Lesson-learned

DND would benefit from early and appropriate project management capability involvement from Assistant Deputy Minister (Materiel) with the Project Sponsor's office on a consistent basis for its high risk projects.

Recommendation #2

The Department of National Defence needs to ensure that adequate Project management capability is assigned to the Project Sponsor project office as early as the Identification phase and that it is appropriately augmented until the project reaches Definition phase.

Statement of Requirement

Observation

In accordance with the National Defence Act, and as amplified in the PWGSC Supply Manual, DND is responsible for defining operational and technical requirements. This stated, under DND policy there has been no requirement to validate Statement of Operational Requirements prior to approval. As a result, Assistant Deputy Minister (Materiel) relied on the military, the Royal Canadian Air Force in the case of the CF-18 replacement, to define its requirements, and none of the parties believed the Statement of Operational Requirements had to be validated.

The justifications for the sole source procurement strategy relied on the assertion by the Royal Canadian Air Force that the Canadian requirements were such that only the F-35 could meet them. PWGSC agreed to the sole source procurement strategy primarily on the strength of a letter from the Assistant Deputy Minister (Materiel) to the Assistant Deputy Minister, Acquisitions Branch, PWGSC, in which DND asserted that only one firm was capable of supplying a 'fifth generation' fighter aircraft.

Public Works and Government Services Canada Specific Role and Responsibilities

PWGSC is accountable to safeguard the integrity of the federal government's procurement process by ensuring that its procurement practices are fair, open and transparent. This involves numerous activities, including ensuring that clients have described their requirements in an objective manner and that these requirements have been justified. Inherent in this requirement, is the need to ensure that sole source justifications are reasonable and well supported.

Impact

Ineffective challenge or oversight over procurement strategies and the Statement of Operational Requirements may not result in obtaining the best value for money.

Lesson-learned

PWGSC should not rely exclusively on client departments in ensuring that statements of operational requirement are written in an objective manner. Consistent external independent oversight may help ensure an adequate challenge function for both the Statement of Operational Requirements and procurement method.

Recommendation #3

To ensure that Statement of Operational Requirements are objective and adequately supported for future high dollar value, complex projects, Public Works and Government Services Canada should:

Seek the opinion of independent third party expertise where it feels that the department does not have the requisite knowledge or expertise to determine whether a client's description of its operational requirements is reasonable. This could be done through a senior panel of independent officials overseeing the Options Analysis phase results and the Statement of Operational Requirements (as it is currently being done through the National Fighter Procurement Secretariat) or through the hiring of independent third party experts.
Engage industry in the review of the requirements, through formal industry consultation mechanisms; and
Ensure that the procurement decision (in particular sole source) is based on well-documented, objective and rigorous analysis with necessary expertise.

Roles and Responsibilities

Observation

There is dispersed accountability and responsibility for certain steps of the acquisition process. This was more acute in the CF-18 replacement because of the unique context of the JSF Program and the misalignment of the project and program activities.

DND, PCO, Industry Canada, TBS and PWGSC were involved in the discussions that led to the signing of a Memorandum to Cabinet in 2010 while only DND, Industry Canada and TBS were involved with respect to the 2002 and 2006 MOU.

The National Defence Act gives the Minister of DND authority to define the necessary materiel required by the Canadian Armed Forces.

PCO provides the Prime Minister with public service support and is the Cabinet secretariat. PCO is responsible for consulting involved departments and agencies and providing analysis to the Prime Minister on proposed policies^{Footnote 9}.

TBS has a central oversight role to play in government-wide management practices and ensuring value for money^{Footnote 10}.

As indicated previously, The Defence Production Act establishes the Minister of PWGSC as the exclusive authority to buy or otherwise acquire defence supplies and construct defence projects required by the Department of National Defence.

For the CF-18 replacement, DND was the lead department and required a replacement for its aging fighter fleet. Industry Canada was responsible for facilitating industrial participation.

Through our discussion, we have noted that each of these stakeholders considered that they acted diligently throughout the MOUs and the Memorandum to Cabinet. However, key players, PWGSC in particular, were not sufficiently involved at critical stages of the process.

Impact

In the unique context of the JSF Program, the dispersed accountability and responsibility has contributed to weaknesses or gaps in the oversight, review and approval.

Lesson-learned

To ensure sufficient oversight and diligence in an environment of dispersed accountability and responsibility, there needs to be a clear understanding of the role and responsibilities of key stakeholders in the process.

Recommendation #4

The responsibilities, accountabilities and requirements for sponsoring departments and Public Works and Government Services Canada to consult and inform key stakeholders in the process, including the Treasury Board Secretariat and the Privy Council Office, should be included in one guidance document in order to provide increased clarity to officials managing future major military acquisitions.

Shortcomings identified by the auditor general of canada

As part of the review, one of the objectives was to determine whether the shortcomings identified by the Auditor General in the acquisition process have been effectively addressed.

The Auditor General's report presented a number of findings directly related to the planned procurement.

On April 3, 2012, the Government announced a comprehensive response to the Auditor General's report. The response indicated that it accepted the Auditor General's recommendation and conclusion and that, in response, it would be implementing a Seven-Point Plan (Refer to the National Fighter Procurement Secretariat web site: National Fighter Procurement Secretariat).

At the time of our review, the National Fighter Procurement Secretariat was putting in place many of the steps necessary for the implementation of the Seven-Point Plan. In fact, our review of the acquisition process is intended to support the sixth point of the Plan (with the exception of the sustainment costs), in providing lessons learned that are intended to help the government in improving similar acquisitions in the future. The National Fighter Procurement Secretariat provides regular updates on the status of each step of the plan on its website.

The shortcomings the Auditor General identified in the acquisition processes are currently being addressed through the implementation of the Seven-Point Plan.

Conclusion

Recent changes in the process for identifying capability deficiencies and the new investment planning process should improve governance and oversight, including timely PWGSC participation. In an effort to further improve the acquisition process, we believe that the implementation of our recommendations will improve the alignment of project and program activities and ensure that key stakeholders are informed and approvals are sought in a timely fashion. Additional independent oversight will contribute to decisions being based on objective and well documented options analysis.

This will ultimately increase the level of due diligence for any future acquisitions of a similar nature, risk, and/or complexity.

Appendix A: Stakeholders Interviewed

National Defence

Vice-Chief of Defence Staff

Rear-Admiral M., Lloyd: Chief of Force Development
Brigadier-General S.A., Brennan: Director General, Capability and structures integration, Chief Force Development
Major-General Poulter I.C.: Chief of Programme

Assistant Deputy Minister (Finance and Corporate Services)

Kevin Lindsey: Assistant Deputy Minister, Chief Financial Officer, Finance and Corporate Services
Karen Cahill: Director General, Strategic Finance and Financial Arrangements
Kirk King: Strategic Finance and Financial Arrangements

Assistant Deputy Minister (Materiel)

Rear-Admiral P. Finn: Chief of Staff
Troy Crosby: Director, Next Generation Fighter Capability, Project Management Office
Corey Lee: Finance and Supply Team Manager Next Generation
Fighter Capability - Project Management Office
Michael Slack: Director Continental Materiel Cooperation Chief of Air

Staff – Royal Canadian Air Force

Colonel J.P.A., Pelletier: Director Air Requirements
Lieutenant Colonel Tim Shopa: Director Air Requirements 5 - Fighters and Trainers

Chief Review Services

Chris Currie: Principal

Industry Canada

Aerospace, Defence and Marine Branch

Jeffrey Waring: Director, F-35 Industrial Participation

Public Works and Government Services Canada

Acquisition Branch

Tom Ring: Assistant Deputy Minister
Joanne Provencher: Director General, Major Military Procurements

Next General Fighter Capability Project

Frederick Theiner: Senior Manager

Office of Audit and Evaluation

Linda Anglin: Chief Audit and Evaluation Executive

Treasury Board Secretariat

Investment, Project Management and Procurement Policy Directorate

Elisa Mayhew: Executive Director
Greg Kenney: Senior Director

Defence and Immigration Division

Ian Matheson: Executive Director, International Affairs, Immigration, Defense
Christopher Burt: Advisor/Economist

Privy Council Office

Economic and Regional Development Policy

Rob Pinker: Senior Policy Analyst

Other

Alan Williams: Author, Research Associate at Queen's University
- Assistant Deputy Minister Supply Operations Services (retired), PWGSC (1995 - 1999)
- Assistant Deputy Minister (Materiel) (retired), DND (1999 - 2005)
Dan Ross: Assistant Deputy Minister (Materiel) (retired), DND (2005 - 2012)

Appendix B: Overview of Process and Key Players

Image Description

This image shows the five phases of the procurement process, in the order of: Identification, Option Analysis, Definition, Expenditure Approval (TB) and Implement Procurement.

Table Summary

This table illustrates the 5 phases; Identification, Options Analysis, Definition, Expenditure Approval (TB) and Implement Procurement and who is responsible for Project leadership, Process Owner, Participants, Key activities, Key Output and Suggested MOU alignment.

	Identification	Options Analysis	Definition / Expenditure Approval (TB)		Implement Procurement
Project Leadership	Project Sponsor (Royal Canadian Air Force, Royal Canadian Navy, Canadian Army)	Project Sponsor (Royal Canadian Air Force, Royal Canadian Navy, Canadian Army)	Assistant Deputy Minister (Materiel), Project Management Office		PWGSC
Process Owner	Chief of Force Development	Chief of Programme	Assistant Deputy Minister (Materiel)		PWGSC / Assistant Deputy Minister (Materiel)
Participants	Chief of Programme	Assistant Deputy Minister-Policy Assistant Deputy Minister (Materiel) (sometimes) Assistant Deputy Minister (Finance)	DND Project Sponsor (Royal Canadian Air Force, Royal Canadian Navy, Canadian Army) PWGSC Central Agencies Chief of Programme Industry Canada		DND Project Sponsor (Royal Canadian Air Force, Royal Canadian Navy, Canadian Army) PWGSC TBS Industry Canada
Key Activities	Capability-Based Planning Identify capability deficiency Identification of possible solutions Initiating the Statement of Requirements (SOR) Rough cost estimates Highlight the risks Identify the resources	Options analysis Assess cost and benefits Refinement of SOR Establishment of project Senior Review Board Preparing of a business case for Memo to Cabinet (when policy coverage is required or needs to be re-affirmed)	Definition Substantive cost estimates Finalize SOR Creation of a Project Management Office Project brief to the Major Crown Project Interdepartmental Oversight Committee Expenditure Approval (TB) Senior Project Advisory Committee Establishment of the procurement strategy Preparation of necessary support for Treasury Board Submission (TB Sub.) – Project approval		Request for Proposals process Contract management Management budget and technical aspects of projects
Key Output	Forces Capability Plan that informs the Investment Plan Project Brief	Project Charter Project Complexity and Risk Assessment Business Case Memo to Cabinet TB Sub. for project approval and expenditure authority	Definition Study Project Management Plan Final SOR	TB Sub. expenditure authority	TB Sub. for contract approval
Suggested MOU Alignment	Conceptual and Developmental MOUs		MOU with Production Components

Appendix C: Policy Framework

In considering possible lessons learned, it is important to consider the policies that are in place currently and those that were in effect at the time of the signing of the three MOUs.