2012-601-1 Evaluation of the Parliamentary Precinct Volume 1: The Long-Term Vision and Plan (Final report)

2012-601-1 Final report (PDF, 847KB)

April 1, 2015

Main points

What was examined

1. This evaluation examined the ongoing relevance and performance of Parliamentary Precinct, Long Term Vision and Plan (LTVP). In the Public Works and Government Services Canada (PWGSC) Program Alignment Architecture (PAA) for 2012-13, the Parliamentary Precinct is situated under the Accommodation and Real Property Services program activity and is designated as Sub-activity 1.2.6. The sub-program is administered by the Parliamentary Precinct Branch (PPB). The contents of this document present Volume 1 of this evaluation.
2. Volume 1 of this evaluation addresses the LTVP for the Parliamentary Precinct. The LTVP represents the business line of highest materiality for the PPB. The LTVP was developed over a twenty-five year planning horizon to address significant deterioration experienced by the Parliamentary Precinct over the past several decades and the long term accommodations requirements for the Parliamentary Partners. Volume 1 of the Evaluation examines the LTVP's implementation from June 1, 2007 to March 31, 2013 using a logic model developed with the help of the Program. Volume 1 assesses the achievements of the LTVP against its immediate outcomes, while focusing on the key risks most likely to affect the achievement of its intermediate and ultimate outcomes, using the Evaluation Matrix prepared for the LTVP business line.
3. Volume 2 of the evaluation examines the conduct of the continual operations business line for the period from April 1, 2007 to March 31, 2013. Under this second business line, the Parliamentary Precinct sub-program provides general-purpose accommodation as well as property and facility management to its clients. Volume 2 will examine the relevance and performance of this business line separately from the LTVP using the Logic model and Evaluation Matrix prepared by the Evaluation Directorate.

Why it is important

4. The Parliamentary Precinct has one of the highest concentrations of federally-designated heritage properties in Canada. Deterioration of these buildings and a continuing shortage of modernized office space to support Parliamentary operations are pressing issues, which the LTVP is intended to address. For these reasons, as well as to meet the Department's obligations under the Treasury Board's Policy on Evaluation, it is important to periodically evaluate the LTVP's relevance and performance, to support decision-making regarding its continued implementation.

What we found

5. The Parliament Buildings, including the Peace Tower, are an iconic symbol of Canada. The Centre, East and West blocks of the Parliament Buildings were built between 1859 and 1866 (excluding the Tower and Library). Over a century of daily use and exposure to the environment, combined with evolving accommodation needs, has necessitated their substantial rehabilitation. The rehabilitation of the Parliamentary Precinct, home to the Canadian Parliament, is an investment in the physical and cultural heritage of Canada. The LTVP, as it seeks to ensure the buildings within the Parliamentary Precinct are rehabilitated on behalf of Canadians, represents an appropriate responsibility of the federal government. As the centre of management expertise for major rehabilitation and restoration projects for the Government of Canada, PWGSC has the mandate, knowledge and experience to deliver the LTVP on the behalf of Canadians, its clients and its Parliamentary Partners.
6. In the first years of the LTVP's implementation (2007-13), PPB has made progress against its immediate outcomes. PPB has developed procedures, processes and tools to effectively manage and deliver LTVP projects. Completed projects reviewed in the course of the evaluation have been completed on time, on scope and on budget and ongoing projects are on time, on scope and on budget, as well (Immediate outcome 1). However, reporting of project status with respect to being on-time and on-budget could be improved by the addition of comparative data clearly indicating the approved project budget and schedule, and the date of those approvals. This is the basis for Recommendation 1.
7. PPB has successfully relocated Parliamentarians and their staff out of the West Block, with minimum disruptions to the operations of Parliament and to ensure the building can be effectively renovated. (Immediate outcome 2).
8. PPB is engaged in a number of good project management practices. (Immediate outcome 3). There is room for improvement, however, in the area of performance measurement specific to the LTVP. To this end, the evaluation has made Recommendation 2.
9. Looking ahead to the coming years of the LTVP, no serious risks to the long-term achievement of the intermediate outcomes have been identified. PWGSC is generally recognized as a good project manager and capable of delivering major capital projects involving heritage buildings during the first six years of the LTVP. (Intermediate outcome 1). PPB project managers have also demonstrated sound stewardship of the cultural and physical heritage of the buildings in the Parliamentary Precinct during these years. PPB must compete with other heritage projects for adequate and qualified sub-contractors, with the requisite specialized skills, to carry out work involving heritage buildings. PPB, however, is aware of this potential risk and is taking actions to mitigate any negative impact on the achievement of this outcome. (Intermediate outcome 2).
10. With regards to the LTVP's ultimate outcome (prudent management of real property assets in the Parliamentary Precinct), a potential risk to its longer-term achievement has been noted, in that clients/partners are bound by their own internal policies and standards and not obligated to follow the same Treasury Board policies and government accommodation standards as PWGSC. The PPB has taken steps to mitigate this risk.
11. PPB has implemented a number of practices that contribute to the efficient renovation and rehabilitation of the Parliamentary Precinct such as sequencing of work, adaptability, occupant relocation and the use of efficient technologies. Furthermore, LTVP projects have or will have adopted measures that are intended to contribute to efficiency in energy and water per capita consumption in the Parliamentary Precinct and to the improvement of environmental sustainability.
12. LTVP projects are being implemented with the intent to optimize use of resources through the application of a number of practices such as early industry engagement, expenditure management, and competitive tendering processes. The flexibility in the LTVP's design has allowed for the achievement of economies, as exampled by the approximately $200 million in direct savings resulting from the incorporation of the Government Conference Center into its rehabilitation plans for the Parliamentary Precinct.
13. The use of the construction management approach in establishment of the Rideau Committee Rooms supported the project's economic implementation. Based on thorough analysis and testing, PPB has adopted the Construction Management approach for all ongoing projects; however, it is too early to determine the impact of the approach on the economy of these projects' implementation.

Management response

14. The Parliamentary Precinct Branch acknowledges the recommendations which seek to improve the quality of performance measurement for the Long Term Vision and Plan. The Branch undertakes a variety of performance measurement activities to monitor and assess the results and overall health of the program. In 2014, the PPB initiated a review of all LTVP reporting activities with the view of defining a well-defined set of standardized reporting objectives. The Branch will continue this assessment and undertake to ensure that consistent reliable data is collected to effectively support the Branch in delivering both the LTVP and the broader Parliamentary Precinct program.

Recommendations and management action plan

Recommendation 1: The Assistant Deputy Minister, Parliamentary Precinct Branch, should provide improved comparability with the approved project schedules and budgets when reporting on project status with respect to being on-time and on-budget.

• Management action plan 1: Existing standardized monthly reports will incorporate more prominently a clear comparison of forecasted completion dates and cost estimates against baseline approvals for project schedules and budgets.

Recommendation 2: The Assistant Deputy Minister, Parliamentary Precinct Branch, should develop a comprehensive performance measurement strategy for the Long Term Vision and Plan, and if necessary, develop additional performance indicators and data collection strategies for the Long Term Vision and Plan's intended outcomes.

• Management action plan 2.1: PPB will develop a performance measurement strategy that reflects the requirements outlined by the Treasury Board in its document, Supporting Effective Evaluations: A Guide to Developing Performance Strategies.
• Management action plan 2.2: PPB will examine the need for additional performance indicators and data collection strategies to support the measurement of the Long Term Vision and Plan's intended outcomes, and if so, develop them as part of the performance measurement strategy.

Introduction

1. This report presents the evaluation of the Parliamentary Precinct's results regarding the Long Term Vision and Plan (LTVP). This engagement was included in the Public Works and Government Services Canada (PWGSC) 2012-17 Risk-Based Audit and Evaluation Plan.

Profile

Background

2. The Minister of Public Works and Government Services is the official custodian of the buildings and grounds of the Parliamentary Precinct and is responsible for the provision of general purpose accommodation and the care, upkeep, structural integrity and appearance of the premises, as well as for the management of major renovations and new construction. The buildings and grounds of the Parliamentary Precinct are functionally defined, as it pertains to the Minister's responsibilities, as the premises which the Senate, the House of Commons and the Library of Parliament occupy. They include those enabling Senators and members to perform their parliamentary work and the related premises and grounds required to support that work (excluding constituency offices). The Parliamentary Precinct is composed of 33 Crown owned buildings, and an additional 19 leased buildings. It includes all of the buildings on Parliament Hill, the south side of Wellington Street, the north side of the Sparks Street Mall, and a food production facility in the east-end of the city. (See Appendix A for a detailed map of the area on Parliament Hill).
3. The Parliamentary Precinct houses several thousand federal employees and government officials, including Parliamentarians and their staff. There are currently 52 commercial tenants throughout the Precinct, including 31 retail, utilizing approximately 9,000 m² and generating $2.4 million in annual revenue. Of the buildings and grounds contained within the Parliamentary Precinct, the Parliament Buildings are the most recognizable. Twenty-eight of the 33 (85%) buildings in the Parliamentary Precinct hold a heritage designation.
4. The Parliamentary Precinct Branch (PPB), on behalf of the Minister, is responsible for the management of major renovations and new construction of buildings within the Parliamentary Precinct (examined in this evaluation), as well as for the provision of general purpose accommodation through the care, upkeep, structural integrity and appearance of the premises (examined in Volume 2 of the evaluation). The PPB was established as a stand-alone Branch of PWGSC in 2008. Prior to 2008, services were administered through PWGSC's Real Property Branch. These two business lines of the Parliamentary Precinct Branch are accounted for in sub-program 1.2.6 in the 2012-13 Program Alignment Architecture.
5. In 2001, a Long Term Vision and Plan (LTVP) was introduced by the Department to address significant deterioration experienced within the premises of the Parliamentary Precinct over the past several decades. The objective of the LTVP is to rehabilitate buildings in the Parliamentary Precinct, including the Parliament Buildings. PWGSC was provided with $846.3 million in 2001 for the implementation of the LTVP.
6. In 2005, the Minister of Public Works and Government Services conducted a review the LTVP in partnership with the Parliamentary Partners (Senate, House of Commons, the Library of Parliament) and key government stakeholders such as the National Capital Commission and the Federal Heritage Buildings Review Office. These institutions helped to develop and subsequently fully endorse a revised LTVP, which came into effect in 2007. The remainder of the funding for the 2001 LTVP ($520.3 million) was allocated to the implementation of the 2007 LTVP.
7. The revised LTVP shifted from a fixed 25-year master plan to rolling five-year programs of work over a 25-year planning horizon. The rolling five-year programs of work established short-term cycles for the approval of specific projects for implementation.
8. Activities in the first five-years of the LTVP (2007-08 to 2011-12) focused on:
   • Stabilization and preparation of the West Block for major restoration;
   • Interim space to accommodate Members of Parliament, committee rooms, and other support functions while the West Block was vacated for its restoration;
   • Urgent work to preserve the East Block, Centre Block, and Confederation buildings to halt their deterioration; and
   • Planning for upcoming projects as part of the next five-year cycle (2012-13 to 2017-18).
9. Projects during the first years of the LTVP that have been completed included:
   • The Food Production Facility which was built in the east-end of Ottawa to serve Parliamentary Staff;
   • The establishment of Rideau Committee Rooms to provide Parliamentarians with replacement committee rooms for those made unavailable by the rehabilitation of the West Block;
   • The rehabilitation of La Promenade to accommodate the relocation of Members of Parliament and their staff from the West Block;
   • The rehabilitation of 181 Queen Street so that the House of Commons administration and support staff could be relocated there to enable the rehabilitation of the 180 Wellington Building;
   • The building of the House of Commons Trade Shops at 2086 Walkley Road.
10. Projects that are currently underway are:
    • The West Block is being rehabilitated to address significant deterioration, and to accommodate the House of Commons, and Members of Parliament and their staff on an interim basis during the rehabilitation of the Centre Block;
    • The rehabilitation of the Sir John A. Macdonald Building will provide permanent replacement accommodation for the Confederation Room (Room 200) and an additional multipurpose room; and
    • The 180 Wellington Building once rehabilitated will provide interim accommodations to Members of Parliament and their staff as well as provide an additional multipurpose room for committees.

Authority

11. PWGSC is the custodian of the Parliamentary Precinct as a result of the Department of Public Works and Government Services Act. Under Section 10(1), the Minister is responsible for "the administration of all federal real property and federal immovables not situated in Yukon, the Northwest Territories, or Nunavut, except those under the administration of any other minister, board, or agency of the Government of Canada or any corporation." PWGSC is authorized to carry out renovation, rehabilitation and maintenance within the Parliamentary Precinct, such as those activities detailed in the LTVP.
12. PWGSC's stewardship of the Parliamentary Precinct is further clarified by Section 6.1.9 of the Treasury Board Policy on the Management of Real Property, which states that "Departments must manage buildings they administer so as to conserve their heritage character throughout their life cycles."
13. Under the Parliament of Canada Act, the Parliamentary Partners are responsible for all financial and administrative matters respecting their members, premises, services and staff. This applies to the governance of the LTVP as outlined in their 2005 Memorandum of Understanding with PPB.

Roles and responsibilities

14. PWGSC's PPB is responsible for the implementation of the LTVP.
15. PPB's Major Crown Projects Directorate is responsible for one program under the LTVP: the Major Capital Program; it is composed of one Director General, and three Directors. The Major Capital Program includes the primary projects necessary to advance the overall objective of restoring the key heritage buildings in the Parliamentary Precinct. The Major Crown Projects Directorate is responsible for the major projects being on-scope, on-time and on-budget.
16. PPB's Planning and Operations Directorate is responsible for three programs under the LTVP; the Planning Program; the Recapitalization Program and the Building Components and Connectivity Program. It is composed of one Director General and four Directors. The Planning and Operations Directorate also coordinates reporting to the Deputy Minister on the implementation of the LTVP. The Planning and Operations Directorate is responsible for Planning, Recapitalization and Building Components and Connectivity projects being on-time, on-scope and on-budget.
    • The Planning Program – includes the development of more refined plans and reliable cost estimates for projects in the next Five-Year Program.
    • The Recapitalization Program – includes a series of urgent building interventions (related mostly to exterior masonry repairs) required to ensure on-going viability of buildings and address health and safety issues. These projects are part of the full restoration work that will ultimately be completed on these buildings. Undertaking the projects early in the implementation of the LTVP has been done to reduce the cost and complexity of future projects.
    • Building Components and Connectivity Program – includes projects on technological enhancements to the Precinct.
17. The Parliamentary Partners (the House of Commons, the Senate, and the Library of Parliament) have the responsibility to complete the following in support of the LTVP:
    • Ensure the preservation and enhancement of Parliamentary traditions and operations;
    • Identify short and long-term goals and priorities;
    • Define requirements and performance objectives;
    • Participate in and collaborate on the development of strategies and plans related to the execution of the LTVP, and endorse them or present alternatives;
    • Oversee the implementation of the LTVP Program and its projects to ensure client requirements, performance objectives and strategic goals are met.

Resources

18. From 2001 to the end of March 2013, PWGSC expended approximately $1.1 billion on the execution of the LTVP.
19. As of March 2013, PPB had a total complement of 85 employees, excluding external consultants, agency personnel and construction contractors, to deliver the LTVP and deal with urgent building interventions as required to ensure the ongoing viability of the Parliamentary Precinct buildings.

Logic model

20. A logic model is a visual representation that links a program's activities, outputs, and outcomes; provides a systematic and visual method of illustrating the program theory; and shows the logic of how a program is expected to achieve its objectives. It also provides the basis for developing the performance measurement and evaluation strategies, including the Evaluation Matrix.
21. The logic model for the PPB programs was developed for PPB by the Evaluation Directorate. Volume 1 of the evaluation addresses the LTVP. The activities, outputs and outcomes assessed in Volume 1 of the evaluation are highlighted in the logic model. The logic model is provided in Exhibit 1.

Exhibit 1: Logic model

Focus of the evaluation

22. The objective of Volume 1 of the Evaluation of the Parliamentary Precinct is to determine the LTVP's relevance and performance in accordance with the Treasury Board Policy on Evaluation. The evaluation's assessment covered the period 2007-08 to 2012-13, which were the first six fiscal years of the revised LTVP (2007). Some of the major capital projects reviewed by the evaluation were started before or completed after the assessment period.
23. A project risk assessment was conducted by the Office of Audit and Evaluation for this evaluation to identify the approach and level of effort to be employed in assessing performance (effectiveness, efficiency, and economy). As the revised LTVP was approved in 2007, the approach to the evaluation undertaken for Volume 1 has been to assess the achievement of the LTVP's immediate outcomes within the first six years and to focus on key risks likely to impact the achievement of its planned intermediate and ultimate outcomes over the course of the LTVP's completion. The evaluation assessed the relevance, efficiency and economy of the LTVP, based on Treasury Board suggested methodologies.
24. An evaluation matrix, including evaluation issues, questions, indicators, and data sources, was developed during the planning phase and used to guide the conduct of the evaluation and the preparation of the Report. Information on the approach and methodologies used to conduct this evaluation, as well as limitations encountered in the planning and conduct of the Project and risk mitigation measures taken by the Office of Audit and Evaluation, are located in the "About the evaluation" section at the end of this report.

Findings and conclusions

25. The findings and conclusions below are based on multiple lines of evidence collected during the evaluation in support of the indicators developed to assess questions based on the core issues to be addressed in evaluations (as per Annex A of the Treasury Board Directive on the Evaluation Function). They are presented by Relevance (Continuing need for the Program; Alignment with Federal Priorities and Departmental Strategic Outcome; Appropriate Role and Responsibility for the Federal Government) followed by Performance (Outcome achievement, and Demonstrations of Efficiency and economy).

Relevance

26. Relevance is the extent to which the LTVP addresses a continuing need, is aligned with federal priorities and departmental strategic outcomes, and is an appropriate role and responsibility for the federal government.

Continuing need

27. Continuing need assesses the extent to which the LTVP continues to address a demonstrable need and is responsive to its clients. Continuing need for the LTVP was assessed through a review of the continued relevance of the original rationale, current legislative and policy requirements, and demand for services.

Continued relevance of the original rationale for the Long Term Vision and Plan

28. The original rationale for the Long Term Vision and Plan, when first created in 2001, was to address the deteriorated state of, and the need to modernize the Canadian Parliament Buildings, as well as the long-term accommodation requirements of the Parliamentary Partners. The original rationale remained relevant in 2005 when the LTVP was revised.
29. The Parliamentary Precinct's main buildings have changed little over the last century, having only received sporadic upgrades since their original construction. The House of Commons Accommodation Services has faced criticism from occupants of the West Block due to perceived health and safety issues with the premises. A building condition assessment from 2006 identified several building systems in the West Block as being at critical risk of failure and that the building would be at risk of total failure by 2013.
30. All stakeholders consulted as part of the evaluation agree that there is a need to rehabilitate the buildings to preserve them for future generations. A 2005 public opinion survey revealed that 81% of Canadians agree that the Parliament Buildings are an important Canadian symbol. Furthermore, the deteriorating condition of the Parliament Buildings may pose a risk for the buildings' occupants and the general public due to increasing mechanical, electrical, and masonry failures.

Current legislative and policy requirement for the Long Term Vision and Plan

31. The Treasury Board's Policy on the Management of Real Property, section 6.1.9 mandates the Deputy Head of PWGSC as responsible for ensuring that "the heritage character of federal buildings is respected and conserved throughout their life cycle." As the custodian of the Parliamentary Precinct, PWGSC, through PPB, is responsible for the implementation of the LTVP and maintaining the Parliament Buildings in compliance with the Policy.

Demand for services

32. There is a strong demand for office space to support Parliamentary operations as there is a current shortage of suitable space. Furthermore, following the adoption of the Fair Representation Act on December 16, 2011, the number of seats in the House of Commons will increase from 308 to 338 as of the October 2015 federal election. As a result, there is a need for an expanded House of Commons as well as additional office space for the new Members of Parliament. The LTVP will address the current shortage of suitable office space as well expand the availability of office space to accommodate the expected increase in the number of seats in the House of Commons.

Alignment with federal priorities and departmental strategic outcome

33. The LTVP's alignment with federal government priorities and with the departmental strategic outcome was assessed through a review of federal and departmental priorities related to the LTVP.
34. The restoration of the Parliament Buildings has been identified as an important priority for the Government of Canada. The November 19, 2008, Speech from the Throne states:

    "Canada's institutions are the cornerstone of our democracy, our freedom and our prosperity. Parliament is Canada's most important national institution. It is the only forum in which all Canadians, through their elected representatives, have a voice in the governance of the nation."

35. Economic Action Plan 2013 provided further support for the rehabilitation of the West Block and Government Conference Centre buildings, as well as for the East Block exterior masonry rehabilitation, with $32 million in funding for 2013-14 and an additional $34 million in 2014-15 on an accrual basis (under accrual accounting, revenues are reported when they are earned and expenditures are reported and matched with related revenues as they are incurred).
36. PWGSC's strategic outcome is to deliver "high quality, central programs and services that ensure sound stewardship on behalf of Canadians and meet the program needs of federal institutions." As the implementer of the LTVP, PWGSC is investing to protect the architectural integrity of the Parliamentary Precinct and maintain these historic heritage buildings for its clients/partners and the Canadian public. Additionally, PWGSC, as a provider of high-quality real property and accommodations services, is utilizing the LTVP to fulfill longstanding client/partner demands for additional and modernized office space.

Appropriate role and responsibility for the federal government

37. To determine if the LTVP is an appropriate role for the federal government, the LTVP was examined in light of: transfer to other federal organizations, devolution to another level of government, and transfer to the private sector.
38. Any change in the organization responsible for the LTVP would be a machinery of government decision and it is outside the scope of the evaluation to provide a conclusion on this matter.

Transfer the responsibility of the Long Term Vision and Plan to other Federal Organizations

39. As noted earlier in this report, the Minister for PWGS, under the Department of Public Works and Government Services Act, is the custodian of the Parliamentary Precinct and is responsible for the implementation of the LTVP. Also, as noted under the Parliament of Canada Act, the Senate and the House of Commons have jurisdiction over their accommodations and related requirements.
40. In her 2010 audit on Rehabilitating the Parliament Buildings, the Auditor General recommended that "the Minister of Public Works and Government Services Canada, in cooperation with and with the support of the speakers of the Senate and the House of Commons—and in concert with the Senate Standing Committee on Internal Economy, Budgets and Administration, and the Board of Internal Economy—should develop and propose mechanisms to ensure that responsibility and accountability for the Parliament buildings rest with the Senate and the House of Commons"^{Footnote 1}. The recommendation was beyond the scope of the Department of PWGSC as it was a machinery of government issue, which is the prerogative of the Prime Minister. Responsibility for the LTVP and the maintenance of the Parliamentary Precinct remains with the Minister of PWGS.
41. The LTVP is a major capital restoration and rehabilitation project. Within the Government of Canada, management expertise for major rehabilitation and restoration projects of the Parliamentary Precinct resides within PWGSC's PPB. Both internal and external stakeholders consulted as part of this evaluation have indicated that the Parliamentary Partners do not have a mandate nor the experience or resources required to manage major crown projects. Equally, no other federal government department has this expertise either.

Devolve the responsibility of the Long Term Vision and Plan to Other Levels of Government

42. As the Parliament Buildings are a national symbol of the Government of Canada, it is likely inappropriate to devolve the responsibility for the LTVP to another level of government.

Transfer the responsibility of the Long Term Vision and Plan to the Private sector

43. PWGSC makes extensive use of private sector contractors in its implementation of the LTVP. Private sector expertise has been leveraged in numerous aspects of the rehabilitation and renovation of the Parliamentary Precinct. The complete removal of responsibility and accountability for the Parliamentary Precinct from the federal government, however, is likely inappropriate, as several of its buildings (including the Parliament Buildings) remain key symbols of the Government of Canada.

Conclusions: Relevance

44. There is a continuing need for the LTVP. The original rationale for LTVP (the rehabilitation of the Parliamentary Precinct) remains valid, and there is a policy requirement for the on-going maintenance of its heritage buildings. Furthermore, there is demand for modernized and additional office space to support Parliamentary operations.
45. The LTVP is aligned with federal priorities and PWGSC's strategic outcome. The federal government is committed to preserving the Parliamentary Precinct as a functional complex and its key buildings as historic, cultural, and institutional symbols of Canadian constitutional democracy. By re-investing in heritage assets and providing expanded, modernized accommodations to parliamentarians and their staff, the LTVP is aligned with PWGSC's strategic outcome.
46. The LTVP is an appropriate responsibility for the federal government and is based in legislation. Management expertise for major rehabilitation and restoration projects of the Parliamentary Precinct resides within PWGSC's PPB. The decision to transfer the responsibility for the LTVP to another federal organization is a machinery of government decision, which is the prerogative of the Prime Minister and beyond the scope of this evaluation. PWGSC remains responsible for the LTVP and the ongoing maintenance of the Parliamentary Precinct. Transfer to another level of government or the private sector is not appropriate as the Parliamentary Precinct is a national symbol of the Government of Canada.

Performance

47. Performance is the extent to which a program is successful in achieving its objectives and the degree to which it is able to do so in a cost-effective manner that demonstrates efficiency and economy.

Outcome achievement

48. This evaluation addresses the first six years of the LTVP's implementation. As such, the evaluation examined the extent to which the LTVP achieved its intended immediate outcomes. For LTVP's intermediate and ultimate outcomes, the evaluation examined activities in support of these outcomes to identify any potential risks that might impact their achievement in the longer term. The LTVP's intended outcomes are identified in italics below, followed by an assessment of the extent to which they have been achieved as of March 31, 2013.

Immediate outcome 1: Projects are delivered on time, on budget, and on scope

49. The evaluation examined the achievement of this immediate outcome through its review of a selection of completed and on-going projects.

Completed projects

50. The evaluation examined five completed LTVP projects: the Food Production Facility, the Rideau Committee Rooms, La Promenade, 181 Queen Street, and 2086 Walkley Rd.
51. Of the five completed LTVP projects, the project documentation demonstrates that all five were delivered on time, on scope, and on budget. Based on interviews with key stakeholders and an examination of financial information, the evaluation found that the five projects were completed and ready for occupation as scheduled (on time and on scope) and reported on or below their forecasted budgets by up to nine percent (on-budget). The La Promenade and Rideau Committee Rooms were delivered under budget.
52. The renovation of the Rideau Committee Rooms, La Promenade and 181 Queen Street ensured the interconnections of functions and services, in particular, information technology and communication services within and between these buildings while also providing office accommodation for Members of Parliament and parliamentary staff, as well as committee rooms. In addition, the Rideau Committee Rooms were fitted with up to date technology and the La Promenade building with energy reducing components.

Ongoing projects

53. The evaluation examined three ongoing LTVP projects (the West Block, the Sir John A. Macdonald Building, and the 180 Wellington Building) to determine the extent to which they are on time, scope and budget.

Ongoing projects – Time (Schedule)

54. PPB considers projects on time if the current project critical path schedule estimates that the project will be completed at or prior to the target completion date established at the time the project was approved. PPB has a process in place to review the status of projects on a monthly basis. Project schedules are updated monthly by the Design Consultant (during the design phase) or the Construction Manager (during the construction phase). These schedules are then reviewed by the project's independent Schedule Consultant. The Schedule Consultant has the responsibility to ensure tasks that are behind schedule are reflected accurately; challenge assumptions underlying any changes to the projected completion dates and validate the updated schedule with an emphasis on the project's critical path elements. According to program management, following the validation by the Schedule Consultant and subsequent review by the project management team, an assessment is provided in a Monthly Project Report as to whether the project is on schedule. The Monthly Project Report is provided to PPB's senior management.
55. To evaluate whether the three selected projects are on schedule, the evaluation compared forecast milestone completion dates for the remaining tasks on these projects, as of March 2013, with the forecast completion dates in the project schedules developed at the time these projects received effective project approval. In all three cases, the evaluation found that, although completion dates for some interim milestones had changed, the targeted completion dates for those milestones that indicate project completion – the readiness for occupancy date – were the same as or earlier, than the dates in the original project schedules.
56. The PPB Monthly Project Reports identify the percentage of work completed up to and including the month reported upon. This percentage is based on an analysis of design and construction tasks and sub-tasks weighted to reflect their proportion of the total project work. The Monthly Project Reports for March 2013 identify the West Block Project as 24% complete; the 180 Wellington Building Project as 43% complete; and the Sir John A. Macdonald Building Project as 31% complete. The PPB reports that it uses this figure as a very general indication of the volume of work completed and not for the purposes of measuring the extent to which the project is on schedule. The evaluation noted that the Monthly Report does not compare the percentage of project completion to the completion rates contained in original project schedules. Providing this comparative information would allow measurement of the extent to which the project is on time. It is the opinion of the evaluation that the inclusion of forecast % completion rates would make the report more informative and transparent.

Ongoing projects – Scope

57. The Project Monthly Reports contain assessments of whether the project is within scope. Our review revealed that all monthly status reports included references to minor changes in the scope of work being carried out under these projects. PPB officials report that projects are considered on scope as long as scope changes are not expected to affect the project budget or schedule. As such, the minor changes noted do not result in the project being out of scope. When a major change in scope does occur, one that will affect the project budget or schedule, these are usually of such a nature and magnitude that they normally result in a modified project approval and a re-baselining of the project schedule and/or budget. Given this, Program officials have indicated that, while conceivable, it would appear extremely unlikely that a project would be reported as being out of scope.
58. Based on the above, the value of reporting a project as being "on scope" may be limited. Nevertheless, the monthly reports include discussion of minor scope changes and of issues that could potentially affect the scope of the project. These should, likely, continue to be discussed in these monthly reports.

Ongoing projects – Budget

59. The PPB assesses whether projects are on budget using a similar methodology as that used to assess whether they are on schedule. Each project employs an independent Costing Consultant, whose responsibility is to monitor and record all expenditures on the project, including project management, design and construction and other costs, and to challenge and validate expenditure forecasts for the project. As well, the Costing Consultant prepares a Monthly Costing Report for the project that provides updated expenditure data for costs incurred to date and an updated estimate of the project total costs, based on costs to date and forecast future costs. These forecasts are quite conservative, as they include substantial reserves for risk, cost escalation and contingency. It is the Project Director who determines whether the project is on budget at that point in time based on the Monthly Costing Reports; on revised cost estimates that had not yet been formally reflected on the project's financial management system; and on consultations with the Construction Manager and the PPB Project Management Support Services.
60. To determine the extent to which the selected projects were on budget, the evaluation examined several sources of information, as summarized in Exhibit 2. From the documents that provided original approval of these projects, the Approved Project Budget was obtained. From the Costing Consultant's Monthly Costing Reports for March 2013, the updated Actual and Forecast Expenditures were obtained and the variance of these (if any) with the Approved Project Budget was calculated. From the Project Monthly Reports for these projects for March 2013, the reported overall budget status of the Project (on budget or over budget) was extracted to assess the consistency of the reported budget status in these reports with the variance between the Approved Project Budget and the updated Actual and Forecast Expenditures provided in the Monthly Costing Reports. The results of our review are contained in Exhibit 2.

Exhibit 2: Summary table for Project budgets as of March 31, 2013

61. Exhibit 2 indicates that project management was of the opinion that all of three projects were on budget as of March 2013. For two of the projects (Wellington Building and West Block), the project Actual and Forecast expenditures support this conclusion. In the case of the third project (Sir J. A. Macdonald Building), the Actual & Forecast Expenditures indicate that the project was 6% over budget as of March 2013. The evaluation was informed that the Project Director determined that this project was on budget due to the fact that revised cost estimates for building components and connectivity as well as for risk and contingency cost escalation had not yet been formally submitted to the Costing Consultant and, thus, were not reflected in the latter's report in the form of reduced expenditures. To validate the Project Director's decision, the evaluation reviewed the Monthly Costing reports for additional months in the 2013-14 fiscal year and found that, by January 2014, the Actual and Forecast Expenditures had been reduced to $99.5M (the approved project budget). Given this, the determination that the project was on budget as of March 2013 was reasonable, in our view. As such, the evaluation has found that the selected projects were on budget, after its review of this additional information.

Ongoing projects – Conclusion

62. Overall, the evaluation has found that current projects are on time, scope and budget. The evaluation notes that reporting on project status would benefit from the inclusion of comparative data in the reporting of timeliness and budget that clearly indicates the approved project budget and schedule applicable at the time of the report and the date of these approvals. The addition of this comparative information to the Project Monthly Reports, in particular, would improve the informativeness and transparency of PPB's reporting.

Immediate outcome 2: Relocation of Parliamentarians and their staff is achieved with minimum interruption to operations of Parliament

63. The evaluation assessed the extent to which this outcome was achieved by examining the amount of time Parliamentarians may have lost due to interruptions to parliamentary operations as a result of relocation. There are two components to the achievement of this objective: creating swing space into which occupants can be moved, and moving occupants to this space with minimal interruption.
64. To commence the renovation of the West Block, PPB undertook activities in advance of the relocation of Parliamentarians and their staff, in particular, preparing swing space and support facilities. Buildings such as La Promenade, the Rideau Committee Rooms and 181 Queen Street were acquired and renovated to accommodate Parliamentarians and their staff in time for the renovation of the West Block to progress on schedule. In addition, PPB completed other LTVP projects to accommodate parliamentary staff's relocation, such as the Food Production Facility, and 2086 Walkley Road (the House of Commons Trade Shops). As of March 31, 2013, 15 relocation projects had taken place.
65. In terms of moving Parliamentarians and their staff, external stakeholders reported that no interruptions to the operations of Parliament were experienced as the moves were conducted outside of regular working hours. In interviews, clients/partners corroborated the above finding and reported that they were satisfied with the manner in which the relocation of staff and Parliamentary facilities was executed. Furthermore, clients/partners interviewed also expressed satisfaction with the quality of the temporary accommodations.

Immediate outcome 3: Good management practices are followed

66. The extent to which this outcome has been achieved was assessed by a review of a selection of management practices employed by PPB and the extent of their implementation, as well as through interviews with internal and external stakeholders.
67. PPB defines the established practices found in the National Project Management System (NPMS) as good management practices. Within the NPMS there are established processes and procedures for planning, implementation, and closing of major crown projects. The evaluation conducted a review of management practices implemented at the project level and concluded that PPB has implemented established and well-accepted practices to deliver LTVP projects.
68. The evaluation's file review found that all plans for the five completed and three ongoing LTVP projects adhered to NPMS directives, despite completed projects pre-dating its implementation. Prior to the implementation of NPMS in December 2010, LTVP projects adhered to the Department's Project Delivery System. These conclusions are supported by the 2010 Office of the Auditor General report, which recognized that NPMS guidelines were being appropriately followed by PPB and that the project management practices employed by PPB were generally sound^{Footnote 2}.
69. PPB has an ongoing process for disseminating best practices throughout their projects though the use of an evergreen database for incorporating lessons learned into future LTVP projects. For example, the construction management model proved successful for the Rideau Committee Rooms project and was subsequently implemented on larger projects.
70. For the rehabilitation and renovation of the West Block, Sir John A. Macdonald, 180 Wellington Building, and the Rideau Committee Rooms, PPB selected the construction management model over that of the general contractor method when determining the contracting approach. The literature review and interviews indicated the construction management approach as offering the greatest benefits when dealing with complex heritage renovation projects. More information on the selection of a contracting approach for major capital projects is presented in the Economy section of this Report.
71. In the delivery of the LTVP projects, PPB performs extensive risk management at all stages of a project, such as risk assessments during a project's inception, identification, and design stages. Project risk is also reviewed by project managers on a monthly and project-by-project basis to evaluate the levels of ongoing risk. Any immediate risks are dealt with by the project team and construction manager. When items are deemed to no longer represent a risk, they are removed from the risk registry.
72. To ensure a sufficient level of expertise to deliver LTVP projects, PPB's project management model ensures that PWGSC personnel hold key decision-making roles and are supported by an array of consultants with specialised expertise. External stakeholders have said that this combination of authority and expertise has resulted in well-managed projects.
73. External stakeholders have indicated that the current overall planning process is optimal and should be adopted for future projects, with some suggesting that earlier involvement of private sector consultants would more easily allow their expertise to be incorporated into the projects. Finally, all stakeholders interviewed as part of this evaluation further stated that PPB should continue to maximize their use of existing experience and knowledge during the planning process for future projects. To this end, PPB has developed a reporting structure to co-ordinate existing resources between the Major Crown Projects (MCP) Directorate and the Planning and Integration Directorate for future renovation/rehabilitation projects. Increased coordination, co-operation, and overlapping of staff and planning activities between the two Directorates is expected by PPB to enhance project management team stability over a project's lifecycle as well as tighten integration of individual projects within the overall master plan.
74. Finally, the evaluation found that PPB does not appear to have a formal performance measurement strategy for the LTVP in support of results-based management, as advised by the Treasury Board in their document "Supporting Effective Evaluations: A Guide to Developing Performance Measurement Strategies". Some of the performance measures used in PPB's regular reports are aligned with the LTVP's logic model, but in general, information is collected on individual projects, with variation in the type and quantity of performance information collected across projects.

Intermediate outcome 1: Public Works and Government Services Canada is recognized as a good project manager, capable of delivering major capital projects involving heritage buildings

75. The evaluation examined the extent to which PWGSC is, to date, recognized as a good project manager and the extent to which it was recognized as capable of delivering major capital projects involving heritage buildings. The examination provides a means by which to identify any potential risk to the longer-term achievement of the outcome. In examining this outcome, the evaluation gathered external stakeholders' perspectives on PWGSC's capacity and its project management ability. The evaluation did not identify any risks to the achievement of this outcome in the medium or long term.
76. External stakeholders generally recognize PWGSC as a "good project manager". They also indicated PPB is using the correct approach (construction management) to manage and implement these projects. Furthermore, the Auditor General noted in 2010 that PWGSC had generally sound project management practices in rehabilitating the Parliamentary Buildings that take into consideration the heritage character of the buildings, their age and conditions, the needs of their users and the protection of the environment.
77. PWGSC is recognized as capable of delivering major capital projects involving heritage buildings. External stakeholders acknowledge that PPB has the skill set to deliver major capital projects involving heritage buildings. Parliamentary partners/clients have also acknowledged that PPB-Major Crown Projects has the experience, knowledge, and expertise to deliver major capital projects involving heritage buildings. PPB project managers use an in-depth consultative process (involving the Federal Heritage Buildings Review Office, Parks Canada, the National Capital Commission, and the Heritage Conservation directorate) and historical and cultural analysis to ensure the proper preservation of the heritage buildings, in accordance with industry standards.

Intermediate outcome 2: Buildings in the Parliamentary Precinct are rehabilitated to ensure sound stewardship of their cultural and physical heritage on behalf of Canadians

78. The evaluation examined the extent to which PWGSC is, to date, supporting the achievement of this outcome, by identifying potential risks to its achievement.
79. A potential risk to the achievement of this outcome is the availability of a workforce with special skills due to the heritage status of the buildings. This poses a challenge for construction managers who have to compete with other projects for adequate and qualified sub-contractors, some with specialized skills, to carry out work. External and internal interviewees report that construction managers have hired workers with specialized skills from across the country and the United Kingdom to ensure the necessary level of expertise. Furthermore, PPB is coordinating scheduling of the restoration and rehabilitation projects in a way to ensure the continued availability of expertise once it has been brought on site. As such, this indicates that PPB is aware of and taking appropriate actions to mitigate the potential risk to the achievement of this outcome.

Ultimate outcome: Prudent management of real property assets in the Parliamentary Precinct

80. The evaluation examined the extent to which the PPB has prudently managed real property assets through investments of federal resources in the rehabilitation and restoration projects undertaken within the first years of implementing the 2007 LTVP. The examination provides a means by which to identify any potential risk to longer-term achievement of the outcome. The evaluation assessed prudent investment of federal resources against Treasury Board Directives of major crown projects and by using information gathered from project documentation and external stakeholder interviews.
81. Integral to the project budgets authorized by the Treasury Board, PPB manages substantial contingency funds which are critical for the rehabilitation of heritage buildings and systems, to ensure that projects can be delivered within overall project budgets.
82. PPB project documentation provided evidence of steps taken to ensure that funds expended on LTVP projects are prudently managed. For example, any funds that are expended by the construction managers, as well as any changes to the original estimates, are first approved by PPB project managers, therefore ensuring that the expenditures of project funds are properly authorized. The authorization process allows the project managers to review the changes and ensure they are reasonable and within scope and budget. Additionally, to ensure prudent management of LTVP funds, PPB project managers, construction managers, and independent consultants with third-party oversight responsibilities review work schedules, project risk registries, and budgets on a monthly basis to ensure that projects remain on their critical paths and funds are disbursed appropriately.
83. External stakeholders have confirmed that PPB project managers approve all expenditures and oversee all work. For example, according to external stakeholders all renovation work must be approved and signed off by the PWGSC project manager, prior to commencing the work. Furthermore, PPB commissioned an audit by PricewaterhouseCoopers. The firm conducted a financial expenditure audit on the West Block project for the period June 30, 2011 to December 21, 2012 that concluded that the project expenditures were appropriately spent and accounted for in accordance with the construction management contract.
84. Working with the Parliamentary Precinct Partners adds complexity, which may present a risk to the future achievement of this outcome. The PPB is required to balance sound stewardship and conduct itself in accordance with government rules and regulations on behalf of Canadians while also meeting the accommodation and real property needs of these unique clients/partners, who are not obliged to follow the Treasury Board policies and government accommodation standards. Internal and external stakeholders have indicated that PPB project managers, in delivering the LTVP, negotiate clients'/partners' requirements and find alternative solutions to requests from clients/partners that could not be met given existing budgetary realities and schedule constraints. This indicates that PPB is aware of, and is taking action to mitigate this potential risk to the achievement of the ultimate outcome.

Conclusions: Immediate outcome achievement and risks to achievement of intermediate and ultimate outcomes

85. PPB achieved its immediate outcomes during the six year period under review.
86. PPB has developed procedures, processes and tools to effectively manage and deliver LTVP projects. Completed projects reviewed in the course of the evaluation have been completed on time, on scope and on budget, and ongoing projects are on time, on budget and on scope, as well. However, reporting of project status with respect to budget and schedule could be improved by the addition of comparative data clearly indicating the approved project budget and schedule, and the date of those approvals, applicable at the time reports are prepared. (Immediate outcome 1). This is the basis for Recommendation 1.
87. PPB has successfully relocated Parliamentarians and their staff out of the West Block for renovation, with minimum disruptions to operation of parliament. (Immediate outcome 2).
88. PPB is practicing a number of good project management practices in the implementation of LTVP projects overall. There is room for improvement, however, in the area of performance measurement specific to the LTVP (Immediate outcome 3). To this end, the evaluation has made Recommendation 2.
89. PWGSC is generally recognized as a good project manager and capable of delivering major capital projects involving heritage buildings for the first six years of the LTVP. The evaluation found no serious risk to the long-term achievement of this outcome. (Intermediate outcome 1).
90. The PPB is ensuring sound stewardship of the cultural and physical heritage of buildings in the Parliamentary Precinct on behalf of Canadians. The evaluation identified a potential risk to the long-term achievement of this outcome in that the PPB must compete with other heritage projects for adequate and qualified sub-contractors, some with specialized skills. PPB is aware of, and is taking appropriate steps to mitigate this risk. (Intermediate outcome 2).
91. Project managers have demonstrated prudent management of resources during the first six years of the LTVP. Recognizing the uniqueness of the clients/paterns, the evaluation identified a potential risk to the achievement of the ultimate outcome of the LTVP, in that clients/partners are not obligated to follow Treasury Board policies and government standards with respect to accommodation. The evaluation found that the PPB has taken steps to mitigate this risk to the long-term achievement of this outcome. (Ultimate outcome).

Efficiency and economy

92. Demonstration of efficiency and economy is defined as an assessment of resource utilization in relation to the production of outputs and outcomes. Efficiency refers to the extent to which resources are used such that a greater level of output is produced with the same level of input or, a lower level of input is used to produce the same level of output. Economy refers to minimizing the use of resources. A program has high demonstrable economy and efficiency when there is a high correlation between minimum resources and outcomes achieved.

Efficiency

93. Efficiency refers to the extent to which planning and implementation of LTVP projects optimize the use of resources. Although we were unable to evaluate the extent to which the LTVP had been implemented efficiently by PPB, we did observe a number of practices that contribute to efficiency.

Sequencing

94. The LTVP was designed to conduct the rehabilitation in manageable chunks to create a consistent demand for resources so that these remained available over time, as needed, and to avoid a surge in demand.
95. Further, the sequencing started with the West Block, then proceeds to the East Block (Phase 1), and will finish with the largest and most complex piece of the rehabilitation, the Centre Block, before winding down with the rest of the program of work for the East Block. This allows for the industry to expand and to enhance skills at a steady pace over a period of time.
96. In managing the LTVP projects, project work elements and the projects themselves, where possible, are planned in succession and in staggered project implementation, rather than having all projects completed at the same time. It results in efficient use of resources, which mitigates the shortage of skilled labour.

Adaptability

97. The 2007 LTVP was designed to allow it to adapt to changing conditions so that it could capitalize on opportunities to increase efficiency in the renovation and rehabilitation of the Parliamentary Precinct.
98. For example, although initial plans were to isolate committee rooms in the West Block's North Wing to accommodate Committees, PPB subsequently identified The Canadian Museum of Photography as an alternative location for Committees. This building had been vacant for several years and could be modified and re-utilized to support the Committee Rooms' functions. This enabled the West Block committee rooms to relocate four years earlier than originally planned, at the same time as the Members of Parliament moved to La Promenade Building. This approach also advanced the West Block project's completion by up to two years, and reduced the West Block project costs by approximately $60M as a result of the shorter project timeline and less complex approach to restoring the building.
99. Another example is the integration of the Government Conference Centre into the plan as a suitable replacement for the Senate Chamber. Originally, the plan was to construct a chamber for the Senate by in-filling the courtyard of the East Block much like that planned for the House of Commons in the West Block. The integration of the Government Conference Centre as a replacement, instead, reduced the scope of work that had been planned for the East Block while rehabilitating an additional Crown asset in need of repair.

Relocation

100. The LTVP renovation project for the West Block involved vacating the building before beginning its rehabilitation and renovation. The selected approach of vacating the buildings for a complete overhaul has been demonstrated to be more cost-efficient than renovating an operational building. When asked if construction could proceed while the building was occupied, architects and engineers concluded that, while possible, the approach would not be reasonable, as renovating an occupied building would not allow the detection and repair of root problems, or the replacement of major environmental systems.
101. Other countries, including Austria, Finland, and Switzerland, are currently renovating (or have recently renovated) their national parliaments. Lessons from these cases indicate the necessity of relocating staff to temporary accommodations for the duration of the work; the challenge of carrying out the renovations zone by zone; and the increased expense and duration of the work imposed by the constraints of renovating/rehabilitating an occupied building.
102. The United Kingdom had attempted to renovate the Palace of Westminster without removing the tenants or disrupting Parliamentary operations. Delays and cost overruns were encountered as a result of attempting to keep the buildings fully operational during the renovation and rehabilitation work. As of October 2012, renovations had been put on hold while the UK government searched for a better solution. The United Kingdom is currently examining Canada's LTVP strategy for approaches to future renovations projects.

Technological efficiency

103. In its rehabilitation of the West Block, PPB was able to introduce the use of new technologies which improved the efficiency of its rehabilitation. PPB implemented laser technology rather than traditional cleaning techniques to clean the heritage masonry. PPB reports that laser technology is a dry process which is quicker and allows more flexibility in the sequencing of work. PPB also implemented blasting techniques for excavation rather than hoe ramming. PPB reports that the process is quicker, it creates less interruption (short bursts vs. prolonged ramming) and is less expensive.

Utility efficiency

104. While PPB does not systematically track or report on utility efficiency, the implementation of the LTVP intends for the Parliament Buildings to be more efficient overall than they were prior to the renovation and rehabilitation. Internal and external stakeholders stated that building improvements will result in increased utility efficiency. It is expected that per capita usage of utilities within the Parliamentary Precinct as a whole will be less than before renovations were undertaken. The total utilities consumed, however, will rise due to interior space additions, longer run times and higher usage requirements of several buildings which were not fully occupied prior to renovations (for example, the Sir John A. Macdonald and 180 Wellington buildings).
105. The evaluation found that LTVP projects implemented and/or plan to implement new building standards and measures to improve energy and water consumption. For completed projects, measures implemented in compliance with new building standards include: energy efficient lighting fixtures and occupancy sensors have been installed to reduce electricity use in unoccupied spaces; water saving plumbing systems; and exhaust systems with heat recovery where building ventilation is equipped with heat recovery wheels, thereby reducing utility use. For on-going projects, numerous sustainability and energy efficiency measures have been implemented, such as: high efficiency motors for heating/ventilation/air conditioning/pumps; variable speed motors when 100% capacity is not always required; heat recovery systems to capture and re-use heat from the West Block glass roof. In addition, between 70% and 85% of demolition materials are being diverted from landfills and reused as well as re-cycled. New energy efficient windows will also be used in all three buildings. It is expected that these measures, once implemented, will improve energy and water consumption. For example, the West Block design is targeting a 25% energy consumption reduction in comparison to its previous use as a result of ensuring the building's compliance with current National Energy Code standards.
106. Finally, all heritage buildings, including the West Block, 180 Wellington and Sir John A. Macdonald are being designed to Green Globe standards (sustainability targets specifically designed for heritage buildings). These renovation and rehabilitation projects were designed to meet the Leadership in Energy and Environmental Design Silver rating, which rates buildings on their use of water, energy consumption, materials, resources, and other design elements. The silver rating is the second of four levels of which platinum is the highest possible rating.

Economy

107. Economy refers to minimizing the use of resources. Economy is said to have been achieved when the cost of resources used approximates the minimum amount of resources needed to achieve expected outcomes. Although the evaluation was unable to assess the extent to which the LTVP has been implemented economically by PPB, a number of practices were observed that contribute to economy.

Industry engagement

108. PPB regularly engages with the construction industry to inform them of upcoming work so that industry has an opportunity to plan. The industry has informed PPB that they are, as result, aware of upcoming projects which form part of the LTVP. By providing industry with the opportunity to anticipate upcoming work, suppliers are better able to identify needed resources to respond to competitive tenders for work; for its part, PPB is better informed with regard to the nature and value of resources necessary to complete tasks.

Expenditure management

109. PPB reports that it manages construction costs throughout the life of its projects. At the start, PPB implements cost planning (the establishment of a cost limit) which is followed by cost estimating (the preparation of a detailed cost plan including estimates and a cost breakdown structure). As noted previously, there is also ongoing monitoring of actual costs against estimates.
110. When determining anticipated costs (cost planning), PPB employs several practices including benchmarking against similar projects, use of its cost templates and the use of lessons learned from previous projects. In the case of masonry work two pilot projects were conducted to determine the rehabilitation procedures and to validate in more detail the actual condition of the masonry. The rates associated with those pilot projects were used to benchmark ongoing masonry work. In addition, all changes to construction are reviewed and the cost is given a fair and reasonable assessment by a third party. Pre-construction estimates (cost estimating) engage industry expertise (a Prime Consultant, a Construction Manager and an independent cost specialist to provide a third-party challenge function). When cost estimating, PPB documents assumptions (including exclusions) and confirms contingencies and risks allowances. PPB also employs Value Engineering methods throughout projects to ensure it maximizes cost efficiency by ensuring construction scope is limited to what is essential to meet the desired outcome.

Competitive tendering processes

111. PPB uses competitive tendering processes for all of its procurements. Further, a fairness monitor is engaged for all significant contracting processes. Finally, in all cases where PPB engages a construction manager, the construction manager engages subcontractors using competitive tendering processes that respect government procurement principles. These procurement methods are intended to achieve best value to the Crown.

Flexibility

112. In addition to contributing to efficiency of the program, the integration of the Government Conference Centre into the LTVP as a suitable replacement for the Senate Chamber also contributed to savings. PPB reports that early indicative assessments from 2012 estimated the difference in costs between rehabilitating the East Block and adapting an infill Chamber versus adapting the Chamber within the Government Conference Centre will result in savings of approximately $200M (discounted to present value).
113. In addition to these savings, this approach will permit PWGSC to undertake the rehabilitation of the Centre Block as per its planned schedule. Should the integration of the Government Conference Centre into the LTVP have not occurred, PWGSC projected a $319M expenditure resulting from the inflation impact from delaying the Centre Block project by 4 years (the time required to complete an infill project within the East Block).

Construction management

114. There are four main contracting approaches that can be used to deliver Major Capital projects: design-build, design-bid-build (one general contractor), multiple general contractors, and construction management. Each approach is suitable for a particular set of circumstances and all have attendant advantages and disadvantages that can affect the implementation of a major capital project.
115. Under a Construction Management approach a construction manager enters into multiple contracts with trade contractors and suppliers. The Construction Manager assumes responsibility for the performance of the trade contracts (subcontracts) much as a general contractor would under the traditional method. Under the construction management approach, private sector contractors submit bids to act as construction manager for the overall project. The successful construction manager then engages other subcontractors through competitively tendered processes that respect government principles. When employing this model, the contractor has the responsibility to ensure that the project adheres to a pre-established schedule and to provide expert advice to the building owner, as well as manage the subcontractors. A reported key advantage of the Construction Management approach is that it balances risk, flexibility and efficiency for large complex projects. PPB reports that Construction Management approach represents, in their view, the optimal approach for managing its ongoing projects.
116. In the case of the completed LTVP projects reviewed, PPB adopted the Construction Management approach for the management of the Rideau Committee Rooms project. This approach was not adopted for the other four completed projects reviewed by this evaluation. A review of the benefits of the Construction Management approach found that the project had only three change orders and these were either Crown-initiated or the result of building conditions discovered only after the start of construction. As well, interviewees believed that the Crown had full visibility of the process and had confidence in the abilities of the Construction Manager. The project, which had an aggressive schedule, was completed on time and on budget.
117. In the case of the three on-going projects reviewed, it is too early to determine the extent to which the use of the Construction Management approach is economical. When determining the appropriate contracting approach for the on-going projects, PPB considered a variety of factors. In the case of the West Block Project, PPB commissioned a study in 2010 which analyzed the documented advantages and disadvantages of the Construction Management Approach, and recommended the Construction Manager approach. The study ranked the option high in its ability to provide budget and cost control. The study also noted the success PWGSC had in using the construction management approach for two other projects (the Rideau Committee Room project which is discussed above and the Cliff Street project which was not examined by this evaluation). Furthermore, PPB officials report that the adoption of the Construction Management approach for the West Block Project resulted in the target completion date for this project being brought forward by one year. The same 2010 study assessed the requirements for the rehabilitation of the Sir John A. Macdonald Building and concluded that the Construction Management approach was the preferred option.

Conclusions: Efficiency and economy

118. PPB has implemented a number of practices that contribute to the efficient renovation and rehabilitation of the Parliamentary Precinct such as sequencing, adaptability, occupant relocation and the use of efficient technologies. Furthermore, LTVP projects have or will have adopted measures that are intended to contribute to efficiency in energy and water per capita consumption in the Parliamentary Precinct and to the improvement of environmental sustainability.
119. LTVP projects are being implemented with the intent to optimize the use of resources through the application of a number of practices such as early industry engagement, expenditure management, and competitive tendering processes. The flexibility in the LTVP's design has allowed for the achievement of economies, as exampled by the approximately $200 million in direct savings resulting from the incorporation of the Government Conference Center into its rehabilitation plans for the Parliamentary Precinct.
120. The use of the construction management approach in establishment of the Rideau Committee Rooms supported the project's economic implementation. Based on thorough analysis and testing, PPB has adopted the Construction Management approach for all ongoing projects; however, it is too early to determine the impact of the approach on the economy of these projects' implementation.

Management response

121. The Parliamentary Precinct Branch acknowledges the recommendations which seek to improve the quality of performance measurement for the Long Term Vision and Plan. The Branch undertakes a variety of performance measurement activities to monitor and assess the results and overall health of the program. In 2014, the PPB initiated a review of all LTVP reporting activities with the view of defining a well-defined set of standardized reporting objectives. The Branch will continue this assessment and undertake to ensure that consistent reliable data is collected to effectively support the Branch in delivering both the LTVP and the broader Parliamentary Precinct program.

Recommendations and management action plan

Recommendation 1: The Assistant Deputy Minister, Parliamentary Precinct Branch, should provide improved comparability with the approved project schedules and budgets when reporting on project status with respect to being on-time and on-budget.

• Management action plan 1: Existing standardized monthly reports will incorporate more prominently a clear comparison of forecasted completion dates and cost estimates against baseline approvals for project schedules and budgets.

Recommendation 2: The Assistant Deputy Minister, Parliamentary Precinct Branch, should develop a comprehensive performance measurement strategy for the Long Term Vision and Plan, and if necessary, develop additional performance indicators and data collection strategies for the Long Term Vision and Plan's intended outcomes.

• Management action plan 2.1: PPB will develop a performance measurement strategy that reflects the requirements outlined by the Treasury Board in its document, Supporting Effective Evaluations: A Guide to Developing Performance Strategies.
• Management action plan 2.2: PPB will examine the need for additional performance indicators and data collection strategies to support the measurement of the Long Term Vision and Plan's intended outcomes, and if so, develop them as part of the performance measurement strategy.

About the evaluation

Authority

This evaluation was included in the Public Works and Government Services Canada (PWGSC) 2012-17 Risk-Based Audit and Evaluation Plan.

Approach and methodology

The evaluation was conducted in accordance with the Standard on Evaluation for the Government of Canada. The evaluation took place between December 2012 and September 2013. To assess the evaluation issues and questions, the following lines of evidence were used.

Document review: An initial document review provided an understanding of the LTVP and its context to assist in the planning phase. Documents reviewed included documents provided by PPB, as well as documents written about the Plan. The document review included Speeches from the Throne, federal budgets, legislation, policy documents, submissions for funding, departmental corporate documents (such as annual Reports on Plans and Priorities and Departmental Performance Reports), PPB documents (LTVP Building on a Solid Foundation, Annual reports, Branch Business Plans) and other documents. All documentation was reviewed to examine the Plan's relevance, design and delivery, and performance.

Literature review: A literature review was conducted to: contextualize the Plan both nationally and internationally; provide theoretical background for the Program model; and identify alternative delivery models through an analysis of models used by public sector organizations in Canadian and international jurisdictions. For this evaluation we examined the delivery mechanism within provincial legislatures and international jurisdictions (Government of Australia, Government of the United Kingdom, and the Federal Government of the United States, the Government of Austria, the Government of Switzerland, and the Government of Finland) were included in the scope of the review, based on their comparability to the Canadian legislature. The review relied on publicly available sources of information, focusing on legislative, accountability and funding elements of office accommodation models employed in selected jurisdictions, as well as current operational models.

Interviews: The evaluation team conducted interviews with various stakeholder groups involved in the renovation and rehabilitation of the Parliamentary Precinct. A total of twenty-six (26) stakeholder interviews were conducted, of which eleven (11) were internal stakeholders, while fifteen (15) were external stakeholders. Internal stakeholders refer to PPB employees. External stakeholders include private sector construction managers and general contractors (6), central agency representatives (1), and PPB's client/partners (8). All of the individuals who were asked to participate in an interview accepted the invitation. Interviews in the National Capital Area were conducted in person. Themes and explanations emerging from the transcripts were mapped against the Evaluation Matrix. The qualitative analysis of the interviews provided information about the LTVP's relevance and performance.

Financial analysis: The evaluation examined financial information for five completed and three ongoing LTVP projects. The extensive financial analysis was prepared for three ongoing as well as five completed projects. The information was used to examine efficiency and economy of the LTVP projects.

Operational data analysis: The evaluation team examined the data gathered by PWGSC on energy and water consumption for the Parliamentary Precinct. The evaluation also examined extensively the risk registries for ongoing projects. In parallel to the risk registries the evaluation examined the extent to which project risk allowance and line item risk were used.

Secondary data: The evaluation team relied on a couple of independent external reports: 1. Raymond Chabot Grant Thornton – Parliamentary Precinct Branch Major Crown Projects Phase I Independent Review of Project Management Practices and, 2. PricewaterhouseCoopers Examination of Construction Management Contracts for West Block Rehabilitation Project. The cost audit that was done for the West Block was a onetime audit and limited in scope. Cost audits were not an established process during the period that was evaluated, however cost audits will be part of the next five year plan (2014-19) and part of the submission to Treasury Board of Canada Secretariat (TBS). In addition, independent studies conducted by external third-parties such as the Office of the Auditor General of Canada and corporate reporting data from PWGSC's Departmental Scorecard, Monthly, Quarterly and Annual Reports to the Deputy Minister, Performance Measurement Framework and Departmental Performance Reports were also relied upon.

File review: In accordance to the Terms of Reference, the evaluation team selected three ongoing projects: the West Block, 180 Wellington building and the Sir John A. Macdonald building. A sample of completed projects was selected: the Food Production Facility, the Rideau Committee Rooms, La Promenade, 181 Queen Street, and 2086 Walkley Rd. A file review was conducted to assess the consistency and adherence to the PDS and NPMS project management systems. A structured assessment checklist was used to systematically extract and compile the data on all projects. File review data was used to assess the LTVP's performance, and to corroborate interview and document review data.

Limitations of the methodology

The methodology used for this evaluation has a number of limitations, however, multiple lines of evidence were used to conclude on the Program's achievement outcomes.

Interviews: The purpose of the interviews was to gather the perspectives of stakeholder groups involved in the renovation and rehabilitation of the Parliamentary Precinct. The evaluation team identified from a list provided by PPB of stakeholders both internal and external who had extensive experience and intimate knowledge of the selected projects. Interview guides for all stakeholder groups were developed in accordance with the Evaluation Matrix. Interviewer bias was mitigated by using a semi-structured interview guide and ensuring that two evaluation team members attended every interview.

Document review: Analysis of secondary data was limited to reports prepared by the Program with one exception: the Costing Consultant on the Wellington Building project did not produce their own costing report during the period reviewed by the evaluation but, rather, validated a costing report prepared by PPB's Project Management Support Services (PMSS). Thus, for this building we used the costing report prepared by the latter group

In terms of performance-related information, the evaluation did not assess the validity of the claims made or the reliability of the measurement tools employed.

File review: The file review was limited to the projects that were selected by the evaluation team based on several criteria (such as project size, monetary value and completion state). Findings from the file review were used to corroborate qualitative data from interviews.

Financial analysis: Financial data related to the LTVP's project budgets and expenditures were reviewed and analyzed to assess the efficiency and economy. Financial information was provided by PPB. The evaluation team was not able to access and extract information from SIGMA due to coding issues. Overall, the evaluation team placed some reliance on the external audit by PriceWaterhouseCoopers. The external audit examined the expenditures of the West Block and the auditor rendered an unqualified opinion. The evaluation also examined the allocation of project risk contingencies and the project risk allowance in conjunction with the project risk registry.

Reporting

Findings for Volume 1 were documented in a Director's Draft Report, which was reviewed by the Office of Audit and Evaluation's Quality Assessment function. PPB's Director General was provided with the Director's Draft Report and was requested to validate facts and comment on the Report. A Chief Audit and Evaluation Executive's Draft Report was prepared and provided to the Assistant Deputy Minister, Parliamentary Precinct Branch, for acceptance as the Office of Primary Interest. The Office of Primary Interest was requested to respond with a Management Action Plan. The Draft Final report, including the Management Action Plan, will be presented to PWGSC's Audit and Evaluation Committee for the Deputy Minister's approval in November 2014. The Final report will be submitted to the Treasury Board Secretariat and posted on the PWGSC website.

Project team

The evaluation was conducted by employees of the Office of Audit and Evaluation, overseen by the Director of Evaluation and under the overall direction of the Chief Audit and Evaluation Executive. The evaluation was reviewed by the Quality Assessment function of the Office of Audit and Evaluation.

Appendix A: Long Term Vision and Plan map