2011-608 Evaluation of the Federal Accommodation Program (Final report)

2011-608 Final report (PDF, 691KB)

August 28, 2014

Glossary

For the reader's convenience, bolded terms found throughout the document are defined below:

Accommodation Levy:

the mechanism by which Public Works and Government Services Canada (PWGSC) is funded for accommodation and related services based on either a 13% percent amount on approved salary dollars.

Base Building:

completed building shell which is used to accommodate tenants. It includes all finished common area space such as floors and bearing, demising and enclosing walls, ceilings, and building systems consistent with the designed function and planned general use of the building, its age and location. In the case of office accommodation, the base building would include window coverings and primary identification signage.

Building Rent:

for leased and lease-purchase accommodation, building rent is composed of the contract (net) rent, operating and maintenance expenses, taxes, and escalations (or de-escalations) using the Consumer Price Index;

for Crown-owned accommodation building rent is composed of a survey-based market (net) rent, operating and maintenance expenses and taxes.

Canada Revenue Agency (CRA) Act:

under the Act, CRA has authority over all of its real property decisions, including acquisition, administration, and disposal of real property and immovables. In 2007, CRA exercised its real property authority through an exclusive agreement with PWGSC for Program services. CRA is an example of a reimbursing client.

Community portfolio:

consists of all of the PWGSC-owned and/or leased accommodation assets located within a single community.

Departmental Integrated Investment Plan:

a five-year outlook on investments and acquired services to support PWGSC's Strategic outcome. The current Plan identifies the key investments and acquired services required over 2012–13 to 2016–17 to maintain the Department's asset base and to attain its service and infrastructure priorities. It sets out how investments and acquired services are prioritized, governed and delivered.

Facilities Inventory System:

real property inventory database and lease planning tool used to capture and maintain information on PWGSC-owned, leased and managed real property assets.

Fit-up:

any alteration or improvement to the base building and/or base building systems, including demolition, to prepare the accommodation for occupancy of the client.

Marketable office space:

a space that is suitable for occupancy by any federal or non-federal client/tenant.

National Capital Area (NCA) regional office:

NCA Portfolio Management Sector and the NCA Operations Sector.

National Investment Strategy:

was first approved in 1994 and was later updated for years 2000–05. (See Departmental Integrated Investment Plan for PWGSC's most recent efforts at prioritizing investments and acquired services). It outlined the strategic direction for real property investment activities and the office accommodation portfolio. The strategy established critical targets, objectives and guiding principles for the real property investment activities of PWGSC. It used a portfolio management approach whereby key properties served as a base from which the PWGSC portfolio could expand or contract according to changing requirements.

Non-reimbursing client:

PWGSC provides accommodation and accommodation services without direct charge to the client department. Accommodation and services are funded by a central reserve to which the Treasury Board allocates funding each time a submission including salary dollars is approved (see also Accommodation Levy). The amount of funding is determined using a default approach based on a percentage of approved salary dollars or by fully costing their space requirements. Client status is defined in the Occupancy Instrument.

Normal Business Hours:

7 a.m. to 6 p.m., Monday through Friday.

Occupancy Instrument (OI):

a document which records the rentable and usable areas, annual rental cost, the civic address and the planned general use of the particular parcel of accommodation that is the subject of an occupancy agreement. It also records any additional terms and conditions or variations from the general terms and conditions that govern the provision and occupancy of the accommodation, as agreed to by a department and PWGSC.

for reimbursing clients, the OI records the charges for the accommodation and accommodation services provided.

for non-reimbursing clients, the OI records the value of the accommodation and accommodation services provided without charge.

Occupancy Instrument System:

administers occupancy agreements (known as Occupancy Instrument) between PWGSC and its clients.

Office Accommodation:

includes workstation space, internal access, circulation and support space, and is primarily used to accommodate administrative and related functions, for example headquarters and regional offices.

PWGSC predecessor organization:

PWGSC was formed in 1993 when four departments (Public Works Canada, Supply and Services Canada, the Translation Bureau, and the Government Telecommunications Agency) were combined.

Present value cost of accommodation:

the initial capital, Operating & Maintenance, taxes, management and additional downstream costs associated with an accommodation project. It is used to measure the cost to the Crown of providing the accommodation over the investment horizon and is compared between alternatives with the same investment horizon.

PWGSC Region:

there are six  PWGSC operational regions: Atlantic; Quebec; the National Capital Area; Ontario; Western; and Pacific.

Reimbursing Client:

PWGSC provides accommodation and accommodation services at full charge to the client department. Client status is defined in the Occupancy Instrument. Reimbursing clients pay for their accommodations directly from their operating budget based on market rates, or if the space required (see special purpose spaces) exceeds their approved space use.

Space envelope:

total amount of rentable square metres a client department can occupy by fiscal year based on authorities provided to PWGSC by the Treasury Board for the provision of accommodation in accordance with standards, policies and guidelines.

Special purpose spaces:

additional, non-standard areas required by a department to accommodate specific activities which are unique and essential to departmental programs. Special purpose space requirements that exceed the level supported by PWGSC are funded by the client department.

Surplus real property:

property that is no longer required in support of a department's programs and that shall be sold or transferred within three years of formal notification of the property being surplus to program requirements.

Tenant Accommodation Demand System:

processes client requests for accommodation against predefined space standards.

Unsuitable space:

includes, but is not limited to, office space being fit-up for clients, buildings/offices under renovation, buildings in a non-usable condition and buildings that have been approved for disposal (surplus).

Main points

What was examined

i. This Evaluation examined the ongoing relevance and performance, for the period from April 1, 2007 to March 31, 2013, of the Federal Accommodation Program of the Real Property Branch (RPB). The Evaluation also assessed Program design and delivery.

ii. The Program is located in sub-program 1.2.1 of the Department's 2013–14 Program Alignment Architecture (PAA).

iii. The Federal Accommodation Program provides accommodation and related real property services to federal departments and agencies. The Treasury Board Common Services Policy and the Policy on Management of Real Property name Public Works and Government Services Canada (PWGSC) as the designated custodian of office facilities provided on an obligatory basis to departments and agencies, and give authority to PWGSC to set the standards for office accommodation. As of March 2013, the Department provided work environments for approximately 269,000 employees from over 110 federal organizations. The portfolio of Program assets managed by PWGSC is comprised of facilities owned by the Crown and facilities leased from the private sector, some with an option to purchase. In 2012–13, the Program accounted for $2,618 million in gross expenditures.

Why it is important

iv. The Government of Canada is committed to create a modern workplace that will attract, retain and encourage public servants to work smarter, greener and healthier to better serve Canadians. The Federal Accommodation Program directly supports this commitment through the provision of standards-based accommodations, stewardship of the federal real property portfolio, workplace modernization and integration of technologies. The Program accounts for approximately one-half of PWGSC's program spending.

v. The RPB has been tracking and reporting on the performance of its real property portfolio by using key performance indicators. The Program was subject to the Strategic Review whose results were presented in Budget 2011 and the Deficit Reduction Action Plan, whose results were reported in Budget 2012. The Federal Accommodation Program had not been subject to a formal evaluation in its entirety.

What we found

vi. The Program is relevant as it addresses the continuing need for the provision of accommodation to federal employees through an obligatory government-wide program.

vii. The Program is directly aligned with the delivery of government priorities and is consistent with the Government of Canada's direction to centralize government-wide administrative services for greater efficiency and service consistency. The Program is part of PWGSC's core business lines and as such is strongly aligned with the departmental strategic outcome.

viii. The federal government's role and responsibility to provide accommodation and related services to federal employees is appropriate and the centralized provision of accommodation is aligned with practices employed in other jurisdictions. Decentralization of the Program's responsibility to individual departments and agencies is possible however the majority of clients believe they can focus on their core mandate more efficiently with PWGSC providing centralized office accommodation services. The devolution of Program responsibility to other levels of government or the transfer of total responsibilities to the private sector is not deemed appropriate, as the ultimate accountability for ensuring accommodation standards are met rests with the federal government. However, the private sector can and has played a role in the delivery of accommodation services.

ix. The Program's Design and Delivery model is generally supporting the delivery of the Program. Various stakeholders, however, perceive the space envelope management feature of the Program's design and delivery as not being as transparent and efficient as it could be in managing accommodation costs for the Government of Canada and identified an opportunity to improve communication between clients and PWGSC on accommodation processes and costs. Additionally, various stakeholders reported on inconsistent service delivery arising from multiple points of contact with PWGSC.

x. The Evaluation determined that the Program is generally achieving its objectives. The Evaluation has identified areas of improvement that, once implemented, would support the Program's achievement of its immediate and long term outcomes.

xi. The Program, through its policy framework, processes and tools, provides office accommodations respectful of government standards and space utilization targets, while meeting client requirements, thus enabling them to focus on their core mandates. PWGSC is also taking additional measures to maximize space use (Immediate outcome 1).

xii. PWGSC has developed a clear and comprehensive methodology to guide Program investment decisions, which is aligned with industry practices. However, the Evaluation noted examples of inconsistent application of the methodology and insufficient integration of non-financial and risk factors to arrive at the best overall value decision for the Crown. The Evaluation identified two areas of improvement with respect to investment decision-making: ensuring the best value methodology is consistently applied by integrating qualitative factors into investment analysis and by sharing additional cost information and accommodation option rationales with client organizations (Immediate outcome 2).

xiii. The Program has some flexibility in managing its portfolio and responding to fluctuations in client demand through various ownership and transactional arrangements. Client representatives interviewed reported inconsistent planning advice from various PWGSC Regions, misunderstanding of client program requirements (e.g., specific to security or need for public access); lack of coordination between PWGSC real property teams; and weak corporate memory due to accommodation staff turnover. The Evaluation determined that the planning and approval phases of accommodation projects could benefit from being more responsive and that the Department is presently streamlining accommodation projects processes and delivery. Finally, the Program provides accommodations that are well-maintained. (Immediate outcome 3).

xiv. The Program has a framework to address on a priority basis health and safety issues and provides work environments viewed as safe. The Program continues to improve the affordability and sustainability of work environments through adopting benchmarked standards and industry-recognized best practices in environmental performance (Intermediate outcome).

xv. The Evaluation concluded that the Program manages its portfolio to preserve its integrity, usefulness and value. Maximizing the existing portfolio assets use, maintaining a low vacancy rate, increasing Crown-owned assets market value are the contributions noted by the Evaluation towards sound portfolio management. The Evaluation, however, determined that the development of a long-term strategy would further contribute to the achievement of the Program's long-term outcome (long term outcome).

xvi. The Program has adopted industry best practices and put mechanisms in place to improve its operational efficiency. These include Workplace 2.0, the National Project Management System and the implementation of a corporate real estate model.

xvii. The Program's participation in government cost-containment initiatives has contributed to government-wide economy.

Recommendations and management action plan

Recommendation: 1 The Assistant Deputy Minister, Real Property Branch, should examine the potential benefits to sharing additional information (i.e.: dollar value of accommodation investment options and occupied accommodation, occupied and unoccupied space envelopes where applicable) with Program clients.

• Management action plan 1.1: A Real Property Services Agreement is being developed with Canada Revenue Agency (CRA) to address the recommendations of a 3^rd party review of their quasi-reimbursing status. In the context of this agreement PWGSC and CRA will collaborate in portfolio planning processes, openly sharing information to promote mutual understanding of differing perspectives and priorities, in search of win-win solutions.
• Management action plan 1.2: Development and presentation to Real Property Management Committee (RPMC) of lessons learned and recommendations for other clients based on the CRA pilot project.

Recommendation 2: The Assistant Deputy Minister, Real Property Branch, should examine ways to achieve greater efficiency in the management of client contacts to foster client participation in support of the Program's objectives achievement.

• Management action plan 2.1: Client Consultancy and Real Property Solutions (CCRPS) meets monthly with its top 20 clients. Real Property Branch policies related to accommodation processes and costs will be provided and discussed as required. Clients will also be engaged in discussions on how to achieve efficiencies and value for money. Discussion concerning the consistent management of client contacts will be added as a standing item to the monthly agenda.
• Management action plan 2.2: Client Consultancy and Real Property Solutions holds regular teleconferences with networks of PWGSC Client Service Directors across the country. Discussion concerning the consistent management of client contacts will be added as a standing item to the agenda.
• Management action plan 2.3: Client Consultancy and Real Property Solutions will review and update client contact information regularly.

Recommendation 3: The Assistant Deputy Minister, Real Property Branch, should take steps to ensure that the best value methodology is consistently applied to account for financial, non-financial and risk considerations in investment analyses.

• Management action plan 3.1: Monitoring and oversight: existing investment analysis tools require the consideration of financial, non-financial and risks in all real property investment decisions. Revised Terms of Reference for Real Property Investment Board (RPIB) and Regional Investment Management Board (RIMB) will allow reviews and oversight of Regional Records of Decisions (RoD) by the National Centre of Expertise to ensure national consistency in the application of these tools.
• Management action plan 3.2: Monitoring functional reviews of regional project approvals and RoD will be conducted.
• Management action plan 3.3: Standardization of analytical tools: a national investment analysis tool and training material for investment analyses will be developed and released to replace several regional tools:
  1. Launch of REFIT 3.0
  2. Regional training

Recommendation 4: The Assistant Deputy Minister, Real Property Branch, should develop a long-term strategy for the Federal Accommodation Program and its portfolio of assets to align planning, investment and management of assets with their strategic value.

• Management action plan 4.1: A new Portfolio framework will be developed which will include a 5-year horizon National Portfolio Plan. The National Portfolio Plan will include strategies to respond to government operations and priorities, and effectively manage risk, including changes in demand, over a long-term horizon.

Introduction

1. This report presents the results of the Evaluation of the Federal Accommodation Program. The Program is situated in the 2013–14 Program Alignment Architecture (PAA) as sub-program 1.2.1. The Deputy Minister of Public Works and Government Services Canada (PWGSC) approved the conduct of this Evaluation, on recommendation by the Audit and Evaluation Committee, as part of the 2012–17 Risk-Based Audit and Evaluation Plan. The Evaluation was conducted in accordance with the Standard on Evaluation for the Government of Canada.

Profile

Background

2. PWGSC provides office accommodation to federal departments and agencies on an obligatory basis through the Federal Accommodation Program administered by the Real Property Branch. The Program (sub-program 1.2.1 of the PWGSC PAA) meets the accommodation needs of federal organizations through a portfolio of office facilities and common use assets. This Program is composed of two primary business lines: the provision, on an obligatory basis, of general-purpose accommodation and common use space to support the delivery of government programs (demand management); and optimization of real property investments in direct support of the provision of accommodation (supply management). Each of these business lines are supported by their respective policy and operation activities. The Evaluation assessed the achievement of the shared outcomes for these two business lines through the use of the logic model prepared for the Program.

3. As of March 2013, the Department provided work environments for approximately 269,000 employees from over 110 federal organizations under its Federal Accommodation Program. The Program's portfolio of assets is comprised of office and common-use facilities owned by the Crown and facilities leased from the private sector, some with an option to purchase. The size of the Program's accommodation portfolio fluctuates across years, as accommodation is removed from or added to the inventory. In March 2013, PWGSC managed a total of 6,454,743m² of office space.

4. The Federal Accommodation Program operates concurrently with other PWGSC programs and those of other government departments such as:

• PWGSC's Federal Holdings ( PAA 1.2.2) which manages the appraisals and the divestiture for all acquisitions, dispositions and transfers of real estate; and
• the Real Property Services ( PAA 1.2.3) which provides Real Property services, on a fee for service basis, to the Federal Accommodation and other Programs. The largest part of these services is delivered through Alternative Forms of Delivery (AFD) contracts with an external supplier, SNC-Lavalin O&M Inc; and
• Shared Services Canada (SSC), which became responsible for the provision of Information Technology (IT) services to 43 departments and agencies as a common service organization for the Government of Canada in August 2011.

5. The Federal Accommodation Program works collaboratively with these entities to provide modern workspaces for the Government of Canada. Furthermore, other Real Property Branch sectors share accountabilities for real property operations affecting the Program, such as client relationship management, management of real property Major Crown Projects, implementation of special initiatives, and corporate services.

Authority

6. The Department of Public Works and Government Services Act confers upon the Minister of Public Works and Government Services the authority to provide accommodation and related real property services to federal organizations. The Act grants the Minister the authority to undertake construction, maintenance and repair of public works, federal real property and immovables.

7. The Treasury Board Common Services Policy and the Policy on Management of Real Property name PWGSC as the designated custodian of office accommodation provided on an obligatory basis to departments and agencies, and give authority to PWGSC to set the standards for office accommodation.

Roles and responsibilities in Demand Management

8. Under Demand Management, PWGSC takes on the role of the provider of accommodation and accommodation services to federal departments and agencies. Demand Management refers to a series of policy and operational activities undertaken by the Program in support of analyzing accommodation needs and alternatives; negotiating and establishing occupancy agreements; monitoring performance and forecasting future accommodation demand. The Program is responsible for developing and implementingthe Framework for Office Accommodation and Accommodation Services, the Space Allocation Standards; the Fit-Up Standards and the Space Envelope Regime which guide these activities.

9. The Program is responsible for providing accommodation space to federal departments and agencies. In doing so, it is responsible for the provision of base building components and (as of 2012) Workplace 2.0 Fit Up Standards fit-up components to its client departments.

10. Client departments, however, are responsible for other costs related to the accommodation such as furniture, above-standard information technology and security systems, and moving costs. Client organizations are also responsible for identifying and providing timely and accurate accommodation requirements, planning, and budget information to PWGSC so that it can complete its Demand Management activities.

11. PWGSC characterizes its clients into two types: Reimbursing and Non-Reimbursing. Reimbursing clients are directly charged by PWGSC to fully recover the associated costs of the accommodation they receive as a result of their occupancy agreements with PWGSC. Non-reimbursing clients are not charged directly for the accommodation and accommodation services they receive through their occupancy agreements with the Department; PWGSC is funded by the Treasury Board for the accommodation it provides the client through its Space Envelope.

Roles and responsibilities in Supply Management

12. Under Supply Management, PWGSC takes on a stewardship role on behalf of the Crown. Supply Management refers to a series of policy and operational activities undertaken by the Program in support of planning the Real Property Portfolio; planning Asset and Building management; making decisions on real property investments and ensuring custodianship of assets and buildings. The Program is responsible for developing and implementing investment strategies (building management plans, investment analysis and decision-making boards, building condition reports) and making service requests to Real Property Services, among others. These activities are undertaken in support of deriving best value for the Crown from investments and to ensure a flexible, responsive and well-maintained federal Real Property Portfolio.

13. In the area of investment decision-making, the Treasury Board is responsible for establishing the government-wide administrative policy framework for real property and for approving transactions that exceed $36 million.

National and regional accountabilities in Demand and Supply Management

14. Accountabilities in the areas of Demand and Supply Management are distributed on both national and regional levels within the Department. PWGSC's Real Property Branch (RPB) acts as the corporate real property headquarters; however, the real property function is delivered through PWGSC regional offices and the National Capital Area.

National accountabilities

15. As the head of the RPB, the Assistant Deputy Minister is accountable to the Deputy Minister of PWGSC for the overall performance of the Program and the stewardship of PWGSC's real property portfolio. National accountabilities for the Program are shared between the Directors General of the National Accommodation, Portfolio Management and Real Estate Services Sector and the Program Management Sector.

16. The Program Management Sector exercises national leadership for real property issues within PWGSC, including Demand Management. It leadership holds accountability for establishing Program direction, objectives, and activity standards; managing the Program resourcing framework and funding protocols; developing the Program's policy management framework; interpreting and communicating Central Agency real property policies and strategic directions; and establishing a governance structure.

17. The National Accommodation, Portfolio Management and Real Estate Services Sector exercises national leadership for strategy development under Supply Management. This includes accountability for developing national strategies and frameworks; evaluating real property ownership options; and stewardship of Program assets. The Sector provides functional direction to the PWGSC Regions on all portfolio management matters.

Regional accountabilities

18. PWGSC's regional offices have local accountability for the delivery of the Program.

19. As the head of a PWGSC Region, the Regional Directors General are functionally accountable to the Assistant Deputy Minister of the RPB for the provision of accommodation and related real property services (Demand Management) and the stewardship of the real property portfolio (Supply Management) within their respective region.

20. As part of implementing a Corporate Real Estate Model within the Department, the National Capital Area was established as a real property region. The National Capital Area (NCA) regional office is positioned within the RPB and two  NCA regional offices. Directors General are accountable to the Assistant Deputy Minister of the RPB for a full range of Program services in the region and in Nunavut.

Resources

21. The Program's financial resources are managed through a combination of funding mechanisms: the Regular Operating Budget, used to pay for salaries and operating expenditures of the Program management function; the Special Purpose Allotment (net vote appropriation), used to pay for day-to-day operation and maintenance of portfolio assets; and the Capital Budget, used to renew and renovate Crown-owned office facilities. In addition, the Program has revenue spending authority to re-spend revenues collected in the Special Purpose Allotment mainly from rental payments made by federal clients and commercial tenants.

22. In 2012–13, the Program accounted for $2,618 million in gross expenditures. It is delivered by employees of the RPB and PWGSC Regions, with personnel costs of $70 million.

Logic model

23. A logic model is a visual representation that links a program's activities, outputs, and outcomes; provides a systematic and visual method of illustrating the program theory; and shows the logic of how a program is expected to achieve its objectives. It also provides the basis for developing the performance measurement and evaluation strategies, including the evaluation matrix.

24. A logic model for the Program was developed based on a detailed document review, meetings with Program management, and interviews with key stakeholders. It was subsequently validated with Program staff. The logic model is provided in Exhibit 1.

Exhibit 1: Logic model

Program activities

25. Program activities can be classified into two broad categories: demand management and supply management. Demand management is concerned with short and long-term planning of client accommodation requirements, allocation and provision of space to meet those requirements, and management of occupancy agreements. Supply management focuses on the operational and strategic management of federal real property assets in direct support of the provision of accommodation.

Demand Management

26. The PWGSC Framework for Office Accommodation and Accommodation Services integrates the strategic and operational aspects of managing clients' demand for accommodation. The framework is comprised of a policy, a strategic accommodation planning approach, Government of Canada fit-up standards for office accommodation, and a space envelope regime (i.e., management of the Program through space envelopes which represent the total space entitlements, in square metres, allocated to individual departments and agencies).

27. Upon receipt of client requirements, Program staff document and analyze client accommodation needs and examine alternatives. All exceptions to government accommodation standards undergo a review at the Accommodation Standards Committee. Once appropriate real property inventory is identified in support of a client request, the Program negotiates and establishes service and occupancy agreements with the client. Occupied accommodation is regularly monitored and reported through data integrity and quality assurance reports.

28. The Program undertakes strategic accommodation planning activities by developing client profiles and accommodation demand strategies, which allow the Program to gather key planning and demand information from both a short and long-term perspective.

29. Finally, accommodation and accommodation services provided under the Program are resourced in accordance with established funding protocols and funding transfers.

Supply Management

30. PWGSC is the custodian of program assets across Canada. The Program is resourced to renew the inventory of Crown-owned assets and fund client accommodations through established funding protocols and transfers.

31. The Department plans and manages its real property assets in a manner consistent with the principles of the National Investment Strategy, regional and community-based investment strategies, asset management plans, as well as strategic direction received from investment governing bodies (e.g., Real Property Investment Board, Regional Investment Management Boards).

32. In addition to planning at the portfolio level, the Program develops asset management plans that outline the strategy for managing an individual asset over its Program use and are supported by annual Building Management Plans. All investments made in the Program portfolio assets that exceed $1 million are included in the Departmental Integrated Investment Plan.

33. PWGSC provides custodianship for all Program assets and buildings by monitoring their strategic value and financial, operational and functional performance. This is achieved through the preparation of building condition reports, building performance reviews, assets performance monitoring reports, and other performance indicators.

34. To perform all of the above activities, PWGSC develops and adjusts policies and processes specific to the supply management aspect of the Program. These include the Investment Analysis Policy, the Asset Management Plan Policy, the Asset Performance Monitoring Policy, and related sustainability and heritage policies.

Focus of the evaluation

35. The objective of this Evaluation was to determine the relevance and performance of PWGSC Sub-activity 1.2.1 (Federal Accommodations) in achieving its expected outcomes stated in the Program's logic model in accordance with the Treasury Board Policy on Evaluation. The Evaluation also examined the Program's design and delivery, however, it did not assess the program's compliance with the policies and processes established for either the Demand or Supply Management business lines. The Evaluation assessed the Program for the period from 2007–08 to 2012–13, although data on certain indicators were not available for the entire five-year period.

36. An evaluation matrix—including evaluation issues, questions, indicators and data sources—was developed during the planning phase and used to guide the conduct of the Evaluation and the preparation of this report. Information on the approach and methodologies used to conduct this Evaluation, as well as limitations encountered in the planning and conduct of the project and risk mitigation measures taken by the Office of Audit and Evaluation, are located in the "About the evaluation" section at the end of this report.

Findings and conclusions

37. The findings and conclusions below are based on multiple lines of evidence collected during the Evaluation in support of the indicators developed to assess five core issues to be addressed in government evaluations (as per Annex A of the Treasury Board Directive on the Evaluation Function), as well as the additional issue of Design and Delivery. They are presented in this report in the following order: Relevance (Continuing need for the Program; Alignment with Government Priorities; and Alignment with Federal Roles and Responsibilities); Design and delivery; and Performance (Achievement of Expected Results; and Demonstration of Efficiency and economy).

Relevance

38. Relevance is the extent to which a program addresses a continuing need, is aligned with federal priorities and departmental strategic outcomes, and is an appropriate role and responsibility for the federal government.

Continuing need

39. Continuing need assesses the extent to which a program continues to address a demonstrable need and is responsive to client demand. Continuing need of the Federal Accommodation Program was assessed through: a review of the continued relevance of the original rationale; current legislative and policy requirements; and demand for services.

40. The rationale for the Program remains sound. The provision of accommodation and related real property services to federal employees was originally undertaken by PWGSC's predecessor organization, the Department of Public Works which was established by an Act respecting the Public Works of Canada in 1867. The Department's role in real property management evolved and in 1982 the Treasury Board Common Services Policy named the Department as the common service organization responsible for accommodation services and directed federal organizations to use the Department as their agent for acquiring office accommodation within Canada. In its role as the employer of the federal public service, the federal government continues to be required to provide work place accommodation to its employees and to ensure that provided accommodation protects employee health and safety, respects prescribed standards, and is accessible and barrier-free. Interviews conducted for the Evaluation with client departments and agencies, PWGSC management and Central Agencies have confirmed the continued relevance of the Program's original rationale and mandate.

41. At present, the Program is delivered under the authority of the Department of Public Works and Government Services Act, which provides the powers, duties, and functions to the Minister of Public Works and Government Services for the provision of office accommodation and other facilities to departments. The Treasury Board Common Services Policy and Policy on the Management of Real Property position PWGSC as the designated custodian of federal office accommodation, provided on an obligatory basis to departments and agencies, and grant PWGSC authority to set standards for office accommodation.

42. The Federal Accommodation Program is demand-driven and operates on behalf of other government departments and agencies in support of their programs and services. As of March 2013, the Federal Accommodation Program provided work environments for approximately 269,000 employees from over 110 federal organizations. The Evaluation found that the demand for Program services and core demand for accommodations is directly driven by the size of the public service due to the obligatory nature of the accommodation services. As long as the services remain obligatory, there is a continuing need for the Program.

Alignment with federal priorities and departmental strategic outcome

43. Program alignment with federal priorities and with the departmental strategic outcome was assessed through a review of the following: federal priorities related to Program outcomes and the Program design; and PWGSC's strategic outcome and priorities related to Program outputs and outcomes.

44. The Federal Accommodation Program supports government priorities by providing the necessary infrastructure (office and common-use facilities) to permit government program and service delivery. The Program's design and objectives are aligned with the Government of Canada's strategic direction to centralize government-wide administrative services for greater efficiency and service consistency. The Program has been leveraged to support a number of government initiatives: economic stimulus and deficit-reduction initiatives; the integration of environmental considerations outlined in the Federal Sustainable Development Strategy; compliance with government accessibility standards; and the Public Service Renewal initiative led by the Clerk of the Privy Council through workplace modernization and integration of technologies.

45. The Federal Accommodation Program is part of PWGSC's core business lines as per the Department of Public Works and Government Services Act and is aligned with PWGSC's departmental strategic outcome to deliver "high-quality services and programs that meet the needs of federal organizations and ensure sound stewardship on behalf of Canadians." The Program meets the need of federal departments and agencies by providing accommodation solutions for federal public servants and by using the best overall value for the Crown as a key principle in its investment decision-making process. Also, the Program directly supports PWGSC's organizational priority of Transforming Critical Infrastructure by implementing a major initiative to transform the management of the Department's physical assets.

Appropriate role and responsibility for the federal government

46. To determine if the delivery of the Program is an appropriate role for the federal government, three options were reviewed: decentralization to other federal departments and agencies, devolution of program responsibility to another level of government, and transfer to the private sector.

47. Under the current legislation and policy, the Minister of Public Works and Government Services is responsible and accountable for the provision of accommodation services to federal departments and agencies and for the administration of real property under PWGSC's custody.

48. A review of other jurisdictions indicated that centralized provision of accommodation services, delivered through a common service organization or a special operating entity, was most commonly employed in other jurisdictions (e.g., the United States, Finland, Ontario, Alberta, and British Columbia). However, two of the jurisdictions included in the review, Australia and the United Kingdom delegated the authority for managing accommodation to individual program departments and agencies. Of note, studies conducted in these two jurisdictions documented challenges such as ensuring uniform adherence across government organizations to accommodation policies and standards; the use of inappropriate accommodation arrangements; and inefficient use of space through high vacancy rates.

49. The Evaluation found that the majority of federal departments and agencies did not have the capacity to manage accommodation services. Less than one-third (30%) of client organizations surveyed reported that they had sufficient capacity and expertise to plan for and manage their own office accommodation requirements. Interviews with client representatives further confirmed clients' limited readiness to take on the role and responsibilities associated with the provision of accommodation to their employees, particularly among organizations that did not have custodial responsibility for real property assets or an internal real property function. Although a few custodial organizations interviewed indicated their readiness to manage their own accommodation services, recent studies, such as the 2012 Auditor General report on the Department of National Defence's real property management, the 2012 Evaluation report on Fisheries and Oceans Canada's assets management services, and the 2011 Audit report of Real Property Management at the Department of Foreign Affairs, Trade and Development Canada have documented significant deficiencies in the management of real property assets by departments.

50. Decentralization of the Program to other federal organizations may be of limited benefit to the federal government. According to interviews with clients and PWGSC management, the centralized provision of accommodation contributes to government-wide economies of scale and bargaining power and avoidance of potential duplication of resources. Specifically, the following benefits were named: establishment of a center of expertise in real property; achievement of economies of scale; whole of government view and understanding of horizontal issues; homogenous level of service and standards; consistent reporting and legal/contract management expertise; and potential for co-locating client organizations across facilities. Some interviewees also noted that the increasing complexity of portfolio management, including new accommodation arrangements and transactions (e.g., build-to-lease, lease-purchase, public-private partnerships) and service delivery models (e.g., outsourcing building services) required that expertise be consolidated within one institution. Finally, in the client survey conducted by the Evaluation, 70% (35 out of 50) respondents either agreed (25) or were neutral (10) on the statement that their department or agency could focus on its core mandate more efficiently with PWGSC providing centralized office accommodation services.

51. The Evaluation found that the transfer of responsibility for accommodation and related services to another level of government would not be appropriate. The responsibility to provide accommodation to federal employees cannot rest with the provinces, territories, or municipalities, which lack the legislative authority and mandate to provide administrative services to federal employees, ensure uniform application of accommodation standards across the country, or act as custodians of federal assets and land under the Federal Real Property and Federal Immovables Act.

52. In its role as the employer of the federal public service, the federal government is required to provide work place accommodations to its employees and to ensure that those accommodations comply with all applicable codes and regulations. The provision of accommodation services by PWGSC is obligatory under current federal legislation and policy. As the ultimate accountability for ensuring that accommodation for federal public servants respects the standards set out in the Framework for Office Accommodation and Accommodation Services rests with the federal government, accountability could not be completely transferred to the private sector. However, the private sector is currently engaged in many aspects of the provision of accommodation services to the Government of Canada.

Conclusions: Relevance

53. The Program is relevant as it addresses the continuing need for the provision of accommodation to federal employees through an obligatory government-wide program.

54. The Program is directly aligned with the delivery of government priorities and is consistent with the Government of Canada's direction to centralize government-wide administrative services for greater efficiency and service consistency. The Program is part of PWGSC's core business lines and as such is aligned with the departmental strategic outcome.

55. The federal government's role and responsibility to provide accommodation and related services to federal employees is appropriate and the centralized provision of accommodation is aligned with practices employed in other jurisdictions. Decentralization of the Program's responsibility to individual departments and agencies is possible however the majority of clients believe they can focus on their core mandate more efficiently with PWGSC providing centralized office accommodation services. l. The devolution of Program responsibility to other levels of government or the transfer of total responsibilities to the private sector is not deemed appropriate, as the ultimate accountability for ensuring accommodation standards are met rests with the federal government. However, the private sector can and has played a significant role in the delivery of accommodation services.

Design and delivery

56. Program design and delivery were examined as part of the Evaluation to support program improvements by helping to identify areas of strengths and weaknesses. The Evaluation did not review the pertinence of the Framework for Office Accommodation and Accommodation Services, including the implementation of accommodation standards, in the following analysis of the design and delivery of the Program. The Evaluation identified and analyzed three elements of the Program model which make it unique in comparison to models employed by other jurisdictions: in doing so, stakeholder perspectives on the design elements were collected and a number of program delivery issues were identified.

57. The Evaluation has identified three elements of the Program model which make it unique in comparison to models employed by other jurisdictions. The first element is the Space Envelope regime. The Program controls the use of space by client departments and agencies nationally through its notional space envelopes. Space envelopes are measured in square metres and are price protected from rental rate increases. The vast majority of client departments and agencies have a space envelope which was established based on their 2001 office holdings. Outside of the space envelope regime, certain clients (Human Resources and Skills Development Canada, Canada Revenue Agency, and various Special Operating Agencies) have signed special agreements with the RPB for their specific requirements and directly reimburse the Program for the full cost of accommodation.

58. When a client requires space that exceeds its space envelope, they obtain the Treasury Board's approval for an increase in the Program's appropriation to pay for additional accommodation or through an accommodation levy. Client organizations must pay for any space that exceeds the established accommodation standards and must receive approval from the RPB. Clients surveyed for this Evaluation reported that they were aware of their space envelope allocations, with 86% of clients surveyed agreeing that PWGSC kept them informed about the status of their space envelope and envelope adjustments. Interviews with clients departments and agencies further confirmed that clients frequently communicated with the National Accommodation Management team within the RPB on issues specific to space envelopes and were satisfied with provided updates. Conversely, many client and Central Agency respondents expressed concern regarding the transparency of the current space envelope approach in that the cost of accommodation is not immediately evident and suggested implementing some form of market-based rent to improve transparency. Some respondents also perceived that the current space envelope approach (which values all square metres of accommodation equally) as limited as it did not appear to address the quality, condition and location of accommodations.

59. The second unique Program element is that accountabilities for the provision of some elements of accommodation to federal public servants are shared between PWGSC and client departments and agencies. The two main areas of shared accountabilities between PWGSC and client organizations are the planning and funding of accommodation projects. Client organizations are responsible for identifying and providing PWGSC with timely and accurate accommodation requirements; planning and developing long-term accommodation strategies; and securing funding associated with incremental accommodation requirements and accommodation fit-up (preparing space for occupancy and tenants' requirements) among others. PWGSC is accountable for developing a generic framework of standards in support of clients' stated operational requirements and for providing accommodations which will satisfy those requirements at best value, based on an analysis of all feasible options.

60. When these accountabilities are not exercised in accordance with the Framework for Office Accommodation and Accommodation Services, the Program's service delivery (namely timelines and costs associated with individual accommodation projects) and relationships between PWGSC and client departments can be impacted. For example, there had been instances where clients did not account for the 13% accommodation levy in their funding requests, thus the client space envelope had not been adjusted. Clients had to pay for new space out of their budgets, which affected their program budgets. Interviews conducted for this Evaluation revealed that the majority of clients now have put processes in place to ensure that accommodation requirements are integrated from the onset of a program.

61. An important feature of the shared accountabilities is a split-funding arrangement for the costs related to fit-up of accommodation. In general, funding from the space envelopes managed by PWGSC covers the installation and costs of base building components and initial elements of the interior construction and finishes which are outlined in the 2012 fit-up standards technical reference manual. Client organizations are responsible for funding the purchase cost and installation of specialized requirements (e.g., specialized security or telecommunications systems), as well as furniture, fixtures, and equipment. Fit-up services for the Program are procured through PWGSC's Real Property Services teams or are provided by the clients.

62. For some clients, fit-up costs, although being a one-time investment, may put a significant pressure on their budget, particularly in situations when a decision to relocate a client is taken by PWGSC. Interviewed PWGSC management reported that split-funding was often an issue leading to delays and escalation on accommodation projects. A recent third-party review of the Program's accountability framework concluded that split-funding placed a financial and administrative burden on client organizations and may be increasing costs to the Crown through significant overhead, duplications in project management capacity and project delays, as clients' funds may not be available to proceed with the implementation of the accommodation solution. As an alternative, the third party review recommended implementing a 'tiered' approach to funding various levels of base fit-up requirements, as certain clients would always have increased operational requirements for specialized security, telecommunication systems or other fit-up elements. In 2011, Central Agencies instituted a reverse re-profiling initiative to aid client organizations with fit-up funding by allowing federal organizations to borrow for their share of accommodation fit-up costs against future reference levels. To date, seven organizations, mostly small agencies, have used the initiative to cover their fit-up expenses in the National Capital Area, Quebec and Western Regions.

63. The third unique element of the Program is its regional service provision administered under a national management regime with support from other PWGSC programs. The process employed in the Government of Canada to provide accommodation and related services is presented in the chart in Appendix A. Regional offices are responsible for providing services to client organizations and are supported by policies, directives and funding from various centres of expertise within the national headquarters (i.e., the RPB). Clients submit their space requirements to their corresponding region, which analyzes and identifies the best value option for the Crown and completes the approval process, which varies in the time required depending on the materiality and complexity of the project. Once the approval process is completed, the Program prepares the space for occupancy through services provided by the PWGSC Real Property Services Program.

64. Program management, staff members and client management interviewed reported that the roles and responsibilities of different PWGSC groups involved in accommodation delivery were unclear, resulting in multiple points of client contact and increasing the risk of inconsistent service delivery. The majority of clients interviewed (10 out of 14) identified inconsistencies in accommodation services and advice received from various PWGSC real property teams, including in the application of accommodation standards; inconsistencies in accommodation services and advice received from PWGSC were particularly apparent to the clients who indicated having centralized accommodation management within their organizations. Seventy-seven percent (40 out of 52) of clients surveyed reported centralized real property management within their organization. However, the extent of inconsistency in service delivery could not be confirmed by the Evaluation through other sources. While some interviewees acknowledged PWGSC's efforts to institute a one-stop shop client engagement function, others observed that it lacked decision-making authority and control over accommodation project processes. Furthermore, Program clients identified an opportunity to improve communication between clients and PWGSC on accommodation processes and costs.

Conclusions: Design and delivery

65. The Program's Design and Delivery model is generally supporting the delivery of the Program. Various stakeholders, however, perceive the space envelope management feature of the Program's design and delivery as not being as transparent and efficient as it could be in managing accommodation costs for the Government of Canada and identified an opportunity to improve communication between clients and PWGSC on accommodation processes and costs. As a result, the Evaluation has formulated Recommendation 1 to address this issue. In addition, various stakeholders reported on inconsistent service delivery arising from multiple points of contact with PWGSC: to mitigate this risk to the Program, the Evaluation has made Recommendation 2.

Performance

66. Performance is the extent to which a program is successful in achieving its objectives and the degree to which it is able to do so in a cost-effective manner that demonstrates efficiency and economy.

Outcome achievement

67. The Evaluation examined the degree to which the Federal Accommodation Program achieved its intended immediate, intermediate, and long-term outcomes. The intended outcomes of the Program are identified in italics below, followed by an assessment of the extent to which they have been achieved.

68. The Evaluation assessed the extent to which federal organizations received appropriate solutions for their accommodation requirements and priorities. The Evaluation defined appropriate solutions as accommodations that respect the Government of Canada standards for space allocation and fit-up and meet clients' program needs (defined as their operational requirements to administer their programming). Using multiple lines of evidence, the following was examined: existence of policies and tools for Program delivery; achievement of utilization targets set for Government of Canada office accommodation; and clients' perspectives on the provided accommodations.

69. As of March 2013, PWGSC provided work environments for approximately 269,000 employees from over 110 federal departments and agencies. PWGSC managed 3,885 occupancy agreements signed with client organizations and a portfolio totaling 6,454,743m² of Crown-owned, leased, and lease-purchased office and common-use facilities.

70. The Department has developed and periodically reviews its Framework for Office Accommodation and Accommodation Services, which as indicated previously, comprises a policy, directives, tools and standards. Maximum space allocations applied when planning, acquiring, and monitoring use of office accommodation are outlined in PWGSC's Space Allocation Standards. The space allocation and fit-up standards were updated in 2012 in support of the Economic Action Plan, and incorporated the Government of Canada Workplace 2.0 principles, which intend for more efficient and effective office space use through flexible and open layouts and collaborative technology. Program clients, PWGSC management, and Central Agency officials interviewed agreed that the Department had the necessary policy instruments in place for the provision of accommodation services. A literature review conducted for the Evaluation also confirmed that other public sector organizations commonly used policy instruments, such as accommodation management frameworks and space standards, to manage and control the use of their accommodation.

71. PWGSC has developed tools to better document client requirements and determine space allocations, such as the Client Accommodation Requirements Questionnaire that details client demand and the Workplace 2.0 Space Allocation Standards Calculator that provides high-level estimates for new fit-up in compliance with space allocation and fit-up standards.

72. In accordance with the Framework for Office Accommodation and Accommodation Services, the Department is funded to allocate office accommodation at the minimum level necessary to support the functional requirements of a client organization. As per the commitment made in Budget 2005, PWGSC reduced the level of office space utilization from 21.4m² per employee in 2005 to PWGSC's target of 18.9m² per employee in 2009 through the enforcement of standards. Analyses revealed that utilization rates did not significantly vary between non-reimbursing and reimbursing occupancies and various types of facilities (Crown-owned, lease-purchased and leased), indicating that there is uniform application of accommodation standards across funding and facility types. (Calculations for leased-purchase facilities excluded one client organization which was in the process of moving into that facility at the time of the Evaluation). As part of implementing the Government of Canada Workplace 2.0 initiative, PWGSC committed to further reducing the average level of office space utilization to 16.4m² per employee over time as space is acquired or renovated.

73. PWGSC has established a target of 70% for new occupancy agreements to meet the space allocation limits in support of the Framework for Office Accommodation and Accommodation Services. Accommodation standards may be exceeded for small offices of 25 or fewer people if the requirements are reasonable and cannot be met within the limits. Also, PWGSC may exceed maximum space allocations when renewing accommodation already occupied by a client if this maximizes the long-term economic advantage to the Crown and provides best value to the taxpayer. In the first year (2011–12), the Department achieved 67% compliance and achieved 78% compliance in 2012–13 (the remaining occupancy agreements contained non-compliant items). The 2012–13 result (78%) includes occupancies within maximum space allocations limits (20%) and renewed occupancies that exceeded exceed maximum limits, but represent the best overall value for the Crown (58%). The result for 2012–13 was affected by the implementation of the space allocation limits in response to the Budget 2012 Space Modernization Initiative. All client requests for exemptions to the space allocation limits and fit-up standards are reviewed by the Accommodation Standards Committee, in place since 2005, and signed off by the RPB Assistant Deputy Minister. Client representatives surveyed agreed that the exemptions process was clear (62% of respondents (16/26) and that decisions on requested exemptions were well communicated by PWGSC (63% of respondents (15/25).

74. The Evaluation found that 69% (35/51) of clients surveyed agreed that PWGSC provided accommodation that met their operational requirements, while 20% (10/51) disagreed. According to interviews conducted, clients named mainly the location of the accommodation, the state of the building and the difficulty to obtain accommodation costs estimates or forecasts as contributing factors for not fully meeting their operational requirements.

75. Interviewees across the Program stakeholder groups perceive the condition of Crown-owned space as inferior to leased space. PWGSC reports that all federal accommodations in Crown-owned, leased and lease-purchased facilities provided under the Program comply with applicable standards and codes while all health and safety issues are addressed on a priority basis.

76. The Evaluation assessed the extent to which best value for the Crown is obtained through investments. According to the Departmental National Project Management System Policy, best value is defined as the optimal balance of overall benefits to the Crown and Taxpayers by providing the best combination of quality, service and time at the lowest total cost over the useful life of the asset. To assess this outcome, the Evaluation examined PWGSC methodology for assessing best value accommodation solutions; the extent to which this methodology is applied; and Program stakeholder perspectives on the achievement of best value for the Crown through investments.

77. The Evaluation found that the Program developed a clear methodology for identifying the best value accommodation solutions. According to the methodology developed, investment analyses are required to present accommodation solutions based on financial, non-financial and risk considerations to determine the best value option for the Crown. The Guide for the Preparation of Investment Analysis Reports developed by PWGSC includes a matrix to capture the qualitative analysis. It also provides a template and detailed description of the style, purpose, and suggested content for each section of the Investment Analysis Reports and refers to federal and departmental policies, including the Treasury Board Policy on the Management of Projects, Departmental National Project Management System Policy, Departmental Investment Analysis Policy, and the Directive on Real Property Branch Obligations to Consider Public-Private Partnerships for projects with a capital expenditure of $100 million or more and an asset lifespan of at least 20 years. In addition, the Treasury Board requires the assessment of a full array of options, such as purchase, Crown construct, lease purchase, build to lease, new lease, lease renewal, sale leaseback, and public-private-partnership and other applicable options.

78. The Evaluation found that the current approach developed by the Program is aligned with industry standards and practices as other corporate real estate organizations also assess a variety of financial and non-financial factors as part of their investment analyses. A best practice would be to undertake post-mortem analyses of decisions taken by the Program to inform future decision-making. The Evaluation found that such analyses have been employed by private sector organizations.

79. The Evaluation found that most Program decisions were made by selecting the option with the lowest estimated Present Value Cost of Accommodation: from 2011–12 and 2012–13, 21 out of 23 projects approved by the Assistant Deputy Minister or higher authority were the lowest cost option. The two cases for which the lowest cost option was not retained were two Crown-construct projects where the construction was assessed as higher risk than the lease option, notably due to client requirements.

80. The Evaluation's file review indicates that decisions were made by selecting the option with the lowest cost. The Evaluation reviewed 12 project files and found that investment analyses considered a full range of options appropriate for the scope of the client's demand in a neutral manner which considered costs to the client. The review, however, found calculations for leased-purchase facilities excluded one client organization which was in the process of moving into that facility at the time of the Evaluation. Analysis matrices were found in a third of the files (4/12) and detailed analysis of the likelihood and significance of accommodation option risks in a quarter of files (3/12).

81. PWGSC Program management interviewees confirmed that decisions were often based on financial aspects represented by early estimates of possible scenarios. These respondents note that non-financial factors, such as strategic priorities, were not always taken sufficiently into account in individual investment analyses. The Evaluation found that the issue of insufficient weighting of the relative risks of accommodation projects has been noted by the Office of the Auditor General in its review of the management of leases of the department. Strengthening the assessment and integration of the qualitative factors into investment decision-making presents an opportunity for Program improvement.

82. Although all Program stakeholders interviewed acknowledged that PWGSC was guided by the priority to deliver the best value for the Crown and had the right policy and guidance tools in place, a majority of clients questioned PWGSC's decisions with regard to the chosen option for accommodation as they are neither engaged, nor informed about the results of analyses conducted by the Department. Clients commented that, after submitting their accommodation requirements through the Client Accommodation Requirements Questionnaire, their involvement in the process was limited to providing the funds necessary to implement a Program's accommodation solution chosen by PWGSC. Clients interviewed further observed that if more information about options under consideration were shared openly and transparently and/or if planning were undertaken jointly, they would have a greater support for the implementation of PWGSC's decisions.

83. The Evaluation assessed the extent to which the Department managed the Federal Accommodation Program in a way that ensures a responsive, flexible, and well-maintained federal real property portfolio. Using multiple lines of evidence, the Evaluation examined each of the following aspects of the Program's portfolio: flexibility, responsiveness, and maintenance. Flexibility was assessed through ownership arrangements, the availability of vacant space, and industry best practices. Responsiveness was assessed through client satisfaction, stakeholder perspectives and project timelines. Maintenance was assessed through client satisfaction, the achievement of performance targets and the benchmarking of operating and maintenance expenditures.

84. In recent years, PWGSC has implemented a series of initiatives to improve its ability to deliver the Program and ensure that the federal real property portfolio is responsive, flexible and well-maintained. The main initiatives include revising and enforcing the government accommodation standards; strengthening tools for completing investment analyses; implementing industry-leading project management practices and an integrated information technology system; diversifying the portfolio through new accommodation solutions; and, leveraging private sector expertise and capacity. A document and literature review confirmed that these initiatives were in line with industry-leading practices and reflected evolving labour arrangements and technological advances. Likewise, all Program stakeholders interviewed viewed the above-listed initiatives as a positive step towards ensuring a responsive, flexible and well-maintained real property portfolio.

85. As of March 2013, the Program provided accommodation to federal organizations in 1,795 locations across Canada in Crown-owned and leased facilities. Ownership arrangements of real property assets present an important aspect of portfolio flexibility. PWGSC uses a variety of ownership arrangements (Crown-owned or leased, some with an option to purchase) and transactional arrangements (sale-leaseback, public-private-partnerships and others) to manage its accommodation portfolio. As of March 2013, the portfolio consisted of 331 Crown-owned buildings (representing 47% of total rentable space in square metres), 1,455 locations under lease arrangements (representing 48% of total rentable space in square metres), and nine locations under lease-purchase arrangements (representing 5% of total rentable space in square metres). Benefits of using a mix of ownership and transactional arrangements to manage a real property portfolio, such as the ability to re-profile funds within the portfolio and the ability to manage risks relative to different arrangements, are well documented in the literature and were confirmed by interviews conducted for the Evaluation. As of March 2013, PWGSC managed 1,977 individual leases. When it comes to managing Program costs, leasing is not the most flexible option and it may expose the Program to market price and availability risks.

86. As PWGSC acquires accommodation across Canada, it makes decisions to ensure that the federal portfolio is flexible enough to accommodate space, location and occupancy date requested. One method by which the Program accommodates fluctuations in clients' business requirements is by incorporating options to renew or extend the lease into its contracts. An analysis of the Program's 2012–13 lease inventory revealed that 36% of leases had remaining lease extension options, with 37% of those having more than one option. Staggering the lease expiry profile of the portfolio is also used by the Program to increase portfolio flexibility.

87. Another aspect in ensuring flexibility in responding to client demand is through available vacant space. The Evaluation found that PWGSC maintained a low vacancy rate for its office space. As of March 2013, 1.3% (or 82,131m²) of Program space was vacant, which was significantly lower than the 8.5% average office vacancy rate reported across Canadian downtown and suburban markets. The Program's vacant space is contained in pockets across 168 facilities, with a space median of 240m² per facility. The vacant space could not accommodate urgent large demands, such as those resulting from a new federal organization creation, but could accommodate short-term occupancies. Short-term occupancies allow the Program increased flexibility in the management of fluctuations in client demand.

88. Sixty-three percent of clients surveyed (33/52) reported a high overall level of satisfaction with PWGSC office accommodation services. Departmental data on accommodation projects' scope, schedule and budget revealed that the majority of projects were consistent with their plan; out of the 3 indicators, the on time indicator was the lowest in 2011–12 at 86% (44/55 of projects managed within schedule) and at 80% in 2012–13 (47/59 of projects managed within schedule).

89. With respect to planning and forecasting clients' accommodation requirements, only 46% (23/50) of clients surveyed by the Evaluation agreed that PWGSC effectively performed those activities. Client representatives interviewed reported inconsistent advice on planning from various PWGSC Regions, misunderstanding of client program requirements (e.g., specific to security or need for public access); lack of coordination between PWGSC real property teams; and weak corporate memory due to accommodation staff turnover.

90. The timeliness of processing clients' requests for accommodation by the Program was identified as an area for improvement in client and Program staff surveys. The Evaluation found that the timelines to determine the best accommodation option and to prepare space for occupancy, which may include construction of a new facility or client relocation to a new building, were lengthy. The Program's accommodation project timelines require clients to identify and provide their requirements to the Program several years in advance of the occupancy date. Many client interviewees reported that this presented difficulty due to uncertainty in future program requirements and funding. Meanwhile, 43% of accommodation staff (23/54) surveyed reported frequently receiving information on client requirements at a sufficient level of detail and only 31% (17/55) reported receiving information in a timely manner. Both Program clients and PWGSC accommodation staff and management interviewed articulated a need for streamlining accommodation project approval processes and delivery, which are presently being addressed by the Department. Specifically, the two initiatives that are being reviewed are the Integrated Investment Planning process led by the Finance and Administration Branch and the National Project Management System process led by the Real Property Services Program.

91. Finally, although the current policy framework states that client departments and agencies are accountable for certifying accommodation demand and also for any changes to accommodation during their occupancy, many clients interviewed suggested the Program explore possibilities for early termination of occupancy agreements in situations where program requirements change, such as during the transitional times of the public service downsizing. Changing demand for accommodation (as a result of business reorganizations, corporate downsizing and outsourcing) has been well documented in research the literature reviewed by the evaluation yet implementation of immediate responses can be inhibited by the long-term nature inherent in the management of the real property sector. The flexibility of the Program discussed in the previous section mitigates some risk to the Program's responsiveness in this regard.

92. Surveys conducted with accommodation management in client organizations and federal public servants revealed that Program clients were generally satisfied with the day-to-day maintenance of provided accommodations. A 2010–11 national survey of federal employees commissioned by the Department revealed that 91% of employees reported some level of satisfaction with the services in their building, but called for certain improvements, namely in building temperature, air quality, and office layout. Additionally, 73% (38/52) of accommodation managers reported that PWGSC provided office accommodation that was in good condition and 63% (32/51) agreed that the provided accommodation was well maintained throughout their occupancy.

93. PWGSC provides a bilingual, 24-hour, 7 day a week National Service Call Centre, which records building maintenance requests placed by tenants and dispatches these requests to personnel on location. In 2011–12, satisfaction rating for service calls reported through the call centre was 87% (target established at 85%).

94. In 2011–12, 100% of the Program's facilities remained operational during normal business hours, and the 2011–12 Program data on operating and maintenance expenditures were comparable with the median for private and public sector facilities in two benchmarking exercises in which PWGSC participated.

95. The Evaluation assessed the extent to which the Program provides safe, affordable and sustainable work environments. Using multiple lines of evidence, the Evaluation examined safety through health and safety regulation monitoring and client perspectives; affordability through targets for costs to the Government of Canada and client perspectives; and sustainability through the adoption of best practices and the achievement of targets for environmental performance.

96. As the custodian of federal office accommodation, PWGSC ensures that the buildings and their systems and equipment adhere to applicable health and safety regulations. In order to manage health and safety risks due to the building condition, as well as operation and maintenance, the Program uses information from the Building Condition Reports and the Building Performance Reviews to develop Building Management Plans for Crown-owned facilities (annually) and for leased facilities (every two years). These documents include all scheduled projects for the Program's facilities, including health and safety projects.

97. The 2009 Report of the Auditor General of Canada on health and safety in federal office buildings examined PWGSC's policies and operations in mitigating health and safety risks to tenants. The report found that PWGSC's policies were clear and provided appropriate guidance to staff. However, the report also identified deficiencies in building management practices concerning health and safety issues. In response, PWGSC implemented a management action plan that included surveying its workplaces' compliance with health and safety regulations and updating its Building Management Plan process to ensure that all high priority deficiencies related to health and safety are addressed. The management action plan was fully implemented at the time the Evaluation was conducted.

98. Interviews conducted for the Evaluation with Program clients confirmed that the provided accommodations were generally perceived by clients as safe and compliant with applicable codes and standards. Available data on the number of critical accommodation-related health and safety incidents revealed that PWGSC was below its target of no more than five incidents annually.

99. Optimizing space utilization represents direct savings for the federal government and contributes to the affordability of work environments. For 2011–12, to help ensure and monitor efficient use of space, the Program introduced the performance indicator "percentage of occupancy agreements created during current fiscal year for more than 25 employees compliant with the framework" to measure the percentage of occupancy agreements complying with the Framework for Office Accommodation and Accommodation Services and set the target at 70%. In 2011–12, the Program achieved 67%, but surpassed the target in 2012–13 with 78%.

100. To increase the affordability of the federal work environment, the Program has taken measures to reduce, and contain the growth of, government footprint through the enforcement of accommodation standards. As a result, the average space per employee was reduced to 18.9m² per person in 2012–13 compared to over 21m² per person in 2004–05. Reducing space utilization represents a cost-containment measure and contributes to the affordability of the work environments, considering that the cost of building rent per square metre was $272 in 2007–08 and $321 in 2012–13.

101. From the client perspective, space is perceived to be generally affordable and clients recognize the advantages of the economies of scale generated by the Program. The Evaluation found that the Program has established partnerships with corporate real estate associations and public sector organizations and developed benchmarking standards and reporting on accommodation cost. The Program's results in the first two years of reporting were comparable to the median scores on cost indicators. The Evaluation was unable to comment on the extent to which the Program's accommodation costs were comparable to other real property organizations as the benchmarking studies included a wide variety of international and Canadian public and private sector organization and peer benchmarking groups were not defined.

102. Greening government operations is one of PWGSC's key priorities. In 2011–12, PWGSC developed a strategic framework for green buildings, which details how industry-recognized environmental performance assessment tools are integrated into construction and operation of Crown-owned and leased accommodation. The Program has adopted a number of best practices, such as conducting life-cycle assessments of all major building projects, incorporating green leases into business operations, using an integrated design approach and establishing performance benchmarks. The Program attained its targets with respect to the percentage of accommodation projects that have achieved an industry-recognized level of high environmental performance, but has not yet reached the established target with regards to the desired cumulative percentage of existing Crown buildings over 1,000m² that have been assessed using an industry-recognized tool based on a five-year assessment cycle.

103. The implementation of Workplace 2.0 is part of the Program's priorities and is expected to generate additional environmental benefits by modernizing the physical workspace with new flexible and sustainable office designs that integrate technology.

104. The Evaluation examined the extent to which the Department managed the Program to ensure that the integrity, usefulness, and value of the federal real property portfolio are preserved. Using multiple lines of evidence, the Evaluation examined each of the following aspects of the Program's portfolio: integrity was assessed by the extent of recapitalization efforts for life cycle building renewals and strategic and policy direction for the Program and its assets; usefulness was assessed by the extent of vacant space in the federal real property portfolio; and value was assessed through estimates of the market value of the federal real property portfolio and the return on investment for Crown-owned office assets.

105. The Program has begun meeting its recapitalization targets in recent years. The Department reported exceeding its target of 80% of the recapitalization budget spent in 2010–11 and achieved 96.4% in 2011–12. Capital expenditures per square metre have steadily increased since  2009–10 as a result of the Government of Canada's Accelerated Infrastructure Program, which enabled the Department to undertake some deferred repair projects. As a result, the facility condition index for Program assets improved to 10.6% in 2011–12 from 12% in 2007–08. Interviews conducted with PWGSC management and Central Agencies revealed that funding allocated for capital projects in the National Investment Strategy was not price protected, resulting in a lower recapitalization rate. Discussions between Central Agencies and PWGSC are currently underway to address this issue.

106. In terms of the strategic direction for the Program and its assets, PWGSC does not currently have a national long-term strategy in place. The National Investment Strategy was first developed in 1994 and was later updated for years 2000–05. It outlined the strategic direction for real property investment activities and the office accommodation portfolio. Presently, the 2012–17 Departmental Integrated Investment Plan represents PWGSC's most recent efforts to prioritize investments and acquired services in support of its Strategic outcome.

107. While discussing the national long-term strategy, it should be noted that uncertainty in client demand can impede the Department ability to manage its portfolio strategically. The recent Strategic Review, for example, has had an impact on both short- and long-term planning activities. As part of the Evaluation, program stakeholders indicated a gap in strategic and long-term planning for the Program and its portfolio of assets, as well as strategic investments in maintenance and recapitalization.

108. A few interviewees referenced the Long-Term Vision and Plan of the Parliamentary Precinct Branch as an example of a long-term strategy, which provides value by delineating a 25-year commitment implemented on rolling five-year programs of work. At the regional level, the Program Community-Based Investment Strategies which provide a long-term investment plan for assets within the community portfolio and establishes a direction for regional portfolio planning. It is a demand/supply management plan that combines government priorities, client plans, inventory management and investment priorities in the context of the community portfolio of assets over a specified period of time, usually ten years. Regional investment plans were to be in compliance with the National Investment strategy. There is not currently a national long-term strategy to which they are aligned.

109. With respect to the policy direction for the Program, the Department has begun reviewing its suite of policy instruments specific to the provision of accommodation and related real property services. PWGSC management in the RPB and regional offices interviewed highlighted the positive work undertaken by the Policy Steering Committee in coordinating policy updates across various policy responsibility centres. The core Program documents, the Space Allocation Standards and the Fit-Up Standards technical reference manual were updated in 2012.

110. The Evaluation examined the extent of vacant space in the federal real property portfolio and found 1.3% of marketable office space vacant as of March 2013, which is below the 3.5% Program target and below the 8.5% vacancy rate reported by the private sector in Canadian markets. Vacancy rate was lower in leased and lease-purchased facilities at 0.4% and was higher within Crown-owned building at 2.4%. In addition to vacant marketable space, 5.6% of the Program's portfolio (358,729m²) was unavailable or unsuitable space for occupancy as of March, 2013. This included space in buildings identified as surplus real property and space under maintenance or preparation for occupancy.

111. The market value of Crown-owned assets has increased over time, with an exception in 2008–09 due to an economic recession. In 2011–12, the average national market value of PWGSC's Crown-owned office portfolio reached $1,805 per square metre, representing a 6.5% increase from the previous year. Although the return on investment for Crown-owned office assets was 3.7% and was below PWGSC's target and industry levels, the Program's mandate as a Government of Canada program is to assure the best overall value for the Crown, rather than to seek a financial return on its assets.

112. Turning to an overall view on the Program's contribution to its long term outcome, it is important to note that the implementation of the Workplace 2.0 initiative was viewed by all Program stakeholders interviewed as a positive direction to achieving the integrity, usefulness and value of the federal real property portfolio. The initiative integrates the latest developments in labour and workplace arrangements, consistent with practices in the private sector and other jurisdictions. These include flexible work arrangements, integration of new information technologies, collaborative work environments and a reduced accommodation footprint.

Conclusions: Outcome achievement

113. The Evaluation determined that the Program is generally achieving its objectives. The Evaluation has identified areas of improvement that, once implemented, would support the Program's achievement of its immediate and long term outcomes. Areas for improvement are identified below by recommendation number.

114. The Program is providing appropriate solutions to meet client requirements and priorities, through its policy framework, processes and tools, office accommodations that respect government standards and space utilization targets and meeting client requirements, thus enabling clients to focus on their core mandates. PWGSC is also taking additional measures to maximize space use. (Immediate outcome 1).

115. PWGSC has developed a clear and comprehensive methodology to guide Program investment decisions, which is aligned with industry practices. However, the Evaluation noted examples of inconsistent application of the methodology and insufficient integration of non-financial and risk factors to arrive at the best overall value decision for the Crown. The Evaluation identified two areas of improvement with respect to investment decision-making: ensuring the best value methodology is consistently applied by integrating qualitative factors into investment analysis (Recommendation 3) and by sharing additional cost information and accommodation option rationales with client organizations (Recommendation 1) (Immediate outcome 2)

116. The Program has some flexibility in managing its portfolio and responding to fluctuations in client demand through various ownership and transactional arrangements. The Evaluation determined that the planning and approval phases of accommodation projects could benefit from being more responsive in processing client requests. As the Department is presently streamlining accommodation projects processes and delivery, the Evaluation did not determine a need for a recommendation with regard to responsiveness in processing client requests. Finally, the Program provides accommodations that are well-maintained. (Immediate outcome 3)

117. The Program has a framework to address, on a priority basis, health and safety issues and provides work environments viewed as safe. The Program continues to improve the affordability and sustainability of work environments through adopting benchmarked standards and industry-recognized best practices in environmental performance. (Intermediate outcome)

118. The Program manages its portfolio to preserve its integrity, usefulness and value. Maximizing the existing portfolio assets use, maintaining a low vacancy rate, increasing Crown-owned assets market value are the contributions noted by the Evaluation towards sound portfolio management. The Evaluation, however, has determined that the development of a long-term strategy would further contribute to the achievement of the Program's long-term outcome (Recommendation 4).

Efficiency and economy

119. Demonstration of efficiency and economy is defined as an assessment of resource utilization in relation to the production of outputs and outcomes. A program has high demonstrable efficiency and economy when resources maximize outputs at least cost and when there is a high correlation between minimum resources and outcomes achieved.

120. Efficiency refers to the extent to which the Program optimizes the use of its resources. The efficiency of the Program was assessed by examining the extent to which the Program integrated industry best practices in Program management and delivery. This was assessed through an analysis of operational data, benchmarking with other organizations, a review of occupancy agreements and Program stakeholder perspectives.

121. The Program has put some mechanisms in place to measure its efficiency and adopt industry best practices to improve its performance. PWGSC is a member of numerous real property associations and undertakes studies to assess accommodation models in other jurisdictions. Workplace 2.0, the National Project Management System, and the adoption of a Corporate Real Estate model are key examples of how the Department aligns the Program with industry best practices. Furthermore, the Department is implementing a project to adopt industry-standard real estate business processes supported by an integrated information technology system (NOVUS). NOVUS was scheduled for launch in April 2014 and will coincide with the introduction of other initiatives such as industry recognized floor measurements (Building Owners and Managers Association standards). It will reduce the number of tools used for managing and reporting on various Program activities.

122. A literature review conducted for the Evaluation confirmed that the initiatives noted above were in line with industry-leading practices and reflected evolving work arrangements and technological advances. Likewise, all Program stakeholders interviewed viewed recent initiatives as a positive step; however, they identified some challenges emanating from their implementation. PWGSC management and accommodation staff, as well as representatives from client organizations noted the extensiveness of project documentation required by the introduction of the National Project Management System in 2010 and the length of the project management process, viewing them as disproportionate to the level of risk for low-materiality and space renewal projects. Furthermore, while the majority of PWGSC interviewees considered the Department's Integrated Investment Plan to be an important step towards risk-based management and approval of projects, they suggested that its implementation had resulted in duplications with existing processes for real property projects and increased project timelines and indirect costs. At the time of the Evaluation, PWGSC was engaged in reviewing and streamlining its requirements for both the National Project Management System and the Integrated Investment Plan.

123. A review of occupancy agreements conducted for the Evaluation found that a significant proportion of new occupancy agreements were signed for the duration of less than one year. While this creates flexibility in terms of space commitment, it can create an administrative burden in terms of managing the agreements. To optimize Program space and minimize costs, the Department embarked on several initiatives in recent years aimed at streamlining the management of occupancy agreements. The Program has reduced the number of occupancy agreements with clients from 4,030 in 2008–09 to 3,885 in 2012–13, while increasing the number of public servants accommodated under the Program by approximately 20,000.

124. Economy refers to minimizing the use of resources. Economy is said to have been achieved when the cost of resources used approximates the minimum amount of resources needed to achieve expected outcomes. Economy was assessed through a review of the Program's budget and documentation, and interviews with Program stakeholders.

125. As part of Budget 2005, PWGSC's real property budget was reduced by $300 million per year ongoing from 2009–10. There was an additional $500 million reduction experienced through increases in the space utilization rate and enforcing government allocation standards. The Program reported achieving its targets to date.

126. In support of the public service renewal initiative and the government's 2012 deficit reduction efforts, the Program further lowered the maximum space allowances, which are expected to generate $65.6 million in savings over a seven-year period, as well as an additional $62 million as a result of reduced government accommodation ecological footprint, such as lower electricity use and reductions in greenhouse gas emissions. Reporting on these savings will commence in 2014–15. In addition to the savings associated with the reduced footprint, the implementation of the workplace modernization initiative (Workplace 2.0) is also expected to lead to additional financial savings and environmental benefits.

Conclusions: Efficiency and economy

127. The Program has adopted industry best practices and put mechanisms in place to improve its operational efficiency. These include Workplace 2.0, the National Project Management System and the implementation of a corporate real estate model.

128. The Program's participation in government cost-containment initiatives has contributed to government-wide economy.

Recommendations and management action plan

Recommendation 1: The Assistant Deputy Minister, Real Property Branch, should examine the potential benefits to sharing additional information (i.e.: dollar value of accommodation investment options and occupied accommodation, occupied and unoccupied space envelopes where applicable) with Program clients.

• Management action plan 1.1: A Real Property Services Agreement is being developed with Canada Revenue Agency (CRA) to address the recommendations of a 3^rd party review of their quasi-reimbursing status. In the context of this agreement PWGSC and CRA will collaborate in portfolio planning processes, openly sharing information to promote mutual understanding of differing perspectives and priorities, in search of win-win solutions.
• Management action plan 1.2: Development and presentation to Real Property Management Committee (RPMC) of lessons learned and recommendations for other clients based on the CRA pilot project.

Recommendation 2: The Assistant Deputy Minister, Real Property Branch, should examine ways to achieve greater efficiency in the management of client contacts to foster client participation in support of the Program's objectives achievement.

• Management action plan 2.1: Client Consultancy and Real Property Solutions (CCRPS) meets monthly with its top 20 clients. Real Property Branch policies related to accommodation processes and costs will be provided and discussed as required. Clients will also be engaged in discussions on how to achieve efficiencies and value for money. Discussion concerning the consistent management of client contacts will be added as a standing item to the monthly agenda.
• Management action plan 2.2: Client Consultancy and Real Property Solutions holds regular teleconferences with networks of PWGSC Client Service Directors across the country. Discussion concerning the consistent management of client contacts will be added as a standing item to the agenda.
• Management action plan 2.3: Client Consultancy and Real Property Solutions will review and update client contact information regularly.

Recommendation 3: The Assistant Deputy Minister, Real Property Branch, should take steps to ensure that the best value methodology is consistently applied to account for financial, non-financial and risk considerations in investment analyses.

• Management action plan 3.1: Monitoring and oversight: existing investment analysis tools require the consideration of financial, non-financial and risks in all real property investment decisions. Revised Terms of Reference for Real Property Investment Board (RPIB) and Regional Investment Management Board (RIMB) will allow reviews and oversight of Regional Records of Decisions (RoD) by the National Centre of Expertise to ensure national consistency in the application of these tools.
• Management action plan 3.2: Monitoring functional reviews of regional project approvals and RoD will be conducted.
• Management action plan 3.3: Standardization of analytical tools: a national investment analysis tool and training material for investment analyses will be developed and released to replace several regional tools:
  1. Launch of REFIT 3.0
  2. Regional training

Recommendation 4: The Assistant Deputy Minister, Real Property Branch, should develop a long-term strategy for the Federal Accommodation Program and its portfolio of assets to align planning, investment and management of assets with their strategic value.

• Management action plan 4.1: A new Portfolio framework will be developed which will include a 5-year horizon National Portfolio Plan. The National Portfolio Plan will include strategies to respond to government operations and priorities, and effectively manage risk, including changes in demand, over a long-term horizon.

About the evaluation

Authority

The Deputy Minister for Public Works and Government Services approved the conduct of this Evaluation, on recommendation by the Audit and evaluation Committee, as part of the 2012–17 Risk-Based Audit and Evaluation Plan.

Evaluation objectives

The Evaluation examined the Federal Accommodation Program, delivered by the Real Property Branch and PWGSC regional offices. This Evaluation had two objectives:

• To determine the relevance of the Program: the continued need for the Program, its alignment with governmental priorities and alignment with federal roles and responsibilities;
• To determine the performance of the Program: the achievement of its expected outcomes and a demonstration of the efficiency and economy.

Approach

The Evaluation was conducted in accordance with the Standard on Evaluation for the Government of Canada. The Evaluation took place between April 2012 and September 2013 and was conducted in three phases: planning, examination and reporting. The Evaluation covered Program performance from 2007–08 to 2012–13, although data on certain indicators were not available for the entire five-year period. To assess evaluation issues and questions, the following lines of evidence were used.

Document review: An initial document review provided an understanding of the Program and its context to assist in the planning phase. Documents reviewed included documents provided by the Program, as well as documents written about the Program. Documents reviewed included Speeches from the Throne, federal budgets, legislation, policy documents, requests for funding, departmental corporate documents (such as annual Reports on Plans and Priorities and Departmental Performance Reports), Real Property Branch documents (Our Portfolio—Our Story annual reports, Branch Business Plans) and Program documents. All documentation was reviewed to examine Program relevance, design and delivery, and performance.

Literature review: A literature review was conducted to: contextualize the Program both nationally and internationally; provide theoretical background for the Program model.

Interviews: The Evaluation team conducted key informant interviews with various stakeholder groups involved in the provision of office accommodation to federal employees, including Program management, client departments and agencies, and Central Agencies (the Treasury Board Secretariat and the Department of Finance). Twenty three (23) Program management representatives were interviewed for the purposes of the Evaluation, including 14 from the RPB and nine from six  PWGSC regional offices. In addition, interviews were conducted with accommodation management representatives from 14 client departments and agencies and six Central Agency officials. The Evaluation team identified a list of potential interviewees, which was validated by the Program. All of the individuals who were asked to participate in an interview accepted the invitation. Interviews in the National Capital Area were conducted in person and interviews with PWGSC regional representatives were conducted by telephone. Semi-structured interview guides were used and all interview notes were transcribed. Themes and explanations emerging from the transcripts were mapped against the evaluation matrix. The qualitative analysis of the interviews provided information about the Program's relevance, design and delivery, and performance.

Surveys: The evaluation team developed two electronic surveys to capture Program clients' and PWGSC accommodations space coordinators perspectives on the delivery and effectiveness of office accommodation services provided through the Program. The survey questionnaire included a mix of dichotomous (yes/no), multiple-item scale and open-ended questions. For the client survey, invitations were sent to the clients' national coordinators of space envelopes. Fifty two (52) client organizations completed the survey, which represents a response rate of 56%. Client organizations however had the choice of what questions they wished to answer. For the PWGSC accommodation staff survey, invitations were sent to all client accommodation services staff across PWGSC Regions. Fifty eight (58) responses from staff were received, for a response rate of 68%.

Operational data analysis: To assess the achievement of Program outcomes, data on office accommodation inventory and space use were collected from PWGSC's real property applications, namely from the Facilities Inventory System, Tenant Accommodation Demand System and Occupancy Instrument System. Data on PWGSC's accommodation project delivery specific to on-time, on-scope and on-budget indicators were extracted from SIGMA, PWGSC's financial management and material management system.

Secondary data: Sources of secondary data included tenant and client satisfaction questionnaires; independent studies conducted by external third-parties and internal audit; and corporate reporting data from PWGSC's Departmental Scorecard, Performance Measurement Framework and Departmental Performance Reports.

File review: A file review was conducted to assess the consistency and quality of investment analyses for office accommodation projects and to examine compliance with National Project Management System's processes and principles. A total of 12 accommodation project files from PWGSC Regions and the Major Crown Projects Sector were selected for review by the evaluation team. A structured assessment checklist was used to systematically extract and compile the data on all projects. File review data were used to assess Program performance, and to corroborate interview and document review data.

Financial analysis: Program budget and expenditures data, as well as data on office space savings were provided by the Program and were analyzed to examine Program efficiency and economy.

Limitations of the evaluation

The methodology used for this Evaluation is subject to potential limitations as identified below. The key approach to reducing the limitation of certain methodologies was to ensure that the evaluation team triangulated all data by using a multiple-lines-of-evidence approach.

Document review: Data contained in the documents reviewed were not always up-to-date (for example, did not include fiscal year 2012–13) or included data collected at the real property portfolio level rather than at the Program level. To minimize these limitations, the evaluation team used most recent data from PWGSC's real property applications specific to the Program.

Literature review: The literature review of office accommodation models was limited to publicly available documentation on accommodation and portfolio management practices in other jurisdictions examined by the Evaluation. Interviews with representatives from other jurisdictions were not conducted to validate the findings and/or to provide additional context. Overall, the Evaluation found sufficient level of detail on every jurisdiction through information available on the jurisdictions' official websites, as well as information contained in various independent third-party studies, reviews and audits.

Interviews: The evaluation team used purposive (judgmental) sampling to identify relevant individuals and ensure good representation from all relevant stakeholder groups. Furthermore, Central Agency analysts responsible for the real property portfolio were not extended invitations to be interviewed. Interview guides for all stakeholder groups were developed in accordance with the evaluation matrix. Interviewer bias was mitigated by using a semi-structured interview guide and ensuring that two evaluation team members attended every interview.

Surveys: The relatively small number of respondents (58 and 52 for staff and client surveys, respectively) limited the extent to which comparisons could be made between different groups within the surveys (for example, across PWGSC Regions, or by the type of client).

Operational data analysis: The evaluation team requested Program analysts to extract data concerning office accommodation inventory and office space use from several PWGSC's real property applications. The responsibility for real property database integrity and the quality of data rests with the RPB. The Branch developed a procedure that aims at maximizing the quality of the information gathered and reported, in terms of relevance, accuracy, reliability, transparency, credibility and integrity. For these reasons, the evaluation team did not conduct an assessment of data's quality.

Secondary data: Analysis of secondary data was limited by the availability and relevance of reported data and was used to support other methodologies.

File review: The file review was limited by a small sample size of 12 accommodation projects (six projects complying with National Project Management System light procedures—less than 3,000m² in region or 5,000m² in the National Capital Area, six projects complying with full National Project Management System procedures, including two Major Crown Projects). The projects were selected by the evaluation team based on several criteria (such as regional representation, project size and occupancy start date). Findings from the file review were used to corroborate qualitative data from interviews and surveys, instead of generalizing findings.

Reporting

Findings were documented in a Director's Draft Report, which was reviewed by the Office of Audit and Evaluation's Quality Assessment function. The Program's Directors General were provided with the Director's Draft Report and a request to validate facts and comment on the report. A Chief Audit and Evaluation Executive's Draft Report was prepared and provided to the Assistant Deputy Minister, Real Property Branch, for acceptance as the Office of Primary Interest. The Office of Primary Interest was requested to respond with a Management Action Plan. The Draft Final report, including the Management Action Plan, was presented to PWGSC's Audit and Evaluation Committee for the Deputy Minister's approval in March, 2014. The Final report will be submitted to the Treasury Board Secretariat and posted on the PWGSC website.

Project team

The Evaluation was conducted by employees of the Office of Audit and Evaluation, overseen by the Director of Evaluation and under the overall direction of the Chief Audit and Evaluation Executive. The Evaluation was reviewed by the Quality Assessment function of the Office of Audit and Evaluation.

Appendix A: Accommodation and related services process chart