Office of Audit and Evaluation Review of the Alternative Forms of Delivery service provider's management review of fiscal year 2009-10 construction, design and security costs

2009-10 Final report (PDF, 255KB)

June 23, 2014

Main points

What we examined

1. As part of the Management Action Plan that was developed in response to the PricewaterhouseCoopers' (PwC) 2011 audit, Forensic Audit of Seventy Transactions Under the SNC-Lavalin O&M Contracts, the Alternative Forms of Delivery (AFD) service provider was responsible for conducting a management review of 100% of the third party costs charged to Public Works and Government Services Canada (PWGSC) in fiscal year 2009-10. In addition, as part of the Management Action Plan requirements, this management review was to be provided to PWGSC's Real Property Branch (RPB) and Office of Audit and Evaluation (OAE) to perform a comprehensive review on the service provider's management review.
2. The AFD service provider has provided its report of its management review and supporting documentation to RPB and the OAE. The review included all eight portfolios in which the AFD service provider provides services for the maintenance and upkeep of buildings. These 8 portfolios correspond directly to the 8 contracts awarded by PWGSC to the AFD service provider in 2004. Costs reviewed consisted of the following three categories: (1) design; (2) construction; and, (3) security. The OAE, which undertook a comprehensive review of the report, analyzed the report's findings, and performed sample testing of 80 projects from fiscal year 2009-10, taking ten from each of the eight portfolios. Sample testing was conducted to assess the accuracy of the AFD service provider's reconciliation of the expenditures for design, construction and security costs for the fiscal year 2009-10 billed to PWGSC.
3. The ten projects selected for each portfolio included 5 projects chosen from those identified by the AFD service provider in their management review as containing variances between the amount billed to PWGSC and the amount incurred by the AFD Service provider, and 5 identified by the AFD service provider as not containing variances. The OAE validated these sample projects with the AFD service provider's results by reviewing source documents for each project along with the amount charged to PWGSC.

Why it is important

4. The OAE's work was directed at providing independent validation of the results of the AFD service provider's management review of 100% of the third party costs billed to PWGSC in 2009-10 in three categories. Independent validation of the accuracy and completeness of the AFD service provider's results was essential to help mitigate any risk of understatement of variances by the AFD service provider. It is important that PWGSC is correctly billed in accordance with the contracts for work completed by the AFD service provider to ensure appropriate stewardship of taxpayers' money.

What we found

5. In the report provided to PWGSC, the AFD service provider indicated that it had identified $1,169,346.42 in net variance between what it had billed PWGSC and the actual costs the AFD service provider had incurred. This total variance was across the three cost categories, and represented a net overbilling rate of 0.54% of the total amount billed to PWGSC.
6. The OAE obtained a list of construction, design, and security costs charged to PWGSC for fiscal year 2009-10 from the AFD service provider, which the AFD service provider had used to prepare its report. In order to ensure the AFD service provider's report list of costs was accurate and complete, the OAE reconciled the AFD service provider's list with amounts reported in RPB's tracking system known as the Alternative Forms of Delivery Management System. No discrepancies were identified.
7. The OAE then selected a sample of 80 projects from the list, taking ten projects from each of the eight portfolios. Five of the projects selected from each portfolio had been indicated by the AFD service provider as containing variance in the amounts incurred by the AFD service provider and what was billed to PWGSC, and the other five had been indicated by the AFD service provider as containing no variance by the AFD service provider. The source documents and supporting documentation were reviewed, and payments to the appropriate subcontractors were validated. Through the review of the documentation for the 80 projects the OAE was able to validate each of the AFD service provider's conclusions without any discrepancies. As the comprehensive review work performed by the OAE found no discrepancies with the AFD service provider's list of projects assessed, and no discrepancies with the 80 projects reviewed, we found the AFD service provider's management review was accurate and complete.
8. Based on the list of construction, design, and security costs charged to PWGSC, the OAE calculated the total amount of overbilling to be $1,499,105.60, and the total amount of under-billing to be $329,759.18 as indicated in Appendix A. The net of these two amounts, $1,169,346.42 represents 0.54% of the $217,909,545.89 billed to PWGSC for fiscal year 2009-10.
9. The AFD service provider was of the opinion that any recovery of overbilling should take into account the under-billing which would result in a total reimbursement to PWGSC of $1,169,346.42. RPB, in consultation with Legal Services and Acquisitions Branch, concluded that the recovery of net overbilling was an appropriate resolution. The AFD service provider provided a cheque to RPB in amount of $1,169,346.42.
10. As a result of the fact that net overbilling was identified in fiscal year 2009-10, it is likely that net overbilling occurred during the period of 2005-06 to 2008-09 and during 2010-11. We were informed by RPB that the AFD service provider has conducted a management review for the 2010-11 year using the same methods as its review of the 2009-10 fiscal year, and RPB has engaged the OAE to analyze the report's findings and validate its results. Identification of net overbilling that likely had taken place for years 2005-06 to 2008-09 would be achieved either through extrapolation of the average error rate across the 2009-10 and 2010-11 years' expenditures, or through the comprehensive review of 100% of expenditures for all the years covered by the contracts. The OAE was informed by RPB that the Branch intends to recover the expected net overbilling for fiscal years 2005-06 to 2008-09 through the extrapolation approach.

Management response

The Real Property Branch (RPB) accepts the report and is working with the appropriate stakeholders to implement actions in a timely manner.

Following the receipt of the AFD service provider's review of the 2009-10 third-party costs, the Branch has recovered $1,169,346.42 representing the net variance between under-billing and overbilling. Given the materiality of the 2009-10 results, the AFD service provider was informed that further reviews of the three costs categories for all the years until 2010-11 needed to be done.

Subsequent to the instructions provided at the June 6, 2013 Audit and Evaluation Committee (AEC) meeting to recover the gross amounts of overbilling instead of the net variance, the Branch proceeded with the recovery of extrapolated amounts of gross overbilling for the remaining years up to and including 2010-11 by setting off these amounts from the monthly invoices starting in July 2013 with the last set-off occurring in November 2013.

Following the June 6, 2013 AEC meeting, the issue of the reimbursement of net versus gross amount was examined. Recovery of net overbilling is appropriate because reconciliation of an account pursuant to an audit is not a new claim under the contract, but rather the discovery of an existing discrepancy. Discrepancies can be considered latent errors requiring correction, in which case PWGSC would not be entitled to withhold the underbilled amounts. Therefore, allowing the deduction of the underbilling from the over-billing provides for a fair accounting, negates the issue as to whether this a rectification of an error or a claim, is in keeping with the current agreement between the parties, and provides the basis for additional recoveries.

The AFD service provider has since completed a review of 100% of 2010-11 design, construction and security costs and the results have been validated by the Office of Audit and Evaluation using the same methodology used for 2009-10.

Given that the 2010-11 error rate was found to be substantially lower than the 2009-10 rate, RPB has agreed with the AFD service provider to utilize the average of the two net error rates to settle the amounts owing for fiscal years 2005-06 to 2008-09.

In addition to the recoveries of extrapolated amounts, the Branch has addressed a key recommendation from the external audits and internal reviews, by requiring any future statement of costs submitted by the AFD service provider be based on actual rather than estimated costs, and audited annually by a third-party auditor. This important control ensures that the AFD service provider's actual costs are accurately reflected in the final amounts paid and that the Department is not overcharged or undercharged.

The Branch also implemented a Project Delivery Services (PDS) financial review procedure. The review consists of a detailed assessment of project costs to ensure that claimed expenditures are in accordance with the terms and conditions of the contracts and that the costs claimed are the same as the costs incurred. On a quarterly basis and for each Portfolio, a sampling of completed projects will be reviewed by the AFD audit team.

In addition to these measures, RPB's focus and efforts are on the development of a comprehensive, integrated approach to appropriately address risk for the next generation of contracts.

Introduction

1. The purpose of this report is to provide the results of the Office of Audit and Evaluation's (OAE) comprehensive review of the AFD service provider fiscal year 2009-10 management review. The OAE assessed a subset of projects (80) to determine whether the AFD service provider's expenditures for design, construction and security costs were accurate, included all supporting documentation and were properly billed to Public Works and Government Services Canada (PWGSC).

Background

2. On November 22, 2004, PWGSC awarded the AFD service provider eight contracts for the maintenance and upkeep of the buildings under the purview of the Real Property Branch. On March 10, 2010 a media article raised questions with regards to the reasonability of seven financial transactions (payments made by PWGSC to the AFD service provider).
3. In response to the media article, the Minister of Public Works and Government Services, ordered a review of the transactions as well as a broader examination of the AFD service provider contracts to determine if costs charged were reasonable and supportable and whether there was any evidence of systemic problems.
4. On April 22, 2010, through a competitive bidding process, PWGSC awarded PwC a contract to provide forensic audit services to assist PWGSC's Office of Audit and Evaluation in an independent multi-phased audit consisting of specified forensic audit procedures on the expenditure transactions made by the AFD service provider on behalf of PWGSC as both custodian and tenant and also on behalf of other government department as tenants. In Phase 1, the reasonability and supportability of the transactions identified by the media article were assessed. In Phase 2, forensic audit procedures were conducted with respect to a sample of 70 additional transactions.
5. One of the findings in the Phase 2 Audit was that out of 66 projects containing charges for third party external service providers, the AFD service provider did not charge PWGSC the appropriate amount 36 times (55%). These transactions resulted in $44,245 of overcharges to PWGSC, on a total of $4,342,599 billed for the sample chosen, representing an overbilling rate of 1.0%. In these projects, the AFD service provider either charged PWGSC an amount different from the actual cost incurred and/or was unable to provide appropriate documentation to support the amounts charged. While the majority of samples PwC reviewed contained instances of overcharges, some instances of undercharging were also identified.
6. In accordance with the contracts between PWGSC and the AFD service provider, design and consultant costs, as well as construction and contractor costs are considered allowable pass-through costs without markup for overhead or profit.
7. In response to the PwC findings, the AFD service provider advised that they would refund PWGSC in full for the amounts identified in the PwC audit where PWGSC was overcharged. The AFD service provider proposed that it would conduct a review of 100% of the amounts charged in fiscal year 2009-10 for all eight portfolios or contracts regarding the following three cost categories: (1) design; (2) construction; and (3) security costs.
8. One of the recommendations from the aforementioned PwC forensic audit was that PWGSC should conduct a comprehensive review of the management review the AFD service provider conducted for costs incurred in fiscal year 2009-10. In addition, the results of this comprehensive review would be analyzed to assess whether or not the AFD service provider should conduct further management reviews of the three costs components for fiscal years 2005-06 to 2008-09 for all eight portfolios, which would subsequently be reviewed by PWGSC.
9. As a result of these recommendations the AFD service provider began its management review of 100% of amounts charged to PWGSC in the 2009-10 fiscal year for design, construction and security costs. Roles and responsibilities of the AFD service provider, the OAE, and the RPB are identified below in Exhibit 1. Labour costs incurred by the AFD service provider to conduct its management review of the 2009-10 fiscal year are allowable-pass through costs under the terms and conditions of the contracts. Any amounts relating to the design, construction, or security costs, identified as overbilling would be reimbursed to PWGSC.
10. The AFD service provider completed its management review of 100% of the three cost categories for fiscal year 2009-10 and submitted its findings to PWGSC for review. The AFD service provider's management review identified $1,169,346.42 in net overbillings to PWGSC in fiscal year 2009-10.
11. The work performed by the OAE was intended to provide a level of assurance to RPB and the Deputy Minister that the results of the AFD service provider's 2009-10 management review were accurate and complete.

Exhibit 1: Roles and responsibilities of stakeholders in the Alternative Forms of Delivery service provider's management review of design, construction, and security costs charged to Public Works and Government Services Canada in fiscal year 2009-10

Focus of the Office of Audit and Evaluation's work

12. The OAE used judgmental sampling to select for review the items related to the design, construction and security costs incurred by the AFD service provider. Ten projects from each of the eight portfolios were selected for a total of 80 projects. The ten projects chosen for each portfolio included 5 projects the AFD service provider identified in their management review as having a variance and 5 identified by the AFD service provider as not containing a variance. The work involved a review of source documents and supporting documentation, and then reconciling these to the amounts actually charged to PWGSC.

Findings

13. The AFD service provider's management review provided to the OAE indicated a net variance of $1,169,346.42 in overbilling on total amounts of $217,839,578.08 billed to PWGSC in 2009-10 (0.54% rate) for design, construction and security costs. This report, along with a listing of the design, construction and security costs charged to PWGSC for fiscal year 2009-10 were then reviewed by the OAE to ensure accuracy and completeness. The OAE's review was designed to validate the AFD service provider's management review and was not designed to detect fraud; based on the procedures the OAE carried out on the sample selected, no instances of fraud were identified.
14. In reviewing the AFD service provider's report, the OAE noted that the AFD service provider identified a net variance of $1,169,346.42 in overbilling to the Department. The AFD service provider did not detail the gross overbilling and underbilling in its report. In reviewing the detailed supporting documentation, the OAE identified the total amount of gross overbilling to be $1,499,105.60, and the total under-billing to be $329,759.18 as indicated in Appendix A.
15. The OAE obtained a list of construction, design, and security costs charged to PWGSC for fiscal year 2009-10 from the AFD service provider in order to select a sample for testing. To ensure the AFD service provider's report list of the costs was accurate and complete, the OAE reconciled the AFD service provider's list with amounts reported in RPB's tracking system, the Alternative Forms of Delivery Management System. No discrepancies were identified.
16. After validating the list of fiscal year 2009-10 costs for the three categories, the OAE then selected a sample of 80 projects from the list, using ten projects from each of the eight portfolios. Five of the projects selected from each portfolio had been indicated by the AFD service provider as containing variance in the amounts incurred by the AFD service provider and what was billed to PWGSC, and the other five had been indicated by the AFD service provider as containing no variance by the AFD service provider. The source documents and supporting documentation were reviewed, and payment to the appropriate subcontractors was validated. Through review of the documentation for the 80 projects OAE was able to validate each of the AFD service provider's conclusions without any discrepancies.
17. Based on the comprehensive review work performed by the OAE and RPB which found no discrepancies with the AFD service provider's list of projects assessed, and no discrepancies with the 80 projects reviewed, we found the AFD service provider's management review was accurate and complete.
18. The AFD service provider was of the opinion that any recovery of overbilling should take into account under-billing, which would result in a total (net) reimbursement to PWGSC of $1,169,346.42. RPB, in consultation with Legal Services and Acquisitions Branch, concluded that the recovery of net overbillings was an appropriate resolution and compliant with the terms and conditions of the contracts. The AFD service provider delivered a cheque to RPB for the $1,169,346.42 amount.
19. As a result of the fact that net overbilling was identified in fiscal year 2009-10, it is likely that net overbilling occurred during the period of 2005-06 to 2008-09 and during 2010-11. We were informed by RPB that the AFD service provider has conducted a management review for the 2010-11 year using the same methods as its review of the 2009-10 fiscal year, and RPB has engaged the OAE to analyze the report's findings and validate its results. Identification of net overbillings that likely had taken place for years 2005-06 to 2008-09 would be achieved either through extrapolation of the average error rate across the 2009-10 and 2010-11 years' expenditures, or through the comprehensive review of 100% of expenditures for each year of the contracts. We were informed by RPB that it intends to recover the expected net overbilling for fiscal years 2005-06 to 2008-09 through the extrapolation approach.

Conclusion

20. Based on the results of the chosen sample, and the reconciliation of the costs the AFD service provider provided with the Alternative Forms of Delivery Management System data, the OAE believes the documents and related amounts provided by the AFD service provider relating to its fiscal year 2009-10 management review are valid. RPB, in consultation with Legal Services and Acquisitions Branch, concluded that the recovery of net overbilling was an appropriate resolution. The management review indicated an amount of $1,169,346.42 in net overbilling for fiscal year 2009-10, to be reimbursed to PWGSC by the AFD service provider.
21. As a result of the fact that net overbilling was identified in fiscal year 2009-10, it is likely that net overbilling occurred during the period of 2005-06 to 2008-09 and during 2010-11. The AFD service provider has conducted a management review for the 2010-11 year using the same methods as its review of the 2009-10 fiscal year, and the OAE is analyzing the report's findings. Identification of net overbilling that likely took place for years 2005-06 to 2008-09 should be achieved either through extrapolation of the average error rate across the 2009-10 and 2010-11 years' expenditures, or through the comprehensive review of 100% expenditures for all years of the contracts. We were informed by RPB that PWGSC intends to recover the expected net overbilling for fiscal years 2005-06 to 2008-09 through the extrapolation approach.

Management response

The Real Property Branch (RPB) accepts the report and is working with the appropriate stakeholders to implement actions in a timely manner.

Following the receipt of the AFD service provider's review of the 2009-10 third-party costs, the Branch has recovered $1,169,346.42 representing the net variance between under-billing and overbilling. Given the materiality of the 2009-10 results, the AFD service provider was informed that further reviews of the three costs categories for all the years until 2010-11 needed to be done.

Subsequent to the instructions provided at the June 6, 2013 Audit and Evaluation Committee (AEC) meeting to recover the gross amounts of overbilling instead of the net variance, the Branch proceeded with the recovery of extrapolated amounts of gross overbilling for the remaining years up to and including 2010-11 by setting off these amounts from the monthly invoices starting in July 2013 with the last set-off occurring in November 2013.

Following the June 6, 2013 AEC meeting, the issue of the reimbursement of net versus gross amount was examined. Recovery of net overbilling is appropriate because reconciliation of an account pursuant to an audit is not a new claim under the contract, but rather the discovery of an existing discrepancy. Discrepancies can be considered latent errors requiring correction, in which case PWGSC would not be entitled to withhold the underbilled amounts. Therefore, allowing the deduction of the underbilling from the over-billing provides for a fair accounting, negates the issue as to whether this a rectification of an error or a claim, is in keeping with the current agreement between the parties, and provides the basis for additional recoveries.

The AFD service provider has since completed a review of 100% of 2010-11 design, construction and security costs, and the results have been validated by the Office of Audit and Evaluation using the same methodology used for 2009-10.

Given that the 2010-11 error rate was found to be substantially lower than the 2009-10 rate, RPB has agreed with the AFD service provider to utilize the average of the two net error rates to settle the amounts owing for fiscal years 2005-06 to 2008-09.

In addition to the recoveries of extrapolated amounts, the Branch has addressed a key recommendation from the external audits and internal reviews, by requiring any future statement of costs submitted by the AFD service provider be based on actual rather than estimated costs, and audited annually by a third-party auditor. This important control ensures that the AFD service provider's actual costs are accurately reflected in the final amounts paid and that the Department is not overcharged or undercharged.

The Branch also implemented a Project Delivery Services (PDS) financial review procedure. The review consists of a detailed assessment of project costs to ensure that the claimed expenditures are in accordance with the terms and conditions of the contracts and that the costs claimed are the same as the costs incurred. On a quarterly basis and for each Portfolio, a sampling of completed projects will be reviewed by the AFD audit team.

In addition to these measures, RPB's focus and efforts are on the development of a comprehensive, integrated approach to appropriately address risk for the next generation of contracts.

Office of Audit and Evaluation approach

A sample of projects containing construction, design, and security cost components were chosen from the AFD service provider for the 2009-10 fiscal year. The sample included projects from the eight portfolios that the AFD service provider maintains in relation to the contracts with RPB. The design, construction and security costs incurred by the AFD service provider within the sample project were reconciled with regards to amounts charged to PWGSC.

The sampling method was as follows:

• Judgmental sampling was used to choose 10 projects from each of the AFD service provider's eight portfolios (for a total of 80 projects). This ensured that all regions were included in the sample. The ten projects per portfolio chosen included 5 projects from those identified by the AFD service provider as containing variances, and 5 identified by the AFD service provider as not containing variances.
• The 25 projects that were sampled through the 2009-805: Review of the Revenue and Expenses Controls of the AFD Program, were excluded from the selection of 80 items. The intent was to consolidate the results from the 2009-805 review with the results of the 80 samples chosen. This increased the level of assurance provided since the projects chosen in the 2009-805 review were solely from the three portfolios in the National Capital Area, as a large proportion of the contract costs were situated in this area.

Testing procedures:

• A list of all projects billed to PWGSC in fiscal year 2009-10 in relation to the AFD service provider's portfolios was obtained from the AFD service provider. This list identified the construction, design, and security amounts billed as a part of each project on the list.
• The list was verified to ensure that the total number of projects and amount paid matched PWGSC's records.
• From that list projects were selected for testing.
• Requested from the AFD service provider the selected project documents, along with supporting documentation such as purchase orders, change orders, copies of cancelled cheques (front and back), job status inquiry reports, and other related documents.
• Traced items back to source documents. The review team assessed the accuracy and supportability of the amounts billed to PWGSC.
• Compared amounts assessed by the OAE team to the amounts assessed by the AFD service provider. Any discrepancies were noted and follow up with the AFD service provider was conducted to obtain any required additional information.

Objective

The objective of the engagement was to assess the accuracy of the AFD service provider's reconciliation of the expenditures for design, construction and security costs for fiscal year 2009-10 billed to PWGSC by the AFD service provider with the supporting documentation for costs incurred by the AFD service provider.

Review team

The Review was conducted by members of the OAE, overseen by the Director of Procurement Audit, and under the overall direction of the Chief Audit and Evaluation Executive.

Appendix A