2013-808 Office of Audit and Evaluation Review of the Alternative Forms of Delivery service provider's management review of fiscal year 2010-11 construction, design and security costs (Final report)

2013-808 Final report (PDF, 423KB)

December 24, 2014

Main points

What we examined

1. In March 2010, in response to the media scrutiny of certain expenditures under the Alternative Forms of Delivery (AFD) contracts, the Minister of Public Works and Government Services (PWGSC) requested an independent third-party audit to examine expenditures billed to PWGSC by the AFD service provider. PricewaterhouseCoopers LLP (PwC) was awarded the contract to perform the independent, multi-phased audit of those expenditures.
2. As part of the Management action plan that was developed in response to the PwC 2011 audit, Forensic Audit of Seventy Transactions Under the SNC-Lavalin O&M Contracts, the AFD service provider was responsible for conducting a management review of 100 percent of the third party costs billed to Public Works and Government Services Canada (PWGSC) in fiscal year 2009-10. In addition, as part of the Management action plan requirements, this review was to be provided to PWGSC's Real Property Branch (RPB) and the Office of Audit and Evaluation (OAE) was to perform a comprehensive review on the service provider's management review.
3. In accordance with the Management action plan, the AFD service provider conducted its management review of 2009-10 and the OAE performed its comprehensive review. The reviews found net billing errors of $1,169,346.42. The OAE concluded that as a result of the net billing errors by the AFD service provider in fiscal year 2009-10, it was possible that net billing errors may have occurred during the period of 2005-06 to 2008-09 and during fiscal year 2010-11. In their response to the 2009-10 review, RPB indicated that the AFD service provider had completed a review of 100 percent of 2010-11 construction, design and security costs, and the Office of Audit and Evaluation would complete a similar validation of the AFD service provider's review results using the same methodology used for the 2009-10 review. 
4. The AFD service provider provided its report summarizing its management review for the 2010-11 fiscal year along with supporting documentation to RPB and the OAE. The review included all eight portfolios for which the AFD service provider provides services for the maintenance and upkeep of buildings. These eight portfolios correspond directly to the eight contracts awarded by PWGSC to the AFD service provider in 2004. Costs reviewed consisted of the following three categories: (1) construction; (2) design; and, (3) security. The OAE, which undertook a comprehensive review of the report, analyzed the report's findings, and performed analytical procedures over all of the costs billed to PWGSC, as well as detailed sample testing of 160 projects from fiscal year 2010-11 including 20 projects from each of the eight portfolios. Sample testing was conducted to assess the accuracy and completeness of the AFD service provider's reconciliation of the expenditures for construction, design and security costs billed to PWGSC for the 2010-11 fiscal year. 
5. The 20 projects selected for each portfolio included ten projects chosen from those identified by the AFD service provider in their management review as containing variances between the amount billed to PWGSC and the actual cost incurred by the AFD service provider, and ten identified by the AFD service provider as not containing variances. The OAE validated these sample projects with the AFD service provider's results by reviewing source documents for each project along with the corresponding amount billed to PWGSC.

Why it is important

6. The OAE's work was directed at providing independent validation of the results of the AFD service provider's management review of 100 percent of the third party costs billed to PWGSC in 2010-11. Independent validation of the accuracy and completeness of the AFD service provider's results was essential to help mitigate any risk of understatement or overstatement of billing variances by the AFD service provider. It is important that PWGSC is correctly billed in accordance with the contracts for work completed by the AFD service provider to ensure appropriate stewardship of taxpayers' money.

What we found

7. In the report provided to PWGSC, the AFD service provider indicated that it had identified $401,304.96 in net billing errors between what it had billed PWGSC and the actual costs the AFD service provider had incurred. This represents a decrease of 66 percent from the 2009-10 net billing error of $1,169,346.42. As the reduction in net billing errors represented a risk that all billing errors had not been captured in the 2010-11 report provided to PWGSC, the OAE: performed analytical procedures on actual costs incurred by the AFD service provider to identify unusual trends, and doubled the sample size of its detailed testing to 160 projects from the 80 projects tested in the OAE's 2009-10 review.
8. The analytical procedures did not identify any unusual trends and the OAE found that in general, billing errors had decreased across all cost categories. In addition, the OAE was able to reconcile the number of projects and their total values with RPB, and no discrepancies were found between the billing errors the AFD service provider had reported to PWGSC and what the OAE found. Finally, in all 160 projects reviewed, the billing errors identified in the management review were validated as accurate by the OAE. In light of these findings the OAE concluded the AFD service provider's management review was accurate and complete, and as such, the net billing error was $401,304.96 as reported.
9. Given net billing errors were identified for both the 2010-11 fiscal year and the 2009-10 fiscal year, it is possible PWGSC was overbilled for third party costs from 2005-06 to 2008-09. As such PWGSC should work with the AFD service provider to assess and recover net billing errors for those years.

Management response

The Real Property Branch (RPB) accepts the report and is working with the appropriate stakeholders to implement actions in a timely manner,

Following the receipt of the AFD service provider's review of the 2009-10 third-party costs the Branch recovered $1,169,346.42 representing the net variance between underbilling and overbilling.

The 2010-11 management review indicates that an amount of $401,304.96 net billing errors occurred. This amount was withheld by PWGSC from the AFD service provider and will not be released to them. An extrapolation was made for fiscal years 2005-06 to 2008-09 based upon an average of the previous error rate. The methodology and associated costs were confirmed with the service provider and the amount of $1,441,475 was recovered.

In addition to recoveries of extrapolated amounts, the Branch has addressed a key recommendation of the external audits and internal reviews by requiring any future statement of costs submitted by the AFD service provider be based on actual rather than estimated costs and audited annually by a third-party auditor. This important control ensures that the AFD service provider's actual costs are accurately reflected in the final amounts paid and that the Department is not overcharged or undercharged.

The Branch also implemented a Project Delivery Service (PDS) financial review procedure. The review consists of a detailed assessment of project costs to ensure that the claimed expenditures are in accordance with the terms and conditions of the contracts and that the costs claimed are the same as the costs incurred. A sampling of completed projects for each portfolio is reviewed on a quarterly basis by the AFD audit team.

In addition to these measures, RPB's focus and efforts are on the development of a comprehensive, integrated approach to appropriately address risk for the next generation of contracts.

Recommendation and management action plan

Recommendation 1 (High priority): The Assistant Deputy Minister Real Property Branch should pursue reimbursement for possible billing errors for third party costs by the AFD service provider for fiscal years 2005-06 to 2008-09. 

• Management action plan 1: RPB confirmed the methodology to be applied for the calculation of third party costs billing errors for fiscal years 2005-06 to 2008-09 and the amounts to be reimbursed to PWGSC with the Service provider. A combined total of $1,441,475 was recovered for 2005-06 to 2008-09.

Introduction

1. This engagement was included in the PWGSC 2014-18 Risk-Based Audit and Evaluation Plan.
2. On November 22, 2004, PWGSC awarded the AFD service provider eight contracts for the maintenance and upkeep of the buildings under the purview of the Real Property Branch. To fulfill its obligations under these contracts, the AFD service provider subcontracts maintenance and building related projects to third parties. In accordance with these contracts, subcontractor costs are considered allowable pass-through costs that are billed to PWGSC by the AFD service provider without markup for overhead or profit. One category of such subcontracted costs is construction, design, and security.
3. On March 10, 2010 a media article raised questions with regard to the reasonability of seven financial transactions (payments made by PWGSC to the AFD service provider) under these contracts. The Minister of Public Works and Government Services ordered a review of the seven transactions as well as a broader examination of the AFD service provider contracts to determine if costs billed to PWGSC were reasonable and supportable and whether there was any evidence of systemic problems.
4. As per the Minister's request, on April 22, 2010, through a competitive bidding process, PWGSC awarded PwC a contract to provide forensic audit services to assist PWGSC's Office of Audit and Evaluation. The services were for an independent multi-phased audit consisting of specified forensic audit procedures on the expenditure transactions made by the AFD service provider on behalf of PWGSC, as both custodian and tenant, and also on behalf of other government department as tenants. In Phase 1 of the audit, the reasonability and supportability of the transactions identified by the media article were assessed. In Phase 2 of the audit, specified forensic audit procedures were conducted on a sample of 70 additional transactions for the period of April 1, 2005 to March 31, 2010 to determine whether there was any evidence of systemic problems. 
5. The Phase 2 Audit sample included 66 projects that included charges for construction, design and security. The Audit determined the AFD service provider did not charge PWGSC the appropriate amount for 36 of those 66 projects (55 percent). It further determined PWGSC had been overbilled $44,245 on total billings of $4,342,599 for construction, design and security, representing a billing error rate of 1.0 percent. These billing errors were identified based on validation of instances where the AFD service provider billed PWGSC an amount different from the actual cost incurred and/or was unable to provide appropriate documentation to support the amounts billed. While the majority of samples PwC reviewed contained instances of overbilling, some instances of underbilling were also identified.
6. In response to the PwC audit's findings, the AFD service provider advised that they would refund PWGSC in full net billing errors identified in the PwC audit. Furthermore, the AFD service provider proposed that it would conduct a management review of 100 percent of the amounts billed in fiscal year 2009-10 for all eight portfolios for construction, design, and security costs.
7. PwC recommended PWGSC validate the results of the management review carried out on these cost components for 2009-10. In addition, it recommended PWGSC consider how it will assess the billings for the three costs components for fiscal years 2005-06 to 2008-09 and 2010-11 for all eight portfolios. Finally, it recommended actions be taken to reduce the risk of billing errors in these cost components for 2011-12 and going forward.
8. In 2012, the AFD service provider completed its management review of these cost components for 2009-10. It identified $1,169,346 in net billing errors to PWGSC in fiscal year 2009-10, out of a total of $217,909,546 billed, representing a billing error rate of 0.54 percent. 
9. The OAE validated the management review and concluded that the error rate was valid. In addition the OAE concluded it was possible that billing errors had taken place from fiscal years 2005-06 to 2008-09, and during 2010-11.
10. The AFD service provider proposed to PWGSC that it would conduct a management review of these cost components for 2010-11, and use the results to calculate an average billing error rate from 2009-10 and 2010-11 that could be extrapolated over the 2005-06 to 2008-09 period. PWGSC accepted the proposal to conduct a management review of 2010-11, but did not commit to an extrapolation of the error rate.
11. The purpose of this report is to provide the results of the OAE's validation of the AFD service provider's management review of the three cost components for fiscal year 2010-11. The work performed by the OAE was intended to provide a review level of assurance to Real Property Branch (RPB) and the Deputy Minister that the results of the AFD service provider's management review were accurate and complete. The roles and responsibilities for this work for the OAE, RPB, and the AFD service provider are indicated below in Exhibit 1.

Exhibit 1

Focus of the review

12. The objective of this review was to assess the accuracy and completeness of the AFD service provider's reconciliation of the expenditures for construction, design, and security costs for fiscal year 2010-11 billed to PWGSC by the AFD service provider, with the supporting documentation for costs incurred.
13. The OAE applied analytical procedures to the amounts billed to PWGSC by the AFD service provider. Using judgmental sampling, it selected 20 projects from each of the eight portfolios, for a total sample of 160 projects. The 20 projects chosen for each portfolio included ten projects the AFD service provider identified in its management review as not having a variance. The other ten projects from each portfolio were composed of five projects that had been identified by the AFD service provider as having overbilling and five projects where the AFD service provider had identified underbilling. For each of these samples, the construction, design and security costs were validated against source documents, and reconciled to the amounts billed to PWGSC. The OAE's review was designed to validate the AFD service provider's management review and was not designed to detect fraud; based on the procedures the OAE carried out on the sample selected, no instances of fraud were identified.
14. More information on the review objective, scope, approach and criteria can be found in the section "About the review" at the end of the report.

Statement of conformance

15. The review conforms with the Internal Auditing Standards for the Government of Canada, as supported by the results of the quality assurance and improvement program.
16. Sufficient and appropriate review procedures have been conducted and evidence gathered to support the accuracy of the findings and conclusions in this report and to provide a review level of assurance. The findings and conclusions are based on a comparison of the conditions, as they existed at the time, against pre-established review criteria that were agreed upon with management. The findings and conclusion are only applicable to the entity examined and for the scope and time period covered by the review.

Approach

The Alternative Forms of Delivery service provider's management review for 2010-11

17. The AFD service provider's review identified billing errors that netted to $401,304.96, between what it billed PWGSC for construction, design and security costs and what it actually incurred, on total billings of $258,720,845.74 (representing a 0.16 percent billing error rate). This net billing error was based on identified overbilling of $724,052.50, and underbilling of $322,747.54.
18. The review of construction, design and security costs for 2009-10 identified net billing errors of $1,169,346.42 (representing a 0.54 percent billing error rate). This consisted of $1,499,105.60 in overbilling and $329,759.18 in underbilling. The AFD service provider explained that the decrease in billing errors from 2009-10 to 2010-11 was due to improvements in the AFD service provider's administration of the contract in 2010-11. During fiscal year 2009-10, the business volume of the AFD contracts had increased over prior years due to additional projects being carried out as part of PWGSC's Accelerated Infrastructure Program. They explained that to accommodate the increased business volume, a number of new personnel were hired by the AFD service provider in 2009-10. The large number of new employees and associated learning curve and training issues had the effect of increasing the error rate on transactions in 2009-10. They believe the error rate in 2010-11 decreased as the employees gained a stronger understanding of the control environment and project management processes. Business volumes remained higher in 2010-11, but the AFD service provider was appropriately staffed to handle this higher volume.
19. Due to the significant decrease in billing error rate from the 2009-10 fiscal year to 2010-11, the OAE adjusted its approach to manage any risk that the billing error rate identified in the management review may be understated.
20. To address the risk the billing error rate was understated, the OAE performed the following procedures during its review:
    1. Performed analytical procedures on the total amounts billed to PWGSC in 2010-11 in relation to total amounts billed to PWGSC in 2009-10. Specifically it:
       • Compared billing errors in projects from 2009-10 with billing errors in projects from 2010-11. The analysis included comparing changes in billing errors across portfolios as well as by dollar amount;
       • Examined actual costs incurred by the AFD service provider that were composed of round numbers ( e.g. $10,000);
       • Analyzed hourly rates billed by the AFD service provider's security subcontractor by comparing the hourly billed rates to the allowable rates from the relevant standing offer^{Footnote 2};
       • Performed Benford analysis^{Footnote 3} to identify trends occurring in the distribution of actual costs incurred by the AFD service provider; and
       • Analyzed the frequency of duplicate actual costs incurred by the AFD service provider in both 2009-10 and 2010-11.
    2. Aligned the number and cost of third party projects billed to PWGSC with RPB's records; and
    3. Increased the sample size of projects for detailed testing to 160 projects. This is double the 80 project sample size reviewed as part of the OAE's review of 2009-10 costs. 

Observations

Analytical procedures of the actual costs incurred by the Alternative Forms of Delivery service provider did not identify any unusual trends

21. The OAE obtained a list of construction, design, and security costs billed to PWGSC for fiscal year 2009-10 and 2010-11. Trends in over and underbilling identified in each fiscal year were considered.
22. For example, to identify whether the changes in billing errors in 2010-11 was confined to any one portfolio, the billing errors from 2009-10 and 2010-11 were assessed by portfolio. Overall, the number of billing errors decreased for the majority of the portfolios in 2010-11 in comparison to 2009-10. 
23. In addition, the billing errors were assessed by grouping the errors as follows: (1) small ($1,000); (2) medium (between $1000 and $10,000); and (3) large (over $10,000), and comparing across years. The analysis identified that all groups of billing errors had decreased in 2010-11 in comparison to 2009-10.
24. The number of billing errors by cost categories was also assessed. The number of billing errors decreased for each of the cost categories in 2010-11 in comparison to 2009-10. In light of this the OAE did not focus its detailed testing on any one cost category.
25. It can be unusual for transaction amounts to be composed of round numbers ( i.e. $10,000). The OAE performed analysis on the number of actual costs incurred by the AFD service provider that were round numbers, for each cost category between 2009-10 and 2010-11, and found the ratio of actual costs with round numbers was similar for both fiscal years. Discussion of the results with RPB indicated that it was not unreasonable for round numbers to occur in some cases, due to the nature of some of the projects, and no unusual trends were identified between the two fiscal years.
26. In its provision of services under the AFD contracts, the AFD service provider is required to subcontract its security needs through PWGSC's standing offers. To ensure that appropriate rates were being billed by the security contractors, a rate analysis was conducted on invoices from the primary security subcontractor. The OAE: (1) identified the hourly rates billed by the primary security subcontractor on its security invoices; and (2) compared the hourly rates billed to what was allowable based on the appropriate standing offer. This analysis did not identify any discrepancies between rates billed to the AFD service provider (and subsequently billed to PWGSC) for security services, and what was allowable under the appropriate standing offer.
27. Benford analysis was conducted on the actual costs incurred by the AFD service provider in 2010-11 and 2009-10 which allowed the OAE to analyze the total construction, design and security costs incurred for unusual trends. This analysis indicated that the actual costs from 2010-11 more closely matched the numerical norms as predicted by the Benford analysis. No unusual trends were identified for the 2010-11 actual costs in relation to the 2009-10 actual costs. This indicates reduced risk for the 2010-11 actual costs relative to 2009-10 actual costs. 
28. Finally, the OAE analyzed duplicate actual costs incurred ( e.g., multiple instances of $2,255 being incurred), and found a general trend of fewer duplicate actual costs in 2010-11 compared to 2009-10. As there were fewer duplicate actual costs in 2010-11 this indicates reduced risk for the 2010-11 results relative to 2009-10.
29. Based on the results of the analytical procedures, no unusual trends were identified for 2010-11 actual costs incurred in relation to 2009-10 actual costs incurred. As the changes in billing errors from 2009-10 to 2010-11 were not confined to any one portfolio the OAE decided to sample all eight portfolios equally. This allowed any systemic issues within a given portfolio to be identified. In addition, as the decrease in instances of billing errors were observed whether the billing error was small, medium or large, no specific emphasis on the size of the billing error on a project was made when the OAE selected its sample.

Costs in the supporting records provided by the Alternative Forms of Delivery service provider were complete

30. To ensure the AFD service provider's list of the costs billed to PWGSC was accurate and complete, the OAE reconciled the number of projects, and their total dollar values on the AFD service provider's list with those reported in RPB's tracking system, the Alternative Forms of Delivery Management System (AFDMS).
31. The OAE expected to find that the number of projects and their total dollar values on the list provided by the AFD service provider would agree to RPB's records in AFDMS.
32. We found that initially the list provided by the AFD service provider and RPB's AFDMS system did not agree. A reconciliation was carried out by RPB and the AFD service provider, which identified the differences, had been caused by misclassification as well as timing differences between the two systems. The AFD service provider's report had listed more projects than RPB's AFDMS system due to single projects being counted multiple times due to reclassification within the AFD service provider's system (although the reclassified projects had only been billed to PWGSC once). In addition, the AFD service provider's system had included additional year end transactions which AFDMS had not captured. RPB verified that the additional year end transactions were valid, and the total value of projects on the list accurately recorded total third party costs.
33. Based on our findings, the list provided by the AFD service provider accurately and completely recorded total third party costs billed to PWGSC. The improvements being made to the RPB's AFDMS system as a result of the OAE engagement, 2012-713 Follow-Up Review of the Implementation of Alternative Forms of Delivery Management Actions Plans by Real Property Branch will address the issues identified related to year end transactions.

Billing errors identified by the Office of Audit and Evaluation agreed to the billing errors identified by the Alternative Forms of Delivery service provider

34. The OAE selected a sample of 160 projects from the reconciled list of project maintained by the AFD service provider. Twenty projects from each of the eight portfolios were selected. 
35. The sample of 160 projects was tested in detail, with the OAE agreeing amounts billed to PWGSC to the actual costs incurred as reported on the third party invoices and other supporting documentation. We identified any over or under billing that had occurred for these projects, and compared these to the billing errors identified by the AFD service provider in its management review.
36. We expected to find that any over or underbilling identified by the OAE's review had been accurately identified in the AFD service provider's report.
37. We found the billing errors identified by the OAE aligned with those identified by the AFD service provider. For projects which had been reclassified, actual costs incurred by the AFD service provider were billed to PWGSC a single time, and RPB was able to monitor these projects through its regular project monitoring processes.

Conclusion

38. Through its analytical procedures and detailed testing of 160 projects, the OAE did not identify any unusual trends or discrepancies in billing errors. As such, we conclude that the AFD service provider's management review was accurate and complete. The management review indicates an amount of $401,304.96 net billing errors, which should be reimbursed to PWGSC by the AFD service provider.
39. As a result of the net billing error of $401,304.96 identified for the 2010-11 fiscal year, and net billing error of $1,169,346.42 identified for the 2009-10 fiscal year, it is possible that PWGSC was overbilled for third party costs from 2005-06 to 2008-09. As such PWGSC should work with the AFD service provider to assess and settle any amounts overbilled to PWGSC during those years.

Management response

The Real Property Branch (RPB) accepts the report and is working with the appropriate stakeholders to implement actions in a timely manner,

Following the receipt of the AFD service provider's review of the 2009-10 third-party costs the Branch recovered $1,169,346.42 representing the net variance between underbilling and overbilling.

The 2010-11 management review indicates that an amount of $401,304.96 net billing errors occurred. This amount was withheld by PWGSC from the AFD service provider and will not be released to them. An extrapolation was made for fiscal years 2005-06 to 2008-09 based upon an average of the previous error rate. The methodology and associated costs were confirmed with the service provider and the amount of $1,441,475 was recovered.

In addition to recoveries of extrapolated amounts, the Branch has addressed a key recommendation of the external audits and internal reviews by requiring any future statement of costs submitted by the AFD service provider be based on actual rather than estimated costs and audited annually by a third-party auditor. This important control ensures that the AFD service provider's actual costs are accurately reflected in the final amounts paid and that the Department is not overcharged or undercharged.

The Branch also implemented a Project Delivery Service (PDS) financial review procedure. The review consists of a detailed assessment of project costs to ensure that the claimed expenditures are in accordance with the terms and conditions of the contracts and that the costs claimed are the same as the costs incurred. A sampling of completed projects for each portfolio is reviewed on a quarterly basis by the AFD audit team.

In addition to these measures, RPB's focus and efforts are on the development of a comprehensive, integrated approach to appropriately address risk for the next generation of contracts.

Recommendation and management action plan

Recommendation 1 (High priority): The Assistant Deputy Minister Real Property Branch should pursue reimbursement for possible billing errors for third party costs by the AFD service provider for fiscal years 2005-06 to 2008-09.

• Management action plan 1: RPB confirmed the methodology to be applied for the calculation of third party costs billing errors for fiscal years 2005-06 to 2008-09 and the amounts to be reimbursed to PWGSC with the Service provider. A combined total of $1,441,475 was recovered for 2005-06 to 2008-09.

About the review

Authority

This engagement was included in the PWGSC 2014-18 Risk-Based Audit and Evaluation Plan.

Objective

The objective of the engagement was to assess the accuracy and completeness of the AFD service provider's reconciliation of the expenditures for construction, design and security costs for fiscal year 2010-11 billed to PWGSC by the AFD service provider with the supporting documentation for costs incurred by the AFD service provider.

Scope and approach

This review covered the period from April 1, 2010 to March 31, 2011.

A sample of projects containing construction, design, and security cost components were chosen from the AFD service provider for the 2010-11 fiscal year. The sample included projects from the eight portfolios that the AFD service provider manages as part of the contract with RPB. The construction, design and security costs incurred by the AFD service provider within the sample project were reconciled with regards to amounts billed to PWGSC.

The sampling method was as follows:

• 20 projects were chosen from each of the AFD service provider's eight portfolios (for a total of 160 projects). This ensured that all regions were included in the sample. The 20 projects per portfolio chosen included ten projects randomly selected from those identified by the AFD service provider as containing no variances between actual and incurred costs, five projects randomly selected from projects identified by the AFD service provider as containing overbilling, and five projects randomly selected from projects identified by the AFD service provider as containing underbilling.

Testing procedures:

Initially the OAE reviewed the AFD service provider's management review of 2010-11 construction, design and security costs, and:

• Followed up with RPB on any OAE comments that resulted from the initial review of the AFD service provider's management review of 2010-11 construction, design and security costs.
• Followed up with RPB regarding management action plan items the AFD service provider states they have implemented as part of this exercise in their report. In addition, followed up with RPB regarding the statement in the AFD service provider's report identifying that $106,383.60 had been calculated and reimbursed to PWGSC for the 2011-12 fiscal year.
• Followed up with RPB to ensure that the total number of projects and amounts paid stated in the AFD service provider's report matched PWGSC's records and any discrepancies were explained.

The second step of the project involved analysis of a sample of projects from the list of projects the AFD service provider used to create their management review. This list includes actual costs for third party subcontractors' charges to the AFD service provider in the areas of construction, design and security, as well as the amounts the AFD service provider subsequently billed to PWGSC related to that work.

• Obtained a list from the AFD service provider of third party construction, design, and security projects from fiscal year 2010-11.
• Verified the list provided matches the AFD service provider's management report with regards to the total number of projects and the total amount billed.
• Verified the total billing errors identified on the list matches the total billing errors identified in the AFD service provider's report.
• Selected items to be tested from the list based on above sample criteria.
• Requested from the AFD service provider the selected project documents, along with supporting documentation such as purchase orders, change orders, copies of cancelled cheques (front and back), job status inquiry reports, and other related documents.
• Traced items back to source documents. The review team assessed the accuracy and supportability of the amounts billed to PWGSC. 
• Compared amounts assessed by the OAE team to the amounts assessed by the AFD service provider. Noted any discrepancies and followed up with the AFD service provider and RPB for explanation.

This Review was conducted in accordance with the Institute of Internal Auditors' International Standards for the Professional Practice of Internal Auditing.

Criteria

The procedures for this review were based off the OAE's previous OAE Review of the AFD service provider's management review of 2009-10 Construction, Design and Security Costs. Specific criteria for this review were not identified as the procedures were substantially the same as the ones completed in the AFD 2009-10 review, except would be completed on the AFD service provider's 2010-11 management review. 

Review work completed

Fieldwork for this review was substantially completed on May 30, 2014.

Review team

The review was conducted by members of the Office of Audit and Evaluation, overseen by the Director Procurement Audit and under the overall direction of the Chief Audit and Evaluation Executive.

The review was reviewed by the quality assessment function of the Office of Audit and Evaluation.