2013-602 Final Evaluation of the Sydney Tar Ponds and Coke Ovens Remediation Project (Final report)

2013-602 Final report (PDF, 603KB)

July 28, 2014

Main points

What was examined

i. This evaluation examined the Sydney Tar Ponds and Coke Ovens Remediation Project (STPCORP), a $397.7 M ten-year (2004–14) federal-provincial project, for which Public Works and Government Services Canada (PWGSC) provided federal oversight. The Results-based Management and Accountability Framework created for the Project committed to three program evaluations of the STPCORP. A formative evaluation of the Interim Cost-Share Agreement (ICSA) was completed in 2007 and a mid-term evaluation was completed in 2012. This final evaluation serves as an assessment of results achieved by the Project up to December 31, 2013. However, Project progress and financial data were updated in this report to reflect the Project status in these two regards as of March 2014.

ii. The Project is located in sub-program 1.2.6 of the Department's 2014–15 Program Alignment Architecture.

Why it is important

iii. The purpose of the STPCORP was to remediate one of Canada's most contaminated sites, the Sydney Tar Ponds and the Coke Ovens. With a budget of $397.7 M for work to be completed over 10 years, it was the largest remediation project in Canada at the time of its inception. The contamination was caused by nearly a century of steel production. The federal government acknowledged its responsibility to contribute to the cleanup of the sites, as it operated the Coke Ovens facility through the Cape Breton Development Corporation from 1968 until 1973 and contributed financially to the operation of the steel plant from 1986 to 2001. Two previous attempts at remediation (one a joint federal-provincial initative and one a provincial effort) were unsuccessful. In light of these failures, and based on its experience with major Crown projects, such as the successful Confederation Bridge project, and the proximity of its Atlantic Region to the sites, PWGSC was assigned the lead role for the federal government on the STPCORP and provided with the federal funding for Project implementation (cost-shared with the Province), as well as funding for Project oversight.

iv. PWGSC committed to the conduct of a Final Evaluation of the STPCORP as part of the Results-based Management and Accountability Framework for the Project. The Evaluation was undertaken in response to that commitment. The findings of the Evaluation, particularly the lessons learned with respect to governance and management, may be useful to inform other large remediation projects which the government is, or will be managing.

What we found

v. The findings and conclusions below are based on multiple lines of evidence used during the Evaluation.

vi. The Project effectively remediated the Tar Ponds and Coke Ovens sites. A total of 747 K cubic meters of contaminated soil was solidified and stabilized - 680 K cubic meters from the Tar Ponds; and another 67 K cubic meters from the Cooling Pond site, the Coke Ovens Tar Cell and the Coke Ovens Brook locations. All of this material was treated using solidification/stabilization, followed by installation of an engineered cap at the Tar Ponds site to further prevent migration of contaminants into the local environment. As a result of these measures, contaminant levels on site were reduced/contained and contaminant migration was largely eliminated in the site's ground water and in surface water flowing through it.

vii. The sites were made ready to be used safely by the community. Structures and landscaping has been completed and recreational facilities have been in use on the site since September, 2013. All legal obligations under the original Memorandum of Agreement have been satisfied and the sites have been legally transferred to the Province.

viii. The remediation has contributed to the protection of human health and the environment through improved air and water quality. In the case of air quality the objective of remediation activities has been primarily in terms of reduced odours caused by the contaminants from the site, increasing the quality of ambient air. The remediation efforts were not focused on atmospheric contaminant levels in areas adjacent to the site as these were at or below acceptable levels prior to the start of the Project. The Project has also largely eliminated contaminants in water flowing through the site and, as a result, contributed to improved marine water quality in the harbour. Its precise contribution to improved marine water quality is not known due to the continued impacts of other sources of contaminants on marine water quality.

ix. As a result of the reduction of unpleasant odour, the elimination of visual pollution and the establishment of attractive, functional recreational facilities, the Project has contributed to the community's quality of life. The Project also contributed to the protection of human health through its support of the community's quality of life, although the Evaluation is unable to measure the precise nature and extent of this contribution.

x. Remediation of the sites has eliminated, for the most part, the liability of the federal government related to contamination at the site.

xi. Remediation activities were completed on time and within the scope of the original budget. The Project operated under an effective governance structure that contributed to its success, as evidenced by the completion of remediation ahead of schedule and under budget, despite its size and complexity and the number of partners involved.

xii. A budget of $397.7 M was established in the Final Cost-Share Agreement, of which the federal government committed $277.7 M and the provincial government $120 M. The Project was carried out overall in an economical manner. The Project was completed at a cost of $389.3 M, which is $8.4 M below the Cost-Share Agreement budget of $397.7 M. The Independent Engineer's design suggestions contributed to cost-savings realized by the Project by improving the efficiency of remediation activities. Furthermore, the remediation effort was based on an economical technology choice at the start of the Project.

xiii. Up to March 31, 2013, the Project generated 1301 Full-time Equivalents (FTEs) of employment; an estimated 89%, or 1158  FTEs was generated in the Cape Breton region. In addition, the Project was successful in providing training, skills development and employment opportunities for First Nations, women in non-traditional trades, and African Nova Scotians.

xiv. The local community benefited socio-economically from the Project. $180 M was directly expended in the Cape Breton region up to March 31, 2013. Of this $180 M, $71 M in contracts was awarded to First Nations firms from the region, far exceeding the original target of $19 M. In transforming formerly contaminated sites into areas that can be used for recreation and light commercial/industrial purposes, the Project has contributed to improving the quality of life of the community. Other benefits of the Project included removing the stigma of the Tar Ponds, increased community cohesiveness and enhanced community attractiveness and business prospects.

xv. The Evaluation identified a number of lessons learned from this Project that could contribute to improved design and delivery of other large, complex remediation projects managed by the department.

Introduction

1. This report presents the results of the Final Evaluation of the Sydney Tar Ponds and Coke Ovens Remediation Project (STPCORP). The Deputy Minister for Public Works and Government Services Canada (PWGSC) approved the conduct of this evaluation, on recommendation by the Audit and Evaluation Committee, as part of the 2013–18 Risk-Based Audit and Evaluation Plan. The Results-based Management and Accountability Framework (RMAF) created for the Project committed to three program evaluations of the STPCORP. The formative evaluation of the Interim Cost-Share Agreement was completed in 2007. It assessed the design and delivery, success, and cost effectiveness of the Project to date. The mid-term evaluation was completed in 2012. It served to report on early results and provide additional assurance that the project was proceeding as expected. This final evaluation serves as an assessment of results achieved by the Project up to December 31, 2013. However, Project progress and financial data were updated in this report to reflect the Project status in these two regards as of March, 2014.

Profile

Background

2. Sydney Harbour has been subject to effluent and atmospheric inputs of chemical contaminants from the large coking and steel manufacturing facility which operated from 1901 until closure in 2001. During operation of the steel plant and Coke Ovens, thousands of tons of contaminants migrated via the Coke Ovens Brook Connector into Muggah Creek and, via the creek and ground water, into the tidal estuary that forms at the mouth of the creek. As the sludge containing these contaminants built up in the tidal estuary, they formed the Sydney Tar Ponds.

3. In 1968, the federal government acquired ownership of the steel plant and Coke Ovens and operated these facilities as the Cape Breton Development Corporation until 1973, when they were purchased by the Sydney Steel Corporation. The federal government provided $177 M in funding from 1986 until 2001, which allowed the steel plant to continue operations.

4. During the 1980s, Fisheries and Oceans Canada shut down the lobster fishery in Sydney Harbour due to contaminant migration from the sites. The Tar Ponds and the Coke Ovens were contaminated with a variety of toxic substances, including petroleum hydrocarbons, polycyclic aromatic hydrocarbons, polychlorobiphenyls and metals. Odors also emanated from the site as a result of this contamination.

5. In 1986, the federal and provincial governments agreed to a cleanup initiative that ultimately failed for technical and economic reasons. A subsequent proposal, in 1996, by the Province to bury the contaminants under slag from the steel mill was widely condemned by the community and the idea was dropped.

6. In 1996, the Joint Action Group, a community-based advisory group, was created to recommend a cleanup method. In 1998, a Memorandum of Understanding was signed by the Joint Action Group and the federal, provincial, and municipal governments, establishing a framework for participation in the cleanup.

7. In 2002, the Commissioner of the Environment and Sustainable Development criticized the federal government for not having addressed federal contaminated sites and cited Sydney as being among Canada's worst. The 2004 Federal Budget committed $500 M over ten years for remediation activities, noting Sydney as a priority. In April 2004, the federal government approved support for the STPCORP.

Authority

8. Following the 2004 Federal Budget and federal government approval of the Project, the Governments of Canada and Nova Scotia signed a Memorandum of Agreement in May 2004, pledging their commitment to the cleanup of the Sydney Tar Ponds and Coke Ovens sites. The agreement included commitments to an Environmental Assessment, the hiring of an Independent Engineer, community liaison activities, consultations with First Nations, and an emphasis on sustainable development activities. It also committed the parties to a future agreement that would outline the major elements of the Project.

9. Project activities were scheduled to be completed over ten years (2004–14). An Interim Cost-Share Agreement (ICSA) for the first phase was signed and implemented in 2005. This was followed by a Final Cost-Share Agreement (FCSA) covering the second phase, from 2007 to 2014.

Roles and responsibilities

10. The federal and provincial governments shared responsibility for funding and oversight. The 2007 FCSA defined the cost-sharing arrangement, budget, and reporting requirements; payment procedures; governance and implementation structures; roles and responsibilities; and the Project work breakdown structure.

11. PWGSC was assigned the lead role for the federal government on the Project in light of the two previous failed attempts at remediation, and based on its experience with major Crown projects, such as the successful Confederation Bridge project, and the proximity of its Atlantic Region to the sites. Its role was to oversee federal interests and ensure that the Project was delivered on time, on budget, and within scope. To carry out its role, PWGSC was provided with funding for the federal share of Project implementation (remediation, and related costs) and for Project oversight. Funding was also authorized for the Department of Justice, Environment Canada and Health Canada to carry out their roles on the Project. Natural Resources Canada (NRCan), and Fisheries and Oceans Canada also participated in the Project but did not receive funding. More information on the PWGSC role can be found in this report's section on governance.

12. The Nova Scotia Department of Transportation and Infrastructure Renewal (which was known as the Department of Transportation and Public Works at the time of the signing of the FCSA, in 2007) had the responsibility for project implementation and operations. The provincial Department of the Environment (which, in 2007, was known as the Department of Labour and the Environment) was also involved in the Project as the environmental and occupational health and safety regulatory authority.

13. In August 2004, the Province established the Sydney Tar Ponds Agency (STPA) as a special operating agency to manage and implement the Project. The STPA was responsible for daily project management and implementation activities, support and contract services, work breakdown, documentation of activities, adherence to health, safety and environmental requirements, and communications. The STPA was accountable to the Project Management Committee (co-chaired by federal and provincial representatives), which had ultimate decision-making authority for the Project. The Operational Advisory Committee (oversaw implementation), Environmental Management Committee (advised on environmental issues), and the Community Liaison Committee (represented community issues) reported to the Project Management Committee.

14. The Independent Engineer provided monthly reports directly to the Project Management Committee; exercised a monitoring and validation role to ensure that technical requirements were addressed and that effective management controls were in place; and reviewed and reported on costs and contract compliance, and other aspects of the Project.

Resources

15. The estimated cost of project implementation, including management, engineering and construction and related activities was $397.7 M over ten years (2004 to 2014). Under the FCSA, the federal government committed $277.7 M to the STPCORP and the Government of Nova Scotia committed $120 M. The first $297.7 M in funding was to be shared in a 60-40 ratio between the federal and provincial governments, respectively, while the final $100 M was to be provided by the federal government. As such, any final cost savings would accrue to the federal government. The Agreement stipulated that there would be no cost overruns and, if they did occur, the Province was responsible for them.

16. PWGSC received funding of $282.2 M for its share of the Project implementation costs that were cost-shared with the Province. Of this amount, $4.5 M was designated as a cost-share reserve, to be used in exceptional circumstances only, should the federal share of the Project implementation costs exceed $277.7 M, the amount that was PWGSC's agreed share under the FCSA, as noted above in paragraph 15.

17. The federal government also authorized funding for operating expenses for the three federal departments with significant ongoing involvement in the Project. PWGSC was provided funding of $25.9 M to enable it to carry out its role as the lead federal oversight agency. This funding extends into fiscal year 2014–15 to allow PWGSC to complete financial closure and due diligence aspects of the Project. Environment Canada was authorized $7.6 M and Health Canada $5.5 M to provide technical and scientific advice in the areas of environment and human health.

Remediation methodology

18. The approach to remediation of the sites was multi-faceted. At the Tar Ponds site, contaminated material was mixed in-situ, in small units (cells) and combined with Portland cement in a process called solidification/stabilization, which creates a structurally enhanced, low permeable mass that immobilizes contaminants in place. The purpose is to provide increased structural strength for capping and to decrease the mobility of contaminants. The resulting mass was then capped with several layers of materials, including clay, soils and inert slag from the steel making operation, as well as an impermeable liner.

19. Contaminated material from the Cooling Pond, the Coke Ovens Tar Cell and Coke Ovens Brook was transported off site, either to the Tar Ponds or to a provincial pug mill, where it was solidified and stabilized. The total amount of material remediated using solidification/stabilization was approximately 747 K cubic meters.

20. The material at the Coke Ovens site, with the exception of material from the Tar Cell and Coke Ovens Brook, was not suitable for solidification/stabilization, for a variety of reasons, including the presence throughout the site of underground building materials and piping. Consequently, the Coke Ovens site was remediated through the installation of a surface cap and the capture and treatment of contaminated ground water in a water treatment plant. The Province of Nova Scotia will continue to operate the water treatment facility on a long-term basis.

Logic model

21. A logic model is a visual representation that links a program's activities, outputs and outcomes; provides a systematic and visual method of illustrating the program theory; and shows the logic of how a program is expected to achieve its objectives. It also provides the basis for developing the performance measurement and evaluation strategies, including the evaluation matrix.

22. A logic model for the Project was developed based on a detailed document review, meetings with program managers and interviews with key stakeholders. It was subsequently validated with program staff. The logic model is provided in Exhibit 1.

Exhibit 1: Logic model

Program activities

Remediate the Tar Ponds

23. The remediation of the Tar Ponds was one of the primary activities of the Project. Its main outputs were the solidification/stabilization of the contaminated sediment, the installation of an engineered cap and the provision of surface and ground water controls. The intended outcome for this output is that contaminant migration is eliminated and contaminant levels on-site are reduced/contained.

Remediate the Coke Ovens

24. The remediation of the Coke Ovens was one of the primary activities of the Project. Its main outputs were the containment of contaminants and the provision of surface and ground water controls. The intended outcomes for these outputs are that contaminant migration is eliminated and contaminant levels on-site are reduced/contained.

Develop and support post-remediation land-use

25. The activity of developing and supporting post-remediation land-use had several outputs, including, transfer of ownership of the site to the provincial government, the establishment and approval of a long-term monitoring and maintenance plan, and the completion of structures and landscaping. The intended outcome of these outputs is that the sites are ready to be used safely by the community.

Focus of the evaluation

26. The Results-based Management and Accountability Framework created for the Project committed to three program evaluations of the STPCORP. The formative evaluation of the Interim Cost-Share Agreement was completed in 2007. It focused on the design and delivery, success, and cost effectiveness of the Project to date. The mid-term evaluation was completed in 2012. It focused on the continued relevance of the Project and on early results while also providing additional assurance that the Project was proceeding as expected.

27. The objective of this final evaluation was to determine the Project's performance in achieving its expected outcomes in accordance with the Treasury Board Policy on Evaluation. Relevance was not addressed because it was fully addressed in the mid-term evaluation and the STPCORP has come to an end as of March 2014. The Evaluation assessed the Project for the period of April 2011 to December 2013. However, financial information and data related to completion of final project tasks were updated in this report to reflect status as of March 31, 2014.

28. An evaluation matrix, including evaluation issues, questions, indicators and data sources, was developed during the planning phase. Information on the approach and methodologies used to conduct this Evaluation, as well as limitations encountered in the planning and conduct of the Evaluation are found in the "About the evaluation" section at the end of this report.

Findings and conclusions

29. The findings and conclusions below are based on multiple lines of evidence used during the Evaluation. They address three main aspects of performance: outcome achievement, design and delivery, and efficiency and economy.

Performance

30. Performance is the extent to which a program is successful in achieving its objectives and the degree to which it is able to do so in a cost-effective manner that demonstrates efficiency and economy.

Outcome achievement

31. The Evaluation examined the degree to which this program achieved its intended immediate, intermediate, and departmental program outcomes. The intended outcomes of the Project are identified in italics below, followed by an assessment of the extent to which they have been achieved.

32. To evaluate achievement of this outcome, the Evaluation reviewed the percentage of contaminants targeted for remediation that were successfully remediated under the Project, either through capping and solidification/stabilization at the Tar Ponds or through capping and water treatment at the Coke Ovens site. As well, the Evaluation assessed the effectiveness of remediation, based on the estimated reduced migration of contaminants at the Tar Ponds; and on the effectiveness of ground water treatment at the Coke Ovens. These assessments were based on information and data obtained from environmental research articles and from the Independent Engineer reports and on the views of knowledgeable stakeholders.

33. The Evaluation has found that STPCORP has effectively remediated the Tar Ponds and Coke Ovens sites. The STPCORP has reduced or contained contaminants at the Tar Ponds and Coke Ovens sites and has largely eliminated migration of contaminants into the harbour. The Results-Based Management and Accountability Framework prepared for the Project states "intermediate outcomes of these four activity areas [remediate the Tar Ponds; remediate the Coke Ovens; remediation support; post-remediation] will be to eliminate all pathways to exposure and have the Province take responsibility for sites that are ready for use." Thus, the implication was that the Project could entirely eliminate migration. However, Project officials have indicated that complete elimination was never considered possible. All stakeholders who were able to respond to an interview question regarding the effectiveness of remediation indicated that it has been effective because, in their view, migration has been largely eliminated. These knowledgeable stakeholders included Project officials responsible for environmental monitoring and Environment Canada, Health Canada and NRCan officials who were current or past members of the Environmental Management Committee. These officials ranged from knowledgeable to expert on the environmental aspects of the Project.

34. As noted earlier in the report, a total of 747 K cubic meters of contaminated soil was solidified/stabilized - 680 K cubic meters from the Tar Ponds; and another 67 K cubic meters from the Cooling Pond site, the Coke Ovens Tar Cell and the Coke Ovens Brook locations. All of this material was treated using solidification/stabilization, followed by installation of an engineered cap at the Tar Ponds site to further prevent migration of contaminants into the local environment.

35. The effectiveness of the solidification/stabilization approach can be assessed based on the "contaminant mass flux" (CMF) concept. In the case of the STPCORP, CMF measured the migration, per unit time, of contaminants out of the capped monolith at the Tar Ponds site. The evaluation assessed the effectiveness of remediation based on two measures of mass flux:

1. a comparison of post-remediation CMF levels or values with pre-remediation levels; and
2. a comparison of post-remediation CMF against levels established by the Project Management Committee officials, in 2008, as acceptable performance levels, based on the recommendations of engineering consultants. These recommendations were based on computerized analysis of the expected permeability of the engineered cap and criteria established for post-remediation leachability of the solidified and stabilized material.

36. With respect to the first comparison, the evaluation found, based on Project data, that post-remediation CMF values for the 47 contaminants tested for at the start of the Project ranged from 0.1% to 4.6% of their pre-remediation values. For 44 of these contaminants, the CMF was .7% or less of pre-remediation CMF.

37. With respect to the second comparison, the evaluation found that the post-remediation CMF values were well below the levels established by the Project Management Committee as acceptable. In particular, the CMF values for the three contaminants with the highest post-remediation test results were 13%, 15% and 28% of the CMF values deemed acceptable.

38. Overall, these findings indicate that remediation has been effective, as indicated by post-remediation CMF values which are well below pre-remediation levels and also well below the Project's acceptable post-remediation levels.

39. The bulk of the material (91%) at the Coke Ovens site was not solidified/stabilized but was remediated by installation of a non-engineered cap and by construction of a ground water capture and treatment system. The cap prevents direct contact with contaminants and erosion and limits infiltration of rainfall runoff. The water capture and treatment system enables the treatment of 100% of contaminated ground water flowing through the Coke Ovens site. The Project has established criteria for contaminant levels in the treated water, based on a combination of existing, widely accepted criteria. The treated ground water is required to be below these Project criteria. Based on a review of a sample of weekly post-treatment test results from September 2010 to December 2013, levels of all contaminants are below the Project criteria and, in many cases, below detection thresholds. This water treatment will likely continue to be necessary for an estimated five to 25 years in order to continue to remove contaminants from the water and prevent their migration.

40. In summary, the contaminants at the Coke Ovens and Tar Ponds sites have been remediated, based on the approved remediation plan for the Project. This has resulted in the reduction or containment of all contaminants and has largely eliminated contaminant migration. All stakeholders who addressed this issue agreed that remediation has been effective.

41. To evaluate achievement of this outcome, the Evaluation assessed the extent to which built structures and landscaping are completed and appropriate for planned use; whether a completed and approved long-term monitoring and maintenance plan is in place; and whether the Independent Engineer has confirmed the safety of the site. The Evaluation used multiple lines of evidence to assess achievement of these measures including interviews, project documentation, and the Project's public opinion research.

42. The Tar Ponds site was developed into a public park. The Evaluation found it has been used for a variety of recreational purposes since its official opening in September 2013. The recreational facilities at the site ( e.g., walking and biking trails, football and soccer fields, and play structures) have been in use since then. Project officials have confirmed the sites are ready to be used safely by the community.

43. The Independent Engineer provided the Substantial Completion Certificate on March 14, 2014, indicating that all work on the Project, including the future site use element, is complete. Issuance of this certificate is predicated on there being no safety issues at the sites. Potential safety issues could have included the presence of hazardous materials left on the sites ( e.g. wood with sharp sides or nails projecting; toxic chemicals etc); or structures that were not constructed properly, resulting in safety hazards, as well as any environmental hazards ( e.g. the continued leaching of contaminants from the site at levels that constituted a health hazard).

44. As all federal and provincial obligations under the 2004 Memorandum of Agreement are fulfilled, including the Long-Term Monitoring and Maintenance Plan, responsibility for the site was transferred to the provincial government through a Land Transfer Agreement on March 14, 2014.

45. To evaluate achievement of this outcome, the Evaluation examined the Project's impacts on the local environment, in terms of air and water quality and in terms of estimated reductions in health risks to the local community as a result of the remediation. As well, the Evaluation obtained the views of knowledgeable stakeholders regarding the impacts of the Project on human health and the environment. With regard to air and water quality, and physical health, the views of stakeholders are substantiated by Project and other research data; with regard to quality of life, the views are stakeholder impressions.

46. Based on these lines of evidence, the Evaluation found that the Project contributed to protection of the environment, although the precise contribution, in terms of improved air or water quality, is not known. As well, the extensive environmental monitoring and mitigation activities carried out during the course of the Project ensured that the remediation effort itself did not adversely affect the environment. The Project also contributed to an improvement in the community's quality of life, although the Evaluation is unable to measure the precise nature and extent of this contribution.

47. The impacts of the Project on water quality can be examined from three perspectives: (a) impacts on surface water quality, (b) impacts on ground water quality, and (c) impacts on marine water quality in Sydney harbour.

48. According to the STPCORP Environmental Impact Statement, "surface water is water flowing within streams and channels on the site." This water may originate in surface water or ground water upstream from the site and from rainfall. Surface water, if it comes into contact with contaminants, can affect other ecological components. It can expose animals and plants, in streams and in Sydney Harbour, to contaminants. It can also expose humans to contaminants, through ingestion of the water or through ingestion of animals contaminated by the water.

49. The Evaluation found that the Project improved surface water quality by diverting surface water to creeks flowing around the perimeter of the sites and isolating it from possible contaminants in the ground through the use of impermeable liners in creek beds. These liners prevent surface water in the creeks from coming into contact with the capped, contaminated material in the ground at the Coke Ovens and Tar Ponds sites. This, together with the capping, the solidification/stabilization and the ground water treatment at the Coke ovens has largely eliminated the migration of contaminants into surface water.

50. According to the STPCORP Environmental Impact Statement, "ground water resources are located in bedrock under the Coke Ovens and the Tar Ponds. Potential effects on ground water are a concern for several reasons: the Project directly interacts with ground water resources; ground water represents a potential pathway to other important aspects of the environment such as human/wildlife health; and ground water flows to surface water streams." The water table under the Coke Ovens generally slopes from east to west. Ground water under the residential neighbourhoods north and south of the Coke Ovens generally flows toward the Coke Ovens, and then westward along the valley floor. From the western edge of the Coke Ovens, ground water flows westward to the South Tar Pond most of the time. In the area between the Coke Ovens and the South Tar Pond, ground water generally flows toward Coke Ovens Brook Connector and the South Tar Pond. This flow pattern carries ground water away from nearby homes and toward the South Tar Pond.

51. Because of these geological parameters, ground water was a more critical issue at the Coke Ovens location than at the Tar Ponds. By treating ground water at the Coke Ovens site, the Project has prevented contaminated ground water from flowing into the Tar Ponds site, where remediation activities (solidification/stabilization and capping) have largely eliminated ground water migrating off the site.

52. As noted earlier in this report, treatment of ground water does not eliminate contaminants at the Coke Ovens site but captures them and removes them from water leaving the site. Thus, the treatment prevents, in the shorter term, migration of contaminants; while, in the longer term, it contributes to an absolute reduction in contaminants at the Coke Ovens site.

53. Based on information provided by Project officials, measurement of the overall impacts of remediation on ground water quality at the Coke Ovens site will take many years, because of the very slow rate of attenuation of contaminants in ground water and because of the numerous other potential sources of ground water contamination in the area. However, all ground water leaving the Coke Ovens site will have been treated and thus the migration of contaminants will be prevented.

54. According to Project officials and based on Project monitoring results, marine water quality in Sydney Harbour has improved as a result of the Project, in that surface water and ground water entering the harbour from the Project sites is no longer contaminated.

55. Marine water monitoring was conducted over the course of the Project, primarily to assess whether the remediation activities themselves were having any effect on marine water quality. This monitoring program consistently demonstrated that contaminants were at or below baseline levels or were below detection thresholds over the course of the Project.

56. While these results indicate that the remediation activities did not themselves adversely affect the marine environment, measuring the overall impact of the Project on marine water quality is complicated by the following factors:

1. Levels of contaminants in harbour water can vary as a result of many factors, including migration of contaminants from other industrial sites in the area;
2. Even prior to the start of the Project, contaminant inventories in Sydney Harbour were already undergoing natural containment by the continuous deposition of less contaminated sediments at the bottom of the harbour. Project remediation activities have already and will continue to contribute to shorten the time period required to bring back mean sediment concentrations of metal and organic contaminants to levels below the effect range median (above which organisms are very likely to be negatively affected by the presence of contaminants).

57. Given this, over the long term, the Tar Ponds and Coke Ovens cleanup will have had a positive effect on marine plants, mammals and fish and their habitat, by improving the quality of the habitat in the South Arm of Sydney Harbour. The improvement will occur because the flow of contaminants from the sites has been largely eliminated.

58. Air quality is one of the most important aspects of the environment. It affects the health and well-being of humans, plants, animals and ecosystems.

59. For 100 years, a steel mill and coke ovens located near the centre of Sydney produced air emissions that affected the community and the local environment. Many other related industries existed on the Coke Ovens site. These included a coal tar plant, a cement company, asphalt paving company, a fuel gas and oil company, metal processors, and a brick company. These industries also produced significant emissions.

60. According to the STPCORP Environmental Impact Statement, "The site, since the closure of its industrial activities, has very little effect on the quality of air in Sydney, as the extensive air monitoring conducted by the Sydney Tar Ponds Agency confirms. Sydney's air quality today is better than it has been in a very long time. It is comparable to air quality in Halifax, and much better than that in large Canadian cities such as Montreal and Toronto".

61. Several Tar Ponds and Coke Ovens cleanup activities, such as excavation of contaminated materials, solidification/stabilization, land-farming, and installation of cover materials could have affected air quality. Each of these activities creates dust, emits gases and causes odours.

62. While air quality, at the site and in the adjacent community prior to the Project, was not considered a health issue, it was considered a nuisance, primarily because of the odour. Air quality on the site and in the community was monitored over the course of the project based on two methodologies: ambient air and odours complaints. This monitoring was conducted primarily to monitor the impacts of remediation activities. Data analysis from this monitoring demonstrated that, with very minor exceptions, the remediation activities had no negative impact on air quality, as measured by both exceedences of accepted standards and odours complaints from the community.

63. The Evaluation assessed evidence of air quality in two ways: through ambient air monitoring and through odours complaints monitoring. Ambient air monitoring (the first aspect of air quality) was monitored over the course of the Project, through a 24-hour averaging Ambient Air Monitoring Program. This program assessed the conditions outside the perimeter of the Remediation Area every six days within every month. It was conducted at six fixed monitoring stations, and a Real-time Air Monitoring Program was conducted at a variable number of monitoring locations outside the perimeter but adjacent to active excavation or other work. Criteria have been established specifically for this Project or adopted from other trusted jurisdictions, such as the Ontario Ministry of the Environment or the Canadian Council of Ministers of the Environment. Atmospheric contaminant levels adjacent to the site, as measured by the Project air quality monitoring system, were at or below acceptable levels from 2007 (pre-project) to 2012. Therefore, air quality was not a major issue.

64. The second aspect of air quality dealt with odour management. Odour resulting from the solidification/stabilization activity taking place in the Tar Ponds area was monitored by the solidification/stabilization contractor, by the air monitoring consultant both within and beyond the fence line, and by the STPA through the daily deployment of an odour monitoring technician.

65. To minimize odour, the STPA used odour-suppressing foam to reduce emissions of contaminants. Gas chromatography/mass spectrometry was also used to assist in characterizing air quality in conjunction with regular monitoring using a nasal ranger field olfactometer at 17 sampling stations.

66. Air monitors indicated that the contractor consistently worked safely within prescribed limits and there were no significant exceedence of air quality standards. As solidification/stabilization progressed, odour levels showed a distinct downward trend as mitigation was aggressively applied. This was evidenced by the substantive reduction in complaints from the public following year one as well as on-site observations by odour monitors. Remediation activities on the site of the Project contributed to the elimination of odours emanating from the site, therefore increasing the quality of ambient air.

67. Based on stakeholder interviews, in particular with individuals with expertise in environmental health, the Tar Ponds and Coke Ovens sites did not represent a direct threat to the physical health of the community prior to remediation. The available documentation supports this view. A 2001 study, prepared for PWGSC, of the health risks to members of the community north of the Coke Ovens site from various substances identified only two substances—lead and arsenic—at levels warranting remedial action. It further found that the risks associated with these were higher from food and the municipal water supply than from the soil in the area.

68. While the sites did not pose a direct risk to human health, they did pose an indirect risk, through consumption of fish, in particular shellfish, from the harbour. However, this risk was essentially eliminated through closure of the lobster fishery, the only commercial fishery in the harbour in 1982. As well, studies conducted by the Department of Fisheries and Oceans in 2001 showed that contaminant levels in the marine environment were naturally attenuating prior the Project.

69. Based on interviews with Project stakeholders, including members of the community and health experts, the main benefit of the remediation with respect to human health, has been on the quality of life in the community. As well, several stakeholders cited improvements to overall quality of life, air quality and odour reduction as health benefits of the Project.

70. In conclusion, the Project contributed to protection of the environment through improved air and water quality. In the case of air quality the impact has been primarily in terms of reduced odours from the site, as atmospheric contaminant levels in areas adjacent to the site were at or below acceptable levels prior to the start of the Project. The Project has also largely eliminated contaminants in ground water and surface water flowing through the site and, as a result, contributed to improved marine water quality in the harbour. Its precise contribution to improved marine water quality is not known due to the continued impacts of other sources of contaminants on marine water quality. The remediation activities themselves did not adversely impact on the environment. In transforming formerly contaminated sites into areas that can be used for recreation and light commercial or industrial purposes it has contributed to improving the quality of life of the community.

71. A Joint Review Panel was established for the remediation of the Tar Ponds and Coke Ovens sites in September 2005, and presented its report in July 2006. The Joint Review Panel was composed of a group of independent experts appointed by the Federal Minister of the Environment to conduct and report on an environmental assessment. The Panel's Environment Assessment Report included recommendations on how the Project could support women in non-traditional occupations and African Nova Scotians. Moreover, a framework agreement had been reached reached with First Nations groups regarding their participation prior to the Joint Review Panel's establishment.

72. To evaluate achievement of this outcome, the Evaluation examined primary benefits, including project expenditures that went to regional, provincial, and Canadian firms; the value of contracts with First Nations firms; skills/capacities acquired through project training by First Nations groups, women and African Nova Scotians; and expected use of the site by the community. Secondary benefits examined included the value of contracts obtained subsequently by First Nations firms as a result of competencies developed; subsequent employment realized by women and African Nova Scotians as a result of project training; and other unintended benefits. Discussion of primary and secondary benefits relating to targeted groups is organized by beneficiary group.

73. The Evaluation assessed achievement of these measures using multiple lines of evidence, including interview findings, project documentation (primarily Economic Benefits Reports and data compiled by PWGSC Atlantic Region staff) and the Project's public opinion research.

74. Based on the above measures, the Project resulted in direct economic benefits of $180 M and also provided socio-economic benefits to targeted groups – First Nations, women in non-traditional trades and African Nova Scotians. It also resulted in secondary benefits to the community, such as improved quality of life, although quantitative data on these were, for the most part, not available.

75. The Project achieved a high level of local expenditures. As of March 31, 2013, 52% ($180 M) of expenditures went to Cape Breton and 11% ($38.4 M) to Nova Scotia. Overall, 93% ($320 M) of expenditures went to Canadian firms or individuals. Of the total employment generated by the Project up to March 31, 2013, (2,573,249 person hours or 1301  FTEs), approximately 89% (2,290,191 person-hours or 1,158  FTEs) was in Cape Breton. Twenty-eight out of the 32 contractors for the Project were from the Cape Breton region.

76. Participation by First Nations was a priority in the 2004 Memorandum of Agreement, and was reaffirmed in the October 2005 Protocol Agreement with First Nations representatives. Nineteen million dollars in contracts were identified as "set-asides," where the lead bidder was required to be a First Nations firm (in order to ensure cost-competitiveness, successful bids had to be within 5% of the lowest bid). First Nations firms ultimately won $71 M in contracts out of a total of approximately $323.5 M, most in competitions not involving set-asides. According to interviewees, experience gained on smaller works early in the Project generated opportunities for First Nations to partner with other firms on larger works.

77. A total of 85 persons from First Nations were employed on the Project, resulting in 140 K hours or 70 full time equivalents of employment. In total, First Nations accounted for 5.4% of employment on the Project; for the sake of comparison, First Nations represent 2.1% of the Nova Scotian population and 3.6% of the Cape Breton population. In addition, over $1 M in funding was provided to the Unama'ki Economic Benefits Office, which is focused on upgrading the skills of First Nations individuals in the Cape Breton region and on identifying business opportunities for First Nations firms. This Office recently expanded its operations to all of Nova Scotia and, in line with this change, has changed its name to the Mi'kmaq Economic Benefits Office of Nova Scotia.

78. Interviewees noted that in collaboration with the Unama'ki Economic Benefits Office, additional funding was acquired for training through the federal Aboriginal Skills and Employment Partnership Program (ASEP) and partnerships with businesses involved in the Tar Ponds Project.

79. Interviewees indicated that the Project contributed to First Nations firms obtaining subsequent contracts outside of the Project, although the Evaluation was unable to obtain details regarding specific contracts.

80. Recommendation 34 of the Joint Review Panel's Environmental Assessment on the Project called for a strategy to facilitate training and employment of women in non-traditional trades ( e.g. construction, scientific, and trades and technology).

81. Evidence demonstrates that the strategy was implemented and was successful. $409 K in funding was provided for initiatives in support of awareness, and for training and skills development for women in trades and technology. These initiatives were carried out directly or through a local women's employment and development support centre (the Anne Terry Project) and the Women's Unlimited training program.

82. Training programs funded included, among other initiatives, a portfolio development program (total of 24 participants in 2011 and 2012); a 14 week career decision-making program in trades and technology; a one day Career Course for Young Women provided annually by the STPA (38 participants over three years); leadership and sexual harassment awareness workshops provided by STPCORP staff (78 participants in total); and 3 years of funding for Techsploration – a program that exposes young women to careers in non-traditional fields – at a local high school. Two years of funding was provided for a coordinator for the Ann Terry Project – a centre that provides career counseling and development to women – to provide coaching, guidance, and leadership to those pursuing careers in trades and technology. As well, six students were provided scholarships totaling $18,000.

83. Interviewees noted that the initiatives for women were successful as they met expectations, were positively received, created attitudinal changes, boosted confidence, and created long-term infrastructure. Public Opinion Research showed that most respondents believed the career courses, internships and scholarships contributed to the development of skills/capacity. It was noted that nine Women Unlimited graduates and several interns obtained subsequent employment in trades and technology and that 25 Women Unlimited graduates subsequently enrolled in trades and technology programs. However, interviewees noted it was challenging for women to get jobs in trades and technology because new graduates are generally placed at the bottom of union priority lists.

84. In addition to providing the awareness, education and training initiatives outlined above, the Project also employed forty-two women in non-traditional occupations. The Evaluation was unable to obtain information to inform an assessment of the extent to which the training and education initiatives contributed to these employment figures, nor to determine what percentage of total trades and technology employment on the Project was represented by women.

85. Recommendation 35 of the Joint Review Panel's Environmental Assessment called for a strategy to aid in the employment of African Nova Scotians. Evidence demonstrates that the strategy was implemented successfully. Thirty-five African Nova Scotians were employed in the Project. In total, these individuals accounted for 2.2% of those employed on the Project; for comparison purposes, they represent 1.9% of the Nova Scotian population and .8% of the Cape Breton population. The Project also provided funding ($241 K) for the Sydney African Nova Scotian Employment Centre. This funding was used for training of board members and staff; to support employment of an Economic Development Officer for three years; and for twenty-eight bursaries for African Nova Scotians.

86. In addition to the $241 K of funding, Project partners provided 10 student jobs, and tours and presentations to students. An additional $25 K was provided for the tuition of a Nova Scotian engineering student/intern of African descent. The work of the Economic Development Officer led to the obtaining of outside funding for training programs, including funding for an older worker program (36 participants) and skills gap training (16 participants). In addition, all remediation contracts included a requirement for employment of African Nova Scotians.

87. While these initiatives were successful in terms of participation, interviewees noted that, overall, African Nova Scotians did not maximize opportunities resulting from the Project, due to capacity issues and conflict in the community. The African Nova Scotian Employment Centre struggled with communication, building awareness, recruitment, and responsiveness to businesses. A few interviewees felt more initiatives/opportunities should have been provided. It was noted that, as was the case for women, transforming acquired skills into employment was a challenge for this group because new trades graduates are at the bottom of the union job priority lists. No information was available on subsequent employment opportunities realized by African Nova Scotians, resulting from the skills acquired through training and/or through employment on the Project.

88. Despite these challenges, the Project was successful in terms of ensuring all contracts included a requirement for employment of African Nova Scotians, as well as in providing awareness and training in support of employment of African Nova Scotians and in the employment African Nova Scotians on the Project.

89. The majority of interview respondents cited other Project benefits such as removing the stigma of the Tar Ponds and bringing the community together. This is corroborated by the Project's Public Opinion Research, in which respondents felt the Project would improve the community's reputation. Interviewees also cited sharing lessons learned, improving government's reputation, and enhancing the community's attractiveness/business prospects.

90. Analysis by Nova Scotia Lands in 2009 and 2011 showed that construction did not reduce nearby property values. Interviewees indicated that they expect that the enhanced community attractiveness resulting from the Project will or already has increased adjacent property values. This perception is corroborated by several research papers published prior to the remediation of the site, which provided estimates of the extent to which the Tar Ponds and Coke Ovens reduced housing prices.

91. Interviewees felt that the sites would be used extensively by the community. Several planned to use the facilities, and had already noticed an interest among community members. The public opinion survey also indicated that residents planned to use the site's facilities.

92. Overall, the STPCORP resulted in socio-economic benefits for the community, including direct expenditures in the Cape Breton region of $180 M up to March 31, 2013. Of this amount, $71 M in contracts was awarded to First Nations firms from the region, far exceeding the original target of $19 M.

93. In addition, the Project was successful in providing training, skills development and employment opportunities for First Nations and to women in non-traditional occupations, and to African Nova Scotians. Other benefits of the Project included removing the stigma of the Tar Ponds, increased community cohesiveness and enhanced community attractiveness and business prospects.

94. This is an outcome identified in the PWGSC 2014–15 Program Alignment Architecture for the entire Accommodation and Real Property Services Program. Five additional sub-programs contribute to the achievement of this higher level outcome. While the STPCORP can contribute to this outcome, it cannot, by itself, achieve it.

95. To evaluate the Project's contribution to this outcome, the Evaluation examined the extent to which the Project resulted in economic benefits to the community and to Canada in general, based on stakeholder views regarding the best long-term use of the site compared to alternatives and on the evaluation findings in relation to Intermediate outcome 2: Local Community Socio-economic Benefits. As well, the evaluation obtained the views of legal experts regarding liabilities of the federal government for the site to assess whether they had been minimized. It was found that the STPCORP contributed to the strategically managed provision of a real property portfolio that maximizes economic benefit and minimizes short and long-term liability.

96. Stakeholder views are that the best overall long-term use of the remediated site would be for non-commercial purposes. Many respondents stated that recreational use was the most appropriate use for the remediated Tar Ponds site. Interviewees noted that other uses, such as residential, were not appropriate due to limitations on what can be developed on the solidified and capped material ( i.e., foundations cannot be dug because the area was capped with clay, gravel, and a liner). Interviewees also noted recreational use of the remediated site was appropriate because it meets a need for recreational facilities; connects the community; offers a diverse variety of facilities; and helps increase physical activity. With regards to the remediated Coke Ovens site, most respondents agreed commercial or light industrial use of the Coke Ovens was appropriate given the limitations with foundations and utilities depth due to the cap. However, a few interviewees felt that there was already enough vacant industrial land in the community.

97. The Project made every effort to maximize economic benefits from the sites over the course of the remediation. As discussed in the section on Intermediate outcome 2: Local Community Socio-Economic Benefits, as of March 31, 2013, 93% of Project expenditures occurred in Canada. This represents a very high percentage of expenditures occurring in Canada, considering that some high cost items, such as odour-suppressing foam, could not be purchased in Canada and the requirement for provinces to comply with the North American Free Trade Agreement.

98. By remediating the Tar Ponds and Coke Ovens, most interview respondents, including the PWGSC legal expert, felt that the federal government had substantially reduced, if not eliminated, its liability for the sites. Interviewees noted that the transfer of the land to the Province had largely eliminated the liability; however, several noted that there will always be some, albeit small, liability if there are future contaminant releases.

99. In conclusion, the Project has contributed to the Departmental Program Outcome, of a strategically-managed real property portfolio that maximizes economic benefits and minimizes liability by ensuring the most appropriate future use of the remediated sites; by ensuring that a high percentage of Project expenditures occurred in Canada; and by eliminating, for the most part, the liability of the federal government related to contamination at the site.

Conclusions: Outcome achievement

100. The Project's immediate outcomes have been achieved. Interviews and analysis of documents and data show that STPCORP reduced and contained the contaminants on site and largely eliminated the contaminant migration into Sydney Harbour. The sites are already being used for recreational purposes and are fully ready for safe use by the community.

101. The Project has contributed to the achievement of the intermediate outcomes. The Project has contributed to protection of the environment, through improved air and water quality. In the case of air quality, the impact has been primarily in terms of reduced odours from the site, as atmospheric contaminant levels in areas adjacent to the site were at or below acceptable levels prior to the start of the Project. The Project has also largely eliminated contaminants in ground water and surface water flowing through the site and, as a result, contributed to improved marine water quality in the harbour. Its precise contribution to improved marine water quality is not known due to the continued impacts of other sources of contaminants on marine water quality. The Project has contributed to improving the quality of life in the community; has resulted in direct economic benefits to the community; and has contributed to other socio-economic benefits, such as enhanced community business prospects. It has also significantly benefited First Nations firms and individuals and women pursuing non-traditional careers and African Nova Scotians.

102. The Project has contributed to the Departmental Program Outcome, of a strategically managed real property portfolio that maximizes economic benefits and minimizes liability by ensuring the most appropriate future use of the remediated sites; by ensuring that a high percentage of Project expenditures occurred in Canada; and by eliminating, for the most part, the liability of the federal government related to contamination at the site.

Governance

103. It is essential to the success of large, complex projects of this kind that the roles and responsibilities of the Project partners and of the various elements of the governance structure ( e.g. committees. Independent Engineer) are clearly defined and are well understood by each of the partners. The Evaluation reviewed the project governance structure (composed of Partners, Committees, and Engineers) to assess its contributions to the effectiveness of the Sydney Tar Pond Agency in implementing the Project. Specifically, the Evaluation examined the roles and responsibilities of:

• Partners (Environment Canada, Health Canada, Nova Scotia Department of Transportation and Infrastructure Renewal, Nova Scotia Department of the Environment and the Sydney Tar Ponds Agency and PWGSC).
• Internal committees (Project Management Committee, Operations Advisory Committee and Environmental Management Committee).
• External committee (Community Liaison Committee).
• The Independent Engineer.

104. Exhibit 2 (below) outlines the STPCORP structure and the reporting relationships intended among the entities involved in the governance of the Project.

Exhibit 2: Planned project governance structure

105. In addition, as part of its assessment of project governance, the Evaluation examined the costs of project governance, based on the actual versus originally budgeted costs of the key governance elements, the Sydney Tar Ponds Agency, PWGSC and the Independent Engineer.

106. Based on the above lines of evidence, the Evaluation found that the Project operated under an effective governance structure that contributed to the success of the Project, as evidenced by the completion of remediation activities five months ahead of schedule and $8.4 M under the federal component of the cost-share agreement's budget, despite the size and complexity of the Project and the number of partners involved. As well, the overall cost of Project governance is slightly under budget. PWGSC realized additional savings in its oversight budget and from not requiring use of its cost-share reserve funding. The Project schedule and budget are discussed in detail in the sections on Efficiency and Economy.

107. As described in the sections below, the roles and responsibilities of Project partners were clearly and consistently outlined in Project framework documents and, for the most part, clearly understood by Project stakeholders. In the views of stakeholders interviewed, this contributed to the success of the Project. While the provincial implementing agency, the STPA, at times displayed an incomplete understanding of oversight role played by PWGSC, this did not affect PWGSC effectively carrying out its role on the Project.

108. The Project framework documents reviewed consistently referred to Environment Canada and Health Canada roles as related to their regulatory responsibilities and to their historical associations with the site. Environment Canada is responsible for enforcing federal environmental regulations and to provide technical advice regarding the contaminated sites. Health issues were expected to remain sensitive during the project; as such, Health Canada assisted PWGSC on technical issues, historic studies, and communications and health issues during the environmental assessment.

109. The majority of the interviewees believed that Environment Canada and Health Canada roles were well understood by all parties.

110. The framework documents reviewed confirmed the role of the Province, represented by the Nova Scotia Department of Transportation and Infrastructure Renewal, as the lead for Project implementation and as having responsibility for establishing a single purpose entity (the STPA) to manage and implement the Project. At the provincial level, the Department of the Environment played a regulatory role.

111. The documents consistently affirmed the role of the STPA as being responsible for delivery of the Project, and for the day-to-day management and implementation of the clean-up. Specific responsibilities included: preparing the scope of work, budgets, authorities and expenditures; implementation of activities; planning and obtaining services, equipment and material; direction and control of technical and advisory services; public communications; and ensuring compliance with legislative, regulatory, licensing and permitting requirements. The STPA was also required to report to the Project Management Committee with respect to expenditures, progress and other aspects of Project implementation.

112. The framework documents described PWGSC as the lead federal representative, responsible for negotiating implementation agreements with the Province and for oversight and accountability with respect to federal interests during implementation. The majority of the interviewees thought that PWGSC roles and responsibilities were clearly understood by most partners and there was a strong governance structure for the project within PWGSC.

113. Interviewees confirmed that most stakeholders had a clear understanding of the roles and responsibilities of PWGSC, the provincial government and the STPA on the Project. While several interviewees indicated that the STPA did not fully understand or agree with the amount of oversight provided by PWGSC; this lack of complete understanding or agreement did not prevent PWGSC from effectively exercising its role.

Committees

114. As described in the sections below, the Project's internal committees, as well as the external Community Liaison Committee, had well-defined and comprehensive terms of reference, included appropriate representation of partners and other stakeholders, and functioned effectively. As a result, the Committees contributed to the successful delivery of the Project.

115. Based on the documents reviewed, the Project Management Committee role was to ensure that the "Project is implemented in accordance with the Memorandum of Agreement (…) and to provide leadership in certain non-technical areas including, without limitations, involvement with First Nations, environmental assessments, community consultations and regional economic opportunities". Specific responsibilities included reviewing and approving the scope of work and proposed budgets and cash flow projections prepared by the STPA; generally ensuring the works are conducted in a manner consistent with the Memorandum of Agreement; and consulting with the Independent Engineer on technical matters. This Committee was to include two federal and two provincial representatives, to be co-chaired by the provincial and federal governments and to meet regularly. The Project Management Committee met regularly as required by the Interim Cost-Share Agreement (ICSA), four times each year.

116. Minutes reviewed for the meetings that took place from April 2011 to June 2013, show that all the organizations that should participate were well represented. The chair alternated regularly between PWGSC and the Nova Scotia Department of Transportation and Infrastructure Renewal. The action items were clearly identified as was the responsible organization and status. The Project Management Committee was supported by the Project Management Committee Secretariat. The Secretariat dealt with routine management and reporting issues and oversaw general administration of the FCSA including: providing information on budgets, schedules and costs and ensuring the flow of information between the Project Management Committee and the various organizations ( STPA, partners, the Independent Engineer and the committees). It was composed of a representative from PWGSC and one from Nova Scotia Department of Transportation and Infrastructure Renewal; participants included the STPA Chief Executive Officer and the Independent Engineer.

117. The results of the interviews support the view that the Project Management Committee functioned effectively. Interviewees mentioned that the co-chairs were effective and this Committee dealt with issues as they arose. In addition, it was noted that it was successful because its executive function was kept distinct from the general management of the Project.

118. The Operations Advisory Committee mandate, necessary representation, frequency of meetings required and participation of members are outlined in a consistent fashion in this Committee's Terms of Reference and in other framework documents. Its role was to provide functional and operational advice to senior management. Responsibilities included providing a forum between government partner representatives, the implementing agency (STPA) and the Independent Engineer for discussion of various project items; supporting the implementing agency to ensure the Project was completed on schedule and within budget; and keeping senior federal and provincial government officials fully informed of the progress on various aspects of the Project.

119. The Operations Advisory Committee had 17 members and included representatives from the Project Management Committee Secretariat, a senior representative from PWGSC, the provincial government, the STPA and the Independent Engineer. It was chaired by the Project Management Committee Secretariat and was to meet monthly with additional meetings as needed. It reported to, and received advice from, PWGSC and Nova Scotia Department of Transportation and Infrastructure Renewal on operational issues.

120. Minutes were reviewed for meetings that took place from April 2011 to December 2012. As required, the Operations Advisory Committee met monthly during this period, with few exceptions. All required organizations were well represented. Action items were clearly identified as was the responsible organization and status.

121. In 2012, the Operations Advisory Committee transformed into the "Six Amigos" and, subsequently, into the Senior Management meetings. The "Six Amigos" and the Senior Management meetings both included representatives from the same partner organizations that participated in the Operations Advisory Committee outlined above, but the number of representatives from each organization was much smaller. The "Six Amigos" and the Senior Management meetings both included nine members. The purpose of the "Six Amigos" and the Senior Management meetings was to take care of the closing details of the Project. The Evaluation reviewed meeting minutes for the "Six Amigos", spanning May 2011 to October 2012, as well as minutes from Senior Management meetings dating from October 2012 to July 2013. Both groups, while smaller than the original Operations Advisory Committee, still had the required representation. They also met more frequently - about once a week as opposed to once a month for the Operations Advisory Committee. The minutes reviewed provided action items, responsible organization and status.

122. The results of the interviews suggest that, while the Operations Advisory Committee was effective initially, it became less effective over time. It was noted that the number of participants increased to the point where it became unwieldy. This led to this committee being replaced by the smaller "Six Amigos" and, subsequently, by senior management meetings, with fewer participants, as the project was winding down and they were dealing with closing details.

123. The Environmental Management Committee mandate, necessary representation, required frequency of meetings and participation of members are outlined in its Terms of Reference, which are well defined, comprehensive and consistent. The Environmental Management Committee was to provide guidance to the STPA during the development and implementation of the Project Environmental Management Plan; track the implementation of Joint Review Panel recommendations; identify and review implementation strategies and measures and to provide regular updates to the responsible authorities and Project Management Committee. The Terms of Reference listed eight federal and five provincial departmental members required, as well as two ex-officio members. It also states the chair was to be shared by PWGSC and the provincial government and that meetings were to be monthly during the implementation phase of the project.

124. Meeting minutes were reviewed for the period from March 2011 to March 2013 and indicated that the Environmental Management Committee was effective. It generally did meet monthly; however, there were nine occasions during this period on which the monthly meeting did not occur as there were no issues that required them to meet. In these instances, it was decided not to incur the costs of a meeting. The minutes show that both the federal and provincial organizations were always represented at meetings (five to ten federal, two to five provincial representatives) and that meetings were co-chaired as required. The meeting minutes were clear and the action items, responsible organization and status were well identified. Also, the minutes always started with reference to the previous action items and their status.

125. The interviews indicate that all five interviewees who commented on the effectiveness of this Committee felt that it worked well. However, two interviewees would have liked more information and/or feedback than what was provided from the Project.

126. The Community Liaison Committee was the project's external committee. Its roles and responsibilities, required representation and frequency of meetings, are outlined, in a consistent fashion, in this committee's Terms of Reference and in other framework documents. Based on these, the Community Liaison Committee mandate was to ensure the healthy, two way flow of information between the STPA and the constituent organizations represented on the Community Liaison Committee. It provided community views, concerns, ideas and questions on project plans and activities and kept constituent organizations abreast of project plans, progress and activities. The Terms of Reference clearly states that it was not a decision making forum. Membership consisted of approximately 15 members from organizations with "significant track records" in specific sectors such as business, health and organized labour. Representatives of the Government of Canada and Nova Scotia could attend all meetings but were not members. The Chair was appointed by the government funding partners. The Chair, in consultation with the members and the Chief Executive Officer of the STPA, determined the frequency of meetings.

127. Meeting minutes were reviewed for the period from April 2011 to April 2013. Based on the minutes there were usually at least nine members present at the meetings. The STPA and PWGSC were represented at all meetings. They provided project updates to the Community Liaison Committee and responded to questions from members. There were only a few minor action items stemming from these meetings. The organization responsible for taking action was identified and the action was followed up.

128. In addition to individual interviews with project stakeholders, the Evaluation conducted a group interview with members of the Community Liaison Committee to obtain their views on the role and functioning of this Committee. Those interviewed indicated that, for the most part, members and other stakeholders felt the Community Liaison Committee had appropriate membership and met frequently enough. They also indicated that the Community Liaison Committee effectively conveyed concerns/views of the community to Project officials and that Project officials listened and responded to concerns. Initially, some members thought it would have more of a regulatory/authoritative role, as opposed to an advisory/information exchange role; however, more recent members appeared to be clear as to the advisory/information exchange role of this Committee.

129. According to those interviewed, the Community Liaison Committee provided the community with a voice and effectively conveyed concerns/views of the community and project management responded to concerns expressed. It also had an effective Chair who clearly understood the Committee's role/mandate. The cooperative relations between the representative community organizations and both levels of government enabled the Project to move forward efficiently, by building trust and reducing opposition that could have caused delays and increased costs. However, two interviewees, out of eleven who responded to this issue, mentioned that the Committee could have played more of a challenge role.

130. While the Independent Engineer was not a part of the committee structure on the Project, it played a key role in governance of the Project; consequently, the Evaluation included the Independent Engineer in the review of project governance.

131. The appointment of the Independent Engineer was a joint responsibility of the provincial and federal governments. The role of the Independent Engineer was initially defined as one of ensuring the technical merits of the Project; ensuring appropriate management controls were in place and reporting on cost and contract compliance. Later documents expanded on these responsibilities to define the Independent Engineer role as one of monitoring; validating and reporting to both governments on the Project design and costs to complete; tendering and procurement; quality assurance/quality control programs; and progress of work. Interviews confirmed that the Independent Engineer provided management with objective support, input and financial oversight. The majority of respondents emphasized the importance to effective delivery of having the Independent Engineer on the Project. The Independent Engineer ensured evidence based decisions; helped in avoiding and resolving disputes; monitored contractors; provided solutions for engineering and design issues; and provided a challenge function with respect to contractors and billing. As a result, the Independent Engineer contributed to the successful completion of the Project, as well as to its efficiency and economy.

132. As part of its assessment of project governance, the Evaluation examined the costs of project governance, based on the actual versus originally budgeted costs of the key management and oversight elements, the Sydney Tar Ponds Agency, the Independent Engineer and PWGSC, in its oversight role.

133. The Evaluation found that overall project governance costs were in line with original estimates. Project governance costs amounted to $55.7 M, or 13.2% of the total Project budget. The original estimate for these costs was $56.4 M or 13.4% of the total Project budget.

134. As Exhibit 3 illustrates, PWGSC oversight costs are expected to be 19% ($4.2 M) less than originally budgeted. The costs of the STPA and the Independent Engineer meanwhile are expected to exceed their original budgets by 9% and 13%. These amounts, however, are within the overall contingency allowance of 14%.

Exhibit 3 Project governance costs

135. As noted above, the role of the Independent Engineer expanded over the course of the Project. While the cost of the Independent Engineer exceeded the original budget, specific design suggestions were made by the Independent Engineer that contributed to cost savings on the Project. One Independent Engineer recommendation, the replacement of planned water diversion channels using sheet pilings by a pump around approach, resulted in estimated savings of $38 M. In addition, the Independent Engineer developed the methodology to show that a planned Geosynthetic Clay Layer was not required for the Tar Ponds, enabling significant cost savings on this Project works. Other examples of value-added by the Independent Engineer included identifying problems with Change Orders that led to a revamping of the Change Order process; recommendations for improved ground water pumping at the Coke Ovens site; the instrumental role played in development of the Web Warehouse information management system; and identification of inappropriate oversight charges on construction that had ceased.

Conclusions: Governance

136. The Project operated under an effective governance structure that contributed to the success of the Project, as evidenced by its completion ahead of schedule and under budget, despite its size and complexity and the number of partners involved.

137. The overall cost of project governance is expected to be in line with the original budgeted amount. While costs for the Independent Engineer and the STPA are expected to be higher than original budgeted amounts, the costs appear to have been reasonable as they did not exceed the contingency allocation of 14%.

Efficiency and economy

138. Demonstration of efficiency and economy is defined as an assessment of resource utilization in relation to the production of outputs and outcomes. Efficiency refers to the extent to which resources are used such that a greater level of output is produced with the same level of input or, a lower level of input is used to produce the same level of output. Economy refers to minimizing the use of resources. There is high demonstrable economy and efficiency when resources maximize outputs at least cost and when there is a strong correlation between minimum resources and outcomes achieved.

139. To evaluate the efficiency of the Project, the Evaluation examined time spent on activities as a key input to the completion of the Project's intended outputs. The Evaluation also examined the extent to which changes to the Project's activities increased the efficiency of its design, as measured by cost savings.

140. The Evaluation compared actual or current estimated completion dates and scope for the Project, in total and in part (Project works), contained in the September 2013 Report of the Independent Engineer, with original estimates contained in the FCSA or, where appropriate, with estimates contained in the July 2010 Report of the Independent Engineer.

141. Based on this information, the Evaluation found that the remediation of the Tar Ponds and Coke Ovens sites and related construction work, overall, has been completed five (5) months ahead of its original schedule. The STPCORP was completed with no changes to the overall scope of the Project, which was to remediate the North and South Tar Ponds and the Coke Ovens sites. Furthermore, the property was officially transferred to the Government of Nova Scotia on March 14, 2014, prior to the original targeted completion date.

142. The Evaluation also found that there were delays in the completion of some individual Project works (in one case amounting to 25 months); however, others were finished ahead of schedule by as much as 15 months. The Evaluation also found a number of variances to Project works, which were major enough to have had negative impacts, but without affecting the overall project scope and schedule. These included:

• Greater than expected amount of contaminated material from the Coke Ovens Brook that had to be solidified and stabilized, which necessitated design changes and delayed completion of reconstruction of the Coke Ovens Brook by 25 months; it also increased costs by $2.7 M, compared to the estimate in the FCSA;
• The capping of the Coke Ovens site was split into two phases, resulting in a delay of 11 months in this project work.

143. The Evaluation also found the following changes were made in the implementation of the Project's activities which contributed to improved design efficiency, as measured by their cost savings to the Project:

• The diversion of surface water from the Tar Ponds using sheet pilings was replaced by a pump-around system that was integrated into solidification/stabilization of the Tar Ponds. This change resulted in a savings of $38 M from the original design.
• The construction of an on-site engineered landfill was cancelled and a contract for access to a local, municipal landfill was arranged instead. This change resulted in savings of $2.4 M compared to the estimated costs as of 2008.

144. Based on these findings, the STPCORP has been carried out in an efficient manner. The Project has been completed ahead of its original schedule and according to its original scope. As well, several improvements were made to the efficiency of the remediation design during course of the Project.

145. To evaluate the Project's economy, the Evaluation reviewed the planned versus actual costs for the Project as a whole, and for engineering design and construction management, based on Project financial data. As well, the Evaluation analyzed the cost of solidification/stabilization compared to solidification/stabilization benchmark costs and alternative remediation methods, based on project data and on comparative data obtained during the course of the Mid-Term Evaluation of the Sydney Tar Ponds and Coke Ovens Remediation Project.

146. Based on the above lines of evidence, the Evaluation found that the Project minimized its use of resources, in that total estimated Project expenditures are expected to be less than originally budgeted. However, engineering design and construction management costs are expected to be significantly (63%) higher than originally budgeted. The selected remediation technology of solidification/stabilization was found to be economical.

147. The original budget for the remediation of the Tar Ponds and Coke Ovens sites was $397.7 M. This budget, which was cost-shared between the federal and provincial governments, included a contingency of $57 M. The Project's budget was set in the 2008 federal-provincial final cost share agreement. Under this agreement, the first $297.7 M in costs was to be shared in a 60-40 ratio between the federal and provincial governments. The final $100 M was to be provided by the Federal Government.

148. Federal funding for the Project was provided to the three federal Partners: PWGSC, Environment Canada and Health Canada. PWGSC was provided with $282.2 M for the remediation activities cost-shared with the provincial government. This amount included a cost-share reserve of $4.5 M. PWGSC also received an additional $25.9 M for oversight costs, for total funding of $308.1 M. PWGSC's oversight funding included $3 M that was transferred to the Department of Justice for litigation of class action law suits, with respect to the operations of the Sydney Steel Company, that had been filed by local citizens against the federal and provincial governments (the plaintiffs were unable to obtain certification as a class action suit and consequently, the case did not proceed). The actual amount for oversight was, therefore, $22.9 M.

149. As of March 2014, the Project was estimated to have been completed at a cost of $389.3 M, $8.4 M below the cost-share budget of $397.7 M. PWGSC's share of this cost was $269.3 M, representing a savings to the department of $8.4 M. PWGSC also expects to realize savings in its budget for Project oversight of $25.9 M and in its cost-share reserve of $4.5 M, resulting in total savings of $17.8 M for the department. This savings is comprised of:

1. Remediation cost savings $8.4 M, all of which will accrue to the government;
2. Expected savings of $4.9 M in PWGSC' oversight budget, including approximately $.6 M in unspent litigation funding by the Department of Justice; and
3. Savings of $4.5 M as PWGSC's cost-share reserve will not be required.

150. While the Project, overall, was completed in an economical manner, the Evaluation found that engineering design and construction management costs are expected to be $47.5 M, $12.5 M above the original budget of $35 M. This represents an increase of 36% in the costs of engineering design and construction management. As the increased costs occurred in Project components that fell under the cost-share agreement, these additional costs have been cost-shared between the federal and provincial governments.

151. Based on information provided in interviews with Project officials and from the Sydney Tar Ponds and Coke Ovens Remediation Project: Lessons learned document, this significant increase in engineering design and construction management costs resulted primarily from the following factors:

1. The roles of design engineer and construction management were combined in one general contractor. This created conflict between the two roles, particularly with respect to possible design errors, designs that posed implementation risks or where alternative, less costly designs were available;
2. The lack of clarity in the engineering design and construction management contract with respect to what was to be billed under lump sum provisions and time and materials clauses, respectively; and
3. The lack of clarity in this contract as to what constituted acceptable deliverables, especially with regard to engineering design deliverables, which made liquidated damages portions of contracts virtually unenforceable. Liquidated damages—also referred to as liquidated and ascertained damages—are damages whose amount the parties designate during the drafting of a contract for the injured party to collect as compensation upon a specific breach ( e.g., late performance).

152. Consistent with what was found in the Mid-Term Evaluation of the STPCORP, the Evaluation found that solidification/stabilization was an economical remediation technology choice. The final cost of solidification/stabilization carried out under the Project was $108.67 per cubic meter, slightly more expensive than the estimate of $107.00 per cubic meter contained in the report on the Mid-Term Evaluation of the STPCORP in 2011. As noted in that report, this is significantly less than the US Environmental Protection Agency estimate for solidification/stabilization projects, of CAN$365.00 per cubic meter and well below the costs of incineration ( CAN$1,746.00 - CAN$2,672.00 per cubic meter) also cited in that report.

153. Based on the above findings, the STPCORP was carried out in an economical manner, overall. The Project is expected to be completed at a cost to the federal government of $17.8 M below what was originally budgeted, although engineering design and construction management costs are expected to be above original budgeted amounts. These increased costs were shared between the federal and provincial governments. The remediation effort was based on an economical remediation technology choice.

Conclusions: Efficiency and economy

154. The Project has been carried out in an efficient manner. The STPCORP has been completed on time and within the original scope of the Project. Several improvements were made to the efficiency of the remediation design during the Project. The Independent Engineer's design suggestions contributed to cost-savings realized by the Project.

155. The STPCORP was carried out overall in an economical manner. The Project is expected to be completed at a cost to the federal government of $17.8 M below what was originally budgeted, although engineering design and construction management costs are expected to be significantly above original budgeted amounts. The remediation effort was based on an economical remediation technology choice at the start of the Project.

Conclusions: Performance

Outcomes achievement

156. The Project achieved its immediate outcomes. The Project has successfully reduced or contained contaminant levels and largely eliminated contaminant migration from the sites. It has ensured the sites are ready to be used safely by the community and legal responsibility for the sites has been transferred to the Province.

157. The Project has achieved its intermediate outcomes. It has contributed to the protection of the local environment through improved water and air quality. In the case of air quality, the impact has been primarily in terms of reduced odours, from the site, as atmospheric contaminant levels in areas adjacent to the site were at or below acceptable levels prior to the start of the Project. The Project has also largely eliminated contaminants in ground water and surface water flowing through the site and, as a result, contributed to improved marine water quality in the harbour. Its precise contribution to improved marine water quality is not known because of the continued impacts of other sources of contaminants on marine water quality and the difficulties in measuring the impacts of the Project in isolation from these other sources. In transforming formerly contaminated sites into areas that can be used for recreation and light commercial or industrial purposes, it has contributed to the quality of life of the community. The sites are already being used for recreational purposes and are fully ready for safe use by the community.

158. The Project has provided socio-economic benefits to the community. It has also significantly benefited First Nations firms and individuals, and, to a lesser extent, women pursuing non-traditional careers and African Nova Scotians.

159. The Project has contributed to the Departmental Program Outcome of a strategically managed real property portfolio that maximizes economic benefits and minimizes liability, by ensuring the most appropriate future use of the sites and eliminating, for the most part, the liability of the federal government related to contamination at the site.

Governance

160. The Project operated under an effective governance structure that contributed to the success of the Project, as evidenced by its completion ahead of schedule and under budget, despite its size and complexity and the number of partners involved.

161. The overall cost of project governance is expected to be below the original budgeted amount.

Efficiency and economy

162. The Project has been carried out in an efficient manner. The STPCORP has been completed on time and within the original scope of the Project. Several improvements were made to the efficiency of the remediation design during the Project. The Independent Engineer's design suggestions contributed to cost-savings realized by the Project.

163. The STPCORP was carried out overall in an economical manner. The Project is expected to be completed at a cost to the federal government of $17.8 M below what was originally budgeted, although engineering design and construction management are expected to be significantly above original budgeted amounts. The remediation effort was based on an economical remediation technology choice at the start of the Project.

Lessons learned

164. Based on the analysis conducted in support of design and delivery and of efficiency and economy, and on views provided by Project stakeholders and on a review of the "Lessons learned" document prepared by the Project, the Evaluation has compiled a list of lessons learned regarding Project governance and management, in order to inform future remediation projects and to contribute to improved design and delivery of these projects.

165. The key lessons learned are the following:

1. Clearly defined roles and responsibilities of all Project partners and implementing agencies were critical to the success of the project. It is important to identify both the shared as well as the distinct responsibilities of the respective parties.
2. According to the views expressed by a wide range of project stakeholders, including PWGSC officials, officials of other government departments, some provincial officials, and the Independent Engineer, PWGSC's oversight role was essential to the project's success. PWGSC played a key role in working with project stakeholders to establish a common view to get the remediation completed with the time and money provided, and in line with the principles outlined in the Memorandum of Agreement.
3. It is important to define clear terms of reference for all committees and management structures. Both the Lessons learned document and the interviews confirmed the importance of clearly defined roles and responsibilities.
4. The separation between executive and management authority was a major contributor to project success. Interviewees pointed out that the Project Management Committee worked well because the executive function was kept distinct from the general management of the project and that the joint chairpersonship between the Province and federal governments created co-ownership and was important to the success of the Project.
5. An effective Community Liaison Committee, with a clearly defined advisory and information exchange role contributed greatly to ensuring community engagement and, ultimately, support for the Project. In addition, a strong Community Liaison Committee Chair contributed greatly to the Committee's effectiveness.
6. The Independent Engineer was critical to a Project of this size and complexity, with multiple partners. The Independent Engineer contributed to improved design, more effective oversight of contractors, improved information management, which resulted in significant cost savings to the Project.
7. Recruitment of individuals with the appropriate skills, knowledge and engagement was critical to success. Both the Lessons learned document and interviews, pointed out the importance of experienced, knowledgeable and engaged staff whether for the executing agency the Community Liaison Committee chair, the Independent Engineer or the Project Management Committee.
8. A large contingency fund is essential for a project of this size and complexity. Unlike construction of a building or bridge, for which well defined engineering standards and solutions exist, remediation projects are at high risk of encountering engineering, construction or other technical issues that can significantly add to the cost of the project.
9. The roles of design engineer and that of construction oversight, respectively, should be separated. Combining them creates the potential for conflict of interest and resistance on the part of the design engineer to design improvements.
10. Design engineer contracts should clearly define deliverables requirements and standards, and payment terms; as well, contracts should clearly delineate costs to be paid on a lump sum basis and those to be paid on a time and materials basis.

About the evaluation

Authority - About the evaluation

The Deputy Minister for Public Works and Government Services Canada approved the conduct of this evaluation, on recommendation by the Audit and Evaluation Committee, as part of the 2013–18 Risk-Based Audit and Evaluation Plan.

Evaluation objectives

The Evaluation examined the Sydney Tar Ponds and Coke Ovens Remediation Project, a joint federal-provincial project, for which Public Works and Government Services Canada, Atlantic Region was assigned the lead role for providing federal oversight. This evaluation's objective was to determine the performance of the Project: the achievement of its expected outcomes, design and delivery and a demonstration of the efficiency and economy of the Project. Relevance was not addressed because it was fully addressed in the Mid-Term Evaluation and STPCORP is coming to an end as of March 2014. The Evaluation assessed the Project for the period from April 2011 to September 2013.

Approach

The Evaluation was conducted in accordance with the Standard on Evaluation for the Government of Canada. The Evaluation took place between August 2013 and May 2014 and was conducted in three phases: planning, examination and reporting. To assess the evaluation issues and questions, the following lines of evidence were used.

Document and data review: The Mid-Term Evaluation Report was reviewed to provide baseline data for indicators used in the evaluation. Other documents reviewed included project planning and management documents; reports on audits, evaluations or other reviews of project activities; project progress reports; reports of the Independent Engineer; project financial documents; environmental and land-use reports; and other project-related documents. Data reviewed included project financial data; project activity data; public opinion research data; environmental and health-related monitoring data; project outreach data; data on solidification/stabilization cell test results; contracting data and set-asides data; as well as data on activities in support of local employment/economic development, including the Project's Economic Benefits Reports.

Stakeholder interviews: Interviews were undertaken with officials of PWGSC's Atlantic Region; officials of Environment Canada and Health Canada; officials of the Government of Nova Scotia; staff of the Sydney Tar Ponds Agency; representatives of the Community Liaison Committee; First Nations representatives; and other stakeholders. A total of 32 interviews were conducted, which included several interviews of multiple stakeholders, such as an interview with Community Liaison Committee members.

Site visit: A site visit was undertaken to verify the implementation of long-term land use plans; and readiness of the sites for use by the community.

Literature review: Academic literature pertinent to the outcomes of the project was reviewed, such as a review of literature pertaining to the environmental impact of the project to assess the achievement of environmental outcomes.

Financial analysis: Financial data related to the original estimates, updated estimates, and actual expenditures on specific project components were reviewed, in conjunction with information on factors contributing to differences between original estimates and actual costs incurred. In addition, the Evaluation updated the assessed cost-effectiveness of solidification/stabilization compared to alternatives by recalculating the cost per square cubic meter with finalized solidification/stabilization costs.

Limitations of the methodology

Document and Data Review: The assessment of environmental and socio-economic impacts was limited by the fact that a full assessment would require the acquisition and analysis of data subsequent to the completion of the Project. Without this data, it cannot be determined if there will be a lasting socio-economic impact from the Project. Without long term environmental monitoring data, the durability of the remediation indicated by the tests (compressive strength, leachability, etc.) cannot be fully confirmed. Assessment of socioeconomic benefits was also limited by a lack of data pertaining to employment outside of the project, including indirect/induced benefits and jobs realized due to skills/experience gained on the project.

Interviews: Interviews were purposively selected to include the senior project managers who were involved in the Project and other relevant stakeholders. While most stakeholders invited to interviews accepted the invitations, a few interviewees were not available during the Project's research phase ( e.g., not all Community Liaison Members were available for the group interview). Due to the limited number of project representatives for some of the elements of the remediation ( e.g., local economic benefits), some interview findings were based on a small number of respondents.

Reporting

Findings were documented in a Director's Draft Report, which was reviewed by the Office of Audit and Evaluation's Quality Assurance function. The Project's Director General was provided with the Director's Draft Report and a request to validate facts and comment on the report. A Chief Audit and Evaluation Executive's Draft Report was prepared and provided to the Regional Director General, Atlantic Region, for acceptance as the Office of Primary Interest. The Draft Final report will be presented to PWGSC's Audit and Evaluation Committee for the Deputy Minister's approval in June, 2014. The Final report will be submitted to the Treasury Board Secretariat and posted on the PWGSC website.

Project team

The Evaluation was conducted by employees of the Office of Audit and Evaluation, overseen by the Director of Evaluation and under the overall direction of the Chief Audit and Evaluation Executive. The Evaluation was reviewed by the Quality Assessment function of the Office of Audit and Evaluation.

Appendix A: Project works descriptions

TP = Tar Pond

CO = Coke Oven