Procurement of nursing services for remote and isolated First Nations communities for Health Canada

Table of contents

• 1. Background and introduction
• 2. Project requirement
• 3. Attestation of assurance
• 4. Objectives of the fairness monitor assignment and methodology
• 5. Fairness monitor specific activities and findings
• 6. Reference documents
• Appendix A: Minor deficiencies fairness monitor contractor's final report
• Appendix B: Resolution of important potential fairness deficiencies fairness monitor contractor's final report

1. Background and introduction

Knowles Consultancy Services Inc. and Hill International Inc. in Joint Venture was engaged as the Fairness monitor (FM) to observe the competitive procurement process for the Procurement of Nursing Services for Remote and Isolated First Nations Communities for Health Canada, undertaken by Public Works and Government Services Canada (PWGSC) through Solicitation Number HT360-123541/C. Knowles Consultancy Services Inc. and Hill International Inc. in Joint Venture is an independent third party with respect to this activity.

We hereby submit the Final Report covering our activities commencing with the review of two Requests for Information (RFIs) through the review of a Request for Proposal (RFP), the evaluation of proposals and the selection of the recommended bidders.

This report includes our attestation of assurance, a summary of the scope and objectives of our assignment, the methodologies applied, and specific activities and relevant findings.

2. Project requirement

The Department of Health Canada (HC) is seeking the services of three Contractors for the provision and management of a workforce of nurses to supplement internal HC resources on an as and when requested basis, at different remote, semi-isolated and isolated locations of First Nations in the provinces of Manitoba, Ontario and Quebec. The contracted services will include primary health care and hospital nursing service requirements.

The services provided by the Contractors will encompass the care of patients and the provision of assistance to medical doctors in the treatment of illness; the conduct of programs designed to promote health and prevent diseases; and the provision of advice. The services may also include direct patient care to individuals, families and groups in the home and community and consultation, dependent upon the demands of the individual task authorization. Each contract will also include an option to acquire nursing services for additional regions.

3. Attestation of assurance

The FM hereby provides the following assurance statements concerning the procurement process to obtain the services of three Contractors for the provision and management of a workforce of nurses to supplement internal HC resources on an as and when requested basis, at different remote, semi-isolated and isolated locations of First Nations in the provinces of Manitoba, Ontario and Quebec.

It is our professional opinion that, notwithstanding the minor deficiencies identified in Appendix A to this report, the competitive procurement process we observed was carried out in a fair, open and transparent manner.

Notes:

The consensus evaluations by the Evaluation Team were carried out in accordance with best practice that successfully mitigated the effects of the minor deficiencies identified in Appendix A. It is our opinion that if the consensus evaluations had been carried out through a teleconference or videoconference as was originally planned, it is unlikely that the minor deficiencies would have been mitigated, and as a result, we would have been unable to provide the opinion that the competitive process was carried out in a fair, open and transparent manner.

For all references in this report concerning fairness related comments being provided to project officials, it is confirmed that, as necessary, project officials provided clarification to the Fairness monitor or took appropriate action to address the comments, and as a result these potential fairness deficiencies were not identified in this report. However several of these potential fairness deficiencies that we identified are of such importance that we have provided comments on them in Appendix B to this report.

___________________
Bruce Maynard P. Eng.
FM Specialist

4. Objectives of the fairness monitor assignment and methodology

The overall objective was as follows: provide Public Works and Government Services Canada (PWGSC) with independent observation of project procurement activities; provide fairness related comments to project officials as early as possible so that appropriate action could be taken to resolve the concerns before fairness was impacted; bring any potential fairness concerns to the attention of the Operational Integrity Sector if there is no timely resolution with project officials; and attest to the fairness of the procurement process, including its execution.

To accomplish the objective we undertook the following activities:

• became familiar with the project governance structure;
• reviewed the Request for Information (RFI);
• reviewed the Request for Proposal (RFP);
• reviewed all amendments and addenda to the RFP including questions submitted by bidders and answers provided;
• reviewed the procedures to be used for the evaluation of responses and the guidance provided to the evaluation team;
• observed the evaluation of responses to the RFP to ensure that the specified evaluation and contractor selection procedures and departmental policy were followed and consistently applied during the evaluation and selection process; and
• observed the debriefing of unsuccessful bidders. (This activity will be reported on in an to this report after any debriefings.)

5. Fairness monitor specific activities and findings

Fairness monitor activities and specific findings concerning the request for information phase

Two RFIs were issued through the Government Electronic Tendering System (GETS).

During the period February 21, 2013 to March 18, 2013, we reviewed draft versions of RFI 1 and provided fairness related comments. Appropriate action was taken by project officials.

RFI 1 was published on MERX on March 19, 2013 (Document 1). Amendments 1 to 6 to RFI 1 (Documents 2 to 7) were published on MERX during the period March 22, 2013 to April 3, 2013 including draft versions of the RFP. We reviewed these documents and provided fairness related comments. Appropriate action was taken by project officials. RFI 1 closed on April 10, 2013.

During the period April 22, 2013 to May 13, 2013, we observed industry consultation One-on-One meetings with suppliers, reviewed draft versions of the RFP, and provided fairness related comments. Appropriate action was taken by project officials.

During the period July 10, 2013 to July 12, 2013, we reviewed draft versions of RFI 2 and the RFP and provided fairness related comments. Appropriate action was taken by project officials.

RFI 2 was published on Buy and Sell on July 12, 2013 (Document 8). Amendments 1 to 3 to RFI 2 (Documents 9 to 11) were published on Buy and Sell during the period July 19, 2013 to July 26, 2013, including draft versions of the RFP. We reviewed these documents and provided fairness related comments. Appropriate action was taken by project officials. RFI 2 closed on July 29, 2013.

Although additional One-on-One industry consultations were offered, only one supplier indicated interest but did not make its representatives available in time to take advantage of the offer. The supplier was advised that it was not possible to delay the issuance of the RFP on September 25, 2013. We provided fairness related comments on this matter and project officials took appropriate action.

During the period September 25, 2013 to October 9, 2013, we reviewed draft versions of the RFP and provided fairness related comments. Appropriate action was taken by project officials.

The RFP was posted on Buy and Sell on October 10, 2013.

Fainess monitor activities and findings during the request for proposal phase

During the period October 11, 2013 to October 26, 2013, we reviewed the RFP as published on Buy and Sell (Document 12), Questions and Answers (Qs and As) concerning the RFP, and Amendments 1 to 6 the RFP (Documents 13 to 18), and provided fairness related comments to the contracting authority. All comments were addressed appropriately by project officials and as necessary were dealt with through the Qs and As process and amendments to the RFP.

The RFP closed on October 28, 2013.

Fairness monitor activities and findings during the evaluation phase

During the period January 22, 2014 to February 2, 2014 we reviewed and provided fairness related comments on the Evaluation Guidelines document. Appropriate action was taken by project officials. During the period February 3, 2014 to February 5, 2014, we observed consensus evaluation meetings on the technical requirements and provided fairness related comments. Appropriate action was taken by project officials.

On February 10, 2014 we reviewed the results of the evaluation of proposals and the selection of the recommended bidders as specified in the RFP. We were advised that the calculations and results had been checked through two independent peer reviews. No fairness deficiencies were identified.

6. Reference documents

The following documents are referenced by number in the attached report. Unless otherwise indicated, these documents are available through the Nursing Services for Remote and Isolated First Nations Communities Project Office.

Table summary

The following table includes a list of documents referred to in the report such as the Request for Proposal (RFP), amendments, clarification letters, debriefing letters, etc. Each row is numbered, followed by the document title, then a document identifier such as a date or document number.

No.	Document	Additional information
1	RFI 1	Released on MERX March 19, 2013
2	Amendment 1 to RFI 1	Released on MERX March 22, 2013
3	Amendment 2 to RFI 1	Released on MERX March 22, 2013
4	Amendment 3 to RFI 1	Released on MERX March 25, 2013
5	Amendment 4 to RFI 1	Released on MERX March 26, 2013
6	Amendment 5 to RFI 1	Released on MERX on April 2, 2013
7	Amendment 6 to RFI 1	Released on MERX on April 3, 2013
8	RFI 2	Released on Buy and Sell July 12, 2013
9	Amendment 1 to RFI 2	Released on Buy and Sell July 19, 2013
10	Amendment 2 to RFI 2	Released on Buy and Sell July 24, 2013
11	Amendment 3 to RFI 2	Released on Buy and Sell July 26, 2013
12	RFP	Released on Buy and Sell October 10, 2013
13	Amendment 1 to RFP	Released on Buy and Sell October 10, 2013
14	Amendment 2 to RFP	Released on Buy and Sell October 15, 2013
15	Amendment 3 to RFP	Released on Buy and Sell October 21, 2013
16	Amendment 4 to RFP	Released on Buy and Sell October 23, 2013
17	Amendment 5 to RFP	Released on Buy and Sell October 25, 2013
18	Amendment 6 to RFP	Released on Buy and Sell October 25, 2013

Appendix A: Minor deficiencies fairness monitor contractor's final report

We observed the following minor deficiencies in the procurement process:

• During what were supposed to be individual evaluations by each evaluator, project officials advised us that discussions took place among evaluators concerning how to evaluate certain criteria. Such discussions could have the effect of pre-empting, or limiting, the formal consensus deliberations, thereby limiting the transparency of the process. By definition individual evaluation requires that evaluators not discuss how to evaluate specific criteria.
• Discussions were held between Subject Matter Experts (SMEs) and evaluators without the contracting authority or Fairness monitor present. Based on our discussions with project officials, it appears that the SME input went well beyond answering specific questions and could have resulted in the SMEs in effect advising evaluators as to how to evaluate certain evaluation criteria. Again, by definition individual evaluation requires that evaluators not be provided with advice on how to evaluate responses to evaluation criteria, and that the contracting authority and Fairness monitor observe the input of SMEs to ensure compliance with this requirement.
• A document entitled “Bid Evaluation Procedures” was not provided to the Fairness monitor prior to the commencement of the bid evaluation process which precluded the opportunity to provide fairness related comments. We reviewed the document after the completion of the individual evaluations and identified fairness related issues that should have been addressed before it was provided to the evaluators

Appendix B: Resolution of important potential fairness deficiencies fairness monitor contractor's final report

We wish to note that, notwithstanding the fact that the following potential fairness deficiencies were resolved before they impacted fairness, it is our opinion that they are of such importance that they should be noted in our report.

• Fairness requires that evaluators have sufficient time for full consensus deliberations. Proposed plans for consensus evaluations identified estimated deliberation times for each criterion as 3 to 4 minutes per bidder response with an estimated total time required for evaluation of 4 hours. Besides being far too short as estimates of time for most criteria, the short time period planned would have sent the signal to evaluators that full and careful deliberations were not the priority, and that the earlier discussions among evaluators should be taken into account, thereby limiting the need for full deliberations at the formal consensus evaluation stage. After we identified the concern that we had on this matter, the estimated deliberation times were deleted from the instructions to evaluators. (It is noted that the consensus evaluation of bids required more than 19 hours of discussion, over a period of 3 days, to complete.)
• Officials initially planned to conduct the consensus evaluation of bids by teleconference and after we expressed our concern, officials then proposed to conduct it by videoconference. We stated that in our opinion the conduct of a consensus evaluation of bids by teleconference or videoconference may not demonstrate a fair consensus evaluation. In our opinion a fair consensus evaluation requires that the evaluators meet in person in order to arrive at a consensus score for each bidder on each evaluation criterion. There must be a high degree of personal interaction among evaluators, including eye contact, so that a clear understanding of each evaluator's position is obtained and reflected in the strength/emotions expressed in an evaluator's support of his/her position. We noted that in our opinion the deliberations that take place during a consensus evaluation are similar to the deliberations of a jury in a fair justice system. A fair justice system requires the jury to carry out its deliberations in person. As a result of our expressed concerns, project officials decided to conduct the consensus evaluations on a face to face basis.

to the final report
January 22, 2015

to fairness monitor final report dated February 16, 2014, concerning the procurement of nursing services for remote and isolated First Nations communities for Health Canada

This to the Fairness monitor Final Report covers the period following the conclusion of the evaluation phase and includes contract award and debriefings to unsuccessful bidders.

The contract was awarded on June 18, 2014. On the same day, letters, which we had previously reviewed, were forwarded to the unsuccessful bidders informing them of the results of the procurement process. Each letter provided the evaluation results of the respective unsuccessful bidder along with the total scores of each of the three successful bidders. The letters also offered a detailed debriefing. On July 10, July 22 and August 6, 2014 we observed detailed verbal debriefings by teleconference that were provided to unsuccessful bidders on the evaluation process and their individual results.

Fairness monitor attestation of assurance

It is the opinion of the Fairness monitor that the post evaluation activities, including the debriefings, were carried out in a fair manner. In this context, fairness is defined as decisions made objectively, free from personal favouritism and political influence, and encompasses the elements of openness, competitiveness, transparency and compliance.

___________________
Bruce Maynard P. Eng.
FM Specialist