Applying the Framework for Science and Technology Advice is analogous to an S&T organization reviewing its operations against an identified standard, such as an ISO standard. In the same way that a laboratory will periodically conduct an internal audit of its scientific procedures against accepted technical standards, science and policy organizations within a department or agency would assess their science advice procedures against the principles and guidelines in the Framework.
The worksheet in this section is intended to help science and policy managers self-assess their practices. However, it is not the intention of this guide to restrict the use of the worksheet to self-assessments only; indeed, organizations may choose to have all or part of their science advisory procedures evaluated by a third party (i.e., review and audit groups or external assessors). This is left entirely to the discretion of each department or agency.
In the normal course of events, once a process has been assessed there is no
need for ongoing reassessment unless circumstances change appreciably.3
New
processes that are introduced should be reviewed for adherence to the Framework’s
principles and guidelines.
An initial question that assessors in each department will want to consider is "What is the scope of the assessment: to focus on key science advisory processes (e.g., regulatory assessment procedures) or key issues (e.g., climate change, water) or key initiative (e.g., ecosystem initiatives)?" Answering this question may be the hardest part of applying the Framework. Here is how the Framework describes the scope of science advice:
In general terms, you may envisage two kinds of science advisory processes – formal and informal. A formal process might include, for example, the operation of a standing advisory committee that deals with ongoing policy issues (e.g., disease surveillance, drug approval, species-at-risk designation). An informal process might include scientific and technical networks or procedures that departments use to respond to unforeseen or ad hoc policy issues.
It can be difficult to test informal processes and activities for adherence to the Framework’s principles and guidelines. If you find it hard to apply the Framework to an informal process, you may want to consider formalizing the process. For example, if your department provides feedback to scientists through an oral briefing you may wish to institute a more formal feedback process, such as a "record of decision" memorandum to staff.
When completing the worksheet, departments and agencies should keep in mind that the Framework is meant to apply both to science and to science advice; that is, to the department’s research and development and related scientific activities, as well as to the processes by which science is translated into policy. However, the worksheet does not explicitly assess the conduct and management of federal science and technology. The Council of Science and Technology Advisors’ Science and Technology Excellence in the Public Service (STEPS) report5 identifies the characteristics of excellence in federally conducted and managed science and technology and identifies techniques for assessing S&T excellence.
Even though the focus of the self-assessment worksheet is on mechanisms and how better to align them to the principles and guidelines in the Framework, it is important to note that the success of any mechanism is dependent on the individuals who are providing the expertise, giving the advice, developing the options and making the decisions. Therefore, a key objective of this guide is to raise awareness of the Framework requirements so that managers can ensure that their staff have the types of competencies, training and development, tools and support they will require to fulfill obligations under the Framework.
A major objective of the self-assessment approach is to empower individual science and policy managers to conduct their own assessment and thereby reduce the time and expense of the Framework review process. A general rule of thumb is that the level of effort devoted to the review should match the size, complexity and importance of the science advisory process and decision. For instance, an individual manager might assess the operations of a program advisory committee in one or two hours, using only his or her own time. Assessing a new regulatory process might require more time and possibly the use of an external advisor.6
The self-assessment worksheet includes examples of a variety of "good practices" that are consistent with the principles and guidelines in the Framework for S&T Advice and may be in use by science-based departments and agencies. For example, under the Framework principle of Early Issue Identification, there are a number of different practices, including:
Similarly, the other principles and their guidelines are accompanied by a variety of indicators of good practice. Appendix 5 contains a glossary of good practices.7
You will notice that some principles and guidelines have relatively few good practices listed. This may indicate that departments and agencies are still searching for ways to put the principles and guidelines into action.
The worksheet can be used to assess the degree to which a particular activity adheres to the principles and guidelines of the Framework. The degree to which departmental processes conform to the principles and guidelines can be:
When improvement is called for, assessors are encouraged to propose an improved approach to that process. At this point it might be helpful to refer to the good practices to determine what improvements may be needed.
The worksheet tries to strike a balance between the common aspects of applying the Framework and the individual characteristics of departments and agencies.
|
FRAMEWORK ELEMENT |
DEPARTMENTAL RESPONSE |
SELF-ASSESSMENT |
|
Principle
|
Guidelines
|
Suggested Assessment
Questions
|
Examples of
Good Practices8
|
Describe how you are addressing the assessment questions |
Rate Adherence9
Fully/Largely/
Somewhat/N.A. |
|
I-Early Issue Identification
The government needs to anticipate, as early as possible, those issues for which science advice will be required, in order to facilitate timely and informed decision making. |
I-1 Decision makers should cast a wide net – consulting internal, external and international sources – to assist in the early identification of issues requiring science advice. |
1.0 Are there mechanisms10 in place or is there access to mechanisms that will assist in early issue identification? Do these allow for:
- input from varied sources (internal, external and international)?
- individual employees (scientists, science advisors or policy analysts) to identify issues to management?
- members of the public or stakeholders to aid in early issue identification?
|
- Foresight studies (1)
- Issue scans
- Technology maps (2)
- Early warning systems
- Science "brainstorming" meetings
- Risk committees (3)
- Expert advisory committee
- Membership in "futures networks"
- Environmental monitoring
- Linkages with international scientific organizations
|
|
|
| |
I-2 Decision makers, policy advisors and scientists should communicate emerging issues requiring advice, and improve the connections between research and potential policy or regulatory issues. |
2.0 Are there effective relationships between scientists, advisors, policy analysts and decision makers to ensure identified issues are communicated in a timely manner?
2.1 Are there procedures to assess findings of early issue identification activities?
|
- Science forum (16)
- Science policy teams (4)
- Expert advisory committee
|
|
|
| |
I-3 Departments should support and encourage their science and policy staffs to establish linkages with each other and with external and international experts. |
3.0 Are there procedures through which science and policy personnel can work together to act on the major outcomes of early issue identification?
3.1 Do procedures and mechanisms allow the department and its managers to call on Canadian and international experts as required? |
- Science-policy linkages (5)
- Inventory of experts (7)
- Issue advisors
- Expert advisory committee
- Participation in international fora
|
|
|
| |
I-4 Departments should maximize interdisciplinary and international co-operation, and the use of expertise across government departments and levels of government, to identify, frame and address horizontal issues.
I-5 Departments should maximize the use of new and existing science and expert advisory bodies. |
4.0 Are there mechanisms for engaging other departments and disciplines to help identify emerging issues or to address solutions?
4.1 Are there mechanisms for sharing the results of early issue identification with other government departments with similar mandates? |
- Interdisciplinary teams
- Interdepartmental committees
- Expert advisory committee
|
|
|
|
II-Inclusiveness
Advice should be drawn from a variety of scientific sources and from experts in relevant disciplines, in order to capture the full diversity of scientific schools of thought and opinion. |
II-1 Departments should seek science input and advice from a wide range of sources, and decision makers should consider the multiple viewpoints received.
Departments should also consider engaging external, independent agencies to create advisory panels or to solicit advice on complex or controversial issues. |
5.0 Do research and assessment mechanisms canvass a wide range of multidisciplinary scientific opinion?
5.1 Do science and policy activities consider traditional knowledge?
5.2 Are there procedures or policies that allow for the creation of ad hoc advisory panels to address complex or controversial issues? |
- Inventory of experts (7)
- Citizen science (11)
- Aboriginal specialist group
- Expert committees
|
|
|
| |
II-2 … advice from external and international sources ... is sought when:
- the problem raises scientific questions that exceed the expertise of in-house staff;
- the issue is horizontal... ;
- there is significant scientific uncertainty;
- there is a range of scientific opinion;
- there are potentially significant implications for sensitive areas of public policy; or
- independent scientific analyses can strengthen public confidence.
|
6.0 Have procedures and policies been established to empower and encourage science and policy managers to seek external advice?
|
- Intergovernmental/
international MOUs
- Multidisciplinary teams
|
|
|
| |
II-3 Departments should ensure that the selection of advisors:
a. is matched to the nature of the issue and the breadth of judgement required;
b. is balanced to reflect the diversity of scientific opinions and to counter potential biases; and
c. includes some experts from other, not necessarily scientific, disciplines.
|
7.0 Have robust procedures been established for selecting advisors?
- Are formal qualification criteria considered in the appointment of advisors?
- What measures are in place to ensure selected advisors match the nature and breadth of judgement required?
- Are conflict of interest guidelines in place for advisors?
|
- Regional advisory networks
- Non-traditional advisory networks
- Conflict of Interest and Post-Employment Code for the Public Service (13)
|
|
|
| |
II-4 Departments should ensure that members of external advisory bodies are regularly rotated, with replacements chosen to preserve balance of representation. |
8.0 Are there guidelines or procedures for the rotation of advisory committee members? |
- Guidelines for Expert Panels (17)
|
|
|
| |
II-5 Decision makers should be open to both solicited and unsolicited advice. |
9.0 Are there mechanisms to receive, acknowledge and consider unsolicited advice in research and policy activities? |
- Web consultations and communications (9)
- S&T briefings for central agencies
|
|
|
|
III-Sound Science and Science Advice
The government should employ measures to ensure the quality, integrity and objectivity of the science and science advice it uses, and ensure that science advice is considered in decision making. |
III-1 Departments should:
- ensure that all science and science advice used for decision making is subject to due diligence...;
- ensure that in-house expertise exists to assess and communicate science ... to decision makers;
- ensure that a strong link exists between science advisors and ... policy advisors;
- promote professional practices for those involved in ... science, and provide and enforce conflict of interest guidelines, with these considerations:
i) science advisors should declare any conflicts of interest...;
ii) decision makers ... responsibility for protecting against actual or perceived conflicts of interests; and
- support and encourage government scientists to publish their research findings and conclusions in external, peer-reviewed publications.
|
10.0 Is there a due diligence procedure in place for major science policy activities?
- Are there adequate resources and expertise to assess science?
- Are there adequate resources and expertise to communicate science?
- Does the department apply science advice conflict of interest codes or guidelines to its science advisory procedures?
- Are researchers encouraged to publish their work in peer-reviewed publications?
- Are there formal and/or informal communication channels linking researchers, science advisors and policy advisors?
|
- Science networks (10)
- Peer review (21)
- Departmental science-policy discussion fora (22)
- Multidisciplinary assessment committees
- Science-policy staff exchanges
- Joint science-policy seminars and workshops
- Values and ethics workshops
- Promotion criteria for research scientists modified to include science policy contributions
- Professional development and training programs on the Framework for S&T Advice, science communication and the science-policy interface in government.
- Science assessments (8)
|
|
|
| |
III-2 Decision makers should:
- require that science advice be provided to them unfiltered by policy considerations;
- be conscious of possible biases among the science advisors and in the science advice received; and
- involve science advisors in the identification and assessment of policy options, to help maintain the integrity of the science advice.
|
11.0 Do existing mechanisms permit science advice to be provided unfiltered to decision makers?
11.1 Do briefing papers explicitly discuss the findings and conclusions of the department’s research and assessment activities?
11.2 Are science personnel involved in the development of policy options and outcomes?
11.3 Do policy analysts provide feedback to researchers? |
- Central agency briefings
- Ministerial expert panels
- Feedback mechanisms to scientists
- "Records of decision"
|
|
|
| |
III-3 Scientists and science advisors should:
- have the flexibility, within the issue being examined, to explore the range of conclusions and interpretations that the scientific findings might suggest;
- assist decision makers and science managers to set research priorities and design a research base that will support future science-based decision making; and
- recognize the existence of other considerations in decision making.
|
12.0 Is there an environment that fosters a culture of openness?
12.1 Does the department consult science advisors on major priorities?
12.2 Are efforts being made to inform scientists of the policy process? |
- Whistleblower protection
- Policy on Internal Disclosure of Information Concerning Wrongdoing in the Workplace (14)
- Clear accountability of functions and roles established at each stage of decision making
|
|
|
| |
III-4 Decision makers should take care to exclude personal and political views in formulating the questions to be addressed, and science advisors should clearly distinguish scientific fact and judgement from personal views in their advice. |
13.0 Are there codes of conduct for the provision of science advice? |
- S&T advice code of conduct
|
|
|
|
IV-Uncertainty and Risk
Science in public policy always contains uncertainty that must be assessed, communicated and managed. Government should develop a risk management framework that includes guidance on how and when precautionary approaches should be applied. |
IV-1 Departments should adhere to a government-wide set of risk management guidelines, once they have been developed, to maintain confidence that a consistent and effective approach is being used across government. |
14.0 Are research and policy personnel familiar with risk management guidelines, and do they apply them in their work? |
- Departmental and interdepartmental risk frameworks
- Implementation of the precautionary principle (19)
- Addressing TBS Integrated Risk Management Framework (18)
- Decision-making framework (6)
- Access to training on risk assessment, risk communication and risk management
|
|
|
| |
IV-2 Scientists and science advisors should ensure that scientific uncertainty is explicitly identified in scientific results and is communicated directly in plain language to decision makers. |
15.0 How is scientific uncertainty (or confidence) communicated to policy analysts and decision makers?
15.1 Do studies and policy options address confidence limits?
15.2 What mechanisms exist to communicate the nature and degree of uncertainty and risk? |
- Risk committees (3)
- Uncertainty parameters specified in science advice
|
|
|
| |
IV-3 Decision makers should ensure that scientific uncertainty is given appropriate weight in decisions. |
16.0 Do briefing materials contain a discussion of scientific uncertainty and risk management approaches?
16.1 Have decision makers been informed of risk assessment and management implications? |
- Risk assessment/risk management training for decision makers
|
|
|
| |
IV-4 Starting well before decisions are made, scientists, science advisors and decision makers should communicate to stakeholders and the public the degree and nature of scientific uncertainty and risks, as well as the risk management approach to be used in reaching decisions. |
17.0 What procedure and mechanisms exist to discuss with stakeholders and the public scientific risk and uncertainty?
17. 1 Do consultation documents explicitly discuss scientific uncertainty and risk? |
- Designated science spokesperson(s) appointed
|
|
|
|
V-Transparency and Openness
The government is expected to employ decision-making processes that are open, as well as transparent, to stakeholders and the public. |
V-1 Decision makers should balance the need for timeliness in reaching decisions with the need for effective consultation, while recognizing that transparency is always imperative.
V-2 Decision makers should provide early warning of significant policy and regulatory initiatives to key interest groups and other governments or international organizations, as appropriate.
V-3 Departments should make publicly accessible, on an ongoing basis, all scientific findings and analysis underlying decisions, and demonstrate how the science was taken into account in the decision making or policy formulation
|
18.0 Are there procedures and policies that address consultation and transparency?
18.1 Are there mechanisms that allow decision makers to provide early warning to interest groups, other governments or international organizations of significant policy and regulatory initiatives?
18.2 Are there procedures and mechanisms for making timely information available to the public about the scientific basis for decision making?
|
- Transparency guidelines
- Decision input and review mechanisms
- Consultation plan
- Open sharing of scientific data
- Timely publication of results
- Media engagement strategies
- Media training
- Proactive public involvement strategies
- Published records of decision and policy advice
|
|
|
| |
V-4 Departments should consider using a variety of means (including Web sites, press releases, newsletters, direct communication with stakeholders, public meetings, etc.) to present policy. Science advisors should be given a leading role in explaining their advice, while policy officials should describe how the science advice was secured and how the policies or regulations have been framed in light of the advice.
V-5 Inevitably, circumstances arise where scientific conclusions conflict with existing policies, or where government scientists believe their findings or advice are being muzzled. In these cases, departments should employ a well-defined and transparent procedure involving review by departmental management and then, if necessary, examination by a third party. The process should emphasize early conflict resolution and ensure departments do not restrict release of scientific findings that meet the guidelines for sound science. |
19.0 Is the department maximizing the use of the communication tools available to it to explain science policy decisions? What kind of communication tools are used?
19.1 Do mechanisms exist for considering and responding to dissenting opinions, minority viewpoints and traditional knowledge? |
- Scientific spokesperson(s) appointed
- Use of communication channels to explain S&T advice
- Established internal and external science conflict resolution procedures
- Whistleblower protection for scientists
- Minority input consideration (15)
- Science liaison officers
- Publication of dissenting views in addition to consensus opinions arising out of the scientific peer review processes
|
|
|
|
VI-Review
Subsequent review of science-based decisions is required to determine whether recent advances in scientific knowledge have an impact on the science advice used to reach the decision. |
VI-1 Departments should establish a follow-up procedure that documents the government’s actions in response to science advice and recommendations. Departmental responses should become part of the official record and provide a useful input to subsequent reviews.
V1-2. Departments should review key decisions to determine whether recent advances in scientific knowledge affect the science and science advice used to inform the decision. The time period for review should depend on the state of the science (for example, the level of uncertainty, the rate of change in the scientific knowledge, etc.) and should be identified at the time the decision is made (for example, establish a "best before" date for the science advice).
|
20.0 Does the department have effective follow-up procedures? Are mechanisms in place for follow-up?
20.1 To what degree does the department undertake post-mortems of important science policy decisions?
20.2 Are past science policy decisions reviewed on a regular basis in light of new scientific information? Are there effective procedures and mechanisms to periodically review past science decisions in light of new scientific information? |
- "Lessons learned" reviews
- Post-decision monitoring programs
- State-of-the-art science reviews
- Performance measurement framework (12)
|
|
|
| |
VI-3 When asked to review past decisions, and the science and science advice that supported them, science advisors should have access to all relevant information, including previous analyses and official responses. |
21.0 Does the department have procedures for retaining corporate S&T knowledge? |
- Knowledge management strategies, programs
- External peer review (20)
|
|
|
