The “Science and Decisions: Advancing Risk Assessment” National Academy Sciences (NAS) report was released in 2009. The report can be read or downloaded here for free: https://www.nap.edu/catalog/12209/science-and-decisions-advancing-risk-assessment. All images used in this post were obtained from the report.
As the report contains a lot of good information and is long (~ 300 pages), I have summarized it in two parts. This post is the first part.
Reason for committee formation and scope of report
The Environmental Protection Agency (EPA) requested the National Research Council (NRC) to suggest recommendations that, if implemented, could assist the agency in 1) developing risk assessments that are consistent with current and evolving scientific understanding, and 2) relevant to the risk management missions of the agency. Recommendations should include practical improvements that the EPA could make in the short- and long-term.
Thus, the NRC established the Committee on Improving Risk Analysis Approaches Used by the EPA. The committee largely comprised of people in academia but representatives from other organizations (state government, NGOs, company) with the necessary expertise were also selected.
The major EPA laws that protect human health and the environment are Clean Water Act, Clean Air Act, Safe Drinking Water Act, Toxic Substances Control Act, Federal Insecticide, Fungicide and Rodenticide Act and the Comprehensive Environmental Response, Compensation, and Liability Act. I have briefly covered these laws here. These laws do not mention risk assessment but EPA uses these principles and incorporates safety margins to not underestimate health and ecological risks in the face of uncertainty and variability. Most of the laws require that EPA take into account economics, feasibility, and societal impacts when making regulatory decisions. The EPA also has to demonstrate that they have “substantial evidence” for proposed regulations and that they did not make “arbitrary or capricious” decisions.
Previous NAS committee recommendations on EPA’s risk assessment process
The risk assessment process in EPA is continually evolving as new scientific information and policies are incorporated into the process. However, since scientific knowledge is incomplete and there is no single way to assess data, it is possible to differently interpret the same data. A 1983 NAS committee recognized this and suggested ways to minimize manipulations of risk assessments to support a fixed policy outcome. It elucidated four steps in the risk assessment process and separated assessment of risks from consideration of risk management alternatives (Figure 1), so that the risk manager becomes mindful of the qualitative and quantitative elements of the risk analyses, and of the scientific uncertainties and assumptions (for e.g. extrapolation of high dose effects to low dose effects by assuming a shape for the dose-response curve) in it. The committee also recommended that EPA develop technical guidelines that specify the options that should be used to draw inferences when there is insufficient data. A 1993 NAS committee advised integrating the ecological risk assessment framework into the human health framework (Figure 2). A 1996 NAS committee advocated for full participation of all interested and affected parties in the problem formulation phase of the risk assessment.
While the EPA has made considerable efforts to implement NAS recommendations, the implementations have been uneven and incomplete.
Designing risk assessments
There are five common elements often encountered in decision-making processes:
- Desire to use the best scientific methods and evidence
- Presence of uncertainty that limits ability to make confident conclusions
- Need for timely decisions
- Some compromise among disparate adverse outcomes (health/ecological/economic)
- Little to no short-term feedback on whether the decision has led to desired outcome
These elements need to be kept in mind when designing a risk assessment (Figure 2). A good design finds the best solution to fulfill multiple simultaneous and competing objectives while recognizing there are constraints. The NAS committee described some objectives that EPA could consider when designing a risk assessment (RA), including scope of assessment (should be as large as possible, for e.g., by accounting for all chemical exposure pathways), inclusiveness of input (should consider broader evidence base and use diverse methods to reach conclusions), inclusion of diverse stakeholders (will ensure fairness in RA design), transparency, compliance with laws, consideration of uncertainty and its impacts, and control of inadvertent risks that may arise due to a delay in decision-making. For e.g., gathering more data to reduce uncertainties could delay either registration of new safer products or regulation of existing harmful products. This could increase health/ecological risks.
EPA currently has in place two processes that relate to design of risk assessments. Planning and scoping phase is a discussion between decision-makers (risk managers) and stakeholders that assist the former is defining a risk-related problem (Box 3-1). Problem formulation phase is a discussion between decision-makers and risk assessors (and technically oriented stakeholders) to develop a detailed technical design for the assessment that reflects the broad conceptual design advanced in the scoping stage (Box 3-2).
Reducing uncertainty and variability
There is uncertainty in assessments when information is lacking, incomplete, or incorrect. Uncertainty depends on the quantity, quality, and relevance of data and on the reliability and relevance of models and inferences used to fill data gaps.
The EPA uses some default options when faced with uncertainty. For example, the agency has two sets of procedures to evaluate uncertainties in cancer and non-cancer risk assessments (Figure 3).
A Monte Carlo simulation is a technique that allows one to visualize most or all of the potential outcomes of a decision. It performs risk analysis by building models of possible results by substituting a range of values for any factor that has inherent uncertainty. The EPA sometimes uses this method to better understand the risks of any decision. The agency could also rely on expert judgement, which involves asking a set of carefully selected experts a series of questions that could help reduce uncertainties.
It is important for EPA to know which uncertainties to focus on as this would reduce burdens on the EPA staff and allow them to complete assessments in a timely manner. A sensitivity analyses can help determine the parameters whose uncertainties might most impact a decision. These uncertainties would require a more detailed analyses.
There are important variations among individuals in a population when it comes to susceptibility and exposure (Figure 4). Techniques that diminish uncertainties can also be used here but, unlike uncertainty, variability can only be better characterized and not reduced. Thus, different strategies are needed to address variability. EPA guidelines call for development of separate risk estimates for susceptible groups of individuals and for those in early life stages. However, these guidelines have not been properly implemented and have several flaws. The committee thus encouraged EPA to consistently account for variations in susceptibility and also proposed it estimate individual susceptibilities (not just group susceptibilities).