Science and Decisions

The “Science and Decisions: Advancing Risk Assessment” National Academy Sciences (NAS) report was released in 2009. The report can be read or downloaded here for free: https://www.nap.edu/catalog/12209/science-and-decisions-advancing-risk-assessment. All images used in this post were obtained from the report.

Reason for committee formation and scope of report

The Environmental Protection Agency (EPA) requested the National Research Council (NRC) to suggest recommendations that, if implemented, could assist the agency in 1) developing risk assessments that are consistent with current and evolving scientific understanding, and 2) relevant to the risk management missions of the agency. Recommendations should include practical improvements that the EPA could make in the short- and long-term.

Thus, the NRC established the Committee on Improving Risk Analysis Approaches Used by the EPA. The committee largely comprised of people in academia but representatives from other organizations (state government, NGOs, company) with the necessary expertise were also selected.

EPA’s mandate

The major EPA laws that protect human health and the environment are Clean Water Act, Clean Air Act, Safe Drinking Water Act, Toxic Substances Control Act, Federal Insecticide, Fungicide and Rodenticide Act and the Comprehensive Environmental Response, Compensation, and Liability Act. I have briefly covered these laws here. These laws do not mention risk assessment but EPA uses these principles and incorporates safety margins to not underestimate health and ecological risks in the face of uncertainty and variability. Most of the laws require that EPA take into account economics, feasibility, and societal impacts when making regulatory decisions. The EPA also has to demonstrate that they have “substantial evidence” for proposed regulations and that they did not make “arbitrary or capricious” decisions.

Previous NAS committee recommendations on EPA’s risk assessment process

The risk assessment process in EPA is continually evolving as new scientific information and policies are incorporated into the process. However, since scientific knowledge is incomplete and there is no single way to assess data, it is possible to differently interpret the same data. A 1983 NAS committee recognized this and suggested ways to minimize manipulations of risk assessments to support a fixed policy outcome. It elucidated four steps in the risk assessment process and separated assessment of risks from consideration of risk management alternatives (Figure 1), so that the risk manager becomes mindful of the qualitative and quantitative elements of the risk analyses, and of the scientific uncertainties and assumptions (for e.g. extrapolation of high dose effects to low dose effects by assuming a shape for the dose-response curve) in it. The committee also recommended that EPA develop technical guidelines that specify the options that should be used to draw inferences when there is insufficient data. A 1993 NAS committee advised integrating the ecological risk assessment framework into the human health framework (Figure 2). A 1996 NAS committee advocated for full participation of all interested and affected parties in the problem formulation phase of the risk assessment.

While the EPA has made considerable efforts to implement NAS recommendations, the implementations have been uneven and incomplete.

Designing risk assessments

There are five common elements often encountered in decision-making processes:

1. Desire to use the best scientific methods and evidence
2. Presence of uncertainty that limits ability to make confident conclusions
3. Need for timely decisions
4. Some compromise among disparate adverse outcomes (health/ecological/economic)
5. Little to no short-term feedback on whether the decision has led to desired outcome

These elements need to be kept in mind when designing a risk assessment (Figure 2). A good design finds the best solution to fulfill multiple simultaneous and competing objectives while recognizing there are constraints. The NAS committee described some objectives that EPA could consider when designing a risk assessment (RA), including scope of assessment (should be as large as possible, for e.g., by accounting for all chemical exposure pathways), inclusiveness of input (should consider broader evidence base and use diverse methods to reach conclusions), inclusion of diverse stakeholders (will ensure fairness in RA design), transparency, compliance with laws, consideration of uncertainty and its impacts, and control of inadvertent risks that may arise due to a delay in decision-making. For e.g., gathering more data to reduce uncertainties could delay either registration of new safer products or regulation of existing harmful products. This could increase health/ecological risks.

EPA currently has in place two processes that relate to design of risk assessments. Planning and scoping phase is a discussion between decision-makers (risk managers) and stakeholders that assist the former is defining a risk-related problem (Box 3-1). Problem formulation phase is a discussion between decision-makers and risk assessors (and technically oriented stakeholders) to develop a detailed technical design for the assessment that reflects the broad conceptual design advanced in the scoping stage (Box 3-2).

Reducing uncertainty and variability

There is uncertainty in assessments when information is lacking, incomplete, or incorrect. Uncertainty depends on the quantity, quality, and relevance of data and on the reliability and relevance of models and inferences used to fill data gaps.

A Monte Carlo simulation is a technique that allows one to visualize most or all of the potential outcomes of a decision. It performs risk analysis by building models of possible results by substituting a range of values for any factor that has inherent uncertainty. The EPA sometimes uses this method to better understand the risks of any decision. The agency could also rely on expert judgement, which involves asking a set of carefully selected experts a series of questions that could help reduce uncertainties.

It is important for EPA to know which uncertainties to focus on as this would reduce burdens on the EPA staff and allow them to complete assessments in a timely manner. A sensitivity analyses can help determine the parameters whose uncertainties might most impact a decision. These uncertainties would require a more detailed analyses.

There are important variations among individuals in a population when it comes to susceptibility and exposure. Techniques that diminish uncertainties can also be used here but, unlike uncertainty, variability can only be better characterized and not reduced. Thus, different strategies are needed to address variability. EPA guidelines call for development of separate risk estimates for susceptible groups of individuals and for those in early life stages. However, these guidelines have not been properly implemented and have several flaws. The committee thus encouraged EPA to consistently account for variations in susceptibility and also proposed it estimate individual susceptibilities (not just group susceptibilities).

A unified approach to dose-response assessment

Currently, EPA assesses a chemical’s cancer and non-cancer risks differently (Figure 3). For cancer, it assumes there is no safe chemical dose unless shown otherwise (as usually not more than a 100 animals are treated and one excess tumor in a 100 people is considered unacceptable). For non-cancer, it assumes there is a safe dose below which no effect occurs.

The NAS committee disagreed with this approach and pointed out that individual variability, preexisting health conditions, presence of other chemicals, and background exposures could make a safe dose unsafe. Also, the current approach places more emphasis on cancer and downplays other health effects of chemicals. Thus, NAS advocated for a unified approach for all effects that accounts for individual susceptibility, health, age, gender, genetics, and other exposures. NAS also advised EPA to quantify a chemical’s magnitude of harm (or risk) at various exposure levels to capture population risks. This new framework is described in Figure 4.

NAS also outlined an approach that could be used to select the right model to assess a chemical’s dose-response relationship. This includes gathering information on a chemical’s adverse effects and mode of action, and identifying potential vulnerable groups/individuals and background exposures (Figure 5).

Default options

A default option is defined as the best option to choose when there are data gaps in the risk assessment process. EPA examines all available and relevant data and, when encountered with missing data that are pertinent to the risk assessment, makes certain assumptions to fill the gaps. However, these assumptions could underestimate risk. For example, the agency often assumes all humans of similar ages are equally susceptible, chemicals act independently of background exposures, and that chemicals with inadequate toxicity data have no risks.

Instead of using defaults, NAS suggested that the EPA could employ a range of plausible assumptions (this could however produce a broad range of risk estimates that are practically useless) or specific alternative assumptions to understand how assumptions influence risk. Also, it is preferable to use definitive defaults like fixed uncertainty factors to prevent different interpretation of the same data.

Cumulative risk assessment

A cumulative risk is the combined risks from aggregate exposure to multiple stressors. EPA undertakes a cumulative assessment of chemicals under some laws, but often does not consider non-chemical stressors, individual vulnerability (occurring from different susceptibilities, exposures, and abilities to recover), and background exposures. The NAS committee recommended that the EPA include these by incorporating biomonitoring and epidemiological data. They also supported development and use of simpler analytical tools to ensure cumulative effects are assessed more quickly and frequently.