Reducing Uncertainty and Risk

 

Peer Review of the Potential Impact of Dredging and Disposal on

At-Risk Salmonids in the Lower Columbia River Estuary,

March - August 2001

 

Sustainable Ecosystems Institute

0605 Taylor’s Ferry Rd.

Portland, OR  97219

503.246.5008

www.sei.org

 

 

A journey of a thousand miles must begin with a single step.

¾Lao Tzu

 

 

Background

In the late 1980’s, the Port of Portland joined with other lower Columbia River ports to request that the US Army Corps of Engineers (Corps) consider deepening the river’s navigation channel. The request was based on the ports’ desire to remain competitive in an industry where larger ships with deeper drafts were becoming the norm. The Port of Portland, serving as the overall coordinator for the sponsoring ports, and the Corps co-sponsored a feasibility study, which was completed in August 1999. The study recommended deepening the channel an additional three feet along 103.5 miles of the Columbia River below Vancouver, Washington.

 

Earlier in the year, Congress had authorized the project. Congressional appropriations, however, could not be released without a green light ¾ or a “no-jeopardy Biological Opinion” ¾ from the National Marine Fisheries Service (NMFS) and the US Fish and Wildlife Service (FWS). Congress set a deadline of December 31, 1999, for the two regulatory agencies to weigh in on their opinions. With just weeks to the deadline, NMFS and FWS each issued the Corps separate no-jeopardy Biological Opinions (BiOps) and each attached various conditions for the Corps to meet over time, following completion of the channel deepening project. Essentially, the two BiOps agreed with the Corps’ and the Port of Portland’s Biological Assessment that the proposed channel deepening project would have no significant impact on the region’s 22 federally listed fish and wildlife species.

Less than two months later, in February 2000, a coalition of environmental organizations filed a lawsuit to block the proposed channel deepening project. The lawsuit asked a federal court to set aside the no-jeopardy BiOps and to require a new opinion. The lawsuit alleged the BiOps lacked scientific foundation and implied that the regulatory agencies’ approval had more to do with meeting the reporting deadline (December 31, 1999) for congressional appropriations than with meeting agency BiOp responsibilities.

 

That summer, in a surprising reversal, NMFS withdrew its BiOp and requested a “reconsultation” with the Corps. The two agencies, however, were unable to reach agreement, and by the end of the year, negotiations had collapsed. To resolve the impasse, the principal parties (Army Corps, Port of Portland, NMFS, and USFWS) agreed in January 2001 to retain the independent scientific peer review services of the Sustainable Ecosystems Institute.

 

About the Sustainable Ecosystems Institute

In 1992, Dr. Deborah Brosnan founded the Sustainable Ecosystems Institute (SEI) to give academic science a stronger voice in natural resource management, and to encourage use of science-based, cooperative approaches to environmental problem solving. The organization is a public-benefit, non-profit association of scientists specializing in peer review, a time-honored process of independent review or input from qualified individuals having no personal stake in the outcome of a decision that concerns only the scientific validity of the evidence and analyses underlying a proposed action or decision.

 

SEI is committed to the use of impartial science of the highest standard. The reputation of the group hinges upon maintaining neutrality and on building consensus. An environmental consensus process is one in which all those who have a stake in the outcome aim to reach agreement on outcomes that resolve or advance issues related to environmental sustainability. Although participants may not agree with all aspects of the agreement, consensus is reached if all participants are willing to live with the total package. Consensus processes are typically guided by ten principles outlined below.

 

 

Ten Principles for Building Consensus†

 

1.             Purpose-driven: The parties have a common concern and a commitment to the process.

2.             Inclusive: All stakeholders having a significant interest in the outcome are involved.

3.             Voluntary participation: The parties who are affected or interested participate voluntarily.

4.             Self-design: The parties design the consensus process.

5.             Flexibility: The parties are open to changing the process as circumstances dictate.

6.             Equal opportunity: The proceedings are transparent and information is widely accessible.

7.             Respect for diversity: The parties have a mutual respect for diverse values and interests.

8.             Accountability: Parties are accountable to their constituencies and to the process.

9.             Time limits: The process is guided by a realistic set of deadlines and milestones.

10.         Implementation: Parties are committed to implementation and effective monitoring.

 

† Cormick, Gerald, Norman Dale, Paul Edmond, S. Glenn Sigurdson, and Barry D. Stuart. 1996. Building consensus for a sustainable future: Putting principles into practice. National Round Table on the Environment and the Economy: Ottawa, Canada.

 

 

Fundamental Elements of A Successful Peer Review Process

There are essentially six fundamental elements to a successful peer review process: 1) assessing the problem,  2) gathering an expert team  3) designing the process,  34) facilitating the process, 5) seeking commitments to implement agreements, and 6) evaluating success. Each is described below in general terms, as well as in their relationship to the Columbia River Channel Deepening project.

 

1.  Assessing the Problem. SEI’s first task was to conduct a conflict assessment ¾ the mediator’s method of determining the nature of the problem and the commitment the principal parties have in finding common ground. The nature of the problem is always more than what meets the eye. In the case of the Columbia River Channel Deepening project, the problem on the surface was whether the proposed action (channel deepening) posed a significant threat to listed salmonids. But below the surface, the problems had to do with an erosion of trust among the principal parties, stemming from the withdrawal of the Biological Opinion and the subsequent failure of the parties to reach resolution on a number of issues.

 

Dr. Steven Courtney, SEI Vice President for Programs, served as the mediator. Dr. Courtney assessed that much of the problem stemmed primarily from different interpretations of the facts and that the solution would therefore lie in getting a highly competent, impartial panel of scientists to review the facts and thereby help resolve disputes. The process he proposed ¾?a series of science panel meetings in which the parties laid out for review and discussion the facts as they saw them ¾?would elevate the substantive issues (scientific facts) above the emotional issues (distrust). With some level of commitment obtained from each party, Dr. Courtney was then able to work with the principal parties to identify the critical scientific and technical issues. These issues were primarily those raised by NMFS and FWS in their withdrawal of the Biological Opinion.


2.  Gathering an Expert Team. A skilled mediator is central to a successful peer review process. The mediator is the architect of the peer review process, assessing the problem, working with parties to design the process, facilitating the process through planning and action, and seeking commitments from the parties to implement agreements. The mediator is an independent person, acceptable to all the participants, whose focus and expertise is in managing and shepherding the process and in assisting disputing parties in finding common ground. In effect, the mediator is both a ‘process manager’ and a ‘dispute manager’. A skilled mediator:

Ÿ         Leads by example, modeling the types of behavior expected from others.

Ÿ         Employs active listening skills.

Ÿ         Shares his or her personal thoughts and feelings as a barometer for groups.

Ÿ         Works adroitly behind the scenes, resolving disputes and addressing complex technical issues like scientific uncertainty.

 

Throughout the peer review process, Dr. Courtney filled the critical role of interpreter/translator. Uniquely he combined scientific expertise with an understanding of natural resources policies and constraints. As such, he communicated between the scientific and non-scientific constituents. This ability to move seamlessly between two very different realms is fundamental to a successful scientific peer review process. Not only was Dr. Courtney able to translate complex, technical concepts into everyday language, but he was equally able to reframe general questions from stakeholders in a manner that spoke directly to the science panel’s specific charge. As a result, Dr. Courtney commanded the respect of the science panel, project managers, stakeholders, and the media.

 

Due to the highly technical and complex nature of this particular peer review, it was essential for Dr. Courtney to be supported by a carefully selected team of experts. Operating under a modest budget, Dr. Courtney decided the interests of his clients could best be served by two support staff ¾one to serve as scientific liaison to the science panel and the technical presenters, and the other to manage outreach to stakeholders, media relations, and meeting logistics. Botanist Dr. Tatiana Boyle with experience in sustainable conservation filled the role of science manager; Wendy Hudson with several years experience in program management, filled the role of project manager. All three SEI staff members worked closely with the four agency project managers, and Dr. Courtney also periodically conferred with some of the project managers’ supervisors or made presentations on the progress of the SEI process to joint supervisor meetings.

 

3. Designing the Process. There are essentially three fundamental elements to consider in process design:  1) facilitating group ownership of the process design,  2) clarifying roles and responsibilities, and 3) establishing ground rules. Each is described more fully below.

 

Facilitating group ownership. The next step was to work in concert with the principal parties to ‘design’ the process. Parties tend to be more invested in a process and to respect rules more when they have had a meaningful hand in shaping them. Moreover, when problems arise later in the process, as they inevitably do, the parties have an agreed-upon framework for resolving differences. While this particular peer review model rested on the use of an independent, scientific review panel, SEI nevertheless was able to give the principal parties a sense of ownership in the process design.

 

Accordingly, the parties recommended that the science panel be composed of seven respected scientists with a range of expertise relevant to the project. To maintain its standards of impartiality, SEI reserved the right to select panel members. The principal parties were allowed to object to any SEI nominations, but not to control the selection process. Only one objection was raised for which SEI found an acceptable replacement. In concert with Dr. Courtney, the project managers decided on six science panel meetings over the course of six months, which SEI would organize. The six meetings would be open to the public and would focus on major areas of concern raised in the withdrawal of the Biological Opinion (e.g., sediments and sediment quality, toxic contamination in fish). SEI would keep an audio and visual record of the meetings, and would post relevant information (e.g., presentations, meeting summaries, etc.) on the organization’s web site.

 

Clarifying roles and responsibilities. Roles and responsibilities were clarified among project managers and between project managers and SEI. Among project managers, the roles and responsibilities entailed decisions around standards of conduct and behavior, media relations, confidentiality, and information sharing. For example, the principal parties agreed to an open sharing of information and to refer all media inquiries to SEI. Roles and responsibilities also entailed respecting each other’s values, knowledge, and institutional constraints. For example, the constraints of the action agencies (Army Corps and the ports) are quite different from those of the regulatory agencies (NMFS and FWS).

 

Between the project managers and SEI, roles and responsibilities ran the gamut from identifying necessary products and services to payment for services. For Dr. Courtney, roles and responsibilities included impartiality, focus on technical issues as opposed to policy issues, and strategic planning. For his staff, they included technical assistance, public outreach, meeting logistics, and record-keeping.

 

Establishing ground rules. Ground rules must be acceptable to all the principal parties, and generally answer the familiar questions of who, why, what, how, where, and when. With respect to the Columbia River Channel Deepening project, the answers to these questions were:

 

Who (will be at the table): The principal parties consisted of four project managers, one each from the Army Corps, NMFS, USFWS, and the Port of Portland (representing the six ports along the lower Columbia River). The parties understood that they were permitted to bring technical staff to any of the meetings. Because so many federal agencies were involved in the dispute, strong consensus existed to open the six panel meetings to the interested public.

 

 

Why (goals): The principal parties reached common ground on the goal of the process, which in essence was to determine whether a scientific basis exists to attribute declines of threatened and endangered fish to effects of the project.

 

What (issues and non-issues): The principal parties agreed to the focus of the meetings ¾ the issues that would and would not be addressed. In this case, the agreement was on a range of concerns (raised by NMFS in its withdrawal of the Biological Opinion) as they potentially relate to effects of the channel deepening. Further, the parties agreed to a presentation format in which technical staff would present the best available information for a particular topic and respond to questions from the panel and audience.

 

How (rules of procedure): This concerns roles and responsibilities toward each other, the mediator, and external parties such as the media. The focus is on good-faith efforts resulting in a win-win situation as opposed to positional bargaining resulting in a win-lose scenario. These are most effective when designed by the principal parties and written down. In the case of media relations, for example, the principal parties agreed to refer all inquiries to SEI.

 

Where and when (logistics and schedule): The principal parties charged SEI with finding a suitable neutral location that could accommodate up to 75 people. Meetings occurred monthly. In addition, SEI staff met regularly with the project managers prior to each meeting to set goals and objectives and to review other relevant information. SEI maintained a database of approximately 150 interested parties and informed them regularly of upcoming meetings.

 

 

4. Facilitating the Process. As important as the mediator is to designing the process, so is he or she to facilitating the process. In particular, the mediator plays a crucial role in resolving problems, which invariably arise throughout the course of the process. Because the resolution of disputes can be so time-consuming, it is important that the mediator be supported by a competent team to address the myriad details that go into facilitating the process. In the case of the Columbia River Channel Deepening project, the SEI team was instrumental in coordinating the science panel, organizing the six meetings, and conducting outreach and media relations.

 

Resolving Problems. The attainment of solid, shared understandings is essential, particularly during the early and often sensitive stages of trust-building and negotiation. However, while it is important to clarify roles and responsibilities and to establish ground rules at the outset, it is equally important to maintain flexibility throughout the design process and the peer review proceedings. It is virtually impossible to anticipate every possible development in a peer review or consensus process. By designing flexibility into the process, participants can anticipate and better handle changes or surprises when confronted with them.

 

As a peer review process unfolds, common understandings and decisions reached at the outset of the process can begin to unravel. In the Columbia River Channel Dredging project, for example, the following initial understandings were challenged later in the process:

 

Changes in priority: Initially the issue of salinity intrusion into the lower estuary as a result of the dredging was considered by many to be a priority concern. During the course of the proceedings, however, the science panel saw no ‘clear and present danger’ with regard to the impacts of sediment and sediment quality on fish. The panel determined that sediments appear to have little risk, although some uncertainty remains. Therefore, the panel viewed them as a second-order monitoring priority.

 

Broadened issues: The physical modeling used by the Corps in the original Environmental Impact Statement modeled the potential effects of the project on river flow. The science panel expressed concern that the modeling focused largely on intermediate flow ranges without considering extreme low-flow conditions as were currently being experienced in the region-wide drought. SEI then tasked the Corps with determining the feasibility of conducting additional modeling on this issue within the short timeframe remaining in the SEI process. The Corps agreed to contract for additional modeling and to have results ready for the final panel meeting.

 

Disputes over ‘facts’: Midway through the peer review process, NMFS scientists presented ‘new information’ on the effects of toxic contaminants on fish. While this revelation caused some discomfort among the regulatory agencies (mostly in terms of how the information was revealed), the science panel found the information to be largely speculative and without rigorous empirical evidence.

 

More time needed to complete tasks and reach agreement: The initial process design, agreed to by all parties, called for a penultimate meeting on monitoring and adaptive management. However, as the meeting date drew closer, the principal parties decided they were not yet prepared to address these issues. By arranging a closed session between project managers, SEI staff, and some of the panel members, SEI was able to help the parties through complex negotiations, and ultimately, to commit to a monitoring and adaptive management program.

 

Impasse and doubts about the feasibility of resolution: On more than one occasion, SEI was faced with the very real possibility that the entire process would collapse due to the occasional contentiousness of one or more of the parties. These impasses required Dr. Courtney to play a skillful, behind-the-scenes balancing act with each of the project managers. It required him to remind the parties of the importance of having flexible attitudes and creative approaches to problem-solving. It also required Dr. Courtney to be similarly flexible.

 

Coordinating the Science Panel. The scientific credibility of the science panel is paramount in a peer review process. This is the independent group of scientists that digests highly technical and complex information and renders an impartial opinion on its merit. It is essential that panelists be of sufficient stature to command the respect of the principal parties, as well as of the public and media.

 

Dr. Tatiana Boyle was responsible for recruiting the seven members of the SEI science panel and for all interactions with them. In preparation for each meeting, she compiled review literature recommended by the project managers, as well as copies of technical presentations. In addition, she took care of all the logistical details related to the panel, such as ground transportation, hotel accommodations, breakfast meetings, dinner socials, honoraria payments, and expense reimbursements. Finally, Dr. Boyle ensured that all presentations were posted on the SEI web site.

 

At the June meeting, the project managers expressed frustration with a perceived lack of direction from the panel. Panelists were quick to point out that their charge was not to guide, but to review and evaluate the material presented to them. The onus, therefore, rested with the project managers to know what questions they wanted to have answered and to have the panel’s response to those questions on record. To mitigate the sense of frustration experienced by both project managers and panelists, the SEI team worked with the project managers to develop a questionnaire for the science panel to complete before the July meeting. Dr. Boyle worked with the panel to meet the tight deadline, tabulated panel responses, contacted individual panelists to clarify some of their responses, and wrote a brief summary of the panel’s collective responses.

 

Organizing the Meetings. The location of meetings and the quality of meeting facilities and services play an important role in the overall success of a peer review process. The Columbia River Channel Deepening project was fortunate to be able to conduct most of its meetings on the Vancouver, Washington, campus of Washington State University. The facilities were free and campus staff were very supportive of the SEI process. The closed July meeting was held at the Port of Portland, and the final public meeting at the Downtown Marriott Hotel in Portland.

 

Wendy Hudson handled all meeting logistics. Specifically, she was responsible siting the meetings and arranging for food and beverage service and other logistical details. She recorded all meetings and afterwards transcribed the discussions between panelists and presenters. She arranged to have transcriptions (“Meeting Minutes”) and meeting summaries posted on the SEI web site, along with other meeting information, such as the meeting agenda and a list of attendees. Ms. Hudson also prepared materials for interim planning sessions with the project managers.

 

Conducting Public Outreach and Media Relations. Attention to relations with stakeholders and the media cannot be underestimated. These groups have legitimate concerns and the expectation that they will be heard. Anything short of making a good-faith effort to meet their needs can have significant consequences for a peer review process.

 

Ms. Hudson served as the public point of contact for the project, answering questions from the public and the media. She maintained a database of approximately 150 “interested parties” and sent them regular email announcements about upcoming meetings. For the press, she developed press releases and press packets, and occasionally fielded their calls for more information.

 

5.  Seeking Commitments to Implement Agreements

No peer review process is complete without a commitment from the principal parties to implement agreements. Although a peer review process is not typically characterized by formal agreements, this particular process, with its attention to risk and uncertainty, strongly suggested a need for monitoring and adaptive management. Consequently, SEI staff reacted quickly when the project managers cancelled the July meeting on monitoring and adaptive management.

 

The specifics of the two issues had been a source of disagreement between the action and regulatory agencies, and Dr. Courtney felt it imperative that the principal parties have the opportunity to air their concerns in a safe environment. The closed July meeting provided just such an environment. SEI staff arranged for three of the science panelists to discuss with the project managers broad concepts related to monitoring and adaptive management, which created the comfort-level needed by the project managers to speak candidly.

 

Following the July meeting, Dr. Courtney worked behind the scenes with individual project managers to secure their commitment to a flexible monitoring and adaptive management program. Although details remained to be negotiated, Dr. Courtney did place project managers on record in the final peer review meeting regarding agency pledges to implement a monitoring and adaptive management program with a minimum duration of five years.

 

6. Evaluating Success. Science is rarely able to provide finality on issues. In the absence of certainty, though, science offers the best possible shared ‘theory’ about relevant facts. Recognition of the limitations of science as a decision-making tool, however, should not discourage the integration of science into decision-making processes. Rather, environmental decision making demands that decision makers remain open to new information, which is bound to keep changing during and well beyond the end of a peer review process. Therefore, in terms of evaluating the success of a peer review process, it is important not to focus on whether the science is ‘right’ or ‘wrong’, but on whether it provides a sufficient basis for decision-making, factoring in degrees of risk and uncertainty.

 

By the final science panel meeting in August 2001, the SEI peer review process revealed an insufficient scientific basis to attribute declines of threatened and endangered fish to effects of the proposed channel deepening. Perhaps more importantly, the SEI science panel process successfully achieved a commitment from all parties to participate in the joint development and implementation of a scientifically rigorous monitoring and adaptive management program. At this writing, the ports and the Corps are preparing a new Biological Assessment (BA) that will address many of the issues raised throughout the peer review process, as well as address the structure of a monitoring and adaptive management program.

 

SEI has a standard evaluation form it sends to panelists and clients (in this case, the project managers) following every peer review process. Responses to these forms help SEI improve its format for future processes.

 

 

*      *      *