Response requested comment:

The ISRP requests responses to the following issues. We anticipate that some topics (e.g., 2, 4, and 6) will likely take more time to address than provided in the month-long response period. For those issues, the response should indicate how these questions will be addressed in the future.

1.      The project's objectives need to be modified with inclusion of measurable metrics and timelines. The use of SMART goals and a Gantt chart for achieving them would greatly strengthen the value and feasibility of project outcomes.

2.      A strategic plan is needed to ensure that the modeling and coordination efforts are effective. Important questions that should be addressed include: What is the long-term vision for the program, and what are the intermediate steps that need to be completed to get there?

3.      Over the short term (next three-year cycle), more detail is needed on project deliverables and on the work plan to achieve them. The proposal does not include anticipated quantitative results and benefits or any timeline of when outcomes may be achieved. A primary weakness of this proposal is the lack of clarity of specific outcomes so one can identify when the project will be complete. The proponents are clearly active, especially with model development and coordination of data and model sharing, but it is not clear what activities are occurring, or why, or that it is organized strategically.

4.      Details are needed on how the modeling informs decision-making, including the design and prioritization of restoration. It is not clear how information from the modeling efforts will be used to make decisions about how the tribes will manage their First Foods resources or interact with other co-managers to better manage flows, temperatures, water quality, and habitat to benefit these First Foods.

5.      Communication about the program and its benefits needs to be improved. The website, for instance should be modified to elevate the profile of the project and improve access to information produced by the project.

6.      Environmental change within the Basin is affected by climate change, land use changes, and modifications to the flow regime. Will the project evaluate the relative impact of these factors on environmental conditions in the Basin?

This program has the potential to be highly valuable for the tribes' and the region. It offers an important opportunity to examine restoration actions in the context of climate change and provides the Tribes with the expertise and tools to be an informed voice in policy-level discussions and decisions. However, the ISRP is concerned that a lack of strategic planning has led to a program whose full potential has not been realized.

The ISRP welcomes further dialogue with the project proponents as they develop their response. The response should address the qualifications listed above as well as several qualifications (numbers 1 and 3) from the 2012 ISRP review (ISRP 2012-6) that have not yet been addressed.

Comment:

1. Objectives, Significance to Regional Programs, and Technical Background

The key objectives of the project are not clearly or consistently stated. They generally center on: (1) reproducing and enhancing models for simulating the impacts of climate change, (2) allowing CRITFC personnel to represent tribal interest and expertise in regional technical workshops, and (3) providing opportunities for the tribes to exchange information about the impacts of climate change on flow regimes, water temperature, water quality, and hence on First Foods (salmon, lamprey, and eulachon, as examples) with other regional experts.

The project's stated objectives need to be quantitative and include timelines with milestones for achieving them. As stated, they are only general work statements; consequently, in the future, there will be no way to tell if they have been achieved. Details on the activities needed to accomplish the objectives are too general, making it difficult to conduct a scientific evaluation of the program. For example, the program is missing details regarding how the first two objectives (e.g., database and tributary analysis of flows and temperature) will inform future restoration strategies and planning for climate change. This issue was raised in ISRP 2012-6.

Near the end of the proposal, a specific objective stated is: "The ultimate objective of CRITFC's Climate Change Impacts Project is to develop and integrate various physical hydrological/river operational model outputs (CIS; CMOP) with water quality models outputs (RBM-10 Yearsley 2012 with updates) and fish spatial habitat models (Hatten et al. 2009) and fish survival models (Zabel et al. 2008; McCann et al. 2018)." However, it is not clear how the development and integration of this model will inform adaptation strategies for climate change.

Other activities are missing clear objectives. For example, what are the objectives of the technical review and information transfer, and how will the proponents know if those efforts are successful?

Finally, the proposed dissemination plan to make some data available through their website and flyers may not produce the greatest impact. It would be useful to provide details of the program's dissemination plan, since this is essential to Objective 3. For instance, who are the target audiences, and how effective are the dissemination activities at reaching those audiences? Also, the model produced by this project appears to be valuable to many groups conducting restoration in the Basin. Is it available in the public domain? A strategic dissemination plan is needed to elevate the profile and impact of this work.

2. Results and Adaptive Management

The proponents are clearly busy, working on a range of topics that include Pacific lamprey and eulachon, benthic macroinvertebrates, and food webs. However, the ISRP was unable to understand who produced what results and how the results contribute strategically toward project objectives and/or informing adaptation.

In addition to the proposal lacking information on the key results of program to date, the list of deliverables in the "Contracted Deliverables and Quantitative Metrics" report was not detailed enough to evaluate progress of the project. It repeats the same items year to year and does not indicate what progress was made on them. For example, "Analytical Framework to Develop and Enhance Regional Climate Change Assessments" and "Estimate Changes in Mainstem Hydrology and Water Quality due to Climate Change Impacts" were repeated between years without any indication of what was actually accomplished. There were four topics listed under "Summary of Main Deliverables" (e.g., updates to HYDSIM, including daily time step, residence, time, ecosystem rule curves, etc.), but it is not clear when those deliverables were completed or how they relate to the ultimate needs/deliverables for the project.

A strength of the project is the facilitation of communication among research and technical groups who are modeling climate change impacts to the ecosystems and tribes, so that tribal interests in preserving traditional foods and culture can be represented in future management. The project has led to the proponents' participation on a number of regional technical forums and review processes. It has also resulted in the development of datasets and the CIS modeling tool that can be applied regionally to inform resilient river restoration. In particular, the two-way exchange of information between CRITFC and the wide range of stakeholders is a strength of this project. However, the project would have a greater impact if it was better organized around a coherent long-term strategy and near-term work plans and how technical details are leading to better adaptation actions (Goal 2 and Objective 3).

Some progress has been made in collaborating with groups who are modeling climate change impacts to adapt these results to scenarios of interest to the tribes, such as ecosystem function (although this has not been sufficiently defined). An important outcome is that the development of CIS has provided CRITFC the technical expertise and tools to contribute to major regional planning efforts (e.g., CRT, RMJOC II) and insert ecological objectives into planning and prioritization processes. Some of those efforts have ended, and CRITFC will need to continue to advocate for their seat at these planning forums, an issue that highlights the need for CRITFC to elevate the profile of their expertise and tools.

It would be helpful in the proposal to see some results of the project's analyses. For example, what were the results of the habitat modeling that USGS did with CIS outputs under different operational scenarios? Or what were the results of the collaborative project to study salmon survival and smolt-to-adult survival using CIS? And how were those results disseminated? The ISRP was unable to find much discussion of key results on the website, in the annual reports, or in this proposal. Results of at least one set of analyses are published (e.g., Justice et al. 2017) in peer-reviewed journals, which is commendable.

While the philosophical approach to adaptive management (AM) proposed by Rieman et al. (2015) is a good framework, the details of how AM is used in this project are not thoroughly described in the proposal. The ISRP would appreciate learning about the internal mechanisms being used to achieve an effective AM program and an effective project, as well as learning about any problems being encountered in its application. Even outside of a formal AM framework, the proposal does not describe lessons learned or how the project has adapted over time. The proposal discusses adaptive measures in response to climate change such as modified pumping to increase releases of cool water pool at Lake Roosevelt, selective withdrawal at Grand Coulee, and so forth. However, there was no discussion of adaptive management for this project or how the adaptive measures being analyzed improve management decisions on the ground. In addition, the ISRP would appreciate learning more about the barriers that led to the conclusion (p. 23) that "The development (proposed previously) of an overarching decision support system was found to be an unrealistic and ineffective approach so is no longer included as an objective." Rather than dropping it as an objective, since it is such an important outcome of this project, could the work activities be adapted towards translating model results into information that could be used by decision makers?

3. Methods: Project Relationships, Work Types, and Deliverables

The ISRP was not able to adequately evaluate project relationships without more detailed information on the mechanisms for cooperation and, specifically, the activities for which this project is responsible. In one of several examples, it was difficult to understand the synergy and/or overlap with the Monitoring and Recovery Trends work (BPA project #2009-004-00), which should be capitalized upon to examine the effects of restoration and climate change on fish populations. It was indicated in the proposal (page 28) that this capacity would be available at some point, but it was not clear what tasks remain to achieve this, who would complete those tasks, and by when. Given the critical importance of this type of analysis for the proposed framework, more details on how this work will be completed is needed to evaluate its feasibility and the merit of the approach.

Likewise, it is not always clear what CRITFC's contribution is to some of the technical forums in which they are participating. For example, for the proposal to NOAA with the University of Washington (UW), it appears that the CIS will be integrated with UW's forecasting models to develop some real time forecasting (of temperatures?). The workplan is not explicit enough to know if UW is just using CRITFC's CIS or if CRITFC actually plays a role in the study design, analysis, or dissemination of results. In other cases, such as the regional workshops, the proposal explicitly identifies how CRITFC contributed (e.g., assisted in workshop sessions, contributed to writing a report, and so forth), which was helpful in understanding their role and relationship to the other programs, even though it still was not always clear how it related to the objectives of this project.

There was also some question about how the work products relate to similar products being produced by others in the Basin. For example, how does the projection of stream temperatures relate to or duplicate the work of NorWeST?

Regarding the work types, the proposal's work plan was not detailed enough to evaluate the past and proposed activities. For example, the proposal notes several tasks with similarly vague details such as: "Further work to improve the RBM-10 model integration with CIS is proposed for 2019 and outyears as is exploration of CIS data collaboration with CE Qual W2 two dimensional and other water quality models." This concern about lack of clarity in the work plan was also raised by the ISRP in the 2012 review.

Finally, while four deliverables are discussed, they are worded in a way makes it hard to understand when they are completed and the objectives have been achieved. It is not always clear exactly what the final deliverables will be or what tasks are needed to produce those deliverables.

The response addresses many of the ISRP concerns raised during the review of the original proposal. However, there were several ISRP concerns that were not adequately addressed, as reflected in the Qualifications below. Additional comments on specific elements of the response are below.

1) The response by the sponsors presented much more detailed information on the project objectives and progress to date than was contained in the original proposal. This new information greatly facilitated the ISRP review.

2) The response indicates that there is alignment between this project and other climate research groups in the region. The project sponsors are aware of the latest work in the region to estimate changes to temperature, flow, and habitat due to climate change, as well as the latest work, modest as it may be, to project the effects of these physical changes on fish populations at the regional and local scale. Considerable effort should be expended to ensure that these relationships become more collaborative over time; interactions among programs should be synergistic rather than overlapping. Are there mechanisms in place to ensure that this happens?

3) Objective 1 and 2 and the methods that will be used to address these objectives were well described. As noted in the qualifications above, the description of the approach to Objective 3 was not complete.

4) The data base tool was adequately described in the response.

5) Inclusion of a summary of the results from the flow and temperature projections developed for Toppenish and Satus creeks was very helpful in illustrating the methodology to be used to estimate climate change impacts on tributary habitats.

6) The ISRP’s concern about the lack of a fully-developed adaptive management strategy or formal decision support process for this project was not completely addressed. The response indicates, “This project will develop information and model projections necessary to assist Tribes in updating their resource plans to include restoration and maintenance actions that ensure ecological resilience. Through collaborative partnerships, this project will also provide information to state and federal agencies necessary for those entities to determine what will be required co-manage tribal natural resources.” In a very generic sense, this statement indicates the intent to use the information generated by this project adaptively. But little detail is provided as to how this goal will be accomplished.

The response adequately clarified many of the issues the ISRP raised in the review of the original proposal. However, three qualifications remain: 

While there is a need for Tribal agencies to develop expertise in Climate Change and the consequences for First Foods, this proposal does not contain enough detail to judge whether or not this work will make a meaningful contribution to attaining this goal. Overall, the concept of this proposal involves using downscaled climate projections to predict stream/river temperature and flow, and then using these predictions to project biological responses by fish and other organisms. This approach is similar to that being employed by many ongoing projects in the region and such work can generate important information about climate change impacts on natural resources. This proposal is clear on the general goals, but provides little detailed information on how they will be achieved. Detail about the work elements was insufficient to enable a comprehensive review of the technical adequacy of the study approach. A revised proposal with more specific information about the objectives and methods is required to show that the work is based on cutting-edge science and has a high probability of achieving its stated objectives. In addition, a much more complete description of the adaptive management process, clearly indicating how information generated by this project will be communicated to managers, should be included.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The ISAB/ISRP have contended for the last decade that subbasin plans, and the projects based on these plans, should explicitly consider climate change, human population growth and invasive species in setting restoration priorities. The lack of spatially-explicit predictions of the relative intensity of these impacts has made evaluation of these factors at an individual project scale, or even at the subbasin scale, problematic. This project proposes to generate predictions of climate change at a scale appropriate to estimate impacts on First Food resources. Changes in climate will be used to predict changes in aquatic habitat, and habitat predictions will be linked with fish population models to provide projections of potential impacts to focal populations under a variety of future climate scenarios. These predictions will then be used to help guide mitigation measures. This is clearly a worthwhile goal and would be of considerable significance to regional restoration programs.

The technical background provided in the proposal fails to include some important elements. There is little, if any, acknowledgment of the work being done by the University of Washington Climate Impacts Group, NOAA and others to better understand the biological consequences of climate change in the Pacific Northwest. Although the proposal indicates that these groups will be consulted as this project moves forward, the work that has been completed by these research organizations to date could have been used more effectively to describe the critical nature of the problem that this proposal plans to address. 

There are three general objectives provided in the proposal: 1) estimate changes in mainstem hydrology and water quality due to climate change; 2) incorporate a climate change decision support system through tributary habitat and fish population modeling; and 3) develop sustainable adaptation strategies and tools to respond to climate change. The objectives are vague. OBJ-1 does not clearly indicate how projected changes in mainstem hydrologic patterns will be used to identify likely changes in water quality. OBJ-2 glosses over the considerable difficulty of using stream habitat conditions to predict productivity and capacity parameters that are required for population dynamics modeling. There is considerable experience with this problem in the Columbia Basin, and the proposal should review some of this information and provide some discussion of the high degree of uncertainty associated with this type of prediction and how this might affect the ability to achieve Objective 3, which proposes to develop a Climate Change Scenario Data Base Tool. While a Climate Change Scenario Data Base Tool sounds useful, there is insufficient description of this tool in the proposal to understand what it will actually provide. Including an example or demonstration in the proposal would be useful. Further, the mechanisms by which CRITFC will conduct and coordinate future database tool development with others in the region are not apparent.

2. History: Accomplishments, Results, and Adaptive Management (ISRP Review of Results)

A brief history of the project is provided in the proposal, and a link to a progress report for 2010-2011 project accomplishments is included. One of the major accomplishments in the first year of this project was the development of a unique database tool for modeling and predicting the effects of future climate change and tributary irrigation withdrawals on Columbia River mainstem flows. Little detail was provided on this tool; a description of the tool and an example of its application would have been informative. Also, has the tool been tested and appropriately validated? How effective is the tool in providing reliable predictions? It also would have been informative if the key results from the work to model future temperatures for the Satus and Toppenish watersheds were included in the proposal. Including this information would seem to be possible as the proposal noted that a manuscript is under review. 

The adaptive management component of this project is not fully described. The project sponsors indicate that results of this work will be made available through usual scientific channels (publications and presentations) and will be available on the CRITFC website. A complete adaptive management process must also include a mechanism for transferring information to those implementing restoration measures in a form that they can apply to the problems with which they are dealing. It is not clear that the CRITFC website is sufficient to achieve this transfer. The information that could be generated by this project, particularly the effort to extend predictions of the potential impact of climate change to tributary habitats, would be useful to nearly all practitioners of habitat restoration in the Columbia Basin as well as mangers. Including an expanded adaptive management discussion in this proposal that outlines a process to convey the information generated by this project to people responsible for setting basin policy and restoration priorities would be of great value.

Retrospective Evaluation of Results

A link to a progress report for 2010-2011 indicates that one of the major accomplishments in the first year of this project was the development of a database tool for predicting the effects of future climate change and water withdrawals on Columbia River mainstem flows. However, little detail was provided on the nature of the tool or the accuracy of its predictions.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (hatchery, RME, tagging)

The emerging limiting factor addressed by this project (climate change) is widely appreciated as having the potential to impact the effectiveness of restoration efforts throughout the Columbia Basin. The first two objectives for this project are commendable in that they propose to develop methods to predict climate change impacts on both the mainstem of the Columbia and Snake rivers and for tributary habitats, where most habitat restoration efforts are focused. The third objective, “Develop Sustainable Adaptation Strategies and Tools to Respond to Climate Change” relates to the dissemination of the project results. As noted in the comments on adaptive management above, this objective is a critical component of the project, but the description of steps required to meet this goal are incomplete. A more comprehensive strategy for communicating project results to people in the Columbia Basin working on fish recovery and habitat restoration is required to take full advantage of this effort. The activities under this objective should be expanded to better address this goal.

The proposal appears to treat climate change as if it were the only major change occurring in the Columbia Basin. In fact, human population increase and consequent land use alterations, and invasive species may have as great or greater impact on productive capacity of some aquatic systems than climate change. Results of the study may not be very meaningful unless these other factors are also considered. Are these other impacts factors being incorporated in the study? If so, how?

Project relationships for this effort were adequately covered in the proposal. This project appears to be well aligned with some of the climate change work ongoing in the Columbia Basin and has established relationships with the major research groups in the region that work in this field. 

4. Deliverables, Work Elements, Metrics, and Methods

The deliverables, work elements, and methods that were presented appear to be appropriate for meeting project objectives. However, this section of the proposal is much abbreviated. There is little detail provided as to how the deliverables will be accomplished, and the methods to be used are only very briefly discussed. In fact, the deliverables essentially just repeat the objectives. As noted above, there are ongoing studies being conducted by agencies and universities that are closely related to Deliverable 2. The sponsors could have used this work to better define the work elements to be included under this deliverable. Publications from these projects should be cited in the proposal justification. Also, there is no discussion of the relative availability of data on current habitat condition. It would seem that this type of information would be critical for assessing the potential impact of climate change on tributary fish production (Deliverable 2). Deliverable 3 (Develop Sustainable Adaptation Strategies and Tools to Respond to Climate Change) in particular needs elaboration. This deliverable appears to consist primarily of attending meetings and workshops. No discussion of how possible adaptation strategies will be identified and assessed is provided. 

Much more detail on the protocols that will be employed in executing this study should be included in the proposal. Some additional detail on the hydrological and biological models that will be used, some indication of the range of climate scenarios that will be included, and some indication of where difficulties might be encountered also would have been useful. A complete technical review of the project is not possible without a greater level of detail on the deliverables and work elements.

4a. Specific comments on protocols and methods described in MonitoringMethods.org

Insufficient detail was provided on study protocols to enable a technical review. Therefore, meaningful comments cannot be provided. Methods should be thoroughly described in a revised proposal.