In our preliminary review, we asked the proponents to lead the development of an M&E Matrix for the John Day River basin. The ISRP appreciates the proponents’ leadership, constructive response, and thorough summary of monitoring activities in the John Day River subbasin, for both their project and other collaborating projects. In many ways, the summary of monitoring and evaluation, matrix of monitoring activities, and maps of monitoring locations provided for the John Day River basin is an excellent example of cooperation among projects and identification of collaborative monitoring and evaluation in a geographic area that the ISRP envisioned in our request for M&E matrices.

In the revised proposal, the proponents provide an initial map and table illustrating the relationships between implementation actions and monitoring. The revised proposal thoroughly describes the project’s monitoring activities, includes maps of the locations of all monitoring actions, and provides a matrix of the structured implementation and monitoring plan for three John Day River steelhead populations. The proponents collaboratively developed a table of implementation projects associated with the Oregon Watershed Enhancement Board Focused Investment Partnership (Table 8). For each project, the table identifies the restoration strategy, monitoring tier, type of monitoring (implementation, effectiveness), potential metrics, use of drone monitoring, and project responsible for monitoring. They plan to incorporate the figure and table in ongoing annual reports to BPA. The revised proposal includes a preliminary map of implementation projects and associated monitoring in the John Day River basin. The map distinguishes the type of monitoring as implementation, effectiveness, or fish-in and fish-out for fish population productivity. The proposal also summarizes linkages between implementation projects and monitoring efforts in the North Fork John Day and Middle Fork John Day rivers. Basically, the proponents provided initial information on all aspects of M&E that the ISRP requested in the Response Loop. Their M&E matrix and summary could be used as an example for other geographic areas.

The proponents responded to the ISRP’s question about quantifying the effects of both flow and temperature and representing these factors in the model for the Middle Fork spring Chinook salmon population and indicated they agreed that it would be beneficial to update and extend these analyses. Given the importance of both flow and stream temperature as limiting factors in the John Day River basin, the ISRP encourages the project to continue their efforts to understand and model the effects of flow and temperature on spawner recruitment.

The ISRP repeats from its preliminary review that this is an exemplary project and is a model for other M&E projects.

Preliminary ISRP report comments: response requested

Response request comment:

The John Day River Salmonid Monitoring project is a status and trends study that has received positive reviews from the ISRP since its inception. The John Day River basin is one of the few basins in the interior Columbia region that has had no recent hatchery releases; however, straying from Snake River populations is a major concern. The project steadily has improved its experimental design and refined its field methods and analyses. Its data are critical for regional management, and the project is closely integrated with key management plans and habitat restoration projects. The proposal provides a thorough literature review, with much appreciated hyperlinks to key papers and reports. This is an exemplary project and is a model for other M&E projects.

The ISRP requests a response from this project to provide the following information:

• M&E matrix - lead. Provide a summary of linkages between the monitoring conducted and the implementation projects in the John Day River basin. One of the challenges for ISRP reviewers is understanding the specific monitoring that is being conducted for multiple implementation projects. Habitat restoration projects or hatchery projects implement actions that are intended to address limiting factors and benefit fish and wildlife. Most of these projects do not directly monitor habitat conditions or biological outcomes, but most identify other projects in the basin that monitor aspects of physical habitat or focal fish species. The monitoring project(s) in the basin provides essential monitoring data for habitat, juvenile salmonid abundance and distribution, outmigration, survival, and adult returns for salmon and steelhead. Some monitoring projects focus on status and trends in basins, while others focus on habitat relationships and responses to local actions. It is unclear what monitoring the monitoring project(s) conducts for each implementation project.

The ISRP is requesting a response from this project to summarize the linkages between implementation and monitoring projects in the basin. The summary should provide a table or matrix that specifically identifies what is being monitored for each implementation project, as well as where and when the monitoring occurs. The summary also should explain how the projects are working together to evaluate progress toward addressing limiting factors and identify future actions. A map or maps of locations of monitoring actions would be helpful in this regard. The monitoring information should clearly explain whether the biological monitoring is local information for the specific implementation site or basin scale monitoring of status and trends or fish in/fish out. We are asking all relevant implementation projects to assist your project in producing this summary. In particular, the John Day Partnership recently hired Nick Bouwes to help map and coordinate their M&E efforts, and they will be asked to assist.

Q1: Clearly defined objectives and outcomes

The proposal clearly identifies four major goals for the monitoring and evaluation, and identifies key management questions addressed by the monitoring and analyses. Quantitative implementation objectives are identified for each major goal. There are no SMART objectives for physical or biological outcomes, but these are not directly relevant for an M&E project designed to provide critical data and analyses for the region and for habitat projects therein. The project generates information used by multiple projects in the John Day River basin and the region, and the activities appear to be well coordinated.

Q2: Methods

The ISRP has reviewed the proponents’ methods previously and has found them to be scientifically rigorous and excellent examples of Tier 2 statistical monitoring at the subbasin scale. The project continues to refine and strengthen their methods. They combined probabilistic sampling for steelhead populations and census sampling for adult Chinook populations. Their data are stored in regional databases and are incorporated into regional planning and evaluation. When BPA budget cuts prevented them from continuing their monitoring for certain population and habitat parameters, they obtained external funding for some measurements. Most notably, they obtained alternative funding for Chinook escapement assessments, which allowed them to continue to estimate SARs, a critical need for the Fish and Wildlife Program. They have used a spatial model (Falke et al. 2013) to provide quantitative predictions of redd occurrence probability and probability of spawning at a landscape scale. This information provides an important context for planning and prioritizing restoration actions.

The proposal thoroughly describes study designs, field and laboratory methods, and analytical procedures. Methods, data, sampling locations, and metadata are located in NRIMP, MonitoringResources.org, and Annual Reports. The ISRP appreciates the thorough descriptions of the methods.

The proposal includes a Gantt chart to describe the general operations of the project monthly for 2023 through 2025. While this helps reviewers understand general distribution of the project efforts through time, it does not provide information on the subcomponents of the project. This information should be provided in annual reports and work plans.

In the Survival, Age Structure, and Productivity portion of the Methods section, the proponents mention that environmental covariates and restoration metrics can be incorporated into the models with more time series data. Given the changes in temperature and flow occurring throughout the basin and the tremendous effort going into restoration activities throughout the basin, this is especially critical. Quantifying effects of these factors on fish will be key to manage for future conditions and ensure that restoration is effective. Could the authors attempt to run models that include these factors now, even if results are preliminary? If not, when do the authors anticipate that enough data will be available?

Q3: Provisions for M&E

The proposal describes a thorough adaptive management process based on regularly scheduled internal meetings, annual reports, and consultation with BPA. They also describe a formal process of meeting with the John Day Basin Partnership to inform habitat restoration planning and implementation. Their explanation is much more complete than that provided by other related projects in the Partnership.

It is less clear what triggers modifications to the program. How are changes to the project proposed and implemented? For example, the proposal mentions the need for more information on bull trout migration, possible challenges due to increasing numbers of invasive smallmouth bass, and changing climate, including asynchronous hydrology between the Columbia River and the John Day River. What prompts a redirection of the project to better account for these factors? Does each component have a “threshold of probable concern” that would result in changes to the activities? If so, what are they? As well, when is each action deemed successful, and when is it not?

The proposal also describes potential confounding factors related to climate change and expansion of nonnative predators. Rather than simply identifying and discussing the nature of these confounding factors, they present studies they are currently implementing to address these confounding factors and develop methods to implement and assess future management actions. They also have developed an approach for identifying coldwater refuges that could serve as the basis for habitat protection and restoration in the face of climate change. Equally importantly, they are conducting experiments to determine whether smallmouth bass predation is additive or compensatory, a complex and important question in the Columbia River Basin at this point.

The proponents provide an excellent description of their responses to past Council recommendations and ISRP reviews. They have responded positively and creatively to past suggestions, making improvements that exceed recommendations from the ISRP. This constructive communication between the proponents and the ISRP is an example of how the scientific review process is intended to function.

Q4: Results – benefits to fish and wildlife

The proposal provided 13 pages with informative tables and figures of past progress and outcomes to benefit fish and wildlife from the project since 2000. The proponents identified the importance of this information for management decisions within the Columbia River Basin and groups that are using their information for conservation and restoration actions. They clearly have identified lessons learned from their results and how they are being used for management. The indirect benefits of his project are large as this project strongly contributes to status and trend monitoring for steelhead and Chinook salmon, life-cycle models, regional actions, and management of the hydrosystem.