This is a complex and high-impact project with many partnerships, connections, and reports. The ISRP greatly appreciates the time and attention dedicated to organizing the proposal to effectively communicate the project’s results and the details of the plan for moving forward. Some of the proposal and project strengths are:

• The proposal provides a through context for the regional and tribal history, landscape characteristics, ecological focus, limiting factors, monitoring programs, and relevant management plans for the project.
• The proposal provides a high level of detail in the narrative of the text, and then provided direct links to documents where more information could be found.
• Including hot links to annual reports, umbrella, monitoring, and other reports are always appreciated since it saves the ISRP time in locating the documents.
• The proposal uses maps and figures effectively to communicate results and strategy.
• The project proponents have developed a diverse, scientifically robust, and collaborative approach to developing and implementing monitoring in the basin, and they have demonstrated a commitment to managing data produced by those efforts.
• The highly collaborative nature of the project, from monitoring to data management to the common ground restoration approach, is a core strength.
• The 15-year synthesis is of great value and, once complete, will be an important contribution both to the John Day basin and Columbia River restoration more broadly.

The ISRP notes that the prioritization and adaptation processes are not as linear and structured as for other (and often simpler) projects. However, the processes reflect a thoughtful strategy and appear to be serving the project. Thus, a highly structured, formal decision-making framework does not seem essential to continuous learning for this project.

The project meets scientific review criteria, but the ISRP makes the following suggestions for project improvement. Actions toward addressing these suggestions should begin immediately and the results can be described in future work plans, annual reports, and proposals.

1. SMART Objectives: The proponents should clearly identify the project goals and objectives for the next project period. The objectives should be written as SMART objectives (see proposal instructions). These goals and objectives can be provided in the next annual report. 
   
   
2. M&E matrix - support. As habitat projects and monitoring projects are not presented as part of an integrated proposal or plan, the need for a crosswalk to identify the linkages between implementation and monitoring is extremely important for basins or geographic areas. The ISRP is requesting a response from the Escapement and Productivity of Spring Chinook and Steelhead Project (199801600) to summarize the linkages between implementation and monitoring projects in the basin. During the response loop (September 24 to November 22, 2021), we ask your project to assist them in creating the summary and provide information to them about what is being monitored for your implementation project and where, when, and by whom the monitoring occurs. A map or maps of locations of monitoring actions would be helpful in this regard. In most cases, we are asking a regional coordination project like yours to develop the summary, but during the project presentation discussions, the ODFW project agreed that it would be more efficient for them to lead the effort and rely on your project and others for supporting information and assistance.

Q1: Clearly defined objectives and outcomes

The proposal describes a comprehensive and thoughtful list of goals and objectives to guide the project, but they were not organized in a way that made it easy for the ISRP to understand what exactly is being planned. The goal of the program is to “protect, manage, and restore aquatic habitats” and objectives supporting that goal are provided (pages 2 and 25), but they are not SMART. Each objective does have a “physical benchmark,” which could theoretically be measurable but, as written, they include vaguely-defined benchmarks, such as “functioning appropriately.” Most of the objectives are more like project goals and lack quantitative outcomes with explicit time frames. SMART objectives provide a framework for evaluating the trajectory of the outcomes and are not intended as contractual guarantees or constraints on future funding. On page 27, the proposal includes one outcome for the project (long-term increasing trend in fish populations), a SMART goal for that outcome, and Table 3, which the ISRP understands is to provide some linkages between actions and measuring the progress toward meeting that goal. Table 3 seems to cover a reasonable set of actions and maps them directly to locations, measures of success, and monitoring strategy. However, the table is not easy to interpret, and the ISRP has difficulty understanding what the measurable outcomes might be.

The ISRP finds it difficult to follow the many different levels of goals, objectives, outcomes, and then objectives again. For example, how does Objective 1A1 (in Table 3) relate to SMART goal 1.1? Are they the same? And how does Outcome 1 relate to the objectives listed in pages 2 and 25? The ISRP called for “quantitative objectives and measures of progress towards those objectives” in the 2017 Umbrella Review, and the need for improvement in this project area remains.

Q2: Methods

This project is responsible for identification, development, implementation, and monitoring of restoration projects. The ISRP review of the project monitoring is discussed in the following section. For identification and development, during the next project period, the JDWR Project will use a combination of the:

• CTWSRO Strategy for metrics like restoration potential benefit, limiting factors, and identified target and focal actions.
• John Day Basin Partnership’s Atlas for prioritizing basins for project implementation.
• Structured Implementation and Monitoring Framework (SIM) for setting the implementation timeline.

Appendix B is a map of planned project locations for FY 2023-27 that, in combination with Tables 2 & 3, gives a general sense of what types of projects will be pursued. Given that the projects were not selected at the time the proposal was submitted, the text lacks some details on what exactly will be performed where. This limits the ISRP to a review of the process rather than the projects. Based on the information available in the proposal and in the linked documents, the framework for selecting projects, which applies the Strategy, Atlas, and SIM, appears to reflect meaningful ecological metrics. As well, the framework appropriately relies on collaboration with basin partners.

Q3: Provisions for M&E

Collectively, the monitoring program aims to address both project-level effectiveness and basin-level fish responses. Given the diverse collection of projects and partners, as well as the need for collaboration due to budget limitations, the proponents rely on different study plans and strategies across subbasins and projects. For example, in Fox Creek, the proponents developed a monitoring plan that is based on the BACI design, one that will be implemented over multiple years. In addition, the project uses their Implementation Effectiveness Monitoring (IEM) to revisit specific types of projects after implementation to identify maintenance needs. They are also participating in the Structured Implementation Monitoring framework (SIM) with ODFW for projects in the Middle Fork John Day, as well as the Intensively Monitored Watershed (IMW) and BPA’s Action Effectiveness Monitoring (AEM) program.

While details of the monitoring plans are a bit scarce in the proposal, the proponents provided links to several monitoring plans. These plans include a) SMART monitoring objectives that are measurable, b) hypotheses about what response is expected, c) sampling locations, timelines, and protocols, and d) details on data management. In short, the ISRP is impressed at the robustness and careful design of the monitoring plans and commends the proponents for this comprehensive effort. Furthermore, the ISRP was also impressed by the efforts made in centralizing data management, the development of the web-based project tracker, and the decision to hire a data manager to support that effort.

Regarding project adjustments, the proposal links to the Partnership’s Theory of Change approach and also provides some specific examples of how monitoring feeds back into project decision making and design. The details of the Theory of Change framework provide both narrative examples and a complex figure (Figure 2 of Appendix D) to help illustrate justifications for project selection and the anticipated outcomes of actions, but do not provide much insight on project adjustments. However, the proposal’s specific examples of how monitoring data have been applied in decision making are very informative. For example, the proposal outlines key findings from the IMW related to the important role of temperature, shading, and tributaries (rather than groundwater) and how those findings have impacted their restoration work and prioritization. In addition, the proposal describes pre-determined checkpoints (e.g., 25% canopy coverage after 5 years) in the IEM plans that trigger project review and potential modifications. Another example describes how monitoring data are used to delay project implementation following a very poor return of spring Chinook. In summary, while the proposal did not include the summary of a structured decision-making framework for adaptive management, the ISRP finds that the proponents are effectively collecting and utilizing data about the project to inform decisions and are satisfied that this process serves the project’s needs.

Q4: Results – benefits to fish and wildlife

The proposal includes some informative graphics on results from the past twenty years, including an online map, a table that covers the entire project, and a time series of restoration metrics (miles restored, miles accessed, number of LWD structures, acres of juniper removed, number of riparian plants planted). The results are impressive, though it is not clear to what degree these actions have directly benefited fish. The proponents identify another project that has been responsible for data on recovery of fish populations, though connection of the physical metrics and population data to examine causality is not reported. Given that the data are not designed to be used in that way, it is appropriate that such an analysis is not included. Nevertheless, the ISRP looks forward to what proponents will learn from the targeted monitoring that is planned and underway on how the activities are influencing targeted fish populations.

The proposal also discusses outreach and engagement efforts in multiple places. Perhaps most notable is a short film that highlights project efforts, particularly around collaboration between the Tribes and ranchers along Fox Creek. The film is professional and inspiring, and high impact. Since it was published in Oct. 2019, it has over 1,700 views, has been included in multiple film festivals and will be shown in classrooms throughout Oregon.