The ISRP requests the proponents to provide information on the following condition in the next annual report:

• SMART Objectives. The proponent needs to revise the SMART objectives to clearly identify the expected outcomes (i.e., desired response) in quantitative terms. To be complete, a SMART objective should be accompanied with quantitative implementation objectives that describe restoration actions and monitoring actions to evaluate the extent of success. The ISRP recognizes that budget constraints may not allow implementation of all desired monitoring actions at this time. The proponents should provide revised SMART objectives in the next annual report and work plan and use them as a basis for reporting of progress in future annual reports.

In our preliminary review, we requested responses on three topics. Our final review comments based on the responses are provided under each topic:

1. SMART objectives. The ISRP suggested recasting the objectives to explicitly indicate expected outcomes of various metrics for each year of the project. As currently written, Objectives 1-4 simply indicate general intentions to improve conditions (in summary, 1: restore longitudinal connectivity, 2: restart ecological processes and develop heterogeneous aquatic habitat, 3: restore multichannel planforms, and 4: restore native riparian and wetland vegetation). The proponent did not provide updated objectives with explicit expected outcomes, citing that the budget would not allow adding efforts to measure success. Presumably, that means the proponents will not assess possible failure (i.e., not making things better or making things worse) either. While the current objectives describe the various areas to be treated (in terms of miles or acres) over the five years of the proposed project, they do not describe the desired response levels of increase or decrease over baseline conditions.

Because the objectives are not quantitative, it is difficult to understand how much will be done and how success would be measured. For example, Objective 1 states “Restore longitudinal connectivity (sediment, wood and all fish life stages) to 20+ miles of tributary habitat over the next 5 years (FY21-FY25).” An example of how to change this to a SMART objective would be: Restore longitudinal connectivity for transport of sediment, wood, and all life stages of fish by removing or modifying 10 artificial barriers in 20+ miles of tributary habitat over the next 5 years (FY21-FY25). The addition of one or more Quantitative Implementation Objectives (1.a. 1.b., etc.) would identify specific actions (e.g., barrier types and locations to be removed) that will occur within the 20+ miles, and how much of each action is expected to be completed within a given year. If the objective quantifies the expected outcome, then implementation monitoring could evaluate if the objective was achieved. This implementation monitoring may or may not demonstrate that connectivity increased (i.e., increased fish or wood movement), but there is presumably an assumption that the specific actions would increase connectivity. These assumptions should be justified with existing monitoring information, documented best management practices, and published studies to the extent possible in the proposal and annual reports.

The original ISRP review provided an example for converting Objective 4 to a SMART Objective.

It is assumed that annual reports will document what activities were done and how many stream miles or acres were variously treated. Without additional information about expected outcomes relative to a baseline and a target, it will be difficult for reviewers, and the practitioners, to determine the level of success of this project.

2. Project selection. The proponent provided the requested information on the project selection process. As a project is planned and implemented, it will be important to document in annual reports the process by which the project was chosen.

3. M&E matrix – lead. The Yakima Basin Habitat Project declined to provide a matrix summary of M&E projects in the Yakima River subbasin. The proponents indicated that preparation of the M&E summary is presently beyond the scope and scale of current project resources and budgets. They indicated that assessments of fish-in/fish-out response to aggregate project activities are conducted at Prosser and Roza dams and that status and trend information and data are provided in annual reports. They state that Yakima subbasin projects are not presently funded to conduct effectiveness monitoring. The ISRP reviewed the M&E components of the different projects in the Yakima River subbasin based solely on the information provided in their original proposals, associated documents, and any information provided as part of the Response Loop.

Though they did not provide the requested M&E summary at this time, they think that a matrix would be helpful in basinwide coordination and development of the regional RM&E. They also indicate that basinwide partners have a general desire to create such a matrix and potentially coordinate through the YTAHP. As the NPCC Fish and Wildlife Program develops efforts to identify monitoring activities and coordination between projects in major subbasins, the ISRP encourages this project to contribute its expertise and resources to help create an effective summary for this geographic area. The ISRP has provided additional information on the summary of monitoring and evaluation for geographic areas in the Programmatic Comments of this report. 

Preliminary ISRP report comments: response requested (Provided for context. The proponents responded to the ISRP’s questions; see response link and final review above.)

Response request comment:

This is a long-running project that is seeking a name change and a change in focus. The project originally started out focusing on protection, but for a variety of reasons, they are seeking to shift to restoration with little or no acquisition planned. The proponents provide a good proposal for restoring salmon and steelhead habitat in the Yakima subbasin, including quantitative objectives, well-developed methods, and quantitative results in terms of physical habitat actions. However, the proposal provided little evidence of a strong interaction with other projects that are monitoring response of fish populations and water temperature. This project and other habitat restoration efforts in the Yakima subbasin should clearly describe how each project is coordinating with other habitat projects and fish and habitat monitoring efforts.

If hatchery salmon are released into streams, will they find sufficient habitat to support them? Habitat practitioners, fish biologists, and hatchery managers should be continually collaborating and sharing information. This project is one of the primary habitat restoration projects being conducted in the Yakima subbasin, with others working on other issues such as improving passage and flow. Understanding the cumulative effect of all the habitat projects is critical because an underpinning of the success of the supplementation program being conducted hinges on improving the habitat.

In general, the proponents propose to use a suite of scientifically sound restoration techniques that address known limiting factors in hopes of benefiting the targeted salmonid species. An especially strong point of the proposal is how they conduct a wood project. However, a better linkage to past and expected biological outcomes needs to be documented in the proposal itself.

The ISRP requests the proponents to address the following points in a revised proposal, and to provide a brief point-by-point response to explain how and where each issue is addressed in the revised proposal:

1. SMART objectives. The proponents need to develop a complete set of SMART objectives (see proposal instructions) for this project and incorporate and submit them in a revised proposal, which will provide complete project documentation for future reference on reporting project progress.
2. Project selection. Please describe how a project is selected for implementation. Is there a project selection framework? What criteria are used to prioritize a project? How is information used to evaluate potential habitat projects?
3. M&E matrix - lead. One of the challenges for ISRP reviewers is understanding the specific monitoring that is being conducted for multiple implementation projects. Habitat restoration projects or hatchery projects implement actions that are intended to address limiting factors and benefit fish and wildlife. Most of these projects do not directly monitor habitat conditions or biological outcomes, but most identify other projects in the basin that monitor aspects of physical habitat or focal fish species. The monitoring project(s) in the basin provides essential monitoring data for habitat, juvenile salmonid abundance and distribution, outmigration, survival, and adult returns for salmon and steelhead. Some monitoring projects focus on status and trends in basins, while others focus on habitat relationships and responses to local actions. It is unclear what monitoring the monitoring project(s) conducts for each implementation project.

Given the regional leadership responsibilities of this habitat restoration project, the ISRP is requesting the Yakima Basin Habitat Project (199505100) to summarize the linkages between implementation and monitoring projects in the geographic area. The summary should provide a table or matrix to identify what is being monitored for each implementation project and where and when the monitoring occurs. The summary also should explain how the projects are working together to evaluate progress toward addressing limiting factors and identify future actions. A map or maps could help identify the locations of monitoring actions. The monitoring information should clearly explain whether the biological monitoring is local information for the specific implementation site or basin scale monitoring of status and trends or fish in/fish out. We are asking monitoring and other implementation projects to assist your project in producing this summary.

Q1: Clearly defined objectives and outcomes

The overall goal of the Yakima Basin Habitat Project (YBHP) is to restore watershed processes to aid the recovery of salmonid stocks in the Yakima subbasin. The proposal provides well defined, largely quantitative objectives. The proponents provide a 10-year habitat plan and a detailed prioritization list of restoration projects.

Some objectives are provided, but it would be helpful for the proponents to convert them into more of a SMART format. One reason why this can be useful is that the use of SMART objectives can be used to guide reporting since reporting can track each objective. It is not clear what objectives 5 and 6 were as they were statements. The useful timeline that is provided shows approximately 30 projects being sequenced and could be converted into a display of objectives for this work. For example, each line in the timeline could have additional details on numbers of stream miles affected, or trees to be planted, and so on and when those are expected to happen. While some variability in the timeline will ultimately happen, this is a useful way to display what the project intends to accomplish.

No monitoring objectives are provided even though the project noted it was doing some amount of monitoring. It is not clear what outcomes the project is looking to achieve. For example, is it doing a certain number of projects, planting some amount of riparian zone, or seeing a particular fish response? These could be more clearly specified. The site prep work (such a LIDAR and flow work) could readily be converted into post-project monitoring for a before and after comparison.

YBHP Objective 4 states:

Restore native riparian and wetland vegetation on 60+ acres over the next 5 years (FY21-FY25) to provide stream shading, bank resistance, allochthonous inputs and LWD recruitment.

The objectives should be recast into SMART objectives. As an example, Objective 4 could be reworded to be:

Obj-4: Restore native riparian and wetland vegetation on 60+ acres in to provide <30%> more stream shading, <30%> more bank resistance, <25%> more allochthonous inputs, and <50%> more LWD recruitment with equitable progress per year over the next 5 years (FY21-FY25).

When the objectives are recast in SMART format, it becomes much easier if annual progress reports are organized accordingly to assess progress towards targeted outcomes.

The Timeline provided in Section 7 shows how the proponents plan to implement the overlapping restoration efforts (planning, design, and implementation) across the Yakima subbasin. By itself, however, it does not indicate the extent and expected outcomes of the activities (e.g., number of riparian acres planted, number of miles of stream restored).

Q2: Methods

The restoration approach and methodology are appropriate. To achieve the broad goal of restoring watershed processes to aid salmonid recovery, YBHP uses a three-prong approach: 1) Assess watershed, habitat, and fish conditions/status to inform the prioritization of restoration activities, 2) Protect, restore, and enhance priority watershed and reaches, and 3) Provide educational opportunities and public outreach related to salmon, habitat, and water quality and watershed health. Restoration activities are aimed at restoring stream processes by removing or mitigating watershed perturbances and improving habitat conditions and water quality. Protection activities complement restoration efforts within the subbasin by securing refuges and preventing degradation.

Along with the substantial information about restoration approaches and techniques provided in the proposal, an excellent group of supportive materials are provided that helped to elucidate the assessment, prioritization, design, and implementation process for projects. Detailed methods on adding large wood placements are excellent with a design framework, objectives of wood placement, and site evaluation. However, no other methods are provided for other types of restoration. Are large wood additions the only type of restoration actions being implemented?

Restoration projects include both engineered logjams (ELJs) and unanchored or “loose” wood placements to increase instream complexity, side channel activation, riparian restoration, increased floodplain inundation, and removal of lateral floodplain impediments. A technical guidance report for setting unanchored large wood is provided. This document aids YBHP staff for assessment of site suitability and for design of unanchored large wood. Projects with a low to medium risk are designed in-house and projects with a medium-high to high have in-house conceptual designs that are then contracted out to advance with engineering. YBHP staff develop 2D hydraulic models to evaluate and support a number of assumptions about wood placement.

The project notes that some monitoring is done, but no details are provided on what is monitored, why it is monitored, or what methods are used. What is missing is how the metrics listed in the objectives will be measured and assessed for success. Some verbiage should be provided to describe how the following metrics will be assessed: longitudinal connectivity, restart of physical process, restart of ecological processes, heterogeneity of aquatic habitats, increase in multi-channel planforms, increase in overbank flow frequency, increase in shallow water aquifer recharge, reduction in active channel severity, increase in refuge habitat, increase in native riparian and wetland vegetation, increase in stream shading, increase in bank resistance, increase in allochthonous inputs, increase in wood recruitment, increase in the beaver population, and increase in nutrient availability.

While the proposal contains helpful details on how a project can be designed, it provides only limited descriptions of methods for project selection. Are projects selected based primarily on habitat factors or do fish metrics have a role? How are projects identified and prioritized? Are the right projects being done?

The use of repeat LiDAR to assess changes in erosion and deposition described under Section 5 (Project evaluation and Adjustment process) is a good example of how metrics are assessed, and the description should be moved to Section 4 (Methods). The same is true for descriptions of the use of a small-Unmanned Aerial Vehicle (drone), geomorphic change detection tools, and 2D modeling for assessment of success.

The discussion of the impacts of the 2020 FEMA Region X rescindment of the Policy on Fish Enhancement Structures in the Floodway was enlightening. It will be important to document how this impacts the project, and the proponents are encouraged to keep us informed through annual reports and discussions with your BPA Contract Officer.

Q3: Provisions for M&E

The proponents largely rely on the BPA EIS (1996) to guide the project adjustment process. An example is provided that described how the project derived what best combination of treatment and plant source resulted in higher planted vegetation survival. It would be good to see several other examples from across the spectrum of activities performed under this project.

The proponents explain why this project has migrated away from land acquisition and away from passage projects since its inception in 1997. Starting in 2014, these changes were made in reaction to changes within BPA acquisition process and to decrease duplication and interference with other agencies carrying out similar activities. These changes appear to represent a reasonable and needed adjustment to the project.

YBHP uses information gathered from past project monitoring to inform designs of future projects. The effects of management actions (restoration/enhancement projects) are monitored and evaluated at various intensities (qualitative vs quantitative approaches), and successive projects, phases, or project types are modified in response to these findings. Physical data, such as that produced from LiDAR acquisition have been important to evaluating restoration site characteristics and design. Repeat LiDAR datasets have proved valuable in evaluating project success by producing DEM of Difference (DoD) data that enables comparison of two, spatially identical topographic surfaces to determine quantities and areas of erosion and deposition resulting from a project action.

The adaptive management and iterative project review process for the YKFP involves identifying objectives, strategies, operating assumptions, uncertainties, and risks that are reviewed annually by project scientists. The purpose of these annual reviews is to (re)assess project objectives, progress towards those objectives, and to evaluate whether any strategies or assumptions need to be altered in the face of new information gained over the past year. If reassessment is necessary, the YKFP science teams bring proposals to the YKFP Policy Group for consideration and action.

The proposal states that the "YBHP is prioritized to address tributary limiting factors of reduced stream complexity and channel structure, elevated summer water temperature, reduced floodplain connectivity and function, insufficient large wood in channels, and degraded riparian conditions." The metrics described in the Results section are linked to these limiting factors, except there is no mention of monitoring of water temperature, which might be affected by the habitat actions.

There are almost no details on monitoring, including what is being monitored, why it was selected, and how long it will be monitored. Is another project doing this monitoring? Some project-scale fish monitoring is conducted, but is there basin-scale or larger-scale monitoring that occurs? The YBHP proposal notes that it uses information gathered from past project monitoring to inform designs of future projects. This implies that monitoring is being conducted. The project then notes that the effects of management actions (restoration/enhancement projects) are monitored and evaluated at various intensities (qualitative vs quantitative approaches), and successive projects, phases, or project types are modified in response to these findings. What are the findings being referred to, and is there a database where this information is housed?

The adaptive management process followed was proposed/published in 1996. Is it still viable or is their need to modify what was published nearly 25 years ago?

Q4: Results – benefits to fish and wildlife

The proposal and accompanying documents summarize habitat project results. Since 1997, 1,876 acres of aquatic and floodplain habitat has been purchased and protected. Since the last ISRP geographic review in 2013, two (2) properties (67 acres) have been acquired. Habitat restoration rather than purchase of land to protect habitat has been primary focus in recent years.

The project’s focus is on physical aspects of restoration and uses a suite of methods that are generally considered helpful for restoring fish and in streams and wildlife in riparian corridors. However, it is not clear by the information provided by the proponent in the proposal if or how the project attends to biological monitoring at the site level. Furthermore, it is not clear how the project relates and reacts to existing monitoring activities being done by other entities of the YKFP project in terms of assessment of success and change in restoration planning. The proposal briefly references a number of other projects in the Yakima Basin, including projects such as Yakima Basin Steelhead VSP monitoring (201003000) and the Yakima Monitoring and Evaluation Project (199506325). The proposal also notes an action effectiveness study that examined the response of juvenile salmon to large wood placement. However, the proposal did not clearly demonstrate a close working relationship with projects that are attempting to monitor trends in salmon and steelhead including the response to ongoing habitat restoration.

It would be very informative if the proponents could provide the estimated percent contribution that past and proposed project activities have and will likely contribute to the overall goal of the YKFP (“to restore sustainable and harvestable populations of salmon, steelhead and other at-risk species that were historically present in the Yakima subbasin”) and YBHP (“to restore watershed health and stream habitat to aid recovery of native salmonids in the Yakima River Basin”)?

Like most restoration projects, determining if and how it benefits fish is challenging. While showing fish use of a project is more straightforward, linking projects to a change in juvenile or adult numbers is a challenge. Overall, improving habitat is clearly important, so projects such as this are worthwhile to do.

In the summer of 2020, FEMA Region X rescinded the Policy on Fish Enhancement Structures in the Floodway. This action complicates the habitat restoration actions, which often raise the flood elevations to reconnect the floodplain. The proponent predicts that this conflict between restoration and flood prevention within the floodplain will likely limit benefits and increase costs of restoration.

The ISRP appreciated the detailed response to concerns raised during the review. The response was helpful in clarifying some items. As with other YN projects, links to monitoring need to be strengthened.

The main thrust of this project is land acquisition and protection, and the proposal includes a table listing some 20 properties acquired since 1998. Another table identifies several dozen offreservation fish habitat restoration projects, listing fish limiting factors addressed and project objectives. Biological results were not indicated. The statement is made that the YKFP Ecological Interaction Team (WDFW Ellensburg) is continuing to collect data that may prove definitively that the work is meaningful.

The ISRP requests that the sponsors provide more details about the techniques used for producing the deliverables identified in the proposal. These details should be written into the proposal.

After the site visits and presentation by sponsors, the ISRP is supportive of the overall program, but the proposal does not present enough details. The ISRP suggests a better development of monitoring response before and after projects is needed. The ISRP would like the sponsors to identify monitoring data for the whole basin and provide those links in the proposal as a minimum for providing monitoring data.

The ISRP also requests that the sponsors provide maps showing location of sites selected to produce deliverables, especially for lands sought for purchase. For instance, the sponsors need to justify that these land purchases are important given direction from the Steelhead Recovery Plan.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The introduction should be revised to provide a more clear description of the project. Maps and descriptions of the location and setting are needed. For example, "In addition, the greater habitat picture in the Yakima Basin (formerly part of Project 198812025) necessarily includes all land and water affecting fish, thus land owned by non-willing sellers must be considered in the spectrum of habitat work, along with fish-bearing water regardless of adjacent ownership." What is the meaning of this statement? Can land be acquired from non-willing sellers?

Another example where the meaning was not clear to reviewers is the following, "All these activities are supported by the Yakima-Klickitat Fisheries Program, thus supported by BPA, thus supported by the Yakima Subbasin Plan, etc."

The proposal identified 2 objectives: Prevent further floodplain development; facilitate natural processes that remain (OBJ-1) ("Purchase of floodplain allows natural floodplain processes to continue while curbing development that reduces floodplain complexity en route to near-term profit") and: Improve tributary rearing and spawning habitat (OBJ-2).

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

Accomplishments are listed in an impressive success table that includes acres protected. The ISRP would like an additional table that provides an estimate of what proportion of the total effort needed that these accomplishments constitute. That is, are we 1% done, 10% done, or some other percentage?

The list of projects off-reservation is useful. One project in particular requires further explanation. The objective for the Holmes Habitat Restoration action is so inclusive that specific restoration actions are unclear. More details are needed, especially with reference to "Fix the world."

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

The identification of emerging limiting factors is not adequate. The proposal mentions several existing and long standing limiting factors but consideration of how the project could respond to factors such as climate change, toxics and contaminants, and increasing pressure from invasive species would be very useful.

4. Deliverables, Work Elements, Metrics, and Methods

The deliverables are clearly associated with objectives. The description of metrics used is not adequate to relate project deliverables with a positive influence on fish. The ISRP requests that the sponsors further explain how project actions will lead to increased fish abundance, productivity, diversity, or spatial distribution.