The ISRP requests the proponents provide information on the following condition in the next annual report and future workplans. Because the proposal is important as a guiding document for the project, we encourage the proponents to revise their proposal to reflect these additions.

• SMART objectives. The proponents should address the ISRP’s questions and recommendations regarding SMART objectives in a revised proposal and present those objectives in the next annual report. The proponents should explain how they will assess their project’s performance based on regional targets from the 2021 Columbia River System Biological Opinion (CRS BiOp). Briefly describe how the proponents develop the objective for number of projects to be facilitated and implemented by the Upper Columbia Programmatic Habitat project (UCPH) each year.

The ISRP has provided additional information on the summary of monitoring and evaluation for geographic areas in the Programmatic Comments of this report. The Council and Council staff have stated their support for developing summaries and matrices of the types and locations of monitoring efforts across projects in major geographic areas. The ISRP anticipates that the NPCC Fish and Wildlife Program will identify the specific elements and formats for these RM&E summaries and matrices in the future. Though it is not a condition, the ISRP encourages the Upper Columbia Programmatic Habitat project to use its regional expertise and resources to help create an effective summary of RM&E efforts in the Upper Columbia River.

In our preliminary review, we requested responses on two topics, and our assessments of the proponents’ responses are provided below:

1. SMART Objectives. The revised proposal submitted by the UCPH modified the original Table 2 that identifies targets for flow protected, flow enhanced, entrainment (numbers of screens), improved habitat access, increased stream complexity, and restored riparian and floodplain habitat for 2021-2036 based on the targets established in the 2021 CRS BiOp. Table 3 repeats those BiOp targets and divides them by 3 to account for the 5-yr duration of the proposal. As such, they are not implementation metrics for the UCPH, but rather are metrics for all associated management and recovery entities in the region.

The proponents do not explain how they will use these targets to assess their progress in achieving their first objective, which is related to the CRS BiOp metrics. The objective simply calls for the project to identify and select robust proposals to produce metrics and help achieve targets. The proposal does not describe the analytical approach it will use to evaluate this objective. Is it based on the proportion of the BiOp target that the funded projects achieve for the period from 2021-2026? The targets are for all entities in the Upper Columbia region and not just the UCPH. As such, the funded projects likely will not achieve the full BiOp target, and the proponents will have no specific metric to evaluate the performance of the UCPH itself.

The ISRP recognizes the important regional coordination function of the UCPH, and the Upper Columbia River Salmon Recovery Board (UCSRB) notes on its website that their adaptive approach to implementation provides information on “how effective the actions are, … how the fish are responding, … and how close we are getting to our goals”. The ISAB (ISAB 2018-1) praised the UCSRB for the their quantitative evaluation of the trajectories of their restoration actions in their integrated recovery program reports and for the development of the Biological Strategy with the Regional Technical Team (updated in 2021). For example, SMART objectives for specific restoration actions for the next funding period could be based on updated trajectories and effectiveness of habitat actions identified in the Integrated Recovery Program Habitat Report (UCSRB 2014) for the period from 1996 to 2012.

Figure 3.3 in ISAB 2018-1. Assessment of the quantity (miles, acres, numbers) of recovery actions in the UCR between 1996-2012 by year (red) and cumulatively (blue) (UCSRB 2014b).

An example of a SMART objective based on this information would be:

“Increase off-channel stream length by 6 miles from 2022 to 2026, which would restore XX% of the potential off-channel streams that have been identified for the subbasin and increase the miles of off-channel stream reconnected by the UCSRB in the Upper Columbia River by XX%.”

The database of past restoration actions and current landscape conditions would provide the quantitative information necessary to develop such objectives for the USCRB restoration efforts for the next funding period.

The second and third objectives are quantifiable and can be used to evaluate the project’s progress. The second objective is clearly related to the Biological Strategy. The proposal does not explain how it determined the metric for the third objective—facilitate implementation of two projects annually. Is this based on past performance? What is the relevance of the number of projects as opposed to the intended ecological outcomes?

The fourth objective—manage all UC Programmatic contracts within a budget of $2,000,000 per federal fiscal year—is unnecessary because BPA contractually requires all projects to operate within budget and the objective does not address the ecological recovery focus of the four goals. In relation to this objective, the ISRP found the budget narrative and the budget table to be confusing. The budget table reported only the portion of the annual budget that is used for project administration (roughly $120,000/yr), but the proposal is for the entire $2,000,000 project funded by BPA. The discussion of the budget should be revised to address the allocation and management of the entire amount of funding for the project.

2. M&E Matrix – lead. The UCSRB declined to provide a matrix summary of M&E projects in the Upper Columbia River subbasin. They indicated that preparation of the M&E summary is outside the UCSRB’s scope of work for the UCPH. They said they would assist others, such as NOAA Fisheries or WDFW, but would not lead the development of the matrix. The ISRP reviewed the M&E components of the different projects in the Upper Columbia River subbasin based solely on the information provided in their original proposals, associated documents, and any information provided as part of the Response Loop. Two of the four goals in the proponents’ revised proposal are designed to “1) coordinate with the RTT to maintain alignment among the UC Programmatic and the science underpinning the regional Recovery Plan and Biological Strategy and 2) target Programmatic habitat improvement actions in priority locations that are anticipated to provide the greatest biological benefits to focal species, based on regional Prioritization.” Information contained in the collective data of the RTT, existing prioritization of restoration actions, and the Regional Strategy would provide a more comprehensive foundation for a summary of M&E and habitat improvement actions than is available in many other subbasins.

As the Fish and Wildlife Program develops efforts to identify monitoring activities and coordination between projects in major subbasins, the ISRP encourages this project to contribute its expertise and resources to help create an effective summary of RM&E efforts in the Upper Columbia River. The ISRP has provided additional information on the summary of monitoring and evaluation for geographic areas in the Programmatic Comments of this report. In addition to providing important information for the NPCC Fish and Wildlife Program, development of an overall summary of the M&E efforts and coordination in the Upper Columbia River subbasin would strengthen the restoration efforts of the UCSRB.

Preliminary ISRP report comments: response requested (Provided for context. The proponents responded to the ISRP’s questions; see response link and final review above.)

Response request comment:

Management of spring Chinook salmon in the upper Columbia River was reviewed by the ISAB and found to be scientifically rigorous and an example for other projects throughout the Columbia River Basin (ISAB 2018-1). The Upper Columbia Programmatic Habitat Project has been reviewed extensively over the last 11 years (2010-12, 2010-28, 2013-11, 2014-5, ISRP 2014-10, ISAB/ISRP CU 2016-1, ISRP 2017-2, ISAB 2018-1). The ISRP and ISAB have found that the project has developed one of the more rigorous recovery strategies and prioritization processes for habitat protection and restoration in the region. In conjunction with the Tribes of the upper Columbia River basin, the project coordinates implementation of the Upper Columbia Spring Chinook Salmon and Steelhead Recovery Plan (UCSRB 2007).

The project uses the Upper Columbia Recovery Plan as a framework for identifying limiting factors, prioritizing restoration actions, and working with collaborators to implement restoration actions. The project also uses the updated Biological Strategy developed by the UCR Regional Technical Team (RTT) and collaborates with the RTT to incorporate up-to-date information. The project monitors only the initial implementation of projects but does not conduct post-implementation compliance or effectiveness monitoring. The project states that BPA does not provide funds for monitoring through this project.

While the project is exemplary in many respects, it does not provide SMART objectives for their major intended ecological outcomes or specific projects. Instead, the proposal identifies three general programmatic management objectives to use priority assessments, address priority limiting factors, and generate BiOp habitat mitigation credits for BPA in each project. The previous ISRP review (ISRP 2017-2) listed a qualification for the project to “develop measurable objectives in advance of the next annual review to evaluate progress towards addressing limiting factors for fish and wildlife from restoration actions. These objectives should support the umbrella projects’ implementation strategy used to identify and select projects.” The programmatic statements in the proposal for this review do not adequately address this qualification of the 2017 review.

The ISRP requests the proponents to address the following in a point-by-point response to assist our review of the proposal:

1. SMART objectives. The ISRP requests the project to provide measurable objectives to evaluate progress towards addressing limiting factors for fish and wildlife from restoration actions. These objectives would support the umbrella projects’ implementation strategy used to identify and select projects. The proposal indicates that “the UCSRB and BPA are collaborating to refine and develop additional umbrella goals and SMART(er) objectives to improve evaluation and implementation reporting” and acknowledges SMART objectives will focus resources to support recovery needs. The example the proponents provided in the proposal could serve as a template for the SMART objectives (see proposal instructions).

2. M&E matrix - lead. One of the challenges for ISRP reviewers is understanding the specific monitoring that is being conducted for multiple implementation projects. Habitat restoration projects or hatchery projects implement actions that are intended to address limiting factors and benefit fish and wildlife. Most of these projects do not directly monitor habitat conditions or biological outcomes, but most identify other projects in the basin that monitor aspects of physical habitat or focal fish species. The monitoring project(s) in the basin provides essential monitoring data for habitat, juvenile salmonid abundance and distribution, outmigration, survival, and adult returns for salmon and steelhead. Some monitoring projects focus on status and trends in basins, while others focus on habitat relationships and responses to local actions. It is unclear what monitoring the monitoring project(s) conducts for each implementation project. Given the regional leadership responsibilities of this programmatic project, the ISRP is requesting the Upper Columbia Programmatic Habitat Project to summarize the linkages between implementation and monitoring projects in the Upper Columbia subbasins. The summary should provide a table or matrix to identify what is being monitored for each implementation project and where and when the monitoring occurs. The summary also should explain how the projects are working together to evaluate progress toward addressing limiting factors and identify future actions. A map or maps could help identify the locations of monitoring actions. The monitoring information should clearly explain whether the biological monitoring is local information for the specific implementation site or basin scale monitoring of status and trends or fish in/fish out. We are asking implementation projects to assist your project in producing this summary and encourage you to work closely with the implementation and monitoring projects of the Confederated Tribes of the Colville Reservation (Okanogan Subbasin Habitat Implementation Program and Okanogan Basin Monitoring & Evaluation Program) and Yakama Confederated Tribes (Upper Columbia Habitat Restoration Project).

Q1: Clearly defined objectives and outcomes

The previous ISRP review (ISRP 2017-2) listed a qualification for the project to “develop measurable objectives in advance of the next annual review.” The project did not develop specific SMART objectives for projects. Instead, they provide general programmatic management objectives to use priority assessments, address priority limiting factors, and generate BiOp habitat mitigation credit for BPA in each project. The proposal indicates that “the UCSRB and BPA are collaborating to refine and develop additional umbrella goals and SMART(er) objectives to improve evaluation and implementation reporting” and gave an example. The project should submit specific physical and biological outcomes for the ongoing and anticipated projects and to develop SMART objectives for as many as possible given the available information and specific project details.

The proposal identifies nine ongoing projects but specifically identified outcomes for physical habitat for none of them (Table 1). SMART objectives are not included for these projects. The Timeline section lists the nine ongoing projects and then includes only eight in Table 3. The specific projects to be implemented by FY 20225 are not described in detail. The proposal states that the UC Recovery Plan provides thorough documentation of the goals and objectives as well as specific metrics for habitat, species linked to quantified limiting factors (www.ucsrb.org/science-resources/reports-plans/recovery-plan/), but the document was produced in 2007. The Recovery Plan website included a document with the 2017 updated implementation schedule, but it did not include any actions beyond 2020.

Several important factors and major uncertainties could affect the success of restoration projects implemented under the Upper Columbia Programmatic Habitat Project. These include climate change, ocean conditions, and predation in addition to known limiting factors.

Q2: Methods (based on sound science principles)

The project uses the Upper Columbia Recovery Plan as a framework for identifying limiting factors, prioritizing restoration actions, and working with collaborators to implement restoration actions. The project also uses the updated 2019 Biological Strategy developed by the RTT and collaborates with the RTT to incorporate up to date information. The process developed by this project and the UCSRB is one of the most rigorous restoration prioritization processes in the region. Curiously, the proposal does not highlight the positive review by the ISAB (ISAB 2018-1).

Management of spring Chinook salmon in the upper Columbia was reviewed by the ISAB and found to be scientifically rigorous and an example for other projects throughout the Columbia River Basin (ISAB 2018-1). The project selection process of the project has been refined by the RTT in 2020 and incorporates improvements suggested in the ISAB review of spring Chinook salmon in the upper Columbia River (ISAB 2018-1). Priorities are integrated with regional recovery plans, thoroughly documented, and publicly available through the Prioritization Web Map (https://prioritization.ucsrb.org/).

Q3: Provisions for M&E

The project monitors only the initial implementation of projects but does not conduct post-implementation compliance or effectiveness monitoring. The UCSRB and RTT have developed a rigorous monitoring plan, but the proposal states that BPA does not provide funds for monitoring. The proposal indicates that the PUDs either conduct or fund most of the monitoring in the four basins. These results are evaluated regularly in a series of scheduled technical team and RTT meetings. The project identifies a number of monitoring actions in the four basins but does not describe any specific monitoring that will be conducted. The proposal does not describe how it will coordinate with other monitoring projects, such as the Okanogan Basin Monitoring & Evaluation Program and the Yakama Nation’s Upper Columbia Habitat Restoration Project. In addition, a new pilot BPA project (201700300) is attempting to create an Upper Columbia Habitat Action Effectiveness Monitoring Plan. The project intends to develop goals and objectives for habitat action effectiveness monitoring (AEM), select habitat action sites and available control sites, and select metrics to measure fish responses and test hypotheses. The Upper Columbia Programmatic Habitat Project should coordinate with this project as it develops.

The Upper Columbia Programmatic Habitat Project has an adaptive management process that is based on a formal Recovery Plan, a Biological Strategy developed by the RTT, regional database, annual State of the Science meetings, and consultation with BPA.

Q4: Results – benefits to fish and wildlife

Actions to address limiting factors are tracked and reported by the project, and the project has developed thorough syntheses of the landscape-level outcomes of their previous actions. The project has completed quantitative summaries of their actions for habitat (2014), hatcheries (2017), hydropower (2019), and harvest (2020).

The proposal thoroughly documents the total amount of habitat that the project implementation and its collaboration with other funding partners has restored. They have invested $20 million to restore high priority areas for spring Chinook and steelhead. Eleven of 28 restoration projects have been completed. The project has restored 6 miles of instream habitat, created 2 miles of side channel habitat, restored 105 acres of floodplain and riparian habitat, protected 38 cfs of flow, removed 4 fish barriers and opened 27 miles of habitat to salmon, steelhead, and bull trout, and screened or removed 17 irrigation diversions. As is frequently the case with programmatic habitat projects, the outcomes are quantified in terms of habitat and less information is presented on responses of fish populations. However, the studies and publications of Polivka et al. (2015, 2020) and others, as well as life-cycle and restoration modeling reviewed in the ISAB 2018-1 report, documented several major examples of positive fish responses to restoration. The project’s focus on limiting factors in the highest priority areas and use of the recovery plan and biological strategy as a framework make it a highly likely that these actions are beneficial to fish and wildlife.

 

The UCSRB response to the ISRP’s questions and requests for clarification were reasonably complete in some areas but did not address all of our concerns. In particular, stating that the proposal had been revised without providing full details in the response itself made it difficult for us to determine whether a question had been adequately answered without referring to the complete proposal. The project prioritization process is still unclear. We look forward to learning more about it when the promised check-in report is submitted for ISRP review.

Although we realize that biological effectiveness monitoring cannot be carried out at every restoration site, further explanation is needed on the relationship between this project, which supports on-the-ground habitat improvement actions, and regional population and habitat status and trend monitoring efforts such as ISEMP and CHaMP. The response is somewhat vague on this matter. A project of this scope must be coupled to an equally ambitious monitoring program in order for adequate learning to occur. Therefore, we ask that past and anticipated future monitoring results also be highlighted in the check-in report. This summary should go beyond the organizational flow diagram (Figure 1) and identify key monitoring metrics that will be used to determine if VSP parameters of focal species are improving, how these metrics will be obtained (and by whom), and how will they be interpreted in terms of population status and trends. It is stated that "this project will continue to work with BPA and Council staff to identity whether restoration actions proposed under this project may be candidates for use in the AEM program." We feel that a plan to incorporate restoration actions into the AEM program should be included in the check-in report. The summary of the Twisp Elbow Coulee Project in the response was helpful, and we hope to see more examples using specific references to VSP parameters in the report.

The overall philosophical view that local experts and communities need to be engaged and involved in developing habitat restoration for salmon and steelhead is consistent with the ISAB Landscape Report and other socioeconomic considerations regarding conservation practice. Using the Upper Columbia Salmon Recovery Board as a vehicle to link the Council Fish and Wildlife Program and BPA/NOAA/Reclamation’s non-Accord BiOp obligations is reasonable. The questions for ISRP review is whether sufficient progress in implementation is being made and whether the organization and governance structure is able to achieve the improvements in habitat and salmon survival and abundance in an efficient manner.

At this time it appears the targeted work is somewhat weakly linked to an understanding of habitat forming processes in the subbasins. From the tour, it appeared that knowledge could be better incorporated in the project regarding the status of salmon and steelhead and the improvement needed to achieve (1) viability status goals in the near and medium terms and (2) desired gains in salmon survival sufficient for recovery. It also appears the pace of implementation could be improved. At this time, the UCRSRB has executed only one targeted solicitation, and the RTT did not generate estimates of fish benefits associated with these initial actions.

Connections between the selected actions and habitat forming processes for salmon and steelhead population improvements need to be better reflected in the proposal and reports. In some cases, monitoring and evaluation seems to be underappreciated and misunderstood. The proposal, conversations during the site visit, and the response (both in this programmatic project and other organizations) seem to be focused on monitoring and evaluation of fish use of individual structures constructed by restoration partners – a narrow perspective and not very informative unless these project-scale assessments can be linked to basinwide fish population performance measures. The more relevant questions are: how are these projects affecting subbasin-scale habitat quality and habitat forming processes, and how are these changes in habitat affecting salmon and steelhead viability? These questions will be appropriately addressed by modeling ISEMP/CHaMP habitat and fish-in/fish-out data. Finally, if the projects actually conducting habitat and fish monitoring are not incorporating knowledge of existing restoration sites into their sampling panels and analysis design, the resulting interpretations may be misinformed.

The format in which the project sponsors provided responses to ISRP comments made it difficult to distinguish the material added in the review loop. It did appear that some information was added to address ISRP concerns, but several of the issues we raised were not addressed. The overview of the program provided during the field tour did partially address concerns about relationships among the restoration programs involved in implementation of habitat projects in the Upper Columbia. This program will prioritize and help implement projects funded through this proposal, the Upper Columbia Salmon Recovery Board, the Bureau of Reclamation, and PUDs. These funding sources represent a significant proportion of the habitat restoration funding in this region. There are some other restoration projects in the Upper Columbia region, like OSHIP, that are not discussed in the revised proposal but it would seem some coordination with these programs also would be worthwhile to ensure the most efficient use of restoration resources. The relationships among RM&E programs and how information being generated through these efforts will be used to modify program processes and project prioritization was not fully addressed. Much of the additional information the ISRP requested about the specifics of procedures being used to prioritize projects also was not provided. Nonetheless, the program clearly plays a critical coordinating role for habitat restoration efforts in this region and should be supported. A program status report is proposed for 2013. This report should include a very complete description of program progress since inception; RM&E results and their application to program priorities and processes; and an indication of the extent to which the AEM program will provide RM&E coverage for watersheds without IMW efforts.

Questions for which the ISRP requested a response included:

1) Describe the restoration project review and prioritization process in more detail including the scoring sheet and criteria.

Some additional information on the prioritization process appears to have been added to version of the proposal submitted in response to the ISRP review. Discussions during the field tour also helped to clarify this issue. However, details of the scoring system were only included as a partially complete Excel spreadsheet (in the project description) for one of the restoration areas, and we were unable to conduct a thorough review of the technical merits of the prioritization system being used. The forthcoming check-in report should remedy this.

2) Describe the connection between the implementation project and the RM&E project in more detail and explain how monitoring results are incorporated into restoration decisions.

Description of the RM&E process being used to evaluate projects implemented under this program was expanded somewhat in the response. IMW efforts are being used to evaluate program effectiveness in the Entiat and Methow, and it is anticipated that the NPCC/BPA action effectiveness monitoring (AEM) program will assess a subset of the projects in the non-IMW watersheds. The extent to which the AEM program meets RM&E needs for this program should be carefully assessed as AEM is implemented.

3) Explain the relationships between this project and the other RM&E efforts ongoing in the basin.

The relationships between the IMW programs (Entiat and Methow) and the anticipated coverage provided by the AEM program were discussed briefly. As noted above, the extent to which AEM will meet the RM&E needs of this program remains to be seen. In addition, it was indicated that the Methow IMW effort, supported by the Bureau of Reclamation, may not be well coordinated with the RM&E efforts under the Council’s Program. In addition, no mention of OBMEP or RM&E activities in the Wenatchee was included in the response. This program would be a logical choice as the sponsor of a more formal process to ensure that coordination among all these RM&E projects occurs.

4) Describe how projects selected for funding address some parts of the VSP criteria for viability.

There was very limited discussion of the expected effect of program projects on VSP parameters. Given the availability of IMW information for two of the watersheds in the program area, some estimate of the response of the fish to the implemented and planned projects should be possible.

5) Describe how conflicts of interest are avoided during the RTT review process.

The response to the question about avoiding conflicts of interest in the project selection process was adequate.

Evaluation of Results

This large umbrella-type project has had a lengthy birth and so far only one entirely new restoration solicitation has been implemented. Thus, it is premature to judge whether the Upper Columbia Programmatic Habitat Project is meeting expectations. The project deserves an in-depth ISRP review that should focus on inter-organizational relationships, the habitat restoration prioritization process, and most importantly, a thorough look at the monitoring programs that are being carried out by other organizations, and how population and habitat data will be used to determine the effects of this project on VSP parameters of focal species. Completion of the check-in report in late 2013 or early 2014, coupled with possibly an on-site ISRP visit with project staff, should help ensure that satisfactory progress is being made.

This project covers a very large area of the Upper Columbia Basin and includes an array of restoration activities that have been reviewed and prioritized by a team of regional experts. The administrative structure differs from the traditional BPA funding approach in which the project sponsor is usually a single organization, but instead this project employs a group of partner organizations. Because of this structure the ISRP has had questions about whether the new approach would deliver the administrative efficiencies promised and whether it would lead to better planned and more effective habitat restoration actions in the Upper Columbia. The qualification to this proposal is that project staff should submit a comprehensive report summarizing their progress to date, including areas where they have experienced difficulties and areas where they have clearly achieved their objectives. This check-in report should describe the cooperative activities taking place between the project and other regional restoration efforts such as OSHIP, which was not clearly identified in the proposal. The report should be completed for ISRP review in late 2013 or early 2014. In addition, the ISRP may request a follow-up site visit to better understand the project selection process, the monitoring program, and how results will be used adaptively to plan and prioritize future projects.

This project has made progress in the short time since its inception, but additional information is needed for the ISRP to complete an assessment of its scientific soundness. The proposal lacked sufficient detail in some key places for an evaluation, and so we are requesting responses to the following queries that are listed below. In the presentation and site visits, many of the ISRP's questions were partially clarified, but the proposal needs to be revised to include, document, and update this information. We appreciate the forwarding of the Parrish and Jenkins report on log jams in the White River subsequent to the site visits and presentation, and we are looking forward to receiving additional information relative to the following suggestions and questions. The proposal has distinct administrative and scientific/implementation components, and both require further elaboration.

The restoration project review and prioritization process needs to be more fully described. The scoring sheet and criteria should be provided. The ISRP does not need reviewer names or the project name, but an example of how the selection process takes place is needed.

The linkage between this implementation project and the RME project also needs to be more explicitly described. The adaptive management loop concerning how monitoring results are incorporated into restoration decisions was not sufficiently transparent. Please see our comments on the M&E component of this project in our RME and AP review 2010-44b www.nwcouncil.org/media/33226/isrp2010_44b.pdf.

The relationships between this project and the various RM&E efforts ongoing in the basin should be described. How will project effectiveness monitoring be conducted besides the ISEMP effort on the Entiat? How are projects supported by this project integrated with the large habitat and fish monitoring programs such as ISEMP and CHaMP that have been established in the region?

Some of the ISRP’s questions can be addressed during the response loop, but a thorough examination of the project is beyond the scope of proposal evaluation. The ISRP originally suggested a check-in report in 2013, which will give us an opportunity to review the project in greater detail.

Questions for which the ISRP requests a response include:

1) Describe the restoration project review and prioritization process in more detail including the scoring sheet and criteria.

2) Describe the connection between the implementation project and the RME project in more detail and explain how monitoring results are incorporated into restoration decisions.

3) Explain the relationships between this project and the other RM&E efforts ongoing in the basin.

4) Describe how projects selected for funding address some parts of the VSP criteria for viability.

5) Describe how conflicts of interest are avoided during the RTT review process.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The purpose of the UC Programmatic Habitat Project is to implement habitat restoration projects in high biological priority areas in the Upper Columbia Region (Columbia Cascade Province). It thereby helps satisfy the Action Agencies' obligations under the 2008 FCRPS Biological Opinion, while complementing and benefitting from the Upper Columbia Spring Chinook Salmon and Steelhead Recovery Plan and its recovery infrastructure. This project is still new, having been initiated in 2010 to replace 14 separate BiOp-serving habitat projects from the FY07-09 Fish and Wildlife Program solicitation. Annual budgets are approximately $3.7M, equal to the total FY09 budgets of the 14 projects being replaced. The project is central to the regional program.

This project is relatively new but has been thoroughly reviewed by the ISRP in the past. It is one of the first attempts in the upper Columbia River to create an umbrella project that oversees a suite of habitat restoration actions instead of the more piecemeal approach of submitting a collection of standalone projects. When first reviewed, the ISRP was cautious about endorsing this approach until the technical procedures for prioritizing and selecting individual restoration projects, as well coordinating with ongoing monitoring projects, were sufficiently explained. The Columbia Cascade region holds several fish species that are ESA-listed including spring Chinook, steelhead, and bull trout, although this project is clearly focused on improving habitat for Chinook and steelhead.

There are two levels of technical background to consider: administrative and on-the-ground applications. Since the details of specific on-the-ground actions were understandably general, we assume that the expertise needed to carry out the restoration actions is adequate. As for the administrative technical background, we are somewhat concerned by the number and type of issues have been encountered during the first year of the new programmatic format. Many of the specific problems encountered, for example see section on Adaptive Management, might have been avoided if project staff employed general administrative principles that have emerged from managing and implementing many other large and similar complex restoration projects.

This project is consistent with the objectives articulated in various habitat restoration and salmon recovery plans that have been developed for the Upper Columbia region. This project reviews, selects, and supports projects funded with non-accord BPA project funding. The objectives provided in the proposal are relatively generic; the goal of this effort is to improve habitat sufficiently to meet VSP recovery goals for steelhead and spring Chinook. These are appropriate objectives for this project. The technical background for this project was generally sufficient. A thorough description of the project selection process was provided. One element of the background that was deficient was the description of the relationships among the various habitat restoration and RM&E programs in this region. Habitat projects in this region are supported through this project, Accord funding, Bureau of Reclamation funding, PUD funding and SRFB funding. The extent to which these various habitat programs are coordinated was not clear.

A repeated concern was expressed by project sponsors about two aspects of BPA contract requirements: 1) inability to carry unspent funds into the next year, and 2) change from two-year contracts to one-year contracts. The sponsors claim that these restrictions reduce flexibility to the point where some important habitat projects may not be completed due to unforeseen delays. Are there any alternatives other than changes in BPA's contracting policies that would help to address these problems?

With regard to the technical background, the biological objective is improvement in VSP parameters in specific salmon species in particular watersheds through habitat restoration. The Recovery Plan and the BiOp RPA 35 call for specific improvements in the state of both VSP and habitat required during specific time periods. These need to be included in the proposal. Then the technical background should provide sufficient information so the ISRP can conclude that the project has a reasonable likelihood of completing the actions (assessments, planning, project selection, project implementation) within the time frame required in the BiOp. The accomplishments section of the proposal identifies an administration function that updates MYAPs, develops a targeted solicitation that is funded based on fulfilling the Recovery Plan and BiOp mandates. The time frame for bringing the process to full functioning and the relationship of the many parts of the process should be succinctly outlined in this proposal section. The work elements portion of the proposal summarizes the activities involved in administrative, targeted, and open solicitations. Other sections of the proposal suggest that it takes 3 to 4 years to bring a project to fruition. If this time frame is from project submission to final implementation, only activities from the first solicitation will be completed by 2018 when this project sunsets.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

Because the project has been in place for only about two years the accomplishments to date have been to undertake the first round of restoration action selection and implement some of their prioritized habitat improvements. Results have therefore been limited to demonstrating that the technical review process can function as anticipated and that at least some of the actions can be implemented. As stated in the proposal, one major hurdle was an unusually high flow event that took place in 2011 that delayed the implementation of certain actions by a year. Fortunately, BPA funds were able to be carried over into the next year so the improvements could be initiated; however, the new administrative rules require annual funding with no carryover and the project now faces the difficult task of balancing investments in new restoration with maintaining a contingency fund in the event that unforeseen circumstances delay or prevent implementation. In general, the project sponsors have done a good job of planning for the unanticipated.

The proposal made clear that the long-term goal was to fund targeted restoration activities that involved substantial planning and resources, and gradually reduce the investment in the smaller budget projects submitted through the Open Solicitation process. While this is understandable, we suggest that some project funds be reserved so that there will be room for new restoration ideas.

A previous review of this project in 2010 (ISRP 2010-28) asked that the project sponsors prepare retrospective reports for ISRP review in years 3 and 6 of this 7-year project in order to verify that assumptions about administrative streamlining, project selection efficiency, and action effectiveness are proceeding as anticipated. Specifically, the ISRP asked that the retrospective summary report in year 3 address actions outlined in Figure F-1 (page 52): Watershed Action Team(s) developing Multiyear Action Plans with the Regional Technical Team and Implementation Team subsequently developing targeted solicitations. As well, the retrospective report in year 6 should summarize the implementation of restoration activities following the targeted solicitation, and update the ISRP on monitoring and effectiveness evaluation of restoration actions. Given the dependence on other RM&E efforts to evaluate the effectiveness of this process, these retrospective reports the ISRP requested will allow the sponsors to summarize results from research efforts in the project area that are relevant to project restoration plans, as well as indicate how these results have been incorporated into the project prioritization process. Retrospective reports should clearly present the processes for Open and Targeted Solicitations, the updating of MYAPs, linkages between the WATs, RTT, and project development, and benefits accorded to BiOp and Recovery Plan obligations.

Since this project is now assisting in coordinating restoration actions in the Entiat subbasin, which comprise an IMW, some discussion of the current state of IMW evaluation and what is being learned should be included in the results section of the proposal. The ISRP would anticipate that the RTT is using monitoring information from the Entiat IMW to guide development of other targeted projects in the Entiat and other subbasins.

Evaluation of Results

Rather than provide an evaluation of results specific to this project, the ISRP chooses to identify what these umbrella type projects should be doing in the basin. The Upper Columbia Programmatic Habitat Project constitutes one of a handful of umbrella habitat projects in the Columbia River Basin that aim to adopt a landscape approach to restoration actions, that is addressing the most important restoration priorities in the right place and in the right order over a large area. Such efforts require thoughtful and comprehensive coordination and a willingness to learn from existing monitoring programs. These projects need to be documented with periodic retrospective reports that synthesize the science involved and chronicle whether desired outcomes of this approach are being achieved.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

The relationships between this project and the other habitat efforts in the project area are described briefly, but a more thorough discussion of the interaction among these efforts would have been beneficial in reviewing the proposal.

ISEMP, CHaMP, and other assessment efforts are occurring in this region. ISEMP is focused on assessing the effectiveness of a suite of restoration projects being implemented on the Entiat River. It is not clear to what extent the monitoring programs are assessing the effectiveness of other program projects. The description of the adaptive management approach indicates that project-scale effectiveness evaluations are occurring but who is doing these assessments and how they are integrated with the ISEMP, CHaMP and other RM&E efforts in this region was not explained. Relationships among organizations performing RM&E in the region also should be better described.

Some emerging limiting factors were addressed in the proposal. A climate change assessment has been completed for the Methow and this information will be used by the UCRTT in proposal evaluations. Presumably, the Methow assessment contains information that also will shed some light on the threats from climate change to other watersheds in the project area. The proposal does not mention impacts from chemical pollution, non-native species or increasing human population and consequent development. These factors also should be considered by the UCRT when evaluating project proposals. As the project moves forward it will be important to be alert to such changes, and this is one reason why reserving some funds for the open solicitation may be prudent by facilitating novel restoration actions that address emerging problems. There are approaches in use to gain insights into future streamflows, and these insights can help shape restoration strategies and actions. These include scenario analyses to inform and improve existing flow restoration and habitat projects (see Donley et al 2012. Global Change Biology (2012), doi: 10.1111/j.1365-2486.2012.02773.x). As one example, it is ecologically important to assess, in simulated scenarios, the sensitivity of late summer (July, August, and September) flows to the following variables, both singly and in combination: climate change, changes in the quantity of water used for irrigation, and possible changes to existing water resource policy. Flows can be modeled using the Water Evaluation and Planning system (WEAP; as well as other modeling platforms) under historical and projected conditions (for example, 2020 and 2040) for each scenario.

A question of contracting was raised in the proposal that could benefit from discussion between the Council, BPA, and the sponsor. More than once in the proposal it was stated that “BPA no longer allows two important types of flexibility that would allow us to respond to the inevitable risks of implementing habitat projects and to reuse funds. The first is the ability to carry forward funds that are not obligated to contracts within the budget FY. In good years, unobligated funds will be only the small % retained for contingency. However, if a big project falls through at the last minute (e.g., failure to reach agreement with an infrastructure owner or another funding source stepping in to pay for construction) then we must scramble to find good alternative work to fund.”

Additionally, the ISRP has other specific concerns:

1) How will colonization by beaver and herbivory by native ungulates be addressed? Is there an overarching beaver and ungulate management philosophy for all the projects?

2) Should mussels be included under focal species?

4. Deliverables, Work Elements, Metrics, and Methods

The deliverables are described in considerable detail and are generally appropriate for the objectives of the project. The descriptions for several of the deliverables include multiple, undefined acronyms (AC, ED, PM, AD, WE). It is unclear who these individuals or organizations are, making it difficult to understand how these deliverables are to be accomplished.

In the type of work section, the explanation of the nearly completed targeted solicitation is provided. Much of this text belongs in the results and accomplishments section. It describes the process used to develop the targeted projects, and the status of the first full solicitation under 2010-001-00.

The project relationships section of the proposal states:

"The Upper Columbia Region uses a reach-based action approach to ensure priority habitat projects are implemented with a clear understanding of the existing physical processes. This reach-based approach to project development incorporates information from Tributary Assessments (TA) and Reach Assessments (RA) completed by the Bureau of Reclamation, Yakama Nation, and Colville Confederated Tribes, which ensures restoration actions are based on a sound scientific assessment of channel processes. As reach-level degradations and processes are defined, alternatives development occurs in order to identify, sequence, and prioritize specific actions to restore channel and floodplain connectivity and complexity. In concert with this reach-based approach, the Entiat and Methow subbasins are implementing an IMW approach (ISEMP and Bureau of Reclamation/USGS, respectively), which pairs reach-based actions with Level 3 effectiveness monitoring in order to assess the effectiveness of actions implemented within an experimental framework. The Upper Columbia Region is progressing from a reach-based approach to a landscape-level approach to recovery."

The ISRP needs information sufficient to conclude that the assessments are using appropriate methods, that channel processes and extent of degradation are reasonably interpreted from the assessments, and that restoration alternatives are consistent with best practices.

The collection of restoration actions endorsed by the Upper Columbia Regional Technical Team is typical of those being implemented elsewhere in the interior Columbia River. Emphasis is on improving habitat complexity, fixing faulty fish screens, re-establishing floodplain connections, and adding structure (large wood) to stream channels. The entries in the objectives and project deliverables section of the proposal used the same boilerplate response to each objective; however, a description of how the selected restoration actions address the objectives would have helped. The ISRP does not need an exhaustive description of how anticipated outcomes improve VSP parameters for every action, but a few examples would have been instructive.

Additionally, the proposal did not discuss a concern raised by the ISRP in an earlier review and that is how to avoid the potential for conflict of interest during the review process. We would like some assurance that members of the review team do not play a role in ranking proposals in which they have a vested interest.

The deliverables are sometimes not well related to individual objectives. As written, the deliverables are so vague that they cannot be linked to any one objective. Importantly, the deliverables need to be quantitative so that they can be accurately evaluated for success (or not) at the next program review. Further, the deliverables are mostly about using appropriated funds; they should really address quantitative fish recovery targets, habitat goals, and environmental protections – with timelines for being attained. Otherwise, how will one know if the actions are having their intended effects?

A professional publication, or two, in a refereed journal should be listed as a deliverable. It is important for large scale projects, like this one, to provide leadership in the broader restoration community. For example, the Parrish and Jenkins report that was forwarded to us was well written and represents a potential publication of general interest.

CVs, of a reasonable length, should be provided for key personnel.

Specific comments on protocols and methods described in MonitoringMethods.org

This project does not involve monitoring but claims to be well integrated into existing monitoring programs. In the Entiat River, and likely the Wenatchee River, this appears to be the case. The specific connection between this project and monitoring efforts in the Methow and Okanagan Rivers was a little less clear and deserves additional description.

In general, a description of which monitoring programs are associated with the selected projects would be helpful. The description should include what will be monitored (for example, CHaMP habitat parameters) and how monitoring results will be used to evaluate whether actions met expectations.

Modified by Dal Marsters on 9/26/2013 3:38:50 PM.