The ISRP recommends that the following conditions be addressed in future work plans and annual reports:

1. SMART Objectives. Present all objectives in a SMART format. Comments are provided to aid in this step for three objectives below. Additionally, the basis for addressing each specific goal and objective should be described, including an explicit emphasis on how addressing goals and objectives will lead to increases in survival, abundance, and condition of fish.
2. Ecological benefits of proposed projects. Include expected ecological benefits for each proposed project. These outcomes should ultimately be linked with SMART objectives. This will be especially important given the fact that the budget will increase substantially over the next 5 years.
3. Benefits of completed projects. Use biological data collected by other WDFW programs and Federal agencies or analyses conducted by others to provide evidence of the effectiveness of this project based on benefits to fish and wildlife.

In our preliminary review, we requested a response on the eight topics listed below. Our final comments based on the response are provided after each topic:

1. SMART Objectives. The ISRP appreciates the effort by the proponents to reframe the project goal and objectives. The revised goal emphasizes the protection of habitat for “listed salmonids,” while the biological emphasis in the previous goal was on “aquatic organisms.” Additionally, the objectives are more clearly organized and are closer to being framed in a SMART format than the objectives presented in the original proposal. A few questions remain, however, on several specific objectives. 

Objective 1: Protect and maintain 1,000 acres of public estuary habitat to support no net loss of native habitats in the Lower Columbia River estuary. 

What is the time frame for this objective? How does it relate to the emphasis that, “WDFW protects the existing functional estuarine habitat owned by the agency within the Lower Columbia River estuary”? What is the specific ecological benefit of achieving this objective? This objective would be more effective if the proponents could do more than simply maintain the habitats they acquire. Can actions be taken to improve the habitats? For example, could invasive plant species be removed for conservation purposes?

Objective 2: Restore ecological function to 773 acres of tidal wetland habitat by 2027 to restore access to, and increase capacity of, estuarine habitats used by juvenile salmonids and other species.

The emphasis on the relationship to CEERP objectives is helpful, but a specific statement on benefits should be included in the proposal. Additionally, projects that will contribute to Objective 2 should be listed, and proponents should indicate when they will be initiated.

Objective 3. Engage with 500 Washingtonians regarding the potential tidal reconnection of 687 acres of public tidal wetland habitat by 2027.

The emphasis on outreach is an important part of this project. While engaging 500 individuals will be useful, how will that lead to more trust? This objective could be strengthened by providing context on the mechanisms by which these individuals will be engaged, as well as the organizations they represent.

As one additional comment to consider, the proponents emphasize that other programs monitor fish presence at restored and reference sites. Those data should be evaluated so that biological outcomes can be clearly identified in the objectives. Data and analyses of benefits for fish and wildlife and available summaries from other programs should be provided in the next annual report.

2. M&E matrix – support. The ISRP appreciates that the proponents contributed information for the M&E Matrix. The Council and Council staff have indicated that developing summaries and matrices of the types and locations of monitoring efforts across projects in major geographic areas would provide important information. The ISRP has provided additional information on the summary of monitoring and evaluation for geographic areas in the Programmatic Comments of this report. The Fish and Wildlife Program may identify the specific elements and formats for these RM&E summaries and matrices in the near future. The information and expertise of this project would strengthen future coordinated M&E summaries for geographic areas.

3. List of projects. The addition of the number and acres restored along with the start and end date for each project provides good context, but it would be more useful to state the expected ecological benefits of each project listed as well.

4. Scoring and evaluation processes. The ISRP appreciates the thorough description of the scoring and evaluation process and references for key documents. The explanation of the changes in the scoring and evaluation process in 2020 clarified the process for the ISRP and strengthened the revised proposal.

5. Project selection. The information provided on how projects are selected is helpful.

6. Project evaluation and adjustment. Again, this is good context. Are other projects on properties owned by WDFW in the Lower Columbia River incorporated in the project’s adaptive management? If so, this should be added to Table 1 for more clarity.

7. Confounding factors. The additions made to provide information on efforts to mitigate confounding factors are helpful.

8. Benefits to fish and wildlife. The proponents emphasize that data on growth, survivorship, condition, and other metrics for juvenile salmon are not collected at project sites, but this information is collected by other WDFW programs and Federal agencies and disseminated through CEERP. Those data should be used to partially address the recommendation of the ISRP to document the benefits to juvenile salmonids. Additionally, data may be available on other focal species (i.e., birds) that could be used in a similar manner. Incorporating such biological information in future reports and proposals will be needed to document the efficacy of the program.

Preliminary ISRP report comments: response requested

Response request comment:

The ISRP recognizes the value of the work and the fact that the activities appear to be well coordinated with other estuarine projects. Aspects of the proposal, however, are unclear making it difficult for the ISRP to adequately evaluate the proposed project.

The ISRP requests the proponents to address the following points in a revised proposal and to provide a brief point-by-point response to explain how and where each issue is addressed in the revised proposal: 

1. SMART Objectives. Provide project objectives in a SMART format (see proposal instructions), with an emphasis on describing the expected physical and biological outcomes. This proposal (as well as most other estuary projects) frames objectives in terms of acreage but not fish, and this is a limitation for effectively evaluating the activities. 
   
   
2. M&E matrix - support. As habitat projects and monitoring projects are not presented as part of an integrated proposal or plan, the need for a matrix to identify the linkages between implementation and monitoring is extremely important for basins or geographic areas. The ISRP is requesting a response from the Columbia River Estuary Ecosystem Monitoring Project (200300700) to summarize the linkages between implementation and monitoring projects in the Lower Columbia geographic area. We ask this project (201007000) to assist them in creating the summary and provide information about what is being monitored for this implementation project and where and when the monitoring occurs. A map or maps of locations of monitoring actions would be helpful in this regard. 
   
   
3. List of projects. Provide a complete list of projects for this proposal in an appendix or table. The ISRP noted that each annual report provides a table with details for each project, including what stage the project is in, when it began, and anticipated completion date. These annual reports could be used to generate the project list being requested here. A summary of what is being monitored could also be provided in this table, along with expected outcomes. 
   
   
4. Scoring and evaluation processes. For Survival Benefit Unit (SBU) and the Project Benefit Unit (PBU) scores, describe what these “units” mean ecologically and how they are determined. A citation and a short summary of how the new Expert Regional Technical Group (ERTG) scoring criteria differs from the SBU evaluation would explain the scoring process more clearly. 
   
   
5. Project selection. There are detailed methods presented about how projects are evaluated once selected, but little information on how projects are initially selected. The selection process should be described in more detail. 
   
   
6. Project evaluation and adjustment. Explain the provisions for any overlap with the LCEP AEM monitoring project. While the proponents state that there is a project adjustment process, no specific examples are provided as to where and how it has been used. In fact, the proponents list only one project (Chinook Estuary) subject to an adaptive management process. 
7. Confounding factors. Provide details on what the project is doing to mitigate for the confounding factors as well as the forward-looking actions that are being implemented as part of the on-the-ground activities. As the proponents note, there are serious confounding factors relevant to ecosystem restoration in the LCRE, all of which have roots in landscape-scale effects: 1) climate change, 2) land conversion, and 3) invasive species. 
   
   
8. Benefits to fish and wildlife. Provide empirical information on how the project is restoring capacity, opportunity, and function. Are restoration actions working as intended? Clarify whether or not actions are improving growth, survivorship, condition, or other fish-related factors for juvenile salmon in the restored areas. Also, if possible, provide evidence that actions are not simply creating sites for predators and competitors of juvenile salmon.

Q1: Clearly defined objectives and outcomes

The project is part of CEERP and is working with four other projects to meet the CEERP goal of restoring 4500 acres by 2035 or an average of 300 acres per year. There are general objectives given for this project in that they are going to scope 5-10 projects over a five-year period, design and plan two projects, and build at least one by 2027. Unless this one restoration action was 1500 acres, they would not attain their goal. The ISRP asks the project to provide some insight and perspective on how this goal can be attained.

This proposal, like the others from the Columbia Estuary, is difficult to evaluate because the projects have substantial overlap. The proponents tout the objectives of others while present their objectives in the internal narrative. This project provided objectives in a somewhat SMART format but never proposed specific restoration actions that could be evaluated for potential ecological benefits. In fact, the objectives are based on acreage rather than ecological outcomes. While physical outcomes are useful, the ISRP recommends that the proponents also articulate ecological outcomes for their projects.

An issue in this proposal also common to the other Columbia Estuary proposals is that, while fish performance is a stated criterion, no one seems to be measuring growth, survivorship, condition, or other fish-related factors that would indicate if the actions are benefiting fish. One would expect this to be necessary for a truly functioning adaptive management process.

How does the information in the bulleted list described as “outcomes of the Lower Columbia River Estuary Scoping and Implementation Project” relate to the projects listed in Table 1?

Q2: Methods

The proponents are implementing restoration actions (reconnecting shallow water habitats to the river) using standard engineering approaches. A description of how principles of landscape ecology influence the work being proposed would strengthen the proposal.

The social techniques are never explained in detail, perhaps because each restoration action is unique and requires a tailored approach. The proposal should include a description of the activities to expand public access such as trail bridges, hunting and fishing access, and boating, which should be documented for existing projects.

The methods to evaluate projects are well described once a project begins. The methods for evaluating cost, however, are unclear. What is the decision-making process for assessing cost? Does this affect how and what projects are accomplished?

It would be helpful if information in the Methods was tied to specific project objectives. As presented, it is very difficult to track. The proposal should include specific restoration activities that can be evaluated for their potential ecological outcomes.

In the section on Key Lessons Learned, the authors indicate the importance of conducting outreach with landowners, local governments, user groups, and the general public, as well as the need to formulate a strategy to do this. Is this being done, and if so, what is the plan/approach that will be taken to do this?

Q3: Provisions for M&E

Monitoring projects in the estuary occurs in three tiers, with level 3 occurring for all projects and levels 1 and 2 being applied more selectively. It is not clear when and how the Action Agencies decide to apply each level of monitoring. A better explanation of this would be helpful.

The adaptive management process seems to be more focused on evaluating how projects make it on the list, and this seems to occur continuously. How does geographic location affect how a project is selected and evaluated? While it is clear that substantial monitoring occurs under the AMER, it is not clear how the information and analyses feed back into project selection and evaluation. The only project where adaptive management seems to be occurring specifically within this project is the Chinook Estuary Restoration, but it is not clear what is being done with this information.

In the Project Evaluation and Adjustment Process section, the proponents state that decisions about what projects to advance are informed by results of a prioritization exercise based on disturbance theory applied at site and landscape scales. The proposal should explain the specific application of disturbance theory in the project prioritization process.

Q4: Results – benefits to fish and wildlife

The proponents describe anticipated benefits and objectives from the other regional plans (p. 5) but do not articulate how their proposed project directly relates to them.

At one level, it is understandable that specificity around future projects can frequently damage project potential, as landowners and land managers rarely like to see restoration projects and actions on their properties proposed for the future without their knowledge and consent (p. 29). However, the level of detail in the proposal is insufficient, and more information is needed within the constraints of uncertainty about the exact restoration actions in the future. This is especially perplexing when the proponents are requesting approximately $45,000,000 over 5 years to implement a few restoration actions. The ISRP requires additional details to evaluate if the request is reasonable in terms of the potential ecological outcomes.

The organization, technical background, site selection/descriptions, reporting, vision, and planning described in this umbrella proposal seem to be a model for many of the other estuary projects, if future work plans are successfully achieved. The sponsors also did a good job describing the emerging limiting factors by raising the issues and describing how they were being addressed. Their inclusion of hypotheses for adaptive management was also commendable.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The Inter-Agency Coordination effort described in the proposal is noteworthy. It would be interesting to see results of an evaluation by participants. The project goals and objectives are linked to many regional programs/plans such as the Council’s FWP 2004 Subbasin Plan, the NMFS Estuary Module (2011), and the 2008 BiOp.

The problem statement is well crafted and the objectives are realistic. The sponsors have provided an excellent technical background document which could almost be a template or model for the other estuary umbrella projects.

Although the proposal is lengthy, it is well organized and easy to follow. The work is supported by solid science, and the sponsors appear to have made a serious attempt to incorporate the latest scientific findings into their selection and implementation of projects. Program activities are well-coordinated with other restoration groups/programs in the Lower Columbia River Estuary. 

Program objectives are similar to the other projects reviewed. For individual projects, a hierarchy of expectations is presented as vision, goals, and objectives. In most cases, the objectives tie well to the vision and goals and link to proposed treatments. The objectives are generally not quantitative and do not include an expected timeframe for results. A positive aspect is that unique success criteria are developed for each project. It appears that these are mostly qualitative statements with no time frame for expected results. 

A positive feature of the proposal was a description of how the program linked to and incorporated information and direction from other sources. There was a very useful description of how elements of The Ecosystem Approach to Restoration of the LCRE by Johnson (2003) have been incorporated into the program. Similar discussions would have been useful in the other proposals that were reviewed. This section also mentioned the NPPC program and mentioned MERR but did not discuss how this linked to proposed activities.

Although it was noted that community and landowner support is critical to the long term success of the program, there are no programmatic goals, vision statements or objectives that address this critical element. It appears that this support is limited to landowner and community support for individually proposed projects. Additionally, in the discussion of emerging limiting factors, it was acknowledged that development and resource management in upstream portions of treated watersheds were important in helping to define downstream conditions, but it was stated that influencing watershed land uses was beyond the control of the program. This seems to be a key area that could be addressed in activities designed to involve and inform the local communities and landowners and the general public.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

History of the project is documented in a well written narrative. Diagrams on funding flows and arrangements were presented which make it easy to understand what has been done. The abstracts of project reports are detailed and include photos, so it is easy to see the results of the restoration projects.

This is a fairly new project, but some initial work has been started. Nice photos showing sites are included and some good preliminary data providing fish use by species is included in tables. The accomplishments are well documented and the project’s timely completion of deliverables is 90%. The program appears well organized and managed.

A good example for the incorporation of adaptive management is given for the Chinook restoration project. Additionally, examples were provided to indicate how recent research findings have been incorporated into revised designs for restoration treatments. The adaptive management framework is explained very well. Figure 1, based on Johnson et al. 2013, is very informative and provides good context for a discussion of uncertainties.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

Project relationships look excellent, and it is especially gratifying to see coordination with the long-term IMW study (Kinsel et al. 2009) which is monitoring and evaluating effects of the Abernathy Tidal Restoration in the context of fish population and monitoring in the Abernathy Creek watershed. This type of work is needed to obtain data on the effects of estuary habitat restoration on salmonid survival.

Three emerging limiting factors were identified: 1) land conversion in the contributing watersheds, 2) potential changes to the hydrologic regime because of factors including climate change and FCRPS operations, and 3) invasive plant species. The sponsors have suggested buffers as possible ways to deal with 1 and 2. Control of invasive species, in particular reed canary grass, is a more difficult task, and no real plan to deal with it is offered.

Each tailored question was appropriately answered.

4. Deliverables, Work Elements, Metrics, and Methods

 “Identify and scope 5-10 ecosystem restoration projects in the historical floodplain of the LCRE. (DELV-1) and Design, permit, and plan construction for 2 ecosystem restoration projects in the historical floodplain of the LCRE; build at minimum 1 new project. (DELV-2)”: these deliverables are mainly plans to develop a plan but are appropriate for this kind of project.

“Continue AEMR and O&M for ongoing restoration projects and initiate AM for at least 1 new restoration project to ensure sustainability and resilience. (DELV-3)”: it would be useful to determine how the sponsors plan to assess “sustainability and resilience.” Metrics for these deliverables are not given in MonitoringMethods.org and presumably are being developed in the “science for critical uncertainty” component of the estuary umbrella project. This should be clarified.

Although there have been only five projects to date, an average 90% accomplishment rate is reported. Quantitative metrics are limited to acres and miles. There are a number of other metrics described, but they are in generally qualitative terms. 

The summary of completed projects is well organized and informative. The reporting template that is used would serve as a good example for reporting other Lower Columbia River Estuary project accomplishments. 

The methods provided in MonitoringMethods.org are partially done (41%) following Roegner et al. 2009.

Specific comments on protocols and methods described in MonitoringMethods.org

Current monitoring occurs within the CEERP framework following Roegner et al. 2009. The Abernathy Creek project is contained within an IMW being monitored by the Washington Department of Ecology. Implementation monitoring appears well done and supported by success metrics for each project. A limitation of these metrics is that they are stated qualitatively.

===========QUALIFICATIONS FOLLOW================

 The ISRP’s issues can be dealt with in contracting and future project reviews.

The organization, technical background, site selection/descriptions, reporting, vision, and planning described in this umbrella proposal seem to be a model for many of the other estuary projects, if future work plans are successfully achieved. The sponsors also did a good job describing the emerging limiting factors by raising the issues and describing how they were being addressed. Their inclusion of hypotheses for adaptive management was also commendable.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The Inter-Agency Coordination effort described in the proposal is noteworthy. It would be interesting to see results of an evaluation by participants. The project goals and objectives are linked to many regional programs/plans such as the Council’s FWP 2004 Subbasin Plan, the NMFS Estuary Module (2011), and the 2008 BiOp.

The problem statement is well crafted and the objectives are realistic. The sponsors have provided an excellent technical background document which could almost be a template or model for the other estuary umbrella projects.

Although the proposal is lengthy, it is well organized and easy to follow. The work is supported by solid science, and the sponsors appear to have made a serious attempt to incorporate the latest scientific findings into their selection and implementation of projects. Program activities are well-coordinated with other restoration groups/programs in the Lower Columbia River Estuary. 

Program objectives are similar to the other projects reviewed. For individual projects, a hierarchy of expectations is presented as vision, goals, and objectives. In most cases, the objectives tie well to the vision and goals and link to proposed treatments. The objectives are generally not quantitative and do not include an expected timeframe for results. A positive aspect is that unique success criteria are developed for each project. It appears that these are mostly qualitative statements with no time frame for expected results. 

A positive feature of the proposal was a description of how the program linked to and incorporated information and direction from other sources. There was a very useful description of how elements of The Ecosystem Approach to Restoration of the LCRE by Johnson (2003) have been incorporated into the program. Similar discussions would have been useful in the other proposals that were reviewed. This section also mentioned the NPPC program and mentioned MERR but did not discuss how this linked to proposed activities.

Although it was noted that community and landowner support is critical to the long term success of the program, there are no programmatic goals, vision statements or objectives that address this critical element. It appears that this support is limited to landowner and community support for individually proposed projects. Additionally, in the discussion of emerging limiting factors, it was acknowledged that development and resource management in upstream portions of treated watersheds were important in helping to define downstream conditions, but it was stated that influencing watershed land uses was beyond the control of the program. This seems to be a key area that could be addressed in activities designed to involve and inform the local communities and landowners and the general public.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

History of the project is documented in a well written narrative. Diagrams on funding flows and arrangements were presented which make it easy to understand what has been done. The abstracts of project reports are detailed and include photos, so it is easy to see the results of the restoration projects.

This is a fairly new project, but some initial work has been started. Nice photos showing sites are included and some good preliminary data providing fish use by species is included in tables. The accomplishments are well documented and the project’s timely completion of deliverables is 90%. The program appears well organized and managed.

A good example for the incorporation of adaptive management is given for the Chinook restoration project. Additionally, examples were provided to indicate how recent research findings have been incorporated into revised designs for restoration treatments. The adaptive management framework is explained very well. Figure 1, based on Johnson et al. 2013, is very informative and provides good context for a discussion of uncertainties.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

Project relationships look excellent, and it is especially gratifying to see coordination with the long-term IMW study (Kinsel et al. 2009) which is monitoring and evaluating effects of the Abernathy Tidal Restoration in the context of fish population and monitoring in the Abernathy Creek watershed. This type of work is needed to obtain data on the effects of estuary habitat restoration on salmonid survival.

Three emerging limiting factors were identified: 1) land conversion in the contributing watersheds, 2) potential changes to the hydrologic regime because of factors including climate change and FCRPS operations, and 3) invasive plant species. The sponsors have suggested buffers as possible ways to deal with 1 and 2. Control of invasive species, in particular reed canary grass, is a more difficult task, and no real plan to deal with it is offered.

Each tailored question was appropriately answered.

4. Deliverables, Work Elements, Metrics, and Methods

 “Identify and scope 5-10 ecosystem restoration projects in the historical floodplain of the LCRE. (DELV-1) and Design, permit, and plan construction for 2 ecosystem restoration projects in the historical floodplain of the LCRE; build at minimum 1 new project. (DELV-2)”: these deliverables are mainly plans to develop a plan but are appropriate for this kind of project.

“Continue AEMR and O&M for ongoing restoration projects and initiate AM for at least 1 new restoration project to ensure sustainability and resilience. (DELV-3)”: it would be useful to determine how the sponsors plan to assess “sustainability and resilience.” Metrics for these deliverables are not given in MonitoringMethods.org and presumably are being developed in the “science for critical uncertainty” component of the estuary umbrella project. This should be clarified.

Although there have been only five projects to date, an average 90% accomplishment rate is reported. Quantitative metrics are limited to acres and miles. There are a number of other metrics described, but they are in generally qualitative terms. 

The summary of completed projects is well organized and informative. The reporting template that is used would serve as a good example for reporting other Lower Columbia River Estuary project accomplishments. 

The methods provided in MonitoringMethods.org are partially done (41%) following Roegner et al. 2009.

Specific comments on protocols and methods described in MonitoringMethods.org

Current monitoring occurs within the CEERP framework following Roegner et al. 2009. The Abernathy Creek project is contained within an IMW being monitored by the Washington Department of Ecology. Implementation monitoring appears well done and supported by success metrics for each project. A limitation of these metrics is that they are stated qualitatively.

===========QUALIFICATIONS FOLLOW================

 The ISRP’s issues can be dealt with in contracting and future project reviews.