The ISRP was pleased that the proponents oriented their project around the First Foods concept, especially as a framework for the objectives and the monitoring activities. The CTUIR not only use the concept for project guidance and governance but also include it in the restoration objectives for the North Fork of the John Day basin in Appendix 3. This is a valuable example of how integration of the Tribe’s traditional knowledge with regional scientific knowledge provides strength to both and, at the same time, increases the cultural and social relevance of the overall effort. The ISRP encourages the CTUIR to continue to highlight the benefits of recognizing diverse knowledge sources in order to achieve goals for fish and wildlife in the Columbia River Basin.

The ISRP highlights four suggestions to improve the project and its future evaluation:

1. SMART objectives. The SMART objectives should be in the main proposal narrative in Section 3. Incorporating these would greatly improve the ability of reviewers to evaluate the program accurately.
2. M&E matrix - support. As habitat projects and monitoring projects are not presented as part of an integrated proposal or plan, the need for a crosswalk to identify the linkages between implementation and monitoring is extremely important for basins or geographic areas. The ISRP is requesting a response from the Escapement and Productivity of Spring Chinook and Steelhead Project (199801600) to summarize the linkages between implementation and monitoring projects in the basin. During the response loop (September 24 to November 22, 2021), we ask your project to assist them in creating the summary and provide information about what is being monitored for your implementation project and about where and when the monitoring occurs. A map or maps of locations of monitoring actions would be helpful in this regard.
3. Examples of restoration outcomes. In a previous review, the ISRP asked that monitoring results be emphasized to a greater degree. The proposal does a great job of providing links to other reports, but a few examples showing changes in key properties over time would have been helpful.
4. Benefits to fish and wildlife. Since project activities are starting to be reflected in benefits to fish and to environmental quality, the ISRP recommends that the project develop plans for a comprehensive analysis and a description of the collective benefits of activities over the last two decades. This synthesis would be timely and useful.

Q1: Clearly defined objectives and outcomes

This is a well-functioning and comprehensive project, one that has greatly matured in its approaches and outcomes over the past decade. The objectives, in a general sense, are on target even though the longer-term ecological outcomes may be difficult to predict at this time.

The proponents have internally developed SMART objectives for the funding period, but the details are buried in Appendix 3. The ISRP emphasizes that the SMART objectives should appear as part of the main proposal narrative in Section 3 (Goals and Objectives). This will facilitate future evaluations of progress toward expected outcomes. Additionally, the project could provide more specific outcomes for some objectives. For instance, one objective is to “increase and reestablish in-stream thermal diversity throughout the year.” What metric will be used to quantify thermal diversity and what level of thermal diversity is the desired outcome?

Q2: Methods

The methods are based largely on established best management practices and are reasonable for the actions being proposed.

Table 2 lists examples of outreach efforts associated with certain project objectives, and it is noted later in the Potential Confounding Factors section that landowner priorities sometimes shift. Given the rich scope of this project, reviewers wonder if more outreach opportunities exist, and if they may not be of growing importance given the environmental challenges facing the basin (i.e., invasive species, changing climate). The ISRP also sees great value in communicating project results with others outside the basin, thereby encouraging even broader engagement/outreach efforts.

Q3: Provisions for M&E

The proponents have developed an effective Adaptive Management process, especially with the sustained involvement of additional collaborators. While it is not always clear who will be doing monitoring for specific project outcomes (there are numerous organizations involved), it is important that the data are appropriate for the activity and that they are readily available, which appears to be the case.

The proposal notes that the status of beaver populations is low, but that no formal census is available. Based on their importance to channel maintenance, is a status evaluation warranted? Two additional alternatives would be for the project to develop a few beaver reintroduction projects in areas where beaver could enhance existing efforts and to adopt a beaver-tolerance policy.

Given the emphasis on climate change in the confounding factors section, it is not clear whether monitoring is evaluating the success of efforts to preserve cooler water temperatures. Are these efforts working? In the Confounding Factors section, modeling water temperatures is mentioned, and while a model is good for broadly assessing vulnerability to changes in climate, such large-scale modeling typically does not account for the type of site-specific management actions occurring throughout the basin. Therefore, it would be helpful for future work to investigate water temperatures above and below restoration locations to evaluate the effectiveness of specific actions.

Q4: Results – benefits to fish and wildlife

Project activities are starting to be reflected in benefits to fish as well as to environmental quality. Considering the longevity of the project, a more thorough evaluation of the benefits would be timely. The ISRP suggests that the project develop plans for a comprehensive analysis and description of the collective benefits of activities that have taken place over the last two decades for fish and wildlife.

The sponsors provided an adequate response regarding the physical and biological monitoring related to this project, clarified the relationship between the various entities conducting monitoring on the Middle Fork John Day (project sponsors, ODFW, Middle Fork IMW), and described the data sharing that takes place among these entities. The sponsors’ present data suggests a positive response in fish abundance following habitat enhancement, but only one year of post-project data are available. The data are from three reaches within the project. The sponsors plan to continue to collect post-project data.

It is important that there be effective monitoring and evaluation (M&E) of the effects of this very intensive habitat management project on invertebrates, fish, and riparian habitat within the manipulated areas. The sponsors need to be sure that the monitoring program’s objectives and overall design moving forward are adequate. There needs to be quantitative evidence that the objectives are being attained within a reasonable time frame and that the target fish are the beneficiaries of the actions. The projects are experiments and should be treated as such; each action requires a hypothesis. The data may be collected by others, but the sponsors should use the results of the data analysis to evaluate what is working and what aspects need improvement or alteration.

The sponsors indicate that there is difficulty in detecting fish population responses, but it is not clear why this is so. Even snorkel counts or minnow traps give data for comparisons. The use of regional status and trend data is not sufficient for detecting whether or not local actions are being effective. It would be informative to see the data collected on Chinook spawning (collected by ODFW) and on other components by the IMW within the property.

While considerable monitoring data were presented in the proposal, in future proposals the sponsors should (1) strive to improve the organization of the presentation of monitoring results; (2) provide more detailed interpretation of the data, i.e., not just a table or graph but an explanation of what the data are implying; (3) provide a better description of the design of the project monitoring program; and (4) draw general conclusions. This information is critical for the ISRP’s evaluation of the success of the project. The sponsors indicate that more effort will be placed on analysis of monitoring data and that a monitoring report will be completed by 2016. This prospect is encouraging and signals a commitment by the sponsors to M&E.

 

This project has a history of favorable ISRP reviews and many strong components. However, the ISRP requests a response of the following items:

1) Provide data to show how the past actions are affecting fish abundance or production.

2) Provide details on how this project fits mechanistically with associated regional programs. In other words, which project is responsible for what activities and how is the information shared, integrated and used?

3) Provide more detail on the status of the RM&E program, the specific role of cooperators in RM&E on the Conservation Area, and progress on data analyses and reporting. Details are provided in the narrative of this review.

In addition, the ISRP offers the following recommendations to improve the project. The sponsors do not have to address these recommendations in their response.

1) Develop an overarching restoration model that can be used to guide and integrate the activities. The sponsors need an overall description of how the project and other related projects fit into an overall model of restoration.

2) Develop a more detailed accounting of specific monitoring actions and analyses. The work is very intensive in a small portion of the watershed. Sponsors need to identify some measures within the restoration areas that will show a response.

3) Develop quantitative objectives and timelines that eventually can be used to evaluate success. Some specific 1, 5, and 10-year monitoring benchmarks for progress should be established for this ambitious, intricate project. It is important to specify what kinds of responses the sponsors are anticipating over these time frames.

4) Provide evidence that the proposed modifications to the stream/mine tailings will produce positive results. This evidence may include data, literature review or an adequate rationale to suggest that this action will be successful.

5) Consider alternative actions. For example, would the benefits to fish be greater if the funds were used for other actions over a broader area of the watershed, for example, to control water temperatures?

6) Considering that most invasive species are here to stay, develop strategies that address the presence of hybrid communities, that increase riparian shade to naturally repel invasive plants, and that only institute control measures for species causing extensive ecological harm to the river, such as knotweed does to riparian areas west of the Cascades.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The Oxbow Conservation Area was purchased in 2001 with the commendable goal to “protect, manage, and restore habitat in the Oxbow Conservation Area … to aid in the recovery of Chinook salmon and summer steelhead in the Middle Fork John Day River that supports culturally significant fish populations and other biota for the Confederated Tribes of Warm Springs and secure access to these resources for its tribal members." Protecting and enhancing habitat is anticipated to benefit ESA-listed spring Chinook and summer steelhead and help maintain a sustainable harvest of fish for tribal members. The project sets aside a substantial amount of floodplain, riparian, and upland habitat. Habitat has been severely degraded, so substantial habitat enhancement work is needed.

A comprehensive Habitat Management Plan for the Forrest and Oxbow Conservation Areas was developed in March 2010 (a link to the Plan was provided in the proposal). The Plan has well-defined objectives, general approaches for accomplishing the objectives, a prioritized set of projects related to each objective, and a timeline for completion. That Plan should provide the context, direction, and justification for the work outlined in this proposal. A critical element in this review is whether the work is progressing according to the Plan, whether the sponsors have encountered any difficulties, and how these difficulties will be dealt with. An overview of the Plan in the Problem Statement section of the proposal would have helped set the stage for the proposed work.

The proposal would have been improved significantly if its objectives, deliverables, and timeline for expected results for the project were more directly linked to the Plan, making clear how the proposed work directly contributes to accomplishment of the Plan’s objectives. The objectives in the proposal could have been more closely aligned with the objectives in the Habitat Management Plan. For example:

Objective 1: At some place in the proposal the sponsors should have defined high quality habitat and discussed how it is identified, how much of it is available, and where it is located within the Conservation Area.

Objective 2: This objective is very broad in scope and encompasses at least four objectives in the Plan.

Objective 3: The proposal narrative implies a broad commitment to RM&E. The ISRP concurs that monitoring should be an essential part of the proposed work.

Objective 4: The ISRP concurs that a managed grazing program that both protects riparian vegetation and provides economic benefits is worthwhile, and it can serve to demonstrate to neighboring landowners that conservation values are not necessarily inconsistent with properly managed livestock grazing. The Habitat Management Plan describes in some detail a Riparian Pilot Grazing Project to be developed by 2014. The sponsors should have explained how Objective 4 relates to this pilot project.

Objective 5: The sponsors have put a lot of effort into outreach and education. This is one of the strongest aspects of the proposed work.

Objective 6: A practical necessity.

It would also have been helpful if the proposal had provided more information specifically related to fish. For example, what life stages would most benefit by habitat restoration in the Conservation Area? How much habitat will be created for each life stage and where is this habitat located within the Area? The Habitat Management Plan does not explicitly define quantitative goals for fish abundance and production; however, it would be useful for the sponsors to estimate the abundances of juvenile and spawning fish that can be expected to result from habitat enhancement. Results from fish surveys in “high quality” habitat may be useful in developing this estimate.

It is also important for the sponsors to discuss how they identified and prioritized potential habitat enhancement sites. Was the prioritization based on expected benefits of each project for habitat and fish? If so, how were expected benefits determined?

In addressing these issues, it would also be beneficial if the project sponsors had provided more background and framed their proposed and ongoing activities in terms of an overarching ecological-based model to guide and integrate the restoration activities.

Like most restoration projects in the Basin, this project is small-scale from a landscape perspective. The funds and activities are targeted on a short stretch of the river which has received major perturbations in the past rather than less intense actions at larger and better integrated scales. Therefore, the project by itself may not measurably improve basin-wide salmon conditions abundance as measured in a basin-wide way. With their current data the sponsors may not be able to tie any juvenile improvement to the habitat project. The ISRP suggests that it is important to have a way to assess if investments made in the restoration actions are having a positive outcome on fish abundance or production. The ISRP is concerned that because monitoring is very limited, the adaptive management process will not work as efficiently. Monitoring, when it occurs, is apparently conducted mainly by other projects and partners. The IMW work will work nicely in conjunction with this effort. However, the sponsors could also do some additional basic monitoring, with a well-crafted design, such as some electro-shock runs or minnow traps a couple of times a year.

Although the project may have some significance to regional programs and cooperation with other projects is indicated, it is not clear how this project fits mechanistically with those programs. In other words, which project is responsible for what activities and how is the information shared, integrated and used?

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

The sponsors have successfully implemented numerous specific projects. Photo points and some quantitative information on stream temperate suggest that improvement in riparian vegetation and stream habitat has occurred. However, it is difficult to evaluate the overall success of the project’s efforts to date. While some quantitative information on habitat and fish was presented more explanation would be useful.

It would have been helpful if the sponsors had used the Habitat Management Plan as a framework for structuring a discussion of accomplishments and results. One of the crucial questions that reviewers need to consider is whether the work to date is achieving the objectives set forth in the Plan. Results presented in the proposal should clearly demonstrate the progress that is being made. Several habitat surveys were conducted by various agencies in the early 2000s. The sponsors suggest that these surveys can serve as a baseline which can be compared to current habitat conditions to assess the effectiveness of enhancement actions. This is a viable approach for demonstrating progress. Data should be clearly and concisely explained and general conclusions drawn about whether the project is achieving its overall goals and its future needs.

A strength of this proposal is its excellent outreach and education program. The sponsors have gone to great length in enlisting the participation and support of landowners and other members of the public and keeping them informed of the project’s progress. The sponsors engage in many conservation-oriented programs and projects, including conservation education for children.

Past accomplishments and results are individually summarized in the proposal as follows:

1. Establish Vegetation
2. Install Large Wood Structures
3. Phase 1 Mine Tailings Rehabilitation
4. Phase 2 Mine Tailings Rehabilitation
5. Build Browse Exclosure Cages Around Phase 2 Riparian Planting Zone
6. Maintain Vegetation
7. Information Sign Updating, Access Regulation, Hunting Program
8. Project Outreach and Opportunities, Information Distribution
9. Fish, Weather, and Habitat Monitoring

The text describing the accomplishments and results reinforces the need for an overall model or strategic plan guiding the research to assure that the collective actions are having the desired effect on fish abundance, survivorship or production.

In addition, information provided on 1. The Middle Fork Intensively Monitored Watershed Program 2. Data Gathering Conducted by OCA Staff 3. Bureau of Reclamation Reach Assessment 4. Caged vs. Browsed CREP Planting Study suffers from insufficient integration and analysis of effectiveness in terms of the stated goal. Further, information is given on Complete Grazing Season, Facility Maintenance, Dredge Tailing Restoration Design, and Install Fences. While relevant to the stated goal to restore Chinook and steelhead, connections are not made as to whether the information and actions are improving fish abundance, survival or productivity.

The ISRP suggests that beaver should be considered as an active participant in the restoration. What efforts are being made to include that natural ecosystem engineer in the restoration process?

The project has been ongoing for 12 years so data should be presented to show whether the actions are having any effects on salmonid abundance or productivity. The management plan calls for status and trend and effectiveness monitoring so analysis of data should be presented.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

Project relationships (to other projects) are not easily understood. The sponsors appear to have a close working relationship with the Oregon Department of Fish and Wildlife (ODFW) and the Bureau of Reclamation (BOR). The sponsors state that they use fish data collected by ODFW for project monitoring within the Conservation Area. It is not clear, however, whether these data are collected specifically for monitoring projects in the Conservation Area. The BOR provides technical assistance and assists with development of restoration project designs.

The Habitat Management Plan expresses a clear commitment to RM&E. The Habitat Management Plan broadly outlines the kinds of RM&E that will be undertaken in the Conservation Area including status and trends and effectiveness monitoring. Objective 3 and Deliverable 12 in the proposal explicitly call for RM&E. However the status of the RM&E program is unclear. The sponsors should provide an up-to-date summary of ongoing monitoring activities. Apparently several cooperators will be involved in conducting RM&E. The role of each of these cooperators needs to be clarified. For example, it appears that ODFW will be involved with monitoring fish populations. Is their monitoring program tailored to the objectives of CTUIR’s management plan or is it a part of a larger subbasin-wide program where fish sampling sites happen to occur in the Conservation Area? The sponsors also should clarify the role of the Middle Fork IMW in RM&E. The proposal needs more detail on how the independent monitoring efforts such as the IMW are providing results. Will the IMW design provide evaluation of whether the mine tailing restoration work is having positive results?

The sponsors should discuss how monitoring data from different cooperators will be compiled, who will conduct the data analyses, and when the analyses will be brought up to date. It seems that data analysis and interpretation are lagging behind other proposed work.

It was good to see climate change listed as an emerging limiting factor. The sponsors are encouraged to use the newer climate-hydrology models to prepare forecasts for the John Day River in terms of flows and temperatures for the coming decades (see, for example, Donley et al. 2012. Strategic planning for instream flow restoration: a case study of potential climate change impacts in the central Columbia River basin. Global Change Biology doi: 10.1111/j.1365-2486.2012.02773). The results may be revealing and could help guide the restoration activities.

4. Deliverables, Work Elements, Metrics, and Methods

The Deliverables represent actions and activities that should lead to habitat improvement. They would be more meaningful if they were linked directly to elements of the Habitat Management Plan. The sponsors should provide the rationale for selection of the sites that will be enhanced.

DELV-1: Restore Mine Tailings Site. The restoration of the mine tailing work can act as a demonstration project. More evidence is needed that the proposed modifications will produce positive results. There are no data, literature review, or adequate rationale presented in the proposal to suggest that this action will be successful. DELV-2: Noxious Weed Control. The ISRP suggests that because many invasive species are here to stay, managers are faced with the emergence of dynamic hybrid communities going forward. It may be more effective to develop strategies that accepted the presence of some hybrid communities, increase riparian shade to naturally repel invasive plants, and only institute control measures for species causing extensive ecological harm to the river (for example, knotweed, to the west of the Cascades).

DELV-7: Grazing Program.This activity seems a bit peripheral to the goal of restoring fish abundance and should be carefully evaluated for relevance.

DELV-8: Reduce Forest Fuels.This activity seems a bit peripheral to the goal of restoring fish abundance and should be carefully evaluated.

DELV-11: Irrigation Ditch Efficiency.If the purpose of the project is to restore fish populations, why is water being diverted for pasture? This aspect should be carefully examined and justified, as it seems at odds with the stated goals of the project.

DELV-12: Monitor Fish and Habitat.This monitoring should include documenting hatchery strays on the spawning grounds.

DELV-14: Maintain Restoration Project: Timelines should be established beyond which the each restoration action becomes self-maintaining.

A professional publication (or two) in a refereed journal should be listed as a deliverable. It is important for large scale projects, like this one, to provide leadership in the broader restoration community

Modified by Dal Marsters on 9/26/2013 11:49:18 AM.

ISRP Request 1: Evidence of success in the social arena should be provided, as well as a plan to assess future success. The sponsors indicated that they have engaged in some social activities such as involving local landowners in stream habitat conservation efforts. They also helped sponsor a free fishing day for kids. However, the ISRP would like to see a more formalized plan for social engagement, a long term plan for engaging the public, with more formal milestones and deliverables. Such a plan is not provided in adequate detail in the proposal or the response. A goal of the project is to "Participate and cooperate with the community, agencies, and organizations to facilitate and promote education, recreation, natural resource planning, monitoring, and research of these properties and conservation activities." The entire project has a substantial social component. As such, the social aspects should have activities that can be quantified.

ISRP Request 2: The importance of this study in a broader landscape context should be described. The sponsors only partly addressed this request. They should consider, for example, how land use practices outside the project will influence watershed processes (e.g., occurrence and magnitude of floods) that could affect the project. They should also consider whether fish abundance in the watershed (e.g., redds or number of spawners) is simply following watershedwide trends or whether abundance within the project area is trending higher. These comments are also relevant to the Oxbow Project. This issue should be addressed in future proposals.

ISRP Requests 3 and 4. The sponsors adequately addressed these requests.

The ISRP recommends that the project sponsor develop a long term plan for public engagement and submit it for ISRP review early in 2014.

This project has a history of favorable ISRP reviews and is an impressive demonstration project. However, the ISRP requests a response of the following items:

1) Evidence of success in the social arena should be provided, as well as a plan to assess future success.

2) The importance of this study in a broader landscape context should be described.

3) Provide details on how this project fits mechanistically with associated programs. In other words, which project is responsible for what activities and how is the information shared, integrated and used?

4) Additional information is needed on the approach to data management.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The Forrest Conservation Area was purchased in 2002 with the goal of promoting “natural production and ecological functions to increase the larger focal species recovery effort of the John Day Subbasin.” More specifically, the project will protect and enhance habitat to benefit ESA-listed spring Chinook and summer steelhead and help maintain a sustainable harvest of fish for tribal members. The Project is a key component of the John Day Subbasin Plan as it addresses habitat protection and improvements, passage improvements, flow augmentations, upland improvements, education and outreach, and restoration of altered landscapes that help achieve the full natural production potential of the subbasin and the Columbia River Basin. The project sets aside a substantial amount of floodplain, riparian, and upland habitat. The habitat has been severely degraded and so substantial habitat enhancement is needed. The sponsors have well-defined goals and provided a good discussion of factors limiting fish production on the Conservation area.

A comprehensive Habitat Management Plan for the Forrest and Oxbow Conservation Areas was developed in March 2010 (a link to the Plan was provided in the proposal). The Plan has well defined objectives, general approaches for accomplishing the objectives, a prioritized set of projects related to each objective, and a timeline for completion. The Plan should provide the context, direction, and justification for the work outlined in this proposal. A critical element is whether the work is progressing according to the Plan, whether the sponsors have encountered any difficulties, and how these difficulties will be dealt with. The proposal would have been improved significantly if its objectives, deliverables, and results were more directly linked to the Plan, making clear how the proposed work directly contributes to accomplishment of the Plan’s objectives. An overview of the Plan in the Problem Statement section of the proposal would have helped set the stage for the proposed work.

It would have been helpful if the proposal provided more information on fish. For example, what life stages would be most benefitted by habitat restoration in the Conservation Area? How much habitat will be created for each life stage, and where is this habitat located within the Area? The Habitat Management Plan does not explicitly define quantitative goals for fish abundance and productivity; however, it would be useful for the sponsors to estimate the abundances of juvenile and spawning fish that can be expected to result from habitat enhancement. Results from fish surveys in “high quality” habitat may be useful in developing this estimate.

It also would be helpful if sponsors discussed how they identified and prioritized potential habitat enhancement sites. Was the prioritization based on expected benefits of each project for habitat and fish? If so, how were benefits determined?

The objectives in the proposal could have been more closely aligned with the objectives in the Habitat Management Plan.

Objective 1: At some place in the proposal the sponsors should have defined high quality habitat and discussed how it is identified, how much of it is available, and where it is located within the Conservation Area.

Objective 2: This objective is very broad in scope and encompasses at least four objectives in the Plan.

Objective 3: The proposal narrative implies a broad commitment to RME. The ISRP concurs that monitoring should be an essential part of the proposed work.

Objective 4: The ISRP concurs that a managed grazing program that both protects riparian vegetation and provides economic benefits is worthwhile, and it can serve to demonstrate to neighboring landowners that conservation values are not necessarily inconsistent with properly managed livestock grazing. The Habitat Management Plan describes in some detail a Riparian Pilot Grazing Project to be developed by 2014. The sponsors should have explained how Objective 4 relates to this pilot project.

Objective 5: The sponsors have put a lot of effort into outreach and education. This is one of the strongest aspects of the proposed work.

Objective 6: A practical necessity.

The proposal would be improved if quantitative goals had been provided. For example, how much can fish abundance and freshwater productivity be expected to increase over given time periods? Is it possible to develop these estimates using habitat data?  It would be beneficial to establish some intermediate benchmarks for success at 1, 3, 5, 10, and 15 year intervals. For example, with riparian restoration - what will be the shorter term trophic effects on primary and secondary production?

The sponsors should consider how large disturbances might affect the ability of the project to meet its objectives. Will the restoration work lead to more resilient ecosystems, less flood damage, and vegetation that responds to fire?

The ISRP has the following concerns: 1) While there is considerable activity planned, there should be an overarching model to guide and integrate the activities. 2) The objectives should state quantitative goals and timelines for specific activities to be successful. 3) The monitoring is sparse and has no systematic structure. Monitoring is apparently mostly conducted by other projects and partners. 4) As a result of concerns 1-3, there is no way to determine if the investments made in the restoration actions are having a positive outcome on fish abundance, survivorship or production. The project may have some significance to regional programs but it is not clear how this project fits mechanistically with programs beyond the John Day catchment as well as within it. In other words, which project is responsible for what activities and how is the information shared, integrated and used? Certainly, there is considerable cooperation.

This is a demonstration project for showing harmony of grazing and conservation work. It is a good site for this. The sponsors could benefit from communication with others doing similar work in the Pacific Northwest. For example, the sponsors should contact the Nature Conservancy to share the results of grazing work at Sycan Marsh in the Klamath Basin.

2. History: Accomplishments, Results, and Adaptive Management (Evaluation of Results)

The sponsors have implemented a number of projects intended to improve riparian conditions, quality and quantity of instream habitat, and fish passage. Although some quantitative information is presented most results are shown with high quality, effective photographs illustrating the kinds of enhancement activities that have taken place and habitat conditions pre-and post enhancement. It would have been helpful if the sponsors had used the Habitat Management Plan as a framework for structuring a discussion of accomplishments and results. One of the crucial questions that reviewers need to consider is whether the work to date is achieving the objectives set forth in the Plan. Results presented in the proposal should clearly demonstrate the progress that is being made. Several habitat surveys were conducted by various agencies in the early 2000s. The sponsors suggest that these surveys can serve as a baseline which can be compared to current habitat conditions to assess the effectiveness of enhancement actions. This is a viable approach for demonstrating progress. Data should be clearly and concisely explained and general conclusions drawn about whether the project is achieving its overall goals and its future needs.

A strength of this proposal is its excellent outreach and education program. The sponsors have gone to great length in enlisting the participation and support of landowners and other members of the public and keeping them informed of the project’s progress. The sponsors engage in many conservation oriented programs and projects, including conservation education for children.

Management changes discussed by the sponsors are modification of ongoing enhancement projects that typically would be made in a project like this. There do not appear to be major project wide changes in direction and restoration methodology.

The ISRP is skeptical about the overall project being able to meet the goal by 2030 (see the stated vision in the proposal). There are no testable hypotheses (at least none are provided), the objectives are vague (not quantitative and no specific timelines), monitoring is inadequate and, therefore, the adaptive management process cannot work as intended. Without testable hypotheses and the monitoring data to test them, the Adaptive Management process cannot work efficiently. It is not clear if Structured Decision Making is being used.

The sponsors indicated that “The Forrest Conservation Area’s contracted deliverable history is 81%, which seems an average, reasonable percentage rate of success.” In addition to completion of deliverables, success should be measured ultimately by improvements in the fish population and secondarily in steam and riparian conditions.

In terms of past deliverables, the information provided was limited. For example, statements such as “The Tribes attempt to monitor the property and the restoration projects as much as possible given limited funds and time to do so” and “The Tribes are in the early phases of developing a web based data storage site to host all the data and make it accessible to interested parties.” Monitoring of restoration actions should be a core activity and data management should be well developed. Concerning data, how does the project interface with ISEMP or CHaMP? Further, redd counts are conducted for steelhead but what about Chinook and bull trout? How are lamprey monitored? How are the photo point images used? What data are generated?

P.29: What species is being stocked in the ponds for recreational fishing?

Further, beaver should be an active participant in the restoration. What efforts are being made to include that natural ecosystem engineer in the restoration process?

The project has been in place since 2002 but the few data provided appear to show no response from the fish.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions

Project relationships (to other projects) are not easily understood. The project expresses a clear commitment to RM&E. The sponsors appear to have a close working relationship with the Oregon Department of Fish and Wildlife (ODFW) and the Bureau of Reclamation (BOR). The sponsors state that they use fish data collected by ODFW for project monitoring within the Conservation Area. It is not clear, however, whether these data are collected specifically for monitoring purposes in the Conservation Area and so is suitable for assessment of project effectiveness, or whether it is part of a larger scale monitoring effort in which some of the sample sites happen to fall within the Area.

The Habitat Management Plan broadly outlines the kinds of RM&E that will be undertaken in the Conservation Area including status and trends and effectiveness monitoring.  Objective 3 and Deliverable 12 in the proposal explicitly call for RM&E. The status of the RM&E program, however, is unclear. The sponsors should provide an up-to-date summary of ongoing monitoring activities. Apparently several cooperators will be involved in conducting RM&E. The role of each of these cooperators needs to be clarified. The sponsors also should clarify the role of the Middle Fork IMW in RM&E. The sponsors should discuss how monitoring data from different cooperators will be compiled, who will conduct the data analyses, and when the analyses will be brought up to date. It appears that data analysis and interpretation is lagging behind other proposed work.

It was beneficial to see climate change listed as an emerging limiting factor. The sponsors are encouraged to use the newer climate-hydrology models to prepare forecasts for the John Day River in terms of flows and temperatures for the coming decades (see, for example, Donley et al. 2012. Strategic planning for instream flow restoration: a case study of potential climate change impacts in the central Columbia River basin. Global Change Biology doi: 10.1111/j.1365-2486.2012.02773). The results may be revealing and could help guide the restoration activities.

4. Deliverables, Work Elements, Metrics, and Methods

With some exceptions the Deliverables are not specific regarding project locations and desired outcomes of restoration actions. Deliverable 3 pertains to conduct of RM&E, but few details of the RM&E plan are given. Methods and metrics for RM&E are cited in MonitoringMethods.org, but they would be more meaningful if they were in the context of an RM&E program. The sponsors should provide the rationale for selection of the sites that will be enhanced.

Operation of the nursery is a step in the right direction. However, its capacity to produce plants is relatively limited. The 12,000 plants per year equates to a community density of less than a hectare on a young native riparian stand (~15 years old). Nursery operations should be carefully examined with a focus on increasing capacity in the near future. The scale is small for the work they are doing. They offer the ability to collect materials from the specific restoration sites and local areas. They may want to expand their nursery aspect.

Planting in stages looked like a good strategy to allow stratification. They should monitor the success of this approach.

Most invasive species are here to stay. While some control may be attempted – usually at very high costs – the fact is that managers are faced with the emergence of dynamic hybrid communities going forward. It may be more effective to develop strategies that in some instances tolerate the presence of hybrid communities, increase riparian shade to naturally repel invasive plants, and only institute control measures for species causing extensive ecological harm to the river (for example, knotweed, west of the cascades). Has fire been considered as a control measure? It has been very effective in other fire dominated communities (including riparian zones). See review: Pettit, N.E. and R.J. Naiman. 2007. Fire in the riparian zone: Characteristics and ecological consequences. Ecosystems 10:673-687.

A professional publication (or two) in a refereed journal should be listed as a deliverable. It is important for large scale projects, like this one, to provide leadership in the broader restoration community.

Specific comments on protocols and methods described in MonitoringMethods.org

P. 47: What type of data are you collecting and how are you documenting supporting metadata? “This information is covered by the protocol(s) set up in monitoringmethods.org account linked to this geographical review. Data is documented with the metadata where possible.” A more comprehensive response is needed for this question. Perhaps provide a Table showing how the metadata are linked to each deliverable

Modified by Dal Marsters on 9/26/2013 11:57:27 AM.

This is a well-prepared proposal that is making progress toward its well-stated and well-justified objectives. The Oxbow Conservation Area was purchased as a high priority project in 2001 with BPA mitigation funds by the Confederated Tribes of Warm Springs Reservation of Oregon (Tribes). The Project has since received BPA annual funding for O&M as part of the Memorandum of Agreement between the Tribes and BPA.

The Oxbow property is located in the Camp Creek 5th Field HUC in the John Day subbasin. The subbasin plan identifies the Middle Fork John Day River as the highest priority subwatershed for the John Day subbasin. The valuable property holds a high concentration of adult spring Chinook salmon through the high temperatures and low flows of the summer months in its deep pools. The conservation area offers spawning and rearing habitat to Chinook, summer steelhead and bull trout as well as access to the five fish-bearing perennial tributaries that come into the property from National Forest lands.

Technical and scientific background: The proposal contains good detail of riparian and in-stream problems requiring remediation and describes in some detail past accomplishments. A list of monitoring activities is presented and an M&E document is referenced. The proponents gave adequate responses to past ISRP questions and concerns. In particular, monitoring and evaluation on the Oxbow Ranch appears to be well coordinated with ongoing ODEQ and ODFW monitoring projects for the John Day basin.

This section contains a quite complete description of the Oxbow Conservation Area, including habitat conditions and context. It also contains material that would be more appropriately placed in the sections on project history, objectives and methods. The section on spring Chinook contains a statement that the fish are protected under MSFCMA, when the reference should be to ESA. The table on fish distribution should contain some citation to sources. Overall the section contains good description of the fish and wildlife species and assessments done on their abundance and habitat. Helpful photos are provided. A good description of habitat issues that need to be addressed by the activities proposed in this project is also provided: dredge tailings, fish passage, riparian trees and shrubs, non-native plants and forest health.

Rationale and significance to subbasin plans and regional programs: The proposal establishes good rationales and significance through linkages to the 2000 Fish and Wildlife Program and to the John Day Subbasin Plan. The area in which Oxbow CAP is located is identified as the highest priority for restoration in the subbasin plan. Recovery strategies identified as highest priority in the subbasin plan are consistent with activities contained in the proposed project. The Oxbow CAAP has developed a draft management plan which is under review at BPA. Goals and objectives of that plan are reflected in this proposal. The proposal also notes links to the USFS and NC management plans for the Middle Fork John Day, with ODFW management plans, with the Grant County SWCD, the watershed council, and with Wy-Kan-Ush-Mi-Wa-Kish-Wit.

Relationships to other projects: The proposal lists several other projects to which this project is directly linked and with which it shares resources. A table identifies specific activities that are shared with other projects.

Project history: A history of project development and various funding issues affecting project scale is presented. This is followed by a description of project results by category such as habitat protection, fencing, planting, fish screening, etc. The project clearly has implemented a substantial amount of restoration work. It would be helpful to have a little more evaluation of what these actions mean in the overall context; e.g.; where is the area now relative to where it was, and needs to be.

Objectives: The proposal contains a number of biological objectives that link the subbasin plan and Oxbow CAP management plan. Several work elements are associated with each objective. The objectives are quite general in specification (e.g. "restore stream base flows) but contain a work element that is quite specific (obtain instream leases for water rights). Time lines are specified. Specific details are contained in the work element metrics (admin and budgeting section).

Tasks (work elements) and methods are broadly described, but seemingly appropriate. Several work elements are associated with each objective. Some of the work elements are presented in general, rather than specific terms (e.g. install fence) but do contain discussion that establish the intention, context and rationale in more detail. Other work elements (e.g. replace 4 fish screens) are specific and measurable. All work elements have specific time lines attached. The objectives and work elements cover a lot of ground and consist of reasonable activities, with reference to their motivation in management plans and to monitoring activities (e.g. the grazing plan, water conditions, fish counts, etc). Specific details are contained in the work element metrics (admin and budgeting section).

Monitoring and evaluation: M&E is conducted in a separate grazing management plan. M&E of project results for fish and habitat is also a separate work element. Data are collected and monitoring conducted on stream temperature, bird surveys, habitat condition, stream flow, fish counts, weather, etc. Descriptions of monitoring efforts contained in Section B provide additional detail of the type of assessment, monitoring and evaluation that is part of this project. It would be useful to see the Oxbow Conservation Area Management Plan to see how the monitoring is integrated to inform decision making on the area as a whole. There is quite a bit of monitoring laid out, but not very good indication of what they are looking for in terms of responses.

Facilities, equipment, and personnel: Facilities and personnel are well situated in place with strong ties to related projects. Also note cost-sharing with the Nature Conservancy and other institutions.

Information transfer: Good description of not only routine reporting to BPA, but also specific details on information sharing and coordination with other projects and agencies.

Benefit to focal and non-focal species is well described. Project restoration activities will provide realizable benefits to spring Chinook, steelhead, redband trout and lamprey. Habitat restoration actions will also benefit frogs, white-tail deer, mink, mallard, yellow warbler, black-capped chickadee and western meadowlark. It is reasonable to expect that these benefits will persist over the long term.

This is a well-written proposal with a clear history and clear objectives, methods, M&E, and demonstrated cooperation with other related projects. The Forrest Conservation Area was purchased by the Confederated Tribes of Warm Springs Reservation of Oregon (Tribes) in 2002 as a high priority project with BPA mitigation funds. The project has since received BPA annual funding for O&M as part of the Memorandum of Agreement between the Tribes and BPA. The Conservation Area is 4,232 acres and is split into two geographically separate parcels located along the Upper Middle Fork and Upper Mainstem John Day Rivers in the John Day Subbasin.

Though currently well below its potential for fish and wildlife due to previous habitat degradation, the property contains critical habitat used by spring Chinook, summer steelhead, and a variety of wildlife. Spawning spring Chinook densities on the Middle Fork property are the highest in the basin and the property represents 4,083 Habitat Units (HU) of protection for 7 wildlife mitigation species for BPA. Benefits from this project to focal and non-focal species should persist over the long term.

Previous ISRP reviews of this proposal were very positive and noted that it was an important high priority project. The current project proposal recounts biological results (gains) that have occurred since acquisition of the property.

Technical and scientific background: The technical and scientific background is excellent. It describes in detail the subbasin context and the Forrest Conservation area within it. It includes a description of the property, assessments conducted, baseline conditions, limiting factors, desired future conditions, and restoration strategies to achieve these. It also contains information that probably should be included in other sections (ties to other projects, history, objectives, etc). The section on spring Chinook contains a statement that the fish are protected under MSFCMA, when the reference should be to ESA. Helpful photos are provided. A good description of habitat issues that need to be addressed by the activities proposed in this project is also provided.

Rationale and significance to subbasin plans and regional programs: The proposal establishes good rationales and significance through linkages to the 2000 FCRPS BiOp and to the John Day Subbasin Plan (JD SBP). The conservation area is a key component of the JD SBP. Recovery strategies identified as highest priority in the SBP for are consistent with activities contained in the proposed project. Project actions are motivated by the limiting factors and their corresponding strategies in the JD SBP. The proposal also describes links to the 2002 Fish and Wildlife Program habitat strategies. The proposal also notes links to the Wy-Kan-Ush-Mi-Wa-Kish-Wit.

Relationships to other projects: An extensive list of direct links to and complementarities with other projects is provided. These projects are managed by CTWS, ODFW, CTUIR, OYCC, BOR, Grant SWCD, ODEQ, public schools, USDA NRCS, etc. The proposal describes very strong links with description of the nature of the link.

Project history is extensive and well documented, particularly for a project that is only 3+ years old. A short history of project development and funding is presented, followed by an extensive description of project activities by category such as fencing, planting, CREP, flow enhancements, irrigation improvements, fish screening, etc. The project clearly has implemented a substantial amount of restoration work. Good detail is provided as justification for the activities. A detailed description of monitoring of project activities is included.

Objectives: Objectives relate to those specified in the JD SBP and to specific restoration goals for the Forrest Conservation Area. Objectives are stated in general form, but become more specific in the expression of work elements and quite specific and measurable in the metrics presented in the administrative section. Objectives are reasonable and comprehensive.

Tasks (work elements) and methods: Several work elements are associated with each objective. Some of the work elements are presented in general, rather than specific terms ((e.g. remove vegetation) but do contain discussion that establish the intention, context and rationale in more detail. Other work elements (replace culverts) are specific and measurable. The objectives and work elements cover a lot of ground and consist of reasonable sounding activities, but lack discussion of their motivation contained in the Oxbow proposal. Each work element contains collection of data for monitoring and evaluation. Specific measurable quantities are contained in the work element metrics (admin and budgeting section).

Monitoring and evaluation: A detailed description of monitoring activities is included in the section on project history. Work elements also contain components to "collect, generate, validate field and lab data" with a description of how these data will be used in evaluating success of the strategies. It would be useful to see the Forrest Area Management Plan to see how the monitoring is integrated to inform decision making on the area as a whole.

Facilities, equipment, and personnel: Facilities and personnel are well situated in place with strong ties to related projects. A specific list of equipment and facilities, with functions and conditions noted, is provided.

Information transfer: Good description of not only routine reporting to BPA but also specific details on information sharing and coordination with other projects and agencies. Indirectly addressed through listing of proposed reports.