The 1996 amendment to the Power Act establishes clear standards for ISRP review. The Council 1999 Artificial Production review and most recent Fish and Wildlife Plan establish guidelines for artificial production projects. The 1996 amendment establishes the requirements for quantifiable objectives and a monitoring plan to determine whether or not they are achieved. The Artificial Production Review and Fish and Wildlife Plan establish that the number of fish produced or released is not a sufficient goal; rather, post-release objectives are required. To meet adequate scientific review criteria, post-release metrics need to be established for survival, growth, and harvest. A robust monitoring plan with measurable objectives needs to be designed. The ISRP concludes that this project proposal Does Not Meet Scientific Criteria.

The fundamental basis for this conclusion is that the sponsor (USFWS's Creston National Fish Hatchery, in collaboration with CSKT and MFWP partners) has not provided the kind of information necessary for a scientific review of the biological or fishery benefits and costs. Moreover, the project sponsor has not demonstrated that a monitoring and evaluation plan is available against which to evaluate claimed success and mitigation benefits. Finally, there is no direct support for the sponsor's claim that the lake fisheries divert harvest pressure from local sensitive areas (beyond secondary-level claims from an Ontario MNR website that does not provide data or analysis).

In the preliminary review, and in previous review cycles, the ISRP explicitly requested a linkage to an M&E plan within the context of MFWP's Hungry Horse Mitigation project or as a stand-alone plan; an evaluation summary of biological and fishery data; and evidence of diverted pressure benefiting the local sensitive populations. While the sponsor provided some very basic information which the ISRP identified for inclusion in results reporting, for example the sites stocked and health/pathogen certifications, the broader reporting requested based on a foundational M&E plan was not provided.

The sponsor responded inadequately to the ISRP request for a copy or a linkage to an M&E plan. The sponsor indicated that the CSKT and MFWP recipients of produced fish are responsible for M&E. While this may be the case as a matter of policy, no planning or data-reporting such as a link to evaluations in annual reports or elsewhere was provided. The ISRP acknowledges that the sponsor is requesting funds for an operation and maintenance activity in support of their partners' management activities. Moreover, the ISRP acknowledges that the sponsor may not be ultimately responsible for, nor has been delegated authority for, conducting the monitoring and evaluation required for a science-based program. However, this does not change the fact that the ISRP cannot judge the merits of whether or not production, release, and management of rainbow trout or cutthroat trout in the state and tribal lakes satisfy mitigation goals.

To guide the Council and to assist the sponsor and its partners in meeting the scientific criteria for the stocked lakes fishery programs of CSKT and MFWP, as well as the O&M project for producing trout for stocking by CNFH, the ISRP recommends to the Council that the sponsor and partners produce a collaborative M&E plan within 12-18 months. The plan should include:

a) clear and measurable objectives, not simply conceptual goals, that include benchmarks or targets indicating amount of success;

b) specific working hypotheses, that is responses to management actions;

c) a general approach to testing these hypotheses, including the specific metrics and analyses that will be used for production and post-release performance evaluations; and,

d) the structure of results reporting.

The M&E plan should focus on the whole program, of which trout production is but a single, subordinate objective. Also, the M&E plan should address each partner’s role, not only in operations, but specifically in terms of evaluating whether or not the program is meeting well-defined and quantifiable objectives. The ISRP has found beneficial the inclusion of a conceptual logic pathway describing the program, including possible stopping points if mitigation objectives are not being reached, as well as other adaptive management decision points.

The ISRP also recommends the Council requests that the sponsor and partners produce a retrospective analysis of the "stocked lakes" program within 12 to 18 months, in concert with the M&E plan. The analysis should be a an objective evaluation and assessment of the program's degree of success in meeting its mitigation objects, including identifiable information gaps that would inform the M&E planning.

ISRP Retrospective Evaluation of Results

Creston National Fish Hatchery obtains west-slope cutthroat and rainbow trout eggs, hatches them, and rears the progeny with the goal of distributing 100,000 west-slope cutthroat trout (WCT) and 100,000 rainbow trout (RBT) annually in offsite closed- basin lakes for Montana Fish Wildlife and Parks (MFWP) and for the Confederated Salish-Kootenai Tribe (CSKT) to mitigate Hungry Horse Dam. The objective is to provide fishing opportunities to the public and tribal members that will reduce fishing pressure on cold water habitats selected as recovery areas for native fish populations. Eyed westslope cutthroat trout eggs are obtained from the MFWP Washoe Park State Fish Hatchery (M012 strain) and rainbow trout (various strains – Arlee, Eagle Lake, Kamloops) from the Ennis National Fish Hatchery.

For cutthroat trout survival from eyed eggs to release was 79, 69, 20, 44, and 51 percent for the years 2005 through 2009. The 20 percent survival rate in 2007 was caused by a pump failure. Survival of juvenile trout has been affected by cold water disease. The founding stock at Washoe Hatchery is reported positive for the pathogen. Fish health inspections at Creston have been negative for reportable bacteria and viruses. For rainbow trout survival from eyed eggs to release was 89, 99, 87, 83, and 86 percent for the years 2005 through 2009. Fish health inspections found all rainbow trout lots to be negative for reportable bacteria and viruses.

To meet MFWP and CSKT management requests, Creston NFH stocked the following numbers of trout during the last five fiscal years: FY 2010 WCT 122,611 RBT 102,111; FY 2009 WCT 96,406 RBT 72,922; FY 2008 WCT 97,417 RBT 71,189; FY 2007 WCT 104,840 RBT 86,652; and FY 2006 WCT 99,126 RBT 100,239.

Post stocking survival and harvest is unreported. The ISRP is unable to establish mitigation fishery benefits. The ISRP highly recommends that this program improve its record of results reporting.

The ISRP requests that sponsor provide:

1. A copy of, or a link to, the specific plan used by the cooperators to monitor and evaluate the program’s progress. See comments below for specific information requested in the plan. In the event such a plan is not presently available, the ISRP will recommend to the Council that such a plan be jointly developed among cooperators within 12-18 months.
   
   
2. A copy of, or link to, the summary report of the program’s results to date for the metrics identified in the plan such as production characteristics including size, numbers, and health assays, as well as an historical accounting of lakes that received stocking, post-stocking performance such as growth and survival, and angler-use and harvest characteristics, for example angler effort, total catch, CPUE, percent return to creel, or others as appropriate.
   
   
3. Evidence to evaluate the hypothesis that by directing fishing and harvest opportunities at fishing lakes, pressure on sensitive stream populations is reduced. Evidence might be data-driven analyses or a literature review of empirical data that demonstrates the extent of this redirection.

Generally, the proposal does not provide the kinds of information necessary to adequately judge whether or not it meets scientific criteria required for the Council’s Fish and Wildlife Program. The sponsors propose an ongoing operational project aimed at producing a target of 100,000 rainbow trout (RBT) juveniles and 100,000 westslope cutthroat trout (WCT) juveniles for release into fishing lakes managed by Montana Fish Wildlife and Parks (MFWP) and the Confederated Salish and Kootenai Tribes (CSKT) to meet mitigation requirements of construction and operation of Hungry Horse Dam. Nominally, the stocking of fish into the agency-managed lakes is identified in the “loss statement.” The ISRP identifies two basic kinds of information lacking within the proposal. 

First, the project sponsors indicate that the two cooperating agencies (MFWP and CSKT) will receive and distribute the trout propagated at the Creston NFH. They also indicate that the cooperating agencies will conduct monitoring and evaluation of the stocking operations. The ISRP appreciates that MWFP and the CSKT are responsible for the decision on which lakes will receive the stocked fish and will conduct the evaluations of post-stocking survival and fishery yield. Nonetheless, the decision pathway for determining stocking locations, stocking numbers, and consistencies with agency stocking policies needs a fuller explanation and presentation. MFWP and CSKT have each submitted proposals for RM&E activities associated with Hungry Horse Dam mitigation activities (e.g., 199101903 for MFWP), but these do not specify any Objectives, Deliverables, or Protocols associated with stocking of the Creston NFH fish. Moreover, the sponsors do not summarize or present analyses of results from this project’s past activities beyond numbers produced for 2006-10. In these years, there has been considerable annual variation in production ranging from ~70-122% of target. It is unclear whether the variation is on fishing demand, agency objectives, or simply based on production survival. Therefore, the ISRP cannot objectively evaluate the scientific basis for the program’s success at meeting its (offsite) mitigation objectives for losses associated with operation of Hungry Horse Dam. 

The ISRP requests that the sponsors provide a copy or a link to the specific plan used by the cooperators to monitor and evaluate the program’s progress. In the event such a plan is not presently available, the ISRP recommends to the Council that such a jointly developed plan (among cooperators) be produced within 12-18 months. The plan should minimally include who is responsible for individual M&E pieces, the measurable objectives of the project (such as the targets for catch rates and stocking densities required to meet those targets), a proposed list of recipient waters and stocking densities by year with a description of the lake, including its connectivity to open waters in the subbasin, the metrics used to evaluate effectiveness of stocking toward the objectives, and any specific analytical approaches that will be applied to M&E data. 

The ISRP also requests a copy of or link to the summary report of the program’s results to date for the metrics identified in the plan (if any), such as production characteristics such as size, numbers, and health assays, as well as an historical accounting of lakes that received stocking, post-stocking performance such as growth and survival, and angler-use and harvest characteristics for example angler effort, total catch, CPUE, percent return to creel, or other as appropriate. The ISRP recommends to the Council that such a report should be delivered or reviewed prior to the next annual round of production.

Second, the project sponsors suggest that by directing fishing and harvest opportunities at fishing lakes, pressure on sensitive stream populations is reduced. The sponsors suggest this is a logical outcome, although no supporting data are provided. The ISRP suggests that this is a testable hypothesis and requests such supporting documentation preferably within the data report requested above, especially data-driven analysis or a literature review of empirical data that demonstrate the extent of this redirection of angler effort rather than the possibility of simply creating a different population of harvest angler that has little bearing on the local sensitive waters. An important question that needs to be addressed is: are the anglers using fishing lakes the same as those that would target the sensitive "no kill" waters, or do the lakes and the streams recruit different a kind or population of anglers?

Ultimately, the present review reiterates previous recommendations for a more science-based approach to the proposed work. In earlier reviews, the ISRP “qualified” its recommendation in two primary ways. First, the ongoing production and distribution/stocking of rainbow trout remains at odds with goals of eradicating introgressed hybrids in the basin. The sponsors indicate that only “closed” basins receive trout so that risks are minimal. This reinforces the need for clearly articulated linkage with agency M&E actions on the stocked fish and descriptions of these recipient waters. Second, as described above, the sponsors premise the purpose of production and distribution as diverting fishing pressure on native and sensitive populations elsewhere in the basin. The ISRP seeks some evidence to substantiate this premise.

Response uploaded into Pisces - below is link to the document.

https://pisces.bpa.gov/release/documents/documentviewer.aspx?doc=P125503