Response requested comment:

It is evident that good progress has been made since 2008 in developing cost-effective genetic methods for assessing parentage, demonstrating successful production of progeny from adult translocations, and identifying species, sex, and population structure in both neutral and adaptive genes. To date, the project appears to have been very effective at both enabling and conducting research to support the conservation of Pacific lamprey. 

However, the project is complicated to review because it has effectively become an "umbrella" project requesting funding for a diversity of component projects. Even during the 2010 Category Review, the ISRP noted "the information in the proposal describing the methodology to undertake the remaining sub-objectives (and associated tasks) is too general to serve as a basis for scientific review. These sub-objectives need a response with additional details. ... As the proposal now stands, it is simply too general. It lacks specific, detailed methodology and study design to be considered scientifically justifiable. The proponents should give serious consideration to prioritizing (with rationale) the myriad of conceivable projects that could fall under the broad "plan" as outlined in the present proposal. It would be helpful if the proponents culled those sub-objectives that would not be funded directly by this project and provided more details on the methods that will be used to address lamprey passage and distribution questions." This situation has not yet been addressed. 

Given the information and time available for the 2019 Category Review, it is infeasible for the ISRP to review all the component projects listed in this proposal; and it is infeasible for the proponents to have included in a single proposal all the information needed for rigorous scientific review of this complex project. Accordingly, our review focuses on the extent to which the proponents are providing leadership and scientific expertise to achieve the stated objectives. In particular, we are examining how the component projects are prioritized for funding, how their effectiveness will be monitored and evaluated, and how overall progress toward achieving the objectives of the umbrella project will be assessed. In short, the ISRP is looking for evidence of a process for adaptive management and reassurance that the umbrella role is cost effective. In the future, it may also be useful for the ISRP to undertake a rigorous scientific review of some or all the component projects.

To complete this review, the ISRP requests a response to address the following concerns:

1. Explain how this umbrella project links with other Pacific lamprey recovery efforts in the Basin. What work is being done by CRITFC staff and how much work is subcontracted to other entities? To what extent does this project duplicate or complement the umbrella role of the Lamprey Conservation Initiative (2017-005-00)?
2. Provide quantifiable biological or physical objectives with timelines (i.e., SMART objectives) to support or replace the qualitative objectives (i.e., goals) provided in the current proposal. The quantitative elements could be measurable tasks or deliverables associated with the qualitative objectives. Additional quantitative objectives (or deliverables) and timelines should be provided for each of the component projects. Those objectives will be needed for the proponents (or ISRP) to evaluate the performance of the component projects.
3. Describe the process by which component projects are prioritized and selected for funding. 
4. Describe the procedures by which the effectiveness of each component project will be monitored and evaluated. 
5. Describe the adaptive management process (i.e., review cycles) for assessing progress toward achieving the overall objectives, for adjusting the suite of component projects based on observed outcomes, and for revising objectives.

Comment:

1. Objectives, Significance to Regional Programs, and Technical Background

Each objective states a commitment to engage with appropriate regional forums or working groups to contribute to one of six range-wide themes for restoration identified in the Lamprey Conservation Agreement. These qualitative objectives are expanded into more specific statements of actions and tasks within Table 4.4, Objectives and associated actions, and Project Deliverable Sections. Although some of the deliverables refer to specific activities, they are also vague, not quantifiable, and lack expected benefits or timelines. In the section "Objectives and Deliverables" (pages 20-22/30), the response to the prompt "How the project deliverables help meet this objective" is "to be developed" in every case. 

The most recent annual report (for 2016) provides different, more specific and somewhat quantitative objectives for each of 12 "work elements," but time lines are not stated. 

Anticipated outcomes are not expressed quantitatively. Explicit timelines for completion are not provided for any of the objectives. The ISRP recognizes that specific outcomes and completion dates may be difficult to predict for this project because they depend on decisions to be made collaboratively with other partners.

The proposal clearly explains why this project was initiated and how it is strategically consistent with biological objectives identified in the Pacific Lamprey Restoration Initiative, as well as critical uncertainties identified in the Fish and Wildlife Program. Even so, it is difficult to understand how this diverse project stands in relation to other more specific projects that are also being reviewed by the ISRP in this Category Review. 

2. Results and Adaptive Management

Participation in regional Pacific lamprey forums and working groups has enabled research that is producing a diverse suite of impressive results (listed and summarized in the proposal and annual reports). These results were generated by a collection of focused component projects, some of which are also being reviewed individually, at least in part, elsewhere in this Category Review (e.g., the Willamette Falls Lamprey Project, 2008-308-00). The component studies are too diverse, with too little detail provided in the proposal and annual reports, to enable a rigorous scientific review by the ISRP.

Despite the impressive list of research results, the proposal does not describe progress toward achieving the objectives. The latest report available in Taurus that includes reviewable details about activities associated with this project is the Annual Report for 2016; at that time, results were not yet available for many of the activities supported by this proposal. Still, it is clear that good progress has been made on specific topics such as developing cost-effective genetic methods for assessing parentage, demonstrating successful production of progeny from adult translocations, and identifying species, sex, and population structure in both neutral and adaptive genes. The project proponents have a good record of producing peer-reviewed publications and sharing information with Basin partners. They have also been diligent in their efforts to educate and reach out to the public about the ecological and cultural importance of Pacific lamprey. 

Neither the proposal nor the annual reports describe "lessons learned" or any adaptive management process (i.e., review cycles) by which decisions to alter course would be made. That said, the diversity and time course of results described within this proposal provide some reassurance that the project activities are evolving steadily and creatively as new information is gained and new opportunities are discovered with various partners. 

In sum, this project continues to provide new knowledge on methodological issues and status and trends that will benefit the conservation of Pacific lamprey populations throughout the Columbia Basin and Pacific coast. The development and application of novel genetic methods described in this project are broadly applicable for resolving uncertainties about population structure and the success of adult translocation efforts.

3. Methods: Project Relationships, Work Types, and Deliverables

Together, the proposal and annual reports provide an appropriate overview of methods for the diversity of studies undertaken and cite published literature to justify assertions and support procedures. However, the methods are not described in sufficient detail for a rigorous review, nor is this practical given the large number of different activities subsumed by this project. The project has developed important protocols that are being applied across the Basin; thirteen have been uploaded to the PNAMP web site. 

The proposal contains few details about methods or plans for monitoring and evaluation activities. Many activities supported by this project relate at least indirectly to status and trends monitoring. However, given the diversity of activities associated with this project, it would have been impractical to provide sufficient detail to support a rigorous review of M&E.

This is a proposal to implement four objectives of the Tribal Pacific Lamprey Restoration Plan for the Columbia River Basin. To accomplish this, ten general sub-objectives are identified in this proposal. One task is to finalize the draft lamprey restoration plan. The ISRP believes strongly this objective should be given a priority. Development of the overall tribal lamprey recovery program, including a prioritized list of actions and studies, should precede implementation of field work. The information in the proposal describing the methodology to undertake the remaining sub-objectives (and associated tasks) is too general to serve as a basis for scientific review. These sub-objectives need a response with additional details. When viewed as a research and restoration plan, or part of a plan, the proposal could serve as a basis for designing meaningful project components. The proponents need to develop each specific objective from the Tribal Pacific Lamprey Restoration Plan for the Columbia River Basin with the sub-objectives, addressing each of the major elements. As the proposal now stands, it is simply too general. It lacks specific, detailed methodology and study design to be considered scientifically justifiable. The proponents should give serious consideration to prioritizing (with rationale) the myriad of conceivable projects that could fall under the broad “plan” as outlined in the present proposal. It would be helpful if the proponents culled those sub-objectives that would not be funded directly by this project and provided more details on the methods that will be used to address lamprey passage and distribution questions.