The Columbia Habitat Monitoring Program (CHaMP) monitoring program described in this proposal is very comprehensive and ambitious. The fact that this effort builds on the success of the ISEMP project provides increased confidence that the program can succeed. As the program is implemented, participants should consider developing formal mechanisms for communicating results to mangers. The ISEMP project holds a great deal of promise for answering the questions: “What is the current status of fish habitat in the Columbia River Basin?” and “Are restoration actions currently being undertaken in the Columbia River Basin having the desired effects on both habitat condition and biological response?” We were pleased to see that the current proposal includes a number of new watersheds that will expand the geographical scope and completeness of ISEMP. We were also pleased that the strong emphasis in standardized data collection and spatially balanced and randomized sampling is intended to bring more consistency to monitoring efforts in the Columbia River Basin. In general, the proposal meets scientific criteria, with one qualification. The ISRP also offers additional suggestions for project proponents to consider.

Qualification: The ISRP recommends that ISEMP organize a one-day workshop to discuss the CHaMP approach with the ISRP/ISAB and others. A draft of CHaMP should be circulated to the ISRP/ISAB before the workshop. Specific issues at the workshop should include how previously collected data can be or has been incorporated into CHaMP databases. It would also be useful to summarize how ISEMP priorities have evolved over the years, as well as a publication strategy. This qualification was discussed with the ISEMP project lead on September 13 and was determined to be agreeable. The workshop will be tentatively scheduled for late 2010 or early 2011.

1. Purpose, Significance to Regional Programs, Technical Background, and Objectives

In the Columbia River Basin, there are a wide variety of RM&E projects that often differ in sampling design, methods, metrics, and quality and extent of data analysis, and often do not lead to definitive evaluations of habitat restoration effectiveness. ISEMP is a far reaching project that is based on the reasonable proposition that a standardized set of protocols, procedures, and data collection and analytical methods that can be adopted basinwide is needed to improve data collection precision and accuracy, provide comparability of results within and between subbasins, and so improve the capability of determining the effectiveness of habitat restoration projects. This proposition has been tested in a limited number of watersheds. In this proposal ISEMP is expanding its scope to include several additional watersheds to further evaluate its protocols for study design, data collection, analytical methods, and information dissemination. This effort is worthwhile in that improvements in habitat effectiveness evaluation are badly needed within the Basin.

The technical background is very complete in the proposal. This project is clearly significant to regional restoration programs. A coordinated program for the collection, compilation, and archiving of data on fish populations and habitat condition has been identified in numerous ISRP and ISAB documents as a critical need for the Columbia Basin. Explanation of the significance of CHaMP to regional programs was very thorough. One question had to do with data archiving and database sharing. What will the interface between ISEMP databases and BPA’s Taurus project tracking system be? Will CHaMP data be linked to Taurus in such a way that interested parties can access habitat or population status and trend data (e.g., the CHaMP metadata library) directly, or will these databases be housed separately by ISEMP outside Taurus?

It was gratifying to see that CHaMP will be testing novel remote sensing techniques for assessing habitat condition over large areas, e.g., using green LiDAR or multispectral sensing. This project has also demonstrated leadership in tracking population and life history attributes using PIT-tag arrays and other landscape-based methods.

2. History: Accomplishments, Results, and Adaptive Management

The links provided to reports produced through ISEMP, in the aggregate, provide a very complete picture of the results generated by this project and reflects well on the prospects for success of the CHaMP program. The manner in which the CHaMP data will be used adaptively to modify future monitoring efforts is clearly described, and some examples are provided. However, the link with managers and policy decision makers in the basin is less clearly described. The proposal indicates that a process will be established specifically to utilize the data generated through CHaMP to produce new analysis tools, which will be used to generate the type of information required to determine future direction of restoration efforts and to support fisheries management decisions. But there is another step required to make this process maximally effective; a formal process for communicating the output from the data and analytical tools to non-technical audiences. ISEMP has used periodic newsletters as one mechanism for addressing this function. This approach also would be a reasonable option to consider for CHaMP. But the CHaMP project leaders should devote some effort to developing a consistent process for broadly disseminating program results.

The ISEMP project has expanded in scope perhaps more than any other habitat restoration-related project funded by BPA since its inception. There are now, according to the proposal, ISEMP studies taking place in 26 watersheds in the Columbia River Basin, all of which contain anadromous salmonids. That organizers have succeeded in growing this project in such an impressive fashion reflects well on the willingness of a wide variety of stakeholders (federal and state agencies, tribes, local conservation districts) to enter into cooperative arrangements with the ISEMP project to address large scale restoration status and effectiveness questions. ISEMP has grown to such an extent that many of its component parts could be treated as separate projects.

It was interesting to see how the proposal described past results. There were abundant maps and lists of activities taking place in ISEMP watersheds, but there were relatively few graphs or tables showing how target species have responded to habitat restoration. We were hoping for a little more in the way of biological response findings, since some restoration locations have now been monitored by ISEMP for seven years. The proposal suggests that the results of habitat restoration often require extended monitoring periods (i.e., often decades) in order for their effects to be assessed. We concur, but including a few highlights of some of the most informative results to date would have made the proposal more interesting.

The proposal does an excellent job of describing the formation and evolution of the CHaMP effort, which is in effect an important type of adaptive management, i.e., the development of standardized habitat survey protocols in order to facilitate data analyses and inter-watershed comparisons. Overall, the description of other activities was thorough and informative. One adaptive management question is: have any restoration actions changed as a result of ISEMP findings? In particular, we are interested in knowing if anything is being done differently because evidence is starting to suggest that current approaches are not working as anticipated. Perhaps, as the proposal points out, it is premature to make judgments but if there are any good examples of restoration practitioners learning from past mistakes, they would be worth knowing.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (Hatchery, RME, Tagging)

Generally, the relationships of this program to other habitat and fish monitoring projects in the Columbia Basin are well described in the proposal. The one relationship that could have benefited from a more thorough discussion was the association between CHaMP and PNAMP. Several of the deliverables in the proposal will be co-developed with PNAMP. But the role of each organization in producing these deliverables was not clearly described. A paragraph in the introduction that outlines this relationship and some indication under the shared deliverables of roles and responsibilities would have helped to clarify the division of labor.

The monitoring work includes tagging, other types of fish population assessment, experimental habitat restoration, long-term habitat trend monitoring, habitat protocol standardization, food web studies, and management of very large datasets. The project does not emphasize identifying limiting factors; rather, it is aimed more toward evaluating the responses of aquatic habitat and fish populations to restoration actions at large spatial scales. Overall, the proposal does a good job of relating ISEMP-sponsored monitoring to other restoration and monitoring projects. The restoration questions being addressed in each of the watersheds are appropriate to the issues believed to be limiting to salmon production. The new CHaMP rotating panel (GRTS) monitoring design appears to hold considerable promise in characterizing habitat status and trends. The project is consistent with the call for expanded RM&E in many subbasin plans and regional programs. This work is relevant to most RM&E efforts basinwide and provides a means by which RM&E programs in diverse subbasins can be unified under a common set of protocols and procedures.

The ISRP suggests that pollutants not be overlooked as potentially limiting factors in certain locations.

4. Deliverables, Work Elements, Metrics, and Methods

Deliverables, work elements, metrics and methods are adequately described. Some of the details specific to different work elements could be explained more completely (e.g., in what types of habitat will benthic macroinvertebrates be sampled, and why?), but given the very broad scope of the project and the need to cover all the work elements this is understandable.

Journal publications are listed in many of the deliverables for different objectives, but to date there have not been very many papers published from the ISEMP work. Hopefully this will change in the near future.

What was the rationale for including 25 sample sites in each basin for habitat status and trend monitoring? Did the ISEMP data suggest that this number of sites would provide adequate statistical power? Some support for this number of sites should be provided. It would seem that the number of sample sites required to adequately represent the range of channel types within a basin would vary based on watershed size, variation in topography, geology, land use and other factors. If this is the case, a variable sample size might be more appropriate.

The assessment of stable isotope analysis to characterize trophic aspects of habitat condition in the John Day River is a novel element of this habitat monitoring program and could lead to the development of a very valuable tool. Limiting factors have almost exclusively been restricted to physical habitat or water quality attributes, largely because there was no efficient method for assessing food web conditions. One suggestion about the proposed methodology for this work element: the terrestrial invertebrates should not be lumped together for stable isotope analysis. As with the aquatic invertebrates, the terrestrial insects should be grouped on the basis of functional group (detritivores, herbivores, or predators).

The deliverable that addresses status and trend monitoring in the Wenatchee actually discusses this activity in the Entiat. In fact, the information for the Entiat is repeated in the second deliverable, which actually does address the Entiat. This discrepancy seems like an inadvertent cut-and-paste error, but the Wenatchee information should be added to the first deliverable.

ISEMP suggests, where appropriate, an “experimental approach” where habitat restoration actions in “treatment” streams are compared to reference “untreated” streams. Given the physiographic and biological variability within reaches or tributaries within a subbasin, let alone differences between subbasins, selection of appropriate references and treatments could prove challenging. It probably would be helpful if ISEMP provided guidelines and/or assistance to subbasin investigators for selecting both reference and treatment sites. The same could be said for data analyses. With the large amount of data that will be collected, investigators may need some assistance in data analyses. It is our understanding that ISEMP is planning to provide analytical assistance where needed.

The Columbia Habitat Monitoring Program (CHaMP) monitoring program described in this proposal is very comprehensive and ambitious. The fact that this effort builds on the success of the ISEMP project provides increased confidence that the program can succeed. As the program is implemented, participants should consider developing formal mechanisms for communicating results to mangers. The ISEMP project holds a great deal of promise for answering the questions: “What is the current status of fish habitat in the Columbia River Basin?” and “Are restoration actions currently being undertaken in the Columbia River Basin having the desired effects on both habitat condition and biological response?” We were pleased to see that the current proposal includes a number of new watersheds that will expand the geographical scope and completeness of ISEMP. We were also pleased that the strong emphasis in standardized data collection and spatially balanced and randomized sampling is intended to bring more consistency to monitoring efforts in the Columbia River Basin. In general, the proposal meets scientific criteria, with one qualification. The ISRP also offers additional suggestions for project proponents to consider. Qualification: The ISRP recommends that ISEMP organize a one-day workshop to discuss the CHaMP approach with the ISRP/ISAB and others. A draft of CHaMP should be circulated to the ISRP/ISAB before the workshop. Specific issues at the workshop should include how previously collected data can be or has been incorporated into CHaMP databases. It would also be useful to summarize how ISEMP priorities have evolved over the years, as well as a publication strategy. This qualification was discussed with the ISEMP project lead on September 13 and was determined to be agreeable. The workshop will be tentatively scheduled for late 2010 or early 2011. 1. Purpose, Significance to Regional Programs, Technical Background, and Objectives In the Columbia River Basin, there are a wide variety of RM&E projects that often differ in sampling design, methods, metrics, and quality and extent of data analysis, and often do not lead to definitive evaluations of habitat restoration effectiveness. ISEMP is a far reaching project that is based on the reasonable proposition that a standardized set of protocols, procedures, and data collection and analytical methods that can be adopted basinwide is needed to improve data collection precision and accuracy, provide comparability of results within and between subbasins, and so improve the capability of determining the effectiveness of habitat restoration projects. This proposition has been tested in a limited number of watersheds. In this proposal ISEMP is expanding its scope to include several additional watersheds to further evaluate its protocols for study design, data collection, analytical methods, and information dissemination. This effort is worthwhile in that improvements in habitat effectiveness evaluation are badly needed within the Basin. The technical background is very complete in the proposal. This project is clearly significant to regional restoration programs. A coordinated program for the collection, compilation, and archiving of data on fish populations and habitat condition has been identified in numerous ISRP and ISAB documents as a critical need for the Columbia Basin. Explanation of the significance of CHaMP to regional programs was very thorough. One question had to do with data archiving and database sharing. What will the interface between ISEMP databases and BPA’s Taurus project tracking system be? Will CHaMP data be linked to Taurus in such a way that interested parties can access habitat or population status and trend data (e.g., the CHaMP metadata library) directly, or will these databases be housed separately by ISEMP outside Taurus? It was gratifying to see that CHaMP will be testing novel remote sensing techniques for assessing habitat condition over large areas, e.g., using green LiDAR or multispectral sensing. This project has also demonstrated leadership in tracking population and life history attributes using PIT-tag arrays and other landscape-based methods. 2. History: Accomplishments, Results, and Adaptive Management The links provided to reports produced through ISEMP, in the aggregate, provide a very complete picture of the results generated by this project and reflects well on the prospects for success of the CHaMP program. The manner in which the CHaMP data will be used adaptively to modify future monitoring efforts is clearly described, and some examples are provided. However, the link with managers and policy decision makers in the basin is less clearly described. The proposal indicates that a process will be established specifically to utilize the data generated through CHaMP to produce new analysis tools, which will be used to generate the type of information required to determine future direction of restoration efforts and to support fisheries management decisions. But there is another step required to make this process maximally effective; a formal process for communicating the output from the data and analytical tools to non-technical audiences. ISEMP has used periodic newsletters as one mechanism for addressing this function. This approach also would be a reasonable option to consider for CHaMP. But the CHaMP project leaders should devote some effort to developing a consistent process for broadly disseminating program results. The ISEMP project has expanded in scope perhaps more than any other habitat restoration-related project funded by BPA since its inception. There are now, according to the proposal, ISEMP studies taking place in 26 watersheds in the Columbia River Basin, all of which contain anadromous salmonids. That organizers have succeeded in growing this project in such an impressive fashion reflects well on the willingness of a wide variety of stakeholders (federal and state agencies, tribes, local conservation districts) to enter into cooperative arrangements with the ISEMP project to address large scale restoration status and effectiveness questions. ISEMP has grown to such an extent that many of its component parts could be treated as separate projects. It was interesting to see how the proposal described past results. There were abundant maps and lists of activities taking place in ISEMP watersheds, but there were relatively few graphs or tables showing how target species have responded to habitat restoration. We were hoping for a little more in the way of biological response findings, since some restoration locations have now been monitored by ISEMP for seven years. The proposal suggests that the results of habitat restoration often require extended monitoring periods (i.e., often decades) in order for their effects to be assessed. We concur, but including a few highlights of some of the most informative results to date would have made the proposal more interesting. The proposal does an excellent job of describing the formation and evolution of the CHaMP effort, which is in effect an important type of adaptive management, i.e., the development of standardized habitat survey protocols in order to facilitate data analyses and inter-watershed comparisons. Overall, the description of other activities was thorough and informative. One adaptive management question is: have any restoration actions changed as a result of ISEMP findings? In particular, we are interested in knowing if anything is being done differently because evidence is starting to suggest that current approaches are not working as anticipated. Perhaps, as the proposal points out, it is premature to make judgments but if there are any good examples of restoration practitioners learning from past mistakes, they would be worth knowing. 3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (Hatchery, RME, Tagging) Generally, the relationships of this program to other habitat and fish monitoring projects in the Columbia Basin are well described in the proposal. The one relationship that could have benefited from a more thorough discussion was the association between CHaMP and PNAMP. Several of the deliverables in the proposal will be co-developed with PNAMP. But the role of each organization in producing these deliverables was not clearly described. A paragraph in the introduction that outlines this relationship and some indication under the shared deliverables of roles and responsibilities would have helped to clarify the division of labor. The monitoring work includes tagging, other types of fish population assessment, experimental habitat restoration, long-term habitat trend monitoring, habitat protocol standardization, food web studies, and management of very large datasets. The project does not emphasize identifying limiting factors; rather, it is aimed more toward evaluating the responses of aquatic habitat and fish populations to restoration actions at large spatial scales. Overall, the proposal does a good job of relating ISEMP-sponsored monitoring to other restoration and monitoring projects. The restoration questions being addressed in each of the watersheds are appropriate to the issues believed to be limiting to salmon production. The new CHaMP rotating panel (GRTS) monitoring design appears to hold considerable promise in characterizing habitat status and trends. The project is consistent with the call for expanded RM&E in many subbasin plans and regional programs. This work is relevant to most RM&E efforts basinwide and provides a means by which RM&E programs in diverse subbasins can be unified under a common set of protocols and procedures. The ISRP suggests that pollutants not be overlooked as potentially limiting factors in certain locations. 4. Deliverables, Work Elements, Metrics, and Methods Deliverables, work elements, metrics and methods are adequately described. Some of the details specific to different work elements could be explained more completely (e.g., in what types of habitat will benthic macroinvertebrates be sampled, and why?), but given the very broad scope of the project and the need to cover all the work elements this is understandable. Journal publications are listed in many of the deliverables for different objectives, but to date there have not been very many papers published from the ISEMP work. Hopefully this will change in the near future. What was the rationale for including 25 sample sites in each basin for habitat status and trend monitoring? Did the ISEMP data suggest that this number of sites would provide adequate statistical power? Some support for this number of sites should be provided. It would seem that the number of sample sites required to adequately represent the range of channel types within a basin would vary based on watershed size, variation in topography, geology, land use and other factors. If this is the case, a variable sample size might be more appropriate. The assessment of stable isotope analysis to characterize trophic aspects of habitat condition in the John Day River is a novel element of this habitat monitoring program and could lead to the development of a very valuable tool. Limiting factors have almost exclusively been restricted to physical habitat or water quality attributes, largely because there was no efficient method for assessing food web conditions. One suggestion about the proposed methodology for this work element: the terrestrial invertebrates should not be lumped together for stable isotope analysis. As with the aquatic invertebrates, the terrestrial insects should be grouped on the basis of functional group (detritivores, herbivores, or predators). The deliverable that addresses status and trend monitoring in the Wenatchee actually discusses this activity in the Entiat. In fact, the information for the Entiat is repeated in the second deliverable, which actually does address the Entiat. This discrepancy seems like an inadvertent cut-and-paste error, but the Wenatchee information should be added to the first deliverable. ISEMP suggests, where appropriate, an “experimental approach” where habitat restoration actions in “treatment” streams are compared to reference “untreated” streams. Given the physiographic and biological variability within reaches or tributaries within a subbasin, let alone differences between subbasins, selection of appropriate references and treatments could prove challenging. It probably would be helpful if ISEMP provided guidelines and/or assistance to subbasin investigators for selecting both reference and treatment sites. The same could be said for data analyses. With the large amount of data that will be collected, investigators may need some assistance in data analyses. It is our understanding that ISEMP is planning to provide analytical assistance where needed.