See programmatic comments on coordination projects. A sound scientific proposal should respond to the six questions and related material at the beginning of the regional coordination section.

The proposal is missing an opportunity to take a more systematic approach to coordination: to think about what the sponsors are really trying to achieve, how they will know if they are achieving it, and how they will adapt to changing circumstances or proactively test new approaches and learn from the outcomes. In several places the statement, “WDFW will monitor and report …” is used. What actually will be monitored? How will the variables monitored be measured? Are there hypotheses about relations between these variables?

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives
The proposed work includes data management (storage, management, and reporting) - 10%, monitoring and evaluation (framework and approach) - 10%, developing and tracking biological objectives - 10%, review of technical documents and processes - 10%, project proposal review - 10%, coordination of projects, programs and funding sources within subbasins - 20%, facilitating and participating in focus workgroups on Program issues - 20%, and information dissemination (technical, policy, and outreach) - 10%.

Significance to regional programs: A lengthy statement lists and describes various projects, workshops, programs and fora that WDFW participates in as part of the implementation of the Fish and Wildlife Program through the research plan, subbasin plans, and MERR plan. Can outcomes from these activities be identified?

Problem statement: This describes the 2010 separation from CBFWA and the need to continue funding for the individual states to continue to participate in the implementation of the Fish and Wildlife Program through the coordination and information transmission that was formerly accomplished through CBFWA. A statement of the problem to be addressed is contained in the first paragraph of the "significance to regional programs" section.

Objectives: The project has specific four objectives. Each objective is stated as a task rather than as a desired outcome.

In general, the Washington Department of Fisheries and Wildlife is one of the active participants in the process of conserving and restoring Columbia Basin fish and wildlife that are affected by the building and operation of the hydro system. “WDFW provides the technical and scientific expertise needed to address mitigation issues related to fish and wildlife management in the Columbia River Basin.” How will the outcomes associated with the four project objectives be measured?

2. History: Accomplishments, Results, and Adaptive Management (ISRP Review of Results)

Explanation of performance: short statement indicating that deliverables have been on schedule.

Major accomplishments: WDFW has participated in the formation of a tri-state coordination effort and continued participation in various fora. Participation in a variety of meetings is enumerated. The proposed plan of work offers few specifics. “Past accomplishments” lists meetings, conference calls, and briefings. This is a good sample to provide information on outcomes and what coordination activities lead to better outcomes.

Management Change: The statement is made, “It should involve stating hypotheses then implementing actions, monitoring, reporting, and evaluating outcomes to provide a clear sequential structure to decisions required in the continuing evolution and implementation of the Program." Expanding this statement into hypotheses for the plan of work would be desirable.

Adaptive Management: An 8-step process for coordination is identified. A reference should be made to BPA [Bonneville Power Administration]. 1997. Wildlife mitigation program final environmental impact statement. DOE/EIS - 0246. U.S. Department of Energy, Portland, OR., “Each of the following eight steps is required to support a transparent, accountable, and effective planning, implementation and evaluation process.” What is presented is a general statement of the principles of adaptive management, without specific indication of how the project will incorporate it in its operations. Could this process be filled in for one of the key coordination issues?

ISRP Retrospective Evaluation of Results

This is a new project, so technically there are no results to evaluate. Historical data on performance is available with the project, “Proposal RESCAT-1989-062-01 - Program Coordination and Facilitation Services provided through the Columbia Basin Fish and Wildlife Foundation.” See the section, “Reporting & Contracted Deliverables Performance.”

Financial performance and history: “Beginning April 1, 2011, WDFW began coordination as an entity independent of CBFWA.” This project has no financial history or review of progress. Previous work was completed under the management of the Columbia Basin Fish and Wildlife Foundation. No explanation of financial performance or financial history is provided.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (hatchery, RME, tagging)

Project relationships: A list of related coordination projects is provided without explanation of the specific types of interaction.
Emerging limiting factors: A brief description of strategies employed to address climate change and invasive species, with links to references. Are there regional coordination limiting factors?

Geographic interests: The Columbia Basin and specifically for the State of Washington overall, which encompass the Lower Columbia, Middle Columbia, Upper Columbia, and Snake River Recovery areas.

Value-added: The proposed plan of work offers few specifics. The list of 85 meetings, conference calls, and briefings is a good sample to provide information on outcomes and what coordination activities lead to better outcomes. The value-added describes just a couple of outcomes. Can coordination outcomes be related to "improving the status of fish and wildlife resources in the Columbia Basin"? “Regional coordination includes but is not limited to participation in various forums, technical committees, and workgroups associated with implementing the MERR Plan, Subbasin Plans, and the Council's Research Plan.” These are essentially inputs to coordination. What are the outcomes from these regional coordination activities? How do they improve the status of fish and wildlife resources? How, for example, can the statement, “Increase the efficiency and effectiveness of RM&E efforts by facilitating communication and coordination among project sponsors and funding agencies within the Basin,” be measured and documented? Listed are some 30 statements that might be converted into hypotheses or used as cases for testing hypotheses. What are the most effective ways of organizing and coordinating? What types of coordination activities work best? What percentage of the meetings is facilitated and does this improve outcomes? Does coordination provide value? How would the value of coordination be measured and compared against the costs?
Deliverable 3 is about documentation. Where are the specifics of documentation addressed?

4. Deliverables, Work Elements, Metrics, and Methods

Deliverables: The project has six deliverables related to the objectives. These include documentation of participation in meetings and types of coordination, and the preparation of reports. A summary list of meetings attended, coordination activities performed and contributions to documents is enumerated. It would be desirable to develop metrics about outcomes and measures of outputs from coordination activities.

Methods and metrics: These are described in greater detail in the "objectives" section than in the "deliverables" section because the objectives are written as tasks. Not much detail is presented on how these activities will be accomplished or metrics to be used to evaluate performance.

Work elements: Two work elements are identified – 122. Provide Technical Review and 189. Coordination-Columbia Basinwide for which no metrics are associated. Can metrics be identified to go with these work elements? Ideally, the hypothesis(es) developed in the proposal would be measured during the course of the coordination activities and results presented in the report on this project. There are many ideas discussed in the proposal that are amenable to this approach. Selecting a few of the most important questions, concerns, or hypotheses and monitoring them is recommended.

The program coordination part of the work elements section provides more specific enumeration of coordination activities. Value added by these activities is generally referenced in the case of fish passage and lamprey recovery but without specific examples of the contribution of WDFW staff. The same comment applies to opportunities to evaluate performance – it is presented as a possibility but without a specific plan.

4a. Specific comments on protocols and methods described in MonitoringMethods.org

The protocols for the two work elements are published but do not provide adequate guidance on the methods and metrics. Guidance is available from ISRP (2007-14:2). Project sponsors can strengthen the science in proposals by developing methods and metrics for the most important project objectives.

Modified by Dal Marsters on 4/17/2012 3:04:04 PM.