The proposal should include 1) a better statement of objectives by separating them from tasks and deliverables to word them as outcomes; 2) a description of what and how work will be done; and 3) a description of how activities will be monitored and evaluated for effectiveness.
The proposal provides lengthy descriptions of the coordination needs of the USRT, the past history of the project, and the limiting factors facing the coordination. It presents far less detail on specifically how the project would address the stated need, and how it would measure the degree of its effectiveness.

1. Purpose: Significance to Regional Programs, Technical Background, and Objectives

The proposed work has seven components: 1. data management (10%); 2. monitoring and evaluation (10%); review of technical documents and processes (6-10%); project proposal review (6-10%); coordination and development of projects (20%); facilitation and participation in workgroups (20%); information dissemination (20%). Some activities are stated as being contingent on the budget increase to add an assistant director. The budget request does not make a strong case for why additional personnel are needed to perform the coordination tasks described and for the expense estimated.

Significance to regional programs: The statement makes reference to tribal coordination and its relation to the implementation of 2008 FCRPS BiOp RPAs and the 2009 Fish and Wildlife Program, in particular, its coordination provisions. It also cites the relationship to the LSRCP and several other regional programs. The Fort McDermitt Paiute-Shoshone Tribe joined the CBFWA in 2011. Because of USRT problems with its previous executive director, this is essentially a new project.
The Burns Paiute Tribe, Shoshone-Bannock Tribes, and Shoshone-Paiute Tribes, and the Fort McDermitt Paiute-Shoshone Tribe make up the membership of the Upper Snake River Tribe Coordination (USRT). The Fort McDermitt Paiute-Shoshone Tribe is an addition to USRT with this proposal.

Problem statement: A very detailed problem statement begins with a description of the USRT goal "to facilitate Tribal unity to protect and nurture all Compacting Tribes’ rights, languages, cultures and traditions in addressing issues related to the Upper Snake River Basin." This is followed by a history of Northwest Power Act implementation, the early role of the tribes in the Fish and Wildlife Program, and the tribes' eventual development of the USRT compact to better represent their collective interests. A good case is made for a strong need to coordinate among individual USRT member tribes that are dispersed over a large area, and for the benefits to members of having a collective voice. The problem statement also acknowledges the ISRP document identifying the need for output and impact metrics.

Objectives: The proposal has four objectives. Each of the objectives is worded as a task rather than as identifying desired outcomes. A short list of activities accompanies each objective. Proposed objectives seek to provide technical assistance and coordinate regarding fish, wildlife, and habitat; land, water, and air; cultural resources; and federal trust responsibility. The objectives will be accomplished through such deliverables as USRT commission meetings, policy decision documents, information sharing, assessments of fish and wildlife losses, regional coordination, contract administration and reporting, and outreach and education.

Emerging limiting factors: The statement notes the historical vulnerability of indigenous people to climate change and argues that holistic management approaches developed over time to address environmental variability supports the need for tribal sovereignty in management and the value of tribal approaches to regional adaptation to climate change. They argue for greater tribal participation in climate change policies.

2. History: Accomplishments, Results, and Adaptive Management (ISRP Review of Results)

Financial performance and history: The project's budget since 2008 is presented. The project gets cost share from member tribes and the BIA. The financial history described actions taken to recover from past accounting irregularities and the implementation of better practices including services of a CPA, a financial policy, regular financial reporting, and other monitoring practices. The existing budget is considered by the sponsors to be inadequate to coordination needs; an increase is requested.

Performance: Recaps the history of financial problems and a high staff turnover rate. Reports have been completed but not by reporting deadlines. The statement indicates that with the hire of a new Executive Director the situation is stabilizing but sees timely reporting as contingent on receiving the requested increase in funding to be able to hire an assistant director.

Adaptive management: The proposal describes several management actions taken to improve coordination activities that demonstrate learning from experience and experimentation with new practices for the purpose of improving performance. These include rotating locations of intertribal coordination meetings, formation of an internal technical work group, and beginning to address data consistency issues.

ISRP Retrospective Evaluation of Results

The project financial history goes back to 2008. USRT has put into place many financial controls to prevent shortfalls in future budgets. During 2011 USRT members had to reallocate coordination funds to support USRT operations. USRT has not completed reports in a timely fashion due to patterns of the first USRT Executive Director, who was terminated for cause. Currently 100% of reports are completed. The new Executive Director has been extensively evaluated.

The proposal presents a very informative discussion of USRT’s history and does an excellent job of assessing the problems USRT has faced and the actions taken to correct these problems.

USRT is being funded by the Environmental Protection Agency to establish an environmental program that will coordinate tribal actions related to climate change.

3. Project Relationships, Emerging Limiting Factors, and Tailored Questions for Type of Work (hatchery, RME, tagging)

Project relationships: the statement provides a long list of BPA-funded projects conducted by member tribes and coordinated through the USRT. It also states the intent to closely link to the CBFWA coordination project. Can the effectiveness of regional coordination in these activities be evaluated?

Regional coordination focus: The geographic location of USRT members is the Upper Snake River and Great Basin. USRT is interested in the Fish and Wildlife Program for the Columbia Basin.

Tailored questions: a detailed description of projects that address issues surrounding the restoration of resident fish.

The proposal suggests that tribal knowledge, practices, and “long-term experience of holistically managing change may be what is needed to base climate change management decisions on.” Would a worthwhile coordination activity under outreach and education be to bring the EPA tribal communities website, Indigenous Peoples Climate Change Assessment Initiative, and Institute for Tribal Environmental Professionals activities to basin decision makers?

4. Deliverables, Work Elements, Metrics, and Methods

Deliverables: The project has seven deliverables. A brief description accompanies each. The deliverables duplicate the objectives, so each deliverable is related to a specific objective. All deliverables are associated with work done by the Executive Director and requested assistant director.

The project sponsor should consider a research plan to evaluate how outreach and education outcomes are observed and measured? Who are the key individuals and groups to be reached? What are the outreach and educational goals, methods to be used, and expected outcomes?

A list of positive accomplishments includes attendance at various regional meetings, hosting a workshop for Columbia River Tribes, and contribution to various regional processes. Can outcomes from these activities be identified and measured?

Seven work elements are identified – 99. Outreach and Education, 114. Identify and Select Projects, 115. Produce Inventory or Assessment, 122. Provide Technical Review, 174. Produce Plan, 189. Coordination-Columbia Basinwide, and 191. Watershed Coordination. Only 99 has metrics, but they are more inputs rather than outcomes. Can output metrics be identified to go with these work elements? Ideally, the hypothesis(es) developed in the proposal would be measured during the course of the coordination activities and results presented in the report on this project. There are many ideas discussed in the proposal that are amenable to this approach. Selecting a few of the most important questions, concerns, or hypotheses and monitoring them is recommended.

Methods and metrics: methods of coordination are provided throughout the document in brief descriptions of objectives, deliverables, and accomplishments. The methods consist of meeting attendance, document development, and coordination and presentations. The proposal associates no metrics with any of the deliverables.

The statement is made that "The effectiveness this work will be monitored following the Independent Scientific Review Panel Memorandum (ISRP 2007-14) which provided NPCC input on evaluation of regional coordination projects." A plan detailing the measurement and evaluation approach should be included in the proposal.

Value added: The statement "Facilitation and coordination of USRT assists Council and BPA in achieving Fish and Wildlife Program objectives in a cost effective manner" is about value-added. Can specific examples of the value added and cost-effectiveness be provided?

4a. Specific comments on protocols and methods described in MonitoringMethods.org

The protocols for the seven work elements are published but do not provide adequate guidance on the methods and metrics. Guidance is available is from ISRP (2007-14:2). The project sponsors can strengthen the science in proposals by developing methods and metrics for the most important project objectives.