Qualifications:

The ISRP recommends that the proponents describe their responses to the ISRP's comments and suggestions below in their upcoming annual report covering FY 2019 accomplishments.

1.      Establish quantitative objectives, guiding hypotheses and timelines for meeting key activities that can be reviewed by the ISRP.

2.      Provide an Adaptive Management (AM) process description for ISRP review. The AM process should include quantitative, time bound objectives and a more comprehensive description of how implementation and effectiveness monitoring will be aligned to improve project use and efficiency.

3.      Develop a strategic framework to guide project direction and priorities to move toward longer term project goals and, as well, a description of the major activities and skills required.

4.      Provide evidence regarding the database's level of use. The ISRP would like to be confident that the project is being widely used and that it is improving restoration actions and outcomes.

Comment:

The project appears to be well focused on providing comprehensive and efficient data management. The primary user is the Kalispel Tribe, but there are a number of partner agencies providing funding to support their use of the data. The proponents describe long term goals as well as information on who and how it will be used, but the information is not supported by a strategic plan or a list of major activities that should occur over the next several years.

1. Objectives, Significance to Regional Programs, and Technical Background

The primary goal is to assist in coordinating resource management strategies by addressing the status of resident fish, wildlife, and their habitats using a centralized data repository. There is a desire to publicly disseminate project and resource information, so that independent parties can "rapidly access primary or secondary data types, research metadata, and download reports related to data sets." Also, a long-term goal is identified as the project functioning as "a decision support system, to guide natural resource management managers in their strategies and activities." Unfortunately, the latter goal statement was confusing to the ISRP because it was not clear how the database addresses it, especially since there is no long-term strategy or action plan in place to guide actions or decisions over the long term.

There are six detailed and quite clearly stated objectives. They are qualitative and have no specific time frames for completion. A major plus is that each objective has a description describing metrics/criteria for measuring successful accomplishment. As previously noted, there are no objectives linked to longer term goals for the project. An example is that there is a stated desire to have the data system "be a place to disseminate natural resource information to the public and act as a data repository for future wildlife and fisheries managers," but there are no objectives focused on user recruitment, marketing of services to potential partner agencies or publics, or measuring current user satisfaction and desires for specific products and services. There has been tracking of actual use, but no description of how that information is used to guide management or decision-making.

Regarding regional significance, the proponents list a number of programs for which the database has relevance, but the details of how it contributes to the programs are more assumed and not clearly documented. Regional significance appears limited to the non-anadromous areas of the Basin. That is not necessarily a problem, but the ISRP is curious if some of the data sets housed within the database have wider utility. If so, linkages to other Columbia Basin databases (e.g., StreamNet, PNAMP, and so forth) are not readily apparent.

2. Results and Adaptive Management

The project appears to be meeting the near term needs of the Kalispell Tribe, yet there is only a limited description of actual accomplishments over the last four to five-years. Also, there is no summary of major accomplishments from inception to the present, which would be useful for evaluating the project. There are biannual accomplishment reports provided but they are narrowly focused on the accomplishment of specific objectives.

Although there is some tracking of project effectiveness, there is no evaluation or identification of lessons learned to guide future project direction and operations. As well, there is no mention of adaptive management (AM) or that it is an important project component. The proponents portray that project effectiveness is measured in several ways and that the range of users, including the public, that submit and maintain data sets will reflect project effectiveness through operational usage. Currently there is tracking of "the volume of data sets uploaded by licensed users, the number of data sets downloaded by the general public and added a hit counter at the web interface level." There is no discussion as to trends in use over the last few years or whether this use appears to be meeting user needs. Further, how are data on number of users and dataset volumes downloaded being used to improve the project? The proponents note that they receive and respond to feedback, but it is not clear from whom or that this feedback process is structured in a way that leads to more than an ad hoc set of edits.

Based on the Explanation of Financial History, it appears that the project has made some minor additions to the functionality of the website. Have any additional modifications been made? For instance, the proponents state that they were tracking project effectiveness as the number of users and the volume of datasets uploaded and downloaded. However, none of those results was presented aside from stating that the database is the repository for 60 datasets. Are these tracking data being used to modify the website? Who is using these data and how? What new insights or restoration plans/actions have been developed as a result?

Based on the workflow for submitting and providing QA/QC of datasets, it seems that substantial personnel time is required to get datasets verified and uploaded. The database would benefit from implementing algorithms that require less human attention to have datasets formatted, documented, and uploaded. Related to this, what is the long-term outlook for the database? Will it need this level of support in perpetuity?

Some of the issues mentioned above suggest the proponents need to pause and carefully think about the design and function of their database. The proponents were honest about their lack of success with the former database (JSAP DB), which was scrapped in 2009. However, the design of the current database also seems to have some inefficiencies, and the ISRP feels that more thought could be given to database design and longevity. For instance, no threats to program investments and the project are mentioned. For this database to have the maximum utility and impact, especially given its ambitious goals, the objectives for and design of the database need to be strategic.

3. Methods: Project Relationships, Work Types, and Deliverables

The proponents list many projects for which this database provides an information repository, 60% of which are BPA-funded projects.

The proposed work and deliverables include some relatively minor modifications to the database, as well as the ongoing activities needed to maintain it. The proposal states that, "Future versions will incorporate field survey report documents, data analysis tools, and summary statistics information." While this functionality is potentially useful, it is not explained well enough for the ISRP to understand its merit or limitations, or even if it will be incorporated in this project.

The ISRP would like to understand why the work under Deliverable 1 (Provide templates for downloading primary datasets into the Intermountain Province GEDMS) needs to be subcontracted rather than done "in-house."

The ISRP has a concern about Deliverable 2 (p. 15). Will the public care about primary data? If resource managers want to share what they are learning about the environment, habitat, and focal species progress within their jurisdiction, the ISRP suspects that they will probably find that analyzed (and interpreted) results are more useful. Are there provisions to see that these are done and provided to the public? Perhaps some links to published analyses using these data would be helpful to the public, especially those wondering what has been learned or accomplished?

The proponents anticipate (p. 16) that within the FY 2019-2023 timeframe additional equipment needs to be acquired. This likely will include a data server, desktop or laptop computers, and additional software. This may also include having to rent facilities for housing the equipment. Were these potential expenditures in the proposed budget?