Abstract
In natural resources, monitoring (i.e., systematic data collection over time to detect changes) is critical for making informed management decisions about resources of interest. To be useful for making decisions, monitoring data needs to reliably depict the true conditions of resources of interest. Error (i.e., the deviation of an estimated value for a sample from the population parameter) can influence the reliability of data. In this dissertation, I highlight and address several different topics related to error in natural resource monitoring which have previously been underexplored. I present a comprehensive classification of potential sources of error. The classification describes 19 potential sources of error and 4 effects which can influence how error is contributed to estimates of indicators in monitoring programs. A full description of each potential source of error is given including a definition and potential causes, influences, and reduction strategies. The classification organizes potential sources of error by the stage of a monitoring program (design, collection, post-collection, and post-monitoring) and if they influence how well the population of interest is represented or the construct (i.e., resource) of interest is measured. The main goals of this classification are to bring awareness to the many different potential sources of error that can be contributed throughout a natural resource monitoring program and to encourage thoughtful design (or modification) of a monitoring program to mitigate contributions of different sources of error.
The remaining chapters focus on potential sources of error that occur during the implementation of a field-based monitoring program. Multiple observers collecting data for a monitoring program can contribute error due to between-observer differences. I first describe and demonstrate an application of mixed-effects models with relaxed homogeneous variance assumptions (called heterogeneous-variance mixed-effects models) to quantify between-observer variance in natural resource monitoring programs with large spatial and temporal extents. Two case studies from the Bureau of Land Management’s (BLM) terrestrial Assessment Inventory and Monitoring (AIM) program illustrate several potential uses of these models. These uses include calculating the magnitude of observer variance to guide indicator selection and appropriate indicator estimate calculations, determining appropriate data aggregation and comparison, understanding the influences of changes to a monitoring program on observer variance, identifying previously unknown influences on observer variance, and suggesting practices or further changes to monitoring programs to reduce observer variance in future data collection efforts.
Observer differences are also explored with a paradigm to describe how and why observers implement methods differently during data collection and a process to modify a program to reduce method implementation problems. First, I define method variations (i.e., implementation of a step of a method that differs from the intended protocol) and deviations (i.e., method variations that result in a different measured or observed value than the intended protocol) and describe five different types of method variations and deviations (ambiguity, miscommunication, drift, honest mistakes, and apathy). Each method variation and deviation type is distinguished by having different underlying causes and requiring different strategies to mitigate the occurrence. Second, I propose a process to evaluate variations and deviations for methods used within a data collection program. The goal of the process is to identify the possible variations for each step of a method, assess which deviations are most problematic, and strategically revise the program to tackle problematic deviations. Continued evaluation of data collection programs and reporting findings and decisions from all phases of the process are encouraged. This framework was exemplified by an evaluation of the BLM terrestrial AIM program. The overall goal of this paradigm and process is to understand and mitigate problems during the implementation of methods to standardize data collection and reduce contributions to error.
Lastly, I discuss the implications of mistakes that occur while implementing the collection and post-collection phases of a monitoring program. I describe and demonstrate how not all mistakes contribute to error and how not all error will lead to inappropriate monitoring decisions. Given clearly defined monitoring objectives, the potential impact of different mistakes can be evaluated, and decisions can be made on the best ways to allocate resources to reduce problematic mistakes and produce useful data.