Abstract
In natural resource management, reliable monitoring data is necessary to make informed management decisions. Multiple observers collecting data for a monitoring program can introduce variance due to between-observer differences which reduces data reliability. We introduce an application of mixed-effects models with relaxed homogeneous variance assumptions (called heterogeneous-variance mixed-effects models) to quantify between-observer variance in natural resource monitoring programs with large spatial and temporal extents. We used monitoring data from the Bureau of Land Management's Assessment, Inventory, and Monitoring program to describe and demonstrate the use of the method. In two example analyses, we identified differences in observer variance across regions and years and associated the differences in these examples with features of the AIM monitoring program. These examples illustrate several potential uses of the models which include calculating the magnitude of observer variance to guide indicator selection and appropriate indicator estimate calculations, determining appropriate data aggregation and comparison, understanding the influences of changes to a monitoring program on observer variance, identifying previously unknown influences on observer variance, and suggesting practices or further changes to monitoring programs to reduce observer variance in future data collection efforts. This statistical technique requires a dataset with a large number of samples and observers and is therefore best suited to large monitoring programs. Reliable metadata about observers and the program history is also required to build the model and interpret the results. The model could also be expanded to answer additional questions about the influence of observers on monitoring data.
•Observer variance can reduce the quality of natural resource monitoring data.•We propose heterogeneous-variance mixed-effects models to quantify this variance.•An important model application is identifying sources of observer variance.•These results can help direct variance reduction strategies in monitoring programs.•The models can also guide appropriate data use and parameter estimation.