IMPLEMENTING CONFIDENCE THRESHOLDS TO ENABLE LARGE SCALE BIODIVERSITY MONITORING ACROSS CALIFORNIA
Ian A Dwight; California Department of Fish and Wildlife; ian.dwight@wildlife.ca.gov; Patrick T. Freeman, Justin P. Suraci, Lindsey N. Rich
Automated bioacoustic classifiers like BirdNET, which process sound data and identify vocalizing taxa to species, require some manual validation of classifier-based predictions. However, manual validation becomes time prohibitive as the scale of monitoring increases. To address this challenge, we leveraged over two decades of acoustic monitoring datasets and calculated species-specific ‘thresholds’ that balance the inclusion of false positives with the omission of false negatives. BirdNET predictions with confidence scores below the threshold are removed, eliminating the need for additional manual validation. We evaluated trade-offs in precision, recall, and the sensitivity of null occupancy estimates to a range of threshold values applied to BirdNET predictions for 272 bird species statewide. Thresholds were based on 196,925 annotated predictions, from which we derived 2,403 ecoregion-specific and 761 statewide logistic regression-based thresholds across four levels of certainty. Precision improved and recall declined as threshold certainty increased. This trade-off improved when using ecoregion-specific versus statewide thresholds. Occupancy estimates decreased with higher certainty thresholds, but post-hoc filtering techniques such as requiring two detections per sampling occasion reduced this sensitivity. These results offer a practical framework for applying detection thresholds to the growing volume of acoustic data collected across California, supporting the implementation of statewide biodiversity monitoring.
Avian Conservation and Change Across Landscapes 
Wednesday 2:45 PM
 

Speaker Bio:
Ian Dwight is a Senior Environmental Scientist with the California Environmental Monitoring and Assessment Framework in the Department of Fish and Wildlife. He earned his undergraduate and graduate degrees at the University of California, Davis, and has over 13 years of experience in wildlife research and management, including four years with CDFW. Although his career has centered on gamebird conservation, he now focuses on developing and implementing biodiversity monitoring approaches for a wide range of taxa across California.