From noise to nature: diffusion models to advance wildlife detection in aerial imagery

Automated wildlife detection in aerial imagery can expand the scale and efficiency of ecological monitoring, but model development is often constrained by limited training data, uneven class representation, and poor coverage of diverse environmental conditions. These constraints are especially acute for rare, elusive, or endangered species, where large, diverse, and well-annotated image datasets may be difficult or impossible to obtain. Synthetic imagery generated by fine-tuned diffusion models could help address these limitations, but its utility for wildlife detection remains insufficiently tested. Here, we evaluated whether diffusion-generated imagery can improve object detection performance in a remote sensing workflow using humpback whales (Megaptera novaeangliae) in drone imagery as a benchmark case study to simulate data limitations relevant to rare or poorly observed species. We fine-tuned Stable Diffusion XL 1.0 using low-rank adaptation on a small independent set of real whale images to generate abundant synthetic aerial imagery, and trained YOLOv8 object detection models using controlled combinations of real and synthetic data. We found that the addition of synthetic imagery improved localization accuracy measured using mAP50–95 when real training data were limited in quantity or diversity, provided little additional benefit when real data were sufficient, and substantially reduced all performance metrics when used alone. These findings suggest that diffusion-generated synthetic imagery can strengthen detection models under data-limited conditions but should complement, rather than replace, real imagery. We conclude by outlining practical applications and key research priorities for integrating synthetic data into wildlife detection workflows.

Duke Scholars

Author David William Johnston Marine Science and Conservation