Transformers For Recognition In Overhead Imagery: A Reality Check

Conference Paper

There is evidence that transformers offer state-of-the-art recognition performance on tasks involving overhead imagery (e.g., satellite imagery). However, it is difficult to make unbiased empirical comparisons between competing deep learning models, making it unclear whether, and to what extent, transformer-based models are beneficial. In this paper we systematically compare the impact of adding transformer structures into state-of-the-art segmentation models for overhead imagery. Each model is given a similar budget of free parameters, and their hyperparameters are optimized using Bayesian Optimization with a fixed quantity of data and computation time. We conduct our experiments with a large and diverse dataset comprising two large public benchmarks: Inria and DeepGlobe. We perform additional ablation studies to explore the impact of specific transformer-based modeling choices. Our results suggest that transformers provide consistent, but modest, performance improvements. We only observe this advantage however in hybrid models that combine convolutional and transformer-based structures, while fully transformer-based models achieve relatively poor performance.

Full Text

Duke Authors

Cited Authors

  • Luzi, F; Gupta, A; Collins, L; Bradbury, K; Malof, J

Published Date

  • January 1, 2023

Published In

  • Proceedings 2023 Ieee Winter Conference on Applications of Computer Vision, Wacv 2023

Start / End Page

  • 3767 - 3776

International Standard Book Number 13 (ISBN-13)

  • 9781665493468

Digital Object Identifier (DOI)

  • 10.1109/WACV56688.2023.00377

Citation Source

  • Scopus