Skip to main content
Journal cover image

Convolutional neural networks rarely learn shape for semantic segmentation

Publication ,  Journal Article
Zhang, Y; Mazurowski, MA
Published in: Pattern Recognition
February 1, 2024

Shape learning, or the ability to leverage shape information, could be a desirable property of convolutional neural networks (CNNs) when target objects have specific shapes. While some research on the topic is emerging, there is no systematic study to conclusively determine whether and under what circumstances CNNs learn shape. Here, we present such a study in the context of segmentation networks where shapes are particularly important. We define shape and propose a new behavioral metric to measure the extent to which a CNN utilizes shape information. We then execute a set of experiments with synthetic and real-world data to progressively uncover under which circumstances CNNs learn shape and what can be done to encourage such behavior. We conclude that (i) CNNs do not learn shape in typical settings but rather rely on other features available to identify the objects of interest, (ii) CNNs can learn shape, but only if the shape is the only feature available to identify the object, (iii) sufficiently large receptive field size relative to the size of target objects is necessary for shape learning; (iv) a limited set of augmentations can encourage shape learning; (v) learning shape is indeed useful in the presence of out-of-distribution data.

Duke Scholars

Published In

Pattern Recognition

DOI

ISSN

0031-3203

Publication Date

February 1, 2024

Volume

146

Related Subject Headings

  • Artificial Intelligence & Image Processing
  • 4611 Machine learning
  • 4605 Data management and data science
  • 4603 Computer vision and multimedia computation
  • 0906 Electrical and Electronic Engineering
  • 0806 Information Systems
  • 0801 Artificial Intelligence and Image Processing
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Zhang, Y., & Mazurowski, M. A. (2024). Convolutional neural networks rarely learn shape for semantic segmentation. Pattern Recognition, 146. https://doi.org/10.1016/j.patcog.2023.110018
Zhang, Y., and M. A. Mazurowski. “Convolutional neural networks rarely learn shape for semantic segmentation.” Pattern Recognition 146 (February 1, 2024). https://doi.org/10.1016/j.patcog.2023.110018.
Zhang Y, Mazurowski MA. Convolutional neural networks rarely learn shape for semantic segmentation. Pattern Recognition. 2024 Feb 1;146.
Zhang, Y., and M. A. Mazurowski. “Convolutional neural networks rarely learn shape for semantic segmentation.” Pattern Recognition, vol. 146, Feb. 2024. Scopus, doi:10.1016/j.patcog.2023.110018.
Zhang Y, Mazurowski MA. Convolutional neural networks rarely learn shape for semantic segmentation. Pattern Recognition. 2024 Feb 1;146.
Journal cover image

Published In

Pattern Recognition

DOI

ISSN

0031-3203

Publication Date

February 1, 2024

Volume

146

Related Subject Headings

  • Artificial Intelligence & Image Processing
  • 4611 Machine learning
  • 4605 Data management and data science
  • 4603 Computer vision and multimedia computation
  • 0906 Electrical and Electronic Engineering
  • 0806 Information Systems
  • 0801 Artificial Intelligence and Image Processing