Skip to main content

Analyzing animal behavior via classifying each video frame using convolutional neural networks.

Publication ,  Journal Article
Stern, U; He, R; Yang, C-H
Published in: Sci Rep
September 23, 2015

High-throughput analysis of animal behavior requires software to analyze videos. Such software analyzes each frame individually, detecting animals' body parts. But the image analysis rarely attempts to recognize "behavioral states"-e.g., actions or facial expressions-directly from the image instead of using the detected body parts. Here, we show that convolutional neural networks (CNNs)-a machine learning approach that recently became the leading technique for object recognition, human pose estimation, and human action recognition-were able to recognize directly from images whether Drosophila were "on" (standing or walking) or "off" (not in physical contact with) egg-laying substrates for each frame of our videos. We used multiple nets and image transformations to optimize accuracy for our classification task, achieving a surprisingly low error rate of just 0.072%. Classifying one of our 8 h videos took less than 3 h using a fast GPU. The approach enabled uncovering a novel egg-laying-induced behavior modification in Drosophila. Furthermore, it should be readily applicable to other behavior analysis tasks.

Duke Scholars

Altmetric Attention Stats
Dimensions Citation Stats

Published In

Sci Rep

DOI

EISSN

2045-2322

Publication Date

September 23, 2015

Volume

5

Start / End Page

14351

Location

England

Related Subject Headings

  • Software
  • Sexual Behavior, Animal
  • Pattern Recognition, Automated
  • Neural Networks, Computer
  • Machine Learning
  • Image Interpretation, Computer-Assisted
  • Image Enhancement
  • Drosophila
  • Behavior Observation Techniques
  • Animals
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Stern, U., He, R., & Yang, C.-H. (2015). Analyzing animal behavior via classifying each video frame using convolutional neural networks. Sci Rep, 5, 14351. https://doi.org/10.1038/srep14351
Stern, Ulrich, Ruo He, and Chung-Hui Yang. “Analyzing animal behavior via classifying each video frame using convolutional neural networks.Sci Rep 5 (September 23, 2015): 14351. https://doi.org/10.1038/srep14351.
Stern, Ulrich, et al. “Analyzing animal behavior via classifying each video frame using convolutional neural networks.Sci Rep, vol. 5, Sept. 2015, p. 14351. Pubmed, doi:10.1038/srep14351.

Published In

Sci Rep

DOI

EISSN

2045-2322

Publication Date

September 23, 2015

Volume

5

Start / End Page

14351

Location

England

Related Subject Headings

  • Software
  • Sexual Behavior, Animal
  • Pattern Recognition, Automated
  • Neural Networks, Computer
  • Machine Learning
  • Image Interpretation, Computer-Assisted
  • Image Enhancement
  • Drosophila
  • Behavior Observation Techniques
  • Animals