Automatic emotion and attention analysis of young children at home: a ResearchKit autism feasibility study.

Journal Article (Journal Article)

Current tools for objectively measuring young children's observed behaviors are expensive, time-consuming, and require extensive training and professional administration. The lack of scalable, reliable, and validated tools impacts access to evidence-based knowledge and limits our capacity to collect population-level data in non-clinical settings. To address this gap, we developed mobile technology to collect videos of young children while they watched movies designed to elicit autism-related behaviors and then used automatic behavioral coding of these videos to quantify children's emotions and behaviors. We present results from our iPhone study Autism & Beyond, built on ResearchKit's open-source platform. The entire study-from an e-Consent process to stimuli presentation and data collection-was conducted within an iPhone-based app available in the Apple Store. Over 1 year, 1756 families with children aged 12-72 months old participated in the study, completing 5618 caregiver-reported surveys and uploading 4441 videos recorded in the child's natural settings. Usable data were collected on 87.6% of the uploaded videos. Automatic coding identified significant differences in emotion and attention by age, sex, and autism risk status. This study demonstrates the acceptability of an app-based tool to caregivers, their willingness to upload videos of their children, the feasibility of caregiver-collected data in the home, and the application of automatic behavioral encoding to quantify emotions and attention variables that are clinically meaningful and may be refined to screen children for autism and developmental disorders outside of clinical settings. This technology has the potential to transform how we screen and monitor children's development.

Full Text

Duke Authors

Cited Authors

  • Egger, HL; Dawson, G; Hashemi, J; Carpenter, KLH; Espinosa, S; Campbell, K; Brotkin, S; Schaich-Borg, J; Qiu, Q; Tepper, M; Baker, JP; Bloomfield, RA; Sapiro, G

Published Date

  • 2018

Published In

Volume / Issue

  • 1 /

Start / End Page

  • 20 -

PubMed ID

  • 31304303

Pubmed Central ID

  • PMC6550157

Electronic International Standard Serial Number (EISSN)

  • 2398-6352

Digital Object Identifier (DOI)

  • 10.1038/s41746-018-0024-6


  • eng

Conference Location

  • England