Long-term prediction of μeCOG signals with a spatio-temporal pyramid of adversarial convolutional networks


Conference Paper

© 2018 IEEE. Video prediction into sufficiently long future has many potential applications. Modeling long-term dynamics for times series is challenging with convolution neural network structure, which is usually good for capturing short-term dependencies. In this work, we propose to embed the convolutional neural network within a spatial-temporal pyramid structure, to exploit both long-term and short-term temporal dependency and capture both macro-scale and micro-scale spatial structures. The prediction at a given scale is conditioned on the features extracted from a lower scale and past observations from the current scale. In order to overcome the blurry issue caused by the mean square error loss, we add a critic model with Wasserstein distance based adversarial loss to complement MSE. We compare our spatio-temporal pyramid model against a single scale convolution network as well as a model with multiple spatial scales only, and demonstrate that our pyramid structure performs better for predicting up to 24 future frames.

Full Text

Duke Authors

Cited Authors

  • Wang, R; Song, Y; Wang, Y; Viventi, J

Published Date

  • May 23, 2018

Published In

Volume / Issue

  • 2018-April /

Start / End Page

  • 1313 - 1317

Electronic International Standard Serial Number (EISSN)

  • 1945-8452

International Standard Serial Number (ISSN)

  • 1945-7928

International Standard Book Number 13 (ISBN-13)

  • 9781538636367

Digital Object Identifier (DOI)

  • 10.1109/ISBI.2018.8363813

Citation Source

  • Scopus