Skip to main content

Deconvolutional paragraph representation learning

Publication ,  Journal Article
Zhang, Y; Shen, D; Wang, G; Gan, Z; Henao, R; Carin, L
Published in: Advances in Neural Information Processing Systems
January 1, 2017

Learning latent representations from long text sequences is an important first step in many natural language processing applications. Recurrent Neural Networks (RNNs) have become a cornerstone for this challenging task. However, the quality of sentences during RNN-based decoding (reconstruction) decreases with the length of the text. We propose a sequence-to-sequence, purely convolutional and deconvolutional autoencoding framework that is free of the above issue, while also being computationally efficient. The proposed method is simple, easy to implement and can be leveraged as a building block for many applications. We show empirically that compared to RNNs, our framework is better at reconstructing and correcting long paragraphs. Quantitative evaluation on semi-supervised text classification and summarization tasks demonstrate the potential for better utilization of long unlabeled text data.

Duke Scholars

Published In

Advances in Neural Information Processing Systems

ISSN

1049-5258

Publication Date

January 1, 2017

Volume

2017-December

Start / End Page

4170 / 4180

Related Subject Headings

  • 4611 Machine learning
  • 1702 Cognitive Sciences
  • 1701 Psychology
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Zhang, Y., Shen, D., Wang, G., Gan, Z., Henao, R., & Carin, L. (2017). Deconvolutional paragraph representation learning. Advances in Neural Information Processing Systems, 2017-December, 4170–4180.
Zhang, Y., D. Shen, G. Wang, Z. Gan, R. Henao, and L. Carin. “Deconvolutional paragraph representation learning.” Advances in Neural Information Processing Systems 2017-December (January 1, 2017): 4170–80.
Zhang Y, Shen D, Wang G, Gan Z, Henao R, Carin L. Deconvolutional paragraph representation learning. Advances in Neural Information Processing Systems. 2017 Jan 1;2017-December:4170–80.
Zhang, Y., et al. “Deconvolutional paragraph representation learning.” Advances in Neural Information Processing Systems, vol. 2017-December, Jan. 2017, pp. 4170–80.
Zhang Y, Shen D, Wang G, Gan Z, Henao R, Carin L. Deconvolutional paragraph representation learning. Advances in Neural Information Processing Systems. 2017 Jan 1;2017-December:4170–4180.

Published In

Advances in Neural Information Processing Systems

ISSN

1049-5258

Publication Date

January 1, 2017

Volume

2017-December

Start / End Page

4170 / 4180

Related Subject Headings

  • 4611 Machine learning
  • 1702 Cognitive Sciences
  • 1701 Psychology