Skip to main content

Towards generating long and coherent text with multi-level latent variable models

Publication ,  Conference
Shen, D; Celikyilmaz, A; Zhang, Y; Chen, L; Wang, X; Gao, J; Carin, L
Published in: ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference
January 1, 2020

Variational autoencoders (VAEs) have received much attention recently as an end-to-end architecture for text generation with latent variables. However, previous works typically focus on synthesizing relatively short sentences (up to 20 words), and the posterior collapse issue has been widely identified in text-VAEs. In this paper, we propose to leverage several multi-level structures to learn a VAE model for generating long, and coherent text. In particular, a hierarchy of stochastic layers between the encoder and decoder networks is employed to abstract more informative and semantic-rich latent codes. Besides, we utilize a multi-level decoder structure to capture the coherent long-term structure inherent in long-form texts, by generating intermediate sentence representations as high-level plan vectors. Extensive experimental results demonstrate that the proposed multi-level VAE model produces more coherent and less repetitive long text compared to baselines as well as can mitigate the posterior-collapse issue.

Duke Scholars

Published In

ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

ISBN

9781950737482

Publication Date

January 1, 2020

Start / End Page

2079 / 2089
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Shen, D., Celikyilmaz, A., Zhang, Y., Chen, L., Wang, X., Gao, J., & Carin, L. (2020). Towards generating long and coherent text with multi-level latent variable models. In ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference (pp. 2079–2089).
Shen, D., A. Celikyilmaz, Y. Zhang, L. Chen, X. Wang, J. Gao, and L. Carin. “Towards generating long and coherent text with multi-level latent variable models.” In ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, 2079–89, 2020.
Shen D, Celikyilmaz A, Zhang Y, Chen L, Wang X, Gao J, et al. Towards generating long and coherent text with multi-level latent variable models. In: ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. 2020. p. 2079–89.
Shen, D., et al. “Towards generating long and coherent text with multi-level latent variable models.” ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference, 2020, pp. 2079–89.
Shen D, Celikyilmaz A, Zhang Y, Chen L, Wang X, Gao J, Carin L. Towards generating long and coherent text with multi-level latent variable models. ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference. 2020. p. 2079–2089.

Published In

ACL 2019 - 57th Annual Meeting of the Association for Computational Linguistics, Proceedings of the Conference

ISBN

9781950737482

Publication Date

January 1, 2020

Start / End Page

2079 / 2089