Variational Diffusion Autoencoders with Random Walk Sampling

Conference Paper

Variational autoencoders (VAEs) and generative adversarial networks (GANs) enjoy an intuitive connection to manifold learning: in training the decoder/generator is optimized to approximate a homeomorphism between the data distribution and the sampling space. This is a construction that strives to define the data manifold. A major obstacle to VAEs and GANs, however, is choosing a suitable prior that matches the data topology. Well-known consequences of poorly picked priors are posterior and mode collapse. To our knowledge, no existing method sidesteps this user choice. Conversely, diffusion maps automatically infer the data topology and enjoy a rigorous connection to manifold learning, but do not scale easily or provide the inverse homeomorphism (i.e. decoder/generator). We propose a method (https://github.com/lihenryhfl/vdae) that combines these approaches into a generative model that inherits the asymptotic guarantees of diffusion maps while preserving the scalability of deep models. We prove approximation theoretic results for the dimension dependence of our proposed method. Finally, we demonstrate the effectiveness of our method with various real and synthetic datasets.

Full Text

Duke Authors

Cited Authors

  • Li, H; Lindenbaum, O; Cheng, X; Cloninger, A

Published Date

  • January 1, 2020

Published In

Volume / Issue

  • 12368 LNCS /

Start / End Page

  • 362 - 378

Electronic International Standard Serial Number (EISSN)

  • 1611-3349

International Standard Serial Number (ISSN)

  • 0302-9743

International Standard Book Number 13 (ISBN-13)

  • 9783030585914

Digital Object Identifier (DOI)

  • 10.1007/978-3-030-58592-1_22

Citation Source

  • Scopus