Sampling from non-log-concave distributions via stochastic variance-reduced gradient Langevin dynamics

We study stochastic variance reduction-based Langevin dynamic algorithms, SVRG-LD and SAGA-LD (Dubey et al., 2016), for sampling from non-log-concave distributions. Under certain assumptions on the log density function, we establish the convergence guarantees of SVRG-LD and SAGA-LD in 2-Wasserstein distance. More specifically, we show that both SVRG-LD and SAGA-LD require Õ(n+n³/⁴/ε²+n¹/²/ε⁴) -exp (Õ(d+γ)) stochastic gradient evaluations to achieve e-accuracy in 2-Wasserstein distance, which outperforms the Õ(n/ε⁴) exp (Õ(d + γ)) gradient complexity achieved by Langevin Monte Carlo Method (Raginsky et al., 2017). Experiments on synthetic data and real data back up our theory.

Duke Scholars

Author Pan Xu Biostatistics & Bioinformatics, Division of Translational Bi ...