Skip to main content

Go gradient for expectation-based objectives

Publication ,  Conference
Cong, Y; Zhao, M; Bai, K; Carin, L
Published in: 7th International Conference on Learning Representations, ICLR 2019
January 1, 2019

Within many machine learning algorithms, a fundamental problem concerns efficient calculation of an unbiased gradient wrt parameters γ for expectation-based objectives Eqγ(y)[f(y)]. Most existing methods either (i) suffer from high variance, seeking help from (often) complicated variance-reduction techniques; or (ii) they only apply to reparameterizable continuous random variables and employ a reparameterization trick. To address these limitations, we propose a General and One-sample (GO) gradient that (i) applies to many distributions associated with non-reparameterizable continuous or discrete random variables, and (ii) has the same low-variance as the reparameterization trick. We find that the GO gradient often works well in practice based on only one Monte Carlo sample (although one can of course use more samples if desired). Alongside the GO gradient, we develop a means of propagating the chain rule through distributions, yielding statistical back-propagation, coupling neural networks to common random variables.

Duke Scholars

Published In

7th International Conference on Learning Representations, ICLR 2019

Publication Date

January 1, 2019
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Cong, Y., Zhao, M., Bai, K., & Carin, L. (2019). Go gradient for expectation-based objectives. In 7th International Conference on Learning Representations, ICLR 2019.
Cong, Y., M. Zhao, K. Bai, and L. Carin. “Go gradient for expectation-based objectives.” In 7th International Conference on Learning Representations, ICLR 2019, 2019.
Cong Y, Zhao M, Bai K, Carin L. Go gradient for expectation-based objectives. In: 7th International Conference on Learning Representations, ICLR 2019. 2019.
Cong, Y., et al. “Go gradient for expectation-based objectives.” 7th International Conference on Learning Representations, ICLR 2019, 2019.
Cong Y, Zhao M, Bai K, Carin L. Go gradient for expectation-based objectives. 7th International Conference on Learning Representations, ICLR 2019. 2019.

Published In

7th International Conference on Learning Representations, ICLR 2019

Publication Date

January 1, 2019