Skip to main content

Accelerated stochastic mirror descent: From continuous-time dynamics to discrete-time algorithms

Publication ,  Conference
Xu, P; Wang, T; Gu, Q
Published in: International Conference on Artificial Intelligence and Statistics, AISTATS 2018
January 1, 2018

We present a new framework to analyze accelerated stochastic mirror descent through the lens of continuous-time stochastic dynamic systems. It enables us to design new algorithms, and perform a unified and simple analysis of the convergence rates of these algorithms. More specifically, under this framework, we provide a Lyapunov function based analysis for the continuous-time stochastic dynamics, as well as several new discrete-time algorithms derived from the continuous-time dynamics. We show that for general convex objective functions, the derived discrete-time algorithms attain the optimal convergence rate. Empirical experiments corroborate our theory.

Duke Scholars

Published In

International Conference on Artificial Intelligence and Statistics, AISTATS 2018

Publication Date

January 1, 2018

Start / End Page

1087 / 1096
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Xu, P., Wang, T., & Gu, Q. (2018). Accelerated stochastic mirror descent: From continuous-time dynamics to discrete-time algorithms. In International Conference on Artificial Intelligence and Statistics, AISTATS 2018 (pp. 1087–1096).
Xu, P., T. Wang, and Q. Gu. “Accelerated stochastic mirror descent: From continuous-time dynamics to discrete-time algorithms.” In International Conference on Artificial Intelligence and Statistics, AISTATS 2018, 1087–96, 2018.
Xu P, Wang T, Gu Q. Accelerated stochastic mirror descent: From continuous-time dynamics to discrete-time algorithms. In: International Conference on Artificial Intelligence and Statistics, AISTATS 2018. 2018. p. 1087–96.
Xu, P., et al. “Accelerated stochastic mirror descent: From continuous-time dynamics to discrete-time algorithms.” International Conference on Artificial Intelligence and Statistics, AISTATS 2018, 2018, pp. 1087–96.
Xu P, Wang T, Gu Q. Accelerated stochastic mirror descent: From continuous-time dynamics to discrete-time algorithms. International Conference on Artificial Intelligence and Statistics, AISTATS 2018. 2018. p. 1087–1096.

Published In

International Conference on Artificial Intelligence and Statistics, AISTATS 2018

Publication Date

January 1, 2018

Start / End Page

1087 / 1096