Skip to main content

Stochastic variance-reduced cubic regularized Newton method

Publication ,  Conference
Zhou, D; Xu, P; Gu, Q
Published in: 35th International Conference on Machine Learning, ICML 2018
January 1, 2018

We propose a stochastic variance-reduced cubic regularized Newton method (SVRC) for non-convex optimization. At the core of our algorithm is a novel semi-stochastic gradient along with a semi-stochastic Hessian, which are specifically designed for cubic regularization method. We show that our algorithm is guaranteed to converge to an (ϵ, √ϵ)-approximate local minimum within Õ(n4/5/ϵ3/2) second-order oracle calls, which outperforms the state-of-the-art cubic regularization algorithms including subsampled cubic regularization. Our work also sheds light on the application of variance reduction technique to high-order non-convex optimization methods. Thorough experiments on various non-convex optimization problems support our theory.

Duke Scholars

Published In

35th International Conference on Machine Learning, ICML 2018

Publication Date

January 1, 2018

Volume

13

Start / End Page

9597 / 9606
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Zhou, D., Xu, P., & Gu, Q. (2018). Stochastic variance-reduced cubic regularized Newton method. In 35th International Conference on Machine Learning, ICML 2018 (Vol. 13, pp. 9597–9606).
Zhou, D., P. Xu, and Q. Gu. “Stochastic variance-reduced cubic regularized Newton method.” In 35th International Conference on Machine Learning, ICML 2018, 13:9597–9606, 2018.
Zhou D, Xu P, Gu Q. Stochastic variance-reduced cubic regularized Newton method. In: 35th International Conference on Machine Learning, ICML 2018. 2018. p. 9597–606.
Zhou, D., et al. “Stochastic variance-reduced cubic regularized Newton method.” 35th International Conference on Machine Learning, ICML 2018, vol. 13, 2018, pp. 9597–606.
Zhou D, Xu P, Gu Q. Stochastic variance-reduced cubic regularized Newton method. 35th International Conference on Machine Learning, ICML 2018. 2018. p. 9597–9606.

Published In

35th International Conference on Machine Learning, ICML 2018

Publication Date

January 1, 2018

Volume

13

Start / End Page

9597 / 9606