Stochastic variance-reduced cubic regularized Newton method
Publication
, Conference
Zhou, D; Xu, P; Gu, Q
Published in: 35th International Conference on Machine Learning, ICML 2018
January 1, 2018
We propose a stochastic variance-reduced cubic regularized Newton method (SVRC) for non-convex optimization. At the core of our algorithm is a novel semi-stochastic gradient along with a semi-stochastic Hessian, which are specifically designed for cubic regularization method. We show that our algorithm is guaranteed to converge to an (ϵ, √ϵ)-approximate local minimum within Õ(n4/5/ϵ3/2) second-order oracle calls, which outperforms the state-of-the-art cubic regularization algorithms including subsampled cubic regularization. Our work also sheds light on the application of variance reduction technique to high-order non-convex optimization methods. Thorough experiments on various non-convex optimization problems support our theory.
Duke Scholars
Published In
35th International Conference on Machine Learning, ICML 2018
Publication Date
January 1, 2018
Volume
13
Start / End Page
9597 / 9606
Citation
APA
Chicago
ICMJE
MLA
NLM
Zhou, D., Xu, P., & Gu, Q. (2018). Stochastic variance-reduced cubic regularized Newton method. In 35th International Conference on Machine Learning, ICML 2018 (Vol. 13, pp. 9597–9606).
Zhou, D., P. Xu, and Q. Gu. “Stochastic variance-reduced cubic regularized Newton method.” In 35th International Conference on Machine Learning, ICML 2018, 13:9597–9606, 2018.
Zhou D, Xu P, Gu Q. Stochastic variance-reduced cubic regularized Newton method. In: 35th International Conference on Machine Learning, ICML 2018. 2018. p. 9597–606.
Zhou, D., et al. “Stochastic variance-reduced cubic regularized Newton method.” 35th International Conference on Machine Learning, ICML 2018, vol. 13, 2018, pp. 9597–606.
Zhou D, Xu P, Gu Q. Stochastic variance-reduced cubic regularized Newton method. 35th International Conference on Machine Learning, ICML 2018. 2018. p. 9597–9606.
Published In
35th International Conference on Machine Learning, ICML 2018
Publication Date
January 1, 2018
Volume
13
Start / End Page
9597 / 9606