Stochastic Variance-Reduced Cubic Regularized Newton Methods

We propose a stochastic variance-reduced cubic regularized Newton method (SVRC) for nonconvex optimization. At the core of our algorithm is a novel semi-stochastic gradient along with a semi-stochastic Hessian, which are specifically designed for cubic regularization method. We show that our algorithm is guaranteed to converge to an (✏,^p✏)-approximate local minimum within^eO(n^4/5 /✏^3/2 ) second-order oracle calls, which outperforms the state-of-the-art cubic regularization algorithms including subsampled cubic regularization. Our work also sheds light on the application of variance reduction technique to high-order non-convex optimization methods. Thorough experiments on various non-convex optimization problems support our theory.

Duke Scholars

Author Pan Xu Biostatistics & Bioinformatics, Division of Translational Bi ...