Skip to main content

Forward backward greedy algorithms for multi-task learning with faster rates

Publication ,  Conference
Tian, L; Xu, P; Gu, Q
Published in: 32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016
January 1, 2016

A large body of algorithms have been proposed for multi-task learning. However, the effectiveness of many multi-task learning algorithms highly depends on the structural regularization, which incurs bias in the resulting estimators and leads to slower convergence rate. In this paper, we aim at developing a multi-task learning algorithm with faster convergence rate. In particular, we propose a general estimator for multitask learning with row sparsity constraint on the parameter matrix, i.e., the number of nonzero rows in the parameter matrix being small. The proposed estimator is a nonconvex optimization problem. In order to solve it, we develop a forward backward greedy algorithm with provable guarantee. More specifically, we prove that the output of the greedy algorithm attains a sharper estimation error bound than many state-of-the-art multi-task learning methods. Moreover, our estimator enjoys model selection consistency under a mild condition. Thorough experiments on both synthetic and real-world data demonstrate the effectiveness of our method and back up our theory.

Duke Scholars

Published In

32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016

Publication Date

January 1, 2016

Start / End Page

735 / 744
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Tian, L., Xu, P., & Gu, Q. (2016). Forward backward greedy algorithms for multi-task learning with faster rates. In 32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016 (pp. 735–744).
Tian, L., P. Xu, and Q. Gu. “Forward backward greedy algorithms for multi-task learning with faster rates.” In 32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016, 735–44, 2016.
Tian L, Xu P, Gu Q. Forward backward greedy algorithms for multi-task learning with faster rates. In: 32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016. 2016. p. 735–44.
Tian, L., et al. “Forward backward greedy algorithms for multi-task learning with faster rates.” 32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016, 2016, pp. 735–44.
Tian L, Xu P, Gu Q. Forward backward greedy algorithms for multi-task learning with faster rates. 32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016. 2016. p. 735–744.

Published In

32nd Conference on Uncertainty in Artificial Intelligence 2016, UAI 2016

Publication Date

January 1, 2016

Start / End Page

735 / 744