Generalization bounds for learning with linear and quadratic side knowledge
© 2014 University of Illinois at Chicago. All rights reserved. In this paper, we consider a supervised learning setting where side knowledge is provided about the labels of unlabeled examples. The side knowledge has the effect of reducing the hypothesis space, leading to tighter generalization bounds, and thus possibly better generalization. We consider two types of side knowledge, the first leading to linear constraints on the hypothesis space, and the second leading to quadratic constraints on the hypothesis space. We show how different types of domain knowledge can lead directly to these kinds of side knowledge. We prove bounds on complexity measures of the hypothesis space for quadratic side knowledge, and show that these bounds are tight in a specific sense.
