Ensemble Pruning via Graph Neural Networks
Ensemble learning is a pivotal machine learning strategy that combines multiple base learners to achieve prediction accuracy surpassing that of any individual model. Despite its effectiveness, large-scale ensemble learning consumes a considerable amount of resources. Ensemble pruning addresses this issue by selecting a subset of base learners from the original ensemble to form a sub-ensemble, while maintaining or even improving the performance of the original model. However, existing ensemble pruning strategies often rely on heuristic solutions that may fail to capture complex interactions among base learners. To address this limitation, in this work, we model the base learners in an ensemble as a weighted and attributed graph, where node features represent characteristics of each learner and edge weights represent relationships between the base learners. Leveraging this representation, we propose a novel ensemble pruning method based on graph neural networks (GNNs). Our approach incorporates specialized GNN architectures designed for bagging and boosting ensembles. Experimental results demonstrate that our method not only improves prediction accuracy but also significantly reduces inference time across diverse datasets. Our implementation is available at the anonymous repository: https://github.com/TechnologyAiGroup/GRE.
