Model Recycling Framework for Multi-Source Data-Free Supervised Transfer Learning
Increasing concerns for data privacy and other difficulties associated with retrieving source data for model training have created the need for source-free transfer learning, in which one only has access to pre-trained models instead of data from the original source domains. This setting introduces many practical challenges, e.g., efficiently selecting models for transfer without information on source data, and transferring without full access to the source models. So motivated, we propose a framework for parameter-efficient training of models that identifies subsets of related source models for knowledge transfer in both white-box and black-box settings. Consequently, our framework makes it possible for Model as a Service (MaaS) providers to build libraries of efficient pretrained models, thus creating an opportunity for multi-source data-free supervised transfer learning.
