Skip to main content

Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction

Publication ,  Conference
Zhang, J; Li, A; Tang, M; Sun, J; Chen, X; Zhang, F; Chen, C; Chen, Y; Li, H
Published in: Proceedings of Machine Learning Research
January 1, 2023

Due to the often limited communication bandwidth of edge devices, most existing federated learning (FL) methods randomly select only a subset of devices to participate in training at each communication round. Compared with engaging all the available clients, such a random-selection mechanism could lead to significant performance degradation on non-IID (independent and identically distributed) data. In this paper, we present our key observation that the essential reason resulting in such performance degradation is the class-imbalance of the grouped data from randomly selected clients. Based on this observation, we design an efficient heterogeneity-aware client sampling mechanism, namely, Federated Class-balanced Sampling (Fed-CBS), which can effectively reduce class-imbalance of the grouped dataset from the intentionally selected clients. We first propose a measure of class-imbalance which can be derived in a privacy-preserving way. Based on this measure, we design a computation-efficient client sampling strategy such that the actively selected clients will generate a more class-balanced grouped dataset with theoretical guarantees. Experimental results show that Fed-CBS outperforms the status quo approaches in terms of test accuracy and the rate of convergence while achieving comparable or even better performance than the ideal setting where all the available clients participate in the FL training.

Duke Scholars

Published In

Proceedings of Machine Learning Research

EISSN

2640-3498

Publication Date

January 1, 2023

Volume

202

Start / End Page

41354 / 41381
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Zhang, J., Li, A., Tang, M., Sun, J., Chen, X., Zhang, F., … Li, H. (2023). Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction. In Proceedings of Machine Learning Research (Vol. 202, pp. 41354–41381).
Zhang, J., A. Li, M. Tang, J. Sun, X. Chen, F. Zhang, C. Chen, Y. Chen, and H. Li. “Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction.” In Proceedings of Machine Learning Research, 202:41354–81, 2023.
Zhang J, Li A, Tang M, Sun J, Chen X, Zhang F, et al. Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction. In: Proceedings of Machine Learning Research. 2023. p. 41354–81.
Zhang, J., et al. “Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction.” Proceedings of Machine Learning Research, vol. 202, 2023, pp. 41354–81.
Zhang J, Li A, Tang M, Sun J, Chen X, Zhang F, Chen C, Chen Y, Li H. Fed-CBS: A Heterogeneity-Aware Client Sampling Mechanism for Federated Learning via Class-Imbalance Reduction. Proceedings of Machine Learning Research. 2023. p. 41354–41381.

Published In

Proceedings of Machine Learning Research

EISSN

2640-3498

Publication Date

January 1, 2023

Volume

202

Start / End Page

41354 / 41381