Skip to main content
construction release_alert
Scholars@Duke will be down for maintenance for approximately one hour starting Tuesday, 11/11 @1pm ET
cancel

Towards fair decentralized benchmarking of healthcare AI algorithms with the Federated Tumor Segmentation (FeTS) challenge.

Publication ,  Journal Article
Zenk, M; Baid, U; Pati, S; Linardos, A; Edwards, B; Sheller, M; Foley, P; Aristizabal, A; Zimmerer, D; Gruzdev, A; Martin, J; Shinohara, RT ...
Published in: Nat Commun
July 8, 2025

Computational competitions are the standard for benchmarking medical image analysis algorithms, but they typically use small curated test datasets acquired at a few centers, leaving a gap to the reality of diverse multicentric patient data. To this end, the Federated Tumor Segmentation (FeTS) Challenge represents the paradigm for real-world algorithmic performance evaluation. The FeTS challenge is a competition to benchmark (i) federated learning aggregation algorithms and (ii) state-of-the-art segmentation algorithms, across multiple international sites. Weight aggregation and client selection techniques were compared using a multicentric brain tumor dataset in realistic federated learning simulations, yielding benefits for adaptive weight aggregation, and efficiency gains through client sampling. Quantitative performance evaluation of state-of-the-art segmentation algorithms on data distributed internationally across 32 institutions yielded good generalization on average, albeit the worst-case performance revealed data-specific modes of failure. Similar multi-site setups can help validate the real-world utility of healthcare AI algorithms in the future.

Duke Scholars

Altmetric Attention Stats
Dimensions Citation Stats

Published In

Nat Commun

DOI

EISSN

2041-1723

Publication Date

July 8, 2025

Volume

16

Issue

1

Start / End Page

6274

Location

England

Related Subject Headings

  • Magnetic Resonance Imaging
  • Image Processing, Computer-Assisted
  • Humans
  • Brain Neoplasms
  • Benchmarking
  • Artificial Intelligence
  • Algorithms
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Zenk, M., Baid, U., Pati, S., Linardos, A., Edwards, B., Sheller, M., … Bakas, S. (2025). Towards fair decentralized benchmarking of healthcare AI algorithms with the Federated Tumor Segmentation (FeTS) challenge. Nat Commun, 16(1), 6274. https://doi.org/10.1038/s41467-025-60466-1
Zenk, Maximilian, Ujjwal Baid, Sarthak Pati, Akis Linardos, Brandon Edwards, Micah Sheller, Patrick Foley, et al. “Towards fair decentralized benchmarking of healthcare AI algorithms with the Federated Tumor Segmentation (FeTS) challenge.Nat Commun 16, no. 1 (July 8, 2025): 6274. https://doi.org/10.1038/s41467-025-60466-1.
Zenk M, Baid U, Pati S, Linardos A, Edwards B, Sheller M, et al. Towards fair decentralized benchmarking of healthcare AI algorithms with the Federated Tumor Segmentation (FeTS) challenge. Nat Commun. 2025 Jul 8;16(1):6274.
Zenk, Maximilian, et al. “Towards fair decentralized benchmarking of healthcare AI algorithms with the Federated Tumor Segmentation (FeTS) challenge.Nat Commun, vol. 16, no. 1, July 2025, p. 6274. Pubmed, doi:10.1038/s41467-025-60466-1.
Zenk M, Baid U, Pati S, Linardos A, Edwards B, Sheller M, Foley P, Aristizabal A, Zimmerer D, Gruzdev A, Martin J, Shinohara RT, Reinke A, Isensee F, Parampottupadam S, Parekh K, Floca R, Kassem H, Baheti B, Thakur S, Chung V, Kushibar K, Lekadir K, Jiang M, Yin Y, Yang H, Liu Q, Chen C, Dou Q, Heng P-A, Zhang X, Zhang S, Khan MI, Azeem MA, Jafaritadi M, Alhoniemi E, Kontio E, Khan SA, Mächler L, Ezhov I, Kofler F, Shit S, Paetzold JC, Loehr T, Wiestler B, Peiris H, Pawar K, Zhong S, Chen Z, Hayat M, Egan G, Harandi M, Isik Polat E, Polat G, Kocyigit A, Temizel A, Tuladhar A, Tyagi L, Souza R, Forkert ND, Mouches P, Wilms M, Shambhat V, Maurya A, Danannavar SS, Kalla R, Anand VK, Krishnamurthi G, Nalawade S, Ganesh C, Wagner B, Reddy D, Das Y, Yu FF, Fei B, Madhuranthakam AJ, Maldjian J, Singh G, Ren J, Zhang W, An N, Hu Q, Zhang Y, Zhou Y, Siomos V, Tarroni G, Passerrat-Palmbach J, Rawat A, Zizzo G, Kadhe SR, Epperlein JP, Braghin S, Wang Y, Kanagavelu R, Wei Q, Yang Y, Liu Y, Kotowski K, Adamski S, Machura B, Malara W, Zarudzki L, Nalepa J, Shi Y, Gao H, Avestimehr S, Yan Y, Akbar AS, Kondrateva E, Li Z, Wu H-Y, Roth J, Saueressig C, Milesi A, Nguyen QD, Gruenhagen NJ, Huang T-M, Ma J, Singh HSH, Pan N-Y, Zhang D, Zeineldin RA, Futrega M, Yuan Y, Conte GM, Feng X, Pham QD, Xia Y, Jiang Z, Luu HM, Dobko M, Carré A, Tuchinov B, Mohy-Ud-Din H, Alam S, Singh A, Shah N, Wang W, Sako C, Bilello M, Ghodasara S, Mohan S, Davatzikos C, Calabrese E, Rudie J, Villanueva-Meyer J, Cha S, Hess C, Mongan J, Ingalhalikar M, Jadhav M, Pandey U, Saini J, Huang RY, Chang K, To M-S, Bhardwaj S, Chong C, Agzarian M, Kozubek M, Lux F, Michálek J, Matula P, Ker Kovský M, Kopr Ivová T, Dostál M, Vybíhal V, Pinho MC, Holcomb J, Metz M, Jain R, Lee MD, Lui YW, Tiwari P, Verma R, Bareja R, Yadav I, Chen J, Kumar N, Gusev Y, Bhuvaneshwar K, Sayah A, Bencheqroun C, Belouali A, Madhavan S, Colen RR, Kotrotsou A, Vollmuth P, Brugnara G, Preetha CJ, Sahm F, Bendszus M, Wick W, Mahajan A, Balaña C, Capellades J, Puig J, Choi YS, Lee S-K, Chang JH, Ahn SS, Shaykh HF, Herrera-Trujillo A, Trujillo M, Escobar W, Abello A, Bernal J, Gómez J, LaMontagne P, Marcus DS, Milchenko M, Nazeri A, Landman B, Ramadass K, Xu K, Chotai S, Chambless LB, Mistry A, Thompson RC, Srinivasan A, Bapuraj JR, Rao A, Wang N, Yoshiaki O, Moritani T, Turk S, Lee J, Prabhudesai S, Garrett J, Larson M, Jeraj R, Li H, Weiss T, Weller M, Bink A, Pouymayou B, Sharma S, Tseng T-C, Adabi S, Xavier Falcão A, Martins SB, Teixeira BCA, Sprenger F, Menotti D, Lucio DR, Niclou SP, Keunen O, Hau A-C, Pelaez E, Franco-Maldonado H, Loayza F, Quevedo S, McKinley R, Slotboom J, Radojewski P, Meier R, Wiest R, Trenkler J, Pichler J, Necker G, Haunschmidt A, Meckel S, Guevara P, Torche E, Mendoza C, Vera F, Ríos E, López E, Velastin SA, Choi J, Baek S, Kim Y, Ismael H, Allen B, Buatti JM, Zampakis P, Panagiotopoulos V, Tsiganos P, Alexiou S, Haliassos I, Zacharaki EI, Moustakas K, Kalogeropoulou C, Kardamakis DM, Luo B, Poisson LM, Wen N, Vallières M, Loutfi MAL, Fortin D, Lepage M, Morón F, Mandel J, Shukla G, Liem S, Alexandre GS, Lombardo J, Palmer JD, Flanders AE, Dicker AP, Ogbole G, Oyekunle D, Odafe-Oyibotha O, Osobu B, Shu’aibu Hikima M, Soneye M, Dako F, Dorcas A, Murcia D, Fu E, Haas R, Thompson JA, Ormond DR, Currie S, Fatania K, Frood R, Simpson AL, Peoples JJ, Hu R, Cutler D, Moraes FY, Tran A, Hamghalam M, Boss MA, Gimpel J, Kattil Veettil D, Schmidt K, Cimino L, Price C, Bialecki B, Marella S, Apgar C, Jakab A, Weber M-A, Colak E, Kleesiek J, Freymann JB, Kirby JS, Maier-Hein L, Albrecht J, Mattson P, Karargyris A, Shah P, Menze B, Maier-Hein K, Bakas S. Towards fair decentralized benchmarking of healthcare AI algorithms with the Federated Tumor Segmentation (FeTS) challenge. Nat Commun. 2025 Jul 8;16(1):6274.

Published In

Nat Commun

DOI

EISSN

2041-1723

Publication Date

July 8, 2025

Volume

16

Issue

1

Start / End Page

6274

Location

England

Related Subject Headings

  • Magnetic Resonance Imaging
  • Image Processing, Computer-Assisted
  • Humans
  • Brain Neoplasms
  • Benchmarking
  • Artificial Intelligence
  • Algorithms