Skip to main content

Which supervised machine learning algorithm can best predict achievement of minimum clinically important difference in neck pain after surgery in patients with cervical myelopathy? A QOD study.

Publication ,  Journal Article
Park, C; Mummaneni, PV; Gottfried, ON; Shaffrey, CI; Tang, AJ; Bisson, EF; Asher, AL; Coric, D; Potts, EA; Foley, KT; Wang, MY; Fu, K-M ...
Published in: Neurosurg Focus
June 2023

OBJECTIVE: The purpose of this study was to evaluate the performance of different supervised machine learning algorithms to predict achievement of minimum clinically important difference (MCID) in neck pain after surgery in patients with cervical spondylotic myelopathy (CSM). METHODS: This was a retrospective analysis of the prospective Quality Outcomes Database CSM cohort. The data set was divided into an 80% training and a 20% test set. Various supervised learning algorithms (including logistic regression, support vector machine, decision tree, random forest, extra trees, gaussian naïve Bayes, k-nearest neighbors, multilayer perceptron, and extreme gradient boosted trees) were evaluated on their performance to predict achievement of MCID in neck pain at 3 and 24 months after surgery, given a set of predicting baseline features. Model performance was assessed with accuracy, F1 score, area under the receiver operating characteristic curve, precision, recall/sensitivity, and specificity. RESULTS: In total, 535 patients (46.9%) achieved MCID for neck pain at 3 months and 569 patients (49.9%) achieved it at 24 months. In each follow-up cohort, 501 patients (93.6%) were satisfied at 3 months after surgery and 569 patients (100%) were satisfied at 24 months after surgery. Of the supervised machine learning algorithms tested, logistic regression demonstrated the best accuracy (3 months: 0.76 ± 0.031, 24 months: 0.773 ± 0.044), followed by F1 score (3 months: 0.759 ± 0.019, 24 months: 0.777 ± 0.039) and area under the receiver operating characteristic curve (3 months: 0.762 ± 0.027, 24 months: 0.773 ± 0.043) at predicting achievement of MCID for neck pain at both follow-up time points, with fair performance. The best precision was also demonstrated by logistic regression at 3 (0.724 ± 0.058) and 24 (0.780 ± 0.097) months. The best recall/sensitivity was demonstrated by multilayer perceptron at 3 months (0.841 ± 0.094) and by extra trees at 24 months (0.817 ± 0.115). Highest specificity was shown by support vector machine at 3 months (0.952 ± 0.013) and by logistic regression at 24 months (0.747 ± 0.18). CONCLUSIONS: Appropriate selection of models for studies should be based on the strengths of each model and the aims of the studies. For maximally predicting true achievement of MCID in neck pain, of all the predictions in this balanced data set the appropriate metric for the authors' study was precision. For both short- and long-term follow-ups, logistic regression demonstrated the highest precision of all models tested. Logistic regression performed consistently the best of all models tested and remains a powerful model for clinical classification tasks.

Duke Scholars

Altmetric Attention Stats
Dimensions Citation Stats

Published In

Neurosurg Focus

DOI

EISSN

1092-0684

Publication Date

June 2023

Volume

54

Issue

6

Start / End Page

E5

Location

United States

Related Subject Headings

  • Supervised Machine Learning
  • Spinal Cord Diseases
  • Retrospective Studies
  • Prospective Studies
  • Neurology & Neurosurgery
  • Neck Pain
  • Humans
  • Bayes Theorem
  • Algorithms
  • 3209 Neurosciences
 

Citation

APA
Chicago
ICMJE
MLA
NLM
Park, C., Mummaneni, P. V., Gottfried, O. N., Shaffrey, C. I., Tang, A. J., Bisson, E. F., … Chan, A. K. (2023). Which supervised machine learning algorithm can best predict achievement of minimum clinically important difference in neck pain after surgery in patients with cervical myelopathy? A QOD study. Neurosurg Focus, 54(6), E5. https://doi.org/10.3171/2023.3.FOCUS2372
Park, Christine, Praveen V. Mummaneni, Oren N. Gottfried, Christopher I. Shaffrey, Anthony J. Tang, Erica F. Bisson, Anthony L. Asher, et al. “Which supervised machine learning algorithm can best predict achievement of minimum clinically important difference in neck pain after surgery in patients with cervical myelopathy? A QOD study.Neurosurg Focus 54, no. 6 (June 2023): E5. https://doi.org/10.3171/2023.3.FOCUS2372.
Park C, Mummaneni PV, Gottfried ON, Shaffrey CI, Tang AJ, Bisson EF, Asher AL, Coric D, Potts EA, Foley KT, Wang MY, Fu K-M, Virk MS, Knightly JJ, Meyer S, Park P, Upadhyaya C, Shaffrey ME, Buchholz AL, Tumialán LM, Turner JD, Sherrod BA, Agarwal N, Chou D, Haid RW, Bydon M, Chan AK. Which supervised machine learning algorithm can best predict achievement of minimum clinically important difference in neck pain after surgery in patients with cervical myelopathy? A QOD study. Neurosurg Focus. 2023 Jun;54(6):E5.

Published In

Neurosurg Focus

DOI

EISSN

1092-0684

Publication Date

June 2023

Volume

54

Issue

6

Start / End Page

E5

Location

United States

Related Subject Headings

  • Supervised Machine Learning
  • Spinal Cord Diseases
  • Retrospective Studies
  • Prospective Studies
  • Neurology & Neurosurgery
  • Neck Pain
  • Humans
  • Bayes Theorem
  • Algorithms
  • 3209 Neurosciences