Skip to main content

Table 2 Study methods

From: Machine learning models predicting risk of revision or secondary knee injury after anterior cruciate ligament reconstruction demonstrate variable discriminatory and accuracy performance: a systematic review

Author (year)

Primary Outcome

Statistical Software and ML algorithms

Models

Model Evaluation

Training/Test Split

Missing Data Method

Martin (2023)

Revision

R (Version 4.1.11 R Core Team)

Cox lasso

Random survival forest

Gradient boosting

Super learner

Concordance - Harrell C-index

Calibration

75/25

Multiple imputation

Martin (2022a)

Revision

R (Version 3.6.1)

Cox Lasso

Concordance - Harrell C-index

Calibration

NR (external validation study - original model 75/25)

Patients included if they had data for five predictive models from original model

Martin (2022b)

Revision

R (Version 3.6.1)

Cox Lasso

Survival Random Forest

Generalized Additive Model (GAM)

Gradient Boosted Regression Model (GBM)

Calibration

Concordance

75/25

Multiple imputation

Johnson (2023)

All-cause re-operation

SciPy version 1.6.2

MLPClassifier

GaussianNB

LogisticRegression

KNeighborsClassifier

BaggingClassifier

RandomForestClassifier

AdaBoostClassifier

GradientBoostingClassifier

XGBClassifier

AUC

Calibration

AUPRC

F1

Recall

Accuracy

Precision

75/25

Multiple imputation

Lopez (2023)

ACLR post-op outcomes (revision included)

TensorFlow Python open-source coding platform (Google Brain, Alphabet Inc., Mountain View, CA)

Artificial Neural Network ML

Logistic Regression

AUC

Accuracy

80/20

Excluded

Ye (2022)

Graft failure

SPSS (Version 25.0; IBM Corp)

Logistic Regression

Gaussian Naïve Bayes

Random Forest

XGBoost

Isotonic XGBoost

Sigmoid XGBoost

AUC

Accuracy

F1

90/10

NR

Martin (2024)

Revision

R (RStudio 2022.07.1)

Cox Lasso

Concordance - Harrell’s C-index

Calibration

75/25

NA

Jurgensmeier (2023)

Secondary meniscus tear

R 4.1.2 using RStudio version 1.4.1717 (RStudio, Boston, MA)

SVM

Random Forest

XGBoost

Elastic Net

Discrimination - AUROC

Calibration

Brier score

0.632 bootstrapping with 1000 resampled datasets

Multiple imputation

Lu (2022)

Secondary meniscus tear

R 4.1.2 using RStudio version 1.2.5001 (RStudio, Boston, MA).

Random Survival Forests

C-statistic (AUROC) (Concordance)

Calibration

Brier Score

0.632 bootstrapping with 1000 resampled datasets

Multiple imputation

  1. ML: machine learning, AUC: area under the curve, AUROC: area under the receiver operating curve, AUPRC: area under the precision-recall graph, ACLR: anterior cruciate ligament reconstruction, NR: not reported, MA: Massachusetts, CA: California, NR: not reported, NA: not applicable