Abstract
BACKGROUND CONTEXT
Preoperative survival estimation in spinal metastatic disease helps determine the
appropriateness of invasive management. The SORG ML 90-day and 1-year machine learning
algorithms for survival in spinal metastatic disease were previously developed in
a single institutional sample but remain to be externally validated.
PURPOSE
The purpose of this study was to externally validate these algorithms in an independent
population from another institution.
STUDY DESIGN/SETTING
Retrospective study at a large, tertiary care center.
PATIENT SAMPLE
Patients 18 years or older who underwent surgery between 2003 and 2016.
OUTCOME MEASURES
Ninety-day and 1-year mortality.
METHODS
Baseline characteristics of the validation cohort were compared to the developmental
cohort for the SORG ML algorithms. Discrimination (c-statistic and receiver operating
curve), calibration (calibration slope, intercept, calibration plot, and observed
proportions by predicted risk groups), overall performance (Brier score), and decision
curve analysis were used to assess the performance of the SORG ML algorithms in the
validation cohort.
RESULTS
Overall, 176 patients underwent surgery for spinal metastatic disease, of which 44
(22.7%) experienced 90-day mortality and 99 (56.2%) experienced 1-year mortality.
The validation cohort differed significantly from the developmental cohort on primary
tumor histology, metastatic tumor burden, previous systemic therapy, overall comorbidity
burden, and preoperative laboratory characteristics. Despite these differences, the
SORG ML algorithms generalized well to the validation cohort on discrimination (c-statistic
0.75–0.81 for 90-day mortality and 0.77–0.78 for 1-year mortality), calibration, Brier
score, and decision curve analysis.
CONCLUSION and RELEVANCE
Initial results from external validation of the SORG ML 90-day and 1-year algorithms
for survival prediction in spinal metastatic disease suggest potential utility of
these digital decision aids in clinical practice. Further studies are needed to validate
or refute these algorithms in large patient samples from prospective, international,
multi-institutional trials.
Keywords
To read this article in full you will need to make a payment
Purchase one-time access:
Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online accessOne-time access price info
- For academic or personal research use, select 'Academic and Personal'
- For corporate R&D use, select 'Corporate R&D Professionals'
Subscribe:
Subscribe to The Spine JournalAlready a print subscriber? Claim online access
Already an online subscriber? Sign in
Register: Create an account
Institutional Access: Sign in to ScienceDirect
References
- Assessment and management of patients with metastatic spinal cord compression: a multidisciplinary review.J Clin Oncol. 2018; 37: 61-71
- A population-based study of malignant spinal cord compression in Ontario.Clin Oncol. 2003; 15: 211-217
- Surgical management of spinal metastases.Oncologist. 2004; 9: 188-196
- Characteristics of patients who survived < 3 months or >2 years after surgery for spinal metastases: can we avoid inappropriate patient selection?.J Clin Oncol. 2016; 34: 3054-3061
- Malignant spinal-cord compression.Lancet Oncol. 2005; 6: 15-24
- Direct decompressive surgical resection in the treatment of spinal cord compression caused by metastatic cancer: a randomised trial.Lancet. 2005; 366: 643-648
- Effect of a randomized controlled trial on the surgical treatment of spinal metastasis, 2000 through 2010: a population-based cohort study.Cancer. 2014; 120: 901-908
- Predictive factors of survival in a surgical series of metastatic epidural spinal cord compression and complete external validation of 8 multivariate models of survival in a prospective North American multicenter study.Cancer. 2018; 124: 3536-3550
- Predicting 90-day and 1-year mortality in spinal metastatic disease: development and internal validation.Neurosurgery. 2019; 85: E671-E681
- Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement.BMC Med. 2015; 13: 1
- New prognostic factors and scoring system for patients with skeletal metastasis.Cancer Med. 2014; 3: 1359-1367
- Updating and validating the Charlson comorbidity index and score for risk adjustment in hospital discharge abstracts using data from 6 countries.Am J Epidemiol. 2011; 173: 676-682
- MissForest—non-parametric missing value imputation for mixed-type data.Bioinformatics. 2011; 28: 112-118
- Towards better clinical prediction models: seven steps for development and an ABCD for validation.Eur Heart J. 2014; 35: 1925-1931
- Verification of forecasts expressed in terms of probability.Mon Weather Rev. 1950; 78: 1-3
- Decision curve analysis: a novel method for evaluating prediction models.Med Decis Making. 2006; 26: 565-574
- Spine metastases: current treatments and future directions.Eur J Cancer. 2010; 46: 2696-2707
- Integrating evidence-based medicine for treatment of spinal metastases into a decision framework: neurologic, oncologic, mechanicals stability, and systemic disease.J Clin Oncol. 2017; 35: 2419-2427
- State of the art treatment of spinal metastatic disease.Neurosurgery. 2018; 82: 757-769
- Predicting survival for metastatic spine disease: a comparison of nine scoring systems.Spine J. 2018; 18: 1804-1814
- Scoring system for the preoperative evaluation of metastatic spine tumor prognosis.Spine. 1990; 15: 1110-1113
- A revised scoring system for preoperative evaluation of metastatic spine tumor prognosis.Spine. 2005; 30: 2186-2191
- Surgical strategy for spinal metastases.Spine. 2001; 26: 298-306
- Survival after surgery for spinal and extremity metastases: prognostication in 241 patients.Acta Orthop Scand. 1995; 66: 143-146
- Prognostic factors and a scoring system for patients with skeletal metastasis.J Bone Joint Surg Br Vol. 2005; 87: 698-703
- Prediction of survival in patients with metastases in the spinal column: results based on a randomized trial of radiotherapy.Cancer. 2005; 103: 320-328
- Development of a prognostic survival algorithm for patients with metastatic spine disease.J Bone Joint Surg Am Vol. 2016; 98: 1767-1776
- Assessing the performance of prediction models: a framework for traditional and novel measures.Epidemiology. 2010; 21: 128-138
- A new framework to enhance the interpretation of external validation studies of clinical prediction models.J Clin Epidemiol. 2015; 68: 279-289
- Prediction models need appropriate internal, internal-external, and external validation.J Clin Epidemiol. 2016; 69: 245-247
- Poor performance of clinical prediction models: the harm of commonly applied methods.J Clin Epidemiol. 2018; 98: 133-143
- A calibration hierarchy for risk models was defined: from utopia to empirical data.J Clin Epidemiol. 2016; 74: 167-176
Article info
Publication history
Published online: September 07, 2019
Accepted:
September 4,
2019
Received in revised form:
September 3,
2019
Received:
April 8,
2019
Footnotes
FDA device/drug status: Not applicable.
Author disclosures: AVK: Nothing to disclose. AKA: Nothing to disclose. ZP: Nothing to disclose. AC: Nothing to disclose. AS: Nothing to disclose. QCBST: Nothing to disclose. PTO: Nothing to disclose. DMS: Consulting: Baxter (B), DePuy Synthes (B), Globus (B), Medtronic (C), NuVasive (B), Stryker (C). JHS: Consulting or Honorarium: Scientific Advisory Board: Chordoma Foundation (Nonfinancial); Speaking and/or Teaching Arrangements: AO Spine (Travel Expense Reimbursement), Stryker Spine (B).
Identification
Copyright
© 2019 Published by Elsevier Inc.
