External Validation of the Bone Metastases Ensemble Trees for Survival (BMETS) Machine Learning Model to Predict Survival in Patients With Symptomatic Bone Metastases

PURPOSE The Bone Metastases Ensemble Trees for Survival (BMETS) model uses a machine learning algorithm to estimate survival time following consultation for palliative radiation therapy for symptomatic bone metastases (SBM). BMETS was developed at a tertiary-care, academic medical center, but its validity and stability when applied to external data sets are unknown. PATIENTS AND METHODS Patients treated with palliative radiation therapy for SBM from May 2013 to May 2016 at two hospital-based community radiation oncology clinics were included, and medical records were retrospectively reviewed to collect model covariates and survival time. The Kaplan-Meier method was used to estimate overall survival from consultation to death or last follow-up. Model discrimination was estimated using time-dependent area under the curve (tAUC), which was calculated using survival predictions from BMETS based on the initial training data set. RESULTS A total of 216 sites of SBM were treated in 182 patients. Most common histologies were breast (27%), lung (23%), and prostate (23%). Compared with the BMETS training set, the external validation population was older (mean age, 67 v 62 years; P, .001), had more primary breast (27% v 19%; P = .03) and prostate cancer (20% v 12%; P = .01), and survived longer (median, 10.7 v 6.4 months). When the BMETS model was applied to the external data set, tAUC values at 3, 6, and 12 months were 0.82, 0.77, and 0.77, respectively. When refit with data from the combined training and external validation sets, tAUC remained . 0.79. CONCLUSION BMETS maintained high discriminative ability when applied to an external validation set and when refit with new data, supporting its generalizability, stability, and the feasibility of dynamic modeling.