Forecasting accuracy of the hollow fiber model of tuberculosis for clinical therapeutic outcomes

Background. The hollow fiber system model of tuberculosis (HFS-TB), in tandem with Monte Carlo experiments, represents a drug development tool (DDT) with the potential for use to develop tuberculosis treatment regimens. However, the predictive accuracy of the HFS-TB, or any other nonclinical DDT such as an animal model, has yet to be robustly evaluated. Methods. To avoid hindsight bias, a literature search was performed to identify clinical studies published at least 6 months after HFS-TB experiments' quantitative predictions. Steps to minimize bias and for reporting systematic reviews were applied as outlined in the Preferred Reporting Items for Systematic Reviews and Meta-Analyses. Publications were scored for quality of evidence. Accuracy was calculated using the mean absolute percentage error, then summated with weighting assigned by sample size and quality-of-evidence score. Given the lack of a gold-standard tuberculosis DDT, the forecasting accuracy of a completely unreliable tool was also calculated from 1000 simulated experiments for a random or "total guesswork" model. Results. The quantitative forecasting accuracy (95% confidence interval [CI]) for the "total guesswork" model was 15.6% (95% CI, 8.7%-22.5%); bias was -0.1% (95% CI, -2.5% to 2.2%). Twenty clinical studies were published after HFS-TB experiments predicted optimal drug exposures and doses, susceptibility breakpoints, and optimal combination regimens. Based on these clinical studies, the predictive accuracy of the HFS-TB was 94.4% (95% CI, 84.3%-99.9%), and bias was 1.8% (95% CI, -13.7% to 6.2%). Conclusions. The HFS-TB model is highly accurate at forecasting optimal drug exposures, doses, and dosing schedules for use in the clinic.