A Bayesian hierarchical nonlinear mixture model in the presence of artifactual outliers in a population pharmacokinetic study

The purpose of this study is to develop a statistical methodology to handle a large proportion of artifactual outliers in a population pharmacokinetic (PK) modeling. The motivating PK data were obtained from a population PK study to examine associations between PK parameters such as clearance of dexmedetomidine (DEX) and cytochrome P450 2A6 phenotypes. The blood samples were sparsely sampled from patients in intensive care units (ICUs) while different doses of DEX were continuously infused. Conventional population PK analysis of these data revealed several challenges and intricacies. Especially, there was strong evidence that some plasma drug concentrations were artifactually high and likely contaminated with the infused drug due to blood sampling processes that are sometimes unavoidable in an ICU setting. If not addressed, or if arbitrarily excluded, these outlying values could lead to biased estimates of PK parameters and miss important relationships between PK parameters and covariates due to increased variability. We propose a novel population PK model, a Bayesian hierarchical nonlinear mixture model, to accommodate the artifactual outliers using a finite mixture as the residual error model. Our results showed that the proposed model handles the outliers well. We also conducted simulation studies with a varying proportion of the outliers. These simulation results showed that the proposed model can accommodate the outliers well so that the estimated PK parameters are less biased.