The effect of sampling health facilities on estimates of effective coverage: a simulation study

Background: Most existing facility assessments collect data on a sample of health facilities. Sampling of health facilities may introduce bias into estimates of effective coverage generated by ecologically linking individuals to health providers based on geographic proximity or administrative catchment. Methods: We assessed the bias introduced to effective coverage estimates produced through two ecological linking approaches (administrative unit and Euclidean distance) applied to a sample of health facilities. Our analysis linked MICS household survey data on care-seeking for child illness and childbirth care with data on service quality collected from a census of health facilities in the Savanes region of Cote d’Ivoire. To assess the bias introduced by sampling, we drew 20 random samples of three different sample sizes from our census of health facilities. We calculated effective coverage of sick child and childbirth care using both ecological linking methods applied to each sampled facility data set. We compared the sampled effective coverage estimates to ecologically linked census-based estimates and estimates based on true source of care. We performed sensitivity analyses with simulated preferential care-seeking from higher-quality providers and randomly generated provider quality scores. Results: Sampling of health facilities did not significantly bias effective coverage compared to either the ecologically linked estimates derived from a census of facilities or true effective coverage estimates using the original data or simulated random quality sensitivity analysis. However, a few estimates based on sampling in a setting where individuals preferentially sought care from higher-quality providers fell outside of the estimate bounds of true effective coverage. Those cases predominantly occurred using smaller sample sizes and the Euclidean distance linking method. None of the sample-based estimates fell outside the bounds of the ecologically linked census-derived estimates. Conclusions: Our analyses suggest that current health facility sampling approaches do not significantly bias estimates of effective coverage produced through ecological linking. Choice of ecological linking methods is a greater source of bias from true effective coverage estimates, although facility sampling can exacerbate this bias in certain scenarios. Careful selection of ecological linking methods is essential to minimize the potential effect of both ecological linking and sampling error.