Correction to: Sensitivity analysis for an unobserved moderator in rct-to-target-population generalization of treatment effects (The Annals of Applied Statistics, (2017))

We regret an error in the article Nguyen et al. (2017), hereafter referred to as the (pub-lished) article. In correcting this error, we no longer recommend the methods in Section 4 of the article. The rest of the article and, most importantly, the sensitivity analyses proposed in Section 3 for moderators observed in the RCT but not in the target population, are unaffected. 1. The article and the content affected. The published article asks how to handle unobserved treatment effect moderators when using data from a randomized controlled trial (RCT) to estimate the average treatment effect for a target population (TATE). To set a foundation for considering this question, the article first presents two methods for estimating TATE when the moderators are observed both in the RCT and in a target population dataset: outcome-model-based TATE estimation relies on an outcome model with treatment-moderator inter-action, and weighting-based TATE estimation relies on weighting the RCT sample to mimic the target population’s distribution of the moderators. Building on this foundation, the article tackles two cases with unobserved moderators. In the first case, some treatment effect moderators (denoted V) are observed in the RCT but not in the target population. In the second case, we are concerned about possible effect moderation (represented generically as moderation by an unobserved U) that is completely unobserved, not even in the RCT. We call these the V case and the U case, respectively. In the V case, the article proposes (in Section 3) an outcome-model-based, an weighting-based, and a weighted-outcome-model-based sensitivity analysis. These sensitivity analysis methods for moderators V observed in the RCT but not in the target population are sound, and are NOT affected by the error we report in this note. In the U case, the article proposes (in Section 4) a bias-formula-based and a weighting-plus-bias-formula-based sensitivity analysis. These two sensitivity analysis methods for effect moderation by factors not observed in the RCT are affected by the flawed argument we explain below. The data example in this article, which represents a V case, is NOT affected by the error which only concerns the U case. 2. The flawed argument concerning the U case. Section 4 proposes sensitivity analyses for the U case (where concern is about effect moderation by factors not observed in the RCT), based on defining U as the remaining composite moderator after accounting for observed moderators (Z). That is, U is a composite variable that captures all effect moderation forces other than Z, and it is independent of observed covariates, including moderators Z and confounders X. (Intuitively, U is a combination of all the remaining moderators, after X, Z have been “regressed out.”) The argument was that, due to this independence, a regression model without U fit to the RCT sample can recover the coefficient representing effect moderation by Z, β_za, so the TATE formula (Formula Presented) can be used which is the basis of the bias-formula-based sensitivity analysis. (To remind of notation, SATE is the average treatment effect on the RCT sample, β_ua is the unknown parameter representing effect moderation by U, ∆_Z is the difference in mean Z between the target population and the RCT and ∆_U is the unknown difference in mean U between the target population and the RCT.) The other part of the argument was that, due to this independence, weighting based on X, Z does not change the distribution of U, so, after weighting, we still have the same simple ∆_U in the TATE formula which can be used as a sensitivity parameter (without having to deal with a weighted trial sample mean U that is different from and, thus, even more obscure than the original trial sample mean U). This is the basis of the weighting-plus-bias-formula-based sensitivity analysis. This argument is flawed. Both parts of this argument hang on the idea of a composite U independent of X, Z. The problem is that, with Z and U both differentially distributed between the RCT sample and the target population (the motivation for sensitivity analysis for U), the association of Z and U is generally different between the RCT sample and the target population due to collider bias when conditioning on sample membership. Thus, independence of U and Z does not occur in both places. It is independence in the RCT sample that would give the result of recovering β_za and of weighting not changing the distribution of U. But for the notion of U to be meaningful, we need it to be independent of X, Z in the target population instead, because the RCT sample is not the inference target. Unfortunately we do not have both. In addition, there is another flaw, which is that regressing out X, Z results in U being uncorrelated with X, Z, not independence. Replacing independence with uncorrelatedness, we also lose the claim that weighting based on X, Z does not change the distribution of U. To make clear the above point about the Z–U association differing between the RCT sample and the target population, consider the simple case in the causal graph in Figure 1, where the RCT sample is also drawn from the target population but is not representative of it. (The case where the RCT sample is from a different population requires a more complex graph, but the problem is essentially the same.) Z and U both influence sample membership (they influence the probability of RCT participation), making S a collider on the graph. Conditioning on S = 1 therefore changes the joint distribution of these variables from the p_z,u in the population to p_z,u|S=1 in the RCT sample. Essentially, the general strategies we rely on for sensitivity analysis here (an outcome model that captures effect moderation and/or a weighting procedure that balances the distribution of moderators) help with the V case but not the U case. This has been pointed out in (Formula Presented) Nguyen et al. (2018) which provides further elaboration on the application scope of V case methods. 3. Conclusion. The error detailed above nullifies the methods proposed for the U case in Nguyen et al. (2017), Section 4. The rest of the paper, including the Introduction (Section 1), the presentation of the two TATE estimation strategies (Section 2), the sensitivity analyses for moderators V observed in the RCT but not in the target population (Section 3), the data example (Section 5) and the discussion (Section 6) are NOT affected by this error.