A Case Study of the Impact of Data-Adaptive Versus Model-Based Estimation of the Propensity Scores on Causal Inferences from Three Inverse Probability Weighting Estimators

In investigations of the effect of treatment on outcome, the propensity score is a tool to eliminate imbalance in the distribution of confounding variables between treatment groups. Recent work has suggested that Super Learner, an ensemble method, outperforms logistic regression in nonlinear settings; however, experience with real‐data analyses tends to show overfitting of the propensity score model using this approach. We investigated a wide range of simulated settings of varying complexities including simulations based on real data to compare the performances of logistic regression, generalized boosted models, and Super Learner in providing balance and for estimating the average treatment effect via propensity score regression, propensity score matching, and inverse probability of treatment weighting. We found that Super Learner and logistic regression are comparable in terms of covariate balance, bias, and mean squared error (MSE); however, Super Learner is computationally very expensive thus leaving no clear advantage to the more complex approach. Propensity scores estimated by generalized boosted models were inferior to the other two estimation approaches. We also found that propensity score regression adjustment was superior to either matching or inverse weighting when the form of the dependence on the treatment on the outcome is correctly specified.

Section: Discussionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Should a propensity score model be super? The utility of ensemble procedures for causal adjustment

Alam

Moodie

Stephens

2018

“…Data-adaptive regression methods have recently been adopted in the missing data and causal inference literature to tackle the problem of model misspecification. [8][9][10][11][12][13][14][15][16][17] Techniques such as classification and regression trees, adaptive splines, neural networks, 1 -regularization, support vector machines, boosting and ensembles, etc, offer flexibility in functional form specification that is not available for traditional approaches such as the Cox proportional hazards model. However, under inconsistency of one nuisance estimator, the large-sample analysis of the resulting data-adaptive doubly robust estimators requires empirical process conditions that are often not verifiable.…”

Section: Introductionmentioning

confidence: 99%

Statistical inference for data‐adaptive doubly robust estimators with survival outcomes

Díaz

2019

The consistency of doubly robust estimators relies on the consistent estimation of at least one of two nuisance regression parameters. In moderate‐to‐large dimensions, the use of flexible data‐adaptive regression estimators may aid in achieving this consistency. However, n1/2‐consistency of doubly robust estimators is not guaranteed if one of the nuisance estimators is inconsistent. In this paper, we present a doubly robust estimator for survival analysis with the novel property that it converges to a Gaussian variable at an n1/2‐rate for a large class of data‐adaptive estimators of the nuisance parameters, under the only assumption that at least one of them is consistently estimated at an n1/4‐rate. This result is achieved through the adaptation of recent ideas in semiparametric inference, which amount to (i) Gaussianizing (ie, making asymptotically linear) a drift term that arises in the asymptotic analysis of the doubly robust estimator and (ii) using cross‐fitting to avoid entropy conditions on the nuisance estimators. We present the formula of the asymptotic variance of the estimator, which allows for the computation of doubly robust confidence intervals and p values. We illustrate the finite‐sample properties of the estimator in simulation studies and demonstrate its use in a phase III clinical trial for estimating the effect of a novel therapy for the treatment of human epidermal growth factor receptor 2 (HER2)–positive breast cancer.

“…However, the large sample analysis of treatment effect estimates based on machine learning requires hard-to-verify assumptions, and often yield estimators which are not n 1/2 consistent, and for which no statistical inference (ie, p values and confidence intervals) is available. Nonetheless, data-adaptive estimation has been widely used in estimation of causal effects from observational data (for a few examples, see previous works 7,[9][10][11][12] ). Indeed, the statistics field of "_targeted learning" (see, eg, van der Laan et al [13][14][15] ) is concerned with the development of optimal (n 1/2 consistent, asymptotically normal, and efficient) estimators of smooth low-dimensional parameters through the use state-of-the-art machine learning.We develop estimators for analyzing data from randomized trials with missing outcomes, when the missingness probabilities and the outcome regression are estimated with data-adaptive methods.…”

mentioning

confidence: 99%

Doubly robust inference for _targeted minimum loss–based estimation in randomized trials with missing outcome data

Díaz

Laan

2017

Missing outcome data is one of the principal threats to the validity of treatment effect estimates from randomized trials. The outcome distributions of participants with missing and observed data are often different, which increases the risk of bias. Causal inference methods may aid in reducing the bias and improving efficiency by incorporating baseline variables into the analysis. In particular, doubly robust estimators incorporate estimates of two nuisance parameters: the outcome regression and the missingness mechanism (i.e., the probability of missingness conditional on treatment assignment and baseline variables), to adjust for differences in the observed and unobserved groups that can be explained by observed covariates.To obtain consistent estimators of the treatment effect, one of these two nuisance parameters mechanism must be consistently estimated. Such nuisance parameters are traditionally estimated using parametric models, which generally preclude consistent estimation, particularly in moderate to high dimensions. Recent research on missing data has focused on data-adaptive * corresponding author: ild2005@med.cornell.edu 1 arXiv:1704.01538v1 [stat.ME] 5 Apr 2017 estimation of the nuisance parameters in order to achieve consistency, but the large sample properties of such estimators are poorly understood. In this article we discuss a doubly robust estimator that is consistent and asymptotically normal (CAN) under data-adaptive consistent estimation of the outcome regression or the missingness mechanism. We provide a formula for an asymptotically valid confidence interval under minimal assumptions. We show that our proposed estimator has smaller finite-sample bias compared to standard doubly robust estimators. We present a simulation study demonstrating the enhanced performance of our estimators in terms of bias, efficiency, and coverage of the confidence intervals. We present the results of an illustrative example: a randomized, double-blind phase II/III trial of antiretroviral therapy in HIV-infected persons, and provide R code implementing our proposed estimators.