Causal Inference in Data Science: A/B Testing & Randomized Trials with Covariate Adjustment
Efficiency & Statistical Power gains from Conditional Covariate Adjustment in A/B Testing & Randomized Trials
1: Background and Motivation
Causal Inference is a field that touches several domains and is of interest to a wide range of practitioners including Statisticians, Data Scientists, Machine Learning Scientists, and other Computational Researchers.
To date I have written several pieces on methods/topics in the Causal Inference space. These include:
- Efficient Sampling Frameworks in Causal Inference
- Doubly Robust Estimation Techniques in Causal Inference
- G-Estimation of Semi-Parametric Structural Nested Models
- Recovery of Causal Effects in the presence of M-Bias Structures
- The need for Marginal Structural Modeling in AB Tests / Randomized Trials for Informative Censoring Adjustment
- Valid Inferential Coverage with Multiple Comparisons
This piece concerns A/B Tests (aka randomized trials) and specification of statistically efficient conditional sampling estimators via covariate adjustment. A mathematically rigorous justification and computational simulation is provided for benefits of adjusting for strong predictors of the outcome of interest, even in the absence of confounding.
The contents of the piece are as follows:
2: Simple Idealized A/B Test with Continuous Outcome
We will begin by exploring the benefits of covariate adjustment in an A/B testing framework under the assumption that the outcome of interest is continuous:
2.1: Specification of Toy Example
Let us postulate a simple “idealized” A/B test with randomized binary intervention A and continuous normally distributed outcome Y. By “idealized” we…
