Difference-in-differences (DID) designs are widely used across disciplines, such as social sciences and economics, to estimate the effect of interventions or policies. Often DID regression models are used to estimate the average treatment effect or average treatment effect on the treated. This is straightforward in cases where the treated group is exchangeable with the control group. However, many interventions are applied or given to a group based on a risk score. To the best of our knowledge, methods for assessing the impact of risk stratified interventions do not exist. There has been little guidance on how to analyze the impact of risk stratified interventions. We provide considerations on how dealing with a risk stratified intervention alters the estimand of interest and provide simulations to show how typical estimators of these estimands perform across various models. We consider the traditional DID model, a risk score adjusted model, and propose a novel DID model that includes interactions between the risk score and treatment. We illustrate these considerations using no-show visit data before and after the implementation of an intervention sending extra reminders to patients with a high-risk of a no-show.
Assessing risk stratified interventions with difference-in-differences
Maricela Cruz, Kaiser Permanente Washington
Authors: Maricela Cruz, Susan M Shortreed, Yates Coley
2023 AWM Research Symposium
Pure and Applied Talks by Mathematicians Enhancing Diversity in Graduate Education (EDGE) [Organized by Quiyana M. Murphy and Sofía Martínez Alberga]