2024 Causal Science Center Conference Poster Session Recap

On October 11th, the Stanford Causal Science Center held a one-day conference with presentations by graduate students and postdocs working on methods for and applications in causal inference. If you missed the in-person poster session, check out the posters and abstracts from our speakers below. 

Congratulations to all of our poster presenters from the 2024 Causal Science Center Conference! 

Poster Title: Design-based inference for generalized network experiments with stochastic interventions

Presenter: Ambarish Chattopadhyay (Postdoctoral Fellow, Stanford Data Science advised by Prof. Guido Imbens)

Summary: We develop design-based approaches to inference for a range of causal effects in complex network experiments, where some units are ineligible to receive treatment.

Abstract: A growing number of researchers are conducting randomized experiments to analyze causal relationships in network settings where units influence one another. A dominant methodology for analyzing these experiments is design-based, leveraging random treatment assignments as the basis for inference. Most existing methods, however, can only be applied to experiments where all units in the network are eligible to receive the treatment. This restriction represents an important limitation because many modern network experiments involve some units that are not eligible for treatment assignment or outcome measurement. A prominent example is bipartite network experiments, where treatment is randomized among one set of units while the outcome is measured for a separate set of units. Conducting design-based inference for such experiments is challenging due to the inherent dependence within and between the ineligible and eligible units. In addition, although the existing methods focus on the average treatment effect among all treatment-eligible units, researchers may be interested in estimating causal effects for different target populations in the network, such as treatment-ineligible units or a group that includes some of both treatment-eligible and ineligible units.

In this paper, we generalize this design-based approach to accommodate complex experiments with a variety of causal estimands and different target populations. We propose a broad class of causal estimands based on stochastic interventions for generalized network experiments. Using a design-based approach, we show how to estimate these causal quantities without bias and develop conservative variance estimators. We apply our methodology to a randomized experiment in education where participation in an anti-conflict promotion program is randomized among selected students. Our analysis estimates the causal effects of treating each student or their friends among different target populations in the network. We find that the program improves the overall conflict awareness among students but does not significantly reduce the total number of such conflicts.

Poster Title: When Does Interference Matter? Decision-Making in Platform Experiments

Presenter: Anushka Murthy (Graduate Student, Management Science and Engineering, Stanford Data Science Scholar, advised by Prof. Ramesh Johari)

Summary: We study an experimenter who runs an A/B test on a two-sided platform, employs a naive (ignoring experimental unit covariance) difference-in-means GTE estimator and naive variance estimator, and uses a t-test to test the null hypothesis of GTE=0, and we find that ignoring correlation when forming the t-test statistic controls the type I error and results in higher power than if unbiased estimators were used for a broad class of treatments.

Abstract: We take the point of view of an experimenter who would like to run an A/B test on their two-sided platform to decide whether to launch a new feature. The experimenter randomizes a portion of the customers to treatment and analyzes the experimental data using a na ̈ıve difference-in-means (DM) estimator to estimate the global treatment effect (GTE), as well as a standard associated na ̈ıve variance estimator (ignoring covariances). Ultimately, to decide whether to launch the feature, the platform uses a standard frequentist hypothesis testing paradigm: they test the null hypothesis of no treatment effect, using a t-test statistic with the na ̈ıve DM estimator and the na ̈ıve variance estimator.

Unfortunately, the A/B test suffers from interference between experimental units because customers interact with the same (limited) inventory. It is well-known that this interference can lead to bias in the DM estimator and much recent work (citations omitted due to space constraints) has been done on potential debiasing techniques. However, prior work does not characterize the impact of this interference on the platform’s subsequent decision rule. This impact is nontrivial, since it depends not only on the bias on the DM estimator, but also on its variance, the behavior of the na ̈ıve variance estimator, and their interaction through the test statistic and subsequent decision rule.

In our work, using a benchmark Markov chain model for the two-sided platform, we study the impact of interference on the false positive probability and statistical power when the experimenter uses naïve estimation. We characterize these quantities for a particular class of treatments that are “monotone” (i.e., where the direction of treatment effect is the same for all system states). We find surprisingly in this setting that the false positive probability is correctly controlled despite the presence of interference, and that the statistical power is larger than that achieved by using any unbiased estimator. In other words, in this setting the platform is actually better off not using a debiased estimator. Using simulations, we also investigate the same quantities when treatment is not necessarily monotone, and provide insights into conditions under which debiasing may be valuable for the platform.

Poster Title: Optimizing Adaptive Experiments: A Unified Approach to Regret Minimization and Best-Arm Identification

Presenter: Chao Qin (Postdoctoral Scholar, Graduate School of Business / Columbia DRO) 

Summary: We propose a unified model that simultaneously accounts for within-experiment performance and post-experiment outcomes, and provide a sharp theory of optimal performance in large populations that not only unifies canonical results but also uncovers novel insights.

Abstract: Practitioners conducting adaptive experiments often encounter two competing priorities: maximizing total welfare (or ‘reward’) through effective treatment assignment and swiftly concluding experiments to implement population-wide treatments. Current literature addresses these priorities separately, with regret minimization studies focusing on the former and best-arm identification research on the latter. This paper bridges this divide by proposing a unified model that simultaneously accounts for within-experiment performance and post-experiment outcomes. We provide a sharp theory of optimal performance in large populations that not only unifies canonical results in the literature but also uncovers novel insights. Our theory reveals that familiar algorithms, such as the recently proposed top-two Thompson sampling algorithm, can optimize a broad class of objectives if a single scalar parameter is appropriately adjusted. In addition, we demonstrate that substantial reductions in experiment duration can often be achieved with minimal impact on both within-experiment and post-experiment regret.

Poster Title: Difference-in-differences using longitudinal wastewater SARS-CoV-2 concentrations to evaluate the impact of Stanford’s COVID-19 public health policies

Presenter: Elana Chan (Graduate Student, Civil and Environmental Engineering, advised by Prof. Alexandria Boehm)

Summary: We applied difference-in-differences to longitudinal concentrations of SARS-CoV-2 RNA in wastewater to evaluate the impact of COVID-19 policies on the spread of COVID-19 among a university population.

Abstract: Public health policy impact evaluation is challenging to study because randomized controlled experiments are infeasible to conduct, and policy changes often coincide with non-policy events. Quasi-experiments do not use randomization and can provide useful knowledge for causal inference. Here we demonstrate how longitudinal wastewater monitoring of viruses at a small geographic scale may be used in a quasi-experimental design to evaluate the impact of COVID-19 public health policies on the spread of COVID-19 among a university population. We first evaluated the correlation between incident, reported COVID-19 cases and wastewater SARS-CoV-2 RNA concentrations and observed changes to the correlation over time, likely due to changes in testing requirements and testing options. Using a difference-in-differences approach, we then evaluated the association between university COVID-19 public health policy changes and levels of SARS-CoV-2 RNA concentrations in wastewater. We did not observe changes in SARS-CoV-2 RNA concentrations associated with most policy changes. Policy changes associated with a significant change in campus wastewater SARS-CoV-2 RNA concentrations included changes to face covering recommendations, indoor gathering bans, and routine surveillance testing requirements and availability.

Poster Title: Sparse-group SLOPE: adaptive bi-level selection with FDR-control

Presenter: Fabio Feser (Visiting PhD Student, Biomedical Data Science, advised by Prof. Chiara Sabatti)

Summary: A new penalized regression approach designed to control the false discovery rate with applications in genetics.

Abstract: A new high-dimensional approach for simultaneous variable and group selection is proposed, called sparse-group SLOPE (SGS). SGS achieves false discovery rate control at both variable and group levels by incorporating the SLOPE model into a sparse-group framework and exploiting grouping information. Aproximal algorithm is implemented for fitting SGS that works for both Gaussian and Binomial distributed responses. Through the analysis of both synthetic and real datasets, the proposed SGS approach is found to outperform other existing lasso- and SLOPE-based models for bi-level selection and prediction accuracy. Further, model selection and noise estimation approaches for selecting the tuning parameter of the regularisation model are proposed and explored.

Poster Title: Efficient combination of observational and experimental datasets under general restrictions on outcome mean functions

Presenter: Harrison Li (Graduate Student, Statistics, advised by Prof. Art B. Owen)

Summary: We show how to optimally leverage modeling and/or structural assumptions linking observational and experimental datasets to maximize precision in estimating finite-dimensional treatment effects.

Abstract: A researcher collecting data from a randomized controlled trial (RCT) often has an auxiliary observational dataset that is confounded or otherwise biased for estimating causal effects. Researchers commonly impose structure on the outcome mean function — the conditional expectation of the outcome of interest given observed covariates — in the two datasets to effectively combine them. Such structure can arise, for instance, from outcome-mediated selection bias or confounding bias of a low-dimensional parametric form. We propose a succinct framework to derive the efficient influence function and semiparametric lower bound for any identifiable pathwise differentiable estimand under a broad range of assumptions on the outcome mean function. We find that with homoskedastic outcomes and a constant propensity score in the RCT, even strong parametric assumptions cannot improve the efficiency bound for estimating various average treatment effects. We then propose one-step estimators that leverage double machine learning to ensure they achieve the efficiency bound with nonparametric mean function estimates. Ultimately, we empower a researcher with custom modeling restrictions on the outcome mean function to systematically construct causal estimators that maximially leverage their assumptions for variance reduction. We empirically demonstrate substantial precision gains from our estimator using various numerical studies and data examples.

Poster Title: Regularized DeepIV with Model Selection

Presenter: Hui Lan (Graduate Student, Institute of Computational and Mathematical Engineering, advised by Prof. Vasilas Syrgkanis)

Summary: We present novel theoretical guarantees for a Tikhonov-regularized variant of the seminal DeepIV method in nonparametric instrumental variable regression, and further demonstrate that this approach enables model selection procedures that can achieve the oracle rates in the misspecified regime.

Abstract: In this paper, we study nonparametric estimation of instrumental variable (IV) regressions. While recent advancements in machine learning have introduced flexible methods for IV estimation, they often encounter one or more of the following limitations: (1) restricting the IV regression to be uniquely identified; (2) requiring minimax computation oracle, which is highly unstable in practice; (3) absence of model selection procedure. In this paper, we analyze a Tikhonov-regularized variant of the seminal DeepIV method, called Regularized DeepIV (RDIV) regression, that can converge to the least-norm IV solution, and overcome all three limitations. RDIV consists of two stages: first, we learn the conditional distribution of covariates, and by utilizing the learned distribution, we learn the estimator by minimizing a Tikhonov-regularized loss function. We further show that RDIV allows model selection procedures that can achieve the oracle rates in the misspecified regime. When extended to an iterative estimator, we prove that RDIV matches the current state-of-the-art convergence rate. Our results provide the first rigorous guarantees for the empirically well-established DeepIV method, showcasing the importance of regularization which was absent from the original work.

Poster Title: Root cause discovery

Presenter: Jinzhou Li (Postdoctoral Scholar, Statistics, advised by Prof. Emmanuel Candes)

Abstract: We study the problem of root cause discovery, which aims to find the intervened variable in one interventional sample using observational samples as a reference. This problem is motivated by the real-world challenge of discovering the disease-causing gene in rare disease patients. We consider a linear structural equation model where the causal ordering is unknown. We begin by examining a simple method of using marginal z-scores, and we characterize the conditions under which this method succeeds or fails, showing that it generally cannot identify the root cause. We then prove, without any additional assumptions, that the root cause is identifiable even if the causal ordering is not. Two key ingredients of this identifiability result are the use of permutations and Cholesky decomposition. Furthermore, we characterize the feasible permutations that yield the correct root cause, based on which we propose a valid and efficient method for root cause discovery. Finally, we examine the performance of our method through simulations, and then adapt and apply it to discover disease-causing genes in the high-dimensional gene expression dataset that motivated this study.

Poster Title: Consistency of Neural Causal Partial Identification

Presenter: Jiyuan Tan (Graduate Student, Management Science and Engineering, advised by Prof. Vasilis Syrgkanis and Prof. Jose Blanchet)

Summary: We provide theoretical foundations for neural causal partial identification.

Abstract: Recent progress in Neural Causal Models (NCMs) showcased how identification and partial identification of causal effects can be automatically carried out via training of neural generative models that respect the constraints encoded in a given causal graph. However, formal consistency of these methods has only been proven for the case of discrete variables or only for linear causal models. In this work, we prove consistency of partial identification via NCMs in a general setting with both continuous and categorical variables. Further, our results highlight the impact of the design of the underlying neural network architecture in terms of depth and connectivity as well as the importance of applying Lipschitz regularization in the training phase. In particular, we provide a counterexample showing that without Lipschitz regularization the NCM may not be asymptotically consistent. Our results are enabled by new results on the approximability of structural causal models via neural generative models, together with an analysis of the sample complexity of the resulting architectures and how that translates into an error in the constrained optimization problem that defines the partial identification bounds.

Poster Title: Using extreme events for causal inference: Causality in extremes of time series and extrapolation of causal effects

Presenter: Juraj Bodik (Visiting PhD Student at UC Berkeley, Statistics) 

Summary: We explore various definitions of how an 'extreme event at $X$ causes an extreme event at $Y$' and demonstrate equivalences with classical causality concepts, particularly in time series.

Abstract: As extreme events become more frequent in our society, numerous applications seek to understand their causes and effects, such as extreme weather driven by climate change or extremes in stock market. Causality in the body can differ from causality in tail; the causal structure is often simpler and clearer in the extremes. Despite this, current methodologies struggle to effectively handle extremes, and we need better tools to quantify their impact on an outcome variable. 

In this talk, we introduce a novel framework for Granger-type causality in extremes, designed to infer causal links from extreme events in time series. We explore different definitions of how an ’extreme event at X causes an extreme event at Y’. We establish equivalences between causality in extremes and other causal concepts, including (classical) Granger causality and structural causality. We prove other key properties of causality in extremes and show that the framework is especially helpful under the presence of hidden confounders. We also propose a novel inference method for detecting the presence of causality in extremes from observational data. Our method is model-free, can handle non-linear and high-dimensional data, outperforms current state-of-the-art methods in all considered setups, both in performance and speed, and was found to uncover coherent effects when applied to financial and extreme weather problems. While we focus on time series data, we discuss how these ideas can be used in the potential outcome framework. 

Poster Title: Real-World Causal Inference in Oncology: Regression Discontinuity Designs to Improve Skin Cancer Care

Presenter: Max Schuessler (Graduate Student, Biomedical Data Science, advised by Prof. Robert Tibshirani)

Poster Title: Optimal Mechanisms for Demand Response: An Indifference Set Approach

Presenter: Mohammad Mehrabi (Graduate Student, Graduate School of Business, advised by Prof. Stefan Wager and Prof. Omer Karaduman)

Summary: We study optimal demand response when users have indifference sets of consumption profiles, showing that while dynamic pricing is not always optimal, it becomes optimal as the number of consumers grows, and we propose algorithms to derive it.

Abstract: Renewable energy sources like solar and wind generate electricity at uncontrollable times, posing challenges for efficient grid utilization. By leveraging consumer flexibility through demand-response programs, energy consumption can be shifted to align with renewable production. We investigate optimal demand response using home energy management systems (HEMS) that compute "indifference sets" of acceptable consumption profiles, receive grid signals, and control devices accordingly. We demonstrate that while dynamic pricing doesn't generally achieve optimal demand response, it becomes asymptotically optimal as the number of consumers grows. An efficient algorithm derives these optimal dynamic prices by querying HEMS about planned consumption under different prices. Our approach, tested in an OpenDSS grid simulation, achieves meaningful demand response without causing grid instability.

Poster Title: Price Experimentation and Interference

Presenter: Orrie Page (Graduate Student, Management Science and Engineering, advised by Prof. Ramesh Johari and Prof. Gabriel Weintraub) 

Summary: We characterize the bias the arises in A/B tests when firms modify a continuous parameter like price.

Abstract: In this paper, we examine biases arising in A/B tests where firms modify a continuous parameter, such as price, to estimate the global treatment effect of a given performance metric, such as profit. These biases emerge in canonical experimental estimators due to interference among market participants. We employ structural modeling and differential calculus to derive intuitive characterizations of these biases. We then specialize our general model to a standard revenue management pricing problem. This setting highlights a key pitfall in the use of A/B pricing experiments to guide profit maximization: notably, the canonical estimator for the expected change in profits can have the wrong sign. In other words, following the guidance of canonical estimators may lead firms to move prices in the wrong direction, inadvertently decreasing profits relative to the status quo. We introduce a novel debiasing technique for such experiments, that only requires that the firm equally split experimental units between treatment and control. We apply these results to a two-sided market model and show how this “change of sign” regime depends on model parameters such as market imbalance, as well as the price markup. We also extend our revenue management model to an alternative specification in which firms set a fixed sales fee across heterogeneous products, and show that all our results extend to this setting as well.

Poster Title: Dynamic Local Average Treatment Effects

Presenter: Ravi Sojitra (Graduate Student, Management Science and Engineering, advised by Prof. Vasilas Syrgkanis)

Summary: We provide identification, estimation, and inference for Dynamic Local Average Treatment Effects, and we prove that sequentially extending identifying assumptions in Imbens and Angrist (1994) are not sufficient for theidentification of Dynamic LATEs.

Abstract: We enable identification, estimation, and inference for Local Average Treatment Effect (LATE) estimands in Dynamic Treatment Regimes (DTRs) with noncompliance (e.g. digital recommendation and medical treatment). These are settings where treatment is encouraged in each time period depending on previous encouragements, treatments, and states (e.g. short-term outcomes and time-varying covariates). Although one may hope to leverage estimates of the effects of switching from one sequence of treatments to another, such quantities are not estimable under noncompliance and standard identifying assumptions for DTRs and LATEs. We introduce two conditions to enable the identification of Dynamic LATEs that quantify effects for subpopulations that would comply with encouragements. First, we show that One-Sided Noncompliance enables identification of all Dynamic LATEs corresponding to treating in a single period only. Second, we introduce a generalization of Staggered Adoption to prove the identification of effects of treating in multiple time periods. Informally, this generalization requires the treatment effect of not continuing to comply to be independent of whether one does not continue to comply. We also show that a sequential extension of Monotonicity in Imbens and Angrist (1994) is not sufficient for the first result and an additional assumption is necessary for the second result. Finally, we use the automatic debiased machine learning framework to perform estimation and inference based on our identification results.