Publications & Presentations
Publications
Sensitivity Analysis for Causal ML: A Use Case at Booking.com (2024)
Authors: Philipp Bach, Victor Chernozhukov, Carlos Cinelli, Lin Jia, Sven Klaassen, Nils Skotara, Martin Spindler
Conferences: Knowledge Discovery in Databases (KDD) 2024
Causal Machine Learning enables flexible estimation of causal effects from observational data, but its validity hinges on untestable assumptions like the absence of unobserved confounders. This paper emphasizes the importance of sensitivity analysis in assessing the robustness of causal findings when such assumptions may be violated, illustrating its practical relevance through a Booking.com use case.
• Presented at KDD 2024 Workshop - Causal Inference and Machine Learning in Practice
• Presented at Causal Data Science Meeting 2024
Increasing the sensitivity of experiments with rank transformation
Authors: Lin Jia, Sam Bailey Published in: Booking.ai
At Booking.com, even small improvements in key metrics can drive significant business value, making experimental sensitivity crucial. This work introduces applying t-tests on ranked data—a method conceptually similar to the Mann-Whitney U test—to enhance sensitivity in A/B tests with highly skewed metrics, emphasizing the importance of understanding data distributions, business goals, and simulation-based validation when selecting statistical methods.
Recent Talks
Tutorial: Robust and Streamlined causal analysis with Causal Inference Framework
PyData London 2024
Measuring causal impact is central to data-driven decision-making at Booking.com, yet A/B testing is not always feasible. This tutorial introduces a comprehensive causal inference framework that integrates academic insights with real-world business needs, providing data scientists with a structured, step-by-step approach to conduct robust, transparent, and consistent causal analyses across the full analytical lifecycle.
Unlocking incrementality: Enhancing Business Decisions through robust causal inference at Booking.com
Keynote Speech at Booking.com Science & Analytics Conference 2024
At Booking.com, a comprehensive causal inference framework has been developed to measure product impact beyond A/B testing by integrating academic rigor with practical business needs. The framework outlines five essential steps—from problem formulation to robustness checks—providing a systematic, transparent, and holistic approach that bridges experimental and observational methods to support robust, data-driven decision-making.