A large portion of Leading with People Analytics (LPA) has been focused on regression analysis, a common approach in statistics in which we are concerned with quantifying how changes in variable x are associated with changes in variable y. Another increasingly popular family of methods within the field of statistics is causal inference analysis, in which we are instead concerned with determining whether changes in variable x cause changes in variable y. Many industry engineers and researchers are showing interest in this growing sub-field of statistics, as it offers a more principled approach to understanding the causes behind the results that we see from observations or experiments. Uber, one of the world’s largest ridesharing and food delivery companies, has already formed a causal inference community whose task is to apply causal inference methods that bring richer insights into operations analysis, product development, and other areas critical to improving the user experience.
In the article, “Using Causal Inference to Improve the Uber User Experience,” Harinen and Li layout what is causal inference, why it is important, and how Uber engineers are using various causal inference methods to solve critical data science questions. For example, those at Uber Labs are interested in how experiencing an event like a delay in food delivery can influence customers’ future engagement with the Uber Eats platform. Answering this question requires working with observational data, as running an experiment under the given circumstances is infeasible since it would negatively impact the customer experience. As we’ve discussed during class, simply calculating the difference in future customer engagement between users who experienced a delay versus those who did not would likely not result in a meaningful answer due to the presence of potential sources of confounding such as the number of customer food orders. Therefore, Harinen and Li highlight how Uber engineers are using causal inference methods such as propensity score matching to account for sources of confounding and achieve more causal estimates of treatment effect. This is one of many illustrations of the use of causal inference methods at Uber Labs provided throughout the article. Another particularly interesting example is the use of regression discontinuity to investigate how different levels of dynamic pricing influence customers’ decisions to request a ride on the Uber platform.
Overall, as a data science student who is particularly interested in the various applications of causal inference methods, it is exciting to see that high-profile companies such as Uber are using these methods to better inform decision-making. Although I am a major proponent of the use of causal inference analysis to investigate interesting questions in data science, I am also aware of its limitations. The validity of the estimates achieved using causal inference methods often rests on certain untestable assumptions. Ensuring that these assumptions are reasonable requires a high level of domain expertise. Therefore, I’d imagine that the engineers carrying out causal inference analysis at Uber Labs are in close collaboration with the domain experts who have substantive knowledge of the business problems at hand. I am also curious as to whether Uber is using causal inference analysis to investigate its own employee data. For instance, one interesting application that comes to mind involves the use of regression discontinuity to study how company-level policy changes affect employee satisfaction. There are surely various other applications of causal inference methods, and it will be interesting to learn more as Uber and other major technology companies continue to uncover them.
SOURCE: Harinen, Totte, and Bonnie Li. “Using Causal Inference to Improve the Uber User Experience.” Uber Engineering, 19 June 2019. https://eng.uber.com/causal-inference-at-uber/.