Visit hbs.edu
Mar 6

Does Human-algorithm Feedback Loop Lead To Error Propagation? Evidence from Zillow’s Zestimate

12:30 pm - 2:00 pm EST Hybrid Event / Cotting 107
  • Meng Liu

We study how home sellers and buyers interact with Zillow’s Zestimate algorithm throughout the sales cycle of residential properties, with an emphasis on the implications of such interactions. In particular, leveraging Zestimate’s algorithm updates as exogenous shocks, we find evidence for a human-algorithm feedback loop: listing and selling outcomes respond significantly to Zestimate, and Zestimate is quickly updated for the focal and comparable homes after a property is listed or sold. This raises a concern that housing market disturbances may propagate and persist because of the feedback loop. However, simulation suggests that disturbances are short-lived and diminish eventually, mainly because all marginal effects across stages of the selling process—though sizable and significant—are less than one.

To further validate this insight in the real data, we leverage the COVID-19 pandemic as a natural experiment. We find consistent evidence that the initial disturbances created by the March-2020 declaration of national emergency faded away in a few months. Overall, our results identify the human-algorithm feedback loop in an important real-world setting, but dismiss the concern that such a feedback loop generates persistent error propagation.

Meng Liu is an assistant professor of marketing at Olin Business School, Washington University in St. Louis.​ Her research is empirically oriented, covering topics in ​Economics of AI/Algorithms, Quantitative Marketing, Digital Platforms, and Economics of Digitization.


This event is free and open to everyone. For event inquiries or questions, reach out to us at d3ln@hbs.edu.

Engage With Us

Join Our Community

Ready to dive deeper with the Digital Data Design Institute at Harvard? Subscribe to our newsletter, contribute to the conversation and begin to invent the future for yourself, your business and society as a whole.