Would you stay at a robot’s house? How Airbnb uses machine learning to drive product development
Over the past few years, Airbnb has developed machine learning algorithms that change how hosts and guests interact with listings. It's a major departure from their early human-centered design days. How should the young company balance data-driven design with human-driven design?
How can Airbnb, a design-driven company, use machine learning – a variety of data-driven techniques – to make product development decisions to continue growing?
Since its founding, Airbnb has been known for its strength in design. The founders, Brian Chesky and Joe Gebbia, met at the Rhode Island School of Design[1], and grew the business by leveraging Design Thinking[2]. Since then, Airbnb has become the fourth largest tech startup in the world[3], and now needs to use scalable, data-driven methods to define future features.
In the past few years[4], Airbnb has begun to use machine learning to drive product decisions. It seems to fit their business well; with the sheer number of users they have, they are unable to speak with each and every one of them, but data scientists can analyze how people use the app to find areas to improve. Airbnb tracks actions taken on the platform by millions of guests and hosts, then creates algorithms to extract value from that data, using their observations to find ways to improve the app. This data-driven product development approach provides Airbnb a scalable, objective way to deliver more value to users.
Recently, Airbnb has been building their data science department and new, machine learning-based tools. They are using these tools to increase trust surrounding their system and to improve matches between guests and hosts. For example, to increase trust, Airbnb is using machine learning to fight fraud by identifying irregular user behaviors and shut them down through “friction” blocks[5]. They’re also leveraging machine learning to improve matches in a variety of ways. For hosts, Airbnb estimates fair prices and utilization rates[6], sends reminders to ensure reservations are met[7], and organizes the uploaded pictures in the order that will make the most impact[8]. For guests, Airbnb uses machine learning to analyze how risky it is for a guest to pass on an interesting listing[9], to find listings similar to ones that guests love[10], and to suggest homes listed by hosts likely to accept the guest’s request[11]. Additionally, Airbnb is creating machine learning-based tools that inform their business operations. For example, the data science team created a system that estimates users’ customer lifetime values, even on an individual level[12]. This information could change how the customer service representatives treat their users when they call in for help.
Airbnb’s engineers, who maintain a blog, present at conferences, publish in journals, and contribute to open-source repositories, have described a few ways they want to expand the machine learning operations in the future. Many of them are excited about Automated Machine Learning[13], a way to algorithmically identify algorithms to use to find insights from data. This is a promising frontier of the technology – it will further commoditize machine learning, and allow non-data scientists to use the techniques without much formal training, both within and outside of Airbnb. Additionally, the engineers are working on tools to offer understandable output from their algorithms[14], which provide explanations to users on why the algorithms made their suggestions. This will be important as more decisions are made using machine learning, because the humans behind them need to understand why decisions are made.
As Airbnb continues to grow and prepare for an IPO, the management team needs to consider how to balance the data-centered design with the human centered design they used to favor. It’s important to find a balance – data-driven product development and human centered design-driven product development solve different problems, and using just one technique could lead the organization to completely overlook some problems they’re facing. Focusing too much on data-driven development is tempting, considering it can be done from within the Airbnb office walls.
In addition to balancing the art and science of product development, the Airbnb management team needs to formalize a strategy on how to fight the misuse of data, especially as a way to justify and institutionalize bias. Airbnb has a history with incidents of bias, such as hosts cancelling due to racial discrimination[15]. They fight to prevent these issues, but data-driven product development could lead to more problems. Could algorithms that pair hosts with guests institutionalize the hosts’ biases? Is it possible that demographics would see different houses?
As it continues growing, where should Airbnb draw the line on data-driven development?
[1] Wikipedia. Accessed via web, November 2018. https://en.wikipedia.org/wiki/Brian_Chesky#cite_note-8
[2] “How Design Thinking Transformed Airbnb from a Failing Startup to a Billion Dollar Business.” First Round. Accessed via web, November 2018. https://firstround.com/review/How-design-thinking-transformed-Airbnb-from-failing-startup-to-billion-dollar-business/
[3] Friedman, Zack. “These 197 Companies are the World’s Most Valuable Unicorns.” Forbes. Accessed via web, November 2018. https://www.forbes.com/sites/zackfriedman/2017/05/30/tech-unicorns/#507dcc0d1179
[4] Wiggers, Kyle. “Airbnb details its journey to AI-powered search.” VentureBeat. Accessed via web, November 2018. https://venturebeat.com/2018/10/24/airbnb-details-its-journey-to-ai-powered-search/
[5] Press, David. “Fighting Financial Fraud with Targeted Friction”. Medium. Accessed via web, November 2018. https://medium.com/airbnb-engineering/fighting-financial-fraud-with-targeted-friction-82d950d8900e, https://medium.com/airbnb-engineering/architecting-a-machine-learning-system-for-risk-941abbba5a60
[6] Airbnb Engineering. “Aerosolve: Machine Learning for Humans.” Accessed via web, November 2018. https://medium.com/airbnb-engineering/aerosolve-machine-learning-for-humans-55efcf602665
[7] Cui, Tao. “Contextual Calendar Reminders.” Medium. Accessed via web, November 2018. https://medium.com/airbnb-engineering/contextual-calendar-reminder-key-to-successful-hosting-9be89e1a32fd
[8] Yao, Shijing. “Categorizing Listing Photos at Airbnb.” Medium. Accessed via web, November 2018. https://medium.com/airbnb-engineering/categorizing-listing-photos-at-airbnb-f9483f3ab7e3
[9] Dai, Peng. “Helping Guests Make Informed Decisions with Market Insights”. Medium. Accessed via web, November 2018. https://medium.com/airbnb-engineering/helping-guests-make-informed-decisions-with-market-insights-8b09dc904353
[10] Grbovic, Mihajlo. “Listing Embeddings in Search Rankings”. Medium. Accessed via web, November 2018. https://medium.com/airbnb-engineering/listing-embeddings-for-similar-listing-recommendations-and-real-time-personalization-in-search-601172f7603e
[11] Ifrach, Bar. “How Airbnb uses Machine Learning to Detect Host Preferences.” Medium. Accessed via web, November 2018. https://medium.com/airbnb-engineering/how-airbnb-uses-machine-learning-to-detect-host-preferences-18ce07150fa3
[12] Husain, Hamel. “Automated Machine Learning – A Paradigm Shift that Accelerates Data Scientist Productivity @ Airbnb.” Medium. Accessed via web, November 2018. https://medium.com/airbnb-engineering/automated-machine-learning-a-paradigm-shift-that-accelerates-data-scientist-productivity-airbnb-f1f8a10d61f8
[13] Husain, Hamel. “Automated Machine Learning – A Paradigm Shift that Accelerates Data Scientist Productivity @ Airbnb.” Medium. Accessed via web, November 2018. https://medium.com/airbnb-engineering/automated-machine-learning-a-paradigm-shift-that-accelerates-data-scientist-productivity-airbnb-f1f8a10d61f8
[14] Airbnb Engineering. “Aerosolve: Machine Learning for Humans.” Accessed via web, November 2018. https://medium.com/airbnb-engineering/aerosolve-machine-learning-for-humans-55efcf602665
[15] Solon, Olivia. “Airbnb host who canceled reservation using racist comment must pay $5,000.” The Guardian. Accessed via web, November 2018. https://www.theguardian.com/technology/2017/jul/13/airbnb-california-racist-comment-penalty-asian-american
Cover photo from Business Insider. Accessed via web, November 2018. https://static.businessinsider.com/image/561444eebd86ef175c8b4bb5-750.jpg
(705 Words)
Thanks Alex for sharing this – it’s fascinating to see how Airbnb is making this shift from human-centered design to data-driven design as they continue scaling. I can see how more basic data use can easily be combined with human-centered design. The examples you gave such as fighting fraud and improving matching appear to be data-driven solutions developed through a human-centered problem solving process. My view is that Airbnb needs to test each new data use and solution by answering two questions.
First, can the solution be explained and connected to a human problem and solution? You noted that this is a priority for engineers and that it will become more challenging as data use becomes more advanced.
Second, does the solution produce a better outcome than a human or relative to existing processes. You raised risks such as the risk of institutionalizing bias. This is an important risk that should be considered when applying data-driven solutions. However, a data-driven solution is less biased than a human, then it should be adopted.
Awesome post Alex. It’s interesting to how a data-driven approach can not only influence product / design decisions, but actually be inserted to the user experience and adapt the user flow as they move through it. The question you raised around bias is an important issue. Regulators in both Europe (GDPR) and the US (see https://obamawhitehouse.archives.gov/blog/2016/05/04/big-risks-big-opportunities-intersection-big-data-and-civil-rights) are starting to think about how to combat this. I’d be curious to see if there are new ML techniques that will evolve in order to adjust for biased data or if teams will need to incorporate a compliance-type role to look for discrimination.
This is an interesting article that brings to life machine learning through the lens of AirBnB, a company many millennials are familiar with. To me, the current algorithms findings that help hosts estimate fair prices and create more attractive listings are currently the biggest added value. As a host, you may have a great place to rent out, but you need help on the business side to maximize its full potential. On the guest side, I would love to see more machine learning innovation from AirBnB around recommending cities for travelers to visit based on past preferences. This could be a really cool way to grow the travel market even more and reduce some of the upfront research people are required to do when planning a trip.
Thanks for an interesting perspective. The “bias” issue is striking to me, and reminds me of the discussion we had around the Aspiring Minds case. In that class, someone brought up Amazon’s automated hiring tool (driven by machine learning and AI) that was scrapped last month because it had begun filtering out female candidates (article here: https://www.reuters.com/article/us-amazon-com-jobs-automation-insight/amazon-scraps-secret-ai-recruiting-tool-that-showed-bias-against-women-idUSKCN1MK08G). We’re at an interesting crossroads, where humans and machines are both fueled by long histories of biases, both conscious or unconscious. Until machines can master EQ like (some!) humans can, there will always be gaps in how an ML-based tool helps to hire — or in Airbnb’s case, connect hosts and guests in a fair and un-biased way.