Big Data — A deal recommended for YOU
How is Big Data and Recommendation Engines shaping customers’ preferences and experiences in e-commerce?
Nowadays, many digital firms, from FB and Google to Amazon and Netflix, access, use, and track lots of our data.
Especially e-commerce firms, can access a huge amount of structure and unstructured data from users –from our customer profiles, our historical transactions data, our regional and seasonal buying patterns, and even browsing behavior – that they can then use to model users’ behavior and capture value in many forms: increase sales and revenues, cut inventory and transportation costs, and enhance customer retention, among others.
An interesting way on how big e-commerce companies like Amazon and Alibaba are creating value, is through sophisticated recommendation engines using big data analytics. A recommendation engine or system, is an intelligent filtering system that show users content, that we would “potentially like”, tailored to our preferences. The magic about these discovery assistants is that it expose us to content that we probably wouldn’t have “discovered” as quickly or efficiently by ourselves[1]. By doing this, recommendation engines create value to customers by delivering highly personalized recommendations in real-time, and it can highly affect our decision-making process and become a critical part of our user experience on how we interact with our favorite platforms. And therefore, also captures value to the firms by driving revenues and engagement. According to McKinsey & Company[2], 35% of Amazon revenues in 2015 were derived from its recommendation engine.
>> Steve Jobs — “a lot of times, people don’t know what they want until you show it to them”[3] <<
The most well-known types of recommendation engines are four[4] and use data and algorithms slightly differently:
- Collaborative filtering: is a predictive algorithm based on a matrix of (past) preferences or reviews by users for items. A successful implementation requires an initial set of “transactions” or described preferences, to avoid the cold-start problem: the algorithm cannot create recommendations, unless it can be based on this initial matrix of preferences.
- Content based: algorithms that recommend similar items based on specific item attributes or content. Note that – like described in my previous post with “Fake News” – this can cause some problems of homogeneity if exposing users to content that has always the same characteristics.
- Social and demographic: algorithms that match content and reviews to those liked by friends, friends of friends, and demographically-similar profiles.
- Contextual: algorithm that match recommendations to the user’s current context, for example by analyzing web browsing.
Truth is, that companies are utilizing a hybrid of all of these kinds[5] and implementing machine and deep learning to continuously adjust the different weights of each type and characteristic to improve the relevance of the results.
Recommendations translation to UX/UI
So the magic is all about recommending the exact accurate promotion at the right time? Well… that is part of the story, but not all.
Of course, recommendation accuracy is important, but it is not the only key. The User interface in the delivery of these recommendations is even more powerful. For example, recognizing that there are some items that you buy once versus others that you consume repetitively could be useful not to damage the customer experience.
Sometimes, to solve for some of these issues, explanations on how these recommendations are being provided can help increase trust among users: tags like “People who bought this, also bought…” or “People who watched this, also enjoyed…” are effective in enhancing transparency.
Challenges ahead:
Of course, these recommendation engines are far from perfect, and billionare investments are being destined in dramatically evolving these valuable mechanisms. Still, there also some short-term challenges (many that have been discussed during our course) that should be addressed:
1) “Garbage in, garbage out”: Because all these algorithms rely on data, the availability of good quality big data is the key to adding value to the customer and the organizations. Poor quality data might arise from, for example, redundant applications and databases, or data storage costs difficulting access and use.
2) Sensitivity and Privacy: Given many of the current challenges with data sharing and handling, companies should beware of how much care they are putting in safe handling individual and organizational privacy and data security. Creeping out users, by exposing they are being tracked or showing that they can access they facebook friends profiles, might backlash by losing customer willingness to share other types or even more relevant data in the future.
Other Dimensions:
Using data is not exclusive the particular tool (recommendation engines) or platform (website, app). In fact, Amazon is experimenting with others things like:
Dynamic Pricing: Amazon.com dynamic pricing system monitors competing prices, alerting and informing Amazon’s pricing every 15s, which has also resulted in increased sales to Amazon by allowing the company to offer competitive prices to customers (even in special events like x-mas sales)
Interaction with recommenders, the AI evolution: The way that we will interact with these recommendation engines is eveolving, and will be increasingly important in the future. How long do you believe it will be for Alexa to develop a Q&A interface that will result in the perfect match for your very urgent need? My guess is not very long.
The future space for recommendation systems is still unknowned, but for sure looks to be headed toward a totally customized environment for each user, which will be unlike anything we’ve ever experienced[6].
Sources:
[2] https://www.mckinsey.com/industries/retail/our-insights/how-retailers-can-keep-up-with-consumers
[4] https://yanirseroussi.com/2015/10/02/the-wonderful-world-of-recommender-systems/
[5] https://medium.com/netflix-techblog/learning-a-personalized-homepage-aa8ec670359a
[6] https://techcrunch.com/2015/09/28/the-evolving-landscape-of-recommendation-systems/
Other sources of reference:
https://hbr.org/2017/08/great-digital-companies-build-great-recommendation-engines
https://www.kdnuggets.com/2015/10/big-data-recommendation-systems-change-lives.html
https://www.techemergence.com/use-cases-recommendation-systems/
http://infolab.stanford.edu/~ullman/mmds/ch9.pdf
https://www.wired.com/2015/04/now-anyone-can-tap-ai-behind-amazons-recommendations/
https://link.springer.com/content/pdf/10.1007%2Fs12525-016-0219-0.pdf
https://www.cs.umd.edu/~samir/498/Amazon-Recommendations.pdf
https://www.techemergence.com/artificial-intelligence-retail/