Mirror, Mirror on the Wall: L’Oréal’s ModiFace Uses AI to Find Your Perfect Shade Match
L’Oréal’s acquisition of ModiFace, the leading provider of augmented reality solutions powered by machine learning, could revolutionize the way consumers experience beauty and make purchasing decisions – will competitors follow?
ModiFace’s virtual experimentation technology has been changing the way consumers shop for makeup. It can recommend the perfect shade of foundation based on a photograph or show a user what a specific shade of lipstick would look like on his or her face, not a model selected by the brand or retailer. ModiFace uses machine learning to power its suite of augmented reality offerings, including:
- Beauty try-on simulations on live video: simulations can reproduce precise makeup shades from brand partners and turn them into realistic digital simulations with dynamic lighting adaption
- Real-time facial analysis and feature tracking: precise measurement of 68 unique facial parameters like lip and eye edges, iris size and location, head pose, skin features such as spots, textures and wrinkles1
During early trials, ModiFace ran an assessment of its technology on a broad set of social media users, scanning a sample of 1.6 million photos across 364,012 users to accurately identify a user’s skin tone. The scan accurately detected skin tone with 98.3% accuracy2. This kind of feature recognition is critical in a consumer’s beauty journey, especially in categories like foundation and concealer, where a key consumer concern in the shopping process is how a brand’s new product offerings fits that consumer’s specific skin type and skin color. Retailers and brands spend significant time and resources training in-store staff to approximate those same details for consumers in store to better facilitate the decision-making process.
L’Oréal Acquires ModiFace
L’Oréal, the largest global beauty conglomerate with €26.02 billion in sales in 20173, acquired ModiFace in March 2018 because of machine learning technology’s growing role in the beauty industry. “We see technology bringing so much innovation in the consumer decision journey for beauty,” says chief digital officer Lubomira Rochet4. “It’s the first time L’Oreal acquired a tech company, a non-beauty brand. So it’s really very significant from a strategic standpoint. And what we have in mind with ModiFace is to create the heart of our digital research and development.5”
Beauty has historically been one of the later adopters to technology trends, but with the rise of hugely successful digital-native brands like Kylie Cosmetics, the industry is actively seeking ways technology can give their brands a ‘boost’ in the consumer ecosystem.
In the short-term, ModiFace is currently being integrated into L’Oréal’s new digital arm, the Digital Services Factory, a network that creates digital solutions for the portfolio of brands under management. Before the integration, the revenue model hinged on the number of platforms (e.g., mobile, online, in-store) a brand or retailer wished to purchase, roughly $200,000 to $500,000 per year for use of the technology. Integrating new categories, from hair dye to new tools like live streaming, applied to all brand partners, with no ‘pay-to-play’ discrimination between brands6. The business model fits an odd niche in the L’Oréal portfolio – many of its customers are either direct competitors to the L’Oréal family of brands or retail partners, like Sephora. Historically, ModiFace has powered technological offerings for many of L’Oréal’s direct competitors including augmented reality mirrors for MAC stores, virtual try-on services for Bobbi Brown Cosmetics’ mobile and web sites – two brands owned by the Estée Lauder Companies7.
In the long term, diversified data is critical to the evolution and continuous learning principles of machine learning – the ability to serve brands and retailers beyond the L’Oréal portfolio is critical to maintaining ModiFace’s leading position in the beauty technology marketplace. Competitors have faced similar organizational challenges and succeeded – LVMH has managed to keep its brand incubator Kendo Brands (creator of brands like Rihanna’s Fenty Beauty, Kat Von D and Marc Jacobs Beauty) a separate business line from its retailer, Sephora, by maintaining rigorous guidelines on innovation to prevent imitation.
Future of the Consumer Journey
The ModiFace deal has triggered key competitors to think more critically about how machine learning and augmented realty technologies will complement brand portfolios, like Coty’s acquisition of YouCam Makeup in 20175. Partnerships with technology giants like Samsung give L’Oréal significant access to consumer behavior data. ModiFace’s embedded software in Samsung’s Bixby vision camera create new opportunities for L’Oréal to integrate its marketing prowess with this new source of data.
The future success of the L’Oréal-ModiFace partnership hinges on a few key questions: how exactly will the consumer beauty experience evolve online and offline with AI in the industry spotlight? What changes will beauty conglomerates make to the shopper experience given this wealth of new consumer behavior data?
How consumers and industry players react to the increasing presence of AI and personalization technology will certainly shape the future of the beauty industry.
(Word Count: 761)
[1] ModiFace, “Products,” www.modiface.com, accessed November 2018.
[2] “App Scans Your Social Media Photos for Optimal Skin Tone,” Global Cosmetic Industry, Vol. 184, No. 2 (March 2016): 6.
[3] L’Oréal, 2017 Annual Report (Paris: L’Oréal, 2017), p.1.
[4] “AI technology to take beauty to new reality,” Chain Drug Review, Vol. 40, Issue 9 (March 4, 2018): 28-28.
[5] Jack Neff, “L’Oreal buys ModiFace to get edge in beauty augmented reality and AI,” Ad Age, March 16, 2018, https://adage.com/article/cmo-strategy/l-oreal-acquires-modiface-edge-beauty-augmented-reality-ai/312764/, accessed November 2018.
[6] Hilary Milnes, “Modiface is becoming the go-to provider of augmented reality to beauty brands,” Glossy, October 6, 2017, https://www.glossy.co/new-face-of-beauty/modiface-is-becoming-the-go-to-provider-of-augmented-reality-to-beauty-brands, accessed November 2018.
[7] Allison Collins and Adriana Lee, “Beauty Brands’ ModiFace Dilemma: Other AI and AR providers are seeing an influx of inquiries from beauty companies following the deal”, WWD: Women’s Wear Daily, April 12, 2018, https://wwd.com/beauty-industry-news/beauty-features/beauty-brands-modiface-dilemma-1202644840, accessed November 2018.
Yes!!! This is exactly what the beauty industry needs! As someone who always struggles with finding the right makeup or clothing, I would love to have a computer scan my face and body and suggest ideal products to purchase. My concern is that is it one thing to simulate color shades and completion, but it’s impossible to recreate the feel of certain types of makeup. I’ve tried lip stains, for example, that my friends absolutely loved, but when it came down to my own personal use, the color looked great, but the stain itself was drying and irritated my skin. I suppose it would be as easy as a free return if the product did not satisfy your needs, but I still find it important to try makeup on in person before purchasing.
Thanks for sharing – a very interesting thought piece. ModiFace’s technology is potentially quite revolutionary for the beauty industry. Increasingly in the land of consumer retail, successful companies need to find a way to accommodate the omnichannel experience, and respond to increasing calls by the consumer to provide tailored, customized, on-demand, and convenient solutions. Given the thematic alignment of this product with these trends, I am a big proponent. As we’ve touched on in our marketing class, e-commerce purchasing for beauty is typically reserved for repeat purchases following an in-store experience. However, if we can unlock the brand discovery and trialability pieces of the puzzle previously addressable for online, the effects would be profound for L’Oreal.
We see these sorts of predictive analytics platforms popping up all around the consumer world, such as StitchFix (https://www.stitchfix.com/) in fashion. To respond to your question, I see the beauty industry moving in the same direction. This technology could be a valuable source of tremendous brand loyalty (if the predictive decision-making is actually sound). Additionally, it would dramatically reduce brand marketing expenses, because the data-driven approach would lead to much more targeted marketing efforts. It will also massively disrupt beauty retailers by potentially disintermediating them from the consumer purchasing journey. The flipside though, is it might allow brands to become increasingly virtual, reduce the customer cost to trialing new brands, and further enable brand proliferation, thereby undermining brand equity (if the technology becomes more widespread). Hence, it can also be a threat to the brands themselves at the same time.
It will be very interesting to see how this all plays out. Clearly though, successful beauty brands will need to have a strong captive digital / technology platform if they want to remain competitive.
This product is amazing – I think Sephora may have used similar technology to execute their own AR makeup application tool? I like how here, the product 1) detects skin tone 2) selects SKUs and 3) can virtually apply them. How much easier could the purchase decision get? I do wonder though how l’Oreal will make use of this tool since going direct to beauty brand .com and mobile apps is uncommon versus using retailer apps with broader product selection and better delivery/returns service propositions. It would almost have been more natural for an Amazon or Sephora to bring this company in-house. And I question L’Oreal’s ability to manage data infrastructure and talent vs. a technology firm… But power to l’Oreal and I hope (as a consumer) they make this a popular & well-integrated part of their experience!
@cah, I agree with your assessment – I think a technology company would have more streamlined processes and better access to talent and innovation. The technology you’re referring to at Sephora is actually ModiFace’s product! Sephora is one of ModiFace’s biggest customers. Similarly to L’Oreal, I don’t think Sephora has the internal talent to develop these types of algorithms and predictive analytics.
I really enjoyed learning about how the beauty industry is integrating AI technology to augment the consumer shopping experience. As you state, the beauty industry has been slow to adopt new technologies, perhaps because of trialability being a critical factor in the purchasing decision (especially for color cosmetics). It’s interesting to see how AI tries to address this issue directly.
Personally, I feel a huge opportunity for such a technology: many times I’ve shopped online on Sephora, but delayed (and soon forgot about) a potential purchase because I couldn’t decide on the right color of foundation/lipstick/eye shadow, even after spending a lot of time researching the brand and product. Yet when brands started offering similar technology (generally through mobile apps), I thought of it as a gimmick, and ended up going to the store when I did make any purchases. I’m curious to learn how companies like Modiface can improve trial of its own product, and build credibility as a first mover in the industry.
Thank you for this very interesting article! One aspect that has particularly caught my attention is the fit of ModiFace’s business model within the broader L’Óreal conglomerate. Going forward, I would be curious to see whether the acquisition rationale revolved exclusively around the technology, in order to “supplement” L’Óreal’s brands, or whether they will be pursuing a path of service offering across the entire beauty industry, in a way to create multiple revenue streams.
I am inclined to think that L’Óreal will be better off leveraging this recently acquired technology in a way to booster the online experience offered by its existing brands, unlocking the e-commerce revenue portion associated to more conservative buyers (those who were not willing to buy the wrong product due to inability to try it on first). In this situation, however, I am afraid that this can hinder open innovation in the sector and perpetuate the beauty industry’s profile as a technology laggard.