Spotify: Machine Learning as Recommendation Engine and Musical Composer?
Spotify has invested heavily in machine learning capabilities to power its recommendation and personalization engines in a competitive streaming environment. Is this a competitive advantage the company can maintain?
The Music Streaming Industry & Personalization
The music streaming industry has grown rapidly over the past few years and in 2017, surpassed 100M paying subscribers worldwide. As it has grown, the industry has also become highly competitive. The most popular paid streaming services are Spotify, Apple Music, Deezer, Rhapsody, Google Play and Tidal.[i] Apple Music has approached Spotify’s lead and in May 2018, reported that it had 50M subscribers (paying and on trials)[ii].
With more musical content accessible online than ever before, Spotify must compete to keep users (and in turn brands and advertisers) engaged with and paying for its platform. Core to continued user engagement is a strong recommendation and personalization engine powered by data and machine learning – satisfied listeners will find the right song for each moment throughout the day. Spotify has invested heavily in both capabilities and to win in the space they will need to continue to innovate.
Spotify’s Investment in Machine Learning
Spotify recognized early on that to keep listeners engaged at scale, they needed to use machine learning to personalize recommendations for listeners. In 2014, Spotify acquired EchoNest, a “music intelligence company”[iii] that many of its competitors used in their recommendation engines. Specifically, EchoNest uses machine listening software to analyze songs (including features such as tempo, pitch, vocals, and energy) and crawls the web to index descriptions and reviews of music. The software then combines this information to make recommendations based on both how the music sounds and how it is being reviewed externally.[iv]
Since, Spotify has continued to focus on machine-learning powered personalization. They have made additional acquisitions to increase their technical capabilities such as Mighty TV, Niland, Sonalytic and SoundTrap.[v] As of December 2017, the company logged 150 billion user events daily — including actions such as playing, skipping, following or upvoting a song — and personalized content with algorithms that took into account over 40 different parameters such as user demographics, past listening behavior, time of day and location.[vi] Over 31% of all listening on the platform is now through Spotify’s playlists, including machine-generated playlists such as “Discover Weekly,” “Daily Mix” and “Release Radar.”[vii] Through continued investment in data collection and machine learning, Spotify has differentiated its platform through a seamless music discovery process.
Longer term, Spotify has begun working to use all of the data it has gathered on listener habits not only to recommend hit music, but to create it as well. In coverage of their 2018 Machine Learning Day, the company advertised exploring “how to generate coherent music.”[viii] AI Scientist Francois Pachet leads this charge and runs Spotify’s “Creator Technology Research Lab” in Paris.[ix] This entirely new form of content development would enabled Spotify not only to better engage listeners with more hits, but also to decrease the significant impact of artist royalties on the bottom line.
Competition, Backlash and Recommendations
Even in their cutting-edge investment to use machine learning capabilities to compose songs, Spotify faces competition. Through Project Magenta, Google has also focused on machine learning capabilities to generate music along with other forms of art.[x] As such, Spotify must continue to invest heavily in this research to stay ahead in product development.
However, as it invests in these capabilities, Spotify must also be careful not to alienate either side of its platform – listeners or artists. Listeners have not always responded positively to the data collection required for Spotify’s algorithms. In 2015, the company released updated privacy terms that indicated access to personal data on users’ phones and the CEO, Daniel Elk, had to apologize publicly.[xi] Spotify also risks alienating artists on the platform as it pursues an alternate and competitive path to content generation.
To maintain its relationship with artists and stay true to its mission to “unlock the potential of human creativity”[xii] Spotify should focus its investment on algorithms that co-create music with artists and inform rather than supplant the creative process. Pachet has hinted at a focus in this direction and said that he is building “companions and collaborators.”[xiii] To maintain trust with listeners, Spotify should transparently share data practices and when possible, include listeners in the data collection processes through tooling such as “Line-In” that enables users to suggest metadata for a song while listening.[xiv]
Key Questions Moving Forward
As Spotify scales, they will need to grapple with the long-term ramifications of combining art and science and to consider if at any point their investment in machine learning has gone too far. Does human curation still play any role in music selection and do we need human musicians in the creation process? Further, if and when does collection of listener data cross a line – can tracking a listeners’ every move step past product innovation to become a form of privacy invasion?
Word Count: 788
[i] Anna Nicolaou, “How Streaming Saved The Music Industry,” Financial Times, January 16, 2017, https://www.ft.com/content/cd99b95e-d8ba-11e6-944b-e7eb37a6aa8e, accessed November 2018.
[ii] Anna Nicolaou, “Apple Slices Into Spotify’s Lead In US Music Market,” Financial Times, July 9, 2018, https://www.ft.com/content/ef2be1bc-809f-11e8-bc55-50daf11b720d, accessed November 2018.
[iii] Darrell Etherington, “Spotify Acquires Echo Nest, Gaining Control Of The Music DNA Company That Powers Its Rivals,” TechCrunch, March 6, 2014, https://techcrunch.com/2014/03/06/spotify-acquires-the-echo-nest/, accessed November 2018.
[iv] Jim Lucchese, Competing with Data and Analytics, interviewed by Sam Ransbotham, MIT Sloan Management Review, February 24, 2015, https://sloanreview.mit.edu/article/analytics-in-e-major/, accessed November 2018.
[v] Spotify Technology S.A., Form F-1 Registration Statement (filed February 28, 2018), p.104, from SEC EDGAR, https://www.sec.gov/Archives/edgar/data/1639920/000119312518063434/d494294df1.htm, accessed November 2018.
[vi]Spotify Technology S.A., Form F-1 Registration Statement (filed February 28, 2018), p.15, from SEC EDGAR, https://www.sec.gov/Archives/edgar/data/1639920/000119312518063434/d494294df1.htm, accessed November 2018.
[vii] Ibid., 98.
[viii] Nicola Bortignon, “Spotify ML Day – Coverage,” Spotify Labs, August 22nd, 2018, https://labs.spotify.com/2018/08/22/spotify-ml-day-coverage/, accessed November 2018.
[ix] Kevin Maney, “Spotify, IBM and Google Using AI to Make Human Musicians Extinct?,” Newsweek, February 2, 2018, https://www.newsweek.com/artificial-intelligence-changing-music-799794, accessed November 2018.
[x] Cade Metz, “How A.I. Is Creating Building Blocks to Reshape Music and Art,” New York Times, August 14, 2017, https://www.nytimes.com/2017/08/14/arts/design/google-how-ai-creates-new-music-and-new-artists-project-magenta.html, accessed November 2018.
[xi] Hayley Tsukayama, “Spotify Apologizes for its News Privacy Policy,” The Washington Post, August 21, 2015, https://www.washingtonpost.com/news/the-switch/wp/2015/08/21/spotify-apologizes-for-its-newest-privacy-policy/, accessed November 2018.
[xii] Spotify Technology S.A., Form F-1 Registration Statement (filed February 28, 2018), p.1, from SEC EDGAR, https://www.sec.gov/Archives/edgar/data/1639920/000119312518063434/d494294df1.htm, accessed November 2018.
[xiii] Kevin Maney, “Spotify, IBM and Google Using AI to Make Human Musicians Extinct?,” Newsweek, February 2, 2018, https://www.newsweek.com/artificial-intelligence-changing-music-799794, accessed November 2018.
[xiv] Janko Roettgers, “Spotify Enlists Its Users to Add Music Metadata,” Variety, March 12, 2018, https://variety.com/2018/digital/news/spotify-line-in-music-metadata-1202723757/, accessed November 2018.
I’m really impressed with Spotify’s ability to curate resonant playlists via machine learning. I feel like playlists are a space where customers are willing to trust data and algorithms, if it ultimately leads them to listen to songs that they appreciate. However, I think Spotify is wandering into dangerous territory by beginning to use machine learning to generate original music. I think this will alienate artists and listeners alike. In my opinion, at least, there is so much to music and artistry that data cannot replicate. Content creation is where humans remain essential. I’m curious to see what Spotify finds in this area, but I think it could really turn off many of their key stakeholders.
Great post! I agree with TOMGirl29. I think this is another situation where people may not know what they are looking for next. Using machine learning to identify components of hit songs and then working with artists to apply those elements could cause different songs to start sounding similar. It also may make producers less willing to bet on a new concept. Finally, I think artists tend to be fairly creative people who want to hold onto their individuality, I would be very curious to hear their responses to Spotify’s suggestions.
This article raises the idea of users suggesting metadata. As a user of Spotify, I wonder if there are any additional data points I could supply which would be helpful? Spotify currently offers a “I don’t like this song” or “I don’t like this artist” button for users. Would asking for additional information be too much of a burden on consumers?
Very good points – For privacy, I believe Spotify became too powerful on knowing a lot about what we listen and music taste is some sort of a reflection of personality. I think the company was particularly hit by EU’s GDPR legislation and they tried to navigate by acquiring a company called Echo Nest and started keeping the user data encrypted with their technology. If the user would like his/her data to be removed, the identity associated with the data (i.e. data key) is removed. The consumption data is still in Spotify’s database, but the user associated with that data is no longer known. It looks like they are complying with the legislation, but I think the question of privacy is still unanswered.
Awesome work! I’ve often thought about the possibility of robots beating humans at creating music. Even though it’s tempting to claim that humans will always understand other humans better, the evolution of music recommendation engines suggests otherwise.
1. Machine learning has certainly curated some spot-on playlists, but as with most trends, people will probably tire of the perfect playlist and want more variety. On the other hand, DJ’s may want to pick up an instrument or two, given that algorithms are beating them at their own game.
2. Most companies seem to avoid the issue of data privacy through anonymity, but one might find that musical tastes are highly correlated with geography, culture, and other unique identifiers. Spotify will probably find itself defending its data practices periodically as they scale.