Window into the soul?: Machine learning for song recommendations at Spotify

“Music is like a window do your soul”… “it tells people a lot about who you are and what you care about, whether you like it or not.”
-Christine Hung, Head of Data Solutions at Spotify [4][6]

If music is a window into the soul, one could argue that Spotify knows its customers better than any of their closest friends. The Swedish music-streaming company has collected extensive data on its 170 million active users [9] to develop a deep understanding of listening preferences. By combining this knowledge with machine learning, Spotify is able to create new song preferences by generating individualized song recommendations for its users to enjoy [1].

Compelling song recommendations are critical to the success of Spotify’s user engagement, a key metric for both of the company’s two revenue sources [4]:

• Paid subscriptions (44% of users, 91% of revenue) [9]
• Ads from users who listen for free (56% of users, 8% of revenue) [9]

Paid subscribers have a variety of music streaming options available, with Apple Music being the company’s main competitor [10]. Since most streaming services offer nearly identical content catalogues, platforms must differentiate on user experience. By serving fresh, highly-relevant song recommendations, Spotify is able to retain loyalty to its platform. Moreover, use of machine learning creates a virtuous cycle: as Spotify’s user base grows, the company has more data to refine smarter recommendations, which promote further user growth through retention and referrals [1][5][7].

Further, the profitability of ad-supported listeners is directly tied to the amount of time these users spend streaming: more listening means more opportunities to present advertisements. Providing users with consistently fresh and appealing content indirectly encourages them to spend more time listening to ads.

Spotify tackles machine learning using a number of data sources and approaches:

• Collaborative filtering: Generates recommendations based on the behavior of users with similar listening histories. If, for example, two users have generally similar song histories, and User 1 listens to a song that the User 2 hasn’t heard of, it’s likely that User 2 will like that song (see Image 1 for example). However, a critical downside of collaborative filtering is known as the slow-start problem: new or unpopular songs that don’t yet have historical data are excluded from analyses. [8]

• Audio signal models: Spotify has developed sophisticated models to analyze audio sources themselves. While this offers a direct way to evaluate the characteristics of a song, the techniques to do so are complex and not necessarily reliable. The execution of these models is complicated by the semantic gap, or the ambiguous connection between the audio itself and the traits that influence whether a user likes a song (mood of the song, lyrical themes, etc.). Spotify’s acquisition of Echo Nest, a music analytics platform founded at the MIT Media Lab, should help drive these types of analyses forward. [8]
• Natural Language Processing (NLP): Language from blogs, music review sites and social media can be scraped and analyzed using NLP to identify shared traits among songs based on the words used in association with them. [7]

• Producer Data: Additional sources of data can be gleaned from music producers directly (i.e., production year, lyrics, album images, etc.) [1].
• User feedback: Spotify is able to create feedback loops to refine it’s models using two sources of user feedback. The first is manual “Thumbs Up” or “Thumbs Down” input by users during songs. The second, perhaps more reliable, signal is observed user behavior – i.e., users skipping over songs, listening to songs repeatedly, etc [5].

To improve their product offering and competitive position further, Spotify should consider addressing several additional issues:

• Cold-start problem for users: Spotify has ways to circumvent the cold-start problem for new songs, but it’s less clear how they tackle a lack of historical data for users. By accessing social media posts, networks, and demographic data, Spotify could better customize content to capture new users, driving retention or faster conversion to paid subscriptions. However, they should be cautious in considering which data sources to access given previous controversies related to overreaching privacy policies [3].
• Content generation: Though studies have shown mixed results on machine learning’s ability to predict highly-successful songs, Spotify may consider selling data to music producers, who want more accurate insights into the likelihood of a song’s commercial viability. In a riskier move, Spotify might also benefit from launching its own record label and producing content influenced by its wealth of customer and machine learning data.

Open questions

• Netflix has leveraged machine learning insights to inform in-house content development. Would the approach Netflix has taken with movies work for Spotify and music?
• Apple is moving into music subscription services, and has a much more robust set of data on users than Spotify. How worried should Spotify be about Apple’s ability to compete in music? 

(798 words)

SOURCES
[1] Bernhardsson, E. (2014). Music Discovery at Spotify. In: MLConf. [online] Available at: https://www.slideshare.net/erikbern/music-recommendations-mlconf-2014 [Accessed 13 Nov. 2018].

[2] Dieleman, S. (2018). Recommending music on Spotify with deep learning. [online] GitHub. Available at: http://benanne.github.io/2014/08/05/spotify-cnns.html [Accessed 10 Nov. 2018].

[3] Hern, A. and Rankin, J. (2018). Spotify’s chief executive apologises after user backlash over new privacy policy. [online] the Guardian. Available at: https://www.theguardian.com/technology/2015/aug/21/spotify-faces-user-backlash-over-new-privacy-policy [Accessed 13 Nov. 2018].

[4] Hung, C. (2018). Strata Data Conference 2017 – New York, New York. [online] O’Reilly | Safari. Available at: https://www.oreilly.com/library/view/strata-data-conference/9781491976326/video314445.html?utm_source=oreilly&utm_medium=newsite&utm_campaign=20171206_data_show_christine_hung_related_resources_music_the_window_into_your_soul [Accessed 10 Nov. 2018].

[5] Johnson, C. (2014). Algorithmic Music Discovery at Spotify. [online] Available at: https://www.slideshare.net/MrChrisJohnson/algorithmic-music-recommendations-at-spotify [Accessed 13 Nov. 2018].

[6] Lorica, B. (2018). Machine learning at Spotify: You are what you stream. [online] O’Reilly Media. Available at: https://www.oreilly.com/ideas/machine-learning-at-spotify-you-are-what-you-stream [Accessed 10 Nov. 2018].

[7] Murali, V. (2016). Music Personalization at Spotify. In: RecSys.

[8] Oord, A., Dieleman, S. and Schrauwen, B. (2018). Deep content-based music recommendation. [online] Papers.nips.cc. Available at: https://papers.nips.cc/paper/5004-deep-content-based-music-recommendation [Accessed 10 Nov. 2018].

[9] Spotify (2018). Annual Report for the First Quarter of 2018. [online] Spotify. Available at: https://investors.spotify.com/financials/press-release-details/2018/Spotify-Technology-SA-Announces-Financial-Results-for-First-Quarter-2018/default.aspx [Accessed 10 Nov. 2018].

[10] Steele, A. (2018). Apple Music on Track to Overtake Spotify in U.S. Subscribers. [online] WSJ. Available at: https://www.wsj.com/articles/apple-music-on-track-to-overtake-spotify-in-u-s-subscribers-1517745720 [Accessed 10 Nov. 2018].