Automated music suggestions are nothing new, so to create Discover Weekly we had to find a new ingredient. That main ingredient, it turned out, was other people. We begin by looking at the 2 billion or so playlists created by our users—each one a reflection of some music fan’s tastes and sensibilities. Those human selections and groupings of songs form the core of Discover Weekly’s recommendations. We consider everything from our own professionally created playlists through to regular user’s playlists - and then we fill in the blanks between your listening habits and those with similar tastes. In the simplest terms, if we notice that two of your favorite songs tend to appear on playlists along with a third song you haven’t heard before, we will suggest the new song to you in your Discover Weekly.

But the recipe for Discover Weekly playlists is more complicated than that. Spotify also creates a profile of each user’s individualized taste in music, grouped into clusters of artists and micro-genres—not just “rock” and “rap” but fine-grained distinctions like “synthpop” and “southern soul.” These are derived using technology which learns about emerging genres by having machines read music sites and analyze how various artists are

The idea of creating 80 million thoughtfully personalized playlists on a weekly basis without a creative data solution is unimaginable. Data has not enhanced Discover Weekly, rather it inspired the feature and made it possible.

“Every playlist felt like it was crafted by my musical soulmate” wrote the Wall Street Journal - and this uncanny feeling of a data-driven playlist “getting you” has driven exceptional business results. Over 30 million consumers have used Discover Weekly, and of those that use it, over 75% use it again. In only a year, it has driven over 2.5 billion streams. And the results have not only been phenomenal for the users and Spotify, but also artists. For thousands of emerging or niche musicians, Discover Weekly is driving over 90% of their streaming activity.

Every week, Spotify uses an algorithm to create over 80 million unique Discover Weekly playlists for its users. This algorithm creatively combines user data with metadata about the world’s music to enable discovery in a completely original - and highly successful - way.

The connection between data from 2 billion playlists and your personal taste profile is made by Spotify’s algorithms. The approach include collaborative filtering, most commonly seen in Amazon’s “customers who bought this item also bought…” feature, and natural language processing, which is how we understand music blogs and the titles of playlists. We also use the open-source software Kafka to manage the data in real-time.

Finally, we adopt deep learning—a technique for recognizing patterns in enormous amounts of data, with powerful computers that are “trained” by humans—to improve our Discover Weekly picks. This allows us to recognize musical “outliers”, like a children’s song a parent might be playing for their toddler, and not include it as a data point for creating that parent’s Discover Weekly.

In 2015, Spotify found itself in a crowded category, with well-funded competition including Apple, Google, and Tidal. Differentiating and strengthening its product with new features was a priority - not only to attract new users but also to retain the loyalty of over 80 million existing users.

Spotify focused on music discovery as the key priority - after all, every platform has the same catalogue of 30 million songs. We reasoned that providing music picks that felt both fresh and familiar on a regular basis would lead to a better user experience.

The key opportunity was taking a behaviour like music discovery, which is typically “lean-in” and time intensive, and making it “lean-back” and accessible. The suggestions Spotify made had to feel highly personal, accurate and also unexpected. Any solution would require an entirely new approach.