View profile

How Netflix uses AI and Data to conquer the world - Issue #3

How Netflix uses AI and Data to conquer the world - Issue #3
By Mario Gavira • Issue #3 • View online
Hi from sunny Barcelona! 
What if I tell you that Netflix is NOT in the business of media entertainment?  Keep reading.
Netflix not only has the largest worldwide subscriber base of any business but managed to keep growing it by +25% last year. Its market capitalization competes head to head with Disney, the most-valued entertainment company in the world.

House of Cards
House of Cards
Netflix success story can not be explained without understanding their granular knowledge of their subscriber base and their AI driven focus on personalization.
Netflix not only looks at millions of ratings, searches and “plays” a day, but the entire viewing history of billions of hours of content streamed per month.
It took them 6 years to collect enough viewer data to engineer a show that became an worldwide success: House of Cards.
Since then, Netflix has increasingly used this formula for content creation achieving success rates of 80% compared to 30%-40% success rates of traditional TV shows.
The New York Times went so far as to claim.
 “Netflix is commissioning original content because it knows what people want before they do”
Vulture published this month an in-depth piece on Netflix that looks under the hood of its operations giving some enlightening on the secret sauce of algorithms, big data, and gut instinct that lies behind the success of Netflix.
Let me pull back the curtain for you on some of Netflix key AI and Data insights 
House of Cards
House of Cards
Make the experience personal by testing, testing and testing
Back in 2013 Netflix claimed “There are 33 million different versions of Netflix”. At that time, the company had 33 million subscribers.
Or as Ted Sarandos, Netflix’s chief content officer puts it:
There’s no such thing as a ‘Netflix show’. Our brand is personalization.
Netflix algorithmically adapts the entire user experience to each individual subscriber, including the rows selected for the homepage, the titles selected for those rows, the visuals for each movie, the recommendations of other movies etc.
This ongoing personalization process is driven by what Netflix defines “consumer science”.
We test new ideas with real customers, at scale, and we measure for statistically significant differences in how they engage with our product.
Stranger Things
Stranger Things
The testing goes from finding ways to increase the relevance of the search results, redesigning the UX for a new device to adding a new feature such as showing members what their Facebook friends are watching from Netflix.
Netflix typical testing process will start with the following hypothesis:
Algorithm/feature/design “X” will increase member engagement with our service, and ultimately member retention.
Consumer science allows them to test big bets, try radical ideas, fine-tune each touchpoint of the user experience and build a track record based on real customer value. 
Example of different sign up processes on 2 different devices and countries
Example of different sign up processes on 2 different devices and countries
Ignore traditional segmentation criterias.
Traditional TV networks use standard demographic ratings such as age, race or location for their market segmentation. Netflix instead tracked viewing habits of its subscriber base from early beginning and created almost 2000 clusters, so called “taste communities”.
These segments are not seen as static silos:
“Most people are usually members of a few different communities. We’re complex beings, we’re in different moods at different times.”
House of Cards
House of Cards
The power of recommendations
Netflix’s Senior Data Scientist, Mohammad Sabah stated in 2014:
“75 percent of users select movies based on the company’s recommendations, and Netflix wants to make that number even higher.”
These recommendations are powered by algorithms that are based on the assumption that similar viewing patterns represent similar user tastes.
The taste communities play an instrumental role in these recommendation algorithms:
“We didn’t come out of the gate and say, ‘We think Black Mirror is for this audience or not for that audience,’. But after we launched the show, we’re able to start to see patterns.” The chart shows how folks who liked Black Mirror were also fans of Lost and Groundhog Day. “On the surface, if you thought about Groundhog Daywith Black Mirror, you might not find an obvious similarity.
But the recommendation algorithms go beyond the “taste” criteria. Netflix also includes contextual criterias to find the perfect recommendation for each user in each moment.
We have data that suggests there is different viewing behavior depending on the day of the week, the time of day, the device, and sometimes even the location. 
Most internet companies use batch processing for personalization use cases such as recommendations, but Netflix realized that this was not quick enough for time sensitive scenarios such as new title launch campaigns or strong trending popularity cases.  They moved to a near real-time (NRT) recommendation process to accelereate the learning process and roll out of test results.
Machine Learning Infrastructure for near real time recommendations
Machine Learning Infrastructure for near real time recommendations
A picture is worth more than a thousand words
Money Heist
Money Heist
Netflix sets themselves apart from traditional media companies not only by what they recommend but how they recommend it to their members. A key feature are the image they use to promote each movie or TV show - the so called artworks.
Netflix aims to provide the artwork for each show that highlights the specific visual clue that is relevant for each individual member. 
For each new title different images are randomly assigned to different subscribers, using the taste communities as an initial guideline.
Example of contextual image selection based on the type of profile. The contextual bandit selects the image of Robin Williams, a famous comedian, for comedy-inclined profiles while selecting an image of a kissing couple for profiles more inclined towards romance
Example of contextual image selection based on the type of profile. The contextual bandit selects the image of Robin Williams, a famous comedian, for comedy-inclined profiles while selecting an image of a kissing couple for profiles more inclined towards romance
This translates into hundred of millions of personalized images continously being tested among its subscriber base.
Examples of artworks
Examples of artworks
For the creation of the artwork, machine learning also plays a critical role thanks to a computer vision algorithm that scans the shows and picks the best images that will be tested among the taste communities.
Example of an image selection algorithm
Example of an image selection algorithm
Go beyond standard industry metrics
Netflix does not limit the succes or failure of a show to the size of the audience. Shows with a smaller audience but low production costs can remain profitable operations that add to the breath and depth of its library.
John Ciancutti, former VP of Product Engineering summarised the key criteria for content selection as follows:
“Netflix seeks the most efficient content. Efficient here meaning content that will achieve the maximum happiness per dollar spent. There are various complicated metrics used, but what they are intended to measure is happiness among Netflix members. 
To build its own metrics focused on user experience. each time a member starts to watch a show, a “view” is created in their data system and a large number of events around each view is collected.
Architecture Diagram for each view
Architecture Diagram for each view
2 metrics are critical in defining the success of a show:
  • the drop rate of subscribers midway through an episode never coming back.
  • the “28-day viewership” of a serie, or how many people completed a full season of a show within the first four weeks of its launch.
Altered Carbon
Altered Carbon
Netflix plans to spend $8 billion in own content producing aprox. 1000 original titles this year (more than any TV network in history).
The 2 metrics to decide the investment strategy in new content are:
  • growth of subscriber base
  • hours of engagement per subscriber
“When you start seeing these metrics plateau, then you say there’s a point of diminishing returns on the continued expansion of the library"
Another metric where Netflix challenges traditional industry standards is in defining the potential size of its audience. Netflix user screen time does not only compete with other Media companies, but with any company absorbing users leisure time and money.
Everyone with a phone has a screen and access to the internet. That is our addressable market. The world’s taste, and the world’s time, is what we’re after.
Stranger Things
Stranger Things
Does Netflix entirely rely on machine decision across the organisation?
Netflix´s wealth of data and sophisticated algorithms may lead to think that decisions such as investing or not in a new show is purely driven by machines.
It is not.
Viewing habits combined with smart algorithms are used for predicting consumer behaviour.
“We have projection models that help us understand, for a given idea or area, how large we think an audience size might be, given certain attributes about it,”
But their various projection models and cost analyses, don’t dictate their decisions.
“You have to be very cautious not to get caught in the math, because you’ll end up making the same thing over and over again. And the data just tells you what happened in the past. It doesn’t tell you anything that will happen in the future.”
 Even in a data obsessed company like Netflix, humans are still in command for key investment decisions and data and smart algorithms only support the final decision making process.
“It’s 70 percent gut and 30 percent data,” 
House of Cards
House of Cards
Conclusion
We are witnessing the transformation of the media industry with the rise of technology giants such as Netflix rewriting the rules of the game.
This dynamic has sparked a merger wave these last weeks with  AT&T-Time Warner and Disney-Fox deals.
It will be fascinating to see how the media landscape will reshape in the coming years, but it seems pretty clear that Netflix’s combination of data, algorithmical personalization and massive content investment are likely to keeps us glued to the screen watching their shows.
Breaking Bad
Breaking Bad
So is Netflix a media company?  It certainly competes in this industry, but you might argue that Netflix really is in the business of personalization and recommendation.
More AI articles hot from the Press
Cool or Creepy? 
AI is rapidly expainding in the health sector helping to detect and prevent diseases.  Cool!
The AI That Spots a Stopped Heart - Bloomberg
Google claims its algorithmic models are more accurate in predicting death risk among hospital patients. Are we ready to let machines tell us our final day? Creepy!
Google says its AI is better at predicting death than hospitals
AI and jobs destruction
Steven Pinker famously predicted that the jobs in danger of being replaced by artificial intelligence are “the stock analysts and petrochemical engineers and parole board members,” while blue-collar workers like plumbers and gardeners are likely to have a brighter future. 
The fact that Amazon is moving its massive inventory management process from human to algorithmic driven decisions seem to bring Mr Pinkers vision a little bit closer to reality. 
Amazon’s Clever Machines Are Moving From the Warehouse to Headquarters - Bloomberg
Feeling like having a little debate? 
IBM latest Project Debater might be the right partner for you. The technology uses AI to formulate a debate on the spot against a human, including an initial argument, a rebuttal, and closing remarks.
Leading IBM scientist Mr Gil admitted that they are still far away from a “true autonomous intelligence,” but said:
 “If AI is going to realize its full potential it needs to live in the human world… and we are not black and white…I think this one of the first attempts to live in that messy world"
IBM’s Newest AI Can Probably Argue Better Than You | WIRED
The AI race goes on
The rush to grab AI start ups among top tech giants seems to have no end. This chart from CB insights gives a good overview on the frenzy M&A activity in this field.
CB Insights
CB Insights
Microsoft has made the latest move by acquiring Bonsai, a startup developing machine learning and reinforcement learning systems for industrial companies. Their so called approach of “automated” machine learning,  or AI solutions accesible for all, fits well with Microsoft’s B2B strategy.  
Microsoft to acquire machine learning startup Bonsai, fueling AI efforts – GeekWire
Breaking Bad
Breaking Bad
Thanks for reading, AI enthusiasts!
Did you enjoy this issue?
Mario Gavira

Your regular dose of AI news and trends decrypted for Non Techies, including practical insights and case studies on how AI can have an impact in your daily business.
Carefully curated by non-artificial intelligence!

If you don't want these updates anymore, please unsubscribe here.
If you were forwarded this newsletter and you like it, you can subscribe here.
Powered by Revue