When used effectively, data can transform your business in magical ways and take it to new heights. By: Claire D. Costa
“If the Starbucks secret is a smile when you get your latte… ours is that the Web site adapts to the individual’s taste.”
– Reed Hastings(CEO of Netflix)
Over the past couple of years, Netflix has become the de-facto destination for viewers looking to binge on movies and TV shows.
Last year Netflix announced that it signed on 135 million Paid customers worldwide. In 2019 alone, Netflix made a tad more than$20 billion, by offering its viewers some of the top-tier content at their convenience.
With its humble beginning as a DVD rental platform, one of the principal factors that play a significant role in the success of this Over-The-Top media provider giant is its underlying use of Big Data.
The technology behind the success of Netflix is Big Data, which is capable of dealing with remarkably large sets of data which makes sense as Netflix doesn’t just deal with the content but a ton of related meta-data as well.
Importance of Data Science
“Where there is data smoke, there is business fire.”
— Thomas Redman
Data Science plays a role so important in a majority of online services and helps not only bring more customers but keeps the existing ones happy.
The reason why is because, with Data Science, you get a more realistic picture of your consumers’ taste in the form of graphs and charts that take not just one metric but several as input.
This crucial piece of information helps you in molding your products and services in a way that looks one-of-a-kind to your customers, attracting them to your platform.
Data really powers everything that we do.
— Jeff Weiner
With a company like Netflix that is brimming with data, it’s always a wise decision to put that pile of data to good use. By incorporating concepts like data analysis, machine learning, statistics and deep learning, Data Science can help not just Netflix but any business to grow exponentially regardless of sector.
Netflix Business Model
“I founded Netflix. I’ve built it steadily over 12 years now, first with DVD becoming profitable in 2002, a head-to-head ferocious battle with Blockbuster and evolving the company toward streaming.”
– Reed Hastings
The business model of Netflix seems very simple on the outside. You buy a subscription and Netflix gives you tons of high-quality ad-free content to binge.
But it wasn’t always like this. Back when Netflix started in 1998, they rented out and even sold DVD copies of contentover the mail.
Later on, they included Blu-Ray discs along-side DVDs, but it was the year 2010 when they jumped into the digital streaming scene, starting with Canada.
Since then, Netflix has been at the forefront of curating, streaming and even producing media for a host of devices. Netflix, like any other company, has consistently focussed on optimizing its efficiency by conducting competitions and Hack-Days over the years looking for that gain in performance.
Everyone with a phone has a screen and access to the internet. That is our addressable market. The world’s taste, and the world’s time, is what we’re after.
– Reed Hastings
Some Interesting Facts about Netflix(source) —
- Despite more competition, Netflix still has the largest subscriber count in 2020
- 60 million US adults have a Netflix subscription
- The company is older than most users realize
- 41% of Netflix users are watching without paying thanks to password and account sharing
- Netflix was one of the first streaming services available as an app on different devices
The beauty of Netflix is on the 28th of October they push a button and the film will be in 190 countries at the same moment in 17 languages.
How Netflix Uses Big Data
Considering how long Netflix has been in the streaming business, it has stacked up heaps of data about its viewers, such as their age, gender, location, their taste in media, to name a few.
By gathering information across every customer interaction, Netflix can dive right into the minds of its viewers and get an idea of what they might like to watch next even before they finish a show or movie.
We have data that suggests there is different viewing behavior depending on the day of the week, the time of day, the device, and sometimes even the location.
– Reed Hastings
Netflix has a massive user base of more than 140 million subscribers. Here are some metrics that Netflix tracks to give an individual taste to everyone —
- What day you watch content
- What time you watch content
- The device on which the content was watched
- How the nature of the content
- Searches on the platform
- Portions of content that got re-watched
- Whether content was paused, rewind, or fast forward
- User location data
- When you leave content
- The ratings given by the users
- Browsing and scrolling behavior
Over time, Netflix has deployed several algorithms and mechanisms that make use of this data and generate critical insights that help steer the company in the right direction. Some of these tools and features are:
● Near Real-Time Recommendation Engine
With a sea of users, each user generates hundreds of ratings per day based on what they watch, search and add to their watch-list, this data ultimately becomes a part of Big Data. Netflix stores all of this information and using key machine learning algorithms, it builds a pattern indicating the viewer’s taste. This pattern may never match with another viewer because of how everyone’s taste is unique.
Based on the ratings, Netflix categorizes its media and suggests the viewer what the recommendation system thinks they might like to watch next.
Netflix will know everything. Netflix will know when a person stops watching it. They have all of their algorithms and will know that this person watched five minutes of a show and then stopped. They can tell by the behavior and the time of day that they are going to come back to it, based on their history.
– Mitchell Hurwitz
● Artwork & Imagery Selection
Ever wondered why Netflix shows multiple artworks for a single TV show or movie?
The tool behind this is called AVA, which is essentially an algorithm that selects what artworks and images to show to whom. Short for Aesthetics Visual Analysis, AVA sifts through every video available and identifies the frames that are best suitable to be used as artworks.
AVA takes a lot of metrics into consideration before finalizing on images, such as facial expressions of actors, the scene lighting, areas of interest, positioning of subjects on screen. It even categorizes and sorts artworks to show to users categorized into several taste groups.
Netflix is something I watch.
– Famke Janssen
● Production Planning
Data plays an integral part when creators come up with an idea about a new show or movie. A lot of brainstorming takes place before anything gets on the paper, and that’s where data comes in.
With prior experience in creating new and original content and loads of data about how the viewers perceived the previous content, Big Data helps bring out the possible solutions to many of the challenges faced during the planning phase.
These challenges could include identifying shoot locations, time and day of the shoot, and more. Even with simple prediction models, Netflix can save a significant amount of effort put into planning, further reducing expenses.
Netflix is commissioning original content because it knows what people want before they do.
– The New York Times
Netflix has open-sourced Metaflow, their cloud native, human-centric framework aimed at boosting data scientist productivity.
The idea behind Metaflow was to shift the focus of data scientists from worrying about the infrastructure of models to solving problems. Metaflow allowed them the freedom to experiment with their ideas by offering a set of fine-tuned features that almost makes Metaflow feel like a plug-and-play framework. A few noteworthy features of Metaflow are:
● Ability to work on a distributed computing platform
● Option to snapshot code and data for versioning and experimenting
● High-speed and high-performance S3 client
● Support for most machine learning frameworks
Developed and open-sourced by Netflix, Polynote is a polyglot notebook with support for Scala and various other features. Polynote allows smooth integration of JVM based machine learning platform with Python to data scientists and machine learning researchers. A few highlights of this notebook are:
● Provides insights into kernel status and tasks in execution
● Offers simplistic dependency and configuration management
● Provides IDE-like features such as auto-complete, error highlights, reproducibility, editing, improvements, visibility, data visualization and many more.
The vast pool of data that Netflix operates on is spread across multiple platforms such as Amazon S3, Druid, Redshift and MySql, to name a few. To maintain seamless interoperability among these data stores, Netflix needed a service.
This need for simplicity gave birth to Metacat, whose sole purpose was to provide centralized metadata access for all data stores. Netflix created Metaflow with the intent of serving the following core objectives:
● To unify and provide centralized views of metadata systems
● To offer a singular API for dataset metadata for platforms
● To provide a solution for business and user metadata storage of datasets
“Apache Druid is a high performance real-time analytics database. It’s designed for workflows where fast queries and ingest really matter. Druid excels at instant data visibility, ad-hoc queries, operational analytics, and handling high concurrency.”
Netflix uses Apache Druid for ensuring that its users get a high-quality user experience every time. Delivering a top-notch user experience every time is not a simple feat. It requires constant analysis of several events, gathering the necessary data and analyzing it. This data could be anything from the playback information, to device information, to measuring platform performance and several others. All these event metrics make raw data complicated, and that’s where Druid comes into play.
Druid’s task is to provide real-time analytics on databases where queries execute regularly and at uncertain time-periods. It is highly scalable and offers excellent performance for any given workload.
● Use of Python
Netflix loves Python because of how powerful it is and how excellent it gets when paired with libraries, not to mention how smoothly it integrates with other platforms. Netflix uses Python for managing a host of its mission-critical aspects such as:
● Applications managing the CDN infrastructure
● Analyzing operational data, traffic distribution and operating efficiency
● Prototyping visualization tools
● Gaining insights via statistical tools, data exploration and cleaning
● For maintaining information security
● Managing several core tasks using Jupyter notebooks
● For experimentation using A/B tests
Big Data plays a critical role in not just deciding the functioning of Netflix but also presents them with newer opportunities to grow. New technologies often bring their fair share of issues with them, but at Netflix, they have been tackling those issues head-on, consistently by taking community inputs. By open-sourcing several of the libraries and frameworks to the community, Netflix aims to improve not just itself, but other companies as well. In the end, it would be incorrect to say that Netflix takes all its decisions based on Big Data insights as they still rely on human inputs from a lot of people.