The world’s leading publication for data science, AI, and ML professionals.

Recommendation Systems Explained

Explaining & Implementing Content Based, Collaborative Filtering & Hybrid Recommendation Systems in Python

Image taken by Mohamad Zaheri from Splash
Image taken by Mohamad Zaheri from Splash

Table of Contents

  • What is a Recommendation System
  • What Defines a Good Recommendation
  • K Fold Cross Validation
  • MSE
  • RMSD
  • Data
  • Requirements
  • Code
  • Collaborative Filtering Recommendation System
  • Intuition
  • Advantages
  • Disadvantages
  • Example
  • Implementation
  • Content Based Recommendation System
  • Intuition
  • Advantages
  • Disadvantages
  • Example
  • Implementation
  • Hybrid Recommendation System
  • Intuition
  • Advantages
  • Disadvantages
  • Example
  • Implementation
  • Concluding Remarks
  • Resources

What is a Recommendation System

Recommendation engines are a subclass of machine learning which generally deal with ranking or rating products / users. Loosely defined, a recommender system is a system which predicts ratings a user might give to a specific item. These predictions will then be ranked and returned back to the user.

They’re used by various large name companies like Google, Instagram, Spotify, Amazon, Reddit, Netflix etc. often to increase engagement with users and the platform. For example, Spotify would recommend songs similar to the ones you’ve repeatedly listened to or liked so that you can continue using their platform to listen to music. Amazon uses recommendations to suggest products to various users based on the data they have collected for that user.

Recommender systems are often seen as a "black box", the model created by these large companies are not very easily interpretable. The results which are generated are often recommendations for the user for things that they need / want but are unaware that they need / want it until they’ve been recommended to them.

There are many different ways to build recommender systems, some use algorithmic and formulaic approaches like Page Rank while others use more modelling centric approaches like collaborative filtering, content based, link prediction, etc. All of these approaches can vary in complexity, but complexity does not translate to "good" performance. Often simple solutions and implementations yield the strongest results. For example, large companies like Reddit, Hacker News and Google have used simple formulaic implementations of recommendation engines to promote content on their platform. In this article, I’ll provide an intuitive and technical overview of the recommendation system architecture and the implementation of a few different variations on a sample generated dataset.

What Defines a Good Recommendation?

Identifying what defines a good recommendation is a problem in itself that many companies struggle with. This definition of "good" recommendations help evaluate the performance of the recommender you built. The quality of a recommendation can be assessed through various tactics which measure coverage and accuracy. Accuracy is the fraction of correct recommendations out of total possible recommendations while coverage measures the fraction of objects in the search space the system is able to provide recommendations for. The method of evaluation of a recommendation is solely dependent on the dataset and approach used to generate the recommendation. Recommender systems share several conceptual similarities with the classification and regression modelling problem. In an ideal situation, you would want to see how real users react to recommendations and track metrics around the user to improve your recommendation, however, this is quite difficult to accomplish. Common statistical accuracy measures to evaluate accuracy of a recommender are RMSD, MAE, and k fold cross validation.

K Fold Cross Validation

  • Imagine you’ve built a model which will predict how well a user will rate an item based on a set of features. K fold cross validation can be used to infer the results of the model through accuracy metrics
  • Same idea as a train test split, except we create K many randomly assigned training and test sets
  • Each individual training set / fold is used to train on the recommendation system independently and then measure the accuracy of the resulting systems against the test set
  • We take the average of accuracy score to see how well the recommendation system is learning
  • This method is beneficial to prevent your model from overfitting, however it is a computationally extensive process

MAE (Mean Absolute Error)

Image taken from Statistics How To
Image taken from Statistics How To
  • Represents the average absolute value of each error in rating prediction
  • Lower the MAE score the better

RMSD (Root Mean Square Deviation)

Image taken from Statistics How To
Image taken from Statistics How To
  • A similar metric to MAE but has a stronger penalty for when the prediction is very far from the true value and weaker penalty for when the prediction is closer to the true value
  • Taking the squares off the difference of true and predicted values instead of the sum of the absolute values. This ensures that the resulting value is always positive and is larger when the difference is high and smaller when the difference is low.
  • The lower the RMSD score the better

These metrics are often used to evaluate the quality of a recommendation but they lack various components. Having user data associated with recommendations is essential to know the true quality of a recommendation. Being able to track hit rates of recommendations, engagement to the platform, responsiveness etc. will provide a clearer viewpoint of recommendation quality. Other components to be aware of is to know when to change recommendations when the user hasn’t interacted with them for an X amount of time, or when to re-train recommenders based on new ratings or interactions from the users. You would also want to pay attention to whether or not these recommendations are limiting the user to a subsection of the products, how does the recommender deal with novelty, diversity and selection bias. A/B testing is often the method used to keep track of these metrics (check out my articles on bayesian and frequentist a/b testing).

Data

In the following sections we are going to go more in depth about different methods of creating recommendation engines and the associated implementations in Python. This section will provide a script which will synthesize a dataset associated with books. The dataset will be used for applications of recommendation systems in the following sections, the goal of this article is not to get meaningful results but to show the user the intuition and implementation behind various types of recommendation engines. Hence the results of these recommendations will be meaningless but the methodologies will be similar to those in production grade environments in industry.

Requirements

  • Python = 3.8.8
  • Pandas = 1.2.4

Code

Collaborative Filtering Systems

Intuition

Collaborative filtering is the process of predicting the interests of a user by identifying preferences and information from many users. This is done by filtering data for information or patterns using techniques involving collaboration among multiple agents, data sources, etc. The underlying intuition behind collaborative filtering is that if users A and B have similar taste in a product, then A and B are likely to have similar taste in other products as well.

There are two common types of approaches in collaborative filtering, memory based and model based approach.

  1. Memory based approaches – also often referred to as neighbourhood collaborative filtering. Essentially, ratings of user-item combinations are predicted on the basis of their neighbourhoods. This can be further split into user based collaborative filtering and item based collaborative filtering. User based essentially means that like minded users are going to yield strong and similar recommendations. Item based collaborative filtering recommends items based on the similarity between items calculated using user ratings of those items.
  2. Model based approaches – are predictive models using Machine Learning. Features associated to the dataset are parameterized as inputs of the model to try to solve an optimization related problem. Model based approaches include using things like decision trees, rule based approaches, latent factor models etc.

Advantages

The main advantage to using collaborative filtering models is its simplicity to implement and the high level coverage they provide. It is also beneficial because it captures subtle characteristics (very true for latent factor models) and does not require understanding of the item content.

Disadvantages

The main disadvantage to this model is that it’s not friendly for recommending new items, this is because there has been no user/item interaction with it. This is referred to as the cold start problem. Memory based algorithms are known to perform poorly on highly sparse datasets.

Examples

Some examples of collaborative filtering algorithms :

  • YouTube content recommendation to users – recommending you videos based on other users who have subscribed / watched similar videos as yourself.
  • CourseEra course recommendation – recommending you courses based on other individuals who have finished existing courses you’ve finished.

Implementation

  1. Import data from generate_data function (function provided above) or download the CSV from here
  2. Generate a pivot table with readers on the index and books on the column and values being the ratings
  3. Calculate similarity between items and users using svds
  4. Generate item recommendations based on user_id

    Content Based Systems

Intuition

Content based systems generate recommendations based on the users preferences and profile. They try to match users to items which they’ve liked previously. The level of similarity between items is generally established based on attributes of items liked by the user. Unlike most collaborative filtering models which leverage ratings between target user and other users, content based models focus on the ratings provided by the target user themselves. In essence, the content based approach leverages different sources of data to generate recommendations.

The simplest forms of content based systems require the following sources of data (these requirements can increase based on the complexity of the system you’re trying to build):

  1. Item level data source – you need a strong source of data associated to the attributes of the item. For our scenario, we have things like book price, num_pages, published_year, etc. The more information you know regarding the item, the more beneficial it will be for your system.
  2. User level data source – you need some sort of user feedback based on the item you’re providing recommendations for. This level of feedback can be either implicit or explicit. In our sample data, we’re working with user ratings of books they’ve read. The more user feedback you can track, the more beneficial it will be for your system.

Advantages

Content based models are most advantageous for recommending items when there is an insufficient amount of rating data available. This is because other items with similar attributes might have been rated by the user. Hence, a model should be able to leverage the ratings along with the item attributes to generate recommendations even when there isn’t a lot of data.

Disadvantages

There are two main disadvantages of content based systems.

  1. The recommendations provided are "obvious" based on the items / content the user has consumed. This is a disadvantage because if the user has never interacted with a particular type of item, that item will never be recommended to the user. For example, if you’ve never read mystery books, then through this approach, you will never be recommended mystery books. This is because the model is user specific and doesn’t leverage knowledge from similar users. This reduces the diversity of the recommendations, this is a negative outcome for many businesses.
  2. They’re ineffective for providing recommendations for new users. When building a model you require a history of explicit / implicit user level data for the items. It’s generally important to have a large dataset of ratings available to make robust predictions without overfitting.

Examples

Some examples of content based systems are :

  • Amazon product feed (you’re being recommended products similar to what you’ve previously purchased)
  • Spotify music recommendations

There are many excellent content based systems which are built algorithmically without the dependency on a model based approach. For example companies like Hacker Rank and Reddit have been known to previously used algorithmic approaches to recommend new posts on their platform to users. The key to building an algorithmic approach to content based recommenders lies in defining a set of rules for your business which can be used to rank items. In the case of Reddit, their recommendations are bounded by time of post, number of likes, number of dislikes, number of comments, etc. This can be factored into a formula to generate a score for a post, a high score would yield a high recommendation and vice versa.

Implementation

  1. Import data from generate_data function (function provided above) or download the CSV from here
  2. Normalize book_price, book_ratings, num_pages
  3. One hot encode publish_year, book_genre, text_lang
  4. Given a book_id input, calculate the cosine similarity and return top n books similar to the input

    Hybrid Recommendation System

Intuition

Various methods of recommendation systems have their own benefits and flaws. Often, many of these methods may seem restrictive when used in isolation, especially when multiple sources of data are available for the problem. Hybrid recommender systems are ones designed to use different available data sources to generate robust inferences.

Hybrid recommendation systems have two predominant designs, parallel and sequential. The parallel design provides the input to multiple recommendation systems, each of those recommendations are combined to generate one output. The sequential design provides the input parameters to a single recommendation engine, the output is passed on to the following recommender in a sequence. Refer to the figure below for a visual representation of both designs.

Parallel and Sequential Recommendation System Architecture. Image provided by C.C. Aggarwal, Recommender Systems: The Textbook
Parallel and Sequential Recommendation System Architecture. Image provided by C.C. Aggarwal, Recommender Systems: The Textbook

Advantages

Hybrid systems combine different models to combat the disadvantages of one model with another. This overall reduces the weaknesses of using individual models and aids in generating more robust recommendations. This yields more robust and personalized recommendations for users.

Disadvantages

These types of models generally have high computational complexity and require a large database of ratings and other attributes to keep up to date. Without up to date metrics (user engagement, ratings, etc.) it makes it difficult to retrain and provide new recommendations with updated items and ratings from various users.

Example

Netflix is a company which uses a hybrid recommendation system, they generate recommendations to users based on the watch and search style of similar users (collaborative filtering) in conjunction with movies which share similar characteristics who’ve been rated by users (content based).

Implementation

  1. Import data from generate_data function (function provided above) or download the CSV from here
  2. Use a content-based model (cosine_similarity) to compute the 50 most similar books
  3. Compute the predicted ratings that the user might give these 50 books using a collaborative filtering model (SVD)

  4. Return the top n books with the highest predicted rating

    If you want a step by step guide on how to implement a hybrid recommendation system in a different manner not shown above, you can reference my article on building link prediction recommendation engines with Node2Vec.

Link Prediction Recommendation Engines with Node2Vec

Concluding Remarks

Please be advised that this is just an introduction to recommendation engines. There are many more types and approaches to building out strong recommendation engines which are not covered in this article. Some notable ones are generating recommendations through link prediction, algorithmic approaches, bayesian models, markov models, etc…

The aim of the article was to provide an intuitive understanding and implementation of the foundational methods used for recommendation systems (collaborative filtering, content based and hybrid). Collaborative filtering is the process of predicting the interests of a user by identifying preferences and information from many users, whereas content based systems generate recommendations based on the users preferences and profile. Hybrid systems are often a combination of many recommendation systems.

I’ll list some food for thought for evaluating the quality of a recommendation below. These questions aren’t answered in this article but are imperative when building a good recommendation system.

  • How can you track the churn, responsiveness, engageability and novelty of recommendations?
  • When should you show a user a new recommendation after knowing they haven’t interacted with the previous one for X amount of time?
  • When users rate or interact with new products does it impact your recommendation system immediately, or later on?
  • How do recommendations change as users’ interaction with the platform increases?
  • How do you know your recommendations aren’t limiting your users to only minor subsections of the business?

I purposely tried to implement various versions of each type of recommendation system to give a more broad understanding of their implementations. All the code and data used to generate these results can be viewed here.

Resources


If you enjoyed reading this article, check out these others you might also be interested in :

Bayesian A/B Testing Explained

Monte Carlo Method Explained

Random Walks with Restart Explained


Related Articles