Why you should add Reinforcement Learning to your Data Science Toolbox
And where to start learning.
There are many things you can learn as a data scientist. I bet the first things you learned were about statistics, data analysis, visualization, programming, databases, machine learning and deep learning. When you are familiar with these topics, you continue with more specific ones, like GANs, NLP, the cloud, containerization, software engineering and others. You choose a direction based on things you like and skills you need. This blog post explains why adding Reinforcement Learning (RL) to your data science toolbox is a good choice.
If you don’t know anything about RL and you would like to see an application, you can read this article. A short recap: an agent interacts with an environment by choosing actions. The agent receives observations and rewards. The goal of the agent is to maximize the sum of the rewards. This process happens in episodes, the agent learns and becomes better and better at its job.
In real life, learning works similarly. When you teach a dog to listen to commands, you give him a treat when he shows the correct behavior. If he doesn’t, he gets nothing. You can translate this situation to RL. The state is the command he hears, the action is anything the dog does in response to the command and the reward is the treat. The dog will listen better and better, if he likes treats :)
Warning: if you just started your data science career I don’t recommend starting with RL! It’s an advanced topic and you should have knowledge about math and deep learning.
The three main reasons to learn RL
Why should you learn RL?
1. Many business problems can be solved with RL
RL adds valuable knowledge which you can use for many business cases. Some examples are:
- You can build a next-best-action recommendation system. Imagine you have customers, and you want to find the most profitable action for every customer. You can define states for your customers, like new customer, churner or loyal customer. By testing sales and marketing actions you can find the most profitable actions related to the states.
- Scheduling jobs is another interesting task. You can specify a job set, the resources and the time needed for every job. The agent will find a way to optimize job completion time.
- Automation and robotics. By using RL, robots can learn to perform a task perfectly. Some tasks are hard to engineer, and RL can help with these tasks. A nice example is controlling a network of traffic lights signals: by rewarding the RL agents for less delay, the agents find the perfect way to organize the signals. They perform better than normal traffic lights control mechanisms.
Not convinced yet? Let’s continue to the next reason.
2. You don’t need labeled data
Although labeled data can be useful in an RL problem, it’s not necessary. You can build an RL environment from scratch and the agent will find a way to maximize the sum of rewards.
This is a big deal. Data preparation and collection are challenges data scientists face, and a survey shows that it consumes almost 80% of their time. To make things worse, these are the tasks data scientists find the least enjoyable. By using RL, it’s possible to skip this part. The agent can learn from scratch and doesn’t necessarily need labeled data. On the other hand, creating an environment can be difficult, especially when you just started with RL and want to solve a complex problem. When this is the case, you can start easy (with a minimum viable product) and expand your problem by adding characteristics and by experimenting with different definitions of the states, actions and rewards.
AlphaZero, solely trained via self-play, only knew the rules of chess, shogi and Go. It outperformed all the previously invented algorithms and showed cool new ways to play these games.
3. Distinguish yourself
There are many Data Science learning paths online. You can start your journey at Coursera, DataCamp, Udacity or deeplearning.ai, to name a few. Or you can get a bachelor’s or master’s degree. Most courses in these learning paths will teach you how to program and how to analyze, visualize and model data. In the beginning this is okay, but after you’re able to perform these tasks, you can start to distinguish yourself from the ‘regular’ data scientist. Learning Reinforcement Learning can help you accomplish this.
RL isn’t taught in standard data science certification programs and really cool to master. You can bring a new and unique solution to the table and approach problems from a different angle. If you’re able to develop RL solutions related to business problems you set yourself apart from other data scientists.
Besides that, a RL problem is like a puzzle. It’s fun to think about how to define the states, rewards and actions. The working solution brings positive vibes. Experimenting is nice, because with a small change in rewards, an agent can do really strange things (you might want to check out 3 bloopers here).
Where to start learning
Hopefully you are convinced and want to start learning RL! Below are some resources you can use in your RL journey.
Reinforcement Learning, Second Edition: An Introduction
This book by Sutton and Barto is widely used and contains everything you need to know about RL.
Reinforcement Learning Specialization
This series of courses, developed by the University of Alberta, is a great resource. It starts with the fundamentals and helps you to truly grasp the concepts and algorithms explained in the book. After you completed this specialization, you will be able to create your own environments and solve problems with RL.
Reinforcement Learning Course | DeepMind & UCL
This series of lectures are from DeepMind (the creators of AlphaZero) and start with the basics, but also talk about advanced topics.
Enjoy learning!
Don’t forget to subscribe if you’d like to get an email whenever I publish a new article. ❤
