Opinion
Reinforcement Learning’s Potential in Sports
Reinforcement learning may well change how sports are played and lead to new optimal techniques in the near future.
Recently, while listening to Pieter Abbeel on the Lex Fridman podcast, Abbeel remarked on how the one person he would most like to meet in the world is Roger Federer. When Fridman asked whether robots through reinforcement learning could come close to emulating the silky smooth Federer forehand, Abbeel remarked that while the algorithms such as imitation learning are likely capable, robotic hardware has a long way to go. Abbeel’s answer poses a new question about what competitive sports will look like once robotic hardware is up to the task. We have seen games like chess and go dominated by computers, with those systems even finding brand new tactics and it seems like the world of competitive sports may be no different.
For those unfamiliar with reinforcement learning, here is a great introduction. The basic premise is to teach a decision-making agent to make optimal decisions through providing them with rewards or penalties. In essence, it seeks to emulate how we as humans learn, through providing feedback on decisions that lead to certain outcomes. Games have been a great way to test reinforcement learning frameworks, as they usually provide a clear positive or negative outcome that can be used as the desired reward signal. In recent years large advancements such as Alpha Zero¹ and Mu Zero² have shown that through “self-play”, a framework where the agent plays against itself without any human interaction or prior knowledge, agents are able to obtain super human level abilities in chess, go, shogi, and 40 or so standard Atari games that reinforcement learning researchers commonly use to benchmark algorithms capabilities. Reinforcement learning algorithms have even shown the ability for agents to cooperate with one another and work as a team, if doing so will lead to a better reward.³ Yet, the interest in games for reinforcement learning has not enveloped the world of sports. This can likely be attributed to the lack hardware as Abbeel points out, but also likely due to the lack of software. Current video games like the Madden franchise and NBA 2K have in-depth physics engines that enable incredibly realistic game play, however as most know, it still does not come close to emulating a current real-life environment. The key advancements in go, chess and others have been enabled through perfect virtualization of the desired environment that allows for millions of iterations and training examples. Once this becomes available we could see results in sports that mirror programs like Alpha Zero, that was able to discover new and improved openings in the game of Go, a game that has been played for hundreds of years. Commonly accepted techniques will change like shooting, throwing and perhaps even new improved versions of simple tasks like running. Furthermore, play-calling in sports like basketball and football may approach a Nash equilibrium. We often hear that players today in sports “aren’t even playing the same game” as their predecessors 20 years ago and that may be hold far more truth than ever before once reinforcement learning becomes widely available and used in sports.
Sports continually get disrupted through players that reinvent what is considered normal. Rafael Nadal dramatically shifted the game of tennis towards hitting with more topspin than previously thought possible. Stephen Curry led the game of basketball to value versatility and sharp shooting instead of height. However, once large scale realistic simulations become possible in sports, we will see disruption at a scale and speed that we have never seen before. And while sports have a nostalgia for tradition, the analytics revolution has proved that those who do not adapt will be left behind. In the future it seems likely that if Abbeel ever gets the chance to meet the great Roger Federer he may even have a chance to teach him a thing or two about playing optimal tennis.
Footnotes
- Silver, Hubert, Schrittweiser, Hassabis. AlphaZero: Shedding new light on chess, shogi and go. https://deepmind.com/blog/article/alphazero-shedding-new-light-grand-games-chess-shogi-and-go
- Silver, Schrittwieser, Antonoglou, Hubert, et al. Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model. https://deepmind.com/blog/article/muzero-mastering-go-chess-shogi-and-atari-without-rules
- Open AI. Dota 2 with Large Scale Deep Reinforcement Learning. https://arxiv.org/pdf/1912.06680.pdf
