While I didn’t become profitable in DFS— I got quite close, and had quite a fun time developing a strategy that deployed real money in real-time NBA DFS competitions.

Nate Harner

Jan 19, 2020·9 min read

In late 2018, I (with help from my brother Zach) set out to create a fully automated daily fantasy sports (DFS) strategy. The goal was more to work on a fun side project over the holiday break than to make any money — I had never played any daily fantasy, but I’m a semi-hardcore basketball fan, and a hardcore data fan.

Here’s the story of how, over the course of two months in late 2018, I developed a DFS strategy that, while not-quite profitable, was completely hands-off, and outperformed the average DraftKings player with an overall win rate of 40% and was profitable during the first 2 of 5 months of the 2018–19 NBA season.

From the outset, I laid out a few requirements for the project. I initially limited the scope to NBA double-up and 50/50 competitions on DraftKings.

Requirements

1. Automated: The entire pipeline should be automated, including entering competitions.
2. Backtest-able: The strategy should be fully backtest-able. We should be able to validate that any changes made to any part of the pipeline lead to better performance than in the past.
3. Human Intervention: Zero “expert knowledge” should be required. For instance, if LeBron has consistently underperformed when matched up against Kawhi, the strategy should figure this out, rather than being told-so explicitly.
4. Replicable: Any data used should be public information (i.e., no subscription services, no private historical data like cash lines).
5. Scalable: The pipeline should scale to multiple platforms and sports with minimal additional work.
6. Fast: Development should be fluid — each step of the pipeline should be able to run independently, and long-running or I/O intensive segments should be performed infrequently and offline.

The Pipeline

Here’s an overview of the steps the pipeline would take for “Live-Mode”, or the process that runs every day to enter fantasy leagues. “Simulation-Mode” (backtesting) is simply a reorganization and looping of some of these steps — we’ll get into that later.

1. Stats Retrieval: We start by retrieving historical data that we will use as the basis for making per-player predictions. For the NBA, this includes per-game player and team stats scraped from basketball-reference, as well as auxiliary data such as betting lines and predicted starting lineups.
2. Feature Creation: Using these stats, we construct derived Features that will be the basis of the learning algorithm.
3. Model Fit: Using these derived features, we fit a model that creates a mapping from Features to FantasyPoints.
4. Roster Retrieval: Retrieve the eligible players, injury status, matchups, and, most importantly, cost-per-player (salaries) for the day.
5. Prediction Feature Creation: Apply the same process we use to create Features to transform roster data into the same exact Features we used to create the player model.
6. Prediction: Using the model we created earlier and the PredictionFeatures, we make fantasy point predictions.
7. Team Selection: We run a linear-optimization (maximize predicted fantasy points subject to salary and position constraints) to produce the team we will enter.
8. League Entry: We use a Selenium bot to navigate the fantasy venue, find target competitions (for instance, 50/50 full slates with $25 entry fees), and enter our team.

In the following sections, I’ll go into some (hopefully) interesting details on specific steps in the pipeline.

Feature Creation

In an ideal world, we could give a learning algorithm raw data, and out would pop the information we want (fantasy point predictions). This is the current state-of-the-art in many domains such as image recognition and natural language processing. Although this may very well be possible for this problem, I took the approach of creating handcrafted features that “manually” extract higher order information ahead of time, rather than having the learning algorithm do this.

To create these features, I built a tool called the “Feature Framework” that allows our features to be expressed as a Directed Acyclic Graph (DAG). Note that this is not my novel idea and is a common paradigm in research-driven trading. There are two main advantages of this approach:

1. We don’t have to repeat computation for multiple features that rely on the same or overlapping sub-features.
2. The framework handles wiring features together — we don’t have to explicitly say that FantasyPoints needs to be computed before AvgFantasyPoints.

Case Study: Defense vs Position

We define “Defense vs Position” (DVP) as the historical points allowed by an opponent in a given position over a pre-defined time horizon. This can be interesting in NBA DFS if an opponent typically allows an outsized amount of points against a given position (imagine the Horford v. Jokic matchup).

When we are finished with the feature creation process, we will have a two-dimensional matrix of (player-game by feature) to use in the learning algorithm:

Model Creation

The model’s purpose is to take in Features (specifically, aspects of a player’s performance historically) and produce predictions of how many fantasy points they will score in a given game. We fit this model using all available historical data, which for the 2018–19 season constitutes about 60,000 rows (using 2017–18 data as well).

It’s important to note that we are trying to minimize (PredictedFantasyPoints -RealizedFantasyPoints) in this formulation of the problem. This has at least two potential drawbacks:

1. This approach weighs every player and every game equally, when in reality, some players may be more important than others.
2. Minimizing the error in player predictions may not directly correlate to winning competitions.

Here’s an example of the output weights from a Linear Regression on the previous 5 games of trailing fantasy points, which is about as simple as we can make the model. The cross-validated mean-average-error (MAE) is 6.4 FantasyPoints.

Non-Linear Fitting

We can drastically improve on these fit results by using a more modern, and most importantly, non-linear fitting algorithm. After trying many non-linear models, such as SVM, Neural Networks, and Decision Trees, the XGB model performed the best. This isn’t too surprising, as XGB is regularly used in Kaggle winning entries concerning tabular data. Even on this same set of simple features, XGB lowers the cross-validated MAE to roughly 6.2. On the full set of features, we can achieve an average error of only 5.38 FantasyPoints per player using XGBoost. For comparison, FanDuel’s stock predictions have an error closer to 6.0, and when evaluating paid-subscription models, I could not find any that get lower than 5.5.

Prediction

Prediction is the pipeline step of applying the previously-fit model to day-of data to get fantasy point predictions. For the Linear Regression Model, the computation is a simple dot-product:

XGB predictions are made by traversing a decision tree. Here’s a breakdown of the XGB model’s prediction (27.6) for Al Horford’s FantasyPoints on Jan 30th 2019.

Backtesting

One pitfall in the approach that many (usually much better than me) DFS players take is the inability to evaluate if changes to their strategy are successful. Without the ability to backtest, there may not be enough data to confidently say that changes are beneficial. To address this concern, I made sure that every part of the pipeline could be rerun for any date in the past (without looking into the future of course). Here are the steps of the simulation:

To determine the cash lines for past contests, we can use the RotoGrinders ResultsDB to get historical cash lines from DraftKings “double-up” contests. Note that when I was working on the pipeline, I used private results from Fanduel 50/50 contests that I participated in.

The backtesting workflow is as follows:

1. Create a feature that may have some predictive power (for instance, DvP)
2. Check if some measure of model error decreases
3. Validate that historical competition results improve

Although I didn’t document the improvements as I made them, the backtesting framework allows us to iteratively add-back features to see how they improve our performance. From these results, we can validate that:

1. New features reduce model error
2. Reduced model error leads to improved competition performance

In the end, the best our model achieves is a 39% win rate over the season, where 50% is profitable in the double-up cash lines we are evaluating (the cutoff is ~56% in 50/50 competitions). We also do much better during the first two months of the season. I’d speculate that features that the algorithm doesn’t account for, such as player matchups, become more and more important towards the end of the season.

Lessons for other DFS’ers

If you’ve come to this article to learn how to be a profitable DFS player, I unfortunately can’t get you there — but I can share some of the things I learned that seem necessary to be (somewhat) competitive. It’s likely that with some additional “secret-sauce” or tweaking of predictions, a seasoned player could become profitable under a mostly-automated system.

1. Backtesting: Sometimes you might really think an idea is awesome, only for it to make no improvement in your strategy, or even cause a regression!
2. Starting Lineups: You have to get starting lineups / late scratches right to have a chance of competing, and messing up even one in every five games can erode your edge.
3. Injured & Replacement Players: DFS, especially NBA 50/50s, often comes down to picking a single player that is a “no-brainer” given their salary and newly injured players on the team. If a player is going to get 3x more minutes than normal because the starter above him is injured, you need to have him on your lineup to stand a chance of competing. DFS has gotten so competitive that these “no-brainer” picks (often a player you possibly haven’t heard of) are often at 80%+ ownership in 50/50s.
4. Position can be a noisy variable: Position, and all the features that stem from it (such as DvP), can be noisy. Players like LeBron can be listed as a PF, when in reality he may in some situations play more like a PG. What we are really trying to extract can be different depending on the use-case, but frequently we are more interested in matchups than specific positions, which can be a sparse data point that humans can do a better job analyzing. An algorithm that determines “probable matchups” could be a step forward on this sub-problem.
5. DFS is hard: According to DraftKings, only 14% of players are profitable in a given month. This automated system was profitable in 2 of 5 calendar months during the 2018–19 season, which puts it well ahead of the average human player. Even with a system that gives decent baseline predictions, you still need to have a very large edge over the field to overcome the ~8% cut that DraftKings and FanDuel take in 50/50 and double-up leagues.

Conclusion

Currently, the DFS problem probably requires a lot of manual effort to solve. Although we could achieve performance that surpassed the average human player on DraftKings, we are still quite far way from a profitable strategy. More than anything else, I had a lot of fun building this project and learned a lot about DFS and practical applications of machine learning. If any readers have questions about the project, please feel free to reach out!