A framework for tracking and attributing the components of player effectiveness

In our original paper, we demonstrated that a baseline model using player age, draft pick and position can be used to explain a large proportion of each of the following components – points per minute, time on ground and percentage of season played – across the Afl player cohort.
In this follow up article, we look at the components of an individual player’s performance by breaking down an individual’s performance into two components :
Player Performance = Baseline Characteristics + Player Specific Characteristics
From the above definition we can begin quantify answers to questions about the average player using the baseline model, as well as compare characteristics of individual players using the player attribution model.
- Baseline Model – (1) What percentage of the season can we expect rookies vs experienced players to play? (2) How does player output change as player roles change – moving up or the field relative to the goal square? (3) What is the expected difference in output from an early vs late round draft pick?
- Player Attribution Model—(1) How do we measure a given player’s skill is relative to their age cohort, after adjusting for position? (2) How much of player improvement from one season to the next comes from skill, role/fitness and durability?
We hope that understanding the intuition of these predictors allows us to make better selections for both the core and non-core players for our AFL fantasy team for season ahead.
Exploring the Data Set
Data for players for the 2015–2021 seasons was scraped from Footywire which comprised a total of 39,000 player-match combinations which aggregated down to 3,000 player-season combinations.
In order to better calibrate the model we first consider the characteristics of the average player in dataset at the game level.

Baseline Draft Pick The median draft pick for each match increases from around 10 for rookies and increases to 30 as players mature. Maturity occurs around 24, or in the 5–6th season of play.

Baseline Position Players generally begin as defenders or forwards and graduate to becoming midfielders and rucks with age and experience.
Player Baseline Model v2
The first iteration of the model used player-season aggregates to demonstrate the concept that a players age and draft pick number can be used to create a baseline of expected player output.
The updated model utilises the richness and variability of the dataset at the match level in the model training process in order to achieve our objective of maximising the accuracy of predictions at the season level.
In order to bridge the gap between the match and season level data, we add an additional variable – elapsed Season – to reflect the point in the season where each match observation is made.

There is a fine balance in terms of the predictors we choose – we want a baseline model which generalises well using age and draft pick and a few features and a second player model where specific characteristics of player skills, style and experience allows us to identify consistently outperforming players.
- We also examine the effect on accuracy as a result of incrementally adding potential predictors to the model.
- Additionally as our in-house position model had much greater accuracy, we drop the data points prior to 2015 where granularity of data required for the position model was not available.

Similar to the previous analysis, we look at the three basic models – loess, gradient boosting and random forest – we find that the random forest is the best model to progress forward with, given its results have the lowest mean absolute error (MAE).
Generalised Performance of the Typical Player
The baseline model generalises the typical player in the AFL cohort. We utilise a Q&A format to explain the types of questions that the model can answer.
Q : What features explain the individual components of the average player performance? The variable importance chart shows the relative importance of each feature to the final output of the model – points per minute (xPPM), time on ground (xMatch) and percentage of season played (xSeason).

- Percentage of time spent in midfield and defence is the most important contributor of points per minute (xPPM).
- Age and Draft Pick are the most relevant contributors to season durability (xSeason) – reflecting that "Moneyball" theory is applicable to AFL.
- For all components, the percentage of time spent playing a ruck position is the least relevant to the final output of the baseline model. We note that player height or vertical distance is a more relevant measure, and will revisit our original modelling on player positions to include this as an input.
- Surprisingly the draft type (ie National or Rookie Draft) holds less importance than the actual draft rank number.
Q : How does expected points per minute (xPPM) vary by player position, age and draft rank? As per the variable importance chart, we observe that position is the most important determinant of points per minute. Across all age groups and positions, early round draft picks generally perform better than later round picks.

- Midfielders have higher output per time unit than other positions; their path to peak is much steeper. For Defenders, the differences between players by draft pick is less pronounced thoughout the life of their career.
- Breakout or the age where a player shows sharp performance improvement begins around 21, depending on position, and is most pronounced in midfielders, perhaps self-fulfilling in that "higher touch" players are more likely sent out into the midfield.
Q : How does time on ground (xMatch) vary by player position, age and draft rank? We can see that 60% is the floor for time on ground, anything less would likely indicate injury or subbing in mid-game. Generally age is not a factor for defenders, while in the other two positions peak time on ground – perhaps a function of fitness – is around 24 years of age.

Q : How does percentage of season played (xSeason) vary by player position, age and draft rank? Generally players play a greater proportion of the season as they age, forwards have the flattest curve in comparison to defenders and midfielders.

Exploring the Draft Class of 2016
In conjunction with the baseline model which explains the generalised player behaviour, we present a series of indices which allows us to compare the career development of individual players.
We demonstrate how to apply the model to track the careers of the 2016 draft class. The example provided also has practical use from a scouting and player valuation/salary perspective.
Application #1 : Player Career Index
A baseline index is created by modelling a "median" draft pick, holding position a constant. The player index is adjusted by the baseline for each match played – ie it is an index of a given player’s skill is relative to their age cohort, after adjusting for position.
The "average" player roughly corresponds to a Round 2 draft pick. We can use the baseline model is to appreciate the evolution of a player’s career. At the end of the 2020, the top 6 draft picks of 2016 earned about 75% more fantasy points over the entire season relative to their age cohort. Individual careers were varied, peppered with injury, resting, suspensions and team performance going into the final rounds of the season.

- McCluggage, McGrath and Petrevski-Seton have consistently played almost full seasons since being drafted while the rest of the players have had some bench/injury periods.
- Taranto’s 2020 playing season started later due to shoulder injury losing ground to his peers during the non-playing period – and has caught up in 2021.
Application #2 : Performance Attribution
We can extend the player analysis by attributing performance components across seasons relative to the baseline – to identify how and where players measure up relative to their cohort. We also transform the base measures into KPI descriptors.

- Style/Skill – the stuff that the player does on the field to have more touches or points per minute than his peers.
- Role/Importance/Consistency – the decision by coach’s to assign more time on ground per match to a player.
- Durability – the ability to play more games per season than his peers (including being able to remain injury-free).
We also find that the model is quite handy for identifying players on the verge of "breaking out" – when we consider that skill is a leading indicator of role and durability.
Exploring Team Performance : Richmond
An alternative practical application of the model is to look at team performance by position. We consider similar charts for the Richmond midfield.
In the following chart we can see that the Richmond midfield in 2015–2018 comprised of a group of much higher than average players. From 2018, Mcintosh’s performance started to look more average, while both Graham and Bolton’s developed over the same period.

Bolton’s breakout year in 2019 is more obvious from the following chart. Comparatively we see that overall team’s skill is still high in 2021 but has begun to converge towards the average as players age.

Reflections and Directions for Future Research
In this article, we have presented only a cursory overview of the potential uses of the baseline/attribution models. Each of these models in itself has potential for further development to extend its practical usability in terms of prediction and diagnostic analysis.
- The measure of performance used in the analysis is AFL fantasy points, which is more specifically a measure of player ball touches, which can be an indicator of player tenacity. Supercoach points, which embeds not publically available player statistics, is considered an indicator of player ball use, which could be a better indicator of player skill. The combination of both these two measures will add richness to how we can describe individual players on the field.
- The baseline model also allows us to develop a draft pick index. For a given player, at each age group, we can project the future lifetime games, minutes or points to be earned, which in turn can be used as a measure of player value/worth and translated into expected player salary (again, not publically available, however would certainly be of interest to coaches, agents, managers and owners of teams) – a step closer to the direction of "Moneyball AFL".
- Experience, as defined by the number of games played, has not been included as an input to the model so far. Experience is self-fulling in that the feedback loop implies that a player who has played a lot of games so far will go on to play even more games. It would be interesting to explore this concept further particularly in determining when or whether individuals have reached peak performance during their career.
- So far, the model results have been presented in aggregate. We will look to further refine and understand the boundaries of its predictive value by assessing the relative magnitude of errors at a more granular level ie by age group or position.
- While the data set used only spans back to 2015 which makes model fitting tricky, the broad conclusions are intuitively in line with what we might expect – access to a longer term data set will enhance the model output and usability.
Overall, it has been quite pleasing to be able to extend the use of AFL fantasy points as a measure beyond the AFL fantasy competition and into more practical applications such as understanding both player career development and individual player and team quirks.
Going forward, we also hope to build an app which allows users to explore and compare the performances of groups of players.
References