Which Positions Best Predicted the Outcome of the 2019 Rugby World Cup?

Published in

Towards Data Science

12 min readNov 27, 2020

The aim of the project was to create and compare predictive models for the 2019 Rugby World Cup. The models were created using the results of around 300 matches taken from previous World Cup and Six Nations tournaments, the most basic of which used the difference between the official World Rugby ratings of the two teams on the day of the match, while the more complex ones used aggregates of individual player ratings calculated from their individual histories. These more complex models involve splitting the team into forwards and backs as well as looking at each position individually.

The models investigated are as follows:

Difference in World Rugby ratings.
Difference in whole team ratings as an aggregate of individual positions.
Difference in forwards ratings as an aggregate of individual positions.
Difference in backs ratings as an aggregate of individual positions.
Difference in ratings for each position.

Summary

It was found that the flankers and centres were the best positions to predict the outcome of the 2019 World Cup while fly halves were the worst. The difference in relative strengths of two teams’ forwards was a better prediction of the match result than the difference in relative strengths of the backs.

Data sources

The data sources for developing the models were ESPN and World Rugby. ESPN has data for player, team and competition records going back to the 1870s and would therefore be an excellent source of information for developing individual player histories. The ratings were taken from World Rugby’s website where a list of rankings and ratings can be found on any date going back to October 2003.

Obtaining rating data from World Rugby

World Rugby give a comprehensive explanation for their ‘Points Exchange’ system which changes the rating of the two participants of the match depending on their ratings before the game, home advantage, whether or not the game was part of a World Cup and the result of said match. Using this ‘Points Exchange’ system, it would be possible to create individual player ratings. A player’s rating for their debut game would be set to their team’s official rating on that date and would change following the ‘Points Exchange’ system (where the World Rugby official rating is used for the opposition).

The official World Rugby rating data was stored in a local database using Microsoft SQL Server Management Studio 18 since it would would be very inefficient to get the official rating data from World Rugby’s website every time it was required as only one date’s worth of ratings is shown on the web page at a time. Shown below is a screenshot of World Rugby’s rankings page as an illustration:

https://www.world.rugby/tournaments/rankings/mru taken on 19/11/2020

A combination of Selenium and Beautiful Soup were used to scrape the data from World Rugby’s rankings page as shown in the flowchart below:

A full page of ratings would be put into each row of the output table with the dates the ratings were valid from and to as the first two entries in each row. An extract of the code which generates each table row is shown below. The last line joins all the entries in the “TableRow” list to a single string with each entry for the table separated by a comma. This would allow for the whole line to be inserted into the SQL table as a row.

# Creates an empty list for a row in the output table  
TableRow = []
    
# Puts the valid from and valid to dates as the first two entries in the current row of the output table
TableRow.append("'%s'"%(ValidFrom))
TableRow.append("'%s'"%(ValidTo))
        
# Gets the data from the current day on the webpage to add it to the current row in the output table    
TableRow = getTableRow(driver,TableColumns,TableRow)
    
# Joins the TableRow list as a single string with entries separated by a comma
TableInput = ','.join(TableRow)

The “getTableRow” function is shown below, the input variable “TableColumns” is a list of comprising of the countries for which ratings are required:

def getTableRow(driver,TableColumns,TableRows):
# Function scraping webpage content to a row for the output table
    
    # Opens the page in BeautifulSoup
    content = driver.page_source
    soup = BeautifulSoup(content,'html.parser')
    
    # Finds the whole table of rankings
    RankingsTable = soup.find('table',class_='fullRankings')
    
    # Finds each row in the whole table
    RankingsTableRows = RankingsTable.findAll('tr')
    
    # Loops through countries detailed in the output table
    for Country in TableColumns[1:]:
        
        # Looping through the rows in the table from the webpage
        for RankingsTableRow in RankingsTableRows[1:]:
            
            # Gets the team name from the current row
            TeamNameTable = 
            RankingsTableRow.find('li',class_='teamName') 
            TeamName = TeamNameTable.find(text=True)
            
            # If country in current row is the correct country
            if Country == TeamName:
                
                # Gets the rating for that country
                Rating = RankingsTableRow.findAll('td')[3]                # Adds rating to the row
                TableRows.append(Rating.find(text=True))
                
                # Moves on to the next row in the webpage table
                break
            
    # Returns the row created for the output table
    return TableRows

Once a row of ratings was created it needed to be inserted into the SQL table. The query is as follows:

# Inserts the generated row into the World_Ratings table
Query = """INSERT INTO dbo.World_Ratings
           VALUES (%s);""" %(TableInput)

Once this was complete all the ratings data needed going back to 2003 would be easily accessible through SQL queries. Below is an extract of the World Rugby ratings table:

Image by author — Sample from the World Ratings table

Obtaining match and player data from ESPN

The data for each country was stored in two tables, a player table and a match table. The player table stored each player who has played for that country from 2003 until the present date along with a number with which to identify that player.

Image by author — Example of the England player table

The match data was obtained by performing a number of queries for the ESPN site:

Team record by match list, all venues, all tournaments: to get the complete list of games and the dates on which they occurred.
Team record by match list, home, all tournaments: to find which of these games were home.
Team record by match list, away, all tournaments: to find which of these games played at an opposition’s home ground.
Team record by match list, neutral venue, all tournaments: to find which of these games were played at a neutral venue.
Team record by match list, all venues, Rugby World Cup: to find which of these games were part of a World Cup tournament.

Each page of the ESPN data was within using a while loop which broke when “PageNo” (an integer indicating the current page) was the same as the total number of pages.

The data was scraped using the “read_html” function in Pandas.

# The raw data from the webpage
rawData = pd.read_html(driver.page_source)[3]

Columns in the desired output table could then be added as shown in the example:

# Converts the date from ESPN format to SQL and puts it in the dataframe
Current_Page_Table.Date = rawData['Match Date'].apply(ESPN2SQL_date)

Given the ESPN data is spread across multiple pages, the data frames from each page will need to be concatenated.

# If on the first page, define the whole dataframe as the current page dataframe
if PageNo == 1:
    allMatches = Current_Page_Table
            
# Otherwise concatenates the current page with the dataframe and resets the index
else:
    allMatches = pd.concat([allMatches,Current_Page_Table])
    allMatches.reset_index(drop = True,inplace = True)

The data was inserted into the SQL table row by row using a for loop once all the pages had been scraped and put into the data frame.

Once the basic information was found for each match, the team selection would need to be found. This was done by querying for the player records shown by match list in the ESPN stats and records. This data was put into a Pandas data frame so it could be cleaned for better use later before storing it in the same database as the official rankings.

The names were replaced with the allocated ID numbers from the player table, the position was identified (props, locks, flankers, centres and wings were not differentiated between roles: loose/tight head etc.) with the position being in brackets indicating the player was a reserve. The data was displayed in order of date. An example ESPN player record query is shown below:

http://stats.espnscrum.com/statsguru/rugby/stats/index.html?class=1;orderby=date;spanmin1=12+Oct+2003;spanval1=span;team=1;template=results;type=player;view=match taken on 19/11/2020

This query allowed the team selection for each match to be found and joined into a table with the general information for the match. Each row in the match table was a different fixture, with columns for date, opposition, points difference, venue (home, away or neutral), RWC game (0 or 1) and the number associated with the player filling each position for that game.

Image by author — Example of England match table

Creating ratings for individual players

The training data for the models came from World Cup and Six Nations matches from years prior to 2019. Previous World Cup games would be a good indicator for future World Cup games, and Six Nations results were used to add more unpredictable matches into the training data since group stage World Cup games can sometimes be too predictable to constitute good training.

Each player’s rating would be calculated by using their history, finding any row in the match table where the date was before the date of the match in question and that player’s number in the row. World Rugby’s ‘Points Exchange’ algorithm was then applied to that player’s whole history.

For investigative purposes a player’s rating history can be plotted on the same axes as the official team’s rankings as shown for Manu Tuilagi below:

Image by author — Player rating history for Manu Tuilagi and England’s official rating against date of match

Tuilagi is a good example for why having ratings for individual players is worth doing: he has a history of injury, which can be seen in the long periods where England’s official rating changes but Tuilagi’s does not. In cases where the team is playing a weaker opposition and so a more common starting 15 are not selected or some key players are injured, the generated rating system would take into account the team selection, rather than relying on the official rating of the nation as a whole.

Creating models to predict match outcomes

The sklearn library’s linear regression functions were used to create the models. The method used to create the model is as follows:

Implement a linear regression on the difference between the two teams’ rating systems (multiple linear regressions for the rating systems where aggregates of individual ratings are used) and the points difference of the match in question.
Find the residuals between the real result of the matches and the predicted results from the linear regression.
Use the standard deviation of the residuals to create a normal distribution indicating uncertainty around the predicted result of the match.

Matches can now be simulated using the following equation where the predicted points difference is the value generated from the regression line and the variation variable is generated at random from the normal distributions created by the residuals from the regression.

Simulated points difference = Predicted points difference + Variation

A linear regression was chosen in order to use the normal distributions generated from the residuals. By adding random variation to the predicted points difference a very precise simulated points difference could be calculated. This was how the data was initially inspected before deciding do larger numbers of simulations with the result limited to win/loss (interesting future work would be to run the large scale simulations with win/loss outcomes using other algorithms, such as a random forest).

The following figures show the points difference of matches plotted against the difference in ratings between the two teams, as well as a histogram showing the distribution of the residuals from the regression line alongside a generated normal distribution. The shaded portion of the graphs indicate two standard deviations away from the regression line.

Image by author — Regression and residual plots for official World Rugby rating model

Image by author — Regression and residual plots for full team aggregate rating model

Image by author — Regression and residual plots for forwards aggregate rating model

Image by author — Regression and residual plots for backs aggregate rating model

Similar plots and models were made looking at the difference in rating for each of the individual positions but they are not shown here.

With the capability to simulate matches now established, the models can be evaluated given their ability to predict outcomes in the 2019 Rugby World Cup.

Simulating Rugby World Cup 2019

Correct Predictions of Each Match

Shown below are the results of each match of the 2019 Rugby World Cup simulated 10000 times using each of the models to predict the results.

The results have been split into each group in the group stage, the knockout stage and a summary of the full tournament.

Image by author — Results of simulating each match 10000 times for every model

The ranking of models’ ability to predict the results of the all the matches in the tournament are as follows with the aggregate models in bold:

Centres
Flankers
Full Team Aggregate
Forwards Aggregate
Scrum Half
Wings
Backs Aggregate
Full Back
Props
Locks
Official Ratings
Number 8
Hooker
Fly Half

The order of models’ predictive success changes however when just the knockout stage is examined. It could be argued that the knockout stage is a better indicator of predictive performance than the whole tournament since those matches are typically harder to predict.

The knockout stage was therefore investigated in more detail. The method chosen for doing that would be to run a simulation of the whole knockout stage (rather than each match) 10000 times. A simulation was ended if it created a match which did not occur (for example if New Zealand had got to the final) and the percentage of simulations where all of the matches had the predicted result was used to determine the quality of the model.

Simulation of Whole Knockout Stage

Image by author — Results of simulated the whole knockout stage 10000 times where

The models in order of correct outcomes of the knockout stage shown below. Where the correct number of correct outcomes is the same for multiple models, the percentage where South Africa win the tournament is used to differentiate and if those values are the same, the percentage where New Zealand get bronze is used.

Full Back
Flankers
Centres
Number 8
Scrum Half
Hooker
Locks
Props
Fly Half
Wings
Forwards Aggregate
Official Ratings
Full Team Aggregate
Backs Aggregate

None of the aggregate models managed to fully predict the group stages however the forwards aggregate predicted South Africa would win the tournament more times than the others.

Overall Results

Each model was given a score based on where it ranked in the individual match simulations and the knockout stage simulations (i.e. 1 point for being rated first, 2 for second etc.). A total score for each model is generated from adding the scores from each of those rankings. The models’ scores are as follows:

Flankers/Centres: 4 Points
Full Back: 9 Points
Scrum Half: 10 Points
Forwards Aggregate: 15 Points
Whole Team Aggregate/Number 8/Wings: 16 Points
Props/Locks: 17 Points
Hooker: 19 Points
Backs Aggregate: 21 Points
Official Ratings/Fly Half: 23 Points

From these it can be seen that flankers and centres were the best positions to predict the outcome of a match in the 2019 rugby world cup and fly half was the worst.

The ability to steal opposition ball in the ruck (either directly or by forcing the opposition to concede a penalty) is an extremely sought after skill in the modern game of rugby. This is typically the role of the flanker and so it makes sense that a team with skilled flankers will have a strong advantage in match play; if you can prevent the opposition from keeping the ball and retain more of your own possession you will tend to win games.

Centres are vital in ensuring the ball gets out to the wings in space, providing an excellent attacking platform for the team. They also have a very hard job in defence, tackling incoming players typically at pace. This all means a good centre has to be able to break defensive lines, provide a clean pass at pace to the wings and make some of the hardest tackles in a game. As a result it follows that centres would be a good indicator for performance.