Still debating the value & impact of Page speed?

An often-quoted statistic by Ericsson ConsumerLab is that waiting for a slow loading web page on a mobile device creates as much stress as watching a horror film. A study by Cloud Flare showed the connection between page speed and website conversion rate and if a webpage takes longer than 4 seconds to load on a mobile device, the conversion rate drops to less than 1%.
Page speed is an important thing, especially now that the mobile web has become more prevalent. Google has been stating since 2009 that their goal is to ‘make the web faster’ and between then-and-now has been releasing a raft of different initiatives to help webmasters build their websites with speed in mind.
One of the biggest ways that Google incentivizes website owners to build with page speed in mind, is by publicly stating that page speed is a ranking factor (i.e. is the faster a webpage is, the greater the chance of it ranking highly on Google). All of Google’s resources for helping website owners live on the Make the Web Faster website.
The Google PageSpeed Insights API
The Google PageSpeed Insights API is a tool that lets you query Google Page Speed data programmatically on a per URL basis and it gets its results from two data sources. The first source is the Chrome User Experience (CruX) data set, which is a real-world data set of web page performance data anonymously sourced from opted-in users of Google Chrome.
The second source of data is the open-source Lighthouse project which is a tool that gives predictions of a web pages’ page speed performance and gives recommendations on how to make improvements. It can be accessed in a variety of ways including via a web page, a command-line tool or via the developer tools in Google Chrome.
The PageSpeed Insights API comes back with over 80 different results in its response as a JSON object comprised of data from both CRuX and Lighthouse. A lot of this data is rooted in how webpage speed metrics are measured.
How webpage speeds are measured
There are some page speed metrics that Google has created to measure page speed performance, and the most important of these can be seen on the image below:

The Speed Index ** is a score that is calculated by taking the difference between the First Contentful Paint time and the Visually Ready Time** and in general the lower the Speed Index the better a website performs for page speed.
Why is this important?
Building an enterprise website can be both complicated and expensive and if the website doesn’t perform well in search it can have a detrimental effect on the bottom line of the business the website is representing. If a website is already live but is performing poorly for page speed, it can be expensive to go back in and make updates to make it perform better.
In a competitive business environment where resources are limited, it would be useful to know where to put a budget against making fixes where you know you will get the most benefit. I decided to investigate this further and the first step was to build a data set of page speed performance against rankings and then use Machine Learning to try and understand the data better and get insights.
I built a data set of 100,000 websites using the top 1000 most popular search phrases on Google and running these through serpapi.com to get the positions of the top 100 websites for each keyword phrase. I then queried each of those 100,000 websites against the Pagespeed Insights API to pull out the most important metrics for analysis.
It is interesting to look at a chart for position versus average Speed Index for the top 10 positions on Google for the dataset:

Remember that the lower the Speed Index the better and as can be seen that apart from position 1, there is a clear relationship between Speed Index and position (i.e. the faster the page, the higher position). It would be a gross oversimplification to say that Speed Index drives position (as can be seen by position 1), but there is something there to investigate.
Exploring the data
I needed to be able to take the response from the PageSpeed Insights API and to try to pull out features which can give insight into how best to build pages for performance. I decided to use Speed Index as a predictor against these features. This will be a regression problem as Speed Index is a continuous variable.
Also, as I wanted to be able to try and derive insights from the model I create, I have decided to use more traditional methods of modelling as this will allow me to look at the strengths of coefficients within the model to try and build business insight. Consequently, I didn’t use a neural network.
Using Speed Index **** as the continuous predictor variable I chose the following features as potentially having the largest impact on page speed:
- First-Contentful-Paint – First-contentful-paint marks the time at which the first text or image is painted. Learn more. – unit: ms
- Time-to-Interactive – The TTI metric measures the time from when the page starts loading to when its main sub-resources have loaded and it is capable of reliably responding to user input quickly. Learn more. – unit: ms
- Time-to-First-Byte – Time-to-first-byte identifies the time at which the server sends a response. Learn more. – unit: ms
- DOM Size – The DOM size refers to the number of elements there are in an HTML document and how they relate to each other. Large complicated DOMs can slow down the processing of webpages. Learn more. – unit: no. of elements
- Boot Up Time – This is the time spent parsing, compiling and executing JavaScript. Smaller JavaScript payloads help with this. Learn more. – units ms
- First-Meaningful-Paint – First-meaningful-paint measures when the primary content of a page is visible. Learn more. – units ms
- Total Blocking Time – the sum of time between First Contentful Paint and Time to Interactive, when task length exceeded 50ms, expressed in milliseconds. – units ms
- Network Requests – The number of network requests that were made during page load. – units no. of requests
- Total-Byte-Weight – The sum of the size of all elements requested in a page download. Learn more. – units bytes
With regression models, we have to try and control for multicollinearity, as this can put our model on very shaky ground statistically. Looking at a correlation matrix of all of our features we see the following:

As can be seen, there isn’t a high degree of correlation between the features in the data set and so multicollinearity isn’t an issue. If we look at a pair plot of all features versus the predictor we can see:

We can see a high degree of linearity across most features, particularly for First-Contentful-Paint, Time-to-First-Byte and Time-to-Interactive. This means that we can most likely use a regression model to accurately model this data.
To prepare the data for modelling, it needed to be cleaned (remove outliers, remove null values, check data types are consistent), scaled and deduped (it is possible for the same page to rank for more than one keyword).
Once all these steps were completed, the dataset had 72,218 rows with 13 features. Of these features, 10 were numeric – one as the predictor and the other 9 for modelling.
Modelling the data
The first thing to do was to randomly partition the data set into three separate pieces: a train set, a test set and a final validation set that isn’t used until a candidate model has been selected.
To evaluate the candidate model I also needed to establish a baseline model to be able to accurately check success. As this is a regression problem and a lot of the relationships appeared to be linear then a ‘simple’ linear regression model seemed like a good first step. I put the word ‘simple’ in quotes as it is linear regression in 9-dimensions.
The success metric to determine how well the model is performing is the [r2 error](https://www.bmc.com/blogs/mean-squared-error-r2-and-variance-in-regression-analysis/#:~:text=r2_score()%20method%3A-,Copy,%2C%20pred(y3)%2C%20%E2%80%A6) and after training the model I got the following:
As can be seen, during training the model got an r2 score of 0.76 within the K-fold, meaning that the model wasn’t overfitting and it was performing well. When calling model.predict() **** on the test set the r2 value was 0.77, so this is the starting point to evaluate other candidate models and it isn’t bad.
I tested a variety of other models with various degrees of success: a decision tree regressor (badly over fitted till regularised), Random Forest Regressor, support vector machine linear regressor and a support vector machine polynomial regressor (more out of interest than practical application).

As can be seen, the random forest regressor performed the best with an r2 of 0.78 which is good. The next step is to optimise this result to try and get a better performing model.
Model Optimisation
The next step was to optimise the model hyperparameters using gridsearch cross-validation (GridSearchCV) to tune the model to perform at its optimum level.
After several rounds of GridSearchCV, the optimum model was a Random Forest Regressor with a max_depth = 23 and min_samples_leaf = 3 and this model had an r2 value of 0.79 which is the best so far and a slight improvement over the unoptimised model.
When the optimised model was fitted against the untouched validation data set I continued to get an r2 value 0.79 which meant the model predicted the Speed Index correctly 4 out of 5 times which is a very good result.
Interpreting the model
The real business insight comes from being able to look at the coefficients in the model to be able to decide which one has the most impact on the speed of a web page. It is good that the best performing model is a Random Forest Regressor as it is possible to call the model.featureimportances **** method to see which features contribute the most to the model.
Calling this on the optimum model that I made for this data, we can see the following:

As can be seen, Time-to-Interactive is what overwhelmingly drives the performance when it comes to building a fast webpage. This may seem obvious, however, if you follow the intuition through it makes sense that if you’re Time-to-Interactive is fast then everything else will be fast, but not necessarily the other way around.
For example, if you optimise for First-Meaningful-Paint (when content is visible, but you won’t be able to interact with it) under the rationale that it is better to at least have content visible even if it can’t be interacted with – your page will be slower to load and the user experience will be far worse.
The contribution of Time-to-First-Byte is also significant and as been as this is the first thing that happens when a page loads, it follows that if this is slow then it will have a knock-on effect on everything else.
Conclusion
If you want to build a fast webpage then according to data from the most popular 1000 keywords on Google and the Google PageSpeed Insights API, you need to optimise your build process toward making the page as interactive as possible as quickly as possible. If you do this, the page will be fast and everything else will follow.
Alongside that ensure your server infrastructure is fast so that there are no bottlenecks. Otherwise, this will slow down the whole loading of a page.
To further expand this project, it would be interesting to look at other target variables. Google has just introduced its Core Web Vitals a family of metrics as a way of quantifying the user experience of a page. These would be interesting to study to see if there was a connection between ‘user-friendliness’ and positions in search.
If you’re interested in checking out the code for this project, it is on my Github profile.