Root Mean Squared Error (RMSE)and Mean Absolute Error (Mae) are metrics used to evaluate a Regression Model. These metrics tell us how accurate our predictions are and, what is the amount of deviation from the actual values.

Technically, RMSE is the Root of the Mean of the Square of Errors and MAE is the Mean of Absolute value of Errors. Here, errors are the differences between the predicted values (values predicted by our regression model) and the actual values of a variable. They are calculated as follows :

On close inspection, you will see that both are average of errors.
Let’s understand this with an example. Say, I want to predict the salary of a data scientist based on the number of years of experience. So, salary is my target variable (Y) and experience is the independent variable(X). I have some random data on X and Y and we will use Linear Regression to predict salary. Let’s use pandas and scikit-learn for data loading and creating linear model.
import pandas as pd
from sklearn.linear_model import LinearRegression
sal_data={"Exp":[2,2.2, 2.8, 4, 7, 8, 11, 12, 21, 25],
"Salary": [7, 8, 11, 15, 22, 29, 37 ,45.7, 49, 52]}
#Load data into a pandas Dataframe
df=pd.DataFrame(sal_data)
df.head(3)

#Selecting X and y variables
X=df[['Experience']]
y=df.Salary
#Creating a Simple Linear Regression Model to predict salaries
lm=LinearRegression()
lm.fit(X,y)
#Prediction of salaries by the model
yp=lm.predict(X)
print(yp)
[12.23965934 12.64846842 13.87489568 16.32775018 22.45988645 24.50393187 30.63606813 32.68011355 51.07652234 59.25270403]
Now, we have ‘yp’ – our array of salary prediction and we will evaluate our model by plotting predicted(yp) and actual salary(y). I am using bohek for my visualizations.
from bokeh.plotting import figure, show, output_file
p=figure(title="Actual vs Predicted Salary", width=450, height=300)
p.title.align = 'center'
p.circle(df.Exp, df.Salary)
p.line(df.Exp, df.Salary, legend_label='Actual Salary', line_width=3, line_alpha=0.4)
p.circle(df.Exp, yp, color="red")
p.line(df.Exp,yp, color="red",legend_label='Predicted Salary', line_width=3, line_alpha=0.4)
p.xaxis.axis_label = 'Experience'
p.yaxis.axis_label = 'Salary'
show(p)

From the graph above, we see that there is a gap between predicted and actual data points. Statistically, this gap/difference is called residuals and commonly called error, and is used in RMSE and MAE. Scikit-learn provides metrics library to calculate these values. However, we will compute RMSE and MAE by using the above mathematical expressions. Both methods will give you the same result.
import numpy as np
print(f'Residuals: {y-yp}')
np.sqrt(np.mean(np.square(y-yp))) #RMSE
np.mean(abs(y-yp)) #MAE
#RMSE/MAE computation using sklearn library
from sklearn.Metrics import mean_squared_error, mean_absolute_error
np.sqrt(mean_squared_error(y, yp))
mean_absolute_error(y, yp)
6.48
5.68
This is our baseline model. MAE is around 5.7 – which seems to be higher. Now our goal is to improve this model by reducing this error.
Let’s run a polynomial transformation on "experience" (X) with the same model and see if our errors reduce.
from sklearn.preprocessing import PolynomialFeatures
pf=PolynomialFeatures() #Linear Equation of degree 2
X_poly=pf.fit_transform(X)
lm.fit(X_poly, y)
yp=lm.predict(X_poly)
I have used Scikit-learn PolynomialFeatures to create a matrix of 1, X, and X2 and passed that as input to my model.
Calculating our error metrics and ….
#RMSE and MAE
np.sqrt(np.mean(np.square(y-yp)))
np.mean(abs(y-yp))
2.3974
1.6386
Voilaaa… they are much lower this time. It fits better than our baseline model! Let’s plot y and yp (like how we did before) to check the overlap.

The gap between the 2 lines has reduced. Let’s observe the distribution of residuals or errors(y-yp) using seaborn’s residual plot function
print(y-yp) #residuals
[ 0.333921 0.447306 0.84028668 -0.136044 -4.190238 -0.434767
-0.847751 5.488121 -2.584481 1.083648]
import seaborn as sns
sns.residplot(y, yp)
plt.show()

We see that residuals tend to concentrate around the x-axis, which makes sense because they are negligible.
There is a third metric – R-Squared score, usually used for Regression models. This measures the amount of variation that can be explained by our model i.e. percentage of correct predictions returned by our model. It is also called the coefficient of determination and calculated by the formula:

Let’s compute R2 mathematically using the formula and using sklearn library and compare the values. Both methods should give you the same result.
#Calculating R-Squared manually
a=sum(np.square(y-yp)) # a -> sum of square of residuals
b=sum(np.square(y-np.mean(y))) # b -> total sum of sqaures
r2_value = 1-(a/b)
0.979
#calculating r2 using sklearn
from sklearn.metrics import r2_score
print(r2_score(y, yp))
0.979
Thus, overall we can interpret that 98% of the model predictions are correct and the variation in the errors is around 2 units. For an ideal model, Rmse/MAE=0 and R2 score = 1, and all the residual points lie on the X-axis. Achieving such a value for any business solution is almost impossible!
Some of the techniques we can use to improve our model accuracy include:
- Transform/scale features
- Treat outliers (if many)
- Add new features/feature engineering
- Use different algorithms
- Model hyperparameter tuning
In my next article, I will be explaining some of the above concepts in detail. You can refer to my notebook here for the above code.