Linear Regression (Part 1): types, examples, Gradient descent example

Machine Learning implementation example in 5 minutes(In Part 3). Implement a machine learning model in linear regression in python.

Linear Regression

Prerequisite: Machine Learning Types and list of algorithms

Linear Regression is one of the most common, some 200 years old and most easily understandable in statistics and machine learning. As explained in the previous post it comes under predictive modelling. Predictive modelling is a kind of modelling here the possible output(Y) for the given input(X) is predicted based on the previous data or values.

Types

1. Simple Linear Regression: It is characterized by one independent variable. Consider the price of the house based only one field that is the size of the plot then that would be a simple linear regression.
2. Multiple Linear Regression: It is characterized by multiple independent variables. The price of the house if depends on more that one like the size of the plot area, the economy then it is considered as multiple linear regression which is in most real-world scenarios.

Equation:

Y = a X+ b

Here the main aim is to find the best fit line, which minimizes error(the sum of the square of the distance between points and the line). The distance between the points and line are taken and each of them is squared to get rid of negative values and then the values are summed which gives the error which needs to be minimized.

How is the error minimized??

Consider the example below where there are three points plotted.

Draw a random line where we need to minimize the distance between the line and the three points

Here the error is the distance from the points to the line which is shown in red coloured line.

The error ie. the distance between the points and line are summed up and represented as below.

Now let us move the line in any direction and plot the error.

Here since the line has moved away from the points the error increases and error is summed up and as shown below.

Here the line moves in opposite direction this time since the error was increased previously and results are as below.

The error is minimized here when compared to previous instances.

The error can be still minimized as below.

But since it results in negative values we implement Gradient decent(least squares) where the values of error are squared, which results in only positive values where error can be measured accurately(this is done since distance cannot be measured in negative values).

Mathematically

1. Redline Error: 3 + 4 + 6 = 10 when squared — — 4 + 9 + 25 = 38
2. Yellowline Error: 4 + 5 + 7= 13 when squared — 9 + 16 + 36 = 61
3. Blueline Error: 1 + 2 + 4 = 7 when squared — — 1 + 4 +16 = 21
4. Lest Error: 1–3+1 = -1 which makes no sense but when squared results in 1 + 9 + 1 = 11

So here 61 > 38 > 21 > 11 makes sense rather than handling negative distances which is hard to imagine and handle.

Hence Gradient descent(least squares) is the best here to find the least error when compared to other instances.

In the next article, I have taken a real-world example and implementation of Linear regression(Machine Learning) coming up so stay tuned!!!!

More references:

1. Is Artificial Intelligence real or is it just a hype of this decade??
2. Artificial Intelligence: Definition, Types, Examples, Technologies
3. Artificial Intelligence vs Machine Learning
4. Why Machine learning for achieving Artificial Intelligence? “ The Need for Machine Learning”
5. Machine Learning Types and Algorithms

Next I have Linear Regression (Part 2) coming up make sure to follow me on medium, linkedin, twitter, Instagram to get more updates. And also if you liked this article make sure to give a clap and share it.

Join our WhatsApp community here.