Linear Regression

To the left, it’s the plot of the size vs the price from the boston house pricing dataset. Given this dataset we need to find a relationship between size and the house price so that we could suggest a fair price for a new house to be sold given its size.

Linear Regression is a Linear Model. Which means, we will establish a linear relationship between the input variables(X) and single output variable(Y). When the input(X) is a single variable this model is called Simple Linear Regression and when there are multiple input variables(X), it is called Multiple Linear Regression.

This is called Supervised learning, we take the set of right answers, find a pattern in it and then use it to make predictions. Since we are predicting the house price, which is a real and continuous valued output, the prediction problem is called as regression.

The model or the hypothesis(h)

As we discussed earlier we have an input variable X and one output variable Y. And we want to build linear relationship between these variables. The input variable is called Independent Variable and the output variable is called Dependent Variable. Since we are defining a linear relationship it can be defined as follows:

The θ1 is the coefficient and θo is called bias coefficient, which are also called the parameters of the model or hypothesis h(x). This equation is similar to the line equation y = m * x + b with m = θ1(Slope) and b=θo (Intercept), in Simple Linear Regression model we want to draw a line between X and Y which estimates the relationship between X and Y.

But how do we find these coefficients? That’s the learning procedure. The technique or algorithm used for learning is called as the learning algorithm. We’ll be using Gradient Descent Algorithm .

Given the input X the parameters θo, θ1 control the output Y.

In any supervised machine learning task we start with the labelled datasets. Hence, input features X and output Y are pretty much known and will not change. Hence, we can consider them to be constants in the equation. The real variables in our equation are θ0 and θ1 which influences the prediction.

But we need a way see how to choose the prediction (Y^) so that its close value actual value Y. We need a way to quantify how good the parameters are. That’s where the error/cost functions are used.