I think, whoever starts to learn Machine Learning, checks out the Machine Learning Class by Andrew Ng. I was not an exception. But the mathematical equations and all the formulae made me realize how less I remember my +2 maths. I was stuck at each and every step. But again, it opened a whole new world of Machine Learning.

In one of the videos, there is a deduction of Gradient Descent equation for Linear Regression Model. It took me some time and help to figure that out and now I must write it down. So, this post is all about deriving the Gradient Descent Equation for univariate Linear Regression model.A Univariate Linear Regression model is represented by a straight line equation as below:

ŷ = θ

θj = θ

In one of the videos, there is a deduction of Gradient Descent equation for Linear Regression Model. It took me some time and help to figure that out and now I must write it down. So, this post is all about deriving the Gradient Descent Equation for univariate Linear Regression model.A Univariate Linear Regression model is represented by a straight line equation as below:

ŷ = θ

_{0}+ θ_{1}xThe better fit of the straight line ensures better prediction of data. The best fit of this straight line equation can be obtained by determining the optimum values of θ

So, let the fun begin step by step.

The generalized Gradient Descent equation is as below:_{0 }and θ_{1}. This optimum values of θ_{0 }and θ_{1 }are derived using the Gradient Descent Equation. The mathematical representation of a univariate linear regression model is here.So, let the fun begin step by step.

__Step1:__θj = θ

_{j}- α *^{∂}⁄_{∂θj}J(θ_{1},θ_{2})J(θ

_{1},θ

_{2}) is the Cost Function.

α is a constant called learning rate

__Step2:__J(θ

_{1},θ

_{2}) = 1/2m∑(ŷ - y

_{i})²)

where m = Total number of examples in the Training Dataset.

ŷ = the predicted value = θ

_{0}+ θ_{1}xy

_{i}= actual value of y for i∑ = the sum for i = 1 to m

Now, let's replace J(θ_{1},θ

_{2}) in the above equation:

θj = θ

_{j}-α*

^{∂}⁄

_{∂θj}(1/2m∑(ŷ - y

_{i})²)

__Step3:__The next step is to replace ŷ.

ŷ = θ

_{0}+ θ

_{1}x

θj = θ

_{j}-α*

^{∂}⁄

_{∂θj}(1/2m∑(θ

_{0}+ θ

_{1}x

_{i}- y

_{i})²)

__Step4:__Next comes the Derivative part and we will use the below Differentiation Rules.

Thus, the above equation is modified as below:

θj = θ

_{j}-α/ 2m * ∑

^{∂}⁄

_{∂θj}(θ

_{0}+ θ

_{1}x

_{i }- y

_{i})²

__Step5:__Now, we have to apply the below Differentiation Rule:

θj = θ

_{j}- α/ 2m * ∑ 2(θ

_{0}+ θ

_{1}x

_{i}- y

_{i})²-1 *

^{∂}⁄

_{∂θj}(θ

_{0}+ θ

_{1}x

_{i}- y

_{i})

= θ

_{j}- α/ m * ∑ (θ

_{0}+ θ

_{1}x

_{i}- y

_{i}) *

^{∂}⁄

_{∂θj}(θ

_{0}+ θ

_{1}x

_{i}- y

_{i})

__Step6:__Now we can calculate θ

_{0 and }θ1

_{ }values.

__Let's start with θ___{0}θ

_{0 = }θ

_{0 }- α/ m * ∑(θ

_{0}+ θ

_{1}x

_{i}- y

_{i})

^{∂}⁄

_{∂θ0}(θ

_{0}+ θ

_{1}x

_{i}- y

_{i})

Since all the elements except θ

_{0 }in the above derivative are treated as constants with respect to θ

_{0,}

^{∂}⁄

_{∂θ0}(θ

_{0}+ θ

_{1}x

_{i}- y

_{i}) = 1

Thus,

θ

_{0}= θ

_{0 }- α/ m * ∑(θ

_{0}+ θ

_{1}x

_{i}- y

_{i})

= θ

_{0 }- α/ m * ∑(ŷ - y

_{i})

__Now we can calculate θ___{1}θ

_{1 = }θ

_{1 }- α/ m * ∑(θ

_{0}+ θ

_{1}x

_{i}- y

_{i})

^{∂}⁄

_{∂θ1}(θ

_{0}+ θ

_{1}x

_{i}- y

_{i})

Since θ

_{0 }and y are constants with respect to θ1, the derivative equation is as below:

^{∂}⁄

_{∂θ1}(θ

_{0}+ θ

_{1}x

_{i}- y

_{i}) = x

_{i}

Thus,

θ

_{1 = }θ

_{1 }- α/ m * ∑(θ

_{0}+ θ

_{1}x

_{i}- y

_{i})* x

_{i}

= θ

_{1 }- α/ m * ∑(ŷ - y

_{i}) * x

_{i}

Now, we have derived both the values of θ

_{0}and θ1

_{ }as below:

**θ**y

_{0 = }θ_{0 }- α/ m * ∑(ŷ -_{i}

**)**

**θ**y

_{1 = }θ_{1 }- α/ m * ∑(ŷ -_{i}

**) ***x

_{i}

_{}