Derivation

Complete Derivation

Figure 1: Page 1 of derivation

Figure 2: Page 2 of derivation

Figure 3: Page 3 of derivation

Figure 4: Page 4 of derivation

<aside> 🗒️

Brain Refresher

Here some relations which should be recalled, so that the derivation becomes smoother. These are from 10+2 in Higher School Statistics.

1. $\sum^N x_i = N\bar{x}$
2. $\bar{x}\bar{y} \neq \overline{xy}$
3. $\bar{x}^2 \neq \overline{x^2}$
4. $\sum{(x+y+z)}=\sum x + \sum y + \sum z$ </aside>

Important Pieces

If we consider the linear regression to be of closed form then the $\{\theta\}$ can be solved by assigning the first derivatives to be zero. Hence the expression would look like

$$ \begin{align} \sum_{i=1}^N (\theta_0 + \theta_1 x_i - y_i) &= 0 \\ \sum_{i=1}^N (\theta_0 x_i + \theta_1 x_i^2 - x_i y_i) &= 0 \end{align} $$

Now if we solve equations 1 and 2 for $\theta_0$ and $\theta_1$ then we get as the answer

$$ \begin{align} \theta_1 &= \frac{\bar{x}\bar{y} - \overline{xy}}{\bar{x}^2 - \overline{x^2}}\\ \theta_0 &= \bar{y} - \theta_1 \bar{x}

\end{align} $$

Here the $\theta_0$ acts as the predicted intercept and $\theta_1$ becomes the predicted slope of the newly fitted straight line.