Cross-Validation

We should cross validate the learning of closed form for linear regression into coding exercise. We will use wolfram here. So the plot in Fig. 1 validates the result.

Here we will choose an experimental straight line or linear equation of our choice to generate some synthetic data.

$$ f(x)= 2.4+0.3x $$

With this we create an array of data that we would treat as $\{y_i\}$. And then use the equation 3 and 4 from the appendix Closed Form to compute the predicted slope $(\theta_1)$ and intercept $(\theta_0)$.

Surprisingly LOL, you will find the intercept to be 2.4 and slope to be 3.4 exact.

This is because we assumed at the very beginning of our experiment that we are working with data with accurate linear relationship.

Figure 1: Scatter plots of the synthetically created data and using a least squares fit method for a closed form solution. X axis denotes the index of the dataset not the x data itself. The length of the dataset is 51.

Okay I will provide the minimal code snippet that one can run through a .wl script or a wolfram notebook interface.

<aside> 💡

wolframengine is free or pseudo open source for research and academic purposes.

</aside>

Here is the code snippet. In future I will also paste the python code too.

(*Define the desired function*)
original[xval_] = 2.4 + 0.3*xval
(*Create the dataset*)
exptX = Table[i, {i, 0, 5, 0.1}]
exptY = original[exptX]
(*compute the intercept and slope just write the formulae*)
predSlope = (Mean[exptX] * Mean[exptY] - Mean[exptX * exptY])/((Mean[exptX])^2 - Mean[(exptX)^2])
predIntercept = Mean[exptY] - predSlope*Mean[exptX]
(*generate the new data from predicted slope and intercept*)
predY = Table[predIntercept + predSlope*x, {x, 0, 5, 0.1}]
(*plot for comparing*)
plt=ListPlot[{exptY, predY},
 PlotStyle -> {{PointSize[0.03], Black}, {PointSize[0.02], Red}},
 PlotTheme -> "Detailed",
 FrameLabel -> {Style["X Data", 12, Black], 
   Style["Y Data", 12, Black]}, 
 ImageSize -> Large,
 Background -> None, 
 PlotLegends -> Placed[{"Experiment", "Predicted"}, {Left, Top }], 
 LabelStyle -> Directive[Black, 12],
 FrameTicks -> {None, Automatic}
 ]
(*export into an image file if you are scripting*)
Export["plot.png", plt, ImageResolution-> 300]