Machine Learning: Linear Regression with Regularization)

In this post I would like to make a mental note on linear regressions, and the key focus will be on the algorithm for fitting the regularised version of linear regression.

Fundamentally, linear regression seeks to fit a set of data sample by determining the hyperplane (affine) that minimises the mean squared error (MSE) between the data points and the plane. In statistics we seek to find this particular parametric regression \( r(X=x) = E(Y|X = x) \), and if \( Y|X \) follows normal distribution, it can be shown that the minimum MSE is indeed the maximum likelihood estimator (MLE) of the regression.