Generalized Least Squares (GLS): Relations to OLS & WLS

with Mathematical Derivations & Intuition

Background and Motivation

For anyone pursuing study in Statistics or Machine Learning, Ordinary Least Squares (OLS) Linear Regression is one of the first and most “simple” methods one is exposed to. However, there are a set of mathematical restrictions under which the OLS estimator is the Best Linear Unbiased Estimator (BLUE), i.e. the unbiased estimator with minimal sampling variance. (For a more thorough overview of OLS, the…