Loess from scratch in Python + Animation
Introduction Loess is a class of regression models that allows us to estimate the regression function \(f(X)\) by fitting a simple (and different) model to each point in the domain \(X\). This is done by paying attention to the points closest to a particular target point \(x_0 \in X\). In the classic simple linear regression we have the following model: \[ y_i = f(x_i) + \epsilon_i\] where \(f(x_i)\) is: \[f(x_i) = \beta_0 + \beta_1 \cdot x_i\] and in the case of loess with approximate \(f(x_i)\) by a polynomial weigthed by a function \(w_i\) that assigns higher weights to points \(x_i\) that are closer to \(x_0\): \[f(x_i) = w_i(x_0) \cdot [\beta_0 + \beta_1 \cdot x_i + \beta_2 \cdot x_i^2 + \dots]\] Procedure: Let \(x_i\) denote a set of \(n\) values for a particular variable and let \(y_i\) represent the corresponding response variable. Find the \(k\) closest points to the target point \(x_0\). Read more