1.8 KiB
Linear Models Deep Dive
Building linear models from their mathematical foundations — normal equation and gradient descent implemented by hand — before confirming each result against scikit-learn.
Gradient descent convergence
Implementing batch gradient descent manually and watching the cost function actually decrease iteration by iteration is what makes the "why" behind .fit() concrete instead of abstract:
The bias-variance tradeoff, visualized
Learning curves make the abstract concept concrete — comparing training error against validation error as training set size grows:
Small training sizes show both errors high (underfitting / high bias). As more data is added, the gap between train and validation error reveals how much the model was overfitting to noise.
How regularization actually shrinks coefficients
Ridge, Lasso, and ElasticNet all penalize large coefficients, but differently — this plot shows coefficient magnitude as the penalty strength (α) increases:
Lasso's tendency to zero out weak features entirely (vs. Ridge's smooth shrinkage) is visible directly in the coefficient paths, not just in the theory.
Multiclass decision boundaries
Softmax regression on the Iris dataset — each region shows which class the model assigns highest probability to:
Notebooks
linear_models_notebook.ipynb— original worklinear_models_notebook_polished.ipynb— AI-assisted documentation pass