# Linear Models Deep Dive

Building linear models from their mathematical foundations — normal equation and gradient descent implemented by hand — before confirming each result against scikit-learn.

## Gradient descent convergence

Implementing batch gradient descent manually and watching the cost function actually decrease iteration by iteration is what makes the "why" behind `.fit()` concrete instead of abstract:

![Gradient descent convergence](images/lin-gd-convergence.png)

## The bias-variance tradeoff, visualized

Learning curves make the abstract concept concrete — comparing training error against validation error as training set size grows:

![Learning curve](images/lin-learning-curve.png)

Small training sizes show both errors high (underfitting / high bias). As more data is added, the gap between train and validation error reveals how much the model was overfitting to noise.

## How regularization actually shrinks coefficients

Ridge, Lasso, and ElasticNet all penalize large coefficients, but differently — this plot shows coefficient magnitude as the penalty strength (α) increases:

![Regularization effect on coefficients](images/lin-regularization.png)

Lasso's tendency to zero out weak features entirely (vs. Ridge's smooth shrinkage) is visible directly in the coefficient paths, not just in the theory.

## Multiclass decision boundaries

Softmax regression on the Iris dataset — each region shows which class the model assigns highest probability to:

![Multiclass decision boundaries](images/lin-decision-boundaries.png)

## Notebooks

- [`linear_models_notebook.ipynb`](../linear-models/linear_models_notebook.ipynb) — original work
- [`linear_models_notebook_polished.ipynb`](../linear-models/linear_models_notebook_polished.ipynb) — AI-assisted documentation pass