37 lines
1.8 KiB
Markdown
37 lines
1.8 KiB
Markdown
# Linear Models Deep Dive
|
||
|
||
Building linear models from their mathematical foundations — normal equation and gradient descent implemented by hand — before confirming each result against scikit-learn.
|
||
|
||
## Gradient descent convergence
|
||
|
||
Implementing batch gradient descent manually and watching the cost function actually decrease iteration by iteration is what makes the "why" behind `.fit()` concrete instead of abstract:
|
||
|
||
|
||
|
||
## The bias-variance tradeoff, visualized
|
||
|
||
Learning curves make the abstract concept concrete — comparing training error against validation error as training set size grows:
|
||
|
||
|
||
|
||
Small training sizes show both errors high (underfitting / high bias). As more data is added, the gap between train and validation error reveals how much the model was overfitting to noise.
|
||
|
||
## How regularization actually shrinks coefficients
|
||
|
||
Ridge, Lasso, and ElasticNet all penalize large coefficients, but differently — this plot shows coefficient magnitude as the penalty strength (α) increases:
|
||
|
||
|
||
|
||
Lasso's tendency to zero out weak features entirely (vs. Ridge's smooth shrinkage) is visible directly in the coefficient paths, not just in the theory.
|
||
|
||
## Multiclass decision boundaries
|
||
|
||
Softmax regression on the Iris dataset — each region shows which class the model assigns highest probability to:
|
||
|
||
|
||
|
||
## Notebooks
|
||
|
||
- [`linear_models_notebook.ipynb`](../linear-models/linear_models_notebook.ipynb) — original work
|
||
- [`linear_models_notebook_polished.ipynb`](../linear-models/linear_models_notebook_polished.ipynb) — AI-assisted documentation pass
|