27 lines
1.6 KiB
Markdown
27 lines
1.6 KiB
Markdown
# Linear Models — From Normal Equations to Logistic Regression
|
|
|
|
A from-scratch deep dive into linear models, building each method from its mathematical foundation before confirming the result against scikit-learn.
|
|
|
|
## Overview
|
|
|
|
Rather than just calling `.fit()`, this notebook implements the underlying math directly (normal equation, gradient descent) to build intuition for what scikit-learn's estimators are actually doing internally.
|
|
|
|
## Covered
|
|
|
|
1. **Linear regression via the normal equation** — solving `θ = (XᵀX)⁻¹Xᵀy` directly, then verifying the result matches scikit-learn's `LinearRegression`.
|
|
2. **Batch gradient descent** — implemented manually, with cost-function convergence visualized over iterations.
|
|
3. **Stochastic gradient descent** — compared against batch GD for convergence speed and stability, then via scikit-learn's `SGDRegressor`.
|
|
4. **Polynomial regression** — fitting non-linear (quadratic) data by engineering polynomial features.
|
|
5. **Bias-variance tradeoff** — learning curves showing underfitting vs overfitting as model complexity changes.
|
|
6. **Regularization** — Ridge, Lasso, and ElasticNet compared, with a visualization of how each shrinks coefficients differently as the penalty strength increases.
|
|
7. **Logistic regression** — binary classification via the sigmoid function, with a visualized decision boundary.
|
|
8. **Softmax regression** — multiclass classification on the Iris dataset, with visualized decision regions across all three species.
|
|
|
|
## Tech stack
|
|
|
|
Python, NumPy, scikit-learn, matplotlib
|
|
|
|
## Notebook
|
|
|
|
[`notebook.ipynb`](notebook.ipynb)
|