# Linear Models — From Normal Equations to Logistic Regression

A from-scratch deep dive into linear models, building each method from its mathematical foundation before confirming the result against scikit-learn.

## Overview

Rather than just calling `.fit()`, this notebook implements the underlying math directly (normal equation, gradient descent) to build intuition for what scikit-learn's estimators are actually doing internally.

## Covered

1. **Linear regression via the normal equation** — solving `θ = (XᵀX)⁻¹Xᵀy` directly, then verifying the result matches scikit-learn's `LinearRegression`.
2. **Batch gradient descent** — implemented manually, with cost-function convergence visualized over iterations.
3. **Stochastic gradient descent** — compared against batch GD for convergence speed and stability, then via scikit-learn's `SGDRegressor`.
4. **Polynomial regression** — fitting non-linear (quadratic) data by engineering polynomial features.
5. **Bias-variance tradeoff** — learning curves showing underfitting vs overfitting as model complexity changes.
6. **Regularization** — Ridge, Lasso, and ElasticNet compared, with a visualization of how each shrinks coefficients differently as the penalty strength increases.
7. **Logistic regression** — binary classification via the sigmoid function, with a visualized decision boundary.
8. **Softmax regression** — multiclass classification on the Iris dataset, with visualized decision regions across all three species.

## Tech stack

Python, NumPy, scikit-learn, matplotlib

## Notebooks

- [`linear_models_notebook.ipynb`](linear_models_notebook.ipynb) — original notebook, my own work
- [`linear_models_notebook_polished.ipynb`](linear_models_notebook_polished.ipynb) — same analysis with AI-assisted documentation and formatting added for readability