ML-portfolio/linear-models

Linear Models — From Normal Equations to Logistic Regression

A from-scratch deep dive into linear models, building each method from its mathematical foundation before confirming the result against scikit-learn.

Overview

Rather than just calling .fit(), this notebook implements the underlying math directly (normal equation, gradient descent) to build intuition for what scikit-learn's estimators are actually doing internally.

Covered

1. Linear regression via the normal equation — solving θ = (XᵀX)⁻¹Xᵀy directly, then verifying the result matches scikit-learn's LinearRegression.
2. Batch gradient descent — implemented manually, with cost-function convergence visualized over iterations.
3. Stochastic gradient descent — compared against batch GD for convergence speed and stability, then via scikit-learn's SGDRegressor.
4. Polynomial regression — fitting non-linear (quadratic) data by engineering polynomial features.
5. Bias-variance tradeoff — learning curves showing underfitting vs overfitting as model complexity changes.
6. Regularization — Ridge, Lasso, and ElasticNet compared, with a visualization of how each shrinks coefficients differently as the penalty strength increases.
7. Logistic regression — binary classification via the sigmoid function, with a visualized decision boundary.
8. Softmax regression — multiclass classification on the Iris dataset, with visualized decision regions across all three species.

Tech stack

Python, NumPy, scikit-learn, matplotlib

Notebooks

• linear_models_notebook.ipynb — original notebook, my own work
• linear_models_notebook_polished.ipynb — same analysis with AI-assisted documentation and formatting added for readability