Add docs with detailed write-ups and result visualizations

This commit is contained in:
rezayaghobi-dev
2026-07-11 17:42:47 +03:30
parent 309627fa50
commit e37cc01c75
18 changed files with 146 additions and 1 deletions

View File

@@ -0,0 +1,36 @@
# Linear Models Deep Dive
Building linear models from their mathematical foundations — normal equation and gradient descent implemented by hand — before confirming each result against scikit-learn.
## Gradient descent convergence
Implementing batch gradient descent manually and watching the cost function actually decrease iteration by iteration is what makes the "why" behind `.fit()` concrete instead of abstract:
![Gradient descent convergence](images/lin-gd-convergence.png)
## The bias-variance tradeoff, visualized
Learning curves make the abstract concept concrete — comparing training error against validation error as training set size grows:
![Learning curve](images/lin-learning-curve.png)
Small training sizes show both errors high (underfitting / high bias). As more data is added, the gap between train and validation error reveals how much the model was overfitting to noise.
## How regularization actually shrinks coefficients
Ridge, Lasso, and ElasticNet all penalize large coefficients, but differently — this plot shows coefficient magnitude as the penalty strength (α) increases:
![Regularization effect on coefficients](images/lin-regularization.png)
Lasso's tendency to zero out weak features entirely (vs. Ridge's smooth shrinkage) is visible directly in the coefficient paths, not just in the theory.
## Multiclass decision boundaries
Softmax regression on the Iris dataset — each region shows which class the model assigns highest probability to:
![Multiclass decision boundaries](images/lin-decision-boundaries.png)
## Notebooks
- [`linear_models_notebook.ipynb`](../linear-models/linear_models_notebook.ipynb) — original work
- [`linear_models_notebook_polished.ipynb`](../linear-models/linear_models_notebook_polished.ipynb) — AI-assisted documentation pass