Add docs with detailed write-ups and result visualizations
This commit is contained in:
36
docs/Linear-Models-Deep-Dive.md
Normal file
36
docs/Linear-Models-Deep-Dive.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# Linear Models Deep Dive
|
||||
|
||||
Building linear models from their mathematical foundations — normal equation and gradient descent implemented by hand — before confirming each result against scikit-learn.
|
||||
|
||||
## Gradient descent convergence
|
||||
|
||||
Implementing batch gradient descent manually and watching the cost function actually decrease iteration by iteration is what makes the "why" behind `.fit()` concrete instead of abstract:
|
||||
|
||||

|
||||
|
||||
## The bias-variance tradeoff, visualized
|
||||
|
||||
Learning curves make the abstract concept concrete — comparing training error against validation error as training set size grows:
|
||||
|
||||

|
||||
|
||||
Small training sizes show both errors high (underfitting / high bias). As more data is added, the gap between train and validation error reveals how much the model was overfitting to noise.
|
||||
|
||||
## How regularization actually shrinks coefficients
|
||||
|
||||
Ridge, Lasso, and ElasticNet all penalize large coefficients, but differently — this plot shows coefficient magnitude as the penalty strength (α) increases:
|
||||
|
||||

|
||||
|
||||
Lasso's tendency to zero out weak features entirely (vs. Ridge's smooth shrinkage) is visible directly in the coefficient paths, not just in the theory.
|
||||
|
||||
## Multiclass decision boundaries
|
||||
|
||||
Softmax regression on the Iris dataset — each region shows which class the model assigns highest probability to:
|
||||
|
||||

|
||||
|
||||
## Notebooks
|
||||
|
||||
- [`linear_models_notebook.ipynb`](../linear-models/linear_models_notebook.ipynb) — original work
|
||||
- [`linear_models_notebook_polished.ipynb`](../linear-models/linear_models_notebook_polished.ipynb) — AI-assisted documentation pass
|
||||
Reference in New Issue
Block a user