Add docs with detailed write-ups and result visualizations
11
docs/Home.md
Normal file
@@ -0,0 +1,11 @@
|
||||
# ML Portfolio — Documentation
|
||||
|
||||
Deep dives into each project: the actual approach, decisions made, and results — illustrated with real plots pulled from the notebooks, not just described in text.
|
||||
|
||||
## Projects
|
||||
|
||||
- [Housing Price Regression](Housing-Price-Regression.md) — end-to-end regression pipeline, test R² 0.72
|
||||
- [MNIST Classification](MNIST-Classification.md) — binary/multiclass/multilabel/multioutput, SGD vs Random Forest
|
||||
- [Linear Models Deep Dive](Linear-Models-Deep-Dive.md) — normal equation through regularization, built from scratch
|
||||
|
||||
Each page walks through the reasoning behind the approach, not just the final numbers — the goal is to show how the conclusions were reached, not just what they were.
|
||||
49
docs/Housing-Price-Regression.md
Normal file
@@ -0,0 +1,49 @@
|
||||
# Housing Price Regression
|
||||
|
||||
End-to-end regression pipeline predicting median house value from California census-block data (20,640 rows).
|
||||
|
||||
## The question driving the approach
|
||||
|
||||
Which features actually predict house value, and can a simple, interpretable pipeline get close to what a more complex model would achieve?
|
||||
|
||||
## Exploratory analysis
|
||||
|
||||
Before modeling, checking which raw features correlate with the target:
|
||||
|
||||

|
||||
|
||||
`median_income` stood out as by far the strongest predictor (r ≈ 0.69) — every later decision, including the stratified train/test split, was built around that finding.
|
||||
|
||||
## Model comparison via cross-validation
|
||||
|
||||
Rather than trusting a single train-set score, both baseline models were evaluated with 5-fold cross-validation to get an honest estimate of real-world error:
|
||||
|
||||

|
||||
|
||||
## What the tuned model actually relies on
|
||||
|
||||
After `GridSearchCV` tuning, inspecting which engineered features the Decision Tree leaned on most:
|
||||
|
||||

|
||||
|
||||
The engineered ratio features (rooms-per-house, etc.) pulled real weight here — not just the raw census counts, which justified the extra feature engineering step.
|
||||
|
||||
## Final result
|
||||
|
||||
Test set touched exactly once, after every modeling decision was locked in:
|
||||
|
||||

|
||||
|
||||
| Metric | Value |
|
||||
|---|---|
|
||||
| MAE | $40,600 |
|
||||
| RMSE | $61,901 |
|
||||
| R² | 0.72 |
|
||||
| 95% CI on RMSE | [$59,141 — $64,542] |
|
||||
|
||||
The scatter shows the model tracks the trend well but underpredicts at the high end — expected, since expensive homes are rarer and harder to fit precisely with limited high-price training examples.
|
||||
|
||||
## Notebooks
|
||||
|
||||
- [`housing-ml-notebook.ipynb`](../housing-price-regression/housing-ml-notebook.ipynb) — original work
|
||||
- [`housing_ml_notebook_polished.ipynb`](../housing-price-regression/housing_ml_notebook_polished.ipynb) — AI-assisted documentation pass
|
||||
36
docs/Linear-Models-Deep-Dive.md
Normal file
@@ -0,0 +1,36 @@
|
||||
# Linear Models Deep Dive
|
||||
|
||||
Building linear models from their mathematical foundations — normal equation and gradient descent implemented by hand — before confirming each result against scikit-learn.
|
||||
|
||||
## Gradient descent convergence
|
||||
|
||||
Implementing batch gradient descent manually and watching the cost function actually decrease iteration by iteration is what makes the "why" behind `.fit()` concrete instead of abstract:
|
||||
|
||||

|
||||
|
||||
## The bias-variance tradeoff, visualized
|
||||
|
||||
Learning curves make the abstract concept concrete — comparing training error against validation error as training set size grows:
|
||||
|
||||

|
||||
|
||||
Small training sizes show both errors high (underfitting / high bias). As more data is added, the gap between train and validation error reveals how much the model was overfitting to noise.
|
||||
|
||||
## How regularization actually shrinks coefficients
|
||||
|
||||
Ridge, Lasso, and ElasticNet all penalize large coefficients, but differently — this plot shows coefficient magnitude as the penalty strength (α) increases:
|
||||
|
||||

|
||||
|
||||
Lasso's tendency to zero out weak features entirely (vs. Ridge's smooth shrinkage) is visible directly in the coefficient paths, not just in the theory.
|
||||
|
||||
## Multiclass decision boundaries
|
||||
|
||||
Softmax regression on the Iris dataset — each region shows which class the model assigns highest probability to:
|
||||
|
||||

|
||||
|
||||
## Notebooks
|
||||
|
||||
- [`linear_models_notebook.ipynb`](../linear-models/linear_models_notebook.ipynb) — original work
|
||||
- [`linear_models_notebook_polished.ipynb`](../linear-models/linear_models_notebook_polished.ipynb) — AI-assisted documentation pass
|
||||
48
docs/MNIST-Classification.md
Normal file
@@ -0,0 +1,48 @@
|
||||
# MNIST Classification
|
||||
|
||||
A full tour of classification problem types on the MNIST handwritten digit dataset — binary, multiclass, multilabel, and multioutput — with an emphasis on evaluating models honestly rather than trusting raw accuracy.
|
||||
|
||||
## Why not just report accuracy?
|
||||
|
||||
About 90% of MNIST digits are *not* an 8. A classifier that always guesses "not 8" scores ~90% accuracy while learning nothing. This project compares models with ROC AUC and F1 instead, and validates every "is it learning something real" claim against a `DummyClassifier` baseline.
|
||||
|
||||
## Linear vs ensemble: SGD vs Random Forest
|
||||
|
||||
Same binary task ("is it an 8?"), two very different models:
|
||||
|
||||

|
||||
|
||||
| Model | F1 | ROC AUC |
|
||||
|---|---|---|
|
||||
| SGD (linear) | 0.58 | 0.915 |
|
||||
| Random Forest (ensemble) | 0.90 | 0.997 |
|
||||
|
||||
The gap illustrates a core limitation of linear models: SGD can only draw one straight decision boundary through 784-dimensional pixel space, while Random Forest's ensemble of trees carves non-linear regions that fit the actual shape of the data.
|
||||
|
||||
## Where the multiclass model actually gets confused
|
||||
|
||||
Extending to all 10 digits, the normalized confusion matrix shows exactly which digits get mistaken for which — more useful than a single accuracy number:
|
||||
|
||||

|
||||
|
||||
## Multioutput: predicting an entire image
|
||||
|
||||
Pushed the idea further — instead of predicting a label, predicting all 784 pixel values at once, framed as denoising a noisy digit:
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
<td><img src="images/clf-denoise-original.png" width="200"/></td>
|
||||
<td><img src="images/clf-denoise-noisy.png" width="200"/></td>
|
||||
<td><img src="images/clf-denoise-predicted.png" width="200"/></td>
|
||||
</tr>
|
||||
<tr>
|
||||
<td align="center">Original</td>
|
||||
<td align="center">Noisy input</td>
|
||||
<td align="center">Model's denoised output</td>
|
||||
</tr>
|
||||
</table>
|
||||
|
||||
## Notebooks
|
||||
|
||||
- [`classification.ipynb`](../classification-project/classification.ipynb) — original work
|
||||
- [`classification_notebook_polished.ipynb`](../classification-project/classification_notebook_polished.ipynb) — AI-assisted documentation pass
|
||||
BIN
docs/images/clf-confusion-matrix.png
Normal file
|
After Width: | Height: | Size: 43 KiB |
BIN
docs/images/clf-denoise-noisy.png
Normal file
|
After Width: | Height: | Size: 6.7 KiB |
BIN
docs/images/clf-denoise-original.png
Normal file
|
After Width: | Height: | Size: 3.8 KiB |
BIN
docs/images/clf-denoise-predicted.png
Normal file
|
After Width: | Height: | Size: 5.7 KiB |
BIN
docs/images/clf-roc-comparison.png
Normal file
|
After Width: | Height: | Size: 24 KiB |
BIN
docs/images/housing-actual-vs-predicted.png
Normal file
|
After Width: | Height: | Size: 96 KiB |
BIN
docs/images/housing-correlation.png
Normal file
|
After Width: | Height: | Size: 27 KiB |
BIN
docs/images/housing-cv-comparison.png
Normal file
|
After Width: | Height: | Size: 17 KiB |
BIN
docs/images/housing-feature-importance.png
Normal file
|
After Width: | Height: | Size: 58 KiB |
BIN
docs/images/lin-decision-boundaries.png
Normal file
|
After Width: | Height: | Size: 98 KiB |
BIN
docs/images/lin-gd-convergence.png
Normal file
|
After Width: | Height: | Size: 158 KiB |
BIN
docs/images/lin-learning-curve.png
Normal file
|
After Width: | Height: | Size: 43 KiB |
BIN
docs/images/lin-regularization.png
Normal file
|
After Width: | Height: | Size: 66 KiB |