diff --git a/README.md b/README.md index a13714b..9711432 100644 --- a/README.md +++ b/README.md @@ -1,5 +1,5 @@ +📖 **Full write-ups with result plots:** [docs/Home.md](docs/Home.md) # Machine Learning Portfolio - A collection of end-to-end machine learning projects covering regression, classification, and linear models built from first principles. ![Python](https://img.shields.io/badge/Python-3776AB?style=flat&logo=python&logoColor=white) @@ -25,3 +25,4 @@ Python, pandas, NumPy, scikit-learn, matplotlib, seaborn **Reza** — Computer Networks student, LPIC-1 [GitHub](https://github.com/rezayaghobi-dev) +s \ No newline at end of file diff --git a/docs/Home.md b/docs/Home.md new file mode 100644 index 0000000..f2cf684 --- /dev/null +++ b/docs/Home.md @@ -0,0 +1,11 @@ +# ML Portfolio — Documentation + +Deep dives into each project: the actual approach, decisions made, and results — illustrated with real plots pulled from the notebooks, not just described in text. + +## Projects + +- [Housing Price Regression](Housing-Price-Regression.md) — end-to-end regression pipeline, test R² 0.72 +- [MNIST Classification](MNIST-Classification.md) — binary/multiclass/multilabel/multioutput, SGD vs Random Forest +- [Linear Models Deep Dive](Linear-Models-Deep-Dive.md) — normal equation through regularization, built from scratch + +Each page walks through the reasoning behind the approach, not just the final numbers — the goal is to show how the conclusions were reached, not just what they were. diff --git a/docs/Housing-Price-Regression.md b/docs/Housing-Price-Regression.md new file mode 100644 index 0000000..17ae942 --- /dev/null +++ b/docs/Housing-Price-Regression.md @@ -0,0 +1,49 @@ +# Housing Price Regression + +End-to-end regression pipeline predicting median house value from California census-block data (20,640 rows). + +## The question driving the approach + +Which features actually predict house value, and can a simple, interpretable pipeline get close to what a more complex model would achieve? + +## Exploratory analysis + +Before modeling, checking which raw features correlate with the target: + +![Correlation with target](images/housing-correlation.png) + +`median_income` stood out as by far the strongest predictor (r ≈ 0.69) — every later decision, including the stratified train/test split, was built around that finding. + +## Model comparison via cross-validation + +Rather than trusting a single train-set score, both baseline models were evaluated with 5-fold cross-validation to get an honest estimate of real-world error: + +![Cross-validation MAE comparison](images/housing-cv-comparison.png) + +## What the tuned model actually relies on + +After `GridSearchCV` tuning, inspecting which engineered features the Decision Tree leaned on most: + +![Feature importance](images/housing-feature-importance.png) + +The engineered ratio features (rooms-per-house, etc.) pulled real weight here — not just the raw census counts, which justified the extra feature engineering step. + +## Final result + +Test set touched exactly once, after every modeling decision was locked in: + +![Actual vs predicted](images/housing-actual-vs-predicted.png) + +| Metric | Value | +|---|---| +| MAE | $40,600 | +| RMSE | $61,901 | +| R² | 0.72 | +| 95% CI on RMSE | [$59,141 — $64,542] | + +The scatter shows the model tracks the trend well but underpredicts at the high end — expected, since expensive homes are rarer and harder to fit precisely with limited high-price training examples. + +## Notebooks + +- [`housing-ml-notebook.ipynb`](../housing-price-regression/housing-ml-notebook.ipynb) — original work +- [`housing_ml_notebook_polished.ipynb`](../housing-price-regression/housing_ml_notebook_polished.ipynb) — AI-assisted documentation pass diff --git a/docs/Linear-Models-Deep-Dive.md b/docs/Linear-Models-Deep-Dive.md new file mode 100644 index 0000000..e1bb86d --- /dev/null +++ b/docs/Linear-Models-Deep-Dive.md @@ -0,0 +1,36 @@ +# Linear Models Deep Dive + +Building linear models from their mathematical foundations — normal equation and gradient descent implemented by hand — before confirming each result against scikit-learn. + +## Gradient descent convergence + +Implementing batch gradient descent manually and watching the cost function actually decrease iteration by iteration is what makes the "why" behind `.fit()` concrete instead of abstract: + +![Gradient descent convergence](images/lin-gd-convergence.png) + +## The bias-variance tradeoff, visualized + +Learning curves make the abstract concept concrete — comparing training error against validation error as training set size grows: + +![Learning curve](images/lin-learning-curve.png) + +Small training sizes show both errors high (underfitting / high bias). As more data is added, the gap between train and validation error reveals how much the model was overfitting to noise. + +## How regularization actually shrinks coefficients + +Ridge, Lasso, and ElasticNet all penalize large coefficients, but differently — this plot shows coefficient magnitude as the penalty strength (α) increases: + +![Regularization effect on coefficients](images/lin-regularization.png) + +Lasso's tendency to zero out weak features entirely (vs. Ridge's smooth shrinkage) is visible directly in the coefficient paths, not just in the theory. + +## Multiclass decision boundaries + +Softmax regression on the Iris dataset — each region shows which class the model assigns highest probability to: + +![Multiclass decision boundaries](images/lin-decision-boundaries.png) + +## Notebooks + +- [`linear_models_notebook.ipynb`](../linear-models/linear_models_notebook.ipynb) — original work +- [`linear_models_notebook_polished.ipynb`](../linear-models/linear_models_notebook_polished.ipynb) — AI-assisted documentation pass diff --git a/docs/MNIST-Classification.md b/docs/MNIST-Classification.md new file mode 100644 index 0000000..2449902 --- /dev/null +++ b/docs/MNIST-Classification.md @@ -0,0 +1,48 @@ +# MNIST Classification + +A full tour of classification problem types on the MNIST handwritten digit dataset — binary, multiclass, multilabel, and multioutput — with an emphasis on evaluating models honestly rather than trusting raw accuracy. + +## Why not just report accuracy? + +About 90% of MNIST digits are *not* an 8. A classifier that always guesses "not 8" scores ~90% accuracy while learning nothing. This project compares models with ROC AUC and F1 instead, and validates every "is it learning something real" claim against a `DummyClassifier` baseline. + +## Linear vs ensemble: SGD vs Random Forest + +Same binary task ("is it an 8?"), two very different models: + +![ROC curve comparison](images/clf-roc-comparison.png) + +| Model | F1 | ROC AUC | +|---|---|---| +| SGD (linear) | 0.58 | 0.915 | +| Random Forest (ensemble) | 0.90 | 0.997 | + +The gap illustrates a core limitation of linear models: SGD can only draw one straight decision boundary through 784-dimensional pixel space, while Random Forest's ensemble of trees carves non-linear regions that fit the actual shape of the data. + +## Where the multiclass model actually gets confused + +Extending to all 10 digits, the normalized confusion matrix shows exactly which digits get mistaken for which — more useful than a single accuracy number: + +![Multiclass confusion matrix](images/clf-confusion-matrix.png) + +## Multioutput: predicting an entire image + +Pushed the idea further — instead of predicting a label, predicting all 784 pixel values at once, framed as denoising a noisy digit: + + + + + + + + + + + + +
OriginalNoisy inputModel's denoised output
+ +## Notebooks + +- [`classification.ipynb`](../classification-project/classification.ipynb) — original work +- [`classification_notebook_polished.ipynb`](../classification-project/classification_notebook_polished.ipynb) — AI-assisted documentation pass diff --git a/docs/images/clf-confusion-matrix.png b/docs/images/clf-confusion-matrix.png new file mode 100644 index 0000000..6217f0e Binary files /dev/null and b/docs/images/clf-confusion-matrix.png differ diff --git a/docs/images/clf-denoise-noisy.png b/docs/images/clf-denoise-noisy.png new file mode 100644 index 0000000..989a7c3 Binary files /dev/null and b/docs/images/clf-denoise-noisy.png differ diff --git a/docs/images/clf-denoise-original.png b/docs/images/clf-denoise-original.png new file mode 100644 index 0000000..f739e4a Binary files /dev/null and b/docs/images/clf-denoise-original.png differ diff --git a/docs/images/clf-denoise-predicted.png b/docs/images/clf-denoise-predicted.png new file mode 100644 index 0000000..c33713e Binary files /dev/null and b/docs/images/clf-denoise-predicted.png differ diff --git a/docs/images/clf-roc-comparison.png b/docs/images/clf-roc-comparison.png new file mode 100644 index 0000000..80bdc2a Binary files /dev/null and b/docs/images/clf-roc-comparison.png differ diff --git a/docs/images/housing-actual-vs-predicted.png b/docs/images/housing-actual-vs-predicted.png new file mode 100644 index 0000000..2e5a31e Binary files /dev/null and b/docs/images/housing-actual-vs-predicted.png differ diff --git a/docs/images/housing-correlation.png b/docs/images/housing-correlation.png new file mode 100644 index 0000000..3da39fb Binary files /dev/null and b/docs/images/housing-correlation.png differ diff --git a/docs/images/housing-cv-comparison.png b/docs/images/housing-cv-comparison.png new file mode 100644 index 0000000..af08fd7 Binary files /dev/null and b/docs/images/housing-cv-comparison.png differ diff --git a/docs/images/housing-feature-importance.png b/docs/images/housing-feature-importance.png new file mode 100644 index 0000000..c9451e2 Binary files /dev/null and b/docs/images/housing-feature-importance.png differ diff --git a/docs/images/lin-decision-boundaries.png b/docs/images/lin-decision-boundaries.png new file mode 100644 index 0000000..ff0aa7b Binary files /dev/null and b/docs/images/lin-decision-boundaries.png differ diff --git a/docs/images/lin-gd-convergence.png b/docs/images/lin-gd-convergence.png new file mode 100644 index 0000000..779f564 Binary files /dev/null and b/docs/images/lin-gd-convergence.png differ diff --git a/docs/images/lin-learning-curve.png b/docs/images/lin-learning-curve.png new file mode 100644 index 0000000..2384781 Binary files /dev/null and b/docs/images/lin-learning-curve.png differ diff --git a/docs/images/lin-regularization.png b/docs/images/lin-regularization.png new file mode 100644 index 0000000..febee9d Binary files /dev/null and b/docs/images/lin-regularization.png differ