Initial commit: housing regression, MNIST classification, linear models

.gitignore

@@ -0,0 +1,18 @@
+# Python
+__pycache__/
+*.py[cod]
+.venv/
+venv/
+
+# Jupyter
+.ipynb_checkpoints/
+
+# Model artifacts and data — regeneratable, don't belong in git
+*.pkl
+*.joblib
+datasets/
+*.csv
+
+# OS / editor
+.DS_Store
+.vscode/

README.md

@@ -0,0 +1,27 @@
+# Machine Learning Portfolio
+
+A collection of end-to-end machine learning projects covering regression, classification, and linear models built from first principles.
+
+![Python](https://img.shields.io/badge/Python-3776AB?style=flat&logo=python&logoColor=white)
+![scikit-learn](https://img.shields.io/badge/scikit--learn-F7931E?style=flat&logo=scikit-learn&logoColor=white)
+![pandas](https://img.shields.io/badge/pandas-150458?style=flat&logo=pandas&logoColor=white)
+![NumPy](https://img.shields.io/badge/NumPy-013243?style=flat&logo=numpy&logoColor=white)
+
+## Projects
+
+| Project | Task | Highlight |
+|---|---|---|
+| [housing-price-regression](housing-price-regression/) | Regression | Full pipeline (EDA → feature engineering → tuning → evaluation), test R² 0.72 |
+| [classification-project](classification-project/) | Classification | Binary/multiclass/multilabel/multioutput on MNIST, Random Forest ROC AUC 0.997 |
+| [linear-models](linear-models/) | Linear models | Normal equation, gradient descent, and regularization implemented from scratch |
+
+Each project folder has its own README with the specific approach, results, and notebook.
+
+## Tech stack
+
+Python, pandas, NumPy, scikit-learn, matplotlib, seaborn
+
+## Author
+
+**Reza** — Computer Networks student, LPIC-1
+[GitHub](https://github.com/rezayaghobi-dev)

projects/notebooks/classification-project/README.md

@@ -0,0 +1,36 @@
+# MNIST Classification — Binary, Multiclass, Multilabel & Multioutput
+
+A complete tour of classification approaches on the MNIST handwritten digit dataset (70,000 images, 28×28 pixels).
+
+## Overview
+
+Covers four distinct classification problem types on the same dataset, comparing models honestly with cross-validation rather than relying on raw accuracy (which is misleading on imbalanced classes).
+
+## Approach
+
+1. **Binary classification** ("is it an 8?") — SGD Classifier baseline, evaluated against a `DummyClassifier` to prove the model actually learned something beyond guessing the majority class.
+2. **Threshold analysis** — precision-recall and ROC curves to show the tradeoff between precision and recall as the decision threshold moves, instead of reporting one fixed operating point.
+3. **Model comparison** — SGD (linear) vs Random Forest (ensemble) on the same binary task.
+4. **Multiclass classification** — all 10 digits at once, with a 10×10 confusion matrix (normalized by row) to see which digits actually get confused with which.
+5. **Multilabel classification** — predicting multiple independent yes/no labels per sample simultaneously, including `ClassifierChain` to model dependence between labels.
+6. **Multioutput classification** — predicting all 784 pixel values at once (image denoising as a classification problem).
+
+## Results
+
+| Model / Task | Metric | Score |
+|---|---|---|
+| SGD (binary: "is it an 8?") | F1 | 0.58 |
+| SGD (binary) | ROC AUC | 0.915 |
+| Random Forest (binary) | F1 | 0.90 |
+| Random Forest (binary) | ROC AUC | 0.997 |
+| Multilabel (KNN) | Macro F1 | 0.978 |
+
+The SGD → Random Forest jump illustrates why a linear decision boundary struggles on raw pixel data compared to a tree ensemble that can carve non-linear regions.
+
+## Tech stack
+
+Python, scikit-learn, matplotlib
+
+## Notebook
+
+[`notebook.ipynb`](notebook.ipynb)

projects/notebooks/housing-price-regression/README.md

@@ -0,0 +1,34 @@
+# California Housing Price Prediction
+
+End-to-end supervised regression pipeline predicting median house value from census-block-level data.
+
+## Overview
+
+Full pipeline from raw data to a tuned, evaluated model: EDA, stratified train/test splitting, feature engineering, a leak-free sklearn preprocessing pipeline, baseline model comparison, cross-validation, hyperparameter tuning, and a final held-out evaluation with a confidence interval.
+
+## Approach
+
+1. **EDA** — explored feature distributions, geographic price spread, and correlation with the target. `median_income` was the strongest predictor (r ≈ 0.69).
+2. **Stratified split** — training/test split stratified on binned `median_income`, so both sets represent all income bands proportionally instead of relying on a lucky random draw.
+3. **Feature engineering** — derived ratio features (`rooms_per_house`, etc.) that are more semantically meaningful per-household than the raw per-block counts.
+4. **Preprocessing pipeline** — all transforms wrapped in sklearn `Pipeline`/`ColumnTransformer` to guarantee no data leakage and identical transforms on train/test.
+5. **Model comparison** — Linear Regression vs Decision Tree baselines, evaluated with 5-fold cross-validation (not just training accuracy).
+6. **Hyperparameter tuning** — `GridSearchCV` over tree depth and an RBF-kernel feature transform.
+7. **Final evaluation** — test set touched exactly once, after all modeling decisions were locked in.
+
+## Results
+
+| Metric | Value |
+|---|---|
+| MAE | $40,600 |
+| RMSE | $61,901 |
+| R² | 0.72 |
+| 95% CI on RMSE | [$59,141 — $64,542] |
+
+## Tech stack
+
+Python, pandas, scikit-learn, matplotlib, seaborn
+
+## Notebook
+
+[`notebook.ipynb`](notebook.ipynb)

projects/notebooks/linear-models/README.md

@@ -0,0 +1,26 @@
+# Linear Models — From Normal Equations to Logistic Regression
+
+A from-scratch deep dive into linear models, building each method from its mathematical foundation before confirming the result against scikit-learn.
+
+## Overview
+
+Rather than just calling `.fit()`, this notebook implements the underlying math directly (normal equation, gradient descent) to build intuition for what scikit-learn's estimators are actually doing internally.
+
+## Covered
+
+1. **Linear regression via the normal equation** — solving `θ = (XᵀX)⁻¹Xᵀy` directly, then verifying the result matches scikit-learn's `LinearRegression`.
+2. **Batch gradient descent** — implemented manually, with cost-function convergence visualized over iterations.
+3. **Stochastic gradient descent** — compared against batch GD for convergence speed and stability, then via scikit-learn's `SGDRegressor`.
+4. **Polynomial regression** — fitting non-linear (quadratic) data by engineering polynomial features.
+5. **Bias-variance tradeoff** — learning curves showing underfitting vs overfitting as model complexity changes.
+6. **Regularization** — Ridge, Lasso, and ElasticNet compared, with a visualization of how each shrinks coefficients differently as the penalty strength increases.
+7. **Logistic regression** — binary classification via the sigmoid function, with a visualized decision boundary.
+8. **Softmax regression** — multiclass classification on the Iris dataset, with visualized decision regions across all three species.
+
+## Tech stack
+
+Python, NumPy, scikit-learn, matplotlib
+
+## Notebook
+
+[`notebook.ipynb`](notebook.ipynb)