Initial commit: MLP regularization, CNN BatchNorm comparison, autoencoder denoising

.gitignore

@@ -0,0 +1,20 @@
+# Python
+__pycache__/
+*.py[cod]
+.venv/
+venv/
+
+# Jupyter
+.ipynb_checkpoints/
+
+# Datasets and model artifacts — regeneratable, don't belong in git
+datasets/
+*.h5
+*.keras
+*.pb
+*.ckpt
+*.weights
+
+# OS / editor
+.DS_Store
+.vscode/

README.md

@@ -0,0 +1,28 @@
+# Deep Learning Portfolio
+
+A collection of deep learning experiments built with TensorFlow/Keras — regularization and normalization comparisons, and an autoencoder for image denoising.
+
+![TensorFlow](https://img.shields.io/badge/TensorFlow-FF6F00?style=flat&logo=tensorflow&logoColor=white)
+![Keras](https://img.shields.io/badge/Keras-D00000?style=flat&logo=keras&logoColor=white)
+![Python](https://img.shields.io/badge/Python-3776AB?style=flat&logo=python&logoColor=white)
+
+📖 **Full write-ups with result plots:** [docs/Home.md](docs/Home.md)
+
+## Projects
+
+| Project | Comparison | Result |
+|---|---|---|
+| [mnist-mlp-regularization](mnist-mlp-regularization/) | Baseline MLP vs Dropout | Test loss 0.279 → 0.099 *(not a clean ablation — see project README)* |
+| [cnn-batchnorm-comparison](cnn-batchnorm-comparison/) | Baseline CNN vs BatchNorm | Test loss 0.289 → 0.094 (clean ablation) |
+| [autoencoder-denoising](autoencoder-denoising/) | Noisy → clean image reconstruction | Recognizable digits recovered through a 32-unit bottleneck |
+
+Each project folder has its own README with architecture, training config, and results. The `docs/` folder has deeper write-ups with the actual result plots.
+
+## Tech stack
+
+Python, TensorFlow, Keras, NumPy, matplotlib
+
+## Author
+
+**Reza** — Computer Networks student, LPIC-1
+[GitHub](https://github.com/rezayaghobi-dev)

autoencoder-denoising/README.md

@@ -0,0 +1,33 @@
+# Autoencoder Image Denoising — MNIST
+
+A simple fully-connected autoencoder trained to reconstruct clean MNIST digits from noisy versions of them.
+
+## Architecture
+
+```
+Input(784) → Dense(32, relu)  [encoder/bottleneck]
+           → Dense(784, sigmoid)  [decoder]
+```
+
+Gaussian noise is added to the input images; the network is trained to output the *clean* version, using binary crossentropy loss between the reconstruction and the original image.
+
+## Approach
+
+1. Load MNIST, normalize pixel values to [0, 1]
+2. Add Gaussian noise (factor 0.5) to create noisy training inputs
+3. Train the autoencoder to map noisy → clean, compressing through a 32-unit bottleneck
+4. Evaluate by visually comparing original, noisy, and reconstructed images
+
+## Result
+
+![Denoising result](autoencoder-denoising-result.png)
+
+Despite the bottleneck compressing each 784-pixel image down to just 32 values, the network recovers a recognizable digit — showing the encoder is learning a meaningful compressed representation rather than just copying pixels through.
+
+## Tech stack
+
+TensorFlow / Keras, NumPy, matplotlib
+
+## Notebook
+
+[`encoder-noisy-image.ipynb`](encoder-noisy-image.ipynb)

cnn-batchnorm-comparison/README.md

@@ -0,0 +1,33 @@
+# CNN BatchNorm Comparison — MNIST
+
+Comparing a baseline CNN against the same architecture with Batch Normalization added, on MNIST digit classification.
+
+## Architecture
+
+```
+Conv2D(32, 3x3, stride 2) → Conv2D(64, 3x3, stride 2) → Flatten → Dense(10, softmax)
+```
+
+The BatchNorm version inserts `BatchNormalization` + separate `Activation('relu')` after each conv layer, instead of the activation being baked into the conv layer directly.
+
+## Clean ablation
+
+Both models use the **identical optimizer and learning rate** (SGD, lr=0.001) — Batch Normalization is the only variable that changed. This makes the result directly attributable to BatchNorm, unlike the [MLP regularization comparison](../mnist-mlp-regularization/README.md) in this repo where two variables changed at once.
+
+## Results
+
+| Model | Test loss (categorical crossentropy) |
+|---|---|
+| Baseline CNN | 0.289 |
+| CNN + BatchNorm | 0.094 |
+
+BatchNorm alone (same optimizer, same learning rate) cut test loss by roughly two-thirds — consistent with its known effect of stabilizing and accelerating convergence, especially under plain SGD at a conservative learning rate.
+
+## Tech stack
+
+TensorFlow / Keras, NumPy, matplotlib
+
+## Notebooks
+
+- [`conv-pred.ipynb`](conv-pred.ipynb) — baseline
+- [`conv-pred-BN.ipynb`](conv-pred-BN.ipynb) — with BatchNorm

docs/Autoencoder-Denoising.md

@@ -0,0 +1,19 @@
+# Autoencoder Image Denoising — MNIST
+
+## Setup
+
+A minimal fully-connected autoencoder: `Input(784) → Dense(32, relu) → Dense(784, sigmoid)`. Gaussian noise (factor 0.5) is added to MNIST digits, and the network is trained to reconstruct the *original clean* image from the noisy input — binary crossentropy loss between reconstruction and ground truth.
+
+## Why a 32-unit bottleneck matters
+
+Compressing a 784-pixel image down to 32 values and then reconstructing it forces the network to learn which features actually matter for representing a digit, rather than memorizing pixels. If the reconstruction still looks like a recognizable digit after that bottleneck, it's evidence the encoder learned a genuinely useful compressed representation.
+
+## Result
+
+![Original, noisy, and denoised digits](images/autoencoder-denoising-result.png)
+
+Despite the aggressive compression (784 → 32 → 784), the reconstructions are clearly recognizable digits — the noise is substantially removed and the core digit shape is preserved.
+
+## Notebook
+
+[`encoder-noisy-image.ipynb`](../autoencoder-denoising/encoder-noisy-image.ipynb)

docs/CNN-BatchNorm.md

@@ -0,0 +1,31 @@
+# CNN BatchNorm Comparison — MNIST
+
+## Setup
+
+A baseline CNN (`Conv2D(32) → Conv2D(64) → Flatten → Dense(10, softmax)`) compared against the identical architecture with BatchNormalization inserted after each conv layer. Both models use the same optimizer and learning rate (SGD, lr=0.001) — BatchNorm is the only variable that changed, making this a clean ablation.
+
+## Training curves
+
+**Baseline CNN:**
+
+![Baseline CNN loss curve](images/cnn-baseline-loss.png)
+
+**CNN + BatchNorm:**
+
+![BatchNorm CNN loss curve](images/cnn-bn-loss.png)
+
+The BatchNorm version's curve drops faster and settles lower — visible directly in the training dynamics, not just the final number.
+
+## Results
+
+| Model | Test loss |
+|---|---|
+| Baseline CNN | 0.289 |
+| CNN + BatchNorm | 0.094 |
+
+Since the optimizer and learning rate were held constant, this ~3x reduction in test loss is attributable to BatchNormalization itself — consistent with its known role in stabilizing gradients and accelerating convergence.
+
+## Notebooks
+
+- [`conv-pred.ipynb`](../cnn-batchnorm-comparison/conv-pred.ipynb)
+- [`conv-pred-BN.ipynb`](../cnn-batchnorm-comparison/conv-pred-BN.ipynb)

docs/Home.md

@@ -0,0 +1,9 @@
+# Deep Learning Portfolio — Documentation
+
+Deep dives into each experiment: architecture, training configuration, and results — illustrated with real plots from the notebooks.
+
+## Projects
+
+- [MLP Regularization](MLP-Regularization.md) — Dropout comparison (with an honest note on a confounded variable)
+- [CNN BatchNorm Comparison](CNN-BatchNorm.md) — clean ablation, BatchNorm's effect isolated
+- [Autoencoder Denoising](Autoencoder-Denoising.md) — noisy-to-clean image reconstruction through a compressed bottleneck

docs/MLP-Regularization.md

@@ -0,0 +1,41 @@
+# MLP Regularization — MNIST
+
+## Setup
+
+A baseline multilayer perceptron (`Dense(500) → Dense(100) → Dense(10)`) compared against the same architecture with `Dropout(0.2)` added after each hidden layer.
+
+## ⚠️ Two variables changed, not one
+
+Before looking at the results: the baseline uses SGD at `lr=0.001`, while the dropout run uses SGD at `lr=0.01` — a 10x higher learning rate, in addition to Dropout being added. This means the comparison below shows what happened between two different configurations, not the isolated effect of Dropout. A rigorous ablation would hold the learning rate constant. This is flagged directly rather than presenting a confounded result as clean evidence for Dropout's effect.
+
+## Training curves
+
+**Baseline:**
+
+![Baseline loss curve](images/mlp-baseline-loss.png)
+
+**With Dropout (and higher learning rate):**
+
+![Dropout loss curve](images/mlp-dropout-loss.png)
+
+## Where each model gets it wrong
+
+**Baseline misclassifications:**
+
+![Baseline misclassified digits](images/mlp-baseline-misclassified.png)
+
+**Dropout version misclassifications:**
+
+![Dropout misclassified digits](images/mlp-dropout-misclassified.png)
+
+## Results
+
+| Model | Test loss |
+|---|---|
+| Baseline (no dropout, lr=0.001) | 0.279 |
+| Dropout (0.2, lr=0.01) | 0.099 |
+
+## Notebooks
+
+- [`mnist-mlp-pred.ipynb`](../mnist-mlp-regularization/mnist-mlp-pred.ipynb)
+- [`mnist-mlp-pred-dropout.ipynb`](../mnist-mlp-regularization/mnist-mlp-pred-dropout.ipynb)

mnist-mlp-regularization/README.md

@@ -0,0 +1,40 @@
+# MLP Regularization — MNIST
+
+Comparing a baseline multilayer perceptron against a Dropout-regularized version on MNIST digit classification.
+
+## Architecture
+
+Both models share the same base architecture:
+
+```
+Dense(500, relu) → Dense(100, relu) → Dense(10, softmax)
+```
+
+The dropout version adds `Dropout(0.2)` after each hidden layer.
+
+## ⚠️ Honest note on this comparison
+
+This is **not a clean ablation** — two things changed between the baseline and dropout runs, not one:
+
+| | Baseline | Dropout version |
+|---|---|---|
+| Dropout | None | 0.2 after each hidden layer |
+| Optimizer | SGD, lr=0.001 | SGD, lr=0.01 |
+
+The learning rate is 10x higher in the dropout run. That means the test loss improvement below **can't be attributed to Dropout alone** — the higher learning rate is a confound. Both notebooks are included as-is because they reflect real experimentation, but the results should be read as "these two configurations produced these two outcomes," not as proof that Dropout alone caused the improvement. A clean ablation would hold the learning rate fixed and vary only Dropout.
+
+## Results
+
+| Model | Test loss (categorical crossentropy) |
+|---|---|
+| Baseline (no dropout, lr=0.001) | 0.279 |
+| Dropout (0.2, lr=0.01) | 0.099 |
+
+## Tech stack
+
+TensorFlow / Keras, NumPy, matplotlib
+
+## Notebooks
+
+- [`mnist-mlp-pred.ipynb`](mnist-mlp-pred.ipynb) — baseline
+- [`mnist-mlp-pred-dropout.ipynb`](mnist-mlp-pred-dropout.ipynb) — with Dropout