Initial commit: MLP regularization, CNN BatchNorm comparison, autoencoder denoising

docs/MLP-Regularization.md

@@ -0,0 +1,41 @@
+# MLP Regularization — MNIST
+
+## Setup
+
+A baseline multilayer perceptron (`Dense(500) → Dense(100) → Dense(10)`) compared against the same architecture with `Dropout(0.2)` added after each hidden layer.
+
+## ⚠️ Two variables changed, not one
+
+Before looking at the results: the baseline uses SGD at `lr=0.001`, while the dropout run uses SGD at `lr=0.01` — a 10x higher learning rate, in addition to Dropout being added. This means the comparison below shows what happened between two different configurations, not the isolated effect of Dropout. A rigorous ablation would hold the learning rate constant. This is flagged directly rather than presenting a confounded result as clean evidence for Dropout's effect.
+
+## Training curves
+
+**Baseline:**
+
+![Baseline loss curve](images/mlp-baseline-loss.png)
+
+**With Dropout (and higher learning rate):**
+
+![Dropout loss curve](images/mlp-dropout-loss.png)
+
+## Where each model gets it wrong
+
+**Baseline misclassifications:**
+
+![Baseline misclassified digits](images/mlp-baseline-misclassified.png)
+
+**Dropout version misclassifications:**
+
+![Dropout misclassified digits](images/mlp-dropout-misclassified.png)
+
+## Results
+
+| Model | Test loss |
+|---|---|
+| Baseline (no dropout, lr=0.001) | 0.279 |
+| Dropout (0.2, lr=0.01) | 0.099 |
+
+## Notebooks
+
+- [`mnist-mlp-pred.ipynb`](../mnist-mlp-regularization/mnist-mlp-pred.ipynb)
+- [`mnist-mlp-pred-dropout.ipynb`](../mnist-mlp-regularization/mnist-mlp-pred-dropout.ipynb)