34 lines
1.3 KiB
Markdown
34 lines
1.3 KiB
Markdown
# CNN BatchNorm Comparison — MNIST
|
|
|
|
Comparing a baseline CNN against the same architecture with Batch Normalization added, on MNIST digit classification.
|
|
|
|
## Architecture
|
|
|
|
```
|
|
Conv2D(32, 3x3, stride 2) → Conv2D(64, 3x3, stride 2) → Flatten → Dense(10, softmax)
|
|
```
|
|
|
|
The BatchNorm version inserts `BatchNormalization` + separate `Activation('relu')` after each conv layer, instead of the activation being baked into the conv layer directly.
|
|
|
|
## Clean ablation
|
|
|
|
Both models use the **identical optimizer and learning rate** (SGD, lr=0.001) — Batch Normalization is the only variable that changed. This makes the result directly attributable to BatchNorm, unlike the [MLP regularization comparison](../mnist-mlp-regularization/README.md) in this repo where two variables changed at once.
|
|
|
|
## Results
|
|
|
|
| Model | Test loss (categorical crossentropy) |
|
|
|---|---|
|
|
| Baseline CNN | 0.289 |
|
|
| CNN + BatchNorm | 0.094 |
|
|
|
|
BatchNorm alone (same optimizer, same learning rate) cut test loss by roughly two-thirds — consistent with its known effect of stabilizing and accelerating convergence, especially under plain SGD at a conservative learning rate.
|
|
|
|
## Tech stack
|
|
|
|
TensorFlow / Keras, NumPy, matplotlib
|
|
|
|
## Notebooks
|
|
|
|
- [`conv-pred.ipynb`](conv-pred.ipynb) — baseline
|
|
- [`conv-pred-BN.ipynb`](conv-pred-BN.ipynb) — with BatchNorm
|