32 lines
1.1 KiB
Markdown
32 lines
1.1 KiB
Markdown
# CNN BatchNorm Comparison — MNIST
|
|
|
|
## Setup
|
|
|
|
A baseline CNN (`Conv2D(32) → Conv2D(64) → Flatten → Dense(10, softmax)`) compared against the identical architecture with BatchNormalization inserted after each conv layer. Both models use the same optimizer and learning rate (SGD, lr=0.001) — BatchNorm is the only variable that changed, making this a clean ablation.
|
|
|
|
## Training curves
|
|
|
|
**Baseline CNN:**
|
|
|
|
|
|
|
|
**CNN + BatchNorm:**
|
|
|
|
|
|
|
|
The BatchNorm version's curve drops faster and settles lower — visible directly in the training dynamics, not just the final number.
|
|
|
|
## Results
|
|
|
|
| Model | Test loss |
|
|
|---|---|
|
|
| Baseline CNN | 0.289 |
|
|
| CNN + BatchNorm | 0.094 |
|
|
|
|
Since the optimizer and learning rate were held constant, this ~3x reduction in test loss is attributable to BatchNormalization itself — consistent with its known role in stabilizing gradients and accelerating convergence.
|
|
|
|
## Notebooks
|
|
|
|
- [`conv-pred.ipynb`](../cnn-batchnorm-comparison/conv-pred.ipynb)
|
|
- [`conv-pred-BN.ipynb`](../cnn-batchnorm-comparison/conv-pred-BN.ipynb)
|