DL-portfolio/docs/CNN-BatchNorm.md

CNN BatchNorm Comparison — MNIST

Setup

A baseline CNN (Conv2D(32) → Conv2D(64) → Flatten → Dense(10, softmax)) compared against the identical architecture with BatchNormalization inserted after each conv layer. Both models use the same optimizer and learning rate (SGD, lr=0.001) — BatchNorm is the only variable that changed, making this a clean ablation.

Training curves

Baseline CNN:

CNN + BatchNorm:

The BatchNorm version's curve drops faster and settles lower — visible directly in the training dynamics, not just the final number.

Results

Model	Test loss
Baseline CNN	0.289
CNN + BatchNorm	0.094

Since the optimizer and learning rate were held constant, this ~3x reduction in test loss is attributable to BatchNormalization itself — consistent with its known role in stabilizing gradients and accelerating convergence.

Notebooks

• conv-pred.ipynb
• conv-pred-BN.ipynb