CNN BatchNorm Comparison — MNIST

Comparing a baseline CNN against the same architecture with Batch Normalization added, on MNIST digit classification.

Architecture

Conv2D(32, 3x3, stride 2) → Conv2D(64, 3x3, stride 2) → Flatten → Dense(10, softmax)

The BatchNorm version inserts BatchNormalization + separate Activation('relu') after each conv layer, instead of the activation being baked into the conv layer directly.

Clean ablation

Both models use the identical optimizer and learning rate (SGD, lr=0.001) — Batch Normalization is the only variable that changed. This makes the result directly attributable to BatchNorm, unlike the MLP regularization comparison in this repo where two variables changed at once.

Results

Model	Test loss (categorical crossentropy)
Baseline CNN	0.289
CNN + BatchNorm	0.094

BatchNorm alone (same optimizer, same learning rate) cut test loss by roughly two-thirds — consistent with its known effect of stabilizing and accelerating convergence, especially under plain SGD at a conservative learning rate.

Tech stack

TensorFlow / Keras, NumPy, matplotlib

Notebooks

conv-pred.ipynb — baseline
conv-pred-BN.ipynb — with BatchNorm

1.3 KiB Raw Permalink Blame History