MLP Regularization — MNIST

Comparing a baseline multilayer perceptron against a Dropout-regularized version on MNIST digit classification.

Architecture

Both models share the same base architecture:

Dense(500, relu) → Dense(100, relu) → Dense(10, softmax)

The dropout version adds Dropout(0.2) after each hidden layer.

⚠️ Honest note on this comparison

This is not a clean ablation — two things changed between the baseline and dropout runs, not one:

	Baseline	Dropout version
Dropout	None	0.2 after each hidden layer
Optimizer	SGD, lr=0.001	SGD, lr=0.01

The learning rate is 10x higher in the dropout run. That means the test loss improvement below can't be attributed to Dropout alone — the higher learning rate is a confound. Both notebooks are included as-is because they reflect real experimentation, but the results should be read as "these two configurations produced these two outcomes," not as proof that Dropout alone caused the improvement. A clean ablation would hold the learning rate fixed and vary only Dropout.

Results

Model	Test loss (categorical crossentropy)
Baseline (no dropout, lr=0.001)	0.279
Dropout (0.2, lr=0.01)	0.099

Tech stack

TensorFlow / Keras, NumPy, matplotlib

Notebooks

mnist-mlp-pred.ipynb — baseline
mnist-mlp-pred-dropout.ipynb — with Dropout

1.4 KiB Raw Blame History