49 lines
2.1 KiB
Markdown
49 lines
2.1 KiB
Markdown
# MNIST Classification
|
|
|
|
A full tour of classification problem types on the MNIST handwritten digit dataset — binary, multiclass, multilabel, and multioutput — with an emphasis on evaluating models honestly rather than trusting raw accuracy.
|
|
|
|
## Why not just report accuracy?
|
|
|
|
About 90% of MNIST digits are *not* an 8. A classifier that always guesses "not 8" scores ~90% accuracy while learning nothing. This project compares models with ROC AUC and F1 instead, and validates every "is it learning something real" claim against a `DummyClassifier` baseline.
|
|
|
|
## Linear vs ensemble: SGD vs Random Forest
|
|
|
|
Same binary task ("is it an 8?"), two very different models:
|
|
|
|
|
|
|
|
| Model | F1 | ROC AUC |
|
|
|---|---|---|
|
|
| SGD (linear) | 0.58 | 0.915 |
|
|
| Random Forest (ensemble) | 0.90 | 0.997 |
|
|
|
|
The gap illustrates a core limitation of linear models: SGD can only draw one straight decision boundary through 784-dimensional pixel space, while Random Forest's ensemble of trees carves non-linear regions that fit the actual shape of the data.
|
|
|
|
## Where the multiclass model actually gets confused
|
|
|
|
Extending to all 10 digits, the normalized confusion matrix shows exactly which digits get mistaken for which — more useful than a single accuracy number:
|
|
|
|
|
|
|
|
## Multioutput: predicting an entire image
|
|
|
|
Pushed the idea further — instead of predicting a label, predicting all 784 pixel values at once, framed as denoising a noisy digit:
|
|
|
|
<table>
|
|
<tr>
|
|
<td><img src="images/clf-denoise-original.png" width="200"/></td>
|
|
<td><img src="images/clf-denoise-noisy.png" width="200"/></td>
|
|
<td><img src="images/clf-denoise-predicted.png" width="200"/></td>
|
|
</tr>
|
|
<tr>
|
|
<td align="center">Original</td>
|
|
<td align="center">Noisy input</td>
|
|
<td align="center">Model's denoised output</td>
|
|
</tr>
|
|
</table>
|
|
|
|
## Notebooks
|
|
|
|
- [`classification.ipynb`](../classification-project/classification.ipynb) — original work
|
|
- [`classification_notebook_polished.ipynb`](../classification-project/classification_notebook_polished.ipynb) — AI-assisted documentation pass
|