01 · Undergraduate Thesis · Computer Vision
How do CNNs and Vision Transformers actually arrive at a diagnosis?
Benchmarking Vision Transformers Against CNNs for Plant Disease Diagnosis: Performance, Robustness, and Visual Interpretability
Two models can reach similar accuracy while learning very different representations.
My undergraduate thesis is a computer vision study comparing ResNet-50 and ViT-B/16 inside a controlled three-stage pipeline (leaf localization, species classification, species-conditioned disease prediction) on roughly 54,000 leaf images across 13 species and 38 classes.
- YOLO leaf localization
- Species classifier
- ResNet-50 vs. ViT-B/16
- Grad-CAM · attention rollout
Evidence
- ViT-B/16 beat ResNet-50 at every stage: 93.8% vs. 92.7% on disease classification, 90.6% vs. 89.4% end to end.
- ViT attribution maps were 1.83 nats more diffuse (saliency entropy 5.67 vs. 3.84, p < 0.001, n = 1,100).
- The CNN put 71% of its attribution mass in the top 10% of pixels, against 49% for the ViT, and overlapped lesions more (IoU 0.61 vs. 0.44).
PyTorchCNNsVision TransformersYOLOGrad-CAMAttention RolloutInterpretabilityRepresentation LearningHPC