🧠 Small VLM Benchmark

Vision-Language Models ≤ 4B parameters · Tested with llama.cpp · AI-assisted summary
S Top Tier A Strong B Good C Mediocre D Poor F Failing Tests: OCR (a digital table, English; and handwritten German text) · Image Description (with a badger, most tested (here) VLMs think that's a raccoon) · Triple question at once (My LLM abbreviated it as 3Q). 4 images in total

🏆 Top Tier (S–A)

Qwen3.5 4B 4B
S ⭐ Top Pick
OCR: ★★★★☆ Desc: ★★★★★ 3Q: 3/3
✓ Excellent OCR, warns about errors. May need an instruction to not generate HTML output. Good image caption and recognized the badger.
⚖️ Apache 2.0
Gemma4 E2B 4B
A
OCR: ★★☆☆☆ Desc: ★★★★★ 3Q: 3/3
✓ Great descriptions, answered my triple question. OCR was weak for me.
⚖️ Apache 2.0
Qwen3.5 2B 2B
A
OCR: ★★★☆☆ Desc: ★★★★★ 3Q: 3/3
✓ The triple question is answered.
⚖️ Apache 2.0
Ministral3 3B 3B
A
OCR: ★★★☆☆ Desc: ★★★★★ 3Q: 3/3
✓ Recognizes the badger. Answers all three questions.
⚖️ Apache 2.0
MiniCPM V 4.6 0.9B
A
OCR: ★★★★☆ Desc: ★★★★☆ 3Q: 3/3
✓ Small but mighty. All three questions answered.
⚖️ Apache 2.0
Youtu VL 4B 4B
A
OCR: ★★★★☆ Desc: ★★★★☆ 3Q: 3/3
✓ Answered the triple question.
⚖️ Custom
Granite Vision 4.1 4B 4B
B
OCR: ★★★★☆ Desc: ★★★★★ 3Q: 3/3
✓ Recognizes the badger. Note that it is a deriative of Granite Vision 4.0 3B.
⚖️ Apache 2.0

📊 Mid Tier (B–C)

InternVL3.5 2B 2B
B
OCR: ★★★☆☆ Desc: ★★★★☆ 3Q: 2.5–3/3
~ Might answer those 3 questions.
⚖️ Apache 2.0
h2ovl mississippi 800m 0.8B
B
OCR: ★☆☆☆☆ Desc: ★★★★☆ 3Q: 2–3/3
~ Hallucinates with the default temperature.
⚖️ Apache 2.0
h2ovl mississippi 2B 2B
B
OCR: ★★☆☆☆ Desc: ★★★★☆ 3Q: 2–3/3
~ Hallucinates with the default temperature.
⚖️ Apache 2.0
Ovis2 1B 1B
B
OCR: ★★★★☆ Desc: ★★★★☆ 3Q: 2/3
~ Good OCR for that size. Recognizes the badger. Third question went off-topic.
⚖️ Apache 2.0
Ovis2.5 2B 2B
B
OCR: ★★★★★ Desc: ★★★★★ 3Q: 2/3
~ Nice OCR. Recognizes the badger. Third question (about the non-existing dog) fails.
⚖️ Apache 2.0
Qwen3.5 0.8B 0.8B
B
OCR: ★★☆☆☆ Desc: ★★★★☆ 3Q: 2/3
~ Hallucinates. The triple partially and mostly fails.
⚖️ Apache 2.0
dots mocr 3B
B
OCR: ★★★☆☆ Desc: ★★★★☆ 3Q: 2.5–3/3
~ ✓ It recognized the badger.
⚠️ FAKE OPEN SOURCE RELEASE - not actually Apache 2.0 (why would they add it to their model card?), a subject to their restrictive terms of use.
FastVLM 0.5B 0.5B
C
OCR: ☆☆☆☆☆ Desc: ★★☆☆☆ 3Q: 2/3
✗ Failed to comply with the OCR task.
⚖️ Custom
FastVLM 1.5B 1.5B
C
OCR: ★☆☆☆☆ Desc: ★★★☆☆ 3Q: 2–3/3
✗ Just goes off-topic and spams.
⚖️ Custom

📉 Low Tier (D–F)

LFM 2.5 VL 450M 450M
D
OCR: ★☆☆☆☆ Desc: ★★★☆☆ 3Q: 0–1/3
✗ Really sucks at OCR. Fails the multi-question.
⚖️ Custom
LFM 2 VL 1.6B 1.6B
D
OCR: ★★☆☆☆ Desc: ★★★★☆ 3Q: 1/3
✗ Bad at OCR. Better descriptions of images, but it fails to answer the questions.
⚖️ Custom
MobileVLM V2 1.7B 1.7B
D
OCR: ☆☆☆☆☆ Desc: ★★☆☆☆ 3Q: 1/3
✗ Prompt ignoring. Too short captions (1–2 sentences).
⚖️ Apache 2.0
MobileVLM V2 3B 3B
D
OCR: ☆☆☆☆☆ Desc: ★★☆☆☆ 3Q: 1–2/3
✗ Slightly better. Prompt ignoring.
⚖️ Apache 2.0
moondream2 2025-06-21 2B
D
OCR: ★★☆☆☆ Desc: ★★★☆☆ 3Q: 1/3
✗ Bad OCR. Recognizes the badger. Fails the triple.
⚖️ Apache 2.0
SmolVLM2 256M Video 0.25B
D
OCR: ★☆☆☆☆ Desc: ★★☆☆☆ 3Q: 0/3
✗ My least favorite one, I hated how it described my images.
⚖️ Apache 2.0
SmolVLM2 500M Video 0.5B
D
OCR: ★☆☆☆☆ Desc: ★★★☆☆ 3Q: 0/3
✗ My least favorite one (x2).
⚖️ Apache 2.0
SmolVLM2 2.2B Instruct 2.2B
D
OCR: ★☆☆☆☆ Desc: ★☆☆☆☆ 3Q: 0/3
✗ Sucks at OCR. Hallucinates more than the 500M one.
⚖️ Apache 2.0
Heron NVILA Lite 2B 2B
D
OCR: ★★★☆☆ Desc: ★★★☆☆ 3Q: 0/3
✗ Passable OCR. Completely fails the triple.
⚖️ Apache 2.0
deepseek-vl-1.3b-chat 1.3B
D
OCR: ☆☆☆☆☆ Desc: ★★★☆☆ 3Q: 0/3
✗ Terrible OCR. Recognizes the badger. Only answers 1 question and ignores the rest.
⚖️ Custom
InternVL3.5 1B 1B
D
OCR: ★★☆☆☆ Desc: ★★★★☆ 3Q: 1–2/3
✗ Sucks at OCR. Couldn't answer all my questions.
⚖️ Apache 2.0
GLM edge v 2b 2B
D
OCR: ★☆☆☆☆ Desc: ★☆☆☆☆ 3Q: ⚠️
✗ Sucks overall due to misbehaving, and it feels like they forgot to add censorship into the 2B model (no refusals?). The 5B one isn't good either
⚖️ Custom

❌ Failed / Untested

Janus Pro 1B 1B
F
✗ Goes off-topic, hence it is useless.
⚖️ Apache 2.0
Mobile O 0.5B 0.5B
F
✗ Was unable to test it again. Image generation/edit throws an error for me. I only remember it sucked at OCR, could not even read single words properly.
⚖️ CC-BY-NC-4.0
Granite Vision 3 2B 2B
F
✗ Image inputs seem not to work at all.
⚖️ Apache 2.0
Cheers (by ai9stars) 3B
F
✗ Failed to get loaded.
⚖️ Apache 2.0
Perception-LM-1B 1B
F
✗ CUDA OOM. Gated repo (with their requirements), a bad license.
⚖️ Gated
deepseek-vl2-tiny 3B
F
✗ Untested, had no luck to do.
⚖️ Custom
kosmos-2.5-chat 1B
F
✗ Extremely low benchmark results from others. Not retested by me.
⚖️ MIT

📄 OCR-Only Models

Dedicated OCR models (not general VLMs)
MinerU2.5-Pro 1.2B 1.2B
Decent OCR, slight improvements for the 2605 version.
⚖️ Apache 2.0
smoldocling 256M 256M
I prefer it over Docling. Small errors.
⚖️ Apache 2.0
LightOnOCR 2 1B 1B
✓ Very nice OCR. Seemed to work better with vLLM rather than with llama.cpp.
⚖️ Apache 2.0
Docling 258M 258M
Not working with llama.cpp. Makes errors.
⚖️ Apache 2.0
GLM-OCR 1B 1B
✗ Hallucinates, likes including irrelevant Chinese characters.
⚖️ MIT
HunyuanOCR 1B 1B
✗ Makes errors and spams. Bad custom license.
⚖️ Hunyuan Community License
DeepSeek OCR 2 3B
✗ Hallucinations only (llama.cpp).
⚖️ Apache 2.0
Falcon OCR 0.3B 0.3B
✗ CUDA OOM. Was very slow when was able to run it.
⚖️ Apache 2.0

Summary

My Favorites: Qwen3.5, Gemma 4, Ovis 2, MiniCPM V 4.6

License Warnings: dots mocr (not actually Apache 2.0), HunyuanOCR (restrictive), Mobile O (CC-BY-NC)

Tests: OCR (German handwritten + English digital) · Image description (badger) · Triple question (claw color, potion location, dog presence - no dog)

This amateur benchmark conducted with llama.cpp and Transformers · 16-bit