Moondream2: A 1.9B VLM That Runs on a Raspberry Pi
Moondream2 is a 1.9B parameter vision-language model that fits in 1.2GB RAM when quantized, enabling image captioning, visual Q&A, and object detection on embedded hardware and edge devices.
Most vision-language models start at 7B parameters. LLaVA-7B requires 14GB VRAM in float16. For applications that need vision understanding without cloud API latency - embedded systems, mobile apps, privacy-sensitive processing - the model must fit in available RAM.
Moondream2 achieves 1.9B parameters through architectural choices that sacrifice breadth for efficiency: a smaller vision encoder, aggressive weight sharing, and training data focused on the most common vision-language tasks.
Hardware Requirements
In 4-bit quantization via GGUF format, Moondream2 requires 1.2GB RAM - enough to run on a Raspberry Pi 5 (8GB model), a mid-range smartphone, or any laptop with integrated graphics. Speed varies:
M2 MacBook Pro (CPU): ~3 seconds per image captioning
from transformers import AutoModelForCausalLM, AutoTokenizer
from PIL import Image
model = AutoModelForCausalLM.from_pretrained(
"vikhyatk/moondream2",
revision="2025-01-09",
trust_remote_code=True,
)
tokenizer = AutoTokenizer.from_pretrained("vikhyatk/moondream2", revision="2025-01-09")
image = Image.open("photo.jpg")
enc_image = model.encode_image(image)
# Image captioning
caption = model.answer_question(enc_image, "Describe this image.", tokenizer)
print(caption)
# Visual Q&A
answer = model.answer_question(enc_image, "How many people are in the image?", tokenizer)
print(answer)
# Object detection (returns bounding boxes)
objects = model.detect(image, "person")
print(objects) # [{"x_min": 0.2, "y_min": 0.1, "x_max": 0.5, "y_max": 0.9}, ...]
Moondream Server for Batch Inference
The Moondream GitHub includes a FastAPI server that batches image requests and caches vision encodings. For pipelines processing thousands of images, caching the encoded image representation (before the text generation step) reduces compute by ~60% when the same image is queried with multiple questions.
# Start the moondream server
pip install moondream
python -m moondream.server --model 2b-int8
Comparison to LLaVA-7B
Metric
Moondream2
LLaVA-7B
Parameters
1.9B
7B
RAM (4-bit)
1.2GB
4GB
Image captioning quality
Good
Better
Object detection
Built-in
Requires prompt tuning
Edge deployment
Yes
No (too slow)
VQA accuracy (VQAv2)
~74%
~80%
For edge deployments where LLaVA-7B is impractical, Moondream2 captures most of the value at a fraction of the resource cost. For server-side inference where quality is the priority, LLaVA-7B or larger models are preferable.
Practical deep-dives on LLMs, developer tools, and AI engineering. No filler. Unsubscribe any time.
// written byFIG. AUTH-01
538
Mahmudul Haque Qudrati
CEO & ML Engineer
CEO and ML Engineer at Pristren. Builds AI-powered software for teams and writes about machine learning, LLMs, developer tools, and practical AI applications.
