Llama 3.2 Vision: Meta's First Multimodal Open-Source Model

Updated
7 min read
Llama 3.2 Vision: Meta's First Multimodal Open-Source Model
TL;DR

Llama 3.2 introduces vision capability to the Llama family with 11B and 90B vision models, plus 1B and 3B text-only variants for on-device deployment.

Vision Comes to the Llama Family

Meta's Llama 3.2 is the first Llama release to include multimodal models. The lineup spans four sizes:

  • 1B - text-only, designed for on-device mobile inference
  • 3B - text-only, edge deployment with stronger reasoning
  • 11B - vision + text, single GPU friendly (24GB VRAM)
  • 90B - vision + text, highest capability, requires multi-GPU

The vision models use cross-attention layers added to the language backbone to process image features - a cleaner integration than adapter-based approaches.

MMMU Benchmark

The Massive Multidisciplinary Multimodal Understanding (MMMU) benchmark tests vision models on college-level questions across 30 subjects requiring both image understanding and domain knowledge:

Click any column to sort
4 rows
Llama 3.2 90B Vision60.3%
Llama 3.2 11B Vision50.7%
GPT-4o mini60.0%
Claude 3 Haiku50.2%

The 90B matches GPT-4o mini at a fraction of the API cost - or free if self-hosted.

Running With Ollama

bash
# 11B vision model  -  runs on RTX 4090 (24GB) or M2/M3 Max Mac
ollama pull llama3.2-vision:11b

# 90B vision model  -  requires 2-4x A100s
ollama pull llama3.2-vision:90b

# Text-only on-device variants
ollama pull llama3.2:1b
ollama pull llama3.2:3b

Using Vision Capabilities

python
import ollama

# Image analysis
response = ollama.chat(
    model="llama3.2-vision:11b",
    messages=[
        {
            "role": "user",
            "content": "What does this diagram show? Describe the data flow.",
            "images": ["path/to/architecture-diagram.png"]
        }
    ]
)
print(response["message"]["content"])

Document Understanding

Llama 3.2 Vision handles:

  • Scanned documents - extract text, tables, and structure from PDFs
  • Charts and graphs - read data values and describe trends
  • Screenshots - analyze UI, identify errors, extract information
  • Photographs - describe content, identify objects, read text
python
# Via HuggingFace transformers
from transformers import MllamaForConditionalGeneration, AutoProcessor
from PIL import Image

model = MllamaForConditionalGeneration.from_pretrained(
    "meta-llama/Llama-3.2-11B-Vision-Instruct",
    device_map="auto"
)
processor = AutoProcessor.from_pretrained("meta-llama/Llama-3.2-11B-Vision-Instruct")

image = Image.open("invoice.png")
inputs = processor(image, "Extract all line items and totals from this invoice.", return_tensors="pt")
output = model.generate(**inputs, max_new_tokens=512)
print(processor.decode(output[0]))

On-Device With 1B and 3B

The 1B and 3B text-only models are optimized for mobile via ExecuTorch (Meta's mobile inference framework). They fit in 500MB-1.5GB of device memory - practical for iOS and Android applications that need local inference without a network call.

Summary

Llama 3.2 brings competitive vision capability to the open-source ecosystem. The 11B vision model is particularly compelling: single-GPU, commercially licensed, and matching GPT-4o mini on MMMU. Get the weights at HuggingFace and read the release post at Meta AI.

#llama-3.2#meta#vision#multimodal#open-source

One AI engineering post, weekly

LLM benchmarks, prompt techniques, and token-cost breakdowns, delivered to your inbox.

What should we test or explain next?

Comments

Have a question or something to add? Sign in to join the discussion.

Mahmudul Haque Qudrati — CEO & ML Engineer at Pristren

Mahmudul Haque Qudrati

CEO & ML Engineer

Software architect, full-stack engineer, and machine learning practitioner. Founder & CEO of Pristren, CTO at Venture Kick-backed PIN Sport Application GmbH (Basel, Switzerland), Software Development Lead at DailyFrame GmbH (building Dimetra patient transport software), and creator of Zlyqor. Focused on custom workflow automation, scalable web and mobile applications, and production AI/ML systems.