Nomic Embed: The First Fully Open 8192-Context Embedding Model
nomic-embed-text-v1.5 supports 8192-token context with Matryoshka embeddings at multiple dimensions - fully open training data, code, and weights under Apache 2.0.
Most popular embedding models - all-MiniLM-L6-v2, OpenAI text-embedding-3-small, BGE-base - have a maximum context of 512 tokens. For short sentences and paragraphs, this is fine. For embedding full documents, meeting transcripts, or long legal clauses, 512 tokens forces chunking strategies that break semantic context across boundaries.
nomic-embed-text-v1.5 raises that limit to 8192 tokens with no performance degradation on short-text tasks.
Matryoshka Embeddings
Traditional embedding models produce fixed-dimension vectors (e.g., 768-dim). Matryoshka Representation Learning (MRL) trains models so that the first N dimensions of a longer vector are themselves a high-quality embedding. nomic-embed-text supports 512, 256, and 128-dim slices from its 768-dim output:
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("nomic-ai/nomic-embed-text-v1.5", trust_remote_code=True)
# Full 768-dim embedding
texts = ["search_query: How does attention mechanism work in transformers?"]
embeddings_768 = model.encode(texts)
# 256-dim Matryoshka slice (truncate, then renormalize)
import numpy as np
embeddings_256 = embeddings_768[:, :256]
embeddings_256 = embeddings_256 / np.linalg.norm(embeddings_256, axis=1, keepdims=True)
print(f"Full embedding shape: {embeddings_768.shape}")
print(f"Compressed embedding shape: {embeddings_256.shape}")
Team workspace
Ship faster with chat, meetings, and projects in one place — Zlyqor.
nomic-embed uses prefixes to signal the embedding use case:
# For retrieval queries
query_embedding = model.encode(["search_query: What causes transformer models to hallucinate?"])
# For documents being indexed
doc_embedding = model.encode(["search_document: Hallucination in LLMs occurs when..."])
# For classification/clustering (no prefix)
cluster_embedding = model.encode(["Machine learning is a subset of artificial intelligence"])
MTEB Performance
nomic-embed-text-v1.5 achieves an MTEB (Massive Text Embedding Benchmark) average of 62.3% - competitive with OpenAI's text-embedding-3-small (62.3%) and substantially better than all-MiniLM-L6-v2 (56.3%). The key differentiator is the 8192 context at equivalent quality.
nomic-embed-vision
Nomic also released nomic-embed-vision, a CLIP-compatible image embedding model aligned to the same embedding space as nomic-embed-text. This means you can embed both images and text into the same vector space and do cross-modal retrieval without a separate model:
ollama pull nomic-embed-text
curl http://localhost:11434/api/embeddings -d '{
"model": "nomic-embed-text",
"prompt": "search_document: Your document text here"
}'
Full Openness Under Apache 2.0
Unlike most embedding models where the training data and pipeline are proprietary, nomic-embed releases training code, training data details, and weights under Apache 2.0 - enabling auditing, fine-tuning, and commercial deployment without restriction.
Practical deep-dives on LLMs, developer tools, and AI engineering. No filler. Unsubscribe any time.
// written byFIG. AUTH-01
530
Mahmudul Haque Qudrati
CEO & ML Engineer
CEO and ML Engineer at Pristren. Builds AI-powered software for teams and writes about machine learning, LLMs, developer tools, and practical AI applications.
