Why Chroma for Prototyping
Chroma is the fastest way to add a vector store to a Python LLM app. It runs in-memory or on-disk with a single import - no Docker, no server, no external dependencies. When you are ready for production, flip to client-server mode with the same API.
Installation
pip install chromadbIn-Memory Mode
import chromadb
client = chromadb.Client() # ephemeral, in-memory
collection = client.create_collection("docs")
collection.add(
documents=["PagedAttention manages KV cache as pages.", "HNSW is a graph ANN index."],
ids=["doc1", "doc2"],
)
results = collection.query(
query_texts=["how does LLM memory work?"],
n_results=2,
)
print(results["documents"])Chroma uses all-MiniLM-L6-v2 (via the chromadb default embedding function) automatically - you do not need to manage embeddings yourself.
Persistent Mode
client = chromadb.PersistentClient(path="./chroma_db")Data persists across process restarts in the specified directory. This is sufficient for single-server production deployments with millions of documents.
Metadata Filtering
collection.add(
documents=["LangGraph is a state machine framework.", "Instructor adds Pydantic to LLMs."],
metadatas=[{"category": "agents"}, {"category": "tooling"}],
ids=["doc3", "doc4"],
)
results = collection.query(
query_texts=["build an agent"],
where={"category": "agents"},
n_results=1,
)The where filter uses MongoDB-style operators: $eq, $ne, $gt, $in, $and, $or.
Custom Embedding Function
Use any embedding model by wrapping it:
from chromadb import EmbeddingFunction
from sentence_transformers import SentenceTransformer
class LocalEmbedder(EmbeddingFunction):
def __init__(self):
self.model = SentenceTransformer("all-mpnet-base-v2")
def __call__(self, input: list[str]) -> list[list[float]]:
return self.model.encode(input).tolist()
collection = client.create_collection("docs", embedding_function=LocalEmbedder())LangChain Integration
pip install langchain-chromafrom langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings
vectorstore = Chroma(
collection_name="docs",
embedding_function=OpenAIEmbeddings(),
persist_directory="./chroma_db",
)
retriever = vectorstore.as_retriever(search_kwargs={"k": 4})Client-Server Mode for Production
chroma run --host 0.0.0.0 --port 8000Connect from your app:
client = chromadb.HttpClient(host="localhost", port=8000)Chroma vs Qdrant vs Pinecone
| Self-host | Yes | Yes | No |
| In-process | Yes | No | No |
| Hybrid search | No | Yes | Yes |
| Production scale | Medium | High | High |
Full documentation at docs.trychroma.com.




Comments
Have a question or something to add? Sign in to join the discussion.