What Is Haystack?
Haystack is an open-source NLP framework by deepset designed specifically for building production retrieval-augmented generation (RAG) pipelines. Version 2.0, released in 2024, was a complete redesign that made the framework more modular and production-ready.
While LangChain and LlamaIndex are broad frameworks covering many LLM use cases, Haystack is narrower and deeper: it's built around the retrieval pipeline as the core abstraction, and it shows in the quality of the retrieval-specific tooling.
Component-Based Architecture
Every Haystack pipeline is a directed acyclic graph of components. Each component has typed inputs and outputs, making pipelines composable and testable in isolation.
The four key component types:
- DocumentStore — the vector database (Elasticsearch, OpenSearch, Weaviate, Qdrant, Chroma, in-memory)
- Retriever — fetches relevant documents (BM25, dense, hybrid)
- Ranker — re-ranks retrieved documents for relevance
- Generator — the LLM that produces the final answer
from haystack import Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.generators import OpenAIGenerator
from haystack.components.builders import RAGPromptBuilder
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
# ... index documents ...
pipeline = Pipeline()
pipeline.add_component("retriever", InMemoryBM25Retriever(document_store=document_store))
pipeline.add_component("prompt_builder", RAGPromptBuilder(template=prompt_template))
pipeline.add_component("llm", OpenAIGenerator(model="gpt-4o-mini"))
pipeline.connect("retriever", "prompt_builder.documents")
pipeline.connect("prompt_builder", "llm")
result = pipeline.run({"retriever": {"query": "What is RAG?"}})
YAML Pipeline Definition
Pipelines can be serialized to YAML for version control and deployment:
components:
retriever:
type: haystack.components.retrievers.in_memory.InMemoryBM25Retriever
init_parameters:
document_store: !component InMemoryDocumentStore
llm:
type: haystack.components.generators.OpenAIGenerator
init_parameters:
model: gpt-4o-mini
connections:
- sender: retriever.documents
receiver: prompt_builder.documents
Hybrid Retrieval
Haystack's hybrid retrieval combines BM25 (keyword) and dense (embedding) retrieval, then merges results with a reciprocal rank fusion algorithm. This consistently outperforms either method alone, especially for technical documents where both exact term matching and semantic understanding matter.
Evaluation Framework
Haystack includes a built-in evaluation framework with metrics for retrieval quality (MRR, MAP, NDCG) and generation quality (faithfulness, answer relevance). This lets you objectively measure whether a pipeline change actually improved performance.
deepset Cloud
For teams that want managed pipelines, deepset Cloud wraps Haystack in a hosted service with monitoring, A/B testing, and deployment management.
Haystack vs LangChain vs LlamaIndex
Haystack is more opinionated and production-focused. LangChain has a broader ecosystem and more integrations. LlamaIndex has deeper support for complex index structures. If your primary use case is a retrieval-heavy production pipeline, Haystack's evaluation framework and YAML-defined pipelines are a significant operational advantage.
