What Is Haystack?
Haystack is an open-source NLP framework by deepset designed specifically for building production retrieval-augmented generation (RAG) pipelines. Version 2.0, released in 2024, was a complete redesign that made the framework more modular and production-ready.
While LangChain and LlamaIndex are broad frameworks covering many LLM use cases, Haystack is narrower and deeper: it's built around the retrieval pipeline as the core abstraction, and it shows in the quality of the retrieval-specific tooling.
Component-Based Architecture
Every Haystack pipeline is a directed acyclic graph of components. Each component has typed inputs and outputs, making pipelines composable and testable in isolation.
The four key component types:
- DocumentStore - the vector database (Elasticsearch, OpenSearch, Weaviate, Qdrant, Chroma, in-memory)
- Retriever - fetches relevant documents (BM25, dense, hybrid)
- Ranker - re-ranks retrieved documents for relevance
- Generator - the LLM that produces the final answer
from haystack import Pipeline
from haystack.components.retrievers.in_memory import InMemoryBM25Retriever
from haystack.components.generators import OpenAIGenerator
from haystack.components.builders import RAGPromptBuilder
from haystack.document_stores.in_memory import InMemoryDocumentStore
document_store = InMemoryDocumentStore()
# ... index documents ...
pipeline = Pipeline()
pipeline.add_component("retriever", InMemoryBM25Retriever(document_store=document_store))
pipeline.add_component("prompt_builder", RAGPromptBuilder(template=prompt_template))
pipeline.add_component("llm", OpenAIGenerator(model="gpt-4o-mini"))
pipeline.connect("retriever", "prompt_builder.documents")
pipeline.connect("prompt_builder", "llm")
result = pipeline.run({"retriever": {"query": "What is RAG?"}})
