Langfuse: Open-Source LLM Observability You Can Self-Host

The Observability Gap in LLM Apps

Traditional APM tools (Datadog, New Relic) capture HTTP latency and error rates. They tell you a request took 4 seconds - but not that 3.8 seconds was an LLM call, that the prompt had 1,200 tokens, that the model used was gpt-4o, or that the user-visible answer was a hallucination. Langfuse fills this gap with LLM-native observability.

Core Hierarchy

Trace → Span → Generation

• A Trace represents one user interaction (e.g., one chat message)
• Spans are logical steps within a trace (retrieve documents, format prompt, call LLM)
• A Generation is a specific LLM call with input/output tokens, model name, latency, and cost

Python SDK Integration

pip install langfuse

Decorator-Based Tracing

from langfuse.decorators import observe, langfuse_context
from openai import OpenAI

client = OpenAI()

@observe()
def retrieve_docs(query: str) -> str:
    # Simulate vector retrieval
    return f"Documents for: {query}"

@observe()
def generate_answer(docs: str, question: str) -> str:
    langfuse_context.update_current_observation(
        input={"docs": docs, "question": question}
    )
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": f"Context: {docs}"},
            {"role": "user", "content": question},
        ],
    )
    answer = response.choices[0].message.content
    langfuse_context.update_current_observation(output=answer)
    return answer

@observe()
def answer_question(question: str) -> str:
    docs = retrieve_docs(question)
    return generate_answer(docs, question)

result = answer_question("What is PagedAttention?")

Every @observe() call automatically creates a span in the trace. Langfuse captures function arguments as input and return values as output.

Prompt Management With Versioning

Store, version, and A/B test prompts in the Langfuse UI:

from langfuse import Langfuse

lf = Langfuse()
prompt = lf.get_prompt("answer-question", version=3)
compiled = prompt.compile(context="...", question="...")

Changing a prompt in production no longer requires a code deploy.

Dataset Creation From Production Traces

Mark any trace as a dataset item directly from the UI. Build ground-truth datasets from real user interactions, then run batch evaluations to compare prompt versions or models.

LLM-as-Judge Scoring

from langfuse import Langfuse

lf = Langfuse()
lf.score(
    trace_id="trace-xyz",
    name="faithfulness",
    value=0.92,
    comment="Answer matches source documents",
)

Automate this with an evaluator function that runs after each generation.

Self-Hosting With Docker

git clone https://github.com/langfuse/langfuse.git
cd langfuse
docker compose up -d

The stack includes PostgreSQL, Redis, and the Langfuse web server. Full instructions in the self-host guide.

User-Level Cost Attribution

Pass user_id to attribute token costs to individual users - essential for multi-tenant SaaS billing:

langfuse_context.update_current_trace(user_id="user-456", session_id="session-789")

The dashboard then shows cost-per-user histograms and per-session token usage.