Langfuse: Open-Source LLM Observability You Can Self-Host

Updated
8 min read
Langfuse: Open-Source LLM Observability You Can Self-Host
TL;DR

Langfuse brings full tracing, prompt versioning, dataset evaluation, and cost attribution to LLM apps - and you can run the entire stack on your own servers.

The Observability Gap in LLM Apps

Traditional APM tools (Datadog, New Relic) capture HTTP latency and error rates. They tell you a request took 4 seconds - but not that 3.8 seconds was an LLM call, that the prompt had 1,200 tokens, that the model used was gpt-4o, or that the user-visible answer was a hallucination. Langfuse fills this gap with LLM-native observability.

Core Hierarchy

Trace → Span → Generation

  • A Trace represents one user interaction (e.g., one chat message)
  • Spans are logical steps within a trace (retrieve documents, format prompt, call LLM)
  • A Generation is a specific LLM call with input/output tokens, model name, latency, and cost

Python SDK Integration

bash
pip install langfuse

Decorator-Based Tracing

python
from langfuse.decorators import observe, langfuse_context
from openai import OpenAI

client = OpenAI()

@observe()
def retrieve_docs(query: str) -> str:
    # Simulate vector retrieval
    return f"Documents for: {query}"

@observe()
def generate_answer(docs: str, question: str) -> str:
    langfuse_context.update_current_observation(
        input={"docs": docs, "question": question}
    )
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": f"Context: {docs}"},
            {"role": "user", "content": question},
        ],
    )
    answer = response.choices[0].message.content
    langfuse_context.update_current_observation(output=answer)
    return answer

@observe()
def answer_question(question: str) -> str:
    docs = retrieve_docs(question)
    return generate_answer(docs, question)

result = answer_question("What is PagedAttention?")

Every @observe() call automatically creates a span in the trace. Langfuse captures function arguments as input and return values as output.

Prompt Management With Versioning

Store, version, and A/B test prompts in the Langfuse UI:

python
from langfuse import Langfuse

lf = Langfuse()
prompt = lf.get_prompt("answer-question", version=3)
compiled = prompt.compile(context="...", question="...")

Changing a prompt in production no longer requires a code deploy.

Dataset Creation From Production Traces

Mark any trace as a dataset item directly from the UI. Build ground-truth datasets from real user interactions, then run batch evaluations to compare prompt versions or models.

LLM-as-Judge Scoring

python
from langfuse import Langfuse

lf = Langfuse()
lf.score(
    trace_id="trace-xyz",
    name="faithfulness",
    value=0.92,
    comment="Answer matches source documents",
)

Automate this with an evaluator function that runs after each generation.

Self-Hosting With Docker

bash
git clone https://github.com/langfuse/langfuse.git
cd langfuse
docker compose up -d

The stack includes PostgreSQL, Redis, and the Langfuse web server. Full instructions in the self-host guide.

User-Level Cost Attribution

Pass user_id to attribute token costs to individual users - essential for multi-tenant SaaS billing:

python
langfuse_context.update_current_trace(user_id="user-456", session_id="session-789")

The dashboard then shows cost-per-user histograms and per-session token usage.

#langfuse#observability#tracing#open-source#self-hosted

One AI engineering post, weekly

LLM benchmarks, prompt techniques, and token-cost breakdowns, delivered to your inbox.

What should we test or explain next?

Comments

Have a question or something to add? Sign in to join the discussion.

Mahmudul Haque Qudrati — CEO & ML Engineer at Pristren

Mahmudul Haque Qudrati

CEO & ML Engineer

Software architect, full-stack engineer, and machine learning practitioner. Founder & CEO of Pristren, CTO at Venture Kick-backed PIN Sport Application GmbH (Basel, Switzerland), Software Development Lead at DailyFrame GmbH (building Dimetra patient transport software), and creator of Zlyqor. Focused on custom workflow automation, scalable web and mobile applications, and production AI/ML systems.