The Problem With Raw LLM Outputs
LLMs produce text that can be malformed, unsafe, or structurally incorrect. In production apps you need to handle: outputs that are not valid JSON when you expected JSON, responses that contain PII (names, emails, phone numbers), toxic language in customer-facing chatbots, and hallucinated content that violates business rules. Guardrails AI centralises this validation logic.
Installation
pip install guardrails-ai
guardrails hub install hub://guardrails/valid_python # Example hub validator
Core Concepts
- Guard: wraps an LLM call with validators
- Validator: a function that checks output and returns pass/fail
- on_fail: action when validation fails (
reask,refrain,fix,exception) - RAIL spec: XML-based schema (legacy) — Pydantic is preferred in 2026
Basic Guard With Pydantic
from guardrails import Guard
from guardrails.hub import TwoWords
from pydantic import BaseModel
class ProductSummary(BaseModel):
name: str
one_line_description: str
guard = Guard.from_pydantic(ProductSummary)
response = guard(
openai.chat.completions.create,
prompt="Describe vLLM in one line.",
model="gpt-4o-mini",
max_tokens=128,
)
print(response.validated_output) # ProductSummary(name='vLLM', one_line_description='...')
If the model returns malformed output, Guard sends the Pydantic validation error back as a reask prompt and retries.
PII Detection Validator
guardrails hub install hub://guardrails/detect_pii
from guardrails import Guard
from guardrails.hub import DetectPII
guard = Guard().use(DetectPII(pii_entities=["EMAIL_ADDRESS", "PHONE_NUMBER"], on_fail="refrain"))
response = guard(
openai.chat.completions.create,
prompt="Summarise this support ticket: John called from john@acme.com about billing.",
model="gpt-4o-mini",
)
# If PII is detected in the output, response.validated_output = None (refrain)
Toxicity Validator
guardrails hub install hub://guardrails/toxic_language
from guardrails.hub import ToxicLanguage
guard = Guard().use(ToxicLanguage(threshold=0.5, on_fail="exception"))
try:
response = guard(openai.chat.completions.create, prompt="...", model="gpt-4o-mini")
except Exception as e:
print(f"Toxicity detected: {e}")
Custom Validator
from guardrails.validators import Validator, register_validator, PassResult, FailResult
@register_validator(name="no-code-snippets", data_type="string")
class NoCodeSnippets(Validator):
def validate(self, value: str, metadata: dict):
if "```" in value or "def " in value:
return FailResult(
error_message="Response must not contain code snippets.",
fix_value=value.split("```")[0].strip(),
)
return PassResult()
guard = Guard().use(NoCodeSnippets(on_fail="fix"))
on_fail Actions
| Action | Behaviour |
|---|---|
| reask | Send validation error back to LLM and retry |
| refrain | Return None (silently skip bad output) |
| fix | Apply the fix_value from the validator |
| exception | Raise ValidationError |
| noop | Log but return the invalid output |
Server Mode for Microservices
guardrails start --config config.py --port 8000
Call the validation server from any language via HTTP — useful when your LLM app is not Python.
Full documentation at guardrailsai.com/docs and the Hub validator marketplace.
