Blog
Latest articles
5 min readAnthropic Message Batches API: 50% Off Claude for Async Workloads
Anthropic's Message Batches API gives you 50% off Claude pricing for requests that can wait up to 24 hours. Here is how to use it and which Claude workloads benefit most.
9 min readLocal LLM vs. API: When Running Models Yourself Actually Saves Money
A GPU server costs $300-800/month. At low query volume, API access is cheaper. At high volume, local wins. Here is the break-even analysis with real numbers.
8 min readHow to Reduce LLM Output Tokens by 40-60% Without Losing Quality
Output tokens cost 3-6x more than input tokens. Specific prompt instructions and format choices can cut output length by 40-60% for the same information, with a direct impact on your bill.
9 min readLLM Rate Limiting and Cost Controls: How to Prevent Runaway API Bills
Runaway LLM bills happen without rate limits and budget alerts. Here is how to implement per-user limits, global budget controls, and circuit breakers that protect your margins.
9 min readThe Cost-Per-Task Framework: How to Actually Measure AI ROI
Tracking API spend alone tells you nothing about ROI. The right metric is cost per meaningful task - and comparing it to the non-AI cost of doing the same work.
8 min readHow Much to Budget for LLM API Costs at Each Startup Stage
LLM costs scale from $0-50/month at pre-product to $500-5,000/month at growth stage. Here is what to expect, where to optimize, and the rule of thumb that keeps AI spend sustainable.
5 min readGroq vs. Together AI vs. Fireworks AI: Fast LLM Inference Compared
Three fast, cheap inference platforms for open source LLMs. Groq is the fastest, Together AI has the broadest model selection, Fireworks specializes in production-grade function calling.
8 min readQuantization Explained: How to Run LLMs 4x Cheaper With Minimal Quality Loss
Quantization reduces model weight precision from FP32 to INT4, cutting memory and compute by 4-8x. Q4_K_M is the sweet spot for most use cases - near full quality at a fraction of the size.
9 min readFlash Attention Explained: The Engineering Trick Behind Long-Context LLMs
Flash Attention rewrites transformer attention to be IO-aware, reducing memory from O(n²) to O(n). It enables 128k context windows and cuts training costs by 2-4x. Here is how it works.
13 min readHow to Evaluate LLMs: Benchmarks, Vibes, and Building Your Own Evals
Benchmarks are gamed and vibes do not scale. Here is how to build real evaluations that tell you whether an LLM actually works for your specific use case.
8 min readMMLU, HumanEval, and Chatbot Arena Explained: What AI Benchmarks Actually Measure
A plain-English explanation of every major LLM benchmark: what each one tests, how it scores, and what a 1% difference actually means in practice.
8 min readLM-as-Judge: Using LLMs to Evaluate LLM Outputs
LM-as-judge works well for relative preference ranking but breaks down for absolute quality scores. Here is how to set it up and avoid the major failure modes.