Weights & Biases for LLM Fine-Tuning: Track Every Run and Compare Results
W&B provides experiment tracking for fine-tuning runs and LLM tracing via Weave, letting you compare models, trace agent calls, and manage models in a production registry.
Mahmudul Haque Qudrati
CEO & ML Engineer
April 22, 2026
8 min read
One AI engineering post, weekly
LLM benchmarks, prompt techniques, and token-cost breakdowns — not another AI news roundup.
Fine-tuning is iterative. You run with lr=2e-4, then try 5e-4. You experiment with 100 examples, then 500. Without tracking, you lose the context of what you tried and why something worked. Weights & Biases (W&B) captures every hyperparameter, metric, and artifact automatically.
Integrating with HuggingFace Trainer
The simplest integration requires one line:
from transformers import TrainingArguments
training_args = TrainingArguments(
output_dir="./outputs",
num_train_epochs=3,
per_device_train_batch_size=4,
learning_rate=2e-4,
report_to="wandb", # this one line
run_name="llama3-qlora-v2",
)
W&B automatically logs training loss, validation loss, learning rate schedule, gradient norms, and system metrics (GPU utilization, memory). No other code changes needed.
Team workspace
Ship faster with chat, meetings, and projects in one place — Zlyqor.
Direct Logging
For custom training loops or logging additional metrics:
import wandb
wandb.init(
project="llm-fine-tuning",
config={
"model": "meta-llama/Meta-Llama-3-8B",
"lora_r": 16,
"dataset_size": 500,
"learning_rate": 2e-4,
}
)
for step, batch in enumerate(train_loader):
loss = train_step(batch)
wandb.log({"train/loss": loss, "train/step": step})
wandb.finish()
W&B Weave for LLM Tracing
Weave is W&B's LLM observability layer. It automatically instruments OpenAI and Anthropic API calls when you add two lines:
import weave
from openai import OpenAI
weave.init("my-llm-app") # that's it - OpenAI calls are now traced
client = OpenAI()
response = client.chat.completions.create(
model="gpt-4o",
messages=[{"role": "user", "content": "Hello"}]
)
Every call is logged with model, input, output, latency, token counts, and cost. For agent workflows, you see the full call tree showing how sub-calls compose.
Comparing Fine-Tuned vs Base Model
Weave's Evaluation feature lets you run both models on the same dataset and compare outputs side-by-side with automatic metrics:
import weave
@weave.op()
def evaluate_model(model_name: str, prompt: str) -> str:
# call your model and return the output
...
evaluation = weave.Evaluation(
dataset=test_cases,
scorers=[accuracy_scorer, format_scorer],
)
asyncio.run(evaluation.evaluate(base_model_predict))
asyncio.run(evaluation.evaluate(finetuned_model_predict))
Model Registry
After fine-tuning, register your model artifact for production:
artifact = wandb.Artifact("llama3-customer-support", type="model")
artifact.add_dir("./outputs/final_model")
wandb.log_artifact(artifact)
# Link to registry for deployment tracking
artifact.link("my-org/model-registry/llama3-customer-support", aliases=["v2", "production"])
Resources
#weights-&-biases#mlops#fine-tuning#experiment-tracking#weave
Mahmudul Haque Qudrati
CEO & ML Engineer
Visionary leader with extensive experience in machine learning and software development. Drives strategic innovation and business growth.
More from Mahmudul
GPT-6 Astra AGI: How to Actually Use It and Get Its Best Performance
8 min read
DeepSeek vs MiniMax vs Kimi in Coding Benchmarks: Real Scores, Real Prices, Real Tradeoffs
8 min read
GPT-6 Astra Benchmarks, Pricing, and Safety: What Developers Need to Know
5 min read
Claude Fable 5.1: Same Price, 75% Cheaper Cache, and Real Tradeoffs
4 min read
Related Articles
6 min readHow to Use Claude to Make Videos Like Vox and Others
Claude can help you make Vox-style videos by generating scripts, editing with code, and automating animation. Here's a practical guide with real workflows and costs.
5 min readOpenAI Ends Cursor Model Access on Nov 12, 2026: What Developers Need to Know
OpenAI will terminate Cursor's access to its models on November 12, 2026, following SpaceX's acquisition. This guide explains the timeline, why it happened, and practical steps to migrate your workflow.
3 min readOx Alpha That Became GLM 5.3 Flash: From Mystery to Preview, Everything We Know
Ox Alpha, the anonymous AI model that topped coding benchmarks, turned out to be GLM-5.3-Flash from Zhipu. Here's the full story, from mystery to official preview, with evidence and practical details.
// discussion
Comments