Toolformer: Teaching LLMs to Use Tools Without Human Annotations

Updated
9 min read
Toolformer: Teaching LLMs to Use Tools Without Human Annotations
TL;DR

Toolformer learns to call external APIs - calculators, search engines, calendars - by self-supervising on when API calls improve prediction, requiring no human-labeled examples of tool use.

The Annotation Bottleneck for Tool Use

Teaching an LLM to use tools effectively requires examples: (prompt, tool call, tool result, final answer) tuples. Collecting these by hand is expensive and limits which tools can be taught. Toolformer (arXiv:2302.04761) by Schick et al. at Meta AI proposes a self-supervised approach: let the model generate its own tool-use annotations, keep only the ones that actually help, and train on them.

The Five Tools

Toolformer learns to use five APIs:

  1. Calculator: arithmetic operations with exact results
  2. Wikipedia search: retrieve relevant passages for a query
  3. Question answering system: answer factual questions
  4. Calendar: get the current date/time
  5. Machine translation: translate text between languages

Each tool call is represented as a special token sequence: [API_NAME(args) -> result]. For example: "The flight takes [Calculator(3 * 60 + 45) -> 225] 225 minutes."

Self-Supervised Data Generation

The pipeline:

  1. Sample positions: For each text document in the training corpus, identify positions where a tool call might help (using the model's own uncertainty as a signal).

  2. Propose API calls: Prompt the model to suggest what tool call would be useful at each position, with a few-shot example to guide the format.

  3. Filter on utility: Execute each proposed API call. Keep the call only if including the result in the context reduces the model's perplexity on the subsequent text more than a threshold. This ensures only genuinely helpful calls are kept.

  4. Fine-tune: Train the base language model on the filtered dataset with real API results inserted.

python
def filter_api_calls(model, text, position, api_call, api_result, threshold=1.0):
    """
    Keep an API call only if it reduces perplexity on the continuation.
    """
    continuation = text[position:]

    # Perplexity without API call
    base_context = text[:position]
    ppl_without = compute_perplexity(model, base_context, continuation)

    # Perplexity with API call and result inserted
    api_context = text[:position] + f"[{api_call} -> {api_result}]"
    ppl_with = compute_perplexity(model, api_context, continuation)

    # Keep if reduction exceeds threshold
    return (ppl_without - ppl_with) > threshold

def compute_perplexity(model, context, text):
    input_ids = tokenizer(context + text, return_tensors="pt").input_ids
    context_len = len(tokenizer(context).input_ids)
    with torch.no_grad():
        loss = model(input_ids, labels=input_ids).loss
    return torch.exp(loss).item()

Why It Generalizes Better Than Few-Shot

Few-shot prompting for tool use works at inference time but does not internalize tool use into the model weights. The model must parse the few-shot format at every call and can be confused by prompts that differ from the examples. Toolformer bakes tool use into the model - it knows when to call a calculator as a natural part of text generation.

Limitations vs Modern Function Calling

Toolformer was demonstrated on GPT-J (6B) - a relatively small model by 2026 standards. It handles five fixed tools versus the arbitrary function calling in GPT-4, Claude, and Gemini. Modern function calling (OpenAI function calling, Claude tool use) uses RLHF-based training on much larger models with schema-defined tools, producing more reliable and generalizable tool use. But Toolformer's self-supervised principle - use perplexity reduction to filter genuine tool utility - influenced the data generation strategies for modern function-calling training.

Results

Toolformer substantially outperforms GPT-J baseline on tasks requiring calculation, fact retrieval, and temporal reasoning. A 6.7B Toolformer model matches or exceeds much larger models (OPT-66B, GPT-3) on QA and math benchmarks where tool use is allowed.

Further Reading

#toolformer#tool-use#self-supervised#apis#meta-ai

One AI engineering post, weekly

LLM benchmarks, prompt techniques, and token-cost breakdowns, delivered to your inbox.

What should we test or explain next?

Comments

Have a question or something to add? Sign in to join the discussion.

Mahmudul Haque Qudrati — CEO & ML Engineer at Pristren

Mahmudul Haque Qudrati

CEO & ML Engineer

Software architect, full-stack engineer, and machine learning practitioner. Founder & CEO of Pristren, CTO at Venture Kick-backed PIN Sport Application GmbH (Basel, Switzerland), Software Development Lead at DailyFrame GmbH (building Dimetra patient transport software), and creator of Zlyqor. Focused on custom workflow automation, scalable web and mobile applications, and production AI/ML systems.