Function calling (also called tool use) is a mechanism where an LLM, instead of writing an answer in prose, returns a structured JSON object specifying which function to call and with what parameters. The calling application then executes the function and returns the result to the model. This eliminates the fragility of parsing free-form text and gives you guaranteed structured output for actions like database queries, API calls, and computations.
Why Function Calling Matters
Before function calling, the only way to get an LLM to take actions was to ask it to output JSON or XML and then parse that output. This worked inconsistently. The model might forget to close a bracket, add an explanation before the JSON, or use slightly different key names than expected.
Function calling solves this by making structured output a first-class feature of the API. When you define functions, the model knows exactly what format to use and that the output will be passed to executable code. Hallucination of function schemas drops dramatically when the model is explicitly operating in "tool use" mode versus trying to format free-form text as JSON.
The result: more reliable pipelines, fewer parsing errors, and the ability to build agent-style applications where the model decides which tool to use and when.
OpenAI Function Calling Syntax
With OpenAI's API, you define tools in the tools parameter:
const response = await openai.chat.completions.create({
model: "gpt-4o",
messages: [{ role: "user", content: "What is the weather in Paris?" }],
tools: [
{
type: "function",
function: {
name: "get_weather",
description: "Get current weather for a location",
parameters: {
type: "object",
properties: {
location: {
type: "string",
description: "City name, e.g. Paris, France",
},
unit: {
type: "string",
enum: ["celsius", "fahrenheit"],
},
},
required: ["location"],
},
},
},
],
});
const toolCall = response.choices[0].message.tool_calls?.[0];
if (toolCall) {
const args = JSON.parse(toolCall.function.arguments);
// args.location === "Paris, France"
// args.unit === undefined or "celsius"
}
Anthropic Tool Use Syntax
Anthropic uses a similar but slightly different structure:
const response = await anthropic.messages.create({
model: "claude-3-5-sonnet-20241022",
max_tokens: 1024,
tools: [
{
name: "get_weather",
description: "Get current weather for a location",
input_schema: {
type: "object",
properties: {
location: { type: "string" },
unit: { type: "string", enum: ["celsius", "fahrenheit"] },
},
required: ["location"],
},
},
],
messages: [{ role: "user", content: "What is the weather in Paris?" }],
});
const toolUseBlock = response.content.find((b) => b.type === "tool_use");
if (toolUseBlock && toolUseBlock.type === "tool_use") {
const args = toolUseBlock.input;
// Execute the function with args
}
The core concept is identical: define a schema, the model returns a structured call, you execute it and return the result.
Common Function Calling Patterns
Web Search
{
name: "search_web",
description: "Search the web for current information",
parameters: {
type: "object",
properties: {
query: { type: "string", description: "Search query" },
num_results: { type: "integer", default: 5 }
},
required: ["query"]
}
}
Database Query
{
name: "query_database",
description: "Run a SQL query against the production database",
parameters: {
type: "object",
properties: {
sql: { type: "string", description: "SELECT query only" },
limit: { type: "integer", maximum: 100 }
},
required: ["sql"]
}
}
Calculator
{
name: "calculate",
description: "Evaluate a mathematical expression accurately",
parameters: {
type: "object",
properties: {
expression: { type: "string", description: "Math expression, e.g. '(42 * 3.14) / 2'" }
},
required: ["expression"]
}
}
Error Handling
What happens when the model calls a function incorrectly? Two scenarios:
Schema violations: the API validates against your schema before returning. If the model tries to pass an unexpected parameter type, the API rejects it and asks the model to retry internally. You rarely see schema errors in practice.
Runtime errors: when your code executes the function and encounters an error (network failure, invalid data, etc.), you return the error to the model as the tool result. Well-designed prompts instruct the model how to handle errors. For example:
messages.push({
role: "tool",
tool_call_id: toolCall.id,
content: JSON.stringify({ error: "Database connection failed", code: "DB_ERROR" }),
});
The model will typically acknowledge the error and either retry, ask for clarification, or explain the problem to the user.
Parallel Function Calls
Modern models (GPT-4o, Claude 3.5) can call multiple functions simultaneously in a single response. If you ask "What is the weather in Paris and London?", the model may return two parallel tool calls rather than sequential calls.
// The response may contain multiple tool calls
const toolCalls = response.choices[0].message.tool_calls;
// Execute all in parallel
const results = await Promise.all(
toolCalls.map(async (call) => {
const args = JSON.parse(call.function.arguments);
const result = await executeFunction(call.function.name, args);
return { id: call.id, result };
})
);
Parallel calls significantly speed up multi-step agentic workflows.
Function Calling vs Structured Output Prompting
Function calling is not the only way to get structured output. You can also use:
- JSON mode (
response_format: { type: "json_object" }) for guaranteed JSON output - Structured outputs with strict schemas (OpenAI's newer feature)
- Prompt engineering ("Respond with only JSON, no other text")
When to use function calling over these alternatives:
- You need the model to decide which action to take (not just format output)
- You are building an agent that needs multiple rounds of tool use
- The function has side effects (writing to a database, making an API call)
When structured output prompting is simpler:
- You just need the model's answer in a specific JSON format
- There are no external actions to execute
- Single-turn extraction tasks where the schema is fixed
Keep Reading
- Streaming LLM Responses Guide — Streaming works differently with tool use
- LLM Embeddings Explained — Another core primitive for building LLM applications
- How Large Language Models Work: Complete Guide — The foundational understanding behind tool use
Pristren builds AI-powered software for teams. Zlyqor is our all-in-one workspace — chat, projects, time tracking, AI meeting summaries, and invoicing — in one tool. Try it free.
