Function Calling in LLMs: How to Get Structured Actions From AI
Function calling gives LLMs a structured way to request execution of specific functions with typed parameters, eliminating the need to parse free-form text outputs.
Function calling (also called tool use) is a mechanism where an LLM, instead of writing an answer in prose, returns a structured JSON object specifying which function to call and with what parameters. The calling application then executes the function and returns the result to the model. This eliminates the fragility of parsing free-form text and gives you guaranteed structured output for actions like database queries, API calls, and computations.
Why Function Calling Matters
Before function calling, the only way to get an LLM to take actions was to ask it to output JSON or XML and then parse that output. This worked inconsistently. The model might forget to close a bracket, add an explanation before the JSON, or use slightly different key names than expected.
Function calling solves this by making structured output a first-class feature of the API. When you define functions, the model knows exactly what format to use and that the output will be passed to executable code. Hallucination of function schemas drops dramatically when the model is explicitly operating in "tool use" mode versus trying to format free-form text as JSON.
The result: more reliable pipelines, fewer parsing errors, and the ability to build agent-style applications where the model decides which tool to use and when.
OpenAI Function Calling Syntax
With OpenAI's API, you define tools in the tools parameter:
What happens when the model calls a function incorrectly? Two scenarios:
Schema violations: the API validates against your schema before returning. If the model tries to pass an unexpected parameter type, the API rejects it and asks the model to retry internally. You rarely see schema errors in practice.
Runtime errors: when your code executes the function and encounters an error (network failure, invalid data, etc.), you return the error to the model as the tool result. Well-designed prompts instruct the model how to handle errors. For example:
The model will typically acknowledge the error and either retry, ask for clarification, or explain the problem to the user.
Parallel Function Calls
Modern models (GPT-4o, Claude 3.5) can call multiple functions simultaneously in a single response. If you ask "What is the weather in Paris and London?", the model may return two parallel tool calls rather than sequential calls.
// The response may contain multiple tool calls
const toolCalls = response.choices[0].message.tool_calls;
// Execute all in parallel
const results = await Promise.all(
toolCalls.map(async (call) => {
const args = JSON.parse(call.function.arguments);
const result = await executeFunction(call.function.name, args);
return { id: call.id, result };
})
);
Parallel calls significantly speed up multi-step agentic workflows.
Function Calling vs Structured Output Prompting
Function calling is not the only way to get structured output. You can also use:
Pristren builds AI-powered software for teams. Zlyqor is our all-in-one workspace - chat, projects, time tracking, AI meeting summaries, and invoicing - in one tool. Try it free.
Practical deep-dives on LLMs, developer tools, and AI engineering. No filler. Unsubscribe any time.
// written byFIG. AUTH-01
530
Mahmudul Haque Qudrati
CEO & ML Engineer
CEO and ML Engineer at Pristren. Builds AI-powered software for teams and writes about machine learning, LLMs, developer tools, and practical AI applications.
Claude 3.5 Sonnet Review: What It Does Better Than GPT-4o (and Where It Falls Short)
An honest, benchmark-driven comparison of Claude 3.5 Sonnet vs GPT-4o covering coding, document analysis, multimodal tasks, pricing, and real-world verdict.
