Function calling in LLMs enables large language models to interact deterministically with external software systems, databases, internal APIs, and enterprise microservices. Rather than relying on fragile prompt engineering or raw text generation, function calling converts natural language user requests into strictly validated JSON arguments.
This engineering guide examines how function calling operates under the hood, cross-provider implementations across OpenAI, Anthropic Claude, and Google Gemini, schema generation with Pydantic and Zod, parallel execution patterns, self-correction error loops, and defense against indirect tool injection attacks.
How Function Calling Operates Under the Hood
Large language models are inherently probability engines generating sequences of tokens. In early agent architectures, developers instructed models to output raw JSON strings wrapped in markdown fences. This approach frequently failed due to missing closing brackets, hallucinated keys, and unescaped quotes.
Modern function calling solves this architectural weakness through constrained sampling and grammar compilation:
- Schema Registration: The client transmits a JSON Schema detailing available function names, descriptions, parameter names, types, and required fields.
- Context Encoding: The inference engine encodes function signatures into internal system context.
- Grammar Constrained Decoding: When the model decides to invoke a tool, token generation is constrained strictly to the allowed JSON grammar of the matching parameter schema.
- Tool Call Emission: The model emits a dedicated tool call block containing the target function name and validated JSON arguments.
- Client-Side Execution: Your application inspects the call, executes the corresponding native function in your backend environment, and collects the returned output.
- Synthesis Turn: The application feeds the tool execution result back to the model, which synthesizes a natural language response for the user.
[ User Request ] ──► [ Model Evaluates Tool Signatures ]
│
▼
[ Model Emits Tool Call: { name, args } ]
│
▼
[ Backend Executes Native Function ]
│
▼
[ Return Tool Result to Model ]
│
▼
[ Final Synthesized Response ]
Cross-Provider Implementations Side by Side
While the architectural pattern remains consistent, the major AI providers expose distinct SDK conventions and parameter naming patterns.
OpenAI Function Calling in Node.js and TypeScript
The OpenAI platform uses the tools parameter array, supporting both single and parallel invocations. Complete technical specifications are detailed in the official OpenAI Function Calling Guide.
import OpenAI from "openai";
const openai = new OpenAI();
const tools: OpenAI.ChatCompletionTool[] = [
{
type: "function",
function: {
name: "get_weather_forecast",
description: "Retrieve meteorological forecast for a specified geographical location",
parameters: {
type: "object",
properties: {
location: { type: "string", description: "City and state or country" },
units: { type: "string", enum: ["celsius", "fahrenheit"] }
},
required: ["location", "units"],
additionalProperties: false
},
strict: true
}
}
];
async function runOpenAIToolFlow(userQuery: string) {
const messages: OpenAI.ChatCompletionMessageParam[] = [
{ role: "user", content: userQuery }
];
const response = await openai.chat.completions.create({
model: "gpt-4o",
messages,
tools,
tool_choice: "auto"
});
const responseMessage = response.choices[0].message;
if (responseMessage.tool_calls && responseMessage.tool_calls.length > 0) {
for (const toolCall of responseMessage.tool_calls) {
if (toolCall.type === "function") {
const functionName = toolCall.function.name;
const functionArgs = JSON.parse(toolCall.function.arguments);
console.log(`Calling ${functionName} with arguments:`, functionArgs);
// Execute local function
const toolResult = { temperature: 22, conditions: "Clear sky", location: functionArgs.location };
messages.push(responseMessage);
messages.push({
role: "tool",
tool_call_id: toolCall.id,
content: JSON.stringify(toolResult)
});
}
}
const finalResponse = await openai.chat.completions.create({
model: "gpt-4o",
messages
});
console.log("Final Answer:", finalResponse.choices[0].message.content);
}
}
runOpenAIToolFlow("What is the current weather forecast for Tokyo in celsius?");
For complete cost optimization across OpenAI endpoints, review our OpenAI API and Batch Processing Guide.
Anthropic Claude Tool Use in Python
Anthropic Claude natively supports tool use via the Messages API. The model signals tool requests by setting stop_reason="tool_use" and delivering structured content blocks, as documented in the Claude Tool Use Overview.
import anthropic
client = anthropic.Anthropic()
tools = [
{
"name": "query_database_inventory",
"description": "Query real-time stock levels for warehouse product SKUs",
"input_schema": {
"type": "object",
"properties": {
"sku_id": {"type": "string", "description": "Warehouse SKU identifier"},
"warehouse_region": {"type": "string", "enum": ["us-east", "us-west", "eu-central"]}
},
"required": ["sku_id", "warehouse_region"]
}
}
]
messages = [
{"role": "user", "content": "How many units of SKU-8842 remain in the eu-central warehouse?"}
]
response = client.messages.create(
model="claude-3-5-sonnet-latest",
max_tokens=1024,
tools=tools,
messages=messages
)
if response.stop_reason == "tool_use":
tool_block = next(b for b in response.content if b.type == "tool_use")
print(f"Executing Tool: {tool_block.name}")
print(f"Tool Arguments: {tool_block.input}")
# Mock database retrieval
database_payload = {"sku_id": "SKU-8842", "available_stock": 420, "reserved": 15}
follow_up = client.messages.create(
model="claude-3-5-sonnet-latest",
max_tokens=1024,
tools=tools,
messages=[
{"role": "user", "content": "How many units of SKU-8842 remain in the eu-central warehouse?"},
{"role": "assistant", "content": response.content},
{
"role": "user",
"content": [
{
"type": "tool_result",
"tool_use_id": tool_block.id,
"content": str(database_payload)
}
]
}
]
)
print("Claude Response:", follow_up.content[0].text)
To combine tool use with batch discounts and caching, review our Anthropic Claude API and Message Batches Guide.
Google Gemini Function Calling in Python
Google Gemini accepts native Python function declarations or JSON schema specifications directly into the client tools list, as documented in the Gemini Function Calling Documentation.
from google import genai
from google.genai import types
client = genai.Client()
def get_stock_ticker_quote(ticker: str) -> dict:
"""Retrieve financial market pricing for a given equity ticker symbol.
Args:
ticker: The stock ticker uppercase symbol e.g. NVDA, GOOG
"""
return {"ticker": ticker, "current_price": 142.50, "currency": "USD"}
response = client.models.generate_content(
model="gemini-2.0-flash",
contents="Check the current equity market price for ticker GOOG.",
config=types.GenerateContentConfig(
tools=[get_stock_ticker_quote]
)
)
if response.function_calls:
for call in response.function_calls:
print(f"Gemini requested function: {call.name} with args: {call.args}")
Schema Generation with Pydantic and Zod
Handwriting raw JSON Schemas is error-prone and leads to drifting documentation. Industry standards rely on schema generation libraries that infer types directly from validated data structures.
Python Schema Extraction with Pydantic
Using Pydantic Models guarantees runtime validation while producing compliant JSON Schema definitions:
from typing import List, Optional
from pydantic import BaseModel, Field
class FilterCriterion(BaseModel):
field_name: str = Field(description="Database column identifier")
operator: str = Field(description="Comparison operator e.g. eq, gt, lt, in")
value: str = Field(description="Target comparison scalar value")
class DatabaseQueryPlan(BaseModel):
collection: str = Field(description="Target database collection or table")
filters: List[FilterCriterion] = Field(description="List of applied query filters")
limit: int = Field(default=50, description="Maximum records to retrieve")
sort_by: Optional[str] = Field(default=None, description="Optional column to order by")
# Export schema definition for model tools parameter
tool_definition = {
"name": "execute_database_query",
"description": "Execute structured queries against enterprise datastores",
"parameters": DatabaseQueryPlan.model_json_schema()
}
TypeScript Schema Extraction with Zod
In Node.js and TypeScript environments, Zod Validation enables zero-redundancy schema generation:
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
const IncidentTicketSchema = z.object({
ticket_id: z.string().describe("Unique identifier for the incident ticket"),
severity: z.enum(["SEV1", "SEV2", "SEV3"]).describe("Incident severity classification"),
assigned_team: z.string().describe("Engineering team responsible for resolution"),
escalate_to_pager: z.boolean().describe("Whether on-call rotation should be alerted")
});
const jsonSchema = zodToJsonSchema(IncidentTicketSchema, "IncidentTicket");
Tool Choice Policies and Forcing Execution
Applications often require strict control over whether a model is permitted to answer conversationally or forced to invoke a specific tool.
OpenAI Tool Choice Configurations
tool_choice: "auto"The model decides autonomously between generating text or executing one or more tools.tool_choice: "none"Disables tool execution, forcing text output.tool_choice: "required"Mandates that the model must execute at least one tool before generating a conversational response.tool_choice: { type: "function", function: { name: "target_func" } }Forces execution of an exact specified tool.
Anthropic Claude Tool Choice Configurations
tool_choice: {"type": "auto"}Default behavior allowing Claude to choose.tool_choice: {"type": "any"}Forces Claude to invoke at least one tool.tool_choice: {"type": "tool", "name": "target_func"}Forces Claude to execute the designated tool name.
Parallel Tool Execution Patterns
When a user query spans multiple entities (for instance: "Compare server load across us-east, us-west, and eu-west"), models can return multiple tool calls simultaneously.
Executing parallel tool calls sequentially introduces unnecessary latency. Use asynchronous concurrency primitives to dispatch all tools in parallel:
import OpenAI from "openai";
const openai = new OpenAI();
async function handleParallelToolCalls(toolCalls: OpenAI.ChatCompletionMessageToolCall[]) {
// Execute all tool calls concurrently across microservices
const executionPromises = toolCalls.map(async (call) => {
if (call.type === "function") {
const args = JSON.parse(call.function.arguments);
const result = await fetchRegionalServerLoad(args.region);
return {
role: "tool" as const,
tool_call_id: call.id,
content: JSON.stringify(result)
};
}
throw new Error("Unsupported tool call type");
});
const toolResponseMessages = await Promise.all(executionPromises);
return toolResponseMessages;
}
To scale concurrent agent workflows across complex pipelines, review our architectural guide on AI Agents Production State and Reliability and AutoGen Multi-Agent Framework.
Defensive Error Handling and Self-Correction Loops
External functions fail due to network timeouts, invalid arguments, database deadlocks, or permission rejections. Crashing the agent loop on tool exceptions degrades user experience.
Instead of terminating execution, feed the error back into the model context. Language models possess strong self-correction capabilities when provided with informative error messages.
[ Model Calls Tool with Invalid Parameter ]
│
▼
[ Tool Execution Fails: "Invalid date format, expected YYYY-MM-DD" ]
│
▼
[ Send Error Message as Tool Result ]
│
▼
[ Model Self-Corrects and Emits Valid Tool Call ]
Implementing a Self-Correction Loop in Python
def execute_with_self_correction(client, messages, tools, max_attempts=3):
attempts = 0
while attempts < max_attempts:
response = client.chat.completions.create(
model="gpt-4o",
messages=messages,
tools=tools
)
message = response.choices[0].message
if not message.tool_calls:
return message.content
messages.append(message)
for tool_call in message.tool_calls:
try:
args = json.loads(tool_call.function.arguments)
result = run_native_tool(tool_call.function.name, args)
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": json.dumps({"status": "success", "data": result})
})
except Exception as error:
attempts += 1
messages.append({
"role": "tool",
"tool_call_id": tool_call.id,
"content": json.dumps({"status": "error", "message": str(error)})
})
print(f"Tool error encountered. Triggering self-correction turn {attempts}...")
Security Architecture and Defending Against Tool Injection Attacks
Exposing tools to language models creates a direct bridge between untrusted text inputs and executable backend code. Indirect prompt injection attacks occur when malicious instructions embedded inside untrusted data (such as emails, PDFs, or web scraping targets) hijack model reasoning.
Implement defense-in-depth principles:
- Parameterize SQL and Shell Commands: Never pass model-generated arguments directly into string concatenations, SQL statements, or shell subprocesses. Treat every tool parameter as untrusted user input.
- Principle of Least Privilege: Scope tool API credentials to minimal read-only permissions where possible. Isolate destructive tools (deleting databases, sending emails, initiating payments) behind dedicated permission gates.
- Human in the Loop for Irreversible Actions: Require explicit user confirmation before executing tools with financial, destructive, or state-altering side effects.
- Input Schema Whitelisting: Use strict JSON schema validation to reject undeclared properties, disallowing arbitrary payload tampering.
Production Deployment Checklist
Before enabling function calling across production microservices, verify that your implementation satisfies this engineering checklist:
- Validated Schemas: Confirm all tools use schemas generated from Pydantic or Zod models with explicit property descriptions.
- Strict Mode Enabled: Where supported by the provider, enable strict schema compliance to eliminate hallucinated JSON structures.
- Concurrent Execution: Implement Promise.all or asyncio.gather to dispatch parallel tool calls simultaneously.
- Resilient Error Reflection: Wrap all native tool executions in try-catch blocks and reflect caught error descriptions back to the model for self-correction.
- Least Privilege Credentials: Isolate tool execution tokens and prevent destructive commands without explicit human approval.
- Telemetry and Auditing: Log every function invocation name, parsed argument, turnaround duration, and response status to your observability dashboard.



Comments
Have a question or something to add? Sign in to join the discussion.