Function Calling in LLMs: OpenAI and Anthropic Claude

What is Function Calling?

Function calling enables LLMs like Claude and GPT to achieve complex and practical tasks beyond generating text alone. It bridges the gap between the model's conversational intelligence and external APIs, databases, and custom code.

OpenAI unleashed the power of function calling first, changing the game for how developers interact with LLMs. In a recent announcement from Anthropic, Claude joined the function calling party, bringing this critical capability to Claude 3 models.

How Function Calling Works

The Process

Critical Point: You Execute Functions

The model does not execute or call functions on the provider's side; it only returns the function name and arguments which you will have to parse and call yourself in your code. This is a security feature—the LLM suggests what to do, but you control what actually happens.

Model Support

Anthropic's Implementation

Messages API Structure

The Anthropic Messages API provides a clear mechanism for function calling using the tools, tool_use, and tool_result structures. This clean API design makes implementation straightforward.

Tool Definition Format

Tools are defined with:

• name: Unique identifier for the tool
• description: What the tool does (the model uses this to decide when to call it)
• input_schema: JSON Schema defining expected parameters

Common Use Cases

Database Queries

Convert natural language questions into database queries. The model determines what data to fetch, you execute the query, and the model formats results into readable answers.

API Integration

Enable the LLM to interact with external services: send emails, create calendar events, fetch weather data, search the web, or call custom business APIs.

Data Processing

Define functions for complex calculations, data transformations, or analysis that would be difficult or unreliable for the LLM to do through text generation alone.

Multi-Step Workflows

Chain multiple function calls together. The model can call one function, analyze results, and decide to call another function based on those results.

Implementation with LlamaIndex

LlamaIndex provides clean abstractions for function calling with both OpenAI and Anthropic. The FunctionCallingAgent class handles the complex orchestration automatically, letting you focus on defining tools and business logic.

Best Practices

Write Clear Tool Descriptions

The model relies entirely on tool descriptions to decide when to use them. Be specific about what each tool does, when to use it, and what parameters it needs.

Validate Inputs

Even though models are good at following schemas, always validate function arguments before execution. Models can occasionally generate invalid parameters.

Handle Errors Gracefully

When a function call fails, return clear error messages to the model. The model can often retry with different parameters or try an alternative approach.

Limit Function Scope

Each function should do one thing well. Don't create overly complex functions with many parameters—the model will struggle to use them correctly.

Return Structured Data

Function results should be structured (JSON) rather than free-form text when possible. This helps the model parse and use the data effectively.

Security Considerations

Advanced Patterns

Parallel Function Calling

Modern models can request multiple independent function calls simultaneously. Execute them in parallel to reduce latency.

Conditional Tool Availability

Don't always provide all tools. Based on conversation context or user permissions, dynamically adjust which tools are available to the model.

Function Calling Chains

Design workflows where one function's output becomes another's input. The model can orchestrate complex multi-step operations automatically.

Testing Function Calling Agents

Create comprehensive test suites covering:

• Correct function selection for various queries
• Proper argument extraction and formatting
• Error handling when functions fail
• Behavior when no appropriate function exists
• Multi-step workflows requiring multiple function calls