Tool calling

Tool calling

Tools let a model ask your code to do something: read a file, run a command, call an API.
The gateway passes your schemas through and returns the model’s calls in the shape your
client expects.

Check capabilities.tools on the model first. A model without it cannot use tools, and
sending schemas to it wastes prompt tokens.

A complete loop

{
  "model": "glm-5.3-flash",
  "messages": [{ "role": "user", "content": "Read /etc/hostname" }],
  "tools": [{
    "type": "function",
    "function": {
      "name": "read_file",
      "description": "Read a file from disk.",
      "parameters": {
        "type": "object",
        "properties": { "path": { "type": "string", "description": "Absolute path." } },
        "required": ["path"]
      }
    }
  }]
}

The reply carries the call:

{
  "choices": [{
    "index": 0,
    "message": {
      "role": "assistant",
      "content": null,
      "tool_calls": [{
        "id": "call-1",
        "type": "function",
        "function": { "name": "read_file", "arguments": "{\"path\":\"/etc/hostname\"}" }
      }]
    },
    "finish_reason": "tool_calls"
  }]
}

Then send the result back and let the model finish:

{
  "model": "glm-5.3-flash",
  "messages": [
    { "role": "user", "content": "Read /etc/hostname" },
    { "role": "assistant", "content": null, "tool_calls": [{ "id": "call-1", "type": "function", "function": { "name": "read_file", "arguments": "{\"path\":\"/etc/hostname\"}" } }] },
    { "role": "tool", "tool_call_id": "call-1", "content": "vps-prod" }
  ],
  "tools": [ /* same schemas */ ]
}

The second turn must end with finish_reason: "stop" and text. If it asks for another
tool, that is the model doing more work, not a loop.

arguments as string or object

Clients disagree: some send function.arguments as a JSON string, some as an
object. Both are accepted and normalised before the upstream call.

This matters because forwarding an object verbatim produced a provider 400 that the
router then retried across every account, which in a dashboard reads as a full outage when
the cause was one caller’s payload shape.

tool_choice

Reasoning models and tools

A reasoning model may spend its budget thinking before calling a tool. If the cap is too
small it returns empty content with no tool call and finish_reason: "stop".

The gateway raises a cap below 256 for a model that reasons, and reports the change as
cap_raised_from in the response. Leave max_tokens out rather than setting it low.

Streaming tool calls

In a stream, one call arrives as several frames. The name appears once, on the frame that
opens the call; later frames carry arguments fragments to concatenate. See
Streaming for the exact frames.

The Anthropic path

/messages takes Anthropic tool definitions and returns Anthropic tool_use blocks with
stop_reason: "tool_use". The gateway does not translate one dialect into the other, so
Claude Code sees what it expects and an OpenAI client does too.

What is refused