Generate responses

POST /v1/llm/responses — one request shape for every language model in the catalog.

Request

POST https://relay.speko.dev/v1/llm/responses

{
  "routing": { "mode": "auto", "objective": "quality" },
  "input": [
    { "type": "message", "role": "system", "content": [{ "type": "text", "text": "You are a concise assistant." }] },
    { "type": "message", "role": "user", "content": [{ "type": "text", "text": "Summarize our return policy." }] }
  ],
  "tools": [
    { "name": "lookup_policy", "description": "Fetch a policy document", "parameters": { "type": "object", "properties": { "topic": { "type": "string" } } } }
  ],
  "reasoning_effort": "medium",
  "max_output_tokens": 500,
  "stream": false
}

routingobject

The routing object. LLM entries differ in capabilities — auto mode only considers providers that support what your request uses (tools, structured output).

inputItem[]required

The conversation so far. There is no server-side conversation state and no previous_response_id — resend the full history on every request.

toolsFunctionTool[]

Optional function tools: name (required, unique), description, parameters (JSON Schema, passed through).

response_formatobject

Optional structured output: {"type": "json_schema", "name": "...", "schema": {...}, "strict": true}.

max_output_tokensnumberrequired

Must be positive. Required because an unbounded generation cannot be priced at admission. This is the hard limit for hidden reasoning tokens plus visible output tokens.

reasoning_effortstring

Optional. One of none, minimal, low, medium, high, or xhigh. When supplied, auto routing considers only models that can honor that control, including models that can explicitly disable reasoning for none. Omit it to place no reasoning constraint on routing.

temperaturenumber

0–2. Cannot be combined with an enabled reasoning_effort.

top_pnumber

Greater than 0, at most 1. Cannot be combined with an enabled reasoning_effort.

streamboolean

true switches the response to server-sent events.

Input items

input is a list of items — a closed union on type. Fields belonging to a different item type are rejected, never ignored.

Type	Fields	Direction
`message`	`role` (`system` \| `user` \| `assistant`), `content` (list of `{"type":"text","text":"..."}` parts)	input and output
`function_call`	`call_id`, `name`, `arguments` (JSON encoded as a string)	input (history) and output
`function_result`	`call_id`, `result` (opaque text, may be empty)	input only
`structured_json`	`json` (raw JSON value)	output only

There is no tool role — tool results are function_result items. Output messages are always assistant-authored.

Response

{
  "id": "resp_rreq_...",
  "route": { "provider": "openai", "model": "gpt-5.2", "region": "us-east-1", "attempt_id": "ratt_..." },
  "output": [
    { "type": "message", "role": "assistant", "content": [{ "type": "text", "text": "Returns are accepted within 30 days..." }] }
  ],
  "stop_reason": "stop",
  "usage": { "input_tokens": 180, "cached_input_tokens": 96, "output_tokens": 64, "reasoning_tokens": 12 }
}

idstring

Speko-minted: resp_ + the request ID. Provider response and conversation IDs never appear.

stop_reasonstring

stop, max_output_tokens, or tool_call.

usageobject

When the provider reports a reasoning split, token counts are mutually exclusive lines: cached_input_tokens are not repeated inside input_tokens, and reasoning_tokens are not repeated inside output_tokens. If a provider does not report a reasoning split, its inclusive output count remains in output_tokens. See Usage.

Tool calling flow

Send your messages plus tools. If the model decides to call one, the response has stop_reason: "tool_call" and a function_call item in output.
Run the function yourself.
Resend the full history — including the function_call item — plus a function_result item with the same call_id.
The model continues from there.

Request

Input items

Response

Tool calling flow

On this page