Speko Docs
Language models

Chat completions

POST /v1/chat/completions — every language model in the catalog over the OpenAI Chat Completions protocol, from any OpenAI SDK.

/v1/chat/completions speaks the OpenAI Chat Completions protocol. Point an OpenAI SDK, or any product that accepts a custom OpenAI-compatible URL, at the Router and keep the rest of your code. Behind it is the same routing, failover, and billing as Generate responses; only the wire format differs.

SettingValue
URLhttps://router.speko.dev/v1/chat/completions
SDK base URLhttps://router.speko.dev/v1
AuthAuthorization: Bearer sk_speko_...
Modelauto, auto:<objective>, or <provider>/<model>

Building on Agora Conversational AI? Jump to the Convo AI configuration.

Quick start

curl -N https://router.speko.dev/v1/chat/completions \
  -H "Authorization: Bearer $SPEKO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [
      { "role": "system", "content": "You are a concise voice assistant." },
      { "role": "user", "content": "What is the capital of France?" }
    ],
    "reasoning_effort": "none",
    "stream": true
  }'

No Idempotency-Key is needed here: the Router mints one when the header is absent. See Headers and idempotency.

With an OpenAI SDK, change the base URL and the key. The examples also send one Idempotency-Key per request, so the SDK's automatic retries cannot generate twice, and check that the stream reached its finish_reason:

import os
import uuid

from openai import OpenAI

client = OpenAI(
    base_url="https://router.speko.dev/v1",
    api_key=os.environ["SPEKO_API_KEY"],
)

# One key per request. The SDK resends it on its own retries; reuse it only
# to retry this exact request. Any new request gets a new key.
request_key = str(uuid.uuid4())

stream = client.chat.completions.create(
    model="auto",
    messages=[{"role": "user", "content": "What is the capital of France?"}],
    reasoning_effort="none",
    stream=True,
    extra_headers={"Idempotency-Key": request_key},
)
finish_reason = None
for chunk in stream:
    if not chunk.choices:
        continue
    choice = chunk.choices[0]
    if choice.delta.content:
        print(choice.delta.content, end="", flush=True)
    finish_reason = choice.finish_reason or finish_reason
if finish_reason is None:
    raise RuntimeError("the stream was cut off; discard the partial reply")
import { randomUUID } from 'node:crypto';
import OpenAI from 'openai';

const client = new OpenAI({
  baseURL: 'https://router.speko.dev/v1',
  apiKey: process.env.SPEKO_API_KEY,
});

// One key per request. The SDK resends it on its own retries; reuse it only
// to retry this exact request. Any new request gets a new key.
const requestKey = randomUUID();

const stream = await client.chat.completions.create(
  {
    model: 'auto',
    messages: [{ role: 'user', content: 'What is the capital of France?' }],
    reasoning_effort: 'none',
    stream: true,
  },
  { headers: { 'Idempotency-Key': requestKey } },
);
let finishReason: string | null = null;
for await (const chunk of stream) {
  const choice = chunk.choices[0];
  if (!choice) continue;
  process.stdout.write(choice.delta.content ?? '');
  finishReason = choice.finish_reason ?? finishReason;
}
if (finishReason === null) {
  throw new Error('the stream was cut off; discard the partial reply');
}

Choosing a model

model selects how the Router routes the request:

modelRouting
auto, or omittedYour key's routing policy; without one, auto routing with the balanced objective.
auto:<objective>Auto routing ranked for balanced, quality, latency, or cost. See Routing.
<provider>/<model>Pins one model, for example openai/gpt-4.1-mini or anthropic/claude-sonnet-5.

Pinnable models are the kind: "llm" entries in GET /v1/models: join provider and id with a slash. The id may itself contain slashes; everything before the first slash is the provider, so "baseten/deepseek-ai/DeepSeek-V4.1-Flash" pins Baseten's deepseek-ai/DeepSeek-V4.1-Flash.

In auto modes, a provider failure before the first output fails over to the next-ranked provider, as on streamed responses. The response's model field names the provider/model that actually served the request.

Any other value, such as a bare "gpt-4.1", is refused with a 400 that quotes it:

{
  "error": {
    "message": "the request body is invalid: model: \"gpt-4.1\" is not a Speko model; use \"auto\", \"auto:<objective>\" or \"<provider>/<model>\"",
    "type": "invalid_request_error",
    "code": "invalid_request",
    "param": null
  }
}

Request

modelstring

How to route the request. See Choosing a model.

messagesMessage[]required

The conversation so far. The Router keeps no conversation state: send the full history on every request. See Messages.

max_tokensintegerdefault: 1024

The output limit, including hidden reasoning tokens. max_completion_tokens is accepted as a synonym; if both are sent they must be equal. Must be positive. The Router reserves credit for this ceiling before it dispatches the request.

streambooleandefault: false

true returns server-sent events.

stream_options.include_usagebooleandefault: false

On a stream, adds a final chunk carrying usage.

reasoning_effortstring

One of none, minimal, low, medium, high, or xhigh. With auto, only models that can honor the value are considered. Omit it to leave each model at its provider default. See Reasoning and sampling.

temperaturenumber

0–2. Not allowed together with an enabled reasoning_effort.

top_pnumber

Greater than 0, at most 1. Not allowed together with an enabled reasoning_effort.

toolsTool[]

Function tools: {"type": "function", "function": {"name", "description", "parameters"}}. parameters is JSON Schema, passed through. strict is accepted but not enforced: call arguments are checked to be valid JSON, not checked against the schema.

tool_choicestring

Only "auto", the default.

Messages

RoleContentNotes
system, developertextBoth are system instructions.
usertext
assistanttext, tool_calls, or bothAn assistant message with neither is dropped.
tooltextAnswers the call named by tool_call_id.

Text content can be a string, an array of strings, an array of {"type": "text", "text": "..."} parts, or a mix of both. Consecutive strings in an array are separate lines; text parts are fragments of one message and join with no separator. Image, audio, and file parts are refused. name is ignored.

Ignored and refused fields

These are accepted and change nothing: user, metadata, store, seed, service_tier, n: 1, parallel_tool_calls: true, modalities: ["text"], response_format: {"type": "text"}, null values, and unknown fields, including Agora's context.

These are refused with 400 invalid_request, and the message names the field:

FieldAccepted values
n1 only
parallel_tool_callstrue only. The Router cannot limit a provider to one call per turn, and dropping calls would lose work the model asked for.
response_format{"type": "text"} only. For JSON output, use /v1/llm/responses.
tool_choice"auto" only
stop, logprobs: true, top_logprobs, logit_bias, audio, prediction, web_search_optionsnot supported
frequency_penalty, presence_penalty0 only
functions, function_callnot supported; use tools

Reasoning and sampling

Many models in the catalog reason before they answer, and reasoning delays the first token by seconds. For voice, send "reasoning_effort": "none": auto then routes only to models that can answer without reasoning, and the first token typically arrives in about a second instead of several.

temperature and top_p cannot be combined with an enabled reasoning_effort: send them with none or with reasoning_effort omitted.

"reasoning_effort": "low" (or any value other than none) together with temperature or top_p is refused with 400 invalid_request. Some clients, including Agora's Convo AI, send temperature and top_p by default.

Response

A non-streaming request returns one chat.completion:

{
  "id": "resp_rreq_01k9x2m7c4f8h3j6n5q0r2t1vw",
  "object": "chat.completion",
  "created": 1794478510,
  "model": "openai/gpt-4.1-mini",
  "choices": [
    {
      "index": 0,
      "message": { "role": "assistant", "content": "The capital of France is Paris." },
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 24,
    "completion_tokens": 8,
    "total_tokens": 32,
    "prompt_tokens_details": { "cached_tokens": 0 },
    "completion_tokens_details": { "reasoning_tokens": 0 }
  }
}
idstring

The Router's response ID: resp_ + the request ID from the Speko-Request-ID header.

modelstring

The provider/model that served the request.

choices[0].message.contentstring | null

The reply text, or null when the model returned no text. Alongside tool calls it is often null, but some models add a little whitespace.

choices[0].message.tool_callsToolCall[]

Present when the model called tools: {"id", "type": "function", "function": {"name", "arguments"}}.

choices[0].finish_reasonstring

stop, length (hit max_tokens), or tool_calls.

usageobject

OpenAI's inclusive totals: prompt_tokens includes cached tokens and completion_tokens includes reasoning tokens; the _details fields give the splits. Billing follows the Router's usage for the same request.

Streaming

With "stream": true the response is Content-Type: text/event-stream: one data: line per chat.completion.chunk, then data: [DONE].

data: {"id":"resp_rreq_...","object":"chat.completion.chunk","created":1794478510,"model":"openai/gpt-4.1-mini","choices":[{"index":0,"delta":{"role":"assistant","content":"The"},"finish_reason":null}]}

data: {"id":"resp_rreq_...","object":"chat.completion.chunk","created":1794478510,"model":"openai/gpt-4.1-mini","choices":[{"index":0,"delta":{"content":" capital of France is Paris."},"finish_reason":null}]}

data: {"id":"resp_rreq_...","object":"chat.completion.chunk","created":1794478510,"model":"openai/gpt-4.1-mini","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}

data: {"id":"resp_rreq_...","object":"chat.completion.chunk","created":1794478510,"model":"openai/gpt-4.1-mini","choices":[],"usage":{"prompt_tokens":24,"completion_tokens":8,"total_tokens":32,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens_details":{"reasoning_tokens":0}}}

data: [DONE]
  • The first chunk carries "role": "assistant".
  • Tool calls stream as delta.tool_calls with an index per call: the first chunk of a call carries id, type, and function.name; later chunks append to function.arguments.
  • The chunk with finish_reason comes last, then the usage chunk if stream_options.include_usage is set, then [DONE].
  • While a model is thinking, the Router sends : heartbeat comment lines to keep proxies from closing the connection. SSE clients, including the OpenAI SDKs, skip them.
  • A failure after the stream started ends it with one data: {"error": {...}} line in the error shape and no [DONE]; the OpenAI SDKs raise it as an error. A stream that ends with neither was cut off: discard the partial reply.
  • The SDK iterators consume [DONE] and also end quietly on a closed connection, so they cannot tell a finished stream from a cut-off one. Track finish_reason as the examples above do: a stream that ends without a chunk carrying one is incomplete, and text already shown from it is partial. Checking for [DONE] itself requires reading the raw SSE stream.

Tool calls

The flow is OpenAI's. Send tools; when the model calls one, the response has finish_reason: "tool_calls":

{
  "role": "assistant",
  "content": null,
  "tool_calls": [
    {
      "id": "call_a1b2c3",
      "type": "function",
      "function": { "name": "get_weather", "arguments": "{\"city\":\"Paris\"}" }
    }
  ]
}

Run the function, then resend the full history with that assistant message and a tool message carrying the same tool_call_id:

{
  "model": "auto",
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Current weather for a city",
        "parameters": { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] }
      }
    }
  ],
  "messages": [
    { "role": "user", "content": "What's the weather in Paris?" },
    {
      "role": "assistant",
      "content": null,
      "tool_calls": [
        { "id": "call_a1b2c3", "type": "function", "function": { "name": "get_weather", "arguments": "{\"city\":\"Paris\"}" } }
      ]
    },
    { "role": "tool", "tool_call_id": "call_a1b2c3", "content": "{\"temp_c\": 18, \"sky\": \"clear\"}" }
  ]
}

A model may call several tools in one turn; answer each call with its own tool message.

Errors

Errors use OpenAI's shape. code is the Router's error code and the HTTP status is the Router's:

{
  "error": {
    "message": "a Bearer authorization header is required",
    "type": "authentication_error",
    "code": "authentication_failed",
    "param": null
  }
}
typecode
invalid_request_errorinvalid_request, capability_unsupported, payload_too_large
authentication_errorauthentication_failed
insufficient_quotainsufficient_credit, budget_exhausted
rate_limit_errorrate_limited, concurrency_exhausted
api_errorevery other code, such as provider_unavailable or provider_error

param is always null; the message names the offending field. Retry rate_limited, concurrency_exhausted, and provider_unavailable with backoff, with the same Idempotency-Key (see Retries in the OpenAI SDKs). The Speko-Request-ID response header identifies the request for support.

Headers and idempotency

Idempotency-Key is optional on this endpoint, unlike other Router POSTs. Without one, the Router mints a key, so every request is new. Send your own to make retries safe: the idempotency rules apply, hashed over the body exactly as sent.

Retries in the OpenAI SDKs

The OpenAI SDKs retry a request up to twice by default (after a timeout, a dropped connection, 409, 429, or a 5xx) and send no Idempotency-Key. If the Router had already dispatched the first attempt, each retry is a new, separately billed generation.

Create a new key for every request, as the quick start does, and send that same key on every attempt of that request: the SDK resends it on its own retries, and your own retries should reuse it too. A retry of a request the Router already dispatched then gets 409 request_already_started instead of a second generation. Any request with a different body gets a new key, including the follow-up that carries tool results within the same conversation turn; reusing a key for a different body is a 409 idempotency_conflict. For voice turns, where a retried reply would arrive too late anyway, you can also turn SDK retries off with max_retries=0 (Python) or maxRetries: 0 (TypeScript).

Responses carry the usual route headers: Speko-Request-ID, Speko-Provider, Speko-Model, and Speko-Region.

Agora Conversational AI

Convo AI can use the Router as its LLM. Agora's cloud calls the URL directly, so pass the full chat-completions URL and a Speko API key.

In the agent's start request, set llm to:

{
  "vendor": "custom",
  "style": "openai",
  "url": "https://router.speko.dev/v1/chat/completions",
  "api_key": "sk_speko_...",
  "system_messages": [
    { "role": "system", "content": "You are a friendly voice assistant. Keep replies short." }
  ],
  "greeting_message": "Hi there! How can I help you today?",
  "failure_message": "Sorry, give me a moment.",
  "max_history": 15,
  "params": {
    "model": "auto",
    "max_tokens": 1024,
    "reasoning_effort": "none"
  }
}

With Agora's Python server SDK, the same configuration is:

import os

from agora_agent.agentkit.vendors import CustomLLM

llm = CustomLLM(
    base_url="https://router.speko.dev/v1/chat/completions",
    api_key=os.environ["SPEKO_API_KEY"],
    model="auto",
    params={"reasoning_effort": "none"},
    greeting_message="Hi there! How can I help you today?",
    failure_message="Sorry, give me a moment.",
    max_history=15,
    max_tokens=1024,
)
  • Set model. Agora's sample code defaults to "mock-model", which the Router refuses. Use "auto", "auto:latency", or a pinned "<provider>/<model>".
  • Keep reasoning_effort at none. It keeps the first token fast, and Agora's samples also send temperature and top_p, which an enabled reasoning_effort would refuse. See Reasoning and sampling.
  • Everything else works as Agora sends it: the greeting replayed as an assistant turn before the first user turn, system and tool content as arrays of strings, tool calls and their results, and the extra context field, which the Router ignores.

On this page