Chat completions
POST /v1/chat/completions — every language model in the catalog over the OpenAI Chat Completions protocol, from any OpenAI SDK.
/v1/chat/completions speaks the OpenAI Chat Completions protocol. Point an
OpenAI SDK, or any product that accepts a custom OpenAI-compatible URL, at the
Router and keep the rest of your code. Behind it is the same routing,
failover, and billing as Generate responses; only the
wire format differs.
| Setting | Value |
|---|---|
| URL | https://router.speko.dev/v1/chat/completions |
| SDK base URL | https://router.speko.dev/v1 |
| Auth | Authorization: Bearer sk_speko_... |
| Model | auto, auto:<objective>, or <provider>/<model> |
Building on Agora Conversational AI? Jump to the Convo AI configuration.
Quick start
curl -N https://router.speko.dev/v1/chat/completions \
-H "Authorization: Bearer $SPEKO_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "auto",
"messages": [
{ "role": "system", "content": "You are a concise voice assistant." },
{ "role": "user", "content": "What is the capital of France?" }
],
"reasoning_effort": "none",
"stream": true
}'No Idempotency-Key is needed here: the Router mints one when the header is
absent. See Headers and idempotency.
With an OpenAI SDK, change the base URL and the key. The examples also send
one Idempotency-Key per request, so the SDK's automatic
retries cannot generate twice, and check that
the stream reached its finish_reason:
import os
import uuid
from openai import OpenAI
client = OpenAI(
base_url="https://router.speko.dev/v1",
api_key=os.environ["SPEKO_API_KEY"],
)
# One key per request. The SDK resends it on its own retries; reuse it only
# to retry this exact request. Any new request gets a new key.
request_key = str(uuid.uuid4())
stream = client.chat.completions.create(
model="auto",
messages=[{"role": "user", "content": "What is the capital of France?"}],
reasoning_effort="none",
stream=True,
extra_headers={"Idempotency-Key": request_key},
)
finish_reason = None
for chunk in stream:
if not chunk.choices:
continue
choice = chunk.choices[0]
if choice.delta.content:
print(choice.delta.content, end="", flush=True)
finish_reason = choice.finish_reason or finish_reason
if finish_reason is None:
raise RuntimeError("the stream was cut off; discard the partial reply")import { randomUUID } from 'node:crypto';
import OpenAI from 'openai';
const client = new OpenAI({
baseURL: 'https://router.speko.dev/v1',
apiKey: process.env.SPEKO_API_KEY,
});
// One key per request. The SDK resends it on its own retries; reuse it only
// to retry this exact request. Any new request gets a new key.
const requestKey = randomUUID();
const stream = await client.chat.completions.create(
{
model: 'auto',
messages: [{ role: 'user', content: 'What is the capital of France?' }],
reasoning_effort: 'none',
stream: true,
},
{ headers: { 'Idempotency-Key': requestKey } },
);
let finishReason: string | null = null;
for await (const chunk of stream) {
const choice = chunk.choices[0];
if (!choice) continue;
process.stdout.write(choice.delta.content ?? '');
finishReason = choice.finish_reason ?? finishReason;
}
if (finishReason === null) {
throw new Error('the stream was cut off; discard the partial reply');
}Choosing a model
model selects how the Router routes the request:
model | Routing |
|---|---|
auto, or omitted | Your key's routing policy; without one, auto routing with the balanced objective. |
auto:<objective> | Auto routing ranked for balanced, quality, latency, or cost. See Routing. |
<provider>/<model> | Pins one model, for example openai/gpt-4.1-mini or anthropic/claude-sonnet-5. |
Pinnable models are the kind: "llm" entries in GET /v1/models:
join provider and id with a slash. The id may itself contain slashes;
everything before the first slash is the provider, so
"baseten/deepseek-ai/DeepSeek-V4.1-Flash" pins Baseten's
deepseek-ai/DeepSeek-V4.1-Flash.
In auto modes, a provider failure before the first output fails over to the
next-ranked provider, as on streamed responses.
The response's model field names the provider/model that actually served
the request.
Any other value, such as a bare "gpt-4.1", is refused with a 400 that
quotes it:
{
"error": {
"message": "the request body is invalid: model: \"gpt-4.1\" is not a Speko model; use \"auto\", \"auto:<objective>\" or \"<provider>/<model>\"",
"type": "invalid_request_error",
"code": "invalid_request",
"param": null
}
}Request
modelstringHow to route the request. See Choosing a model.
messagesMessage[]requiredThe conversation so far. The Router keeps no conversation state: send the full history on every request. See Messages.
max_tokensintegerdefault: 1024The output limit, including hidden reasoning tokens. max_completion_tokens is accepted as a synonym; if both are sent they must be equal. Must be positive. The Router reserves credit for this ceiling before it dispatches the request.
streambooleandefault: falsetrue returns server-sent events.
stream_options.include_usagebooleandefault: falseOn a stream, adds a final chunk carrying usage.
reasoning_effortstringOne of none, minimal, low, medium, high, or xhigh. With auto, only models that can honor the value are considered. Omit it to leave each model at its provider default. See Reasoning and sampling.
temperaturenumber0–2. Not allowed together with an enabled reasoning_effort.
top_pnumberGreater than 0, at most 1. Not allowed together with an enabled reasoning_effort.
toolsTool[]Function tools: {"type": "function", "function": {"name", "description", "parameters"}}. parameters is JSON Schema, passed through. strict is accepted but not enforced: call arguments are checked to be valid JSON, not checked against the schema.
tool_choicestringOnly "auto", the default.
Messages
| Role | Content | Notes |
|---|---|---|
system, developer | text | Both are system instructions. |
user | text | |
assistant | text, tool_calls, or both | An assistant message with neither is dropped. |
tool | text | Answers the call named by tool_call_id. |
Text content can be a string, an array of strings, an array of
{"type": "text", "text": "..."} parts, or a mix of both. Consecutive strings
in an array are separate lines; text parts are fragments of one message and
join with no separator. Image, audio, and file parts are refused. name is
ignored.
Ignored and refused fields
These are accepted and change nothing: user, metadata, store, seed,
service_tier, n: 1, parallel_tool_calls: true, modalities: ["text"],
response_format: {"type": "text"}, null values, and unknown fields,
including Agora's context.
These are refused with 400 invalid_request, and the message names the field:
| Field | Accepted values |
|---|---|
n | 1 only |
parallel_tool_calls | true only. The Router cannot limit a provider to one call per turn, and dropping calls would lose work the model asked for. |
response_format | {"type": "text"} only. For JSON output, use /v1/llm/responses. |
tool_choice | "auto" only |
stop, logprobs: true, top_logprobs, logit_bias, audio, prediction, web_search_options | not supported |
frequency_penalty, presence_penalty | 0 only |
functions, function_call | not supported; use tools |
Reasoning and sampling
Many models in the catalog reason before they answer, and reasoning delays the
first token by seconds. For voice, send "reasoning_effort": "none":
auto then routes only to models that can answer without reasoning, and the
first token typically arrives in about a second instead of several.
temperature and top_p cannot be combined with an enabled
reasoning_effort: send them with none or with reasoning_effort omitted.
"reasoning_effort": "low" (or any value other than none) together with
temperature or top_p is refused with 400 invalid_request. Some clients,
including Agora's Convo AI, send temperature and top_p by default.
Response
A non-streaming request returns one chat.completion:
{
"id": "resp_rreq_01k9x2m7c4f8h3j6n5q0r2t1vw",
"object": "chat.completion",
"created": 1794478510,
"model": "openai/gpt-4.1-mini",
"choices": [
{
"index": 0,
"message": { "role": "assistant", "content": "The capital of France is Paris." },
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 24,
"completion_tokens": 8,
"total_tokens": 32,
"prompt_tokens_details": { "cached_tokens": 0 },
"completion_tokens_details": { "reasoning_tokens": 0 }
}
}idstringThe Router's response ID: resp_ + the request ID from the Speko-Request-ID header.
modelstringThe provider/model that served the request.
choices[0].message.contentstring | nullThe reply text, or null when the model returned no text. Alongside tool calls it is often null, but some models add a little whitespace.
choices[0].message.tool_callsToolCall[]Present when the model called tools: {"id", "type": "function", "function": {"name", "arguments"}}.
choices[0].finish_reasonstringstop, length (hit max_tokens), or tool_calls.
usageobjectOpenAI's inclusive totals: prompt_tokens includes cached tokens and completion_tokens includes reasoning tokens; the _details fields give the splits. Billing follows the Router's usage for the same request.
Streaming
With "stream": true the response is Content-Type: text/event-stream: one
data: line per chat.completion.chunk, then data: [DONE].
data: {"id":"resp_rreq_...","object":"chat.completion.chunk","created":1794478510,"model":"openai/gpt-4.1-mini","choices":[{"index":0,"delta":{"role":"assistant","content":"The"},"finish_reason":null}]}
data: {"id":"resp_rreq_...","object":"chat.completion.chunk","created":1794478510,"model":"openai/gpt-4.1-mini","choices":[{"index":0,"delta":{"content":" capital of France is Paris."},"finish_reason":null}]}
data: {"id":"resp_rreq_...","object":"chat.completion.chunk","created":1794478510,"model":"openai/gpt-4.1-mini","choices":[{"index":0,"delta":{},"finish_reason":"stop"}]}
data: {"id":"resp_rreq_...","object":"chat.completion.chunk","created":1794478510,"model":"openai/gpt-4.1-mini","choices":[],"usage":{"prompt_tokens":24,"completion_tokens":8,"total_tokens":32,"prompt_tokens_details":{"cached_tokens":0},"completion_tokens_details":{"reasoning_tokens":0}}}
data: [DONE]- The first chunk carries
"role": "assistant". - Tool calls stream as
delta.tool_callswith anindexper call: the first chunk of a call carriesid,type, andfunction.name; later chunks append tofunction.arguments. - The chunk with
finish_reasoncomes last, then the usage chunk ifstream_options.include_usageis set, then[DONE]. - While a model is thinking, the Router sends
: heartbeatcomment lines to keep proxies from closing the connection. SSE clients, including the OpenAI SDKs, skip them. - A failure after the stream started ends it with one
data: {"error": {...}}line in the error shape and no[DONE]; the OpenAI SDKs raise it as an error. A stream that ends with neither was cut off: discard the partial reply. - The SDK iterators consume
[DONE]and also end quietly on a closed connection, so they cannot tell a finished stream from a cut-off one. Trackfinish_reasonas the examples above do: a stream that ends without a chunk carrying one is incomplete, and text already shown from it is partial. Checking for[DONE]itself requires reading the raw SSE stream.
Tool calls
The flow is OpenAI's. Send tools; when the model calls one, the response
has finish_reason: "tool_calls":
{
"role": "assistant",
"content": null,
"tool_calls": [
{
"id": "call_a1b2c3",
"type": "function",
"function": { "name": "get_weather", "arguments": "{\"city\":\"Paris\"}" }
}
]
}Run the function, then resend the full history with that assistant message
and a tool message carrying the same tool_call_id:
{
"model": "auto",
"tools": [
{
"type": "function",
"function": {
"name": "get_weather",
"description": "Current weather for a city",
"parameters": { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] }
}
}
],
"messages": [
{ "role": "user", "content": "What's the weather in Paris?" },
{
"role": "assistant",
"content": null,
"tool_calls": [
{ "id": "call_a1b2c3", "type": "function", "function": { "name": "get_weather", "arguments": "{\"city\":\"Paris\"}" } }
]
},
{ "role": "tool", "tool_call_id": "call_a1b2c3", "content": "{\"temp_c\": 18, \"sky\": \"clear\"}" }
]
}A model may call several tools in one turn; answer each call with its own
tool message.
Errors
Errors use OpenAI's shape. code is the Router's
error code and the HTTP status is the Router's:
{
"error": {
"message": "a Bearer authorization header is required",
"type": "authentication_error",
"code": "authentication_failed",
"param": null
}
}type | code |
|---|---|
invalid_request_error | invalid_request, capability_unsupported, payload_too_large |
authentication_error | authentication_failed |
insufficient_quota | insufficient_credit, budget_exhausted |
rate_limit_error | rate_limited, concurrency_exhausted |
api_error | every other code, such as provider_unavailable or provider_error |
param is always null; the message names the offending field. Retry
rate_limited, concurrency_exhausted, and provider_unavailable with
backoff, with the same Idempotency-Key (see
Retries in the OpenAI SDKs). The
Speko-Request-ID response header identifies the request for support.
Headers and idempotency
Idempotency-Key is optional on this endpoint, unlike other Router POSTs.
Without one, the Router mints a key, so every request is new. Send your own to
make retries safe: the idempotency rules apply, hashed
over the body exactly as sent.
Retries in the OpenAI SDKs
The OpenAI SDKs retry a request up to twice by default (after a timeout, a
dropped connection, 409, 429, or a 5xx) and send no Idempotency-Key.
If the Router had already dispatched the first attempt, each retry is a new,
separately billed generation.
Create a new key for every request, as the quick start does,
and send that same key on every attempt of that request: the SDK resends it
on its own retries, and your own retries should reuse it too. A retry of a
request the Router already dispatched then gets 409 request_already_started
instead of a second generation. Any request with a different body gets a new
key, including the follow-up that carries tool results within
the same conversation turn; reusing a key for a different body is a
409 idempotency_conflict. For voice turns, where a retried reply would
arrive too late anyway, you can also turn SDK retries off with
max_retries=0 (Python) or maxRetries: 0 (TypeScript).
Responses carry the usual route headers:
Speko-Request-ID, Speko-Provider, Speko-Model, and Speko-Region.
Agora Conversational AI
Convo AI can use the Router as its LLM. Agora's cloud calls the URL directly, so pass the full chat-completions URL and a Speko API key.
In the agent's start request, set llm to:
{
"vendor": "custom",
"style": "openai",
"url": "https://router.speko.dev/v1/chat/completions",
"api_key": "sk_speko_...",
"system_messages": [
{ "role": "system", "content": "You are a friendly voice assistant. Keep replies short." }
],
"greeting_message": "Hi there! How can I help you today?",
"failure_message": "Sorry, give me a moment.",
"max_history": 15,
"params": {
"model": "auto",
"max_tokens": 1024,
"reasoning_effort": "none"
}
}With Agora's Python server SDK, the same configuration is:
import os
from agora_agent.agentkit.vendors import CustomLLM
llm = CustomLLM(
base_url="https://router.speko.dev/v1/chat/completions",
api_key=os.environ["SPEKO_API_KEY"],
model="auto",
params={"reasoning_effort": "none"},
greeting_message="Hi there! How can I help you today?",
failure_message="Sorry, give me a moment.",
max_history=15,
max_tokens=1024,
)- Set
model. Agora's sample code defaults to"mock-model", which the Router refuses. Use"auto","auto:latency", or a pinned"<provider>/<model>". - Keep
reasoning_effortatnone. It keeps the first token fast, and Agora's samples also sendtemperatureandtop_p, which an enabledreasoning_effortwould refuse. See Reasoning and sampling. - Everything else works as Agora sends it: the greeting replayed as an
assistant turn before the first user turn, system and tool content as
arrays of strings, tool calls and their results, and the extra
contextfield, which the Router ignores.