Streaming responses
Server-sent events from /v1/llm/responses with stream true.
Enabling
Set "stream": true on a response request. The response becomes Content-Type: text/event-stream.
curl -N https://relay.speko.dev/v1/llm/responses \
-H "Authorization: Bearer $SPEKO_API_KEY" \
-H "Idempotency-Key: $(uuidgen)" \
-H "Content-Type: application/json" \
-d '{
"routing": { "mode": "explicit", "provider": "cerebras", "model": "llama-4-maverick-17b-128e-instruct" },
"input": [{ "type": "message", "role": "user", "content": [{ "type": "text", "text": "Stream me a haiku." }] }],
"max_output_tokens": 60,
"stream": true
}'Event set
The event names form a closed set:
| Event | Payload |
|---|---|
response.created | {"id": "resp_...", "route": {...}} — the admitted route |
response.item.added | A new output item has started |
response.text.delta | Incremental message text |
response.function_call_arguments.delta | Incremental JSON for a tool call's arguments |
response.item.completed | The current output item is complete |
response.completed | Terminal: final stop_reason and usage |
error | Terminal: the standard error envelope |
Exactly one terminal event — response.completed or error — ends every stream, never both and never neither. If the connection closes without one, the stream was truncated; discard partial output or retry with the same idempotency key.
Heartbeats
While a provider is thinking, the relay emits SSE comment lines as keepalives:
: heartbeatIgnore them; they exist to keep intermediaries from idling out the connection.
Failover on streams
response.created is deliberately not counted as the start of output: in auto mode, a provider failure after response.created but before the first content event still fails over transparently to the next provider. Once a response.text.delta (or any output item) has arrived, failures terminate the stream with an error event instead.