Speko Docs
Concepts

Provider params

Canonical STT/TTS fields, provider-scoped providerOptions, workload defaults, and how params behave across failover.

Routing picks the provider; provider params tune what that provider does. There are two layers:

  • Canonical fields (L1) - one cross-provider name, translated to each vendor's wire param by Speko.
  • providerOptions (L2) - provider-scoped passthrough, validated against Speko's param registry. A provider's options apply only when that provider serves the request.

Where params are accepted

SurfaceFieldsValidated as
POST /v1/transcribe - X-Speko-Stt-Options header (JSON, max 4 KB)workload, diarization, speakersExpected, smartFormat, fillerWords, profanityFilter, providerOptions (plus keywords, language)batch
WS /v1/transcribe/stream - config framesame STT request fields; diarization is not served on this route (live speaker labels are an agent-session feature)streaming
POST /v1/synthesize - bodyworkload, providerOptionsstreaming
/v1/sessions, /v1/sessions/phone, /v1/agents - sttOptions / ttsOptionssame STT request fields; diarization labels speakers live (see Speaker labels on live calls); TTS: workload, providerOptions (providerOptions capped at 2 KB serialized)streaming

Each providerOptions param in the registry declares the surfaces it supports. Sending one on the wrong surface is rejected with unsupported_surface - for example Deepgram's streaming-only endpointing on /v1/transcribe. Canonical fields use the failover rules below. diarization has batch mappings on /v1/transcribe and live mappings on agent sessions; WS /v1/transcribe/stream serves neither.

Canonical fields (L1)

FieldTypeMeaningWhen a candidate cannot serve it
diarizationbooleanSeparate speakers in a batch transcript, or on a live agent calldemote (candidate excluded)
speakersExpectedintegerHint of how many speakers to expectdrop
smartFormatbooleanProvider-side formatting (punctuation, casing, numbers)drop
fillerWordsbooleanKeep filler words ("um", "uh") instead of removing themdrop
profanityFilterbooleanMask profanity in the transcriptdrop
wordTimestampsbooleanPer-word start/end timings in the done frame's wordsdemote (candidate excluded)

Per-provider wire mapping:

FieldProviderWire paramSurfaces
diarizationDeepgramdiarizebatch, agent sessions (not Flux)
diarizationAssemblyAIspeaker_labelsbatch, agent sessions
diarizationMetamode: DIARIZATIONagent sessions
diarizationElevenLabsdiarizebatch, agent sessions (Scribe v3 Live)
diarizationSonioxenable_speaker_diarizationbatch, agent sessions
diarizationGooglediarizationConfigbatch
speakersExpectedAssemblyAIspeakers_expectedbatch
speakersExpectedElevenLabsnum_speakersbatch
speakersExpectedGooglediarizationConfig.maxSpeakerCountbatch
smartFormatDeepgramsmart_formatstreaming, batch
smartFormatAssemblyAIformat_text + punctuate (streaming: format_turns)batch
smartFormatGooglefeatures.enableAutomaticPunctuationstreaming, batch
smartFormatxAIformatbatch
fillerWordsDeepgramfiller_wordsstreaming, batch
fillerWordsAssemblyAIdisfluenciesbatch
profanityFilterDeepgramprofanity_filterstreaming, batch
profanityFilterAssemblyAIfilter_profanitybatch
wordTimestampsGeminimode.timestamp_granularities: ["word"]batch
wordTimestampsDeepgramper-word words (not Flux)batch

On /v1/transcribe, diarization and wordTimestamps are batch features. Either one makes the done frame carry words: an array of { text, start, end, confidence?, speaker? } with start/end in seconds from the start of the audio (multiply by 1000 for subtitle cue times). wordTimestamps is served by gemini-3.5-transcribe and by Deepgram's batch models (not Flux). Gemini cannot combine it (or diarization) with keywords, so the term list is dropped for that request, and Google caps timestamped audio at 30 minutes. Pin Gemini to be sure of it. A demote-class field excludes candidates that cannot serve it rather than selecting one that can, and automatic routing does not rank the pre-recorded Gemini row, so an unpinned request answers 422 no_capable_provider unless a Deepgram model is on the language's ladder. When Gemini fails, a pinned request falls back to Deepgram if Deepgram ranks for the language:

-H 'X-Speko-Stt-Options: {"wordTimestamps":true}' \
-H 'X-Speko-Constraints: {"allowedProviders":{"stt":["gemini:gemini-3.5-transcribe"]}}'

The Router needs no pin — its automatic routing filters candidates on the capability itself. WS /v1/transcribe/stream does not forward speakers. Speaker labels from diarization-capable providers are normalized to S0, S1, ... on word-level output, regardless of the vendor's native labeling.

Deepgram Flux models deliberately accept none of the canonical or provider-specific params in these tables.

Speaker labels on live calls

Set sttOptions.diarization: true on an agent (or per call on /v1/sessions and /v1/sessions/phone) when several people share the far end of the call: a meeting, a speakerphone, a front desk. In the dashboard this is the Speaker labels switch in the agent's transcription settings.

{ "sttOptions": { "diarization": true } }

What changes on the call:

  • Routing. Speech recognition runs on a model that labels speakers live: Soniox, Deepgram Nova, AssemblyAI streaming, Meta Muse Voice Transcribe, or ElevenLabs Scribe v3 Live. Automatic routing leaves the others out of the chain. Pinned models that cannot label speakers are left out too, and the response carries a param_unsupported_on_primary warning naming the provider that was skipped. If none of the models you pinned can label speakers, the session is refused with no_capable_provider (param: "diarization"). If a session's transcription runs through the compatibility proxy instead of a direct stage, it starts without labels and the response carries the same warning with provider: "speko-proxy".
  • Transcript. Each caller turn in GET /v1/sessions/:id/transcript carries speaker: S1, S2, ... numbered in the order the voices were first heard. When one turn holds more than one voice, speakerSegments splits it into [{ speaker, text }]; otherwise it is an empty array. text is always the words as spoken, without tags. Agent and system turns have speaker: null.
  • The agent. Once a second voice has been heard, the model sees each caller line tagged with its speaker ([Speaker 1], [Speaker 2]) and is told not to say the tags. A call with a single voice is handled exactly as it is without the option.

Limits to plan for:

  • A label is attached to each stretch of speech the recognizer finalizes. Two people talking back to back without a pause can land under one label.
  • A very short turn (under about a second) may come back without a speaker; it is then recorded with speaker: null.
  • Labels are numbered per call and are not voice identities: S1 on one call is unrelated to S1 on the next. A recognizer's speaker ids only mean something within one connection to it, so after a reconnect, a recognizer failover, or a mid-call change that restarts recognition (a language switch, for example), a voice heard before comes back under a new label rather than risk two people sharing one. The agent is only told about speakers once two voices have been heard on the same connection.
  • Replies are not drafted ahead of the end of the caller's turn on these calls, because a draft would be written before the speaker is known. Expect a slightly later first word than on the same agent without the option.
  • Meta Muse labels speakers in a mode Meta describes as not tuned for low latency, so turns can end later on that model than without the option.
  • ElevenLabs Scribe v3 Live sends a segment's speakers after its text. The worker holds each segment for them, for at most 0.8 seconds, so turns can end later than without the option. When Scribe later decides two labels were one person, lines already recorded keep the label they had.
  • A turn with more than 32 changes of speaker is returned with its dominant speaker and full text, without speakerSegments.
  • Labels are least reliable in the first seconds of a call and on narrowband phone audio.
  • Speech-to-speech agents have no transcription stage, so the option does not apply to them.
  • speakersExpected is not used on live calls.

providerOptions (L2)

{
  "providerOptions": {
    "deepgram": { "dictation": true },
    "elevenlabs": { "timestamps_granularity": "word" }
  }
}
  • Keys are provider ids; each inner object is validated against the registry tables below.
  • Unknown providers, unknown params, wrong types, invalid enum values, and numbers outside a strict 0-1 range return 400 invalid_provider_options with per-param details.
  • Numbers outside a documented non-unit range pass through with an out_of_range warning - the provider decides.
  • A provider's options apply only when that provider serves the request. On failover they never cross to the winning provider; the losing provider's dict is discarded.
  • Options for a provider outside your allowedProviders pins produce a provider_not_pinned warning; they still apply if routing picks that provider.
  • Spelling a canonical field inside providerOptions while also setting it at the top level (for example providerOptions.deepgram.diarize next to diarization, or voice_settings.speed next to speed) is rejected as canonical_collision - set one spelling.

STT params

ProviderParamTypeValuesSurfacesEffect
Deepgrampunctuatebooleanstreaming, batchAdds punctuation and capitalization to transcripts.
Deepgramnumeralsbooleanstreaming, batchFormats spoken numbers as numerals in supported languages.
Deepgramendpointinginteger10-2000streamingSilence duration in milliseconds before speech finalization.
Deepgramutterance_end_msinteger1000-5000streamingWord-gap duration in milliseconds before an UtteranceEnd event.
Deepgramvad_eventsbooleanstreamingEmits speech-start voice activity events.
Deepgramdetect_languagebooleanbatchDetects the dominant language of each channel.
Deepgramdictationbooleanstreaming, batchConverts spoken dictation commands into punctuation and layout.
Deepgrammeasurementsbooleanstreaming, batchFormats spoken measurement units as abbreviations.
AssemblyAIend_of_turn_confidence_thresholdnumber0-1streamingSets the confidence threshold for semantic end-of-turn detection.
AssemblyAImin_end_of_turn_silence_when_confidentinteger0-5000streamingSets the minimum confident end-of-turn silence in milliseconds.
AssemblyAImax_turn_silenceinteger100-10000streamingSets the maximum silence in milliseconds before forcing a turn end.
AssemblyAIformat_turnsbooleanstreamingFormats final streaming turns with punctuation and casing.
AssemblyAImultichannelbooleanbatchTranscribes each channel of a recording separately.
AssemblyAIlanguage_detectionbooleanbatchDetects the language of a prerecorded transcript.
ElevenLabsvad_silence_threshold_secsnumber0.3-3streamingSets the VAD silence duration in seconds before committing a turn.
ElevenLabscommit_strategyenumvad, turn_predictionstreamingScribe v3 Live only. Ends a segment on silence (vad, the default) or when the model predicts the speaker has finished.
ElevenLabsmultilang_modebooleanstreamingScribe v3 Live only. Follows speakers who switch languages during the session.
ElevenLabstag_audio_eventsbooleanbatchIncludes non-speech audio event tags in the transcript.
ElevenLabstimestamps_granularityenumword, characterbatchSelects word- or character-level transcript timestamps.
ElevenLabsnumbers_formatenumdigits, wordsbatchScribe v3 only. Writes numbers as digits or as words.
ElevenLabssecondary_language_codesarray of stringsISO 639-1 or 639-3 codesbatchScribe v3 only. Other languages spoken in the audio besides the main one. A code for a language Scribe does not support is refused.
Sonioxenable_language_identificationbooleanstreaming, batchIncludes detected language information on transcript tokens.

Google and xAI expose only the canonical fields above. OpenAI and Cartesia expose no tunable STT params yet.

TTS params

ProviderParamTypeValuesEffect
ElevenLabsvoice_settingsobjectsee belowOverrides the selected voice's per-request generation settings.
ElevenLabsapply_text_normalizationenumauto, on, offControls provider-side text normalization.
MiniMaxvolumenumber0.5-2Controls output volume multiplier.
MiniMaxpitchinteger-12 to 12Shifts pitch by semitones.
Cartesiaspeednumber0.6-1.5Controls speaking-rate multiplier. Collides with top-level speed.
Cartesiavolumenumber0.5-2Controls output volume multiplier.
Fish Audiolatencyenumnormal, lowSelects normal quality or low-latency generation.
Blandexpressivenessnumber0-1Controls expressive variation in generated speech.
Blandstabilitynumber0-1Controls consistency in generated speech.

voice_settings (ElevenLabs) properties:

PropertyTypeValuesEffect
stabilitynumber0-1Controls consistency versus expressive variation.
similarity_boostnumber0-1Controls similarity to the source voice.
stylenumber0-1Controls style exaggeration.
use_speaker_boostbooleanEnables additional speaker-similarity processing.
speednumber0.7-1.2Speaking-rate multiplier. Collides with top-level speed - set one.

When ElevenLabs serves a request that sets workload or providerOptions.elevenlabs, a top-level speed outside 0.7-1.2 is clamped to the nearest bound, so 1.25 is sent as 1.2. A voice_settings.speed you set yourself is sent unchanged.

Failover behavior

  • Drop-class params are omitted for a candidate that cannot serve them. The request still runs; the omission is reported as a param_dropped warning and the param is absent from applied.
  • Demote-class params (diarization) remove incapable candidates from the failover ladder before the first attempt, each reported as a candidate_excluded warning. If no candidate remains, the request fails with 422 { "error": "no_capable_provider", "code": "NO_CAPABLE_PROVIDER", "param": "diarization" } instead of silently running without the capability.

When a request carries any of the new fields, the response reports what actually applied:

  • /v1/transcribe: the final done SSE event gains effectiveParams: { provider, model, applied, defaultsVersion }, plus warnings when non-empty.
  • /v1/transcribe/stream: a metadata frame after ready carries the same object.
  • /v1/synthesize: response headers X-Speko-Applied-Params (JSON array of param names) and X-Speko-Warnings (JSON array, capped at 2 KB).

Requests without any of the new fields get byte-identical responses to before these fields existed.

Workload defaults

workload: 'conversation' | 'transcription' | 'narration' applies per-provider defaults from the registry. Defaults are presence-keyed: no workload, no defaults. Explicit fields always override defaults. Responses echo defaultsVersion (currently 2026-08-06) so a registry change is visible.

ModalityProviderWorkloadDefaults applied
STTDeepgramconversationsmartFormat: false, punctuate: true, fillerWords: true, vad_events: true
STTDeepgramtranscriptionsmartFormat: true
STTAssemblyAIconversationformat_turns: true
STTAssemblyAItranscriptionsmartFormat: true
STTElevenLabstranscriptiontimestamps_granularity: "word", tag_audio_events: false
TTSElevenLabsnarrationvoice_settings: { stability: 0.5, similarity_boost: 0.75, use_speaker_boost: true }

A default only applies when the candidate's surface and model support the param. On /v1/synthesize, workload: "narration" additionally prefers a 24 kHz output ladder when the request sets no sampleRate and at least one candidate serves that rate.

Use-case vocabulary

Three vocabularies coexist on different surfaces:

Router key useCasePlatform workloadVertical useCaseId (/v1/recommend-stack)
phone_agentconversationany of the 10 vertical profiles
realtime_agent (UI: "In-app voice assistant")conversationany of the 10 vertical profiles
transcriptiontranscription-
voice_content (UI: "Narration")narration-
translation, other (legacy, API-only)--

Vertical profiles: customer-support, sales-outbound, healthcare, telephony-receptionist, voice-assistant, education-tutoring, restaurant-ordering, mental-health-coaching, legal-intake, financial-banking-ivr.

Guides: Batch transcription, Narration.

On this page