@spekoai/client

@spekoai/client is the browser-side companion to @spekoai/sdk. It connects a browser tab to a Speko voice session: capture the user's microphone, play the agent's audio, and exchange structured events such as transcripts and status changes.

Your server must mint a short-lived session token and return only browser-safe credentials. Never expose a Speko or provider root API key to browser code. For VoiceConversation, audio uses Speko's browser media transport. For RealtimeVoiceConversation, Speko reserves credit and mints a scoped provider credential, then audio flows directly between the browser and OpenAI, xAI, or Gemini Live.

Install

npm install @spekoai/client
# or
pnpm add @spekoai/client

The package does not expose low-level media transport types on its public surface, so most apps only import from @spekoai/client directly.

Choose a browser path

Use RealtimeVoiceConversation for provider-direct S2S. Your backend creates a session with mode: 's2s'; the browser connects straight to the provider with the returned short-lived credential:

const session = await fetch('/api/realtime-session', { method: 'POST' }).then((r) =>
  r.json(),
);

const conversation = await RealtimeVoiceConversation.create({
  ...session,
  onMessage: ({ source, text, isFinal }) => console.log(source, text, isFinal),
});

See Provider-direct speech-to-speech for the backend and browser flow.

Use VoiceConversation when your backend creates mode: 'cascade' and returns LiveKit transport credentials:

import { VoiceConversation } from '@spekoai/client';

const conversation = await VoiceConversation.create({
  transportToken,   // from server
  transportUrl,     // from server

  onConnect: ({ conversationId }) => console.log('connected', conversationId),
  onDisconnect: ({ reason }) => console.log('disconnected', reason),
  onMessage: ({ source, text, isFinal }) =>
    console.log(source, text, isFinal),
  onStatusChange: (status) => console.log('status', status),
  onModeChange: (mode) => console.log('mode', mode),
  onError: (err) => console.error(err),
});

await conversation.setMicMuted(true);
conversation.setVolume(0.8);
conversation.sendUserMessage('hello');
conversation.sendContextualUpdate('user switched to the checkout page');

await conversation.endSession();

See Build a voice agent for the cascade worker side.

Per-session variables (dynamic variables)

One agent, different values per session: a driver's name, a tenant id, a booking reference, a scoped access token. Your backend sets them when it mints the session. POST /v1/sessions accepts these per-session fields alongside agentId:

Your backend mints the session and forwards only the two transport fields:

// Server-side. SPEKO_API_KEY never leaves this process.
const session = await fetch('https://api.speko.dev/v1/sessions', {
  method: 'POST',
  headers: {
    Authorization: `Bearer ${process.env.SPEKO_API_KEY}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    mode: 'cascade',
    agentId: 'agent_123',
    variables: {
      driver_name: driver.firstName,
      driver_id: driver.id,
      fleet_tier: driver.tier,
    },
    toolSecrets: {
      fleet_api_token: await mintFleetToken(driver.id),
    },
    webhookTags: { tenant: driver.tenantId },
  }),
}).then((r) => r.json());

// Forward these two and nothing else.
return { transportToken: session.transportToken, transportUrl: session.transportUrl };

The page connects knowing none of it:

const { transportToken, transportUrl } = await fetch('/api/voice-session', {
  method: 'POST',
}).then((r) => r.json());

const conversation = await VoiceConversation.create({ transportToken, transportUrl });

The page cannot forge these values. They are bound server-side before the transport token exists, and the browser never receives the fields — so tampered page code can only reconnect with values already fixed. When the variable is a driver_id or a tenant id, that is the difference between something the agent may act on and a string the client asserted.

Per-session values are set at mint and hold for the session's lifetime; the browser has no path to change them mid-call.

What the SDK owns

• Connecting with supplied short-lived provider-direct or LiveKit credentials.
• Acquiring the microphone with sensible constraints (echo cancellation, noise suppression, auto gain — all togglable via audioConstraints).
• Playing remote audio.
• Normalizing provider or LiveKit transcript and status events into the same callbacks.
• Sending typed user turns via sendChatMessage.
• Mic mute, speaker volume, output device selection.
• Tearing everything down on disconnect, including releasing the OS microphone capture.

What it doesn't do

• Mint sessions from API keys. Keep SPEKO_API_KEY on your server. Browser code should only receive short-lived session tokens. Per-session prompt values, tool credentials, and webhook routing are set on that server-side mint — see per-session variables.
• Configure the agent from the browser. ConversationOverrides, sendUserMessage, and sendContextualUpdate publish data packets that no agent reads. Session config is a server-side concern.
• Retries. A failed connect() throws a SpekoClientError. Retry logic belongs in your app's UX.
• Tool calls, guardrail hooks, MCP, VAD score streaming. Deferred — see the package's ROADMAP.md.

Reference

• VoiceConversation — the primary API surface.
• RealtimeVoiceConversation — provider-direct S2S over OpenAI WebRTC or xAI/Gemini Live WebSockets.
• Callbacks & events — every hook the SDK exposes.
• Data channel protocol — wire format for inbound / outbound packets.
• Errors — SpekoClientError and its codes.