Speko Docs

iOS (Swift)

Run a Speko voice conversation in a native iOS app with the SpekoClient Swift package.

SpekoClient joins a Speko voice session from an iOS app. It captures the mic, plays the agent, and streams a reconciled transcript. It runs over LiveKit, or Daily when the session says so.

let conversation = try await VoiceConversation.create(options: VoiceConversationOptions(
    transport: SpekoTransport(rawValue: session.transport) ?? .livekit,
    transportToken: session.transportToken,
    transportUrl: session.transportUrl
))

The three values come from your backend. See Mint a session on your backend.

Install

Add the package in Xcode with File > Add Package Dependencies… and the URL https://github.com/SpekoAI/client-swift.git, or in Package.swift:

dependencies: [
    .package(url: "https://github.com/SpekoAI/client-swift.git", from: "0.1.0"),
],
targets: [
    .target(name: "MyApp", dependencies: [.product(name: "SpekoClient", package: "client-swift")]),
]

The SDK needs iOS 15 and Xcode 16.3 or later.

0.1.0 is not published yet. Until the SpekoAI/client-swift mirror is tagged, add packages/client-swift as a local package.

CocoaPods works too: pod 'SpekoClient', '~> 0.1'. That build is LiveKit-only: Pipecat's iOS client is not on CocoaPods. Use Swift Package Manager if your agents run on Pipecat.

Platform setup

Microphone permission

Add a purpose string to your app's Info.plist. iOS terminates the app on the first mic access without it.

<key>NSMicrophoneUsageDescription</key>
<string>Talk to the voice assistant.</string>

create asks for record permission before it joins the room. A denial throws MICROPHONE_FAILED and leaves the session token unused. Retry the same session text-only with micEnabled: false.

Audio session

During a call the SDK sets AVAudioSession to .playAndRecord, mode .voiceChat, with .defaultToSpeaker and .allowBluetooth. Your previous category, mode and options come back when the call ends. Pass manageAudioSession: false if your app owns the audio session, for example with CallKit.

Background audio

The SDK does not force background audio. To keep a call alive with the screen locked, turn on Background Modes > Audio, AirPlay, and Picture in Picture in your target.

Quickstart

  1. Mint a session on your backend and return transport, transportToken and transportUrl.
  2. Add NSMicrophoneUsageDescription.
  3. Create the conversation and read its events.
import SpekoClient

struct SessionResponse: Decodable {
    let transport: String
    let transportToken: String
    let transportUrl: String
}

@MainActor
final class CallModel: ObservableObject {
    @Published var conversation: VoiceConversation?

    func start(backend: URL) async throws {
        var request = URLRequest(url: backend)
        request.httpMethod = "POST"
        let (data, _) = try await URLSession.shared.data(for: request)
        let session = try JSONDecoder().decode(SessionResponse.self, from: data)

        let conv = try await VoiceConversation.create(options: VoiceConversationOptions(
            transport: SpekoTransport(rawValue: session.transport) ?? .livekit,
            transportToken: session.transportToken,
            transportUrl: session.transportUrl
        ))
        conversation = conv

        for await event in conv.events {
            if case let .disconnect(details) = event {
                print("Call ended: \(details.reason)")
            }
        }
    }

    func mute(_ muted: Bool) async throws { try await conversation?.setMicMuted(muted) }

    func end() async { await conversation?.endSession() }
}

events is the primary API. It buffers from the start of create. A loop that starts after create returns still sees .connect. It has one consumer, keeps the newest 256 unread events, and finishes after .disconnect. For UI state, observe status, mode and transcript instead.

Render the transcript

Read transcript, not raw messages. Transcript updates are cumulative: a segment arrives again and again with growing text. The SDK merges them into one entry per speaker turn, ordered by startedAt. VoiceConversation is an ObservableObject, so SwiftUI observes it directly.

struct TranscriptView: View {
    @ObservedObject var conversation: VoiceConversation

    var body: some View {
        List(Array(conversation.transcript.enumerated()), id: \.offset) { _, line in
            Text("\(line.source == .agent ? "Agent" : "You"): \(line.text)")
                .italic(!line.isFinal)
        }
    }
}

Example app

packages/client-swift/Examples/SpekoDemo is a SwiftUI app with a transcript, mute, typed chat and end. Open SpekoDemo.xcodeproj, pick an iPhone simulator or device, and Run. Paste your backend URL, or a token and URL, then tap Start call.

API reference

VoiceConversationOptions

FieldTypeRequiredDefaultDescription
transportTokenStringYesFrom your backend.
transportUrlStringYesFrom your backend.
transportSpekoTransportNo.livekit.livekit or .daily, from the session response.
micEnabledBoolNotruefalse joins text-only. No permission prompt.
audioConstraintsAudioConstraintsNoall trueechoCancellation, noiseSuppression, autoGainControl.
manageAudioSessionBoolNotrueConfigure and restore AVAudioSession.

VoiceConversation

The class is @MainActor. Keep a strong reference until you call endSession().

MemberDescription
static create(options:) async throwsConnects and returns the live conversation.
eventsAsyncStream<VoiceConversationEvent>. Buffered (newest 256), one consumer.
status@Published ConversationStatus: .connecting, .connected, .disconnecting, .disconnected.
mode@Published ConversationMode: .speaking while the agent talks, else .listening.
transcript@Published [ConversationMessage]: the reconciled transcript.
getId()LiveKit room name, or the Daily room name.
isOpen()true while connected.
endSession() asyncLeaves the call. Safe to call twice.
setMicMuted(_:) async throwsNo-op on a text-only session.
setVolume(_:)Agent volume, 0.0 to 1.0. LiveKit only.
sendChatMessage(_:) async throwsTyped user turn. The agent answers by voice.
canPlaybackAudiotrue while connected. iOS has no autoplay block.
startAudioPlayback() asyncNo-op, kept for parity.

Events

VoiceConversationEventWhen
.connect(conversationId:)The session is live. Fires inside create.
.disconnect(details)The call ended. details.reason is .user, .agent, .error, .timeout or .unknown.
.message(message)One raw transcript update.
.transcript(messages)The full reconciled transcript after an update.
.statusChange(status)Status changed.
.modeChange(mode)The agent started or stopped speaking.
.error(error)Non-fatal error. The call continues.

The browser's onAudioPlaybackBlocked has no iOS equivalent.

ConversationMessage

FieldTypeDescription
sourceMessageSource.agent or .user.
textStringThe turn's text so far.
isFinalBoolfalse while the turn is still being transcribed.
segmentIdString?Stable key of the turn.
startedAtDate?When the turn began.

Errors

SpekoClientError carries a code. Failures in create are thrown. Failures during a call arrive as .error on events.

CodeCauseWhat to do
MICROPHONE_FAILEDRecord permission denied, mic publish failed, or the mic died mid-call.Ask the user to allow the mic in Settings, or retry with micEnabled: false.
CONNECTION_FAILEDThe transport rejected the join.Mint a fresh session. Tokens are short-lived.
CONNECTION_TIMEOUTDaily only: the agent was not ready within 30 seconds.Retry with a new session.
NOT_CONNECTEDA send ran before the call connected.Wait for .connected.
UNSUPPORTED_TRANSPORT.daily on a build without the Daily SDK: macOS or CocoaPods.Install with Swift Package Manager on iOS.
INVALID_OPTIONStransportToken or transportUrl is empty.Check your backend response.

Limitations

  • RealtimeVoiceConversation (provider-direct speech-to-speech), CallListener and the embed element are not yet available on mobile.
  • Daily runs on iOS only, through Swift Package Manager. Daily ships an iOS-only binary.
  • setVolume has no effect on Daily. Pipecat's iOS client exposes no playback volume.
  • On Daily, getId() returns the room name. Pipecat's iOS client does not expose the session id.

Troubleshooting

SymptomFix
The app crashes when the call startsNSMicrophoneUsageDescription is missing from Info.plist.
MICROPHONE_FAILED on every callThe user denied the mic earlier. Link to Settings, or join text-only.
CONNECTION_FAILED on the second callThe token was reused. Mint one session per call.
Agent audio plays from the earpieceYour app reset the audio session. Leave manageAudioSession on, or set .defaultToSpeaker yourself.
Call stops when the app goes to the backgroundTurn on the Audio background mode.
Transcript repeats wordsYou appended raw .message events. Render transcript instead.

Next step

On this page