iOS (Swift)
Run a Speko voice conversation in a native iOS app with the SpekoClient Swift package.
SpekoClient joins a Speko voice session from an iOS app. It captures the mic, plays the agent, and streams a reconciled transcript. It runs over LiveKit, or Daily when the session says so.
let conversation = try await VoiceConversation.create(options: VoiceConversationOptions(
transport: SpekoTransport(rawValue: session.transport) ?? .livekit,
transportToken: session.transportToken,
transportUrl: session.transportUrl
))The three values come from your backend. See Mint a session on your backend.
Install
Add the package in Xcode with File > Add Package Dependencies… and the URL https://github.com/SpekoAI/client-swift.git, or in Package.swift:
dependencies: [
.package(url: "https://github.com/SpekoAI/client-swift.git", from: "0.1.0"),
],
targets: [
.target(name: "MyApp", dependencies: [.product(name: "SpekoClient", package: "client-swift")]),
]The SDK needs iOS 15 and Xcode 16.3 or later.
0.1.0 is not published yet. Until the SpekoAI/client-swift mirror is tagged, add packages/client-swift as a local package.
CocoaPods works too: pod 'SpekoClient', '~> 0.1'. That build is LiveKit-only: Pipecat's iOS client is not on CocoaPods. Use Swift Package Manager if your agents run on Pipecat.
Platform setup
Microphone permission
Add a purpose string to your app's Info.plist. iOS terminates the app on the first mic access without it.
<key>NSMicrophoneUsageDescription</key>
<string>Talk to the voice assistant.</string>create asks for record permission before it joins the room. A denial throws MICROPHONE_FAILED and leaves the session token unused. Retry the same session text-only with micEnabled: false.
Audio session
During a call the SDK sets AVAudioSession to .playAndRecord, mode .voiceChat, with .defaultToSpeaker and .allowBluetooth. Your previous category, mode and options come back when the call ends. Pass manageAudioSession: false if your app owns the audio session, for example with CallKit.
Background audio
The SDK does not force background audio. To keep a call alive with the screen locked, turn on Background Modes > Audio, AirPlay, and Picture in Picture in your target.
Quickstart
- Mint a session on your backend and return
transport,transportTokenandtransportUrl. - Add
NSMicrophoneUsageDescription. - Create the conversation and read its events.
import SpekoClient
struct SessionResponse: Decodable {
let transport: String
let transportToken: String
let transportUrl: String
}
@MainActor
final class CallModel: ObservableObject {
@Published var conversation: VoiceConversation?
func start(backend: URL) async throws {
var request = URLRequest(url: backend)
request.httpMethod = "POST"
let (data, _) = try await URLSession.shared.data(for: request)
let session = try JSONDecoder().decode(SessionResponse.self, from: data)
let conv = try await VoiceConversation.create(options: VoiceConversationOptions(
transport: SpekoTransport(rawValue: session.transport) ?? .livekit,
transportToken: session.transportToken,
transportUrl: session.transportUrl
))
conversation = conv
for await event in conv.events {
if case let .disconnect(details) = event {
print("Call ended: \(details.reason)")
}
}
}
func mute(_ muted: Bool) async throws { try await conversation?.setMicMuted(muted) }
func end() async { await conversation?.endSession() }
}events is the primary API. It buffers from the start of create. A loop that starts after create returns still sees .connect. It has one consumer, keeps the newest 256 unread events, and finishes after .disconnect. For UI state, observe status, mode and transcript instead.
Render the transcript
Read transcript, not raw messages. Transcript updates are cumulative: a segment arrives again and again with growing text. The SDK merges them into one entry per speaker turn, ordered by startedAt. VoiceConversation is an ObservableObject, so SwiftUI observes it directly.
struct TranscriptView: View {
@ObservedObject var conversation: VoiceConversation
var body: some View {
List(Array(conversation.transcript.enumerated()), id: \.offset) { _, line in
Text("\(line.source == .agent ? "Agent" : "You"): \(line.text)")
.italic(!line.isFinal)
}
}
}Example app
packages/client-swift/Examples/SpekoDemo is a SwiftUI app with a transcript, mute, typed chat and end. Open SpekoDemo.xcodeproj, pick an iPhone simulator or device, and Run. Paste your backend URL, or a token and URL, then tap Start call.
API reference
VoiceConversationOptions
| Field | Type | Required | Default | Description |
|---|---|---|---|---|
transportToken | String | Yes | From your backend. | |
transportUrl | String | Yes | From your backend. | |
transport | SpekoTransport | No | .livekit | .livekit or .daily, from the session response. |
micEnabled | Bool | No | true | false joins text-only. No permission prompt. |
audioConstraints | AudioConstraints | No | all true | echoCancellation, noiseSuppression, autoGainControl. |
manageAudioSession | Bool | No | true | Configure and restore AVAudioSession. |
VoiceConversation
The class is @MainActor. Keep a strong reference until you call endSession().
| Member | Description |
|---|---|
static create(options:) async throws | Connects and returns the live conversation. |
events | AsyncStream<VoiceConversationEvent>. Buffered (newest 256), one consumer. |
status | @Published ConversationStatus: .connecting, .connected, .disconnecting, .disconnected. |
mode | @Published ConversationMode: .speaking while the agent talks, else .listening. |
transcript | @Published [ConversationMessage]: the reconciled transcript. |
getId() | LiveKit room name, or the Daily room name. |
isOpen() | true while connected. |
endSession() async | Leaves the call. Safe to call twice. |
setMicMuted(_:) async throws | No-op on a text-only session. |
setVolume(_:) | Agent volume, 0.0 to 1.0. LiveKit only. |
sendChatMessage(_:) async throws | Typed user turn. The agent answers by voice. |
canPlaybackAudio | true while connected. iOS has no autoplay block. |
startAudioPlayback() async | No-op, kept for parity. |
Events
VoiceConversationEvent | When |
|---|---|
.connect(conversationId:) | The session is live. Fires inside create. |
.disconnect(details) | The call ended. details.reason is .user, .agent, .error, .timeout or .unknown. |
.message(message) | One raw transcript update. |
.transcript(messages) | The full reconciled transcript after an update. |
.statusChange(status) | Status changed. |
.modeChange(mode) | The agent started or stopped speaking. |
.error(error) | Non-fatal error. The call continues. |
The browser's onAudioPlaybackBlocked has no iOS equivalent.
ConversationMessage
| Field | Type | Description |
|---|---|---|
source | MessageSource | .agent or .user. |
text | String | The turn's text so far. |
isFinal | Bool | false while the turn is still being transcribed. |
segmentId | String? | Stable key of the turn. |
startedAt | Date? | When the turn began. |
Errors
SpekoClientError carries a code. Failures in create are thrown. Failures during a call arrive as .error on events.
| Code | Cause | What to do |
|---|---|---|
MICROPHONE_FAILED | Record permission denied, mic publish failed, or the mic died mid-call. | Ask the user to allow the mic in Settings, or retry with micEnabled: false. |
CONNECTION_FAILED | The transport rejected the join. | Mint a fresh session. Tokens are short-lived. |
CONNECTION_TIMEOUT | Daily only: the agent was not ready within 30 seconds. | Retry with a new session. |
NOT_CONNECTED | A send ran before the call connected. | Wait for .connected. |
UNSUPPORTED_TRANSPORT | .daily on a build without the Daily SDK: macOS or CocoaPods. | Install with Swift Package Manager on iOS. |
INVALID_OPTIONS | transportToken or transportUrl is empty. | Check your backend response. |
Limitations
RealtimeVoiceConversation(provider-direct speech-to-speech),CallListenerand the embed element are not yet available on mobile.- Daily runs on iOS only, through Swift Package Manager. Daily ships an iOS-only binary.
setVolumehas no effect on Daily. Pipecat's iOS client exposes no playback volume.- On Daily,
getId()returns the room name. Pipecat's iOS client does not expose the session id.
Troubleshooting
| Symptom | Fix |
|---|---|
| The app crashes when the call starts | NSMicrophoneUsageDescription is missing from Info.plist. |
MICROPHONE_FAILED on every call | The user denied the mic earlier. Link to Settings, or join text-only. |
CONNECTION_FAILED on the second call | The token was reused. Mint one session per call. |
| Agent audio plays from the earpiece | Your app reset the audio session. Leave manageAudioSession on, or set .defaultToSpeaker yourself. |
| Call stops when the app goes to the background | Turn on the Audio background mode. |
| Transcript repeats words | You appended raw .message events. Render transcript instead. |