Test before you deploy
Run simulated callers against a draft and catch failures before real callers do.
A test case is a simulated call. Speko plays a caller with a goal, lets your agent respond, and scores the result. Test runs are free and never count toward Monitoring.
Generate test cases
- Open the agent and select Evaluation in the sidebar.
- Click Generate from agent.
Speko reads the agent's instructions, tools, and knowledge and writes 4 to 12 cases. Click Regenerate after a big change. It adds new cases and keeps the ones you have.
Write your own
Click Scenario and describe the call in plain words, for example: "Caller wants to move Tuesday's appointment to Friday but doesn't remember the time." The case passes when the simulated caller reaches its goal.
Click Phone call instead to test a real outbound call. Your agent dials the number you enter from its assigned phone number. You can choose the codec and network quality to simulate a bad line.
To check something specific, open Advanced options and pick a check:
| Check | Passes when | Scored by |
|---|---|---|
| Calls a tool | The agent calls the named tool, or any tool if left blank | Rule |
| Says a phrase | The agent says the phrase, ignoring case | Rule |
| Responds in time | The agent starts speaking within the limit after the caller stops. Default 3.5 s. | Rule |
| Handles voicemail | The test call reaches voicemail, and the agent's message contains your phrase if you set one. Phone calls only. | Rule |
| Switches language | The agent replies mainly in the target language | AI judge |
| No made-up facts | The agent invents no facts, prices, policies, or promises | AI judge |
| Judged behavior | Your own rule, written in plain words | AI judge |
Rule checks are exact and can block a deploy. AI-judged checks are advisory.
Pick a tough caller
Each case can use a caller character. A character has its own voice, pace, and habits: a caller in a noisy cafe, someone who interrupts, someone who says "mm-hm" over the agent. Set it with the chip on each row.
Browse them all under Evals → Personas, and click Hear this persona to listen. You can also turn a real call into a persona with Import from a real call.
Run and read results
Click Run on a case. Results show Pass, Fail, Inconclusive, or Incomplete (the call hit its turn limit before the goal was confirmed).
When a draft exists, runs test the version selected by the Testing: Draft / Live switch in the bottom bar. Test the draft, then deploy.
Click a result to open it. You can listen to the call and read the evidence for each check. When a judged check fails, click Improve the prompt. Speko drafts a fix to the instructions. Review it and click Apply prompt, then run the case again.
Apply prompt updates the live agent right away. It does not create a draft.
Share scenarios across agents
A test case belongs to one agent. A scenario belongs to the workspace. Create one under Evals → Scenarios → Add scenario. Describe the caller's goal and what success looks like. Then attach it to any number of agents. Attached scenarios run in each agent's full test run. Track record on the scenario page shows every result.
Block deploys on failures
By default, a failed test never stops a deploy. To change that, a workspace owner or admin turns on Block deploys on a failed check under Settings → Organization → Advanced.
When it is on, Deploy is refused while the live version has a failed rule check. The bar shows See which case failed and Deploy anyway. Inconclusive results never block.