Monitor real calls
Score every real call against plain-language checks, group failures into issues, and get alerted.
Monitoring tells you how an agent does with real callers. After each real call ends, an AI judge scores it against a list of checks. Each call gets one result. Failed calls are grouped into issues, and alerts tell you when something gets worse.
Open the agent and select Monitoring. The page has three parts: Health, Issues, and Recent calls. Checks and Alerts open from the buttons at the top.
What counts as a real call
Monitoring counts real calls only. These calls are never counted:
- Calls from Evaluation: test runs, reliability gates, and red-team runs.
- Calls placed from the dashboard's test buttons, such as Place call and the web test panel.
- Calls created with
"test": truein the sessionmetadata. See Marking test calls. - Calls you marked as a test in Monitoring.
Test results stay under Evaluation.
Checks
Open Monitoring → Checks. An agent has three kinds of checks.
Basic checks
Every agent has five basic checks. They are the same for every agent and always on.
| Check | What the judge asks |
|---|---|
| No made-up facts | Did the agent avoid stating figures, policies, dates, names, availability, or promises that are not in its instructions, tool results, or what the caller said? Facts that may come from one of the agent's tools, and routine disclosures such as "this call is recorded", do not fail it. |
| Follows its rules | Did the agent stay within the rules in its instructions: what it must always do, must never do, and must not discuss? |
| Answers what the caller asked | If the caller asked something, did the agent answer it or say clearly why it could not? A call-screening prompt is not a question. |
| No repeating loops | Did the agent avoid repeating a question the caller already answered, or going back to a step it already finished? Asking again because the answer was unclear or did not fit the options is not a loop. |
| Ends the call properly | If the conversation reached its natural end, did the agent wrap up or say goodbye instead of cutting off or going silent? |
Ends the call properly does not apply when the caller hung up first or stopped answering. The agent cannot close a call that nobody is on.
Goal checks
Goal checks test what this agent is for, such as Confirms the patient's name before giving results.
Speko drafts goal checks for you from the agent's instructions and its workflow step instructions. Each check shows where it came from: the agent instructions or a named workflow step.
- Speko drafts checks when you create an agent.
- Speko drafts again when you deploy changed instructions. If the instructions did not change, nothing is drafted.
- Drafted checks are on right away. Click Turn off on any check you do not want.
- To draft now, click Draft from instructions.
A redraft never overwrites a check you already have. If you edited a check or turned it off, your version stays.
To add your own check, type it in plain words, for example Never promises a start date, and click Add check. It is on right away. Click Edit to change the wording of any check.
Only checks that are on judge calls. A check the judge cannot decide for a call, because nothing in the call touched it, is marked Did not apply and never fails the call.
Measured from call records
Some failures are read from the call record, not from the AI judge. Each one fails the call as a system error.
| Check | Fails when |
|---|---|
| Agent never spoke | The caller spoke and the agent never answered. |
| Long silence | The caller waited 8 seconds or more for the agent to reply. |
| Call dropped | The session ended with an error, not a hang-up. |
| Audio failure | Speech recognition or the voice provider failed during the call. |
Call results
Every real call gets exactly one result. Speko applies these rules in order and uses the first one that matches.
| Result | Meaning |
|---|---|
| System error | The call broke: a measured check failed, the session failed, or Speko could not analyse the conversation. This is a platform problem, not something to fix in the instructions. |
| Handled correctly | Nobody could be helped, and ending the call was right: voicemail, an answering machine, no answer, a busy line, a declined call, a wrong number, or the caller hung up or stopped answering. |
| Agent mistake | At least one check failed, either a basic check or a goal check. |
| Success | No check failed. |
Success rate = (success + handled correctly) ÷ scored calls.
Health
The Health part of the Monitoring page shows numbers for the last 24 hours, 7 days, or 30 days, each compared with the period before:
- Success rate, with a daily chart and the 80% goal line.
- Real calls: how many real calls were scored.
- Response time: the median time the agent took to reply, with the 95th percentile.
- Agent mistakes, System errors, and Handled correctly.
Click a number to switch the chart to it.
Issues
Failed calls with the same cause are grouped into one issue. One issue is one failing check or one kind of system error, such as Long silence before answering. Each issue shows:
- whether it is an Agent mistake or a System error
- how many calls it affected, and when it was last seen
- its trend: New, Rising, Falling, or Steady
Open an issue to read the judge's latest reason and the calls behind it. Click Make test case to turn a failing call into a test case under Evaluation, so you can test the fix before you deploy it.
Calls
Open Recent calls → View all to see every scored real call. Filter by result or by issue, or search. Open a call to see:
- the recording and transcript
- the call result and, for a failure, the reason
- the verdict on each check: passed, failed, or did not apply, with the judge's reason
If a call was really a test, open it and click Mark as test call. It leaves the numbers, issues, and alerts. To undo, click Undo in the message, or Count as real call on the same call.
Alerts
- Open Monitoring → Alerts.
- Under Send to, add a Slack channel, one or more email addresses (comma-separated), or a webhook URL, and click Save.
- Turn on the alerts you want.
| Alert | Fires when |
|---|---|
| Success rate drops below 60% | The success rate over the last 20 judged real calls is below 60%. It waits until at least 10 calls are judged. |
| A call has a system error | A real call ends in a system error: a dropped call, a silent agent, or a provider failure. |
| A new issue appears | A call fails for a reason this agent has not had in the last 30 days. |
| Responses get slow | A call's 95th-percentile response time is above 3 seconds. |
Each alert fires at most once every 10 minutes, so a burst of bad calls sends one alert. Recent alerts lists what fired.
- Slack: alerts post through the connected Speko Slack app. Invite the app to the channel first.
- Email: each address gets the alert.
- Webhook: Speko sends a signed
POSTusing Standard Webhooks. Verify it with the signing secret shown on the page.
API
Every Monitoring action is also available through the API. All paths are under /v1/agents/{agent_id}.
| Method and path | What it does |
|---|---|
GET /monitoring/summary?period=7d | Health numbers. period is 24h, 7d, or 30d. |
GET /monitoring/issues?period=7d | Issues for the period. |
GET /monitoring/issues/{issue_key} | One issue and its calls. |
POST /monitoring/calls/{session_id}/test | Body {"test": true} marks a call as a test. {"test": false} undoes it. |
GET /criteria | List goal checks. |
GET /criteria/basic | List the basic checks. |
POST /criteria | Add a goal check: slug, name, prompt, and optional status (on by default). |
PATCH /criteria/{criterion_id} | Change a check, or set status to on or off. |
DELETE /criteria/{criterion_id} | Delete a check. |
POST /criteria/suggest | Draft checks from the agent's instructions now. |
GET /monitors, POST /monitors | List or create alerts. |
PATCH /monitors/{monitor_id}, DELETE /monitors/{monitor_id} | Change or delete an alert. |
GET /monitors/{monitor_id}/events | Alerts that fired. |
A check slug uses lowercase letters, digits, and underscores. Slugs that start with basic_ are reserved.