Use each recording to produce an acceptance question. The following is a proposed desk worksheet—not a test we have executed and not a claim about any supplier’s capabilities.
| Trial scenario | Evidence to request | Proposed acceptance question |
|---|---|---|
| Caller requests an eligible appointment | Recording, tool trace and final booking record | Does the correct appointment exist under the approved rules? |
| Caller requests unavailable or prohibited work | Agent response and any resulting system changes | Did the agent avoid inventing availability or making an unauthorised commitment? |
| Caller corrects a name, address or number | Audio, transcript and saved record | Was the correction applied everywhere it mattered? |
| Caller interrupts or there is background noise | Audio and observed outcome | Did the agent recover without losing the task or repeatedly talking over the caller? |
| Scheduling or CRM write fails | Error trace, retry behaviour and caller message | Did the agent avoid falsely confirming completion? |
| Caller requests a person | Routing events and receiving-agent context | Did the person receive the caller and the necessary context? |
| Nobody accepts the transfer | Fallback record and follow-up ownership | Was the caller given an approved next step without a routing loop? |
| A model, prompt or integration changes | Before-and-after regression results | Is the change acceptable across the relevant call types? |
What should the investment case measure?
Our recommended governing measure is:
Total workflow cost ÷ unique, correctly completed, policy-compliant tasks.
Use your own measured inputs. Include failed attempts in the costs; distinguish autonomous completion from human rescue; and avoid counting a repeat call as a second successful task.
For an AI voice agent ROI calculator or AI call center cost-per-resolution comparison, build separate lines for:
- Supplier usage and platform charges.
- Implementation and integration.
- Monitoring and quality review.
- Human follow-up and corrections.
- Repeat contacts.
- Actual cash savings, avoided hiring and possible recovered contribution.
Treat this as a proposed accounting structure, not a universal industry standard. Ask finance to approve the definitions and the attribution assumptions before interpreting a supplier’s reported savings.
What should leave the viewing meeting?
- One bounded workflow and a list of prohibited actions.
- A named human fallback owner with actual coverage hours.
- A baseline drawn from comparable calls.
- A test pack with expected conversation outcomes and system states.
- A normalised quote with commercial exclusions made explicit.
- A risk review covering the specific calling and data flows.
- A written expand, narrow, continue or stop decision rule.