The hidden bill
A request is a technical event. An outcome is something a person can use: a correctly classified document, a usable campaign, or a caller reaching the right destination. If a cheaper response needs another attempt or a human repair, the first price comparison leaves out part of the bill.
A lesson from AIReceptionist
Our contribution makes a smaller realtime model an explicit configuration option while keeping the full model as the default. That creates a choice; it does not prove that the smaller model handles interruptions, transfers, or caller intent equally well. Those are the outcomes a live evaluation needs to protect.
How to make the decision
Define what counts as an accepted call before testing. Add model usage, telephony, retries, and review effort across the workload, then divide by accepted outcomes. Track response time separately so a cheaper result does not conceal a longer wait. Keep failed attempts in the numerator.
An illustrative comparison
If ten attempts cost $1 and all ten succeed, the cost is $0.10 per accepted outcome. If ten cheaper attempts cost $0.80 but only five succeed, it is $0.16. These are illustrative numbers, not results from AIReceptionist.
About the evidence
Configuration tested locally. Caller outcomes require live evaluation.
Project detail and source links ↗