The turn pipeline
Every message runs the same six stages in the same order. Three fast small-model calls around one large one, then a check. The order is the contract — it is what makes the behaviour predictable enough to debug.
A small model classifies the message: language, intent, safety, sentiment.
Chitchat and abuse should never pay for retrieval or a large-model call. This is also where a request to speak to a human is recognised as one.
Turns a follow-up into a standalone query using conversation history.
“And its price?” retrieves nothing on its own. Rewritten against the previous turn it becomes a query that actually matches a document.
Hybrid vector plus full-text search, with tables, actions and procedures loaded in parallel.
Vector search finds paraphrases; text search finds exact identifiers like order numbers and SKUs. Fusing both beats either alone.
A cross-encoder scores every candidate against the query and orders them.
Retrieval is recall-oriented and noisy. This is the stage that decides whether there is a real answer here — below the threshold, the agent abstains.
The large model answers, asks a clarifying question, or proposes a tool call.
The only stage that writes prose, and the only one allowed to propose an action. It cannot execute one.
A second model checks the answer is grounded in the retrieved chunks and addresses the question.
Catches the confident paragraph assembled from adjacent-but-wrong context. A failure becomes an abstention, never an unvalidated answer.
Why abstention is a feature
After reranking, every candidate carries a score. If the best one falls below the configured threshold, the pipeline stops and the agent says it does not know — it does not hand the model a pile of weak context and hope.
This is the single most important behaviour in the product. A support agent that invents a refund policy costs more than one that admits ignorance, because a wrong answer is acted on and a missing one is escalated.
Outcomes
Every turn ends in exactly one outcome, recorded on the trace:
| Outcome | Meaning |
|---|---|
ANSWERED | Grounded answer produced and validated. |
CLARIFIED | The question was ambiguous; the agent asked rather than guessed. |
ABSTAINED | Nothing retrieved above the confidence threshold, or validation failed. |
ESCALATED | Handed to a human, with the full conversation attached. |
BLOCKED | A guard refused an action. The reason is recorded — see below. |
ERROR | A stage failed. The turn is logged rather than silently dropped. |
Block reasons
When a turn is blocked, the trace says which guard did it. These are checks in the execution path, not instructions in a prompt:
NEVER_TESTED— the action has never passed a test call, so it is not offered to the model at allIDENTITY_REQUIRED— the action needs a verified visitor and this one is anonymousCONFIRMATION_REQUIRED— a write was proposed; it executes on the next turn, after the customer agreesNOT_AVAILABLE— the action is disabled, or does not exist in this workspace
What a trace records
Every turn writes one trace row, readable from any conversation in the Inbox. It holds the raw query, the rewritten query, the gate’s intent classification, how many candidates were retrieved, the top chunks with their vector, text and rerank scores, whether the result fell below threshold, the model used, input and output token counts, the groundedness and relevance verdicts, per-stage latency, and the computed cost.
That is deliberately more than you need day to day. It exists so that when someone asks “why did it say that”, the answer is a record rather than a theory.
Cost
Three small-model calls and one large one per turn. The small calls are cheap enough to be rounding errors; the generate stage dominates. Cost per turn is computed from real token counts and rolled up per conversation and per resolution on the Analytics page.
Autonomous resolution is computed on a schedule rather than at write time — a conversation counts as resolved when it goes quiet without escalation for the configured window. See Escalation.