Deflection dashboards reward bots that never give up; users reward bots that know when to. The difference between a helpful assistant and an abandonment machine is engineered escalation: confidence thresholds, an honest "I don't know," frustration detection, and a warm handoff that carries the full context to a human.
Tier: Core · Workflow area: Assistants & RAG · ~35 minutes
This playbook designs the exit paths for your support bot — whether it is ServiceNow Virtual Agent, Freshservice Freddy, or a custom RAG chatbot you built in the labs — so that every conversation ends in one of three deliberate states: answered, honestly declined, or escalated with context. It implements UC-42's core promise: deflection without user abandonment.
Define the three legitimate endings. Write them down before touching config: Answered (grounded response, user confirms or silently leaves after the answer), Honest decline ("I don't know this one — here is the fastest human path"), and Escalated (warm handoff with context). Any conversation ending another way — loop, dead end, silent timeout — is a defect, not a deflection.
Make the model grade its own grounding. In a custom RAG bot, wrap the answer step with a refusal-capable prompt:
Answer the user's question using ONLY the knowledge-base passages below.
Rules:
- If the passages fully answer the question, answer in under 120 words
and cite the article ID.
- If the passages partially answer it, answer the covered part and say
exactly what you could not verify.
- If the passages do not answer it, reply with the single token
NO_ANSWER — do not guess, do not apologize at length.
Question: {user_question}
Passages: {retrieved_chunks}
In code, treat two signals as the confidence gate: top retrieval score below your cutoff (start around 0.55 cosine similarity and tune against logs) or a NO_ANSWER reply. Either one routes to the honest-decline flow. In vendor platforms, the equivalent knobs live in the generative-answer settings — search the admin console for "confidence threshold" or "fallback topic" and record the path in your runbook.
Write the honest decline like a professional, not an apology. One template, used everywhere:
I don't have a reliable answer for that — and I'd rather connect you
than guess. I can raise a ticket with what you've told me so far, or
type "agent" to reach a person now. Nothing you've typed is lost.
The three ingredients are non-negotiable: admit the limit, offer two concrete next steps, promise context is preserved.
Detect frustration with cheap triggers plus one classifier. Hard triggers first — they need no ML: the user types agent, human, person, or help alone; the same intent fires twice without resolution; the user rephrases the same question a third time; more than 60 seconds of profanity-adjacent input. For the softer cases, run a lightweight classifier over each user turn:
Classify the user's latest message in this support chat as one of:
CALM, CONFUSED, FRUSTRATED, URGENT.
Return the label and a 5-word reason. Message: "{turn}"
Prior turns: {last_3_turns}
FRUSTRATED or URGENT plus any hard trigger means escalate now, mid-flow, without finishing the decision tree.
Build the warm-handoff payload. Escalation that drops the user into a queue with a blank ticket is abandonment with extra steps. The handoff carries:
ESCALATION HANDOFF — chat #8841
User: A. Sharma (asked about: VPN fails after password change)
Bot attempted: KB-0312 (VPN re-auth steps) — user reports step 3 fails
Frustration signal: same question rephrased 3x
Suggested category/priority: Network / P3
Full transcript attached. User was told a human sees this context.
Generate the summary with the LLM, but pin the transcript verbatim beneath it — the agent picking it up must be able to check the summary against the source.
Guarantee the bypass. The word "agent" reaches a human (or creates a ticket, off-hours) from any point in any flow, including mid-form. Test it from ten random states. A single flow that swallows the keyword is a trapped user.
Rehearse the failure cases before launch. Feed the bot ten deliberately unanswerable questions and ten angry rephrasing sequences from a test account. Every run must end in a clean decline or a warm handoff. File anything else as a launch blocker.
NO_ANSWER routes to decline; hard triggers OR classifier routes to handoff; the handoff posts the payload to your ITSM API and returns the ticket number to the user in-chat.Handoff payloads and transcripts carry names, asset details and whatever users pasted in frustration — keep them inside your ITSM tenant, redact them before any external LLM call, and never quote a user's transcript in a public channel.
Design the three endings first — answered, honestly declined, escalated with full context — and tune thresholds from real conversation logs; a bot that hands off gracefully at the right moment deflects more over time than one that fights to the last turn, because users come back to it.