Skip to content
Patterns

Routing by intent

One choice question decides which handler, queue or team gets a request. When the model is not sure, the request goes to a fallback instead of the wrong place.

When to use it

  • Requests arrive in one place (an inbox, a chat, a form, an agent's input) and need to go to one of several handlers.
  • You route with keyword rules today, and they miss requests written in other words.
  • A wrong route is costly: the customer waits while the request bounces between teams.

How it works

Ask one choice question with one option per destination, and describe each option in a few words. The answer names the pick, its confidence and the probabilities of every option. Route on the pick when the confidence clears your threshold; otherwise send the request to a fallback: a triage queue, a person, or a larger model.

  1. State

    The request

    • “I moved last week, can my order go to the new flat?”

  2. Question

    Which handler?

    • intentchoice
    • change_address · 0.94

  3. confidence ≥ 0.70

    The change_address handler runs.

    confidence < 0.70, or other

    Triage, with the top two options as a hint.

Each option maps to a handler; unsure requests go to triage. Values are illustrative.

The smallest version picks a support queue and sends the unsure messages to triage:

from typesafe_sdk import Choice, TypeSafeClient # Reads TYPESAFE_BASE_URL (https://api.vakyn.com) and TYPESAFE_API_KEY from the environment.client = TypeSafeClient()QUEUES = {    "billing": "Charges, invoices and refunds",    "shipping": "Delivery, tracking and damaged parcels",    "account": "Sign-in, password and profile",}  def route(message: str) -> str:    result = client.system_one(        state=message,        questions={"queue": Choice(instructions="Which queue should handle this?", criteria=QUEUES)},    )    answer = result.choices["queue"]    if answer.confidence >= 0.7:        return answer.choice  # sure enough: route automatically    return "triage"  # unsure: a person decides  print(route("I changed my email address and now the password reset link never arrives."))

The same idea with handlers instead of queue names. Each option maps to a function; an other option with no description catches the requests that fit none of them.

from typesafe_sdk import Choice, TypeSafeClient # Reads TYPESAFE_BASE_URL (https://api.vakyn.com) and TYPESAFE_API_KEY from the environment.client = TypeSafeClient()THRESHOLD = 0.70 # One option per handler. The descriptions are what the model reads; "other" catches the rest.INTENTS = {    "refund": "Wants money back for an order",    "change_address": "Wants to change where an order is delivered",    "cancel_plan": "Wants to stop a subscription",    "bug_report": "Reports something in the app that does not work",    "other": None,}  def start_refund(msg): ...def update_address(msg): ...def cancel_plan(msg): ...def file_bug(msg): ...def send_to_triage(msg, reason): ...  HANDLERS = {    "refund": start_refund,    "change_address": update_address,    "cancel_plan": cancel_plan,    "bug_report": file_bug,}  def handle(message: str):    result = client.system_one(        state=message,        questions={"intent": Choice(instructions="What does the customer want?", criteria=INTENTS)},    )    intent = result.choices["intent"]    if intent.choice in HANDLERS and intent.confidence >= THRESHOLD:        return HANDLERS[intent.choice](message)    # Unsure, or none of the above: a person picks, with the model's top two as a hint.    top_two = sorted(intent.probabilities.items(), key=lambda kv: kv[1], reverse=True)[:2]    return send_to_triage(message, reason=top_two)

Tips

  • Write options that do not overlap. When two options could both be right, the probability splits between them and the confidence drops, so more requests go to the fallback.
  • Add an other option. Without one, the model has to pick a destination even for a request that fits none, and the probabilities spread out.
  • To route on more than one fact, ask them together in the same request: the team as a choice, the urgency as a score. See fan-out.
  • For how to set the threshold and what to do below it, see routing by confidence.

Loading the docs…