Jev in CrewAI: a Flow @router with a confidence threshold

What you will build

A Flow that routes a support ticket to the billing, technical or sales agent. It is the same scenario as our n8n guide, this time in CrewAI.

• ask_jev, marked @start(), sends the ticket to Jev with two questions: which team, and is it urgent.
• pick_team, marked @router, returns the branch name based on Jev’s confidence.
• run_team_agent listens to the three branches and starts the chosen team’s agent.
• send_to_human receives the tickets nobody could classify.
• The Flow state doubles as the log: model version, probabilities, confidence and path taken.

Why a @router rather than a manager agent

In a crew, routing is often left to an LLM. A manager agent reads the ticket and delegates. Each decision then costs an LLM call, and its reasoning lives in free text.

Flows are CrewAI’s orchestration layer. Python code decides what runs next, and crews or agents do the work. The @router is the natural place for a typed decision. Jev returns a team, probabilities and a confidence there. Your code applies the threshold.

Before you start

• Python 3.10 or later.
• A Jev API key, created in the TypeSafe console, in the TYPESAFE_API_KEY variable. Jev sign-ups are temporarily paused, so check your access.
• The key for your current LLM. The example uses Claude Haiku 4.5 through ANTHROPIC_API_KEY. Swap in the model your agents use.

pip install "crewai[anthropic]" typesafe-sdk

CrewAI’s anthropic extra pins its own version of the Anthropic SDK. Install this project in a dedicated virtual environment to avoid conflicts.

The full code

The file is under 80 lines. Copy it as is, then read the next sections to adapt it.

import logging

from crewai import LLM, Agent
from crewai.flow.flow import Flow, listen, or_, router, start
from pydantic import BaseModel
from typesafe_sdk import Choice, Noul, TypeSafeClient, TypeSafeError

JEV_MODEL = "jev-1.13.0"  # pinned: a threshold is calibrated for one model version
THRESHOLD = 0.6  # starting point only, calibrate it on your own tickets
SHADOW = True  # start here: Jev is logged, your current LLM still decides
TEAMS = {"billing": "Payments, invoices, refunds", "technical": "Bugs, outages, integrations",
         "sales": "Pricing, quotes, new accounts"}
QUESTIONS = {
    "department": Choice(instructions="Which team should handle this ticket?", criteria=TEAMS),
    "urgent": Noul(instructions="Does the message express urgency?")}
PROMPT = "Which team should handle this ticket: billing, technical or sales? One word only.\n\n"

jev = TypeSafeClient(model=JEV_MODEL)  # reads TYPESAFE_API_KEY, retries 429 and 5xx itself
current_llm = LLM(model="anthropic/claude-haiku-4-5", temperature=0)  # the model you use today
log = logging.getLogger("ticket_routing")
AGENTS = {team: Agent(role=f"{team.title()} support agent", goal=f"Resolve tickets about: {scope}",
                      backstory="You answer customer tickets clearly and briefly.", llm=current_llm)
          for team, scope in TEAMS.items()}

class TicketState(BaseModel):
    ticket: str = ""
    jev_choice: str = ""
    confidence: float = 0.0  # stays 0.0 when Jev fails, which sends the ticket to the fallback
    probabilities: dict[str, float] = {}
    urgent: float | None = None
    model: str = ""
    error: str = ""
    department: str = ""
    decided_by: str = ""
    threshold: float = THRESHOLD

class TicketFlow(Flow[TicketState]):
    @start()
    def ask_jev(self):
        try:
            res = jev.system_one(state=self.state.ticket, questions=QUESTIONS)
        except TypeSafeError as err:  # 401, 422, or 429/529 once the SDK retries are spent
            self.state.error = type(err).__name__
            return
        dept = res.choices["department"]
        self.state.jev_choice, self.state.confidence = dept.choice, dept.confidence
        self.state.probabilities, self.state.model = dept.probabilities, res.model
        self.state.urgent = res.nouls["urgent"].noul

    @router(ask_jev)
    def pick_team(self):
        s = self.state
        if SHADOW or s.confidence < THRESHOLD:  # Jev unsure or down: your current LLM decides
            answer = current_llm.call(PROMPT + s.ticket)
            s.department, s.decided_by = answer.strip().lower(), "llm"
        else:
            s.department, s.decided_by = s.jev_choice, "jev"
        log.info(s.model_dump_json(exclude={"ticket"}))
        return s.department if s.department in TEAMS else "human"  # an unknown label runs nothing

    @listen(or_("billing", "technical", "sales"))
    def run_team_agent(self):
        return AGENTS[self.state.department].kickoff(self.state.ticket).raw

    @listen("human")
    def send_to_human(self):
        return "Sent to the human triage queue."

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    text = "Hi, my September invoice shows up twice on my statement. Can you refund the duplicate?"
    print(TicketFlow().kickoff(inputs={"ticket": text}))

1. The start step that calls Jev

The client pins the model version. TypeSafeClient(model="jev-1.13.0") sets the model for every call. The jev-latest alias moves with each release, and your thresholds move with it.

Jev gets two typed questions. A choice picks the team among three described options. A noul returns the probability that the message is urgent. The step stores the answers in the Flow state, a Pydantic model.

The SDK already retries 429 and 5xx twice, including the 529 overload code. It honours the Retry-After header. If the error persists, the step records its name and returns. The confidence then stays at 0, which sends the ticket to the fallback.

2. The @router that picks the branch

The decision lives in pick_team. The @router(ask_jev) decorator runs it as soon as the Jev step finishes. The method returns a string, and that string triggers the listeners with the same name.

• Confidence above the threshold: the branch is Jev’s choice.
• Confidence below it, a Jev error or shadow mode: your current LLM picks, through LLM.call.
• An LLM answer outside the list: the branch becomes human.

That last guard matters. We checked it with CrewAI 1.15.22: a label no listener waits for starts nothing. The Flow ends without an error, and kickoff() simply returns the label. A ticket can vanish silently.

The starting threshold is 0.6, the value used in TypeSafe’s examples. It is not a production value. The “Choosing the right threshold” section explains how to measure it.

3. The branches that start the agents

run_team_agent listens to the three labels through or_. It starts the chosen team’s agent with Agent.kickoff, which returns a result whose text is in .raw.

If each team has its own crew, write one listener per label. Each one calls its crew with kickoff(inputs=...). Give these methods a name that differs from the label. CrewAI 1.15 rejects a listener named billing that listens to "billing", because it would trigger itself in a loop.

4. Start in shadow mode

Keep SHADOW = True for the first weeks. Jev answers on every ticket and its answer is logged. But your LLM still picks every branch. Your customers see no difference.

For each ticket, the log gives you Jev’s choice and your LLM’s choice. Measure how often they agree before Jev makes a single real decision. Then set SHADOW to False to turn the threshold on.

5. The decision log

The router writes the Flow state as JSON, without the ticket text. Here is the line for a billing ticket handled by Jev, with the sample values from the docs:

{"jev_choice":"billing","confidence":0.81,"probabilities":{"billing":0.88,"technical":0.12,"sales":0.0},"urgent":0.12,"model":"jev-1.13.0","error":"","department":"billing","decided_by":"jev","threshold":0.6,"id":"66faddc1-c1a6-47f6-a45b-61601be72b85"}

• model: the exact version that answered, as returned by the API.
• probabilities and confidence: what lets you recalibrate the threshold.
• decided_by: the path taken, jev or llm.
• id: the Flow identifier, added by CrewAI. It ties the decision to the rest of the run.

How we tested this code

We ran this file as is, with crewai 1.15.22 and typesafe-sdk 0.7.1. The real CrewAI Flow and the real Jev SDK were running. Jev’s HTTP calls went to a fake API that returned the responses documented by TypeSafe. The agents and the LLM were replaced by test doubles.

• Confidence of 0.81: Jev picks billing, the LLM is not called.
• Confidence of 0.10: the LLM picks, the technical branch runs.
• Repeated 529 error: the LLM picks, the error stays in the log.
• An LLM answer outside the list: the ticket goes to human.
• Shadow mode: the LLM decides, Jev’s choice stays in the log.

We also ran the Jev part of this code against the real API on 23/09/2026: jev-1.13.0 answered billing, confidence 1, in 403 ms from France, for 368 input tokens.

Choosing the right threshold

You cannot guess a threshold. You measure it on your data, in your language.

1. Take a few hundred past tickets where the right team is known.
2. Run them through the Flow in shadow mode.
3. For each threshold between 0.5 and 0.95, compute the share of tickets Jev would handle alone, and its error rate on that share.
4. Keep the lowest threshold whose error rate you can accept.

Jev is trained on English first. TypeSafe recommends testing on your own content before using it in another language. The n8n guide walks through the same method.

Before production

• Data: Jev is hosted in the United States. Pseudonymise personal data before sending it, and have your DPO approve the transfer.
• Version: keep jev-1.13.0 pinned. Recalibrate before moving to the next one.
• Log: keep every decision, including the LLM’s.
• Costs: Jev replaces the routing decision, not the agents’ work. The agents are still billed at their LLM’s price.
• Sensitive decisions: high confidence is not authorisation. Any outcome unfavourable to a person must go through a human.

Want to see Jev decide before writing any code? Try the demo with your own sample tickets.

Jev and TypeSafe are trademarks of TypeSafe AI, Inc. This article is independent and not affiliated with TypeSafe AI.