Jev in LangChain: a LangGraph router with a confidence threshold

What you will build

A four-node graph that routes a support ticket to billing, technical or sales. It is the same scenario as our n8n guide, written in Python this time.

• ask_jev sends the ticket to Jev with two questions: which team, and is it urgent.
• is_jev_sure is a conditional edge. It compares Jev’s confidence with your threshold.
• accept_jev keeps Jev’s choice when the confidence is high enough.
• llm_fallback calls your current LLM when Jev is unsure or does not answer.
• log_decision logs the model version, the probabilities, the confidence and the path taken.

Why LangGraph for this routing

LangGraph is now the way to route in the LangChain ecosystem. Since LangChain 1.0, the old routing chains such as LLMRouterChain live in the langchain-classic package. Routing is now described with nodes and conditional edges.

This model suits Jev well. The classifier becomes one more node. The threshold rule becomes an edge anyone can read. The rest of your graph does not change.

Before you start

• Python 3.10 or later.
• A Jev API key, created in the TypeSafe console, in the TYPESAFE_API_KEY variable. Jev sign-ups are temporarily paused, so check your access.
• The key for your current LLM. The example uses Claude Haiku 4.5 through ANTHROPIC_API_KEY. Swap in the model you already use.

pip install typesafe-sdk langgraph langchain langchain-anthropic

The full code

The file is under 80 lines. Copy it as is, then read the next sections to adapt it.

import json
import logging
from typing import Literal, TypedDict

from langchain.chat_models import init_chat_model
from langgraph.graph import END, START, StateGraph
from pydantic import BaseModel
from typesafe_sdk import Choice, Noul, TypeSafeClient, TypeSafeError

JEV_MODEL = "jev-1.13.0"  # pinned: a threshold is calibrated for one model version
THRESHOLD = 0.6  # starting point only, calibrate it on your own tickets
SHADOW = True  # start here: Jev is logged, your current LLM still decides
TEAMS = {"billing": "Payments, invoices, refunds", "technical": "Bugs, outages, integrations",
         "sales": "Pricing, quotes, new accounts"}
QUESTIONS = {
    "department": Choice(instructions="Which team should handle this ticket?", criteria=TEAMS),
    "urgent": Noul(instructions="Does the message express urgency?")}
jev = TypeSafeClient(model=JEV_MODEL)  # reads TYPESAFE_API_KEY, retries 429 and 5xx itself
log = logging.getLogger("ticket_routing")

class Route(BaseModel):
    department: Literal["billing", "technical", "sales"]

# Your current LLM router, unchanged. It only runs when Jev is unsure or unavailable.
llm = init_chat_model("anthropic:claude-haiku-4-5", temperature=0).with_structured_output(Route)

class Ticket(TypedDict, total=False):
    ticket: str
    jev_choice: str
    confidence: float
    probabilities: dict[str, float]
    urgent: float
    model: str
    error: str
    department: str
    decided_by: str

def ask_jev(state: Ticket) -> Ticket:
    try:
        res = jev.system_one(state=state["ticket"], questions=QUESTIONS)
    except TypeSafeError as err:  # 401, 422, or 429/529 once the SDK retries are spent
        return {"error": type(err).__name__, "confidence": 0.0}
    dept = res.choices["department"]
    return {"jev_choice": dept.choice, "confidence": dept.confidence,
            "probabilities": dept.probabilities, "urgent": res.nouls["urgent"].noul,
            "model": res.model}

def is_jev_sure(state: Ticket) -> Literal["accept_jev", "llm_fallback"]:
    return "llm_fallback" if SHADOW or state["confidence"] < THRESHOLD else "accept_jev"

def accept_jev(state: Ticket) -> Ticket:
    return {"department": state["jev_choice"], "decided_by": "jev"}

def llm_fallback(state: Ticket) -> Ticket:
    route = llm.invoke(f"Which team should handle this support ticket?\n\n{state['ticket']}")
    return {"department": route.department, "decided_by": "llm"}

def log_decision(state: Ticket) -> Ticket:
    record = {k: v for k, v in state.items() if k != "ticket"} | {"threshold": THRESHOLD}
    log.info(json.dumps(record))
    return {}  # next: hand the ticket to state["department"]

builder = StateGraph(Ticket)
for node in (ask_jev, accept_jev, llm_fallback, log_decision):
    builder.add_node(node)  # the node name is the function name
builder.add_edge(START, "ask_jev")
builder.add_conditional_edges("ask_jev", is_jev_sure)
builder.add_edge("accept_jev", "log_decision")
builder.add_edge("llm_fallback", "log_decision")
builder.add_edge("log_decision", END)
graph = builder.compile()

if __name__ == "__main__":
    logging.basicConfig(level=logging.INFO)
    text = "Hi, my September invoice shows up twice on my statement. Can you refund the duplicate?"
    print(graph.invoke({"ticket": text})["department"])

1. The node that calls Jev

The client pins the model version. TypeSafeClient(model="jev-1.13.0") sets the model for every call. The jev-latest alias moves with each release, and your thresholds move with it.

The node asks two typed questions. A choice picks the team among three described options. A noul returns the probability that the message is urgent. Answers come back under the same names, in res.choices and res.nouls.

The SDK already handles transient errors. By default it retries 408, 429 and 5xx twice, with growing delays. It honours the Retry-After header and stops after 30 seconds in total. Jev’s 529 code, which means overloaded, falls in that range.

If the error persists, the SDK raises a TypeSafeError. The node catches it and returns a confidence of 0. The ticket then goes to your LLM, and the graph keeps running. The error name stays in the log.

LangGraph also offers a per-node retry_policy. Do not stack both mechanisms. Three SDK attempts times three node attempts make nine calls for a single ticket.

2. The conditional edge on confidence

The whole decision lives in is_jev_sure. The function reads the state and returns the name of the next node. add_conditional_edges plugs it onto the output of ask_jev.

The Literal["accept_jev", "llm_fallback"] return type is not decoration. LangGraph uses it to know the possible destinations and to draw the graph. You do not need to pass a mapping table.

The starting threshold is 0.6, the value used in TypeSafe’s examples. It is not a production value. The “Choosing the right threshold” section explains how to measure it.

3. The fallback to your LLM

llm_fallback reuses your current router, unchanged. The example uses init_chat_model with structured output. The model can only answer with one of the three teams, thanks to the Literal type on the Route class.

This node only runs in three cases:

• Jev’s confidence is below the threshold;
• Jev returned an error, even after retries;
• shadow mode is on.

In every other case your LLM is not called. That is where the savings come from: Jev charges $0.042 per million input tokens, and its output tokens are free.

4. Start in shadow mode

Keep SHADOW = True for the first weeks. Jev answers on every ticket and its answer is logged. But your LLM still makes every decision. Your customers see no difference.

For each ticket, this log gives you Jev’s choice and your LLM’s choice. You can measure how often they agree before Jev makes a single real decision. Then set SHADOW to False to turn the threshold on.

5. The decision log

log_decision writes one JSON line per ticket. Here is the line for a billing ticket handled by Jev, with the sample values from the docs:

{"jev_choice": "billing", "confidence": 0.81, "probabilities": {"billing": 0.88, "technical": 0.12, "sales": 0.0}, "urgent": 0.12, "model": "jev-1.13.0", "department": "billing", "decided_by": "jev", "threshold": 0.6}

• model: the exact version that answered, as returned by the API.
• probabilities and confidence: what lets you recalibrate the threshold later.
• decided_by: the path taken, jev or llm.
• threshold: the threshold applied that day.

The ticket text is not logged. Ship these lines to your usual tool. They are what you recalibrate with, and what you show an auditor.

How we tested this code

We ran this file as is, with langgraph 1.2.12, langchain 1.4.2 and typesafe-sdk 0.7.1. The real Jev SDK was running, but its HTTP calls went to a fake API. That API returned the responses documented by TypeSafe. The SDK accepts a transport parameter for this.

• Confidence of 0.81: Jev decides, the LLM is not called.
• Confidence of 0.10: the graph takes the fallback.
• Repeated 429 error: three calls in total, then the fallback.
• 401 error: no retry, immediate fallback.
• Shadow mode: the LLM decides, Jev’s choice stays in the log.

We also ran the Jev part of this code against the real API on 23/09/2026: jev-1.13.0 answered billing, confidence 1, in 394 ms from France, for 368 input tokens.

Choosing the right threshold

You cannot guess a threshold. You measure it on your data, in your language.

1. Take a few hundred past tickets where the right team is known.
2. Run them through the graph in shadow mode.
3. For each threshold between 0.5 and 0.95, compute the share of tickets Jev would handle alone, and its error rate on that share.
4. Keep the lowest threshold whose error rate you can accept.

Jev is trained on English first. TypeSafe recommends testing on your own content before using it in another language. The n8n guide walks through the same method step by step.

Before production

• Data: Jev is hosted in the United States. Pseudonymise personal data before sending it, and have your DPO approve the transfer.
• Version: keep jev-1.13.0 pinned. Recalibrate before moving to the next one.
• Log: keep every decision, including the LLM’s.
• Sensitive decisions: high confidence is not authorisation. Any outcome unfavourable to a person must go through a human.

Want to see Jev decide before writing any code? Try the demo with your own sample tickets.

Jev and TypeSafe are trademarks of TypeSafe AI, Inc. This article is independent and not affiliated with TypeSafe AI.