Jev vs classifiers: fine-tuning, zero-shot, rules, embeddings

Jev is not the only way to classify text without a generative LLM. Rules, a fine-tuned small model, embeddings, a zero-shot classifier: each has its ground. Here is a fair comparison, including where Jev loses.

The short answer

Jev wins when you have no labelled data and your labels keep changing. A fine-tuned small model or plain rules win when labels are stable, examples are plentiful, or EU hosting is mandatory.

Generative LLMs used as classifiers get their own comparison: Jev vs LLM.

Your options for text classification

Approach	Training data	Setup	Changing labels	Calibration	Latency and cost	EU hosting
Rules, keywords	None	Quick at first	Edit the code	No probabilities	Negligible	Yes
Fine-tuned small model	Many examples, or few with SetFit	Days to weeks	Relabel, retrain	Decent in-domain, needs tuning	Low, fixed hosting cost	Yes
Embeddings + classifier	A few examples per label	Days	Add examples	Needs calibrating	One embedding call plus local compute	Yes if self-hosted
Zero-shot NLI	None	Hours	Rewrite the labels	Not guaranteed for your task	One pass per label	Yes
Jev	None	Hours	Rewrite the options	Calibrated per TypeSafe, to verify	70 to 500 ms per TypeSafe, $0.042 per million tokens	No, United States
LLM routers	Preferences or evals	Varies	Not the same job	Varies	Added to the LLM call	Depends on vendor

Setup times are rough estimates from our own experience, not measurements.

Rules and keywords: when are they enough?

Rules win when the signal is explicit and stable. A form field, an order number, a known sender: no AI needed.

Data: none, just domain knowledge.
Labels: every change is a code change, with its tests.
Calibration: none. A rule matches or it does not.
Latency, cost, hosting: negligible, on your own servers.

Their weak spot is well known: paraphrases, typos, unexpected languages. “You took my money twice” slips past a “refund” rule. A good habit: keep rules for the certain cases and send the rest to a model.

Jev vs fine-tuning a small BERT-style model

A fine-tuned small model is still the benchmark when labels are stable and examples are plentiful.

Data: labelled examples for every label. SetFit cuts that need sharply: with 8 examples per class, its authors come close to a RoBERTa model trained on 3,000 examples, on a customer review dataset.
Setup: collect, label, train, evaluate, deploy.
Labels: a new category means relabelling and retraining.
Calibration: pre-trained transformers are fairly well calibrated in-domain, and temperature scaling improves them. Out of domain, check it again.
Latency and cost: local inference, no external call. Cost is mostly fixed: hosting, labelling, maintenance.
Hosting: anywhere you like, the EU included.

Jev wins with no labelled data, with shifting labels, or when you need to ship fast. Jev cannot be fine-tuned: it adapts through the instructions and options of each request. The two also combine: TypeSafe shows how to train a classic model on Jev’s probabilities.

Embeddings and nearest neighbours: the pragmatic option

Embeddings are a sound middle ground when you have a few examples per label. Each text becomes a vector. A logistic regression or a nearest-neighbour lookup gives the label.

Labels: with nearest neighbours, you just add examples.
Calibration: a neighbour vote is not a calibrated probability. A logistic regression can be calibrated with scikit-learn, using sigmoid, isotonic or temperature scaling.
Cost: OpenAI’s text-embedding-3-small costs $0.02 per million tokens, less than Jev’s $0.042.
Hosting: an open-source embedding model runs wherever you want.

The catch: similarity is not a decision. Two tickets that look alike can belong to two teams. Jev answers the question you ask, not how similar texts are.

Looking for a zero-shot classifier alternative?

A zero-shot NLI classifier is the closest thing to Jev in day-to-day use. You give labels in plain language, with no training data.

The method comes from Yin, Hay and Roth at EMNLP 2019. A natural language inference model judges whether the text entails a hypothesis built for each label, such as “This text is about billing”. facebook/bart-large-mnli, under the MIT licence, is the best-known example.

Labels: instant change, just rewrite the hypotheses.
Cost and latency: one text and hypothesis pair per label, so both grow with the number of labels.
Calibration: nothing guarantees entailment scores are calibrated for your task.
Languages: bart-large-mnli is trained on English. xlm-roberta-large-xnli covers 15 languages, French included.
Hosting: on your own servers.

Jev also takes detailed option descriptions, JSON states and several question types per call. We know of no public comparison of the two on the same data. Test both on yours.

RouteLLM, Not Diamond, Martian: a different problem

These tools pick which LLM answers. They do not decide which team handles a ticket.

RouteLLM, open source under Apache 2.0, sends simple queries to a weak model and the rest to a strong one. Its routers are trained on Chatbot Arena preference data. The project claims up to 85% cost savings while keeping 95% of GPT-4 performance, on its benchmarks.
Not Diamond predicts the best LLM for each query and can train on your own evaluations.
Martian introduced its model router as sending each query to the best LLM in real time.

They complement each other. Jev makes the business decision and can supply the signals that pick the model, such as complexity. Plain code does the rest, as in our live demo.

When a small model or rules beat Jev

Labels have been stable for months: a fine-tuned model needs almost nothing while nothing changes.
You have lots of labelled data: a model trained on your history learns your edge cases. Jev does not train on your data.
EU hosting is required: Jev is hosted in the United States. TypeSafe covers EU transfers with standard contractual clauses, without Data Privacy Framework certification.
Volume is very high: see the numbers below.
Texts are not in English and the stakes are high: English is where Jev is most accurate, per TypeSafe.

At very high volume, Jev’s cost stays modest. For 10 million decisions a month at 500 tokens: 10,000,000 × 500 × 0.042 ÷ 1,000,000 = $210, or €183.20. The constraint lies elsewhere:

the limit is 1,200 requests per minute, so 51,840,000 over 30 days at most;
TypeSafe warns these limits change without notice while it scales;
Jev is a service launched on 15 September 2026, with sign-ups paused for now.

A model you host yourself has none of these dependencies.

When Jev has the edge

No labelled data, or no time to produce it.
Labels that change often: add an option to the request, no retraining.
Several decisions on the same text: choice, score and noul go in the same call.
Usable probabilities out of the box, which TypeSafe says are calibrated. Check on your data.
No infrastructure to host or maintain.

How do you choose without guessing?

Measure candidates on your real decisions, not on a demo. In shadow mode, you replay a sample of past decisions with nothing changed in production. You compare agreement rates and traffic coverage at the same confidence level.

To go further, read what Jev is and what a System One model is.

Jev and TypeSafe are trademarks of TypeSafe AI, Inc. This article is independent and not affiliated with TypeSafe AI.