Vercel's AI Gateway Production Index has two numbers about Jev that sit oddly together. It was the fastest-adopted model in the gateway's history, reaching 35% of paid teams in its first two weeks. Over the same two weeks it accounted for just 0.27% of gateway spend.
A third of paying teams tried it. Together they spent almost nothing on it. These models are cheap by design, which is why your feed has been full of Jev, Clef and Decider. A decision model answers a closed question with a choice and a probability, in one pass, instead of writing text.
Is this message urgent? Which of these five tools fits? Does this trace look like a failure? You hand it the question and the options, and it hands back one option and how sure it is. Daniel Kahneman's name for fast, automatic judgment is System 1, from Thinking, Fast and Slow, and Jev is sold as a System One model. We scouted 199 AI engineering posts, nearly all published since September 1, and read 37 closely, 19 of them about decision models and routing. This post is what they say, plus what we learned running one ourselves. Our take: use one for closed questions, and only after you've checked the number it returns against your own labelled data.
A closed question has a known list of answers: is this message urgent? A decision model hands back one option and how sure it is, in one pass.
An open question needs prose. An LLM still has to write the answer, and code or a person has to parse or read it.
Pick by the question. One of a known list: a decision model can pick it. Prose: an LLM writes it.
Why does everyone try one? The price is real
A practical guide on dev.to lists Jev at $0.042 per million input tokens, with output free. Its price table puts frontier LLMs at $0.20 to $10 per million input tokens, so the input is roughly 5 to 240 times cheaper. The same guide credits it with answering in 70 to 500 milliseconds, and calls those TypeSafe's own numbers, self-run and unreproduced. Our own probe, a handful of calls while building our router, saw 295 to 574 milliseconds, median 0.32 seconds.
Teams that put small models to work report speed too. Cloudflare had its Clef model fetch, render and classify a website in 2.2 seconds. The fastest general LLM they tried, gpt-oss-120b, took 4.7 seconds in the same workflow and returned only two classifications. If you chain a few steps, a Clairic, Clef and GLM-5.3 cascade finished in 19.3 seconds against 61.7 for GLM-5.3 alone, and cost 49% less.
Shuai Guo built a browser agent where Microsoft's Decision-1 picks each step and GPT-6 Luna writes the text. It took 41 seconds. Pure GPT-6 Luna took 156. Big caveat: one run, one task, and it's Decision-1, not Jev. Treat it as a hint.
The cleanest case is a smaller one, and it's the one we'd copy. GitHub replaced an LLM pipeline for secret detection with a fine-tuned ModernBERT classifier. It's a small classifier for one job rather than a general decision model, and it didn't replace the deterministic scanner: regex still finds the candidates. Offline, it had 13% higher precision and 19% higher recall than the LLM pipeline it replaced.
| Team | The job | With a small model | LLM only |
|---|---|---|---|
| Cloudflare | Classify a website | 2.2 s | 4.7 s |
| Cloudflare | Extraction cascade | 19.3 s, 49% cheaper | 61.7 s |
| Shuai Guo | Browser task, one run | 41 s | 156 s |
| GitHub | Secret scan, median | 86% faster | The old LLM pipeline |
GitHub also reports serving cost down more than 98%. Speed is where the case is strongest, though you won't get it everywhere: in Red Hat's benchmark below, an LLM judge had the better latency.
Are decision models accurate? Only on your data
Cloudflare's own run found Clef beat Jev in 3 of 4 areas of TypeSafe's eval suite, which is a vendor talking about its own model, but still. Privatemode compared a GLM-5.3-Flash setup with Jev across 28 text datasets and found them on par, each more accurate on 10. At list prices, a million decisions cost about EUR 62 with Flash and about EUR 16 with Jev. And Hamel Husain and Antaripa Saha found its probabilities well calibrated on their data.
The doubters are loud too. Arcturus Labs puts the problem in one line: "Speed, cost, and ease of use are obvious, but accuracy is the one thing that's hard to check." Red Hat ran a guardrail benchmark on tasks like prompt injection and content safety. Their finding is that decision models like Jev don't reliably beat LLM judges, pre-trained classifiers or open decision models on speed or accuracy.
kapa.ai tested Jev as a reranker on 1,000 cases. It beat every Cohere and Qwen model they tried, and the newer version, 1.14, came within 0.003 of the top. But Voyage AI's rerank-3 won, and it was also the cheapest and the fastest. Their advice, and ours: test on your own data.
The sharpest result comes from Swift at Major League Hacking (MLH). Their router picks a cheap code check or defers to an LLM judge. On eight real checks, out of the box, Jev made three dangerous calls, false positives. Gemini Flash-Lite made none. Swift's summary: "Cents come back. Trust doesn't."
| Test | What | Who won |
|---|---|---|
| Cloudflare, vendor run | Clef vs Jev, TypeSafe suite | Clef, 3 of 4 areas |
| Privatemode | Flash vs Jev, 28 datasets | Tie on accuracy |
| Red Hat | Guardrail tasks | No reliable winner |
| kapa.ai | Reranking, 1,000 cases | Voyage rerank-3 |
| MLH | Eight real checks, first prompt | Flash-Lite, no dangerous calls |
Five tests, and no model won them all. Red Hat adds a useful footnote: decision models arguably existed for years as zero-shot text classifiers, such as Meta's BART-large-mnli from 2019.
Does the prompt still matter? Yes, and examples can hurt
You might assume a model that only picks from a list needs no prompting. MLH found the opposite. They folded their worked examples into Jev's prompt and its dangerous checks doubled. The fix was one sentence, and it's worth stealing:
Imagine the checker runs and passes. Could the check still be false?
If yes, that checker is wrong: defer.That's Swift's wording. On fresh checks, it went from 35 false-positive runs to 0. Swift also draws a line worth keeping: "A blind spot is a prompt bug you fix. A flip is sampling noise you measure for." The noise is real. Privatemode saw up to 3.5% of answers change between identical runs, even at temperature 0.
Prompts don't travel either, so test before you borrow someone else's. Red Hat's results confirm that the style of prompt that works for Jev doesn't necessarily work for Laya, an open-source Jev-style model, and vice versa. Arcturus Labs says why the prompt carries so much weight: none of these models is trained on your process, and "all of that has to live in the prompt."
The dev.to guide has the matching rule for inputs: whatever assembles the state decides what it is allowed to know. It doesn't see your product. It sees the paragraph you gave it, nothing more.
How much should you trust the probability?
The number is the product. Your agent will act on it, so a decision model that returns "urgent, 0.81" is only useful if 0.81 means something.
An LLM can't give you that for free. Amrut Joshi puts it plainly: "0.85" is a plausible continuation of the conversation, not a measurement of anything. Ask again and you get a different number. It's a Magic 8-Ball with every "Ask again later" face filed off: it always answers and always sounds sure.
Jev is trained to do better than that. The dev.to guide says it is trained for calibration, so its confidence is "meaningful in aggregate: higher confidence really does mean higher accuracy." In aggregate is doing real work in that sentence. Hamel and Saha found one Clef Flash group of traces with an average predicted failure probability of about 8%, where 60% contained failures. That was 98 labelled traces from an apartment-leasing agent, so it's a result on that dataset, not a law, but our reading is that a set about that size is enough to start seeing which probabilities hold up. Jev and Liquid's d1-ultra were well calibrated on the same data.
The other trap is the default. Hamel and Saha noticed people using the 0.50 threshold without checking it, and give an example: a model that scored every failure 0.40 and every pass 0.10 would be perfect at a 0.30 threshold and catch nothing at 0.50.
Move the line on purpose. On a slice of the Berkeley Function Calling Leaderboard, MLH raised Jev's bar to 0.8: false positives dropped to 1%, at the price of deferring twice as often. You can make that trade only if you know what each side costs you.
A score, not a verdict. Say a model scores every failure 0.40 and every pass 0.10. It ranks them perfectly; the line decides what happens next.
The default 0.50 catches nothing. Every failure scores under it and takes the cheap path.
Tuned to 0.30 on labelled data, the same model catches every failure. Move the line on purpose, once you know what each side costs.
Our own router has this problem: its floor is 0.5, set by hand and never tuned on labelled routes. That is the default Hamel warns about.
Let code do the counting
These models read well and count badly, and the people closest to them say so. Cloudflare's Clairic post: "Reading is strong, but multi-step arithmetic isn't yet." One- and two-step sums seem reliable; longer chains are where it slips. Their advice is to have Clairic predict the value and let code do the math. The guide says it is "not a calculator": it doesn't count reliably, and error grows with the size of the thing counted.
The guide has another line worth keeping in view: the model can't return an invalid value, but it can return the wrong valid one. A typed answer is still an answer that can be wrong.
The pattern that worked for the people we read has three roles. The decision model picks, the LLM writes, and code acts and counts. That's Shuai Guo's split: the decision model handles closed-ended selection, the LLM open-ended generation, and Python orchestration. MLH put numbers on the same shape. With Jev deciding and Flash-Lite writing arguments only when needed, they measured $0.10 per 1,000 checks against $0.52 for Flash-Lite alone. The decision step itself costs about three cents per thousand.
Our router follows the same rule. It asks the model one closed question with two options, which of the two models takes the newest message, and never asks it to write or count.
What we do: a router that fails open
We run one. On Definable, an agent can have a main model, which is slow and careful, and an optional fast one. For each new message, Jev picks which of the two takes it. Its two options are two short descriptions we wrote: what a fast model is good at and what a slow one is good at.
It reads the newest message and the first 4,000 characters of the agent's last answer. The second part came from that same probe on 2026-10-19. "Ok, go ahead and do it" on its own went to the fast model with a confidence of 0.81. The same words right after the agent had proposed a plan went to the main model at 0.98. That is why we show it the last answer too.
The fast model starts only when Jev picks it at 0.5 or more. Everything else falls to the main model: the router is unsure, errors, takes longer than a second, or the fast model can't take the conversation (an image it can't see, a chat longer than its context window). That is what every run did before there were two models, so when routing fails, the run still answers. After the route, either model can hand the conversation to the other with a short note.
Jev reads the newest message and the first 4,000 characters of the agent's last answer, once per message.
“Ok, go ahead and do it” on its own went to the fast model at 0.81. The fast model starts only at 0.5 or more.
The same words right after the agent proposed a plan went to the main model at 0.98. That's why Jev sees the last answer too.
It fails open. Unsure, an error, slower than a second: everything else falls to the main model, so when routing fails the run still answers.
We route once per new message, never mid-call. That doesn't dodge the cost Nhu Hoang measured. The model can change from one message to the next, and each change means the incoming model reads the chat without its cache. Switching back is cheaper, but it still pays for what was added while it was away. In Hoang's scripted 20-turn test on a 0.9B model, per-request routing delivered only 13 of the 40 percentage points of savings that the raw price difference suggested. Switching at phase boundaries captured 29. As Hoang puts it: "Routing is not just a pricing decision."
Our position is an opinion, not a result: a person's new message is the closest thing a chat has to a phase boundary. We haven't measured what our switches cost. Claude's API cache lasts five minutes, so with Opus as the main model, a chat the fast model held for longer than that comes back cold, the same as an Opus-only agent after a five-minute pause. We wrote about the design in Fast and slow models, and the Router step shows up in traces.
Here's the gap. We haven't tuned that floor against labelled routes, so we won't claim route accuracy we haven't measured. But the trace records each route's confidence, and every handoff from the fast model back to the main one is a free label we haven't used yet. Each one is a hint that a route was wrong, or that the task grew.
A checklist before you trust one
Hamel and Saha write a sentence we'd put on the wall: "None of the ideas discussed in this post are new." A decision model is a classifier with a better API. Evaluate it the way you'd evaluate any classifier.
| Check | Ask yourself |
|---|---|
| Closed question | Is the answer one of a known list? |
| Your data | Did I score it on my own labelled examples? |
| Prompt tests | Did I check examples help, not hurt? |
| Threshold | Did I tune it, or leave 0.50? |
| Calibration | Does 0.8 mean right about 80% of the time? |
| Fallback | What happens when it's unsure or down? |
| Counting | Is code doing the math? |
On Definable, every trace of an agent with a fast model opens with the Router step: the model it picked, how sure it was, and why the main model took over when it did. That's the record a floor like ours gets tuned from.
A cheap classifier is only as good as the labels you checked it against.