Four decision models, one inbox

The short version

Perplexity wins on most things. Jev is the only one that says "certain".

• Perplexity was the most accurate, could safely automate the most mail, was the cheapest, and tied Jev on speed.
• Jev is the only model that says "I'm certain" and means it. 759 answers at 99% or higher, every one correct.
• Clef is accurate but never sure of itself. When it says 75%, it is right 98% of the time.
• Clef-flash is cheap and fast but the weakest of the four.
• Cloudflare says Clef is 2.5x faster than Jev. From the same machine, it was 2.2x slower.

Background

What a decision model does

A chat model writes text. A decision model reads text and returns a probability for every answer you allow. You give it the content and the options. It gives back numbers you can put a threshold on, in one call.

That last column is the whole product. If the number is high you let the machine route the email. If it is low you send it to a person. Whether that works depends on one thing: does 0.96 really mean right 96% of the time? That is what this test measures.

Finding 01

Three vendors, one API

All three companies use the same request format. You send some content, a set of named questions, and the options for each. Cloudflare says so directly: you can switch from Jev to Clef "by changing the endpoint and model". We sent the exact same body to all four.

{
  "model": "…",
  "state": "Subject: Anfrage DN100 …",
  "questions": {
    "category": {
      "type": "choice",
      "instructions": "Classify this email …",
      "criteria": { "Order": "…", "Request": "…", … 10 options }
    }
  }
}

Perplexity and Cloudflare both published open weights. Both are Qwen language models with a small scoring head on top, run once over the input with no text generation. Perplexity's open-source package is named autojev, serves Jev's endpoint path, and accepts jev-1.13.0 as a model name. Jev's own architecture is not public.

Finding 02

Perplexity and Clef are more accurate

The fair way to compare two models on the same emails is to look only where they disagree and count who was right.

Finding 03 · The main one

Three of the four models are never sure

All four models are right about 96 to 98% of the time. But look at how often each one is willing to say it is at least 90% sure.

Clef-flash is right 96% of the time but says it is 90% sure only 3% of the time. Clef is right 98% of the time and sure 43% of the time. Their probabilities are too low. The models are better than they think they are.

Jev and Perplexity sit close to the line: when they say 85%, they mean roughly that. Clef and Clef-flash sit far above it. Clef-flash at "65% sure" was right 99% of the time.

Cloudflare says it trained Clef with a loss designed to improve calibration. Its own public leaderboard lists Jev's calibration score and leaves Clef's blank. On this task, Clef's calibration error is about five times Jev's.

This matters if you want to say "auto-route anything above 99%". With Jev, 759 emails cleared that bar and all 759 were right. Perplexity cleared it 3 times; its highest answer was 99.0%. Clef never went above 97.2% and Clef-flash never above 96.2%. A fixed threshold that works for one model means nothing for another.

Finding 04

So how much can each one automate?

Sort each model's answers from most to least confident, then go down the list until mistakes pass 1 in 100. Because this uses each model's own ranking, it works whatever scale its numbers are on.

Perplexity can handle 97% of the inbox at that bar, Clef 96%, Jev 94%. Clef ranks its answers well even though its numbers are too low, so it still does well here. Clef-flash falls to 84%.

Perplexity goes through 91% of the mail without a single error, Jev 88%, Clef 80%. Clef-flash makes its first mistake at 30%.

We left one email out of this chart, and only this chart. It is a public tender for office cleaning services that mentions a bill of quantities, so it looks like a tender request but is irrelevant to the business. We wrote it as a deliberately hard trap. All four models fell for it. Perplexity was 97.7% sure, which would end its error-free run at 52% on its own. Because every model got it wrong, removing it treats them all the same, and the chart shows what each model does on the rest of the mail.

Jev's first mistake came at 86% confidence, below all 759 answers it was certain about. That is Jev's remaining advantage: its "certain" is a number you can set a rule on without testing first. Perplexity never goes above 99%, so you have to find its cut-off on your own data.

Finding 05

Jev and Perplexity tie on speed. Clef is slower than advertised.

Cloudflare's announcement says Clef is 2.5x faster than Jev and Clef-flash 13x faster. Their own leaderboard notes that Jev's number includes the internet round trip while Clef's was measured on Cloudflare's own servers. Measured the same way, from the same machine, Clef was 2.2x slower than Jev and Clef-flash slightly slower. Clef's slowest call took 5.8 seconds; Jev and Perplexity never went past 0.7.

Finding 06

Perplexity is the cheapest

Perplexity charges $0.04 per million input tokens, Jev about the same, Clef-flash about twice that and Clef six times. For context, two weeks ago the two Gemini models cost $0.80 and $1.79 per 1,000 of the same emails. Perplexity's accuracy (98.0%) sits between theirs (97.5% and 98.5%) at a small fraction of the price.

Finding 07

Jev barely changed in two weeks

4 of 1,565

We ran Jev on the same emails on September 17 and again today. Four answers changed. The average confidence moved by less than one percentage point. Cloudflare's leaderboard says Jev's answers "drifted" between two of their runs; on this task it was stable. It also got faster: typical latency fell from 0.60 to 0.29 seconds.

Finding 08

Images yes, PDFs no, and one silent failure

Perplexity and Cloudflare advertise image input. We sent every model a small image and a small PDF, each showing one word, and asked which word it was.

Jev does not support images and does not say so. It accepts the request, treats the encoded image as text, and returns a confident-looking guess: "INVOICE" at about 40% no matter which word was in the picture. Code that sends Jev an image gets a normal answer back and no error. That is worth knowing before you build on it.

Everything else in this report is text only, so all four models saw exactly the same input.

Reference

API differences that matter

The practical rules that follow: ignore the vendors' own "confidence" fields and use the probabilities, because each vendor computes confidence differently. Ask one question per call on Clef, because its answers to one question shift with what else is in the request. And log the model name and date on every call, because only Jev lets you pin a version.

Reference

The launch claims, checked

Honestly

What this doesn't prove

• Most answers were not checked by a person. For the real emails, Claude Opus 5 decided what "correct" meant. Only the 364 written emails have answers we know for certain.
• One task. Sorting business email into ten buckets. Nothing here says how these models do on other jobs.
• No attachments. 84% of the real emails had files attached. No model saw them.
• The written emails are cleaner than real mail. One author wrote all 364.
• Perplexity and Cloudflare can change their models without notice. These results describe the hosted models on October 2, 2026.

In one sentence

Perplexity's new model is the best all-rounder on this task, Jev is the only one that will tell you it is certain and be right every time, and Cloudflare's Clef is accurate but too modest about it.

Method. 1,565 emails: 1,201 real ones from four company mailboxes and 364 written to cover rare categories, the same set as the September Jev benchmark. Every model got identical text and one "pick one of 10" question with category definitions taken straight from a production prompt. Models: jev-1.13.0, pplx-decider-v1-27b, clef, clef-flash, all called on October 2, 2026 from one US cloud server. Confidence is each model's top probability; the vendors' own confidence fields were ignored. Automation rates use 50 random tie-break orders and were stable. Significance by McNemar's exact test on paired answers. Reference answers for the real emails from Claude Opus 5, which never saw any model's output.