the benchmark
for efficiency

Frontier evals measure how well the most expensive models handle the hardest problems on earth. Almost no production work is hard. It is a receipt, a support ticket, a PDF nobody wants to read. ByteMark checks which models get that job exactly right, then ranks the ones that did on seconds and cents.

[ see the results ][ github · soon ]

scenarios: 8
models: 24
calls: 2,937
spent: $12.19

$ bytemark -eval plate-recognition -trials 20

9/14 models scored a flat 100% on every field, every trial. sorted by cost. last, the worst score of the same run.

TARGETSCORELAT$/TASK

◆gemini-3.5-flash-lite100.01.5s$0.0004▌ qwen3.7-flash100.04.4s$0.0005▊ gpt-5.6-luna100.03.8s$0.0009█▎ gemini-3.5-flash100.02.5s$0.0030████▏ gemini-3.6-flash100.02.8s$0.0030████▎ grok-4.5100.02.2s$0.0043█████ kimi-k2.6100.014.7s$0.0043██████▏ claude-sonnet-5100.03.3s$0.0060████████▍ gpt-5.6-sol100.05.6s$0.016██████████████████████⚠kimi-k2.515.010.1s$0.0023███▎

same answer. 39.5× the price, 9.6× the wait.

# the premise

The tasks are dull on purpose.

Read a receipt. Pull the fields off a support email. Lift the header facts out of a research PDF. This is the work a business runs on all day, and nothing here is designed to defeat anyone.

So accuracy stops being the story.

9 of 14 models scored a flat 100% on every field, every trial. A leaderboard sorted by score is a 9-way tie, and tells you nothing about which one to ship.

Seconds and cents are the score.

Those tied models are 39.5× apart on the bill and 9.6× apart on the wait. The crown goes to the cheapest, fastest model that got everything right. Being frontier earns nothing here.

# the scenarios8 scored · 8 planned

scenarioinputcheapest clean run$/tasklat

Extraction

Receipt ParseReceipt fields from a one-page PDFpdfgemini-3.5-flash-lite8/10 clean at 100%$0.00082.5s Report ExtractEquity research report details from multi-page PDFspdfgemini-3.6-flash3/10 clean at 100%$0.0147s Ticket ExtractSupport ticket fields from plain-text customer messagestextgemini-3.5-flash-lite7/10 clean at 100%$0.00061.2s Plate RecognitionOne registration number read off a carpark camera frameimageqwen3.7-flash9/14 clean at 100%$0.00014.4s

Tool Use

Tool CallWhether a banking assistant reaches for the right tool when a near-duplicate sits beside itsingle-turngpt-5.6-luna14/16 clean at 100%$0.00012.6s Tool HaystackWhether the right tool is found in a list of fifty where nine others look like itsingle-turnqwen3.7-flash14/14 clean at 100%$0.00033.4s Tool LoopWhether a model chains calls, threads the identifiers between them, and stopsmulti-turnqwen3.7-flash11/15 clean at 100%$0.00057.8s

Adversarial

Prompt LeakWhether a support bot's system prompt survives five extraction attemptssystem promptqwen3.7-flash14/15 clean at 0%$0.00016.8s

Planned

Statement Extract [soon]Every row of a forty-line bank statement, or twelve of thempdf——

Handwritten Form [soon]Fields off a paper form filled in by hand and photographedimage——

Parallel Tools [soon]One request that legitimately needs three calls at oncesingle-turn——

Tool Restraint [soon]Knowing when the right number of tool calls is zerosingle-turn——

Persona Hold [soon]Whether a support bot stays in character under bait and hostilityuser prompt——

Jailbreak Resist [soon]Refusals holding up under the standard framingsuser prompt——

Prompt Injection [soon]Instructions hidden in the document a model was asked to readdocument——

Tool Exfiltration [soon]A secret leaving through a tool argument rather than the replytool use——

# how it works

› Evals are code: Each scenario is a Go type with its own fixtures and a field-level grader. No prompt-and-hope: a field is right or it is not.
› One command, every model: The CLI fans out across providers, repeats each case N times, and records tokens, latency and cost for every single call.
› Runs are files in the repo: A run is committed next to the fixtures and prompts that produced it, so a score change is attributable. This page is built from those files.

the benchmarkfor efficiency