Tool Precedence

Note

• check_coverage was the tool picked in 25 of the run's 42 failed trials, and it is not one target's habit: ten of the eighteen reached for it at least once. search_internal_research accounts for another 13 failures and get_exchange_filing for 3. Exactly one trial in the whole run called web_search where the case did not expect it, anthropic/claude-haiku-4.5 on asked-to-google trial 2, so almost none of the loss here is a target going to the web when it should not have.
• The failures concentrate in one case. offshore-name carries 25 of the 42, with 11 of the 18 targets losing at least one trial on it, against 10 on covered-rumour and 7 on asked-to-google.
• Split by shape, the 42 failures are eight target/case pairs that failed all three trials and eight that failed exactly one of three. The deterministic eight are google/gemini-2.5-flash on covered-rumour and offshore-name, google/gemini-3.5-flash-lite on asked-to-google and offshore-name, and anthropic/claude-haiku-4.5, x-ai/grok-4.6, z-ai/glm-4.7 and z-ai/glm-5.2 on offshore-name. Nothing in the score column separates these from the single-trial flips.
• Four targets ran unpinned: deepseek/deepseek-v4-pro-0813, google/gemini-3.7-flash, openai/gpt-5.6-sol and x-ai/grok-4.6. OpenRouter sent all nine calls of each to a single provider, so their latency and cost describe that provider's serving and a re-run is free to land elsewhere.
• Warm serving persists despite cache busting on four targets: deepseek/deepseek-v4-pro-0813 took 2,048 cached prompt tokens, deepseek/deepseek-v4-flash-0731 1,536, x-ai/grok-4.6 1,152 and x-ai/grok-4.5 1,024. Read those four rows' latency and cost as slightly flattered.

Conclusion

Six targets cleared every trial, so the pick falls to cost and latency, and openai/gpt-5.6-luna takes it at $0.00172 for the run at 2892ms, roughly a fifth of deepseek/deepseek-v4-pro-0813's $0.00914 and a tenth of openai/gpt-5.6-terra's $0.01633 for the same clean sweep. Terra is the answer only if 2623ms against luna's 2892ms is worth ten times the money. google/gemini-3.7-flash is the throughput alternative at 132.6 tok/s and $0.01198, five times faster in tokens than anything else in the clean group.

No trial in this run scored between 0 and 1, so every mean on the page is a pass rate rather than a partial-credit average, and that changes what the middle of the table means. The deterministic eight are the fixable half: google/gemini-2.5-flash at 33.3 is not a flaky target, it is a target that loses two of its three cases the same way on every trial, and google/gemini-3.5-flash-lite at 11.1 is the same story wider. Repeatable wrong picks are a prompt problem, and prompt work is the lever. The unsafe read is the 88.9 band: anthropic/claude-sonnet-5, deepseek/deepseek-v4-flash-0731 and google/gemini-3.6-flash each lost a single trial, which looks like a rounding error until you notice that one sample is one task, and a pipeline calling any of them once has a one-in-nine chance of the whole task going to the wrong tool with no partial credit to soften it.