benchr · Issue No. 07
Pricing, benchmarks, and use-case fit for Claude, GPT, Gemini, Llama, Mistral, and the open-weight tier — sourced from official provider documentation.
Featured
A head-to-head comparison covering code review, landing-page copy, citation accuracy, recipe writing, and email drafting. Claude leads on five of the seven; the two GPT-5 wins are not small.
Strong on architecture decisions and long-document analysis. Vision is its weaker side.
The visual-design winner. Still strong on math and structured output.
Excels at vision + reasoning. Average elsewhere. Deprecated by Google in March 2026 in favor of 3.1 Pro.
Cursor, GitHub Copilot, Windsurf, and Cody on the same Markdown-exporter task.
ElevenLabs, OpenAI Whisper, and Cartesia Sonic on latency, accuracy, and naturalness.
Advertised window versus the retrieval zone where the model actually finds information.
The cheapest model for chat, coding, RAG, agents, classification, and summarization.
Where Llama 4, Mistral Large 2, DeepSeek-V3.1, and Qwen 3 stand against the closed labs.
MMLU is saturated above 90%. The benchmarks worth tracking now.
An interactive reference covering pricing, benchmarks, context windows, and capability ratings for 14 frontier and open-weight models.