benchr changelog

Launched the full benchr tool suite: ranked index, charts, cost calculator, model recommender, deprecation tracker, and release timeline. All tools read from models.json — never duplicate data per page.

New models added to the index: Claude Opus 4.8, GPT-5, GPT-5 Mini, Grok 4.3, Kimi K2.6, Mistral Large 3, Llama 4 Scout, Llama 4 Maverick. Removed unverified entries (GPT-5.4, GPT-5.4 mini, Qwen 3.5). All 19 current models verified against official provider pricing pages.

Released May 28, 2026. $5/$25 per 1M standard; $10/$50 Fast Mode. SWE-Bench Pro 69.2%. Source: anthropic.com/news. Review →

GA since May 19, 2026. $1.50/$9.00 per 1M; free API tier. 289 tok/s throughput. Source: ai.google.dev. Review →

Previous models.json had 1M context for GPT-5.5. Corrected to 1,048,576 per OpenAI's API documentation. Source: platform.openai.com/docs.

These entries couldn't be confirmed against official provider sources. Replaced with verified models: GPT-5 ($1.25/$10, Aug 2025) and GPT-5 Mini ($0.25/$2.00). Qwen 3.5 entry removed; Qwen 3.6 remains (verified Apache 2.0, Apr 2026).

Introduced the benchr Rating: capability × 0.65 + price efficiency × 0.35. Formula runs in open JavaScript at assets/js/models.js. Full methodology documented on the rankings page. No provider pays for ranking position.

Prior data in gpt-5-5-review.html incorrectly stated GPT-5 at $2.50/$15. Actual price confirmed at $1.25/$10 from official OpenAI pricing page. Source: openai.com/pricing.

Google confirmed Gemini 3 Pro was sunset March 26, 2026. Successor: Gemini 3.1 Pro. Added to deprecation tracker. Source: ai.google.dev/deprecation.