GLM-5.3 tops the applied LLM leaderboard — five times cheaper than its rivals

Ed-o-meter, a project by Reinvently, is a large-language-model leaderboard built not on academic metrics but on 28 “real-world” tasks that actual users genuinely perform. The assignments are split into five categories: coding, working with data, everyday queries, safety, and tool use. The checks are automatic and binary, answer quality is graded by a separate judge model, and a full run costs the authors roughly 30 dollars. The harness, tasks, and checkers are released as open source under the MIT license (Featherbench).

An open model in the lead

The headline of the latest update, from August 23, is the arrival of GLM-5.3. It is the first model on the board to take all five “turns” of the track at 100%. That comes with a quality score of 9.3 (the third-best result) and a cost of just 0.28 dollars for the full lap of tasks. The only price you pay is patience: the median time to first token is 16.3 seconds. A faster alternative, gpt-5.5, answers in 13.2 seconds and also holds 100% on safety, but it sags on everyday tasks (89%) and costs 1.43 dollars a lap — noticeably more expensive.

Cheap “workhorses” and safety slip-ups

The cheapest performer is gpt-5.6-luna: 0.064 dollars a lap and a 5.3-second response time. But that speed comes at a price — 79% overall and only 33% on safety, so the model is fit only for cheap background tasks with retries. More reliable are haiku-4-5 (96%, with a 0.9-second response) and deepseek-v4-pro (the same 96% even cheaper, but with a record-slow 40-second response — batch processing only).

The gpt-5.6 line (luna, terra, sol) failed the “crash test”: in 11 of 12 jailbreak checks the models coughed up the control “canary.” The Claude trio and gpt-5.5, by contrast, handled safety flawlessly (6/6).

When a filter looks like a failure

The authors devote a separate discussion to a telling case. The opus-5 model posts the best quality score (9.4) and 100% on both everyday tasks and safety, but only 43% on coding. The reason is not weak code: four harmless debugging tasks were blocked by the provider’s classifier before the first token was even generated. The same fate befell fable-5, which declined five assignments. Since two models from the same family hit the wall at once, the authors conclude that Anthropic’s filter blankets the entire series 5 line, and suggest treating this not as a property of the model but as a measurement artifact.

The authors also candidly note their own caveats — from the bias of a judge model grading itself to a false positive on the vegetarian-recipe check. The project invites anyone interested to clone the repository and run the track themselves.

Source: Hacker News