StormEval
  1. StormEval
  2. Models
  3. Claude Opus 5.5

Daily test results · Claude Code · effort medium

Is Claude Opus 5.5 getting worse?

Building baseline, day 7 of 14

No verdict yet: day 7 of a 14-day baseline.

Runs through
Claude Code
Effort
medium
Provider
claude-cli
Model id
claude-opus-5-5
Tasks
New issues in real code
Updated
2026-10-21 04:41 UTC

Rating over time

Higher is better. The band is one standard error either side. The dashed line is the first day. Movement inside the band means nothing.

Was Claude Opus 5.5 nerfed?

No verdict yet

This is day 7 of the 14-day baseline for Claude Opus 5.5. The baseline began on 4 October 2026. If the model runs every day, the baseline is complete on 28 October 2026 and the first verdict is expected on 29 October 2026.

It is too early to say either way. Until the baseline is complete there is nothing to compare a new day with, and a single day moves by chance.

Rating
1707±197 · one standard error
Change since day 1
+48from 1659 on 4 October 2026
Pass rate, latest graded day
100%latest graded day · 3 graded tasks
Days of data
7since 4 October 2026

What the runs took

Median time per task
3m 26smean 3m 34s · 0 timeouts
Output tokens per task
8.7k22% thinking · 580k in
Output tokens per second
43median · mean 41
Estimated cost per task
$0.564at API list prices · $1.69 a day

Estimated cost is tokens at API list prices, not money spent. A dash means no list price is set. Graded counts cover 21 tasks over 7 days; time, tokens and cost can also include finished runs excluded from grading.

Time per task

How long each task took. A task not finished in 15 minutes is a fail, and counts as 15 minutes of time spent.

  • Claude Opus 5.5median 3m 26sslowest 7m 01s · pull-request

By task family

Pass rate, mean time and mean output tokens for each kind of problem.

Modelpull-request
Claude Opus 5.5100%3m 34s8.7k

Day by day

DayRatingPassedTimeoutsTime spentMedian timeOutput tokensOutput tokens / sEst. cost
21 Oct1707 ±1973/3012m 00s3m 37s41k58$2.67
17 Oct1657 ±1973/306m 38s2m 03s14k37$1.01
15 Oct1640 ±1993/3010m 10s3m 08s16k28$1.16
8 Oct1611 ±2033/308m 31s3m 20s22k48$1.21
7 Oct1649 ±2003/309m 47s3m 35s24k45$1.51
5 Oct1595 ±2073/3016m 35s5m 11s33k33$2.01
4 Oct1659 ±2003/3011m 14s3m 00s32k52$2.29

How this is measured

Claude Opus 5.5 gets the same new tasks as every other model. Each task is an issue in a large, working codebase, and the model writes the change that resolves it with its own coding tool, as it would for a pull request. A program runs the tests to check the change. Only one run per task counts toward this model's results.

The model is compared with itself: the last 7 days against its first 14 days. A verdict is only given when the difference is too large to be chance. A task not finished in 15 minutes counts as a fail.

Read the full method

Not a leaderboard. Models run at different effort settings and through different tools.

Other models