Daily test results · API · effort default
Is DeepSeek V4.1 Flash getting worse?
No verdict yet: day 1 of a 14-day baseline.
- Runs through
- API
- Effort
- default
- Provider
- deepseek
- Model id
deepseek-flash- Tasks
- 30 new ones every day
- Updated
- 2026-10-04 05:45 UTC
Rating over time
Higher is better. The band is one standard error either side. The dashed line is the first day. Movement inside the band means nothing.
Was DeepSeek V4.1 Flash nerfed?
No verdict yet
This is day 1 of the 14-day baseline for DeepSeek V4.1 Flash. The baseline began on 4 October 2026. If the model runs every day, the baseline is complete on 17 October 2026 and the first verdict is expected on 18 October 2026.
It is too early to say either way. Until the baseline is complete there is nothing to compare a new day with, and a single day moves by chance.
- Rating
- 1385±68 · one standard error
- Change since day 1
- –needs two days
- Pass rate
- 47%latest day · 30 graded tasks
- Days of data
- 1since 4 October 2026
What the runs took
- Median time per task
- 4m 20smean 4m 23s · 1 timeout
- Output tokens per task
- 63k100% thinking · 289 in
- Output tokens per second
- 242median · mean 248
- Estimated cost per task
- $0.038at API list prices · $1.10 a day
Estimated cost is tokens at API list prices, not money spent. A dash means no list price is set. Numbers cover 30 graded tasks over 1 day.
Time per task
How long each task took. A task with no answer in 8 minutes is a fail, and counts as 8 minutes of time spent.
By task family
Pass rate, mean time and mean output tokens for each kind of problem.
| Model | assign | orders | paths | queens | tsp |
|---|---|---|---|---|---|
| DeepSeek V4.1 Flash | 0%2m 07s32k | 67%5m 37s74k | 50%5m 25s72k | 33%4m 52s84k | 83%3m 51s55k |
Day by day
| Day | Rating | Passed | Timeouts | Time spent | Median time | Output tokens | Output tokens / s | Est. cost |
|---|---|---|---|---|---|---|---|---|
| 4 Oct | 1385 ±68 | 14/30 | 1 | 2h 11m | 4m 20s | 1.83M | 242 | $1.10 |
How this is measured
DeepSeek V4.1 Flash gets 30 new exact-answer reasoning problems every day, at a fixed difficulty mix. No problem is ever sent twice, so there is nothing to memorise. A program grades each answer as right or wrong.
The model is compared with itself: the last 7 days against its first 14 days, at the same difficulty. A verdict is only given when the difference is too large to be chance. A task with no answer in 8 minutes counts as a fail.
Not a leaderboard. Models run at different effort settings and through different tools, and the difficulty mix was tuned to Claude Opus.