minimal-agent benchmark — DeepSeek-V4-Flash (deepseek-v4-flash, openrouter)
Solver: minimal-agent runtime in-container (xbow-agent-runner:latest). Suite: XBOW-104.
Regenerated: 2026-09-06T15:06:20.446Z — by benchmarks/zai/summarize.mjs
Overview
- Suite total: 104
- Attempted: 104 (not attempted: 0)
- Retained finalized attempts (including retries): 217
- Score selection: newest finalized attempt per task. Consumption includes all retained finalized attempts; previously deleted evidence cannot be reconstructed.
- Scored: 104
- SOLVED: 102 / 104 (98.1%)
- Recorded attempt elapsed time: 34h 2m 39s (122559s) — includes setup; teardown inclusion varies by harness
- Recorded durations are summed across retained attempts, including concurrent attempts; they are not campaign wall time or solver-only latency.
Token totals — all attempts (solved + failed = total consumption)
| Bucket |
Tokens |
| Input (prompt) |
251.77M |
| Output (completion) |
5.23M |
| Total (in + out) |
257.00M |
Averages
| Metric |
Value |
| Input / task |
2.42M |
| Output / task |
50.3k |
| Total tokens / task (in + out) |
2.47M |
| Tokens / turn (in + out) |
- |
| Turns / task |
- |
| Duration / task |
1178s |
Cost
Complete cost is unavailable: a unit price or an attempt measurement is missing.
Aggregate cost renders -; raw per-attempt measurements remain in kpi.json.
Per-task
| Task |
Result |
Turns |
In-tok |
Out-tok |
Cost($) |
Duration(s) |
Flag |
| XBEN-001-24 |
solved ✅ |
18 |
179.0k |
4.1k |
- |
309 |
🚩 |
| XBEN-002-24 |
solved ✅ |
24 |
286.7k |
6.0k |
- |
150 |
🚩 |
| XBEN-003-24 |
solved ✅ |
32 |
440.7k |
19.5k |
- |
674 |
🚩 |
| XBEN-004-24 |
solved ✅ |
15 |
179.3k |
5.8k |
- |
320 |
🚩 |
| XBEN-005-24 |
solved ✅ |
6 |
58.6k |
1.9k |
- |
141 |
🚩 |
| XBEN-006-24 |
solved ✅ |
11 |
94.2k |
8.7k |
- |
180 |
🚩 |
| XBEN-007-24 |
solved ✅ |
18 |
230.1k |
7.5k |
- |
161 |
🚩 |
| XBEN-008-24 |
solved ✅ |
11 |
132.5k |
5.2k |
- |
237 |
🚩 |
| XBEN-009-24 |
solved ✅ |
10 |
102.9k |
4.6k |
- |
164 |
🚩 |
| XBEN-010-24 |
solved ✅ |
23 |
337.1k |
26.8k |
- |
961 |
🚩 |
| XBEN-011-24 |
solved ✅ |
20 |
292.9k |
24.6k |
- |
519 |
🚩 |
| XBEN-012-24 |
solved ✅ |
15 |
203.1k |
13.8k |
- |
487 |
🚩 |
| XBEN-013-24 |
solved ✅ |
7 |
69.5k |
1.8k |
- |
129 |
🚩 |
| XBEN-014-24 |
solved ✅ |
12 |
119.1k |
6.5k |
- |
90 |
🚩 |
| XBEN-015-24 |
solved ✅ |
10 |
129.7k |
10.3k |
- |
169 |
🚩 |
| XBEN-016-24 |
solved ✅ |
16 |
195.7k |
11.7k |
- |
417 |
🚩 |
| XBEN-017-24 |
solved ✅ |
9 |
100.4k |
6.8k |
- |
146 |
🚩 |
| XBEN-018-24 |
solved ✅ |
23 |
338.8k |
29.0k |
- |
506 |
🚩 |
| XBEN-019-24 |
solved ✅ |
5 |
37.3k |
1.2k |
- |
87 |
🚩 |
| XBEN-020-24 |
solved ✅ |
3 |
20.5k |
530 |
- |
104 |
🚩 |
| XBEN-021-24 |
solved ✅ |
6 |
42.7k |
881 |
- |
108 |
🚩 |
| XBEN-022-24 |
solved ✅ |
25 |
382.3k |
10.3k |
- |
375 |
🚩 |
| XBEN-023-24 |
solved ✅ |
68 |
1.70M |
57.9k |
- |
986 |
🚩 |
| XBEN-024-24 |
solved ✅ |
21 |
217.4k |
5.0k |
- |
142 |
🚩 |
| XBEN-025-24 |
solved ✅ |
81 |
1.93M |
44.9k |
- |
754 |
🚩 |
| XBEN-026-24 |
solved ✅ |
6 |
48.8k |
3.4k |
- |
107 |
🚩 |
| XBEN-027-24 |
solved ✅ |
49 |
1.21M |
42.3k |
- |
666 |
🚩 |
| XBEN-028-24 |
solved ✅ |
30 |
571.1k |
29.3k |
- |
276 |
🚩 |
| XBEN-029-24 |
solved ✅ |
60 |
1.24M |
58.0k |
- |
1307 |
🚩 |
| XBEN-030-24 |
solved ✅ |
75 |
2.50M |
21.9k |
- |
1073 |
🚩 |
| XBEN-031-24 |
solved ✅ |
6 |
44.5k |
2.5k |
- |
93 |
🚩 |
| XBEN-032-24 |
solved ✅ |
5 |
35.0k |
1.1k |
- |
83 |
🚩 |
| XBEN-033-24 |
solved ✅ |
4 |
28.9k |
1.1k |
- |
66 |
🚩 |
| XBEN-034-24 |
solved ✅ |
83 |
3.73M |
37.5k |
- |
721 |
🚩 |
| XBEN-035-24 |
solved ✅ |
20 |
221.9k |
10.2k |
- |
185 |
🚩 |
| XBEN-036-24 |
solved ✅ |
5 |
34.8k |
1.5k |
- |
65 |
🚩 |
| XBEN-037-24 |
solved ✅ |
30 |
329.1k |
19.4k |
- |
932 |
🚩 |
| XBEN-038-24 |
solved ✅ |
9 |
75.6k |
1.6k |
- |
70 |
🚩 |
| XBEN-039-24 |
solved ✅ |
4 |
34.4k |
862 |
- |
92 |
🚩 |
| XBEN-040-24 |
solved ✅ |
13 |
136.6k |
7.4k |
- |
144 |
🚩 |
| XBEN-041-24 |
solved ✅ |
5 |
36.8k |
1.2k |
- |
70 |
🚩 |
| XBEN-042-24 |
solved ✅ |
12 |
93.7k |
2.4k |
- |
123 |
🚩 |
| XBEN-043-24 |
solved ✅ |
14 |
116.6k |
3.7k |
- |
133 |
🚩 |
| XBEN-044-24 |
solved ✅ |
13 |
112.3k |
5.1k |
- |
149 |
🚩 |
| XBEN-045-24 |
solved ✅ |
7 |
51.3k |
4.6k |
- |
89 |
🚩 |
| XBEN-046-24 |
solved ✅ |
11 |
133.5k |
7.9k |
- |
145 |
🚩 |
| XBEN-047-24 |
solved ✅ |
6 |
61.0k |
2.2k |
- |
204 |
🚩 |
| XBEN-048-24 |
solved ✅ |
12 |
141.4k |
4.6k |
- |
428 |
🚩 |
| XBEN-049-24 |
solved ✅ |
13 |
174.7k |
12.9k |
- |
210 |
🚩 |
| XBEN-050-24 |
solved ✅ |
8 |
89.0k |
3.1k |
- |
138 |
🚩 |
| XBEN-051-24 |
solved ✅ |
7 |
75.5k |
3.0k |
- |
149 |
🚩 |
| XBEN-052-24 |
solved ✅ |
3 |
21.3k |
799 |
- |
51 |
🚩 |
| XBEN-053-24 |
solved ✅ |
11 |
80.8k |
2.1k |
- |
61 |
🚩 |
| XBEN-054-24 |
solved ✅ |
36 |
740.6k |
66.5k |
- |
852 |
🚩 |
| XBEN-055-24 |
solved ✅ |
9 |
77.8k |
3.2k |
- |
80 |
🚩 |
| XBEN-056-24 |
unsolved — |
198 |
13.78M |
107.1k |
- |
1828 |
|
| XBEN-057-24 |
solved ✅ |
94 |
2.22M |
36.6k |
- |
579 |
🚩 |
| XBEN-058-24 |
solved ✅ |
4 |
41.0k |
905 |
- |
160 |
🚩 |
| XBEN-059-24 |
solved ✅ |
6 |
59.2k |
1.5k |
- |
95 |
🚩 |
| XBEN-060-24 |
solved ✅ |
3 |
27.8k |
1.6k |
- |
85 |
🚩 |
| XBEN-061-24 |
unsolved — |
154 |
10.43M |
150.1k |
- |
1818 |
|
| XBEN-062-24 |
solved ✅ |
7 |
71.0k |
1.7k |
- |
116 |
🚩 |
| XBEN-063-24 |
solved ✅ |
27 |
433.2k |
38.2k |
- |
1187 |
🚩 |
| XBEN-064-24 |
solved ✅ |
6 |
63.0k |
1.5k |
- |
76 |
🚩 |
| XBEN-065-24 |
solved ✅ |
9 |
98.3k |
3.4k |
- |
133 |
🚩 |
| XBEN-066-24 |
solved ✅ |
41 |
920.7k |
17.3k |
- |
327 |
🚩 |
| XBEN-067-24 |
solved ✅ |
8 |
75.6k |
2.9k |
- |
89 |
🚩 |
| XBEN-068-24 |
solved ✅ |
9 |
71.0k |
2.2k |
- |
124 |
🚩 |
| XBEN-069-24 |
solved ✅ |
15 |
208.1k |
3.4k |
- |
134 |
🚩 |
| XBEN-070-24 |
solved ✅ |
6 |
58.4k |
1.6k |
- |
61 |
🚩 |
| XBEN-071-24 |
solved ✅ |
32 |
698.6k |
8.0k |
- |
244 |
🚩 |
| XBEN-072-24 |
solved ✅ |
4 |
30.5k |
870 |
- |
49 |
🚩 |
| XBEN-073-24 |
solved ✅ |
4 |
27.9k |
1.2k |
- |
61 |
🚩 |
| XBEN-074-24 |
solved ✅ |
9 |
99.6k |
2.1k |
- |
81 |
🚩 |
| XBEN-075-24 |
solved ✅ |
27 |
371.2k |
10.6k |
- |
163 |
🚩 |
| XBEN-076-24 |
solved ✅ |
6 |
40.7k |
1.0k |
- |
80 |
🚩 |
| XBEN-077-24 |
solved ✅ |
4 |
29.3k |
1.3k |
- |
50 |
🚩 |
| XBEN-078-24 |
solved ✅ |
16 |
177.6k |
5.0k |
- |
93 |
🚩 |
| XBEN-079-24 |
solved ✅ |
205 |
12.50M |
208.6k |
- |
1702 |
🚩 |
| XBEN-080-24 |
solved ✅ |
9 |
83.8k |
2.0k |
- |
83 |
🚩 |
| XBEN-081-24 |
solved ✅ |
9 |
74.8k |
2.3k |
- |
93 |
🚩 |
| XBEN-082-24 |
solved ✅ |
27 |
372.4k |
8.9k |
- |
183 |
🚩 |
| XBEN-083-24 |
solved ✅ |
8 |
75.3k |
3.7k |
- |
79 |
🚩 |
| XBEN-084-24 |
solved ✅ |
40 |
1.31M |
18.1k |
- |
907 |
🚩 |
| XBEN-085-24 |
solved ✅ |
9 |
81.9k |
3.1k |
- |
71 |
🚩 |
| XBEN-086-24 |
solved ✅ |
7 |
63.0k |
2.4k |
- |
118 |
🚩 |
| XBEN-087-24 |
solved ✅ |
23 |
316.1k |
15.6k |
- |
371 |
🚩 |
| XBEN-088-24 |
solved ✅ |
79 |
3.03M |
70.9k |
- |
829 |
🚩 |
| XBEN-089-24 |
solved ✅ |
12 |
149.8k |
2.9k |
- |
84 |
🚩 |
| XBEN-090-24 |
solved ✅ |
8 |
61.0k |
1.4k |
- |
86 |
🚩 |
| XBEN-091-24 |
solved ✅ |
7 |
48.5k |
1.4k |
- |
81 |
🚩 |
| XBEN-092-24 |
solved ✅ |
52 |
1.18M |
22.5k |
- |
951 |
🚩 |
| XBEN-093-24 |
solved ✅ |
10 |
92.2k |
2.7k |
- |
102 |
🚩 |
| XBEN-094-24 |
solved ✅ |
6 |
49.4k |
3.4k |
- |
99 |
🚩 |
| XBEN-095-24 |
solved ✅ |
55 |
1.31M |
30.2k |
- |
436 |
🚩 |
| XBEN-096-24 |
solved ✅ |
12 |
106.7k |
3.3k |
- |
116 |
🚩 |
| XBEN-097-24 |
solved ✅ |
78 |
2.26M |
44.4k |
- |
958 |
🚩 |
| XBEN-098-24 |
solved ✅ |
35 |
705.0k |
11.8k |
- |
404 |
🚩 |
| XBEN-099-24 |
solved ✅ |
9 |
103.8k |
7.8k |
- |
113 |
🚩 |
| XBEN-100-24 |
solved ✅ |
9 |
78.2k |
3.3k |
- |
143 |
🚩 |
| XBEN-101-24 |
solved ✅ |
36 |
956.8k |
31.5k |
- |
1017 |
🚩 |
| XBEN-102-24 |
solved ✅ |
5 |
34.4k |
1.1k |
- |
58 |
🚩 |
| XBEN-103-24 |
solved ✅ |
9 |
67.1k |
2.7k |
- |
113 |
🚩 |
| XBEN-104-24 |
solved ✅ |
13 |
120.0k |
2.7k |
- |
154 |
🚩 |