AA score/cost/time comparison

Snapshot #822 fetched 2026-07-09 23:00:41 · 1.28 MB raw HTML · 569 models

Snapshot

Pick the stored fetch snapshot to compare.

Run

Scoring

Choose the quality benchmark and final score formula.

Mode Calc

Cost adjustment

Tune formulas that account for Cost per Task.

Cost weight Cost floor Cost power

Result set

Control table filtering, ordering, and row count.

Sort Pareto Pareto axis Limit

Top quality model in this view: GPT-5.6 Sol (max) (63.4 pts, $1.04/task) · sorted by score

Cost/task vs quality?

97 models · 13 on the Pareto frontier · combined quality vs Cost per Task

Historic #1 winner?

Top row for each successful snapshot using the current mode/calc/cost settings and score sort.

Latest #1 GPT-5.6 Sol (max) 63.4

Recent winner changes

GPT-5.6 Sol (max) 2026-07-09 18:00:49 · 63.4 Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) 2026-06-19 08:00:57 · 63.1 Claude Opus 4.8 (Adaptive Reasoning, Max Effort) 2026-06-18 12:00:57 · 63.4 Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) 2026-06-11 21:00:18 · 69.2 Claude Opus 4.8 (Adaptive Reasoning, Max Effort) 2026-05-28 20:35:30 · 65.3 GPT-5.5 (xhigh) 2026-05-22 00:01:03 · 64.5

Run?	Fetched?	Winner?	Score?	Qual?	Cost/task?	Time/task?
#822	2026-07-09 23:00:41	GPT-5.6 Sol (max)	63.4	63.4	$1.04	-
#821	2026-07-09 22:00:42	GPT-5.6 Sol (max)	63.4	63.4	$1.04	-
#820	2026-07-09 21:00:43	GPT-5.6 Sol (max)	63.4	63.4	$1.04	-
#819	2026-07-09 20:00:44	GPT-5.6 Sol (max)	63.4	63.4	$1.04	-
#818	2026-07-09 19:00:45	GPT-5.6 Sol (max)	63.4	63.4	$1.04	-
#817	2026-07-09 18:00:49	GPT-5.6 Sol (max)	63.4	63.4	$1.04	-
#816	2026-07-09 14:00:45	Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)	63.1	63.1	$2.75	4.76m
#815	2026-07-09 13:00:46	Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback)	63.1	63.1	$2.75	4.76m

#?	Pareto?	Model?	Released?	Cost/task?	Time/task?	$/Q?	Qual?	ΔTop?	Intel?	Code?	Agent?	Pen?	Score?
1	✓	GPT-5.6 Sol (max) OpenAI	2026-07-09	$1.04	-	0.0164	63.4	0.0	58.9	77.4	54.0	20.2	63.4
2		Claude Fable 5 (Adaptive Reasoning, Max Effort, Opus 4.8 Fallback) Anthropic	2026-06-09	$2.75	4.85m	0.0436	63.1	-0.4	59.9	76.5	52.8	24.4	63.1
3	✓	GPT-5.6 Sol (xhigh) OpenAI	2026-07-09	$0.682	-	0.0109	62.6	-0.8	57.7	78.3	51.8	18.3	62.6
4	✓	GPT-5.6 Sol (high) OpenAI	2026-07-09	$0.453	-	0.0075	60.5	-2.9	55.9	77.2	48.5	16.6	60.5
5		GPT-5.6 Terra (max) OpenAI	2026-07-09	$0.554	1.89m	0.0093	59.7	-3.8	55.0	76.7	47.4	17.4	59.7
6		Claude Opus 4.8 (Adaptive Reasoning, Max Effort) Anthropic	2026-05-28	$1.80	6.93m	0.0304	59.0	-4.4	55.7	74.3	47.2	22.5	59.0
7		GPT-5.5 (xhigh) OpenAI	2026-04-23	$0.863	3.84m	0.0148	58.2	-5.2	54.8	74.9	44.9	19.4	58.2
8	✓	GPT-5.6 Sol (medium) OpenAI	2026-07-09	$0.314	-	0.0054	58.1	-5.3	53.6	76.3	44.5	15.0	58.1
9	✓	Grok 4.5 (high) SpaceXAI	2026-07-08	$0.311	2.59m	0.0054	57.3	-6.1	53.8	72.4	45.7	14.9	57.3
10		Claude Sonnet 5 (Adaptive Reasoning, Max Effort) Anthropic	2026-06-30	$1.53	7.96m	0.0267	57.2	-6.2	53.4	71.5	46.7	21.8	57.2
11		Claude Opus 4.7 (Adaptive Reasoning, Max Effort) Anthropic	2026-04-16	$1.97	5.98m	0.0345	57.2	-6.3	53.5	73.6	44.4	23.0	57.2
12	✓	GPT-5.6 Luna (max) OpenAI	2026-07-09	$0.209	1.52m	0.0037	56.1	-7.3	51.2	71.4	45.6	13.2	56.1
13		GPT-5.5 (high) OpenAI	2026-04-23	$0.610	2.52m	0.0109	56.1	-7.4	53.1	71.6	43.5	17.9	56.1
14		GPT-5.6 Terra (xhigh) OpenAI	2026-07-09	$0.327	1.18m	0.0059	55.6	-7.8	51.6	70.6	44.7	15.1	55.6
15		GPT-5.4 (xhigh) OpenAI	2026-03-05	$0.887	3.49m	0.0163	54.5	-8.9	51.4	71.1	41.1	19.5	54.5
16		GLM-5.2 (max) Z AI	2026-06-16	$0.370	3.19m	0.0068	54.3	-9.1	51.1	68.8	43.1	15.7	54.3
17	✓	GPT-5.6 Luna (xhigh) OpenAI	2026-07-09	$0.139	60s	0.0026	53.5	-9.9	49.1	68.6	42.9	11.4	53.5
18		GPT-5.5 (medium) OpenAI	2026-04-23	$0.344	1.36m	0.0065	53.2	-10.2	50.4	71.5	37.8	15.4	53.2
19		GPT-5.6 Sol (low) OpenAI	2026-07-09	$0.197	-	0.0037	53.0	-10.4	49.4	69.7	40.0	13.0	53.0
20		Gemini 3.5 Flash (high) Google	2026-05-19	$0.586	2.80m	0.0111	52.6	-10.8	50.2	70.1	37.4	17.7	52.6
21		GPT-5.6 Terra (high) OpenAI	2026-07-09	$0.236	45s	0.0045	52.5	-11.0	49.0	67.1	41.3	13.7	52.5
22		Claude Sonnet 4.6 (Adaptive Reasoning, Max Effort) Anthropic	2026-02-17	$1.14	12.2m	0.0226	50.3	-13.1	47.2	63.0	40.8	20.6	50.3
23	✓	GPT-5.6 Luna (high) OpenAI	2026-07-09	$0.0948	34s	0.0019	49.8	-13.6	46.1	63.3	40.1	9.8	49.8
24		GPT-5.6 Terra (medium) OpenAI	2026-07-09	$0.128	24s	0.0026	49.1	-14.3	45.6	64.7	37.0	11.1	49.1
25		Qwen3.7 Max Alibaba	2026-05-19	$1.06	1.57m	0.0223	47.5	-15.9	46.0	66.0	30.6	20.3	47.5
26		GPT-5.6 Sol (Non-reasoning) OpenAI	2026-07-09	$0.200	-	0.0043	47.1	-16.4	41.2	65.1	34.9	13.0	47.1
27	✓	DeepSeek V4 Pro (Reasoning, Max Effort) DeepSeek	2026-04-24	$0.0448	8.25m	0.0010	46.7	-16.8	44.3	59.4	36.4	6.5	46.7
28		MiniMax-M3 MiniMax	2026-06-01	$0.125	4.14m	0.0027	46.1	-17.3	44.4	58.6	35.4	11.0	46.1
29		Gemini 3.1 Pro Preview Google	2026-02-19	$0.291	1.47m	0.0064	45.6	-17.9	46.5	68.8	21.4	14.6	45.6
30		Kimi K2.6 Kimi	2026-04-20	$0.346	14.8m	0.0076	45.4	-18.0	44.2	61.8	30.3	15.4	45.4
31		GPT-5.5 (low) OpenAI	2026-04-23	$0.193	34s	0.0043	44.9	-18.5	43.5	60.9	30.4	12.9	44.9
32		Kimi K2.7 Code Kimi	2026-06-12	$0.178	6.39m	0.0040	44.1	-19.3	41.9	60.8	29.6	12.5	44.1
33	✓	MiMo-V2.5-Pro Xiaomi	2026-04-22	$0.0314	7.11m	0.0007	43.8	-19.6	42.2	60.2	29.1	5.0	43.8
34		GPT-5.6 Terra (low) OpenAI	2026-07-09	$0.101	15s	0.0023	43.1	-20.4	40.5	58.1	30.6	10.0	43.1
35	✓	DeepSeek V4 Flash (Reasoning, Max Effort) DeepSeek	2026-04-24	$0.0223	5.61m	0.0005	42.5	-20.9	40.3	56.2	31.1	3.5	42.5
36		GPT-5.4 mini (xhigh) OpenAI	2026-03-17	$0.479	7.94m	0.0114	42.1	-21.4	40.0	56.1	30.2	16.8	42.1
37		GLM-5.1 (Reasoning) Z AI	2026-04-07	$0.226	5.45m	0.0054	41.9	-21.5	40.2	55.8	29.9	13.5	41.9
38		GPT-5.4 nano (xhigh) OpenAI	2026-03-17	$0.127	7.05m	0.0031	40.6	-22.8	38.2	56.1	27.5	11.0	40.6
39		Qwen3.6 Plus Alibaba	2026-04-02	$0.314	6.41m	0.0077	40.5	-22.9	39.6	54.5	27.6	15.0	40.5
40		GPT-5.6 Luna (medium) OpenAI	2026-07-09	$0.0504	15s	0.0013	39.9	-23.5	38.1	50.7	31.0	7.0	39.9
41		Grok Build 0.1 0616 SpaceXAI	-	$0.212	4.59m	0.0053	39.8	-23.7	39.8	51.5	28.0	13.3	39.8
42		Qwen3.6 27B (Reasoning) Alibaba	2026-04-22	$0.265	6.91m	0.0068	39.3	-24.2	37.1	53.7	27.0	14.2	39.3
43		GPT-5.5 (Non-reasoning) OpenAI	2026-04-23	$0.151	26s	0.0039	39.2	-24.2	35.4	56.5	25.8	11.8	39.2
44		Qwen3.7 Plus Alibaba	2026-06-01	$0.219	6.76m	0.0057	38.5	-24.9	39.0	55.9	20.8	13.4	38.5
45		GPT-5.6 Terra (Non-reasoning) OpenAI	2026-07-09	$0.115	14s	0.0030	38.5	-24.9	34.0	52.3	29.3	10.6	38.5
46		Nemotron 3 Ultra 550B A55B (Reasoning) NVIDIA	2026-06-04	$0.244	2.05m	0.0064	38.1	-25.3	37.8	49.3	27.4	13.9	38.1
47		Claude 4.5 Sonnet (Reasoning) Anthropic	2025-09-29	$0.413	5.20m	0.0110	37.7	-25.7	36.4	52.1	24.6	16.2	37.7
48		GPT-5.1 (high) OpenAI	2025-11-13	$0.269	3.98m	0.0075	35.8	-27.7	36.9	49.4	21.0	14.3	35.8
49		GLM-4.7 (Reasoning) Z AI	2025-12-22	$0.297	4.24m	0.0085	34.8	-28.6	33.7	45.3	25.4	14.7	34.8
50		Grok 4.3 (high) SpaceXAI	2026-04-30	$0.139	2.23m	0.0040	34.6	-28.8	37.6	42.2	24.1	11.4	34.6
51		GPT-5.6 Luna (low) OpenAI	2026-07-09	$0.0405	9.2s	0.0012	34.3	-29.2	33.3	44.2	25.4	6.1	34.3
52		Qwen3.5 397B A17B (Reasoning) Alibaba	2026-02-16	$0.333	5.13m	0.0098	33.9	-29.5	33.7	48.2	19.8	15.2	33.9
53		Qwen3.6 27B (Non-reasoning) Alibaba	2026-04-22	$0.353	2.95m	0.0106	33.5	-30.0	30.5	46.6	23.3	15.5	33.5
54		Qwen3.5 122B A10B (Reasoning) Alibaba	2026-02-24	$0.241	1.92m	0.0073	32.9	-30.5	32.3	45.7	20.7	13.8	32.9
55		GPT-5 (high) OpenAI	2025-08-07	$0.212	2.65m	0.0065	32.7	-30.7	34.7	37.8	25.7	13.3	32.7
56		Mistral Medium 3.5 Mistral	2026-04-29	$1.08	2.96m	0.0339	31.9	-31.5	29.9	46.9	19.0	20.3	31.9
57		Qwen3.6 35B A3B (Reasoning) Alibaba	2026-04-16	$0.179	2.96m	0.0056	31.6	-31.8	31.6	41.9	21.4	12.5	31.6
58		Ring-2.6-1T InclusionAI	2026-05-08	$0.345	3.28m	0.0112	30.8	-32.7	30.6	42.8	18.9	15.4	30.8
59		GLM-4.6 (Reasoning) Z AI	2025-09-30	$0.283	7.13m	0.0092	30.7	-32.7	28.7	45.8	17.7	14.5	30.7
60		Claude 4.5 Haiku (Reasoning) Anthropic	2025-10-15	$0.237	3.61m	0.0079	30.0	-33.5	29.6	43.9	16.4	13.8	30.0
61		GPT-5.6 Luna (Non-reasoning) OpenAI	2026-07-09	$0.0547	7.9s	0.0019	29.3	-34.1	26.6	39.3	22.0	7.4	29.3
62		Qwen3.5 122B A10B (Non-reasoning) Alibaba	2026-02-24	$0.176	53s	0.0061	28.9	-34.5	27.6	43.3	15.8	12.5	28.9
63		MiMo-V2-Flash (Non-reasoning) Xiaomi	2025-12-16	$0.351	-	0.0122	28.8	-34.6	24.7	49.8	12.0	15.5	28.8
64		Grok 4.3 (Non-reasoning) SpaceXAI	2026-04-30	$0.293	1.47m	0.0106	27.6	-35.8	24.8	35.2	22.8	14.7	27.6
65		Gemma 4 26B A4B (Reasoning) Google	2026-04-02	$0.0294	-	0.0012	25.3	-38.1	25.7	39.3	11.0	4.7	25.3
66		NVIDIA Nemotron 3 Super 120B A12B (Reasoning) NVIDIA	2026-03-11	$0.206	2.68m	0.0086	23.9	-39.5	25.4	37.7	8.7	13.1	23.9
67		gpt-oss-120b (high) OpenAI	2025-08-05	$0.0607	2.06m	0.0027	22.5	-40.9	23.8	30.4	13.2	7.8	22.5
68		Gemini 2.5 Pro Google	2025-06-05	$0.198	1.36m	0.0090	22.1	-41.4	25.8	33.3	7.1	13.0	22.1
69		Qwen3 Coder Next Alibaba	2026-02-03	$0.328	2.63m	0.0149	22.0	-41.4	21.1	36.2	8.8	15.2	22.0
70		Gemini 3.1 Flash-Lite Google	2026-03-03	$0.0416	1.05m	0.0019	22.0	-41.5	25.0	34.7	6.2	6.2	22.0
71		Nova 2.0 Pro Preview (medium) Amazon	2025-11-27	$0.173	1.62m	0.0083	20.9	-42.5	21.8	34.0	7.0	12.4	20.9
72		GPT-5 mini (high) OpenAI	2025-08-07	$0.0294	1.89m	0.0015	20.1	-43.3	25.3	15.6	19.4	4.7	20.1
73		Qwen3.5 9B (Reasoning) Alibaba	2026-03-02	$0.164	8.21m	0.0085	19.2	-44.3	21.4	28.7	7.4	12.1	19.2
74		Nova 2.0 Pro Preview (low) Amazon	2025-11-27	$0.213	42s	0.0123	17.3	-46.1	19.6	25.9	6.4	13.3	17.3
75	✓	HyperNova 60B 2605 Multiverse Computing	2026-05-26	$0.0143	1.31m	0.0009	15.9	-47.5	17.8	23.2	6.7	1.5	15.9
76		DeepSeek R1 (Jan '25) DeepSeek	2025-01-20	$0.247	-	0.0160	15.4	-48.0	18.5	24.6	3.1	13.9	15.4
77		Mistral Small 3.1 Mistral	2025-03-17	$0.0410	30s	0.0027	15.4	-48.0	14.7	26.3	5.2	6.1	15.4
78		Nova 2.0 Lite (high) Amazon	2025-10-29	$0.245	5.44m	0.0166	14.8	-48.6	18.2	23.0	3.1	13.9	14.8
79		Mistral Large 3 Mistral	2025-12-02	$0.0599	1.18m	0.0043	13.8	-49.6	15.9	20.1	5.5	7.8	13.8
80		Mistral Medium 3.1 Mistral	2025-08-12	$0.145	56s	0.0105	13.8	-49.6	14.7	20.5	6.2	11.6	13.8
81		Magistral Medium 1.2 Mistral	2025-09-18	$0.747	3.42m	0.0546	13.7	-49.7	17.9	21.3	1.9	18.7	13.7
82		DeepSeek V3 (Dec '24) DeepSeek	2024-12-26	$0.0254	-	0.0020	12.9	-50.5	14.2	23.0	1.6	4.0	12.9
83		gpt-oss-20b (high) OpenAI	2025-08-05	$0.0178	1.69m	0.0014	12.9	-50.5	14.9	20.7	3.1	2.5	12.9
84		DeepSeek V3 0324 DeepSeek	2025-03-25	$0.190	-	0.0149	12.7	-50.7	15.4	21.2	1.5	12.8	12.7
85		Nova 2.0 Pro Preview (Non-reasoning) Amazon	2025-11-27	$0.246	31s	0.0194	12.7	-50.7	14.4	20.9	2.9	13.9	12.7
86		GPT-4.1 mini OpenAI	2025-04-14	$0.0578	39s	0.0047	12.2	-51.2	14.8	20.2	1.7	7.6	12.2
87		Qwen3 Next 80B A3B (Reasoning) Alibaba	2025-09-11	$0.162	1.82m	0.0135	12.1	-51.4	16.7	17.4	2.1	12.1	12.1
88		Llama 4 Maverick Meta	2025-04-05	$0.0316	19s	0.0030	10.6	-52.8	14.3	16.3	1.3	5.0	10.6
89		NVIDIA Nemotron 3 Nano 30B A3B (Reasoning) NVIDIA	2025-12-15	$0.0183	2.31m	0.0018	10.2	-53.2	14.2	14.4	2.0	2.6	10.2
90		Qwen3 30B A3B 2507 (Reasoning) Alibaba	2025-07-30	$0.0715	2.65m	0.0076	9.4	-54.0	14.4	12.1	1.8	8.5	9.4
91		Ministral 3 14B Mistral	2025-12-02	$0.149	2.60m	0.0162	9.2	-54.2	11.1	14.4	2.2	11.7	9.2
92		Magistral Small 1.2 Mistral	2025-09-17	$0.254	1.92m	0.0277	9.2	-54.3	11.3	14.7	1.4	14.0	9.2
93		Mistral Small 3.2 Mistral	2025-06-20	$0.124	53s	0.0148	8.4	-55.1	10.6	12.5	2.0	10.9	8.4
94		Llama 3.3 Instruct 70B Meta	2024-12-06	$0.0798	19s	0.0110	7.2	-56.2	9.4	11.9	0.3	9.0	7.2
95		Ministral 3 8B Mistral	2025-12-02	$0.185	3.06m	0.0279	6.6	-56.8	9.0	9.7	1.2	12.7	6.6
96	✓	Llama 4 Scout Meta	2025-04-05	$0.00851	13s	0.0013	6.4	-57.0	10.0	8.2	1.1	0.0	6.4
97		Ministral 3 3B Mistral	2025-12-02	$0.126	1.90m	0.0288	4.4	-59.0	6.8	4.8	1.6	11.0	4.4