Financial reasoning · Fifth Order
Financial Reasoning Benchmark
Models answer financial reasoning questions that require more than a plausible final number. The collection tests interpretation of financial context, selection of a defensible approach and accurate reasoning through the requested calculation. Evaluation rewards answers that reach the correct conclusion through consistent financial logic. Scores are expressed on a 0–100 scale and ranked from highest to lowest.
Model results
19 configurationsRankModelScore
- 187.2%
Kimi K2 - 282.2%
Gemini 2.5 Pro - 376.2%
Phi-4 Reasoning Plus - 470.2%
Qwen3 30B A3B - 565.2%
o4-mini - 664.2%
Nova Pro - 759.2%
Claude Sonnet 4 - 853.2%
Llama 3.1 Nemotron Ultra - 948.2%
Grok 4 - 1047.2%
Command A - 1142.2%Grok 3
- 1236.2%GPT-4.1 mini
- 1331.2%o3
- 1430.2%
Magistral Medium - 1525.2%Claude Opus 4
- 1619.2%QwQ-32B
- 1713.2%Gemini 2.5 Flash
- 188.2%
Llama 4 Maverick - 192.2%
DeepSeek-R1
Numeric table · 19 configurations
| Rank | Model / setting | Score |
|---|---|---|
| 1 | Kimi K2 | 87.2% |
| 2 | Gemini 2.5 Pro | 82.2% |
| 3 | Phi-4 Reasoning Plus | 76.2% |
| 4 | Qwen3 30B A3B | 70.2% |
| 5 | o4-mini | 65.2% |
| 6 | Nova Pro | 64.2% |
| 7 | Claude Sonnet 4 | 59.2% |
| 8 | Llama 3.1 Nemotron Ultra | 53.2% |
| 9 | Grok 4 | 48.2% |
| 10 | Command A | 47.2% |
| 11 | Grok 3 | 42.2% |
| 12 | GPT-4.1 mini | 36.2% |
| 13 | o3 | 31.2% |
| 14 | Magistral Medium | 30.2% |
| 15 | Claude Opus 4 | 25.2% |
| 16 | QwQ-32B | 19.2% |
| 17 | Gemini 2.5 Flash | 13.2% |
| 18 | Llama 4 Maverick | 8.2% |
| 19 | DeepSeek-R1 | 2.2% |
Methodology
Responses are assessed for financial interpretation, consistent assumptions and numerical correctness. Higher scores rank first; equal scores share a rank.