$ ~/aibattle/blackjack
🂡 Blackjack · Model vs the built-in dealer · hit/stand/double · scored by chip profit · 9000 hands total
🎬 Watch featured replays →
Setup — Blackjack
Standard blackjack against a fixed house dealer (full rules on Wikipedia); this is a solitaire-style benchmark, not model-vs-model:- Model vs the built-in dealer, which plays the fixed house line (hits until 17, then stands). Player actions are hit / stand / double — no split or insurance.
- 500 hands per model, each from a fresh shuffle. Deals are independent — not shared across models — so the luck of the cards differs from model to model.
- Flat 1-unit bet per hand (2 units on a double); a natural pays 3:2.
Because deals aren't shared and the dealer keeps a fixed house edge, a negative field net is expected and short runs are luck-heavy — so models are ranked by mean profit per hand, not total. Bust / double / natural rates show how soundly each one plays.
What the model sees each turn: its own cards and running total, the dealer's up-card, and the legal actions (hit / stand / double) — not the dealer's hole card or the rest of the shoe.
🏆 Leaderboard
| # | model | mean/hand | win% | push% | loss% | bust% | double% | natural% | invalid% | hands | think | tokens/dec | $/1K dec |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 1 |  GPT 5.4 |
+0.060 | 46% | 8% | 46% | 16% | 11% | 5% | 0.0% | 500 | 6.7s | — | — |
| 2 |  MiniMax-M2.7 |
+0.041 | 46% | 9% | 45% | 14% | 12% | 4% | 0.0% | 500 | 38.8s | 7,667 | $9.20 |
| 3 |  GLM-5.1 |
+0.035 | 46% | 7% | 48% | 16% | 10% | 6% | 0.0% | 1000 | 15.8s | 561 | $2.47 |
| 4 | GPT-OSS 120B |
+0.033 | 45% | 8% | 47% | 17% | 13% | 6% | 0.0% | 500 | 1.5s | 147 | $0.09 |
| 5 | GPT 5.5 |
+0.020 | 42% | 12% | 46% | 15% | 9% | 5% | 0.0% | 500 | 2.7s | — | — |
| 6 |  Claude Opus 4.8 |
+0.014 | 44% | 8% | 48% | 16% | 11% | 4% | 0.0% | 500 | 1.3s | — | — |
| 7 | GLM-5.2 |
+0.004 | 44% | 8% | 48% | 15% | 11% | 4% | 0.0% | 1000 | 72.2s | 1,080 | $4.75 |
| 8 |  Qwen3.7 Plus |
+0.004 | 44% | 7% | 49% | 17% | 10% | 4% | 0.0% | 1000 | 5.2s | 1,030 | $1.65 |
| 9 | Claude Sonnet 4.6 |
-0.015 | 43% | 8% | 49% | 17% | 12% | 7% | 0.0% | 500 | 1.2s | — | — |
| 10 |  Kimi K2.6 |
-0.033 | 43% | 8% | 49% | 15% | 9% | 4% | 0.0% | 1000 | 75.3s | 5,919 | $23.68 |
| 11 |  DeepSeek V4 Pro |
-0.033 | 41% | 10% | 49% | 16% | 10% | 4% | 0.0% | 1000 | 89.0s | 239 | $0.83 |
| 12 | MiniMax-M3 |
-0.046 | 42% | 7% | 51% | 19% | 8% | 4% | 0.0% | 1000 | 9.2s | 745 | $0.89 |
columns: mean/hand avg profit/hand (units) · win% / push% / loss% vs dealer · bust% went over 21 · double% doubled down · natural% dealt a blackjack · invalid% illegal actions · hands hands played · think avg sec/decision
Each model plays independent hands against the built-in dealer; the dealer holds an
inherent house edge, so a negative field average is expected. mean/hand = average chips per hand
(a doubled hand pays ±2), the fair comparison since hand counts can differ. bust = player busted;
double = chose to double down; natural = dealt a blackjack. Read directly from per-hand outcomes.