🎲 AI Battle Arena

An evaluation-first arena for strategic interaction between AI agents.
AI Battle Arena cross-environment leaderboard
Haizhong Zheng Β· 07/02/2026
Part 1 Β· What we're building

Why benchmark strategic interaction?

Static benchmarks wear out

  • Saturate as models improve
  • Leak into training data (contamination)
  • Gameable; reward recall over decision-making

Strategic interaction gives a renewable skill signal

  • Interactive β†’ difficulty regenerates with the opponent
  • Demand hidden-info reasoning + long-horizon planning
  • Produce a natural, relative ranking
β†’ We built an arena where models act in 8 controlled game-theoretic environments under one identical pipeline, so results are directly comparable.
Part 1 Β· What we're building

One pipeline, many environments

  • One pipeline Β· many environments β€” same prompt / parse / retry / scoring everywhere
  • Position-neutral, opponent-adjusted β€” round-robin, seat-swapped, bootstrap Elo with error bars
  • Logs that explain β€” every step logged β†’ why-win / why-lose, not just a number
  • Agents, not models β€” agents see only observations, never hidden state β†’ private information stays private
EnvironmentInfo
Kuhn Pokerimperfect
Leduc Hold'emimperfect
Hold'em 1-Handimperfect
Hold'em Matchimperfect
Blackjack (vs dealer)imperfect
Connect Fourperfect
Gomoku-Liteperfect
Colonel Blottosimultaneous
Part 1 Β· What we're building

What we'll cover today

β‘ 

πŸ€– Model Arena β€” raw models, one simple prompt

  • Two strategic-interaction families: perfect-info (Connect Four Β· Gomoku) and hidden-info (Hold'em family Β· Leduc Β· Kuhn Β· Blotto Β· Blackjack)
  • The rankings β€” who wins, and who wins where
  • The findings β€” diagonal blind spot Β· pressure-vs-showdown Β· why models win & lose
β‘‘

πŸ› οΈ Harness Arena β€” beyond the raw model

  • Preliminary: what kind of harness / pipeline helps a model act more strategically
Model Arena Β· results

The cross-environment leaderboard

cross-environment leaderboard
Arena Rank Score β€” average finishing place (ordinal) Β· Arena Elo β€” margin-aware, Β±1 bootstrap SD Β· round-robin & seat-swapped so models that met different opponents stay comparable. Frontier closed models lead, but several open-weight models sit within the error bars, and rankings reshuffle by environment.
Model Arena Β· what strategic environments reveal

Case study

Three environments up close. For each: the ranking first, then two slices of behavior β€” and a replay to watch it happen.

πŸ”΄

Gomoku β€” do they spot & block threats?

πŸƒ

Hold'em 1-Hand β€” how chips are earned: pressure vs showdown

πŸƒ

Hold'em Match β€” chip management over 30 hands

Case study Β· Gomoku

Gomoku β€” the ranking

Gomoku leaderboard
GPT 5.5 and Kimi K2.6 top the field β€” perfect-information, so this is pure board skill. Where does the gap come from?
Case study Β· Gomoku Β· behavior

Offense is solved β€” defense decides

modelwin%win-take%block%late-game error %
Kimi K2.6691007523
GPT 5.568998022
GLM-5.161946832
GLM-5.259966733
DeepSeek V4 Pro58986832
GPT 5.455997333
Claude Opus 4.853926738
Claude Sonnet 4.650986540
Qwen3.7 Plus38936049
MiniMax-M327895061
GPT-OSS 120B18874368
MiniMax-M2.717904770
win-take is saturated (87–100%) β€” everyone grabs an available five. What fans out is defense: block% and late-game errors track win-rate almost 1:1. GPT 5.5 is the field's best blocker (80%, #2); GPT 5.4 sits mid-pack. A threat is missed 12.8% of the time overall.
Case study Β· Gomoku Β· in-depth

The blind spot is on the diagonal

threat axisfacedmissedmiss-rate
horizontal (contiguous in text)428245.6%
vertical (strided across rows)4014912.2%
diagonal β†˜ (down-right)2844315.1%
diagonal ↙ (down-left)2315624.2%
The board is fed as a 2-D grid flattened into 1-D text. Horizontal lines are contiguous β†’ easy; the more a line strays from reading order, the more it's missed. β†˜ flows with reading order, ↙ against β†’ ↙ missed most. A serialization artifact, not a skill gap.
β–Ά watch a full Gomoku battle β€” Kimi K2.6 vs MiniMax-M3
Case study Β· Hold'em 1-Hand

Hold'em 1-Hand β€” the ranking

Hold'em 1-Hand leaderboard
GPT 5.5 runs away (+80 bb/100). Not by holding better cards β€” by how it wins chips.
Case study Β· Hold'em 1-Hand Β· behavior

Where the chips come from

pressure vs showdown map
  • Pressure $ β€” won without showdown (folding opponents out) Β· Showdown $ β€” won at showdown (hand strength)
  • Winners earn by pressure β€” fold equity (GPT 5.5, far right); losers bleed at showdown β€” pay off second-best (bottom)
Case study Β· Hold'em 1-Hand Β· in-depth

Style & mistakes: three models compared

GPT 5.5
Claude Opus 4.8
MiniMax-M2.7
GPT 5.5 β€” loose-aggressive, earns by pressure Β· Claude Opus 4.8 β€” passive "calling station", leaks at showdown Β· MiniMax-M2.7 β€” weakest, drags weak hands to showdown. Aggression backed by discipline tracks Elo.   β–Ά watch: a big fold forced
Case study Β· Hold'em Match

Hold'em Match β€” the ranking

Hold'em Match leaderboard
Same models, longer-horizon poker setting: win-or-lose over 30 carry-stack hands. Chip management now matters more than any single pot.
Case study Β· Hold'em Match Β· why they win & lose

The match-only skill: shift gears with the score

Only across 30 carry-stack hands can a model read ahead vs behind and adapt β€” a single hand can't show it. Winners fight back when behind (aggression ↑, fold less); flat players get ground down.

modelaggression ahead→behindgear-shift Δbust-out%match win%
GPT 5.534% β†’ 51%+1717%65%
GPT 5.431% β†’ 52%+2112%61%
GLM-5.120% β†’ 32%+128%56%
Claude Opus 4.825% β†’ 25%012%40%
Qwen3.7 Plus14% β†’ 16%+25%39%
Top models crank aggression when behind (+17 / +21) and fight for pots; the bottom play the same whether winning or losing (β‰ˆ0) β†’ slowly ground down. Adapting to the scoreboard is the skill 1-Hand can't measure.
Case study Β· Hold'em Match Β· situational play

Short-stacked: gamble, don't freeze

When a model is short-stacked (about to lose the match), the correct play is to push β€” take a shot at a comeback, because playing it safe loses anyway. Aggression bucketed by effective stack:

modeldeep β‰₯40bbmid 15–40bbshort <15bb
GPT 5.540%37%86%*
GPT 5.440%32%24%
Kimi K2.633%24%30%
GLM-5.127%25%50%*
DeepSeek V4 Pro28%21%46%*
GLM-5.228%26%18%
Claude Sonnet 4.631%24%11%
MiniMax-M328%22%35%
Claude Opus 4.826%18%6%
MiniMax-M2.734%35%33%
Qwen3.7 Plus15%12%0%*
GPT 5.5 shoves when short (86%) to seek a comeback β€” it's losing anyway, so take the shot. The weak models freeze (Claude Opus 6%, Qwen 0%) and just die. (* the top models rarely get short at all β€” they're usually ahead β€” so this sample is small / suggestive.)
Case study Β· Hold'em Match Β· does 1-Hand skill carry over?

The single-hand skill mostly transfers

lead trajectory: % ahead at each of 30 hands
Each row = a model; 30 blocks = hands 1β†’30; green = ahead, red = behind. Single-hand strength largely carries into the match: the 1-Hand king GPT 5.5 leads wire-to-wire (green from hand 1), while the 1-Hand basement (Qwen, MiniMax) stays red. The exception is GPT 5.4 β€” mediocre per hand (1-Hand #7) but its gear-shift & discipline lift it to Match #2.
Harness Arena Β· where we're going

Today we measure the model out of the box

  • Current results: a single simple system prompt, each model plays independently β†’ a score
  • That measures the model as-is β€” not its ceiling
  • Scaffolding, tools, search, or training on strategic-interaction logs could move it a lot
  • (cf. the "Harness Arena": are we measuring the model, or the scaffolding?)
model
+
1 simple
system prompt
→
arena
→
score
Harness Arena Β· where we're going

New axis: the pipeline, not just the model

  • Research question: how much does the pipeline move performance?
  • Variants: richer prompts Β· tools/search Β· self-play Β· SFT / RL on interaction logs
  • The arena stays the fixed measuring stick across all variants
model
→
pipeline A
pipeline B
pipeline C
→
arena
same ruler
→
score
Harness Arena Β· early results β€” Hold'em 1-hand

Four pipelines vs the raw model

pipeline  (same model + inference-time scaffold)glm-5.2deepseek-v4kimi-k2p6
aggression_default  β€” bias toward betting65%63%68%
council  β€” multi-view synthesis73%75%82%
best_of_3  β€” sample & select61%58%61%
self_refine  β€” draft & critique70%52%57%

win-rate head-to-head vs the plain baseline (model + one plain system prompt) Β· 50% = tie with plain Β· n = 28–458 hands per cell

  • All four beat the raw model β€” a scaffold alone, no training, lifts 1-hand win-rate
  • Cheapest & robust: aggression_default = one instruction at 1 call β†’ 63–68% everywhere. Strongest: council (multi-view) β†’ 73–82%, but 4 calls
  • More calls β‰  automatic win: best_of_3 modest; self_refine model-dependent (strong on glm, ~tie elsewhere)
Harness Arena Β· pipeline 1 of 4

aggression_default β€” bias toward betting

  • Add one instruction to the prompt (still 1 model call):
    • "Default to aggression when unclear β€” a bet/raise wins two ways (value and fold equity); a call only wins when ahead."
    • "Prefer betting/raising over a passive call unless there's a clear reason."
  • Idea: LLMs play systematically too passively β€” one nudge corrects the bias
  • The cheapest scaffold here β€” no extra calls
vs plainwin%
glm-5.265%
deepseek-v463%
kimi-k2p668%

1 model call / decision

Harness Arena Β· pipeline 2 of 4

council β€” multi-view synthesis

  • Ask the same model for the best action from 3 expert viewpoints (separate calls, no decision yet):
    • aggressive β€” maximize pressure & fold equity
    • value β€” extract/protect, avoid spew
    • unexploitable β€” keep a balanced, hard-to-exploit range
  • A 4th call synthesizes the three views into one action
  • Idea: a single pass has blind spots β€” force opposing perspectives, then converge
vs plainwin%
glm-5.273%
deepseek-v475%
kimi-k2p682%

4 model calls / decision

Harness Arena Β· pipeline 3 of 4

best_of_3 β€” sample & select

  • Generate 3 candidate actions with different leanings:
    • neutral (the obvious line)
    • "consider a more aggressive line"
    • "consider a more cautious / pot-control line"
  • Then select the best of the three
  • Selection lever, not critique: pick among finished answers rather than re-thinking
vs plainwin%
glm-5.261%
deepseek-v458%
kimi-k2p661%

4 model calls / decision

Harness Arena Β· pipeline 4 of 4

self_refine β€” draft & critique

  • Draft an action with reasoning
  • Self-critique it: "is this the best legal action, or is a different one better?"
  • Revise in light of the critique β†’ final action
  • Iterative self-feedback (Self-Refine, Madaan et al. 2023)
vs plainwin%
glm-5.270%
deepseek-v452%
kimi-k2p657%

3 model calls / decision

Future work

Future work

Model Arena is done; Harness Arena is early. Three directions from here:

β‘ 

From evaluation β†’ a training environment

  • Ship the arena as a reinforcement-learning environment β€” so others can not just benchmark / evaluate models, but train them in it (self-play, RL on interaction logs)
β‘‘

From controlled interaction β†’ adversarial security

  • Extend AI Battle Arena into a real adversarial environment. Long-term, the contest isn't human-vs-AI β€” it's AI-vs-AI adversarial security β€” our next project
β‘’

From fixed environments β†’ auto-generated meta-games

  • Procedurally generate brand-new strategic environments that never existed before β€” no rules in any training set, no memorized strategy β€” so the arena tests on-the-fly reasoning on novel rules, not recalled play, and stays contamination-proof as models improve