Structure beats scale: $8K Stratego win, 30-point swing on agent orchestration

TL;DR

• Ataraxos beat the Stratego world champion on an $8,000 training budget.
• The same architecture set state-of-the-art on Hanabi (24.654/25) and topped Dou dizhu bots.
• Mollick invoked the Bitter Lesson to argue agent swarms are starting to self-organize.
• Practitioners countered with a 30-point GAIA swing from orchestration choice alone.
• Production post-mortems blame organizational memory loss as swarm size grows.

Two features, one argument against scale is all you need. Ataraxos beat the reigning Stratego world champion on an $8,000 training budget — roughly 1/100th the examples and 1/30th the self-play games of DeepMind’s 2022 DeepNash — and the same three-part architecture carried over to Hanabi and Dou dizhu. The win isn’t that compute doesn’t matter; it’s that a specific imperfect-information pattern did a lot of the work compute would otherwise have to brute-force.

The second feature is the same argument fought on agent terrain. Ethan Mollick reaches for Sutton’s Bitter Lesson to say swarms are starting to self-organize past hand-crafted scaffolding. Practitioners answer with numbers: a 30-point GAIA swing from orchestration choice alone, up to 50× cost spreads at similar accuracy, and production post-mortems that keep surfacing the same failure mode — agents lose context, loop, and get worse as the swarm grows.

Ataraxos beats top Stratego player on an $8,000 training budget

Source: ars-technica-ai · published 2026-10-01

TL;DR

• Ataraxos beat the reigning Stratego world champion using a three-part architecture trained for under $8,000.
• It used roughly 1/100th the training examples and 1/30th the self-play games of DeepMind’s 2022 DeepNash.
• The same architecture set state-of-the-art on Hanabi (24.654/25) and beat top Dou dizhu bots, suggesting a general imperfect-information pattern.
• A champion co-author flags one real gap: Ataraxos can’t adapt across a long match series the way humans learn opponents.

Generate the hidden state, then search over it

Stratego has been the last major board-game holdout against AI because most of the board is hidden — each side sees only its own piece ranks — and the number of plausible world-states explodes beyond what poker-style subgame solvers can handle ¹. DeepMind’s 2022 DeepNash cracked expert-level play by not modeling the hidden pieces at all: it used model-free Regularized Nash Dynamics to converge toward an unexploitable equilibrium, no search, no belief tracking ².

Ataraxos, published in Nature on 30 September, inverts that bet. It bolts three components together:

flowchart LR
    A[Observed board] --> B[Generative belief net<br/>samples hidden pieces]
    B --> C[Policy-value net<br/>evaluates candidate states]
    C --> D[Decision-time search<br/>over sampled worlds]
    D --> E[Move]

The generative belief network is the load-bearing idea: instead of reasoning over every possible opponent configuration, it samples a handful of plausible ones, and search runs over those ¹. That is what collapses the compute budget. MIT pegs training at under $8,000 — roughly 16 H100s for about a week — with two orders of magnitude fewer training examples than DeepNash ³⁴. This is being pitched as an algorithmic result, not a scaling result, and the numbers back that framing.

It generalizes — that’s the real headline

The Stratego-vs-human angle is the hook; the transfer results are the story for anyone outside games research. The same architecture averaged 24.654/25 in 2-player Hanabi with 77.53% perfect games, edging out SPARTA (24.61), and beat both DouZero and PerfectDou at Dou dizhu ⁵. Cooperative card games, adversarial card games, and adversarial board games under one roof is unusual — it suggests “sample beliefs, then search” is a design pattern for hidden-information decision-making, not a Stratego-specific hack. The obvious targets are negotiation, auction, and security domains where enumerating opponent states is hopeless but sampling them is tractable.

Where the champion pushes back

The sharpest dissent comes from inside the author list. Three-time world champion Vincent de Boer — who co-authored the earlier DeepNash paper — calls Ataraxos’s inability to adjust across a long series of games a “major handicap,” since a human like Niemeijer learns and exploits an opponent’s patterns over dozens of matches ⁶. Ataraxos replans every move from a blank slate. Human opponents also described its style as “preternaturally lucky,” pulling off bluffs they considered too risky to attempt ⁴; whether that reflects better probability calculus or a thin 20-game sample is unresolved.

One structural caveat worth flagging: the Nature paper reports efficiency gains over DeepNash but, per the write-ups available, does not include a head-to-head Ataraxos-vs-DeepNash match ². The superhuman claim rests on the human series. That is a strong result against the best human in history, but a different claim than “best Stratego agent ever built.” Both are probably true; only one is directly demonstrated.

---

Mollick’s agent swarm meets a 30-point orchestration gap

Source: one-useful-thing · published 2026-10-01

TL;DR

• Ethan Mollick argues agent swarms are starting to self-organize, invoking Sutton’s Bitter Lesson against hand-crafted scaffolding.
• Practitioners counter with a 30-point GAIA swing from orchestration choice alone and up to 50× cost spreads at similar accuracy.
• Production post-mortems blame “organizational memory loss” — agents forget context and loop, and it gets worse as the swarm grows.
• The Bitter Lesson stops at clean objectives: domains with strong priors like FFTs still beat raw compute.

The claim

Mollick’s essay reaches for Rich Sutton’s Bitter Lesson — the observation that general methods plus compute keep beating hand-engineered structure — and applies it to multi-agent systems. The dot (one model, one prompt) gives way to the swarm (many models coordinating), and in Mollick’s telling the swarm increasingly plans itself. Scaffolding, routers, hand-tuned handoffs: all of it is transitional cruft that bigger models and more compute will dissolve.

It is a clean story, and in the direction of travel it is directionally right. Uvik’s framework survey finds roughly 57% of organizations already running agents in production, and OpenAI quietly retired its stateless “Swarm” experiment in favor of SDKs with memory, tracing, and guardrails — exactly the trajectory Mollick describes ⁷. His own “Overhang” essay supplies the connective tissue: model capability has outrun what organizations know how to do with it ⁸.

The counter-evidence

The problem is that the people shipping these systems describe a very different bottleneck. A Machine Learning Mastery survey of current agentic deployments reports that the same underlying model can see a 30-point performance delta on GAIA depending solely on orchestration logic, with frameworks differing by up to 50× in cost for comparable accuracy ⁹. That is not compute winning monotonically. That is scaffolding dominating outcomes.

flowchart LR
    A[Same base model] --> B{Orchestration layer}
    B -->|Framework X| C[High GAIA score]
    B -->|Framework Y| D[30 pts lower, 50× cheaper]
    B -->|Framework Z| E[Loops, forgets, fails]

Sriram Narasimhan’s production post-mortem is blunter: multi-agent systems fail because they forget. Agents lose earlier context, loop endlessly, or diverge from the plan, and the error rate climbs in regulated environments where you cannot afford a confident hallucination ¹⁰. These failure modes get worse as the swarm grows, not better — the opposite of what the Bitter Lesson predicts.

Where the lesson actually stops

Two independent critiques frame the limit precisely. Siddharth Yadav’s “Bitter Lesson versus the Garbage Can” argues that most organizations cannot specify their objectives cleanly enough for brute-force optimization to bite in the first place — the lesson needs a well-posed reward ¹¹. Drew Breunig catalogues domains where engineered priors still dominate learned ones: signal processing has FFTs, physical simulation has numerical solvers, and no amount of compute has displaced them ¹².

As Breunig’s critique puts it, compute does not dissolve every structure.

Both critiques land in the same place. The Bitter Lesson applies when the problem can be represented as clean data with a clean objective. “Coordinate a team of agents against a fuzzy business goal with persistent state” is not that problem.

What’s at stake

If Mollick is right, investment in orchestration frameworks, agent memory systems, and hand-tuned handoffs is a depreciating asset — buy compute, wait a cycle, delete the scaffolding. If the practitioners are right, orchestration is the product for the next several years, and the 50× cost gap between frameworks is where the margin lives. The honest read of the research bundle is that Mollick has the trend line and his critics have the error bars. Both matter, and the error bars are currently larger than the trend.

Footnotes

1. The Decoder — https://the-decoder.com/ai-beats-strategos-greatest-player-ending-one-of-the-last-human-strongholds-in-board-games/

   AI techniques developed for games like poker definitely could not scale to Stratego’s explosion of possible universes; Ataraxos uses a three-pronged architecture — policy-value network, generative belief network, and decision-time search — to ‘zoom in’ on plausible board states.

   ↩ ↩²

2. DeepMind blog (DeepNash, 2022) — https://deepmind.google/blog/mastering-stratego-the-classic-game-of-imperfect-information/

   DeepNash avoids modeling hidden pieces explicitly, instead using model-free Regularized Nash Dynamics to converge to an unexploitable Nash equilibrium — a fundamentally different design than Ataraxos’s generative belief approach.

   ↩ ↩²

3. MIT News — https://news.mit.edu/2026/game-playing-ai-stratego-new-champ-0930

   Ataraxos reached superhuman levels using approximately one-hundredth of the training examples and one-thirtieth of the self-play games required by DeepMind’s DeepNash, with training costs under $8,000.

   ↩

4. Remio.ai developer writeup — https://www.remio.ai/post/ataraxos-stratego-ai-beat-the-best-human-and-it-cost-under-8-000-to-train

   Human opponents described Ataraxos’s play as ‘preternaturally lucky,’ pulling off bluffs humans consider too risky; the system was trained on roughly 16 H100 GPUs for about a week.

   ↩ ↩²

5. VMTech analysis — https://vmtech.rs/en/instagram-insights/ataraxos-ai-stratego-hidden-information

   In 2-player Hanabi, Ataraxos averaged 24.654/25 with 77.53% perfect games, surpassing SPARTA’s 24.61; it also defeated DouZero and PerfectDou at Dou dizhu, demonstrating the method generalizes beyond Stratego.

   ↩

6. SuperpowerDaily (summarizing Nature paper) — https://superpowerdaily.com/posts/researchers-ai-beats-top-stratego-players-with-far-less-training-than-deepnash

   Three-time world champion Vincent de Boer considers the AI’s inability to adjust its strategy across a long series of games a ‘major handicap,’ since a human like Niemeijer can learn and exploit an opponent’s patterns over time.

   ↩

7. Uvik, ‘Agentic AI Frameworks’ survey — https://uvik.net/blog/agentic-ai-frameworks/

   Over 57% of organizations now have agents in production; LangGraph dominates stateful workflows while CrewAI leads role-based automation, replacing the earlier experimental ‘Swarm’ project.

   ↩

8. Ethan Mollick, ‘The Overhang’ (One Useful Thing, prior essay) — https://www.oneusefulthing.org/p/the-overhang

   Capability has outrun deployment — the models can already do more than organizations have figured out how to use.

   ↩

9. Machine Learning Mastery, ‘The Current State of Agentic AI’ — https://machinelearningmastery.com/the-current-state-of-agentic-ai/

   The same model can see a 30-point performance delta on GAIA depending solely on orchestration logic, and some frameworks are up to 50x more expensive than others for similar accuracy.

   ↩

10. Sriram Narasimhan, ‘Why Multi-Agent Systems Fail in Production: They Forget’ (Medium) — https://sriram-narasim.medium.com/why-multi-agent-systems-fail-in-production-they-forget-bc8105b2e9e6

    Real-world failures often stem from organizational memory loss — agents forget earlier context or loop endlessly, leading to spiked error rates in regulated environments.

    ↩

11. Siddharth Yadav, ‘The Bitter Lesson versus the Garbage Can’ (sids.in) — https://sids.in/posts/the-bitter-lesson-versus-the-garbage-can

    Most organizations struggle to define their objectives clearly enough for an AI to optimize them; the Bitter Lesson only applies when the challenge can be represented as clean data.

    ↩

12. Drew Breunig, ‘Does the Bitter Lesson Have Limits?’ (dbreunig.com, Aug 2025) — https://www.dbreunig.com/2025/08/01/does-the-bitter-lesson-have-limits.html

    In specialized domains like signal processing, hand-crafted methods such as FFTs remain more efficient than deep learning — compute does not dissolve every structure.

    ↩