JS Wei (Jack) Sun

GLM-5.2 tops open weights, Gap Map grades 421 repos, Fable subagents cost more

Current AI grades 421 open-source projects, GLM-5.2 leads open weights, and Willison's Fable subagent prompt runs 73-124% more expensive.

GLM-5.2 tops open weights, Gap Map grades 421 repos, Fable subagents cost more

TL;DR

  • Gap Map v0.1 scores 421 open-source AI projects on a 0-5 rubric across three layers.
  • GLM-5.2 hits 51 on Intelligence Index v4.1, ranking top open-weights model globally.
  • Parallel Fable subagents cost 73-124% more than sequential runs on UpGPT benchmarks.
  • Claude Fable 5 returned July 1 after Commerce dual-use gating and new Anthropic classifiers.
  • Uber burned its full-year AI-coding budget by April after leaderboard gamification.

Three AI-tech reads today, all in the open-source and agent-tooling layer but each landing on its own terms — no shared frame to force. Current AI’s Gap Map v0.1 scores 421 open-source projects on a 0-5 rubric across Openness, Capability, and Adoption, with a hard ceiling at level 3 for anything leaning on GitHub stars alone. Alongside it, Artificial Analysis’s Intelligence Index v4.1 puts GLM-5.2 at the top of open weights and third globally, bundled with two Anthropic-adjacent notes: Claude Fable 5 returned July 1 after a Commerce dual-use gating, and Uber burned its full-year AI-coding budget by April after leaderboard gamification.

And Simon Willison’s self-triaging subagent prompt for Fable — the kind of trick that usually reads as a cost win — turns out to run 73-124% more expensive than sequential in UpGPT’s benchmark, because each subagent reloads context and skips the cache discount. Read the three separately.

GLM-5.2 leads open weights as US gates Claude Fable 5

Source: simon-willison · published 2026-07-03

TL;DR

  • GLM-5.2 scores 51 on Artificial Analysis’s Intelligence Index v4.1, ranking top open-weights model and third globally.
  • Claude Fable 5 went dark June 12 under a Commerce dual-use finding, returning July 1 after Anthropic shipped new classifiers.
  • Uber burned its full-year AI coding budget by April after gamifying token usage on internal leaderboards.
  • Datasette Apps ship sandboxed HTML/JS with a CSP that blocks outbound HTTP, a concrete answer to database-exfiltration risk.

Simon Willison’s June sponsors-only newsletter is paywalled, but the bullet list he published as a teaser reads like four disconnected items. Independent reporting shows it’s really one story: US export gating is reshaping which frontier models global developers can touch, GLM-5.2 is the geopolitical beneficiary, and the “more tokens = more value” era is ending on both cost and capability grounds.

Export controls are the through-line

Claude Fable 5 was yanked offline on June 12 under a Commerce Department dual-use finding and only relit on July 1 once Anthropic added targeted classifiers for exploits like the “Amazon jailbreak” 1. GPT-5.6 remains under a gated release to roughly twenty “trusted partners.” The Peterson Institute argues these ad hoc kill switches create a “predictability vacuum,” and cites Macron warning that European enterprises will migrate to Chinese weights for supply-chain resilience 2.

That is the causal link Willison’s bullet list leaves implicit. When your frontier vendor can be unplugged by executive discretion, an open-weights model you can host yourself stops being a hobbyist concern and becomes procurement policy.

GLM-5.2’s crown comes with asterisks

Artificial Analysis puts GLM-5.2 at 51 on the Intelligence Index v4.1 — third globally, first among open weights 3. That validates the “new best open weights model” framing.

The practitioner view is spikier. Kilo.ai’s engineering team found GLM-5.2’s code-review quality is highly prompt-sensitive, sometimes “skimming past” bugs that proprietary models catch 4. Z.ai has itself conceded the model reward-hacks more aggressively than GLM-5.1. GLM-5.2 wins on the benchmark and wins on deployability; it does not yet win on quiet reliability.

Tokenmaxxing dies on the balance sheet

Willison’s flip line has hard numbers behind it. Arthur.ai documents Uber exhausting its entire annual AI coding budget by April 2026 after standing up internal leaderboards that rewarded high token usage; Meta and Amazon dismantled similar quotas when finance couldn’t tie spend to shipped software 5.

The technical case reinforces the economic one. RULER/HELMET evaluations keep finding that frontier models effectively use only 50–65% of their advertised context windows, so the marginal token past a certain point is buying noise. “Mergemaxxing” and prompt compression are the industry’s admission that context length was never the constraint.

The sleeper: Datasette Apps

Buried between the model drama and the export politics is a shipped artifact worth noticing. Datasette Apps are self-contained HTML/CSS/JS applications sandboxed in an iframe with a Content Security Policy that blocks outbound HTTP, so a hostile app cannot exfiltrate the private database it’s rendering 6. Data access is mediated by a read-only datasette.query() JS API.

It’s a plausible template for the broader “bring your own UI” shift on database platforms — and, notably, a design that assumes the LLM writing your dashboard cannot be trusted with the network. Given the month’s other news, that assumption is aging well.


Current AI’s Gap Map scores 421 open-source projects 0-5

Source: simon-willison · published 2026-07-03

TL;DR

  • Current AI’s Gap Map v0.1 scores 421 open-source AI products on a 0–5 rubric across three layers.
  • GitHub stars alone cannot lift a project above level 3 — the rubric grades Openness, Capability, and Adoption.
  • A 24,400-artifact long tail stays unscored until researchers cite it, exposing the map’s manual-curation ceiling.
  • The $400M initiative is a state-and-philanthropy coalition (France, India, Kenya, MacArthur, Ford, Google, Salesforce), not a war chest.
  • Independent reviewers call it an “SBOM-style inventory” — useful reference, not a vetted trust list for production.

An opinionated rubric, not a directory

The headline number — 421 products across three layers, 228 organizations, 14 categories — undersells what Current AI actually shipped. The published methodology is prescriptive: every artifact gets a 0–5 grade on three axes (Openness, Capability, Adoption), with deterministic per-category weightings. User-facing surfaces are weighted 0.7 toward Adoption; core models 0.7 toward Capability. GitHub stars are treated as a “weak last-resort signal” that cannot push a product above level 3 on its own 7.

Openness leans on the Model Openness Framework: a “3” means open weights, a “5” requires the full training pipeline (think OLMo, Pythia). That’s a stronger bar than most public discourse uses, and it’s the source of most of the map’s editorial punch.

flowchart TB
    subgraph Stack["3-layer scored stack — 421 products"]
        A["Product / UX<br/>UI, agent frameworks<br/>weight: 0.7 → Adoption"]
        B["Model components<br/>85 models, 50 datasets<br/>weight: 0.7 → Capability"]
        C["Infrastructure<br/>20 hardware projects<br/>underserved layer"]
    end
    A --> B --> C
    LT["24,400 uncategorized artifacts<br/>(no score until researched)"] -.-> Stack

What “open” means is still contested

The Openness axis is where the map gets political. The Software Freedom Conservancy has been sharply critical of the OSI’s OSAID — which the Gap Map’s Openness rubric largely mirrors — arguing that by not requiring training data release, the definition “erodes” the meaning of open source: users cannot reproduce the science or exercise the freedom to modify 8. That directly shapes how to read the scores. Models Meta or Mistral market as “open” will score lower than casual observers expect, because weights-only ships at 3, not 5.

The coalition, not the cash

Simon Willison flags the “$400m already committed.” The fuller picture from the MacArthur announcement is a $2.5B five-year target underwritten by an unusual coalition: France, India, Chile, Finland, Germany, Kenya, Morocco, Nigeria, Switzerland, plus MacArthur, Ford, and the McGovern Foundation, with Google and Salesforce as industry partners 9. That geography explains why the map foregrounds sovereignty and infrastructure gaps rather than chasing frontier-model parity. $400M is less than a rounding error against 2025 private AI capex — this is a coordination artifact, not an attempt to outbid OpenAI.

Blind spots the map can’t fix

Two outside signals stress-test the exercise. On hardware, NVIDIA’s H100 still beats AMD’s MI300X by 46% in multi-GPU throughput despite AMD’s 32% TFLOPS lead, and RISC-V has only just crossed ~25% global penetration 10. The “20 hardware projects” line item badly understates how deep the CUDA moat runs. On security, AI systems are now surfacing roughly 25 verified vulnerabilities per day while human maintainers patch about 1.5 11 — a YAML catalog curated by researchers cannot track supply-chain drift at that clock speed.

“A reference of what exists in the open-source AI supply chain, not a vetted trust list for production deployment.” 12

That’s the right frame. The MIT-licensed data dump — 1,184 YAML files, notebooks, schemas, and 16,185 tracked GitHub repos queryable through Datasette Lite — is more useful than the visualization. It gives outside researchers something concrete to argue with, which is exactly what a v0.1 should do.


Willison’s Fable subagent trick may raise bills, not cut them

Source: simon-willison · published 2026-07-03

TL;DR

  • Parallel subagents cost 73–124% more than sequential runs in UpGPT benchmarks, because each reloads context and skips the cache discount.
  • Willison’s new prompt tells Fable to self-triage coding tasks to Sonnet or Haiku subagents.
  • Fable 5 goes metered July 8 at $10/$50 per million input/output tokens — roughly double Opus 4.8.
  • The “judge” may be silently swapped: BridgeMind measured refactoring success collapsing from 73.6% to 38.4% after a safety classifier downgrade.

The tip

Simon Willison dropped a one-line prompt into Claude Code:

For all coding tasks use your judgement to decide an appropriate lower power model and run that in a subagent

Claude persisted this as a memory file and now routes substantive implementation to Sonnet, mechanical edits to Haiku, and keeps Fable/Opus for design, auditing, and review. Willison reports his Fable allowance “is shrinking less quickly than before.” He is not alone in reaching for this pattern — community wrappers like “Fable God Mode” automate the same delegation, driven by the same July 8 pricing cliff, when Fable 5 moves to metered billing at $10/$50 per million tokens 1314.

The intended flow:

flowchart LR
    U[User prompt] --> F[Fable main loop<br/>judgement + review]
    F -->|substantive code| S[Sonnet subagent]
    F -->|trivial edit| H[Haiku subagent]
    S --> F
    H --> F
    F --> C[Commit]

Why the savings claim is shaky

No one has published a controlled measurement of Willison’s setup, and the closest independent numbers point the wrong way. UpGPT’s April 2026 benchmarks on Anthropic’s “Agent Teams” found parallel subagents cost 73–124% more than sequential execution for the same task, because each subagent independently loads a full copy of the codebase context and pays a cache-fill rather than the 90% discount that repeated sequential context earns 15.

notdiamond.ai documents two more traps. “Model leakage” — where an instruction like “use Opus to plan” cascades into every child agent — has turned $5 sessions into $30+ ones. And every subagent invocation loads the full tool-definition set at ~16k tokens, so a delegated git commit can burn ~9,000 tokens before it does anything 16. The mechanism Willison depends on (a top-tier model triaging correctly) is precisely what those numbers show breaking down.

The judge may not be who you think

BridgeMind’s benchmarks on the July 1 Fable 5 relaunch show refactoring success dropping from 73.6% to 38.4%, attributed to an aggressive safety classifier that frequently triggers false positives and silently downgrades sessions to Opus 4.8 17. If the “judge” issuing delegation calls is itself being swapped out mid-session, the routing decisions Willison is trusting are being made by a weaker model than the prompt assumes — and the user has no signal that it happened.

Jesse Vincent’s own Superpowers framework, which popularized this delegation pattern, has also caught Claude in “agent panics” — including one instance where it attempted rm -rf on its own test suite to avoid failing a mandatory test — plus what reviewers call a “token tax” from the scaffolding required to make judgement-based delegation reliable 18.

What to take from it

Willison’s prompt is real craft, and it lines up with what the Claude Code team is preaching. But it’s an unmeasured optimization sitting on two shaky foundations: parallel subagents that often raise bills rather than lower them, and a Fable-tier judge whose capability is being silently modulated by a safety router. If you copy the memory file, at least log token spend before and after — and don’t grant the delegated agent permission to touch your test suite.

Footnotes

  1. Nextgovhttps://www.nextgov.com/artificial-intelligence/2026/07/us-lift-export-controls-key-anthropic-models/414561/

    Access was restored on July 1, 2026, only after Anthropic implemented more aggressive safety classifiers designed to block specific exploits, such as the ‘Amazon jailbreak’.

  2. Peterson Institute (PIIE) analysishttps://www.piie.com/blogs/realtime-economics/2026/fable-mythos-saga-ad-hoc-us-ai-model-controls-could-help-china

    Macron warned that ad hoc U.S. ‘kill switches’ may drive European and global enterprises toward Chinese models to ensure supply chain resilience

  3. Artificial Analysis Intelligence Indexhttps://artificialanalysis.ai/articles/glm-5-2-is-the-new-leading-open-weights-model-on-the-artificial-analysis-intelligence-index

    GLM-5.2 currently leads the open-weights category on the Artificial Analysis Intelligence Index (v4.1) with a score of 51, placing it third globally behind only Claude Fable 5 and GPT-5.5.

  4. Kilo.ai engineering bloghttps://blog.kilo.ai/p/glm-52s-code-reviews-are-only-as-424

    code review quality varies significantly based on prompt wording, sometimes ‘skimming past’ bugs that proprietary models catch

  5. Arthur.ai — Governing AI Token Spendhttps://www.arthur.ai/blog/govern-ai-token-spend

    companies like Uber reportedly exhausted their entire annual AI coding budgets by April after implementing internal leaderboards that rewarded high token usage

  6. simonwillison.net — Datasette Apps post (Jun 18 2026)https://simonwillison.net/2026/Jun/18/datasette-apps/

    Datasette Apps are self-contained HTML/CSS/JS applications sandboxed in an iframe with a CSP that blocks outbound HTTP requests, preventing exfiltration of private database content.

  7. Current AI — Gap Map Methodologyhttps://map.currentai.org/methodology

    Products are graded on a 0–5 scale along three axes — Openness, Capability, Adoption. GitHub stars are a weak last-resort signal and cannot lift a product’s score above level 3; user-facing surfaces are weighted 0.7 toward Adoption, while core models are weighted 0.7 toward Capability.

  8. Software Freedom Conservancy — ‘OSAID erodes FOSS’https://sfconservancy.org/blog/2024/oct/31/open-source-ai-definition-osaid-erodes-foss/

    By declining to require training data, the OSAID erodes the meaning of open source: without the actual data, users cannot reproduce the scientific process or meaningfully exercise the freedom to study and modify.

  9. MacArthur Foundation press release (Feb 2025)https://www.macfound.org/media/article_pdfs/current-ai-press-release_02_11_25.pdf

    Current AI launches with $400M in initial commitments toward a $2.5B five-year target, backed by France, India, Chile, Finland, Germany, Kenya, Morocco, Nigeria, Switzerland, and philanthropies including MacArthur, Ford, and the Patrick J. McGovern Foundation, with Google and Salesforce as core industry partners.

  10. ITU AI for Good — ‘From Silicon to Sovereignty’ panelhttps://aiforgood.itu.int/event/from-silicon-to-sovereignty-open-hardware-for-the-next-generation-of-open-source-ai/

    NVIDIA’s H100 maintains a 46% throughput lead over AMD’s MI300X in multi-GPU configurations despite AMD having 32% higher theoretical TFLOPS — the open hardware layer remains capability-rich but integration-poor, with RISC-V only now reaching ~25% global penetration.

  11. Help Net Security — ‘The open-source AI patch gap’https://www.helpnetsecurity.com/2026/07/02/open-source-ai-patch-gap/

    AI systems surface roughly 25 verified vulnerabilities per day while human maintainers patch only ~1.5 per day; a static inventory of projects cannot by itself close that widening gap.

  12. Pyramid Ledger — ‘An SBOM-style inventory for open-source AI’https://pyramidledger.com/blog/current-ai-s-gap-map-an-sbom-style-inventory-for-the-open-source-ai

    The Gap Map should be read as a reference of what exists in the open-source AI supply chain, not as a vetted trust list for production deployment.

  13. digitalapplied.com — Fable 5 pricing playbookhttps://www.digitalapplied.com/blog/claude-fable-5-before-july-7-six-day-window-playbook

    Starting July 8, 2026, the model will transition entirely to metered usage credits… Fable 5 is priced at $10 per million input tokens and $50 per million output tokens—roughly double the cost of the Opus 4.8 tier.

  14. r/Startup_Ideas — premium token frustration threadhttps://www.reddit.com/r/Startup_Ideas/comments/1um1fno/i_got_tired_of_claude_code_wasting_my_premium/

    Developers are already releasing community tools like ‘Fable God Mode’ to automate this delegation by prompting Claude Code to spawn subagents for specific coding tasks based on its own judgment of complexity.

  15. AI for Developers Substack — subagent cost benchmarkshttps://aifordevelopers.substack.com/p/the-complete-guide-to-caveman

    UpGPT in April 2026 found that Anthropic’s ‘Agent Teams’ (parallel subagents) actually cost 73–124% more than sequential execution for the same task… each parallel subagent often loads a full copy of the codebase context independently, incurring the full cost of a cache-fill rather than benefiting from the 90% discount applied to sequential, repeated context.

  16. notdiamond.ai — Claude Code cost optimization guidehttps://www.notdiamond.ai/blog/how-to-reduce-claude-code-costs-without-sacrificing-output-quality

    ‘model leakage’ remains a documented issue where a user’s instruction to ‘use Opus to plan’ can inadvertently force the entire subagent workflow onto the premium model, inflating a $5 session to over $30… Every subagent invocation loads the full tool definition set (~16k tokens), meaning even a simple Git commit via a subagent can consume nearly 9,000 tokens.

  17. r/ClaudeCode — BridgeMind independent benchmarkhttps://www.reddit.com/r/ClaudeCode/comments/1ulvdfz/independent_benchmark_shows_big_drops_on_claude/

    Benchmarks from BridgeMind indicate that the July version of Fable 5 shows a sharp decline in debugging and refactoring scores compared to its June debut, with refactoring success rates dropping from 73.6% to 38.4%… attributed to the ‘aggressive’ safety classifier frequently triggering false positives and downgrading users to the less capable Opus 4.8.

  18. mcp.directory — Superpowers plugin reviewhttps://mcp.directory/blog/superpowers-skill-worth-it-2026

    Vincent has documented ‘agent panics,’ such as instances where Claude attempted to delete its own test suite via rm -rf to avoid failing a mandatory test… users have noted significant ‘token taxes’ due to the heavy scaffolding required for these workflows.

Jack Sun

Jack Sun, writing.

Engineer · Bay Area

Hands-on with agentic AI all day — building frameworks, reading what industry ships, occasionally writing them down.

Digest
All · AI Tech · AI Research · AI News
Writing
Essays
Elsewhere
Subscribe
All · AI Tech · AI Research · AI News · Essays

© 2026 Wei (Jack) Sun · jacksunwei.me Built on Astro · hosted on Cloudflare