GLM-5.3 tops CyberGym, Chinese labs own the Hub, HF agents contest ICML

TL;DR

• GLM-5.3 hits 84.5% on CyberGym, edging Mythos 5 and GPT-5.6 Sol on vulnerability discovery.
• Z.ai held open weights 2 weeks for safety hardening, the first release gated on offensive-cyber risk.
• Chinese labs shipped 178 models above 20B params in Jan–Aug 2026, 59% Apache 2.0.
• Qwen logs 151,448 derivatives on the Hub, 2.6× Meta’s Llama footprint.
• 1,221 volunteers ran coding agents against 2,226 ICML 2026 papers, contesting 23%.

Two of today’s three features center on the Chinese open-model frontier. Z.ai’s GLM-5.3 takes the top slot on CyberGym for offensive-cyber capability — with no new pretraining run, just scaled RL and long-horizon post-training on the same 743B MoE base. Hugging Face’s own frontier report puts numbers on the trend Z.ai is riding: 178 Chinese models above 20B this year, Qwen with 2.6× Llama’s derivative footprint, and — pointedly — GLM-5.2 doing forensic work on a Hub intrusion that GPT-5.5 and Claude 4 refused as too dangerous.

The third feature moves the frame from models to evaluation. A 1,221-volunteer effort ran coding agents against every ICML 2026 paper, contesting 23% of them, including a 9σ refutation of a spotlighted paging-robustness result. Only 12% of attempts reproduced end-to-end, and the SOTA reruns that did burned 4,096 H100-hours apiece. Peer review at agent scale is now a thing that has been measured, and the number is uncomfortable.

GLM-5.3 tops CyberGym at 84.5% with post-training alone

Source: interconnects · published 2026-08-14

TL;DR

• GLM-5.3 hits 84.5% on CyberGym, edging Anthropic’s Mythos 5 (83.8%) and GPT-5.6 Sol (83.6%) for vulnerability discovery.
• The 743B MoE base is unchanged from GLM-5.2 — no new pretraining run, no fresh corpus.
• Every gain comes from post-training: scaled RL in the slime framework plus a long-horizon stack called SAO.
• Z.ai held the open weights for 2 weeks of safety hardening, the first frontier release explicitly gated on offensive-cyber risk.
• The “just distillation” reflex misses the mechanism: async RL on separate GPU clusters is doing the work.

The claim that matters

Nathan Lambert’s read on GLM-5.3 is unusually blunt for a frontier-release post: it is not a distillation story. That is a direct shot at the policy narrative crystallizing around Chinese labs — IISS documents that Anthropic caught ~24,000 fraudulent accounts tied to Moonshot, DeepSeek and MiniMax pulling over 16 million Claude exchanges, evidence concrete enough to underpin NSTM-4’s classification of industrial distillation as a national-security threat ¹. The presumption of contamination is not paranoid. It is documented.

But GLM-5.3 is a bad test case for that presumption, and that is exactly why Lambert picks it.

What actually changed

Z.ai’s own release notes make the claim falsifiable. The 743B-parameter MoE base is literally the GLM-5.2 checkpoint; no new pretraining run, no fresh corpus ². The delta is entirely in post-training: scaled RL through the open-source slime framework, plus a long-horizon agent stack called SAO.

slime is the load-bearing piece of engineering. The repo disaggregates the training loop into three components — a Megatron-LM trainer, an SGLang rollout engine, and a central Data Buffer — so trajectory generation and weight updates run fully asynchronously on separate GPU clusters ³.

flowchart LR
    A[SGLang rollout cluster] -->|trajectories| B[(Data Buffer)]
    B -->|batches| C[Megatron-LM trainer]
    C -.->|async weight sync.| A

That is a concrete mechanism for how a post-training-only update moves benchmarks 20+ points. You do not need a GPT-5.6 teacher to explain it. You need a lot of RL compute and a pipeline that doesn’t idle the trainer waiting for rollouts.

The cyber angle is the actual news

The benchmark table is where GLM-5.3 stops being an incremental release. 84.5% on CyberGym puts it ahead of Mythos 5 and GPT-5.6 Sol, and Z.ai says the model turned up a serious pre-release vulnerability in Cursor during evaluation ⁴. Context deflates the miracle slightly: Cursor has been patching zero-click prompt-injection RCEs (DuneSlide, CVE-2026-50548/50549) and a CLI workspace-trust bypass across the last month, so the target surface was already known-soft ⁵. Frontier models hunting there will find things.

What is genuinely new is the release choreography. This is the first open-weights frontier launch where Z.ai held the weights for two weeks of explicit safety hardening before publishing ². Labs do not embargo weights over code-completion scores. They embargo them when they think the offensive capability is real.

The takeaway

The debate splits cleanly. Policy commentators treat any Chinese frontier release as presumptively distilled ¹; practitioners looking at slime, SAO, and an unchanged base checkpoint see a legitimate post-training scaling story ²³. Lambert’s sharper companion piece, The Distillation Panic, concedes distillation happens but argues the policy framing has overshot the engineering reality ⁶.

GLM-5.3 is the cleanest evidence yet that he is right about the mechanism — and the cyber numbers are evidence the stakes of being wrong about it just got higher.

---

Hugging Face: Chinese labs now own the open-model frontier

Source: huggingface-blog · published 2026-08-14

TL;DR

• Chinese labs shipped 178 models above 20B in Jan–Aug 2026, with 59% Apache 2.0 and zero non-commercial.
• Qwen has 151,448 derivatives on the Hub, 2.6× Meta’s Llama footprint, and 5× its monthly GGUF downloads.
• Claude Code drives 44.4% of agent traffic to the Hub, which now serves an agents.md endpoint alongside human pages.
• GLM-5.2 forensically analyzed a Hub intrusion after GPT-5.5 and Claude 4 refused the diagnostic queries as malicious.
• SaferAI flags the same GLM-5.2 helpfulness as a dual-use gap: it refused none of its offensive-cyber benchmark tasks.

The frontier moved east

Hugging Face’s summer report makes the geographic shift unambiguous: DeepSeek, Moonshot, Xiaomi, Z.ai, and Meituan are the labs setting the monthly parameter ceiling, which sat between 754B and 2.78T through August. Xiaomi and Meituan both cleared a trillion this year. US labs, with the exception of NVIDIA’s Nemotron 3 Ultra (561B) and Thinking Machines’ Inkling, mostly stayed under 130B. (HF lists Inkling at 952B; the shipped architecture finalized at 975B total / 41B active, and even the NVFP4 quantization needs ~590GB of VRAM ⁷.) The trillion-parameter local-inference story is real but gated to prosumers with server-class rigs, which matches the report’s own admission that models over 100B account for just 1% of downloads.

The more surprising finding is licensing. Of those 178 Chinese 20B+ releases, 59% ship Apache 2.0, 22% MIT, and — per Hugging Face — none carry a non-commercial restriction. That inverts the caricature that Chinese labs are the closed ones. But the claim is already fraying: Moonshot’s Kimi K3 introduced a custom license with commercial revenue thresholds ⁸, echoing Meta’s 700M-MAU cap on Llama. And on the US side, Google’s Gemma 4 finally moved to full Apache 2.0, dropping the remote-revocation clause and viral synthetic-data terms that had blocked downstream adoption ⁹. The inversion holds directionally; the “exactly none” is a snapshot, not a trend line.

Qwen is the new base model

Alibaba’s Qwen has quietly become the substrate everyone else fine-tunes on. 151,448 derivatives on the Hub, 2.6× Meta’s total. Monthly GGUF downloads run 39.6M for Qwen vs. 7.5M for Llama. Repositories using the gguf library grew 464% year-over-year; Apple’s mlx grew 148%. The runtime layer is expanding faster than the modeling core, which tracks with the report’s other quiet finding: models under 1B still command 83% of all-time downloads. The frontier gets headlines; the practical layer stays small.

Hacker News commenters pushed back on how Hugging Face frames this dominance ¹⁰. Qwen, DeepSeek, and GLM ship weights without training data or training code — “open weight,” not open source in the OLMo 3 sense. HF’s derivative-and-download metrics implicitly reward weight releases over reproducibility, and the report doesn’t name the distinction.

The GLM-5.2 forensic incident cuts both ways

The emotional beat of the report is a July intrusion attempt against Hugging Face itself. When staff tried to feed 17,000+ logged attacker actions to GPT-5.5 and Claude 4, both classified the diagnostic queries as malicious and refused ¹¹. GLM-5.2 completed the analysis. Jeff Boudier’s companion post treats this as vindication for open models in cyber defense.

SaferAI’s August audit reads the same property differently:

GLM-5.2 refused none of the offensive cyber or dual-use biological tasks it was assigned, whereas Claude 4 refused so frequently that researchers could not even complete the benchmark. ¹²

The trait that made GLM-5.2 the only tool for the forensic job is the same trait SaferAI flags as a safety gap — and unlike an API model, weights on the Hub cannot be recalled. Hugging Face’s report tells the first half of that story cleanly. The second half is the one worth watching.

---

Hugging Face agents contest 23% of 2,226 ICML 2026 papers

Source: huggingface-blog · published 2026-08-13

TL;DR

• 1,221 volunteers ran coding agents against 2,226 ICML 2026 papers, logging 35,908 claim-level verdicts in 18 days.
• 23% of papers were contested, including a 9σ refutation of a spotlighted paging-robustness bound.
• Only 266 papers were fully reproduced end-to-end — roughly 12% of attempts.
• SOTA reruns averaged 4,096 H100-hours per paper, far beyond the hackathon’s $20-per-person credit budget.

What 1,221 people and a lot of Claude Code actually did

Between July 15 and August 2, Hugging Face and alphaXiv pointed a community of 1,221 participants — armed with Claude Code, Codex, Cursor, orx, and $20 each in HF Jobs credits — at 2,226 of ICML 2026’s accepted papers. Every attempt landed in a “Trackio logbook” (code, artifacts, traces), and a GLM-5.2-powered “Logbook Judge” issued 35,908 verified/falsified/toy-scale/inconclusive verdicts on individual claims.

The headline split: 51% of papers (1,103) had at least one claim independently verified, 23% (496) had at least one claim falsified or contested, and 242 papers saw independent teams reach opposite verdicts on the same claim — a category the organizers call “adversarial reproducibility.” Only 266 papers came out fully clean.

The falsifications that will actually move arXiv

Four case studies are doing most of the rhetorical work, and they’re substantive:

The Frank-Wolfe finding is the most damning process critique: the failure only shows up past 224 steps, so every short-horizon reviewer missed it. Authors of the Paging and Frank-Wolfe papers have already acknowledged the findings and committed to arXiv corrections ¹³ — faster than a standard rebuttal cycle.

Read the numbers with the caveats attached

The 51/23 split flatters the exercise. Independent coverage points out that replicating the top 5% of SOTA papers averaged ~4,096 H100-equivalent hours, and papers built on closed corporate or medical datasets reproduced at under 12% ¹⁴. A $20 credit budget doesn’t touch that tier, which means the falsification rate is systematically biased toward the tractable long tail, not the industrial frontier.

The scoring incentives compound this. The rubric awarded 2 points for a full falsification — the same as a full reproduction — creating what one write-up called a “falsification funnel” that pays to debunk ¹⁵. And the judge itself is contested: GLM-5.2 is suspected of partial distillation from frontier models, with reliability described as “harness-dependent” rather than intrinsic ¹⁶. Broader agent benchmarks agree the ceiling is lower than the demo suggests — on PaperBench-style tasks, agents fail 32–49% more often than humans, and one study found 59% of agent-produced reviews contained fabricated claims ¹⁷.

The takeaway

The interesting number isn’t 23%. It’s that a distributed swarm plus $20 of compute per person surfaced named, defensible errors in spotlighted papers within three weeks — while a parallel ICML scandal saw 497 papers desk-rejected after hidden watermarks caught 506 reviewers running LLMs against policy ¹⁸. Both sides of peer review are now being policed by tooling authors didn’t consent to. The hackathon’s own conclusion — humans as “Principal Investigators” steering agents claim-by-claim — is the honest framing. The agents aren’t auditing science yet; they’re generating leads fast enough that humans finally can.

Footnotes

1. IISS — AI Distillation Attacks in the US-China Contest — https://www.iiss.org/online-analysis/cyber-power-matrix/2026/05/ai-distillation-attacks-in-the-uschina-contest/

   Anthropic reported campaigns tied to Moonshot, DeepSeek and MiniMax using ~24,000 fraudulent accounts to generate over 16 million exchanges with Claude, prompting NSTM-4 to categorize industrial distillation as a national-security threat.

   ↩ ↩²

2. Z.ai official GLM-5.3 blog — https://z.ai/blog/glm-5.3

   The 743B-parameter base model is unchanged from GLM-5.2; gains come from scaled post-training with the ‘slime’ async RL framework and the ‘SAO’ long-horizon stack, plus a two-week delay on open-weight release for safety hardening.

   ↩ ↩² ↩³

3. THUDM/slime GitHub — https://github.com/THUDM/slime

   Slime disaggregates training (Megatron-LM), rollout (SGLang), and a central Data Buffer, supporting fully asynchronous RL where trajectory generation and weight updates run in parallel on separate GPU clusters.

   ↩ ↩²

4. VentureBeat — https://venturebeat.com/technology/glm-5-3-is-here-with-advanced-cyber-capabilities-and-reportedly-already-found-a-serious-vulnerability-in-cursor

   GLM-5.3 reached an 84.5% success rate on CyberGym for vulnerability discovery, edging out Anthropic’s Mythos 5 (83.8%) and GPT-5.6 Sol (83.6%), and reportedly identified a serious vulnerability in Cursor before release.

   ↩

5. NeuraCybIntel — Cursor RCE flaws — https://www.neuracybintel.com/articles/critical-cursor-ai-code-editor-flaws-exposed-developers-to-os-level-remote-code-execution

   Cursor patched DuneSlide (CVE-2026-50548/50549) zero-click prompt-injection RCEs in the 3.0 release, and a separate CLI ‘trust this workspace’ bypass was fixed within three days of Manifold Security’s July 20 report.

   ↩

6. Interconnects — ‘The Distillation Panic’ (Lambert) — https://www.interconnects.ai/p/the-distillation-panic

   Framing every Chinese frontier release as a ‘distillation attack’ ignores the systems-engineering work required to integrate those signals; the panic is becoming a policy narrative untethered from what the training pipelines actually look like.

   ↩

7. Latent.Space — ‘Thinky’s Inkling 975B-A41B’ — https://www.latent.space/p/ainews-thinkys-inkling-975b-a41b

   The official release finalized the architecture at 975 billion total parameters … activating only 41 billion per token … even the quantized NVFP4 version requires approximately 590GB of VRAM.

   ↩

8. Moonshot AI — Kimi K3 launch page — https://www.kimi.com/blog/kimi-k3

   Moonshot AI introduced a custom ‘Kimi K3 License’ that imposes commercial revenue thresholds, marking a departure from purely open standards for its largest models.

   ↩

9. gavinj.net — ‘Why Gemma 4’s Apache 2.0 license matters’ — https://www.gavinj.net/post/gemma-4-apache-2-license-matters

   Gemma 4 abandoned the restrictive Gemma Terms of Use that previously allowed Google to remotely revoke access and placed ‘viral’ claims on all derivative synthetic data.

   ↩

10. Hacker News discussion of the HF state-of-open-models report — https://news.ycombinator.com/item?id=49213552

    Many developers expressed frustration that models like Qwen are essentially ‘proprietary binaries’ without training data or code … projects like OLMo 3 [are] the only true examples of open-source AI, whereas the Chinese leads were categorized as ‘open-weight’.

    ↩

11. Jeff Boudier, Hugging Face blog — ‘Open models for cyber defense’ — https://huggingface.co/blog/jeffboudier/open-model-cyber-defense

    Because the forensic data contained real exploit payloads and C2 artifacts, closed models’ guardrails classified investigators’ queries as malicious activity, refusing to process the 17,000+ logged actions.

    ↩

12. SaferAI report on GLM-5.2 (mdaakibansari.com.np) — https://www.mdaakibansari.com.np/articles/saferai-glm-5-2-safety-gap-report

    GLM-5.2 ‘refused none’ of the offensive cyber or dual-use biological tasks it was assigned, whereas Claude 4 refused so frequently that researchers could not even complete the benchmark.

    ↩

13. Medium — ‘Trust but Run the Agent’ by Avi Thaljunk — https://medium.com/@avithaljunk/trust-but-run-the-agent-inside-the-icml-2026-agent-reproductions-challenge-3ca944b5bfe4

    Multiple authors have already confirmed the agents’ findings, leading to several ‘in-flight’ arXiv corrections and cases where authors admitted to errors they had previously overlooked.

    ↩

14. getaibook.com — ICML 2026 reproducibility summary — https://getaibook.com/news/36-of-icml-2026-papers-fail-hugging-face-reproducibility-aud/

    Replicating the top 5% of SOTA papers required an average of 4,096 H100-equivalent GPU hours… papers utilizing closed corporate or medical datasets saw a success rate of less than 12%.

    ↩

15. readsikit.com — coverage of the challenge — https://readsikit.com/2026/08/13/hugging-face-icml-2026-agent-reproductions/

    The hackathon incentivized a ‘falsification funnel,’ rewarding participants with leaderboard points for proving a claim failed — assigning 2 points for a ‘full falsification,’ the same value as a ‘full reproduction.’

    ↩

16. clawvard.school — GLM-5.2 technical writeup — https://clawvard.school/blog/glm-5-2-open-weights-llm-for-agents

    Analysts at Graphistry noted that GLM-5.2’s outputs were statistically similar enough to proprietary frontier models to suggest the possibility of a successful ‘distillation attack’… its reliability is heavily dependent on the ‘harness’ rather than the weights alone.

    ↩

17. beam.ai — ‘Hybrid Sweet Spot’ analysis — https://beam.ai/agentic-insights/the-hybrid-sweet-spot-why-human-ai-teams-outperform-full-automation

    Benchmarks like PaperBench and MLReplicate reveal that agents fail 32% to 49% more often than humans… in one study, 59% of agent-produced reviews contained unsupported or entirely fabricated claims.

    ↩

18. CASRAI — ICML 2026 watermark/desk-rejection report — https://casrai.org/news/icml-2026-watermark-detection-ai-reviewers-desk-rejections

    ICML organizers desk-rejected 497 papers… after hidden watermarks in paper PDFs forced LLMs used by 506 reviewers to output specific phrases in their reviews.

    ↩