JS Wei (Jack) Sun

Open weights win the technical debate, lose the governance one

Hugging Face's evidence that dangerous cyber capability lives in scaffolding holds up, but its own RCE bug undercuts the trust-us framing.

Open weights win the technical debate, lose the governance one

TL;DR

  • Hugging Face argues frontier cyber capability lives in scaffolding, not weights — and independent reproductions on Anthropic’s Mythos set back the claim.
  • A 3.6B open model surfaced the flagship FreeBSD exploit; GPT-5.4 and Opus 4.6 cleared most of the benchmark cleanly.
  • Critics including Schneier say a for-profit shouldn’t unilaterally govern OS-rewriting capability, regardless of where the danger lives.
  • CVE-2026-25874 hit HF’s own LeRobot the same week: unsafe pickle, silenced linters, remote code execution.

Hugging Face spent the week making a careful technical argument: dangerous cyber capability is a property of scaffolding and tooling, not of the underlying weights, so locking down weights is the wrong lever. The evidence is stronger than the slogans suggest — a 3.6B open model surfaced the headline FreeBSD exploit on Anthropic’s Mythos benchmark, and frontier closed models cleared most of the same set without trouble.

The trouble is everything around the argument. Bruce Schneier and others point out that even a correct technical claim doesn’t grant a private company the authority to decide, on its own, how OS-rewriting capability gets governed. And the messenger picked an awkward week: CVE-2026-25874 landed in Hugging Face’s own LeRobot codebase — an unsafe pickle path with silenced linters, the kind of supply-chain footgun the openness debate keeps circling. Today’s story is what happens when the engineering case is sound and the political case still isn’t.

Hugging Face’s openness pitch is right on the tech, soft on the politics

Source: huggingface-blog · published 2026-04-21

TL;DR

  • HF argues Mythos-class cyber capability lives in scaffolding, not weights — independent reproductions back this up.
  • A 3.6B open model surfaced the flagship FreeBSD exploit; GPT-5.4 and Opus 4.6 cleanly cleared most of Anthropic’s set.
  • Schneier and others say HF doesn’t go far enough: closed governance of OS-rewriting capability is “not a choice a for-profit corporation should be allowed to make.”
  • Awkward timing: CVE-2026-25874 hit HF’s own LeRobot the same week — unsafe pickle, silenced linters, RCE.

The thesis that actually survived contact with reproducers

Hugging Face’s central technical claim — that Mythos’s vulnerability-finding prowess is a property of the system (compute, code-heavy data, probing scaffolds, autonomy budget), not a secret model — is the part of the essay that has held up. Vidoc Security Lab ran Anthropic’s five headline bugs through stock GPT-5.4 and Claude Opus 4.6 inside an open opencode harness and reproduced the FreeBSD and Botan findings on every attempt; Opus 4.6 even caught the 27-year-old OpenBSD bug 1. AISLE went smaller and cheaper, showing a 3.6B open-weights model could surface the flagship FreeBSD exploit for a fraction of frontier-model cost 2.

SystemSetupResult on Anthropic’s vuln set
Mythos (closed)Anthropic scaffold, gated preview83.1% on CyberGym; full set found
Claude Opus 4.6Public model + open opencode harnessFreeBSD, Botan, OpenBSD reproduced 1
GPT-5.4Public model + open opencode harnessFreeBSD, Botan reproduced 1
3.6B open-weightsAISLE harnessFreeBSD exploit at trivial cost 2

The “jagged frontier” framing in HF’s post is doing real work here: Mythos’s lead is a months-long moat from better plumbing, not a structural capability gap.

The dissent HF gestures at, said louder elsewhere

Where HF stays essayistic, Bruce Schneier is blunt: a capability that can rewrite the security posture of every operating system “should not be governed solely by the internal judgment of its creators… not a choice a for-profit corporation should be allowed to make in a democratic society” 3. Mashable’s panel went further, calling the Glasswing gated rollout a “PR play by Anthropic” once smaller models replicated the headline bugs 4. The question on the table isn’t whether openness is nicer — it’s whether the closed alternative is even defensible when reproduction is this cheap.

It also isn’t very closed. The Mythos preview itself was breached by a Discord group that simply guessed the preview URL inside a third-party vendor environment 5. Gated access, in practice, is a speed bump.

The pickle in Hugging Face’s own kitchen

The same week the openness piece dropped, CVE-2026-25874 (CVSS 9.3–9.8) landed against HF’s own LeRobot: unauthenticated RCE via pickle.loads() over an add_insecure_port() gRPC channel, in code where the dangerous calls had been silenced with # nosec comments — shipped by the organization that built Safetensors specifically to retire pickle 6. “Open” did not mean “audited.” If openness is a structural defense, supply-chain hygiene is the part of the structure HF still owes its readers a serious answer on.

What’s unresolved

Both camps come out worse than their press releases. Anthropic’s quarantine is leaky and arguably theatrical 45; HF’s openness essay is undercut by a pickle deserialization bug it shipped itself 6. The technical diagnosis is solid — discovery is now cheap, reproducible, and decentralizable 12 — but nobody in this debate has named who patches the resulting flood. Cheap discovery without a maintainer plan isn’t defense. It’s just a faster funnel into the same backlog.

Footnotes

  1. Vidoc Security Lab bloghttps://blog.vidocsecurity.com/blog/we-reproduced-anthropics-mythos-findings-with-public-models

    Both GPT-5.4 and Claude Opus 4.6 cleanly reproduced the FreeBSD and Botan vulnerabilities in every attempt… Claude Opus 4.6 also successfully identified the subtle 27-year-old OpenBSD bug

    2 3 4
  2. Sloppish (AISLE reproduction notes)https://sloppish.com/preview-the-reproduction.html

    even a 3.6-billion-parameter model could detect the flagship FreeBSD exploit for a fraction of the cost of frontier models

    2 3
  3. Bruce Schneier bloghttps://www.schneier.com/blog/archives/2026/04/on-anthropics-mythos-preview-and-project-glasswing.html

    should not be governed solely by the internal judgment of its creators… that is not a choice a for-profit corporation should be allowed to make in a democratic society

  4. Mashablehttps://mashable.com/article/claude-mythos-preview-project-glasswing-pr-stunt-cybersecurity-experts

    dismissed the initial announcement as a ‘PR play by Anthropic’

    2
  5. Let’s Data Sciencehttps://letsdatascience.com/blog/anthropic-mythos-discord-leak-guessed-url

    a small Discord-based group gained unauthorized access to the model by simply guessing the preview URL within a third-party vendor environment

    2
  6. The Hacker News (CVE-2026-25874)https://thehackernews.com/2026/04/critical-cve-2026-25874-leaves-hugging.html

    LeRobot utilized unsafe pickle deserialization despite Hugging Face’s long-standing promotion of the more secure Safetensors format… the vulnerable code contained # nosec comments, which explicitly silenced security linters

    2
Jack Sun

Jack Sun, writing.

Engineer · Bay Area

Hands-on with agentic AI all day — building frameworks, reading what industry ships, occasionally writing them down.

Digest
All · AI Tech · AI Research · AI News
Writing
Essays
Elsewhere
Subscribe
All · AI Tech · AI Research · AI News · Essays

© 2026 Wei (Jack) Sun · jacksunwei.me Built on Astro · hosted on Cloudflare