xAI open-sources Grok, Claude fetch leaks memory, Inkling leads US open weights

TL;DR

• xAI open-sourced Grok Build’s 844K Rust lines after a 27,800× data-exfiltration scandal.
• Thinking Machines’ Inkling debuts as the #1 US open-weights model at 41 on Intelligence Index.
• Inkling hallucinates 63% on AA-Omniscience, worse than GLM-5.2 or Kimi K2.6.
• Claude’s web_fetch leaked a user’s name, city, and employer via a nested-link honeypot.
• 98% of production agents ship with all three lethal-trifecta prongs, per CSA.

Today’s tech section leans hard on agent-trust failures. xAI dumps its 844K-line Rust Grok Build codebase on GitHub as damage control after researchers caught the CLI shipping 5.1 GiB of user data for a task that needed 192 KB — with the user-facing privacy toggle never actually governing the upload endpoint. Anthropic, meanwhile, patched a Claude web_fetch bug that let an attacker’s alphabet honeypot pull a user’s name, home city, and employer through nested links, then declined the bounty on a claim of prior internal discovery.

The third lead sits in a different lane: Thinking Machines’ Inkling launches as the top US open-weights model on Artificial Analysis’s Intelligence Index, edging Nemotron 3 Ultra — but the same eval clocks it hallucinating 63% of the time on AA-Omniscience, and its 12B Inkling-Small variant beats the 41B flagship on instruction-following. The round-ups add OpenAI’s GPT-Red sparring partner, Allen AI’s Shippy post-mortem, IBM on router failure modes, and a new voice-AI benchmark.

xAI open-sources Grok Build after 27,800× data-exfil scandal

Source: simon-willison · published 2026-07-15

TL;DR

• xAI open-sourced its 844K-line Rust Grok Build codebase under Apache-2.0 as damage control.
• The CLI shipped 5.1 GiB for a task needing 192 KB — a 27,800× amplification into a hidden GCS bucket.
• The “Improve the model” toggle never governed the upload endpoint, only training use of the data.
• No independent audit confirms Musk’s deletion pledge, and the repo lands as one flattened commit.

What actually left the machine

Willison’s post frames the release as “quite a surprising codebase.” The context that makes it surprising is uglier than the post lets on. Independent wire-level analysis of grok CLI v0.2.93 found the client streaming full Git bundles — 5.1 GiB against 192 KB of actual model traffic, roughly a 27,800× amplification — including files the agent had been explicitly instructed not to read ¹. The exfiltration ran on a separate endpoint from inference: model calls went to a normal responses API, while a /v1/storage path drained sessions into a GCS bucket named grok-code-session-traces ². The user-facing “Improve the model” checkbox only gated training use of that data, not the upload itself.

flowchart LR
    U["Developer's repo<br/>.env, SSH keys, history"] --> C["grok CLI v0.2.93"]
    C -->|"192 KB prompts"| I["responses API<br/>(inference)"]
    C -->|"5.1 GiB traces"| S["storage API → GCS bucket<br/>grok-code-session-traces"]
    T["'Improve the model' toggle"] -. governs .-> I
    T -. does NOT govern .-> S

That second arrow is the whole scandal. A developer who ran grok in ~ was, per one report Willison cites, uploading SSH keys and a password-manager database to a bucket they didn’t know existed.

The remediation and its gaps

xAI’s response arrived in three moves: a July 12 server-side kill of the upload path, Musk’s pledge that retained data would be “completely and utterly deleted,” and now the Apache-2.0 source drop. Andrew Milich — ex-Skiff, ex-Cursor — is fronting the fix and shipped a /privacy slash command that both disables retention and triggers a retroactive purge ³. The open codebase confirms the kill-switch: upload_session_state() in xai-grok-shell/src/upload/trace.rs now returns a hard-coded session_state_upload_unavailable error, and the GCS client in upload/gcs.rs is dead code.

What the release does not confirm is anything about the past. There is no independent audit of the deletion, no timeline, and the repo arrived as a single commit — the git history that would show when the exfil path was added, and by whom, has been flattened. Sam Altman’s public reaction was one word: “concerning” ⁴. Security analysts are telling affected developers to rotate credentials anyway.

Openwashing, or the client audit we needed?

Parts of the developer community read the license choice as tactical. The model weights and training data stay proprietary; only the harness is open. Critics at the-decoder called this “openwashing” — permissive licensing on the periphery to buy goodwill while the core stays closed ⁵. The defense is that the harness is exactly where the breach lived, and 844K lines of Rust (against Codex’s 950K) is a genuinely substantial artifact — not a fig leaf. Willison’s own follow-up, porting the xai-grok-markdown Mermaid-to-Unicode renderer into WebAssembly within a day, is a small demonstration of what an open harness actually enables.

The regulatory context sharpens the stakes. xAI is already the subject of an Irish DPC inquiry citing GDPR Articles 5, 6, 25, and 35 ⁶; a CLI that quietly slurps .env files is close to a textbook Article 25 (data-protection-by-design) failure. The open-source drop verifies the kill-switch exists in code today. It does not verify that yesterday’s data is gone.

Further reading

• Mermaid to Unicode box art (grok-mermaid) — simon-willison

---

Claude’s web_fetch leaked memory via alphabet honeypot

Source: simon-willison · published 2026-07-15

TL;DR

• Ayush Paul pulled a Claude user’s name, home city, and employer via an attacker honeypot.
• The trick: web_fetch refused attacker URLs but followed nested links inside fetched pages.
• Anthropic patched and declined a bounty, claiming prior internal discovery without receipts.
• 98% of production agents ship with all three lethal-trifecta prongs, per Cloud Security Alliance.

The exploit, exactly

Claude’s memory is not a single blob. A daily summarization pass distills past chats into a persistent user biography that gets injected into every new session, and a conversation_search tool queries the full transcript on demand ⁷. That’s the data source. The exfiltration channel was web_fetch, whose defense — praised only months earlier — was a strict allowlist: it would visit URLs the user typed or that web_search returned, and nothing else.

Paul found the “and nothing else” was wrong. web_fetch also followed links embedded in pages it had already loaded. So he built a honeypot that, when hit by a client with Claude-User in the user-agent, served a fake Cloudflare “AI authentication” page instructing the agent to walk an alphabet-shaped link tree to find its user’s profile ⁷:

flowchart LR
    U[User chat] --> C{Claude + memory}
    C -->|web_fetch| H[Honeypot: Cloudflare auth page]
    H -->|nested links /a /b /c...| C
    C -->|recalls name 'ayush'| L[/a → /ay → /ayu → /ayush/]
    L -.server logs.-> A((Attacker))

Each nested request encoded one more character of remembered data into the URL path. The attacker just read their access log.

Why this keeps happening

The Memory Heist is not a one-off. Oasis Security’s concurrent “Claudy Day” chain achieved invisible-prompt-injection exfiltration inside a default Claude session with no third-party connectors ⁸. Anthropic has shipped eight consecutive patches to the Claude for Chrome extension for a bug class where rogue scripts forge isTrusted click events to drive Gmail and Docs reads ⁹. Cisco’s MemoryTrap shows the memory surface itself is now an attack primitive: a poisoned MEMORY.md in a repo survives across Claude Code sessions and reboots ¹⁰.

The common shape: memory + web + tool-use is the useful-agent recipe, and the Cloud Security Alliance’s June 2026 audit found 98% of production agents ship with all three ¹¹. Removing follow-on link navigation from web_fetch closes one loophole in that surface. It does not change the surface.

The bounty fight is the real story

Anthropic patched the bug and declined to pay, saying they’d already identified the nested-link path internally. There’s no external artifact of that prior discovery, and the claim lands against a documented pattern: Johann Rehberger’s earlier Claude exfiltration reports were initially triaged as “model safety” issues rather than security bugs and only reclassified as valid vulnerabilities after public attention ¹².

The interesting fault line is governance, not cryptography.

Deterministic URL allowlists are the strongest lethal-trifecta defense any major lab has publicly committed to, and one nested-link oversight was enough to fully defeat it. That’s a design lesson. The louder lesson is what happens when the researchers who find these holes decide the disclosure channel isn’t worth the effort — and every “we found it internally” without receipts pushes that day closer.

---

Inkling debuts as the top US open-weights model

Source: huggingface-blog · published 2026-07-15

TL;DR

• Thinking Machines’ Inkling ships 975B total / 41B active MoE weights, claiming 77.6% SWE-bench Verified and 97.1% AIME 2026.
• Artificial Analysis ranks it #1 US open-weights at 41 on its Intelligence Index, above Nemotron 3 Ultra.
• Same eval flags a 63% hallucination rate on AA-Omniscience — worse than GLM-5.2 or Kimi K2.6.
• Inkling-Small (12B active) beats the 41B flagship on IFBench (83.4 vs 79.8) and ties on SWE-bench.
• Flagship’s edge is parametric knowledge: SimpleQA Verified drops from 43.9% to 20.9% on Small.

What actually shipped

Inkling is a decoder-only MoE with 256 experts (6 routed + 2 shared per token), a hybrid global/sliding-window attention stack at a 5:1 ratio, and relative attention in place of RoPE. It handles text, image, and audio inputs across a 1M-token context and was trained on 45T tokens. Post-training used ECHO, an RL algorithm that learns an implicit world model without a separate verifier, plus ~30M rollouts across tool-use and math environments ¹³. The release is Apache 2.0 with day-0 support in transformers, vLLM, and SGLang.

The headline benchmark numbers — 97.1% AIME 2026, 87.2% GPQA Diamond, 77.6% SWE-bench Verified, 74.1% MCP Atlas — put Inkling squarely in frontier territory. Artificial Analysis debuts it as the leading US open-weights model on their Intelligence Index at 41, above Nemotron 3 Ultra and Gemma 4 31B ¹⁴. Nathan Lambert called it “the new best American model”; Karpathy praised the companion Tinker platform for delivering ~90% of DIY algorithmic control at ~10% of the infra overhead ¹⁵.

The caveats the launch post skips

Two footnotes matter. First, the 77.6 on SWE-bench Verified is bumping a dataset ceiling: a 2026 project audit found that over 59% of remaining failures on the benchmark are broken test cases rather than model errors ¹⁶. Every frontier model is now clustered against that wall, and the gap between 77% and 80% mostly reflects test-suite noise. Second, that headline depends on cranking the reasoning_effort parameter to max — Kingy’s analysis notes Inkling matches larger proprietary systems on Terminal-Bench 2.1 using roughly one-third the tokens, but only in max-effort mode ¹³. Real inference cost is higher than the spec sheet suggests.

More pointedly: Artificial Analysis measured a 63% hallucination rate on AA-Omniscience, worse than Chinese peers like GLM-5.2 and Kimi K2.6 ¹⁴. Inkling prioritizes attempting an answer over abstaining — a deliberate posture, but one that undercuts its usefulness for knowledge-heavy retrieval tasks.

The Inkling-Small surprise

Third-party coverage surfaces a result the primary release under-plays. Inkling-Small (12B active / 276B total) scores 83.4% on IFBench versus 79.8% for the 41B flagship, and essentially ties it on SWE-bench Verified (77.4 vs 77.6). Where it collapses is factual recall: 20.9% on SimpleQA Verified versus 43.9% for the flagship ¹⁷. Translation: the flagship’s advantage over its smaller sibling is mostly parametric knowledge, not reasoning or instruction-following. If you’re building agents that ground themselves in tools or retrieval, Small may be the better default.

Can you actually run it?

The full BF16 checkpoint wants ~2TB of GPU memory — multiple H200 or B300 systems. NVFP4 gets you to 600GB. For local use, Unsloth’s Dynamic 2.0 1-bit GGUFs compress the model to ~270–290GB with roughly 18% accuracy loss ¹⁸. The catch: vLLM and SGLang GGUF paths are “highly experimental” and missing fused-expert MoE kernels, so production serving still means FP8/NVFP4 on data-center hardware while llama.cpp handles the workstation case ¹⁸. Early Tinker users at Ramp Labs also flagged that the platform can’t yet host custom reward functions, forcing latency-adding remote hosting ¹⁵.

Net: a legitimate frontier open-weights release, best understood as a customization substrate with known factuality weaknesses — not the unqualified leader the launch implies.

Round-ups

OpenAI’s GPT-Red sparring partner hardens GPT-5.6 against attacks

Source: mit-tech-review-ai

GPT-Red is an in-house LLM super-hacker OpenAI trains its flagship models against to stiffen cyber defenses. The company says last week’s GPT-5.6 release, sparred against GPT-Red during training, is its most attack-resistant model to date.

Allen AI shares agent-building lessons from shipping Shippy

Source: huggingface-blog

Shippy, Allen AI’s agent project, exposed the gap between demo-grade and production-grade agents. The team details what broke in evaluation, tool use, and error recovery, and the design choices that kept the agent reliable once real users started sending traffic.

IBM Research unpacks why LLM model routing gets hard fast

Source: huggingface-blog

Routing queries to the cheapest capable model looks trivial until traffic mixes, latency budgets, and quality thresholds collide. IBM Research walks through the failure modes production routers hit and the trade-offs between cost, accuracy, and reliability at scale.

Real World VoiceEQ benchmark scores the human quality of voice AI

Source: huggingface-blog

Real World VoiceEQ is a new benchmark that rates voice models on how human they sound in realistic conversations, not just word-error rate. The framework targets prosody, turn-taking, and emotional fit — dimensions standard speech metrics ignore.

Footnotes

1. mlq.ai — Cereblab wire-level teardown summary — https://mlq.ai/news/xais-grok-build-cli-caught-uploading-entire-codebases-to-google-cloud-without-consent/

   For a coding task needing only 192 KB of traffic, the tool uploaded 5.1 GiB of data—a 27,800x increase… the tool uploaded files it had been explicitly instructed not to read, and it ignored the ‘Improve the model’ privacy toggle.

   ↩

2. The Hacker News — technical audit of endpoints — https://thehackernews.com/2026/07/grok-build-uploads-entire-git.html

   Model inference traffic was sent to a standard responses endpoint, [but] the bulk of the data was directed to a Google Cloud Storage bucket named ‘grok-code-session-traces’ via a /v1/storage endpoint… the ‘Improve the model’ toggle only governed model training rather than the underlying telemetry.

   ↩

3. OpenSourceForU — Milich remediation — https://www.opensourceforu.com/2026/07/xai-open-sources-grok-build-after-repository-upload-controversy/

   Milich, a former lead at the privacy-focused startup Skiff and the coding tool Cursor… stated that executing /privacy within the terminal tool disables data retention moving forward and triggers a retroactive deletion of all previously synchronized user data.

   ↩

4. Ground News — Altman reaction & audit gap — https://ground.news/article/elon-musk-confirms-grok-build-uploads-user-code-spacexai-promises-deletion

   OpenAI CEO Sam Altman responded to the findings with a single word: ‘concerning’… there has been no independent audit to confirm the deletion or a clear timeline for the process.

   ↩

5. the-decoder.com — dissent on ‘openwashing’ — https://the-decoder.com/xai-open-sources-grok-build-on-github-after-massive-data-breach/

   Skeptics argue that without the training data and full model architecture, the release is a form of ‘openwashing’—using a permissive license for peripheral tools to gain community goodwill while keeping the core intelligence closed.

   ↩

6. JURIST — Irish DPC / GDPR context — https://www.jurist.org/news/2026/02/ireland-opens-investigation-into-xs-grok-images/

   xAI is currently the subject of a ‘large-scale inquiry’ by the Irish Data Protection Commission… examining whether xAI violated GDPR Articles 5, 6, 25, and 35.

   ↩

7. Ayush Paul, ‘The Memory Heist’ (original disclosure) — https://www.ayush.digital/blog/the-memory-heist

   Claude’s memory is a two-part system: a daily summarization pass that injects condensed user profiles into every new chat, and a conversation_search tool that queries the full historical transcript… the attacker’s server logs record the sequence of GET requests, such as /a → /ay → /ayu → /ayush, reconstructing private strings one character at a time.

   ↩ ↩²

8. Oasis Security, ‘Claudy Day’ disclosure — https://www.oasis.security/blog/claude-ai-prompt-injection-data-exfiltration-vulnerability

   A vulnerability chain dubbed ‘Claudy Day’ exploited invisible prompt injection to exfiltrate data without any specialized integrations, functioning entirely within a default Claude session.

   ↩

9. CybersecurityNews — Claude for Chrome vulnerability — https://cybersecuritynews.com/claude-for-chrome-vulnerability/

   Despite eight consecutive releases aimed at securing the extension, a critical vulnerability remains exploitable… the flaw stems from a failure to check the isTrusted property of click events, allowing programmatic interaction from rogue scripts to masquerade as user-initiated actions.

   ↩

10. HelpNetSecurity — Cisco ‘MemoryTrap’ disclosure — https://www.helpnetsecurity.com/2026/04/14/idan-habler-cisco-agentic-ai-memory-attacks/

    By embedding malicious instructions in a MEMORY.md file within a repository, attackers can permanently alter the agent’s behavior across all future sessions and reboots.

    ↩

11. Airia — ‘AI Security in 2026: The Lethal Trifecta’ — https://airia.com/blog/ai-security-in-2026-prompt-injection-the-lethal-trifecta-and-how-to-defend/

    A June 2026 assessment by the Cloud Security Alliance found that 98% of production AI agents satisfy all three conditions of the Lethal Trifecta, making them structurally exploitable by design.

    ↩

12. Executive Offense newsletter (beehiiv) — https://executiveoffense.beehiiv.com/p/ai-under-fire

    For other reports, such as those by Rehberger, the company initially dismissed the findings as ‘model safety’ issues rather than technical security vulnerabilities before eventually acknowledging them as valid security flaws following public attention.

    ↩

13. Kingy.ai analysis — https://kingy.ai/blog/inkling-ai-model-benchmarks-specs-open-weights/

    At maximum reasoning effort Inkling matches significantly larger proprietary systems on Terminal-Bench 2.1 while using roughly one-third of the generated tokens; post-training used 30M rollouts across tool-use and math environments.

    ↩ ↩²

14. Artificial Analysis — https://artificialanalysis.ai/articles/thinking-machines-has-released-inkling-the-new-leading-u-s-open-weights-model

    Inkling debuts as the leading U.S. open-weights model on the Intelligence Index at 41, ahead of Nemotron 3 Ultra and Gemma 4 31B, but records a 63% hallucination rate on AA-Omniscience — the model prioritizes attempting an answer over abstaining.

    ↩ ↩²

15. Silicon Republic (Lambert/Karpathy commentary) — https://www.siliconrepublic.com/business/mira-muratis-ai-start-up-unveils-customisable-open-source-model-inkling

    Nathan Lambert called Inkling ‘the new best American model’ and Karpathy praised Tinker for providing ~90% of DIY algorithmic control while removing 90% of the infra overhead — though Ramp Labs noted the platform initially cannot host custom reward functions, forcing latency-adding remote hosting.

    ↩ ↩²

16. SWE-bench project audit — https://www.swebench.com/

    Over 59% of failures on SWE-bench Verified stem from flawed test cases rather than model limitations, suggesting scores near 80% may represent a performance ceiling for the dataset.

    ↩

17. MarkTechPost — https://www.marktechpost.com/2026/07/15/thinking-machines-lab-releases-inkling-a-975b-parameter-open-weights-multimodal-moe-with-41b-active-parameters-and-controllable-thinking-effort/

    Inkling-Small (12B active / 276B total) scores 83.4% on IFBench versus 79.8% for the 41B flagship and matches it on SWE-bench Verified (77.4 vs 77.6), but collapses to 20.9% on SimpleQA Verified versus 43.9% — a sharp knowledge/instruction tradeoff.

    ↩

18. gHacks — https://www.ghacks.net/2026/07/16/thinking-machines-lab-releases-inkling-a-975-billion-parameter-open-weights-ai-model-under-apache-2-0/

    Running the full BF16 checkpoint requires ~2TB of GPU memory (multiple H200/B300 systems); 1-bit Unsloth Dynamic 2.0 GGUF quants fit in ~270–290GB with roughly 18% accuracy loss, and vLLM/SGLang GGUF support remains ‘highly experimental’ with missing fused-expert MoE kernels.

    ↩ ↩²