xAI open-sources Grok, Claude fetch leaks memory, Inkling leads US open weights

Sources

xai-org/grok-build, now open source simonwillison.net

xai-org/grok-build, now open source xAI’s grok CLI tool faced severe community backlash yesterday when it became apparent that running the command in a directory could upload that entire directory to xAI’s Google Cloud buckets. One user reported running it in their home directory and seeing it upload “my SSH keys, my password manager database, my documents, photos, videos, everything”. I’ve not seen an official explanation for why it was doing this, but xAI did respond to the feedback ( Musk :…

Mermaid to Unicode box art (grok-mermaid) simonwillison.net

Tool: Mermaid to Unicode box art (grok-mermaid) While exploring the codebase for the newly open-sourced Grok CLI coding agent I came across xai-grok-markdown/src/mermaid.rs , a “self-contained terminal renderer for Mermaid diagrams” written in Rust. I figured it would be fun to try that out in a browser via WebAssembly. Here’s the prompt I ran in Claude Code for web (Fable 5), and this is what the resulting tool looks like: Tags: tools , rust , webassembly , mermaid , grok , xai

Welcome Inkling by Thinking Machines huggingface.co

How I tricked Claude into leaking your deepest, darkest secrets simonwillison.net

How I tricked Claude into leaking your deepest, darkest secrets I’ve been impressed by the way the Claude web_fetch tool is designed to avoid data exfiltration attacks. Ayush Paul found a hole in that design. To recap: regular Claude chat is at risk of lethal trifecta attacks, because it has access to private data (in the form of memories of your past interactions) and has a tool for accessing online content which can both read hostile instructions and exfiltrate data through the URLs it access…

Meet GPT-Red: an LLM super-hacker OpenAI built to make its models safer technologyreview.com

GPT-Red is an in-house LLM super-hacker OpenAI trains its flagship models against to stiffen cyber defenses. The company says last week’s GPT-5.6 release, sparred against GPT-Red during training, is its most attack-resistant model to date.

Model Routing Is Simple. Until It Isn’t. huggingface.co

Routing queries to the cheapest capable model looks trivial until traffic mixes, latency budgets, and quality thresholds collide. IBM Research walks through the failure modes production routers hit and the trade-offs between cost, accuracy, and reliability at scale.

What building Shippy taught us about building agents huggingface.co

Shippy, Allen AI’s agent project, exposed the gap between demo-grade and production-grade agents. The team details what broke in evaluation, tool use, and error recovery, and the design choices that kept the agent reliable once real users started sending traffic.

Introducing Real World VoiceEQ: Measuring the human quality of voice AI huggingface.co

Real World VoiceEQ is a new benchmark that rates voice models on how human they sound in realistic conversations, not just word-error rate. The framework targets prosody, turn-taking, and emotional fit — dimensions standard speech metrics ignore.

References

mlq.ai — Cereblab wire-level teardown summary mlq.ai

For a coding task needing only 192 KB of traffic, the tool uploaded 5.1 GiB of data—a 27,800x increase… the tool uploaded files it had been explicitly instructed not to read, and it ignored the ‘Improve the model’ privacy toggle.

The Hacker News — technical audit of endpoints thehackernews.com

Model inference traffic was sent to a standard responses endpoint, [but] the bulk of the data was directed to a Google Cloud Storage bucket named ‘grok-code-session-traces’ via a /v1/storage endpoint… the ‘Improve the model’ toggle only governed model training rather than the underlying telemetry.

Ground News — Altman reaction & audit gap ground.news

OpenAI CEO Sam Altman responded to the findings with a single word: ‘concerning’… there has been no independent audit to confirm the deletion or a clear timeline for the process.

OpenSourceForU — Milich remediation opensourceforu.com

Milich, a former lead at the privacy-focused startup Skiff and the coding tool Cursor… stated that executing /privacy within the terminal tool disables data retention moving forward and triggers a retroactive deletion of all previously synchronized user data.

JURIST — Irish DPC / GDPR context jurist.org

xAI is currently the subject of a ‘large-scale inquiry’ by the Irish Data Protection Commission… examining whether xAI violated GDPR Articles 5, 6, 25, and 35.

the-decoder.com — dissent on ‘openwashing’ the-decoder.com

Skeptics argue that without the training data and full model architecture, the release is a form of ‘openwashing’—using a permissive license for peripheral tools to gain community goodwill while keeping the core intelligence closed.

Ayush Paul, ‘The Memory Heist’ (original disclosure) ayush.digital

Claude’s memory is a two-part system: a daily summarization pass that injects condensed user profiles into every new chat, and a conversation_search tool that queries the full historical transcript… the attacker’s server logs record the sequence of GET requests, such as /a → /ay → /ayu → /ayush, reconstructing private strings one character at a time.

Oasis Security, ‘Claudy Day’ disclosure oasis.security

A vulnerability chain dubbed ‘Claudy Day’ exploited invisible prompt injection to exfiltrate data without any specialized integrations, functioning entirely within a default Claude session.

CybersecurityNews — Claude for Chrome vulnerability cybersecuritynews.com

Despite eight consecutive releases aimed at securing the extension, a critical vulnerability remains exploitable… the flaw stems from a failure to check the isTrusted property of click events, allowing programmatic interaction from rogue scripts to masquerade as user-initiated actions.

Airia — ‘AI Security in 2026: The Lethal Trifecta’ airia.com

A June 2026 assessment by the Cloud Security Alliance found that 98% of production AI agents satisfy all three conditions of the Lethal Trifecta, making them structurally exploitable by design.

HelpNetSecurity — Cisco ‘MemoryTrap’ disclosure helpnetsecurity.com

By embedding malicious instructions in a MEMORY.md file within a repository, attackers can permanently alter the agent’s behavior across all future sessions and reboots.

Executive Offense newsletter (beehiiv) executiveoffense.beehiiv.com

For other reports, such as those by Rehberger, the company initially dismissed the findings as ‘model safety’ issues rather than technical security vulnerabilities before eventually acknowledging them as valid security flaws following public attention.

Artificial Analysis artificialanalysis.ai

Inkling debuts as the leading U.S. open-weights model on the Intelligence Index at 41, ahead of Nemotron 3 Ultra and Gemma 4 31B, but records a 63% hallucination rate on AA-Omniscience — the model prioritizes attempting an answer over abstaining.

SWE-bench project audit swebench.com

Over 59% of failures on SWE-bench Verified stem from flawed test cases rather than model limitations, suggesting scores near 80% may represent a performance ceiling for the dataset.

MarkTechPost marktechpost.com

Inkling-Small (12B active / 276B total) scores 83.4% on IFBench versus 79.8% for the 41B flagship and matches it on SWE-bench Verified (77.4 vs 77.6), but collapses to 20.9% on SimpleQA Verified versus 43.9% — a sharp knowledge/instruction tradeoff.

Kingy.ai analysis kingy.ai

At maximum reasoning effort Inkling matches significantly larger proprietary systems on Terminal-Bench 2.1 while using roughly one-third of the generated tokens; post-training used 30M rollouts across tool-use and math environments.

gHacks ghacks.net

Running the full BF16 checkpoint requires ~2TB of GPU memory (multiple H200/B300 systems); 1-bit Unsloth Dynamic 2.0 GGUF quants fit in ~270–290GB with roughly 18% accuracy loss, and vLLM/SGLang GGUF support remains ‘highly experimental’ with missing fused-expert MoE kernels.

Silicon Republic (Lambert/Karpathy commentary) siliconrepublic.com

Nathan Lambert called Inkling ‘the new best American model’ and Karpathy praised Tinker for providing ~90% of DIY algorithmic control while removing 90% of the infra overhead — though Ramp Labs noted the platform initially cannot host custom reward functions, forcing latency-adding remote hosting.