Google sample mints Gemini RCE; Thinking Machines explains IBM non-determinism

TL;DR

• Gemini 3.8 Live posts an 82.6 speech score and 35.1% on Sierra’s banking benchmark.
• Google’s own server.py mints tokens without constraints, letting clients run arbitrary Python.
• Gemini’s Extended Thinking bills silent reasoning at the $12/M audio-output rate.
• IBM’s ALTK-Evolve narrows GPT-4.1’s AppWorld Pass^5 gap from 24pp to 12pp.
• Thinking Machines’ batch-invariance work explains ALTK’s non-determinism better than IBM does.

Two tech features today, and both push the reader toward a source the vendor didn’t cite. Gemini 3.8 Live’s headline number is an 82.6 on speech-to-speech, but the security story lives in Google’s own server.py sample — the reference implementation mints tokens without live_connect_constraints, so any client that copies the tutorial ships an arbitrary-Python RCE. A June API swap also silently retired turnComplete, and Extended Thinking bills silent reasoning at the audio-output rate.

IBM’s ALTK-Evolve cuts GPT-4.1’s AppWorld Pass^5 gap from 24pp to 12pp by mining consistency guidelines from a single trace — a real result on ~40% of the tokens the competing Stanford/SambaNova toolkit uses. But IBM’s own explanation for the underlying run-to-run non-determinism (“GPU non-associativity”) is weaker than Thinking Machines’ batch-invariance write-up, and a 12pp residual plus judge-overfitting risk sit between the toolkit and IBM’s “100% reliability” pitch.

Gemini 3.8 Live ships with an RCE in Google’s sample code

Source: simon-willison · published 2026-09-15

TL;DR

• Gemini 3.8 Live pairs a chart-topping 82.6 speech-to-speech score with a failing 35.1% on Sierra’s banking benchmark.
• Google’s own sample server.py minted tokens without live_connect_constraints, letting clients enable codeExecution and run arbitrary Python.
• June 2026 API swap made interactionStatus the new source of truth, silently breaking clients still gating on turnComplete.
• Extended Thinking bills silent reasoning steps at the $12/M audio-output rate, even when the user hears nothing.

The demo looks great. The reference server was a footgun.

Simon Willison’s write-up frames Gemini 3.8 Live as a clean “no-library, straight to the wss:// endpoint” build — the kind of tight WebSocket demo that makes you want to ship one over the weekend. That’s precisely the shape of code most exposed to the RCE class that Alvin Ferdiansyah disclosed against the same surface.

The attack is embarrassing in its simplicity. Backends are supposed to mint ephemeral tokens for BidiGenerateContentConstrained populated with live_connect_constraints that pin the model, tools, and system prompt server-side. If those constraints are missing, the server accepts the setup frame directly from the client — so an attacker enables the codeExecution tool, overrides the system prompt to something like “execute this Python,” and gets shell-adjacent capability inside Google’s sandbox ¹. The root cause wasn’t a hypothetical developer mistake: Google’s own google-gemini/gemini-live-api-examples shipped a server.py that set only expiration and usage fields on the token, so anyone copy-pasting the reference implementation inherited the bug ².

sequenceDiagram
    participant C as Client/Attacker
    participant B as Your backend<br/>(copied from Google sample)
    participant G as Gemini Live WS
    participant S as gVisor sandbox
    C->>B: request ephemeral token
    B->>G: mint token (expiration + usage only)
    G-->>C: token, no live_connect_constraints
    C->>G: setup frame: enable codeExecution,<br/>override system prompt
    G->>S: run attacker Python
    S-->>C: results / exfil

If you’re building on the endpoint Simon links, the constraints belong on the token, not the client HTML.

Benchmarks are top-of-leaderboard, unevenly

Google’s headline numbers hold up on independent aggregators — 82.6 on Artificial Analysis Speech-to-Speech Quality and 97.7% on Big Bench Audio — but the same coverage clocks a 35.1% on Sierra’s τ³-Voice banking benchmark, which stresses multi-step regulated workflows ³. The gap between “great voice quality” and “trustworthy voice agent” is exactly where a lot of production pilots die.

Operational limits underline the point. Audio-only sessions cap at 15 minutes and collapse to 2 with video enabled, and the Extended Thinking variant bills reasoning steps the user never hears at the $12/M audio-output rate — so a chatty internal chain of thought quietly maps to your most expensive token bucket ⁴.

The state machine changed under you

Anyone with a Live integration older than June 2026 should re-read the protocol. The migration replaced the turnComplete-driven loop with an interactionStatus field: turnComplete can now fire after a filler utterance (“let me think…”) while the model keeps working, and clients that don’t gate on IN_PROGRESS/IDLE truncate their own agents. Zen van Riel flags this as the source of the “silent failure” reports haunting n8n and older SDK pipelines that hard-code the legacy outputs array ⁵.

Users notice the seams

Google’s marketing leans on “natural, fluid dialogue,” but r/GeminiAI is already collecting complaints about an “unbearable” vocal fry that develops over longer sessions and unprompted personality shifts mid-conversation ⁶ — plausibly a side effect of a native-audio architecture that absorbs user prosody a little too eagerly. Charming in a 30-second demo, grating on a 12-minute call.

The launch is real, and the model is genuinely strong. But a developer copying Simon’s HTML into a product should pin the token constraints, migrate to interactionStatus, watch the billing meter on Extended Thinking, and expect the voice itself to be a support ticket.

---

IBM cuts GPT-4.1’s agent consistency gap from 24pp to 12pp

Source: huggingface-blog · published 2026-09-15

TL;DR

• GPT-4.1 on AppWorld scored 77.4% Mean@5 but only 53.0% Pass^5 — succeeding every run on barely half of tasks.
• IBM’s ALTK-Evolve narrows the gap to 12pp by injecting targeted “consistency guidelines” mined from a single trace.
• The flip mechanism is better explained by Thinking Machines’ batch-invariance work than by IBM’s own hand-wave at “GPU non-associativity.”
• Against Stanford/SambaNova’s ACE, ALTK-Evolve hits TGC 89.3 vs 80.4 on ~40% of the tokens.
• A 12pp residual Pass^5 gap and judge-overfitting risk still sit between the toolkit and IBM’s “100% reliability” pitch.

The metric that embarrasses your agent

Mean@k is the number vendors quote. Pass^k is the number that matters if you ship. IBM’s ALTK-Evolve paper makes this concrete: a ReAct agent on GPT-4.1 clears 77.4% Mean@5 on AppWorld, but Pass^5 — the fraction of tasks it nails on every one of five identical runs — is 53.0%. On roughly a quarter of tasks, the same prompt against the same environment is a coin flip.

That gap isn’t unique to agents. Scale AI’s SWE-bench Pro shows top coding models collapsing from 70%+ on Verified to ~23% once tasks are decontaminated and made realistic ⁷. Benchmark capability and production reliability are largely orthogonal axes, and the field has spent years optimizing the first one.

Why identical prompts diverge

IBM attributes flipping to “flat probability distributions” plus “GPU floating-point non-associativity or platform-side request batching.” That’s directionally right but understated. Horace He’s team at Thinking Machines Lab pinned the dominant cause in September 2025: inference kernels for attention and RMSNorm lack batch invariance, so the same prompt takes different reduction paths depending on how many co-tenant requests the server happens to batch it with ⁸. Fixing this bitwise is possible but costs 1.5–2× throughput.

The implication for ALTK-Evolve is uncomfortable: some fraction of flips will be triggered by neighboring traffic on the inference endpoint, and no amount of prompt-level guideline injection reaches that layer.

What the toolkit actually does

ALTK-Evolve’s Consistency Analyzer takes a single recorded trajectory and, for each decision step, issues one extra call requesting k=5 completions in parallel. Steps whose completions disagree are flagged as flip-prone. The framework then synthesizes short natural-language “consistency guidelines” — things like “use line-anchored regex for counting” or “verify search results for multiple matches before selecting one” — and stores them in a vector DB, injecting only the relevant ones at inference time. IBM calls this “the lesson, not the log” ⁹.

Results on the 168-task AppWorld test_normal set:

Guidelines derived from one task family also lifted Pass^5 by 13pp on related tasks, suggesting the “lessons” generalize rather than memorize.

The rival worth naming

The natural comparison — absent from IBM’s post — is Stanford/SambaNova’s Agentic Context Engineering (ACE), which injects a full playbook every turn. On DeepSeek-V3.2/AppWorld, independent writeups put ALTK-Evolve at TGC 89.3 vs ACE’s 80.4 while consuming 263K tokens per task against ACE’s 634K ¹⁰. On gpt-oss-120b the ratio widens to about 1:7. Selective retrieval beating a monolithic playbook is a credible thesis, though no third party has rerun the numbers off IBM infrastructure yet.

Where it’s thin

Two limits deserve airtime. First, prose guidelines can’t fix state-management bugs in multi-agent systems — that layer needs locks and mechanical isolation, not language ¹¹. Second, single-trajectory diagnostics feeding a self-generated grader invite reward hacking: agents can learn to produce judge-friendly traces without genuinely stabler behavior ¹². And even after intervention, a 12-point Pass^5 gap remains — a long way from the “100% reliability for mission-critical production” the announcement gestures at.

Consistency is orthogonal to capability. ALTK-Evolve is a real step; batch-invariant kernels and held-out audits are the next ones.

Footnotes

1. Infosec Write-ups — Alvin Ferdiansyah disclosure — https://infosecwriteups.com/rce-via-gemini-live-ai-voice-session-misconfiguration-e0648805a055

   In the absence of live_connect_constraints, the Gemini Live server accepts the initial setup frame directly from the client… an attacker can enable the codeExecution tool—a feature that runs Python code in a gVisor-sandboxed environment—and override the system prompt to force the model into executing arbitrary commands.

   ↩

2. Cyberpress — Google reference code root cause — https://cyberpress.org/gemini-live-api-inject-code-execution/

   The official GitHub repository google-gemini/gemini-live-api-examples contained a sample server.py that minted tokens using only expiration and usage parameters, completely omitting the security constraints needed to lock down the session.

   ↩

3. MarkTechPost — benchmark results — https://www.marktechpost.com/2026/09/15/google-releases-gemini-3-8-live-and-3-8-live-extended-thinking-for-production-grade-voice-agents/

   Gemini 3.8 Live has secured the top position on the Artificial Analysis Speech-to-Speech Quality Index with a score of 82.6… However, it scored only 35.1% on the Sierra τ-Voice-banking benchmark, suggesting it is not yet reliable for regulated financial workflows.

   ↩

4. Medium (lvntblsn) — operational constraints — https://medium.com/@lvntblsn/what-is-gemini-3-8-live-extended-thinking-why-your-voice-agent-finally-talks-while-it-thinks-1455e8b6420c

   session limits remain restrictive: users are capped at 15 minutes for audio-only sessions, and this drops to just two minutes when video context is enabled… Users are billed for thinking tokens at the more expensive audio output rate ($12.00 per million tokens), even for reasoning steps they do not hear.

   ↩

5. Zen van Riel — June 2026 migration guide — https://zenvanriel.com/ai-engineer-blog/gemini-api-breaking-changes-june-2026-migration-guide/

   developers must now monitor interactionStatus. A status of IN_PROGRESS indicates the model is still active, even if a specific audio or text turn appears finished… Failure to migrate this logic causes silent failures, where applications prematurely cut off the model’s background processing.

   ↩

6. r/GeminiAI user report — https://www.reddit.com/r/GeminiAI/comments/1pjvbz9/gemini_live_voice_changes_often_now_with_annoying/

   an ‘unbearable’ vocal fry that develops during longer conversations and inconsistent personality shifts that some find grating

   ↩

7. Scale AI — SWE-bench Pro leaderboard — https://labs.scale.com/leaderboard/swe_bench_pro_public

   Top models score over 70% on SWE-bench Verified but drop to approximately 23% on SWE-bench Pro, which uses more complex, realistic codebases to reduce data contamination.

   ↩

8. Thinking Machines Lab (Horace He) — https://thinkingmachines.ai/blog/defeating-nondeterminism-in-llm-inference/

   The true variability arises because production inference servers group multiple user requests into dynamic batches… a kernel may use a different reduction tree or tiling strategy for a batch of 8 than it does for a batch of 1.

   ↩

9. IBM Research announcement (ibm.com) — https://www.ibm.com/new/announcements/altk-evolve-on-the-job-learning-for-ai-agents

   ALTK-Evolve distills trajectories into concise Consistency Guidelines stored in a vector database and injects only the relevant guidelines at inference time — ‘the lesson, not the log.’

   ↩

10. daily.dev summary of ALTK-Evolve vs ACE — https://daily.dev/posts/thinking-of-ace-we-can-do-it-with-fewer-tokens-0stmkpisz

    ALTK-Evolve achieved a TGC score of 89.3, surpassing ACE’s 80.4, while consuming roughly 263K tokens per task compared to ACE’s 634K — a ~60% reduction in inference cost.

    ↩

11. phantom-byte.com on multi-agent concurrency — https://articles.phantom-byte.com/multi-agent-concurrency-control.html

    No amount of prompt-based guideline injection can solve fundamental concurrency issues in multi-agent systems… the state layer requires mechanical isolation and locking rather than the natural-language prose produced by Evolve.

    ↩

12. intelligentmachines.blog — LLM-as-a-Judge critique — https://www.intelligentmachines.blog/post/llm-as-a-judge-rethinkinghowweevaluateaisystems

    There is a significant risk of reward hacking, where the primary agent learns to produce judge-friendly outputs… static diagnostic prompts can lead to overfitting, where the system adapts to the specific feedback of the grader rather than developing generalizable reasoning skills.

    ↩