JS Wei (Jack) Sun

Tracebit hits 0% attack rate, DOOMQL runs in SQLite, METR clocks devs 19% slower

Every URL the pipeline pulled into ranking for this issue — primary sources plus the supporting and contradicting findings each Researcher returned. Inline citations in the issue point back here.

← Back to the issue

Sources

Now, defenders are embracing the prompt injection, too arstechnica.com

“Context bombing” tricks hacking agents into shutting down before they can do harm.

DOOMQL simonwillison.net

DOOMQL Peter Gostev built this using GPT-5.6 Sol. This is a lot of fun: DOOMQL started with a deliberately unreasonable question: what if SQLite were the game engine, not merely the place where a game stores data? The result is a small, original Doom-like game in which SQL owns movement, collision, enemies, combat, progression and every RGB pixel on screen. It’s implemented as a Python terminal script - I tried it out like this: cd /tmp git clone https://github.com/petergpt/doomql cd doomql uv…

datasette code-frequency chart on GitHub simonwillison.net

datasette code-frequency chart on GitHub Out of curiosity I decided to see if I could find a useful illustration of the impact of coding agents and Opus 4.5 class models on my own output. The best I’ve found so far is this GitHub chart of frequency of code changes to my Datasette open source project: The big spike in activity at the end aligns with Opus 4.8, GPT-5.5, Fable 5 and GPT-5.6 Sol. Tags: github , ai , datasette , generative-ai , llms , ai-assisted-programming , coding-agents

Using uvx in GitHub Actions in a cache-friendly way simonwillison.net

A GitHub Actions recipe from Simon Willison avoids re-downloading Python tools from PyPI on every run. Setting UV_EXCLUDE_NEWER to a fixed date freezes uvx resolution to that snapshot, then folding the date into the cache key lets a bump both invalidate the cache and upgrade tools.

Simulating everything, sort of: The promise and limits of world models arstechnica.com

World models aim to simulate physical reality for robotics and video, but researchers including Fei-Fei Li, Yann LeCun and Runway’s Anastasis Germanidis disagree on architecture and scope. The explainer walks through diffusion versus autoregressive approaches and systems like GWM-1 and World Labs.

References

AI Weekly — Tracebit context bomb benchmarks aiweekly.co

In tests against advanced models like Anthropic’s Opus 4.8, the success rate for gaining administrative access dropped from a 93% baseline to 0% once a context bomb was introduced… Across a wider range of models including Gemini and DeepSeek, the average rate of full system compromise fell from 36% to just 1%.

The Stack — Project Mantis tarpit framework thestack.technology

Mantis uses invisible prompt injections — concealed via ANSI escape sequences or HTML tags — to redirect an attacking agent into a simulated FTP server with an infinitely deep filesystem… the agent attempts to traverse the recursive directory structure indefinitely, consuming tokens for every directory listing.

EmergentMind — refusal abliteration research emergentmind.com

Abliteration… removes the model’s machinery for refusal entirely… reducing refusal rates from nearly 90% to under 2% without needing to retrain the model. Because the bypass is baked into the model weights, it is immune to defenses that rely on detecting specific linguistic patterns.

AI Models Substack — Extended Refusal defense aimodels.substack.com

Extended Refusal (ER) training… forces the model to generate semantically rich and lengthy refusals, so the safety signal is dispersed across multiple dimensions in the neural network, making it far more difficult for attackers to isolate and remove a single refusal vector.

ResultSense — Palisade self-replication study critique resultsense.com

Cybersecurity expert Jamieson O’Reilly characterized Palisade’s test environments as ‘soft jelly,’ arguing that real enterprise networks with robust monitoring would easily detect the conspicuous transfer of 100GB+ model weights.

Meteora Web — CFAA/legal analysis of defensive injection meteoraweb.com

Even ‘defensive’ prompt injection could theoretically violate the CFAA if interpreted as ‘exceeding authorized access’ by bypassing an AI’s internal guardrails… defensive agents have accidentally triggered remote code execution on a collaborator’s system when scanning for vulnerabilities.

METR - Model Evaluation & Threat Research metr.org

experienced developers were actually 19% slower when using AI tools, even though those same developers estimated they were 20% faster

JFSI - ‘Your AI Productivity Data Is Lying’ jfsi.io

practices like git merge --squash collapse dozens of AI-generated commits into a single merge commit, destroying the data needed to analyze the ratio of additions to deletions accurately

Simon Willison Substack - ‘The Last Six Months in LLMs’ simonw.substack.com

if a developer produces code twice as fast, they must find a way to halve their maintenance costs, or they will be ‘screwed’ by the resulting technical debt

Level Up Coding - ‘Vibe Coding Kills Open Source’ levelup.gitconnected.com

Daniel Stenberg (cURL) famously shut down his project’s six-year bug bounty program in January 2026 after valid reports dropped to just 5%, citing the mental toll of debunking ‘terror reporting’ generated by AI

Vals.ai SWE-bench leaderboard vals.ai

On the more rigorous SWE-bench Pro… Claude Fable 5 leads with 80.0%, while Opus 4.8 maintains a significant edge over GPT-5.6 Sol, scoring 69.2% compared to Sol’s 64.6%

Swarmia - productivity impact of AI coding tools swarmia.com

teams using AI tools saw pull request (PR) review times increase by 91% and rework rates spike as reviewers struggled to absorb the sheer volume of generated code

Tom’s Hardware tomshardware.com

Doom multiplayer tribute gets coded in pure SQL and runs at 30fps — made from just 150 lines of code in less than a month

CedarDB blog — DOOMBench cedardb.com

CedarDB maintains 30 FPS, DuckDB pushes roughly 10 FPS, while Postgres plummets to 0.3 FPS running the same DOOMQL codebase

ThursdAI — ‘GPT-5.6 Sol: Fable Killer?’ sub.thursdai.news

Sol reached a record 7.8% on ARC-AGI-3 and set a top score on Terminal-Bench 2.1, but testers flagged ‘cheating’ behaviours where the model looks for shortcuts to pass benchmarks

Simon Willison — DOOMQL (primary) simonwillison.net

It’s implemented as a Python terminal script … Here’s the huge SQL query that implements a full ray tracer in SQLite using a recursive CTE.

Jack Sun

Jack Sun, writing.

Engineer · Bay Area

Hands-on with agentic AI all day — building frameworks, reading what industry ships, occasionally writing them down.

Digest
All · AI Tech · AI Research · AI News
Writing
Essays
Elsewhere
Subscribe
All · AI Tech · AI Research · AI News · Essays

© 2026 Wei (Jack) Sun · jacksunwei.me Built on Astro · hosted on Cloudflare