Shopify agents rebuild Shop, Datasette splits AI fixes, TryNix boots in browser

TL;DR

• Shopify rebuilt the Shop app fully native in 12 weeks with 6 engineers on an internal agent stack.
• Datasette 1.0a39 patches access-control bugs via a two-human test-first split auditing three frontier models.
• TryNix boots any Nix package from the last 13 years inside a browser tab via qemu-wasm.
• curl killed its bug bounty this year after AI-generated reports collapsed valid-rate from 1-in-6 to near zero.

Three tech ships land today, and none of them are really about the model or the runtime at the center — they’re about what’s wrapped around it. Shopify’s rewrite of the Shop app in twelve weeks with six engineers only works because of a bespoke internal agent stack — Pi, Tardis, Helix — doing Swift↔Kotlin translation, debugging, and ‘AI slop’ review. Datasette’s access-control patches are AI-audited across Claude Fable 5.1, GPT-5.6, and GPT-6 Astra, but the trust comes from a two-human test-first split, not the models. TryNix runs any Nix package from the last thirteen years in a browser tab, and the load-bearing trick is a qemu-wasm VM plus a fake builtins.appendContext derivation that skips Nix evaluation entirely.

The counter-example is sitting inside Datasette’s own release note: curl killed its bug bounty this year after AI-generated reports crushed its valid-report rate. The pattern holds — the AI (or the runtime) is only as useful as the process wrapped around it.

Shopify rebuilds Shop app native in 12 weeks with AI agents

Source: simon-willison · published 2026-09-10

TL;DR

• Shopify rebuilt the Shop app fully native in ~12 weeks with a core team of six engineers
• Post-rewrite gains: 23% faster iOS cold start, 50% on Android, 109 MB smaller binary, 99.95% session stability
• An internal agent stack — Pi, Tardis, Helix — handles Swift↔Kotlin translation, debugging, and “AI slop” review
• Critics dismiss the setup as a 3,000+ engineer luxury with a “disappointing” 500ms screen-load target

The measurable wins

Simon Willison flagged the pivot as “the exact reason you would expect” — agents now absorb enough of the dual-platform tax that native’s productivity penalty stops being decisive. The follow-up numbers make the claim testable. Shopify rebuilt the Shop app native in roughly 12 weeks with six engineers on the core team ¹, and the rewrite delivered a 23% faster iOS cold start, a 50% faster Android cold start, a 109 MB reduction in Android build size, and session stability above 99.95% ². Those are the concrete deltas the original announcement gestures at without listing.

The agent stack behind the pivot

The interesting engineering isn’t Swift or Kotlin — it’s the internal tooling Shopify built to keep two codebases in sync without a human doing translation shifts. “Pi” runs subagents that port features across platforms, “Tardis” debugs against captured live-event streams, and “Helix” is an explicit reviewer whose job is to reject the incoherent output that agents routinely produce ¹.

flowchart LR
    Spec[Product spec / bug] --> Pi{Pi orchestrator}
    Pi --> Swift[Swift/iOS impl]
    Pi --> Kotlin[Kotlin/Android impl]
    Swift --> Tardis[Tardis: live-event debug]
    Kotlin --> Tardis
    Tardis --> Helix{Helix reviewer}
    Helix -->|reject slop| Pi
    Helix -->|approve| Merge[Merge to native codebase]

The Helix layer is the load-bearing part. Enterprise AI-governance benchmarks find that over 80% of agent-generated code ships with at least one vulnerability, and agents frequently produce “globally incoherent” output that passes local tests while violating broader architecture ³. A deterministic review gate isn’t optional at this scale; it’s the thing that makes the whole thesis viable.

Ecosystem fallout

Shopify was a major React Native contributor, and the handoffs are more granular than the blog post lets on. react-native-skia will be forked and renamed by original author William Candillon, with Shopify funding him through end of 2026. FlashList stewardship is moving to Software Mansion — already the maintainers of Reanimated and Gesture Handler, so the community’s brain-drain risk is at least partly hedged. Restyle gets archived outright ⁴.

Does this generalize?

The Hacker News reception was skeptical in specific, quantitative ways. Commenters called Shopify’s own 500ms screen-load benchmark “disappointing” for something being sold as native-quality — a truly native target sits closer to 100ms — and argued the agent workflow is a luxury of a 3,000+ engineer org that can afford to build Pi, Tardis, and Helix in-house ⁵. Smaller teams still get more leverage from a single JS codebase than from paying two native tax bills, even with off-the-shelf agents.

Independent context cuts both ways. Meta shipped Threads fully native, and 2025-era analyses still cite a persistent “Native Ceiling” around BLE, sensors, and high-fidelity A/V that survives React Native’s New Architecture ⁶. So Shopify’s pivot is either a scale-specific curiosity or a leading indicator that the “write once, run anywhere” premium has quietly collapsed. The evidence in this bundle — real perf wins ², a bounded OSS transition ⁴, and a legitimate dispute about agent-maintenance economics ⁵³ — supports treating it as the second, at least for anyone with the engineering budget to build their own Helix.

---

Datasette ships AI-audited fixes via test-first human split

Source: simon-willison · published 2026-09-11

TL;DR

• Datasette 1.0a39 and 0.65.4 patch access-control bugs affecting mixed public/private table deployments on the open web.
• Audit ran three frontier models — Claude Fable 5.1, GPT-5.6, GPT-6 Astra — over roughly a week.
• One human writes the failing test, the other writes the fix — two humans review every issue independently.
• curl killed its bug bounty this year after AI-generated reports collapsed its valid-rate from 1-in-6 to near zero.

What actually shipped

Two patch releases, both addressing improper access control on Datasette instances that expose a mix of public and private tables on the open web. The initial reports came from Sevban Dönmez, a senior offensive-security consultant credited with 100+ zero-days and OWASP Web Security Testing Guide contributions ⁷. From there, Simon Willison and Alex Garcia ran an extended internal audit using three frontier models and spent nearly a week collaborating on the fixes ⁸.

No CVE IDs, no vulnerability classes, no public reproduction. The Datasette team is holding the AI-generated test scripts back temporarily to give operators time to upgrade — a disclosure trade-off the wider community has not settled.

The workflow is the story

The models found the bugs. The process is what makes the release defensible:

flowchart LR
    A[Frontier model audit<br/>Claude Fable 5.1 / GPT-5.6 / GPT-6 Astra] --> B[Candidate issue]
    B --> C[Human A: writes failing test]
    B --> D[Human B: writes fix]
    C --> E[Cross-review<br/>+ different coding agents]
    D --> E
    E --> F[Patch ships]

Splitting test-authoring from fix-authoring forces two humans to independently understand each issue, and running different agents on each side is a cheap way to catch model-specific blind spots ⁸. Willison has been formalizing this as “agentic engineering” — specialist subagents (test runner, code reviewer) in fresh context windows to avoid the token exhaustion that degrades single-agent audits ⁹.

The dissent: AI auditing is drowning other maintainers

The rosy read on frontier-model security work is real. Cisco says it scanned 1.8 billion lines of code in eight weeks — a task previously estimated at eight years — with false-positive rates under 3% once models were given threat models and live environments ¹⁰. Anthropic’s Project Glasswing partners logged over 10,000 high/critical vulnerabilities in a single month ¹¹.

The counter-narrative is just as loud. curl’s Daniel Stenberg shut down his bug-bounty program this year after “AI slop” reports collapsed the valid-rate to near zero, and rclone and others report the same triage burden ¹².

The valid-rate for reports collapsed from one in six to nearly zero.

The Datasette case is pointedly not that pattern. The vulnerability tip came from a credentialed human researcher, and the AI work was done by the maintainers themselves rather than lobbed over the wall by a stranger. “Maintainer-driven audit” and “external submitter with an LLM” have opposite economics for a small project.

What’s still unproven

Cost. Running three frontier models across a codebase for a week is not free, and no smaller OSS project has published a comparable case study. Willison sits at a tier of tooling access most maintainers don’t — the open question is whether the test-first split holds up when the audit budget is $50 instead of whatever Datasette’s was. Until someone reproduces the pattern one rung down, this is an existence proof, not a template.

---

TryNix boots any Nix package in-browser via qemu-wasm

Source: simon-willison · published 2026-09-10

TL;DR

• TryNix boots any Nix package from the last 13 years inside a browser tab via a qemu-wasm x86_64 VM.
• qemu-wasm stays in a TCI interpreter until a block runs 1,000+ times, then JITs a per-block Wasm module.
• A builtins.appendContext fake derivation skips Nix evaluation and pulls prebuilt store paths straight from cache.
• Closures must fit in a ~1.5GB tab budget, capping trynix at small CLI tools, not CI.

A URL that boots Python 3.6.2

Farid Zakaria’s trynix.dev does something that shouldn’t be interactive: click a link, get an x86_64 Linux VM running an arbitrary Nix package from the past 13 years, entirely in your browser tab. ?pkg=python3@3.6.2 boots Python from 2017. There is no backend doing the work — the WebAssembly and package assets ship from a CDN and the emulation happens on your laptop. Environments are URL-addressable, so a link is the whole artifact.

The “magnum opus” framing Zakaria uses is earned, because two separate pieces of engineering had to line up before this became usable rather than a demo.

How it actually stays interactive

Naively porting QEMU to Wasm dies on the cost of WebAssembly.Module — instantiating a fresh module per translation block is ruinously slow. qemu-wasm sidesteps this by starting execution in the Tiny Code Interpreter (TCI) and only promoting “hot” blocks — those executed more than 1,000 times — to a TCG JIT that emits a per-block Wasm module importing QEMU’s shared linear memory ¹³.

On the Nix side, Zakaria’s nixpkgs-multiverse “fast mode” refuses to evaluate Nixpkgs at all. It uses builtins.appendContext to hand the CLI a fake derivation pointing at a known store path, then runs a polynomial-time interval-covering algorithm to pick the fewest historical Nixpkgs revisions that satisfy a request across ~310,000 package versions ¹⁴. Without that shortcut, “boot Python 3.6.2 in five seconds” would be “wait for Nix to evaluate a 2017 tree first.”

flowchart LR
    U[URL: ?pkg=python3@3.6.2] --> M[nixpkgs-multiverse fast mode]
    M -->|fake derivation + store path| C[(Cachix / CDN)]
    C -->|prebuilt closure| Q[qemu-wasm x86_64 VM]
    Q -->|TCI interpreter| H{hot block?<br/>&gt;1000 executions}
    H -->|yes| J[emit per-block Wasm JIT module]
    H -->|no| Q
    Q --> S[interactive shell in browser tab]

The ceilings nobody’s demoing

Community testing sets the honest limits. Closures have to fit inside a browser tab’s ~1.5GB working set — Wasm32’s 4GB address space is the hard wall — and large binaries take 1–2 minutes to boot, which pushes trynix toward small CLI tools rather than a CI replacement ¹⁵. Independent Wasm benchmarks put the baseline slowdown at 45–55% vs native with worst cases of 2–2.5×, a penalty that compounds when you stack a QEMU x86_64 interpreter on top ¹⁶. CheerpX/WebVM’s dedicated x86-to-Wasm JIT is reportedly 10–100× faster than v86 and boots unmodified Debian ¹⁷, but it can’t run arbitrary unmodified kernels the way a QEMU-based approach can. Trynix trades peak speed for generality.

The PR-preview footgun

Zakaria’s companion project, trynix-preview, has a GitHub Action post a boot link on every pull request — “no servers, just browsers.” Buried in the docs is the caveat: it requires allow-unsafe-pr-checkout: true, and he explicitly recommends segregated caches so fork PRs can’t push to a production Cachix ¹⁸. Teams bolting this onto public repos should read that section twice.

Why AI folks should care

Willison filed this under WebAssembly craft, but the AI-adjacent read is sharper: trynix is a URL-addressable, deterministic, sandboxed x86_64 environment that a model or agent could hand to a user — or to itself — as a reproducible execution substrate. Dependency reproducibility is where most agent code-execution stacks quietly break. A link that boots the exact toolchain, every time, from 2017 or 2026, is a primitive worth watching.

Footnotes

1. Shopify Engineering — Shop app migration details — https://shopify.engineering/shop-app-migration

   The Shop app rewrite moved from proof-of-concept to launched native version in about 12 weeks with a core team of six engineers, using an internal agent system ‘Pi’ with subagents for translation, a debugging tool ‘Tardis’ and a reviewer ‘Helix’ to prevent ‘AI slop’.

   ↩ ↩²

2. sesamedisk.com writeup of Shopify’s benchmarks — https://sesamedisk.com/shopify-switching-from-react-native-to/

   Cold startup times decreased by 23% on iOS and 50% on Android, the Android build shrank by 109 MB, and session stability reached over 99.95% after the native rewrite.

   ↩ ↩²

3. Augment Code — AI code governance report — https://www.augmentcode.com/guides/ai-code-governance-framework-for-enterprise-dev-teams

   Over 80% of AI-generated code introduces at least one vulnerability on Agent Security League benchmarks, and agents frequently produce ‘globally incoherent’ code that passes local tests but violates broader system architecture — requiring deterministic validation gates and Skills-style guardrails.

   ↩ ↩²

4. Shopify Engineering — ‘Back to native’ post (library handoffs) — https://shopify.engineering/back-to-native

   React Native Skia will be forked by original author William Candillon under a new name (Shopify sponsors through end of 2026); FlashList stewardship is transitioning toward Software Mansion; Restyle will be archived at the end of 2026.

   ↩ ↩²

5. Hacker News discussion (item 49644607) — https://news.ycombinator.com/item?id=49644607

   Commenters called Shopify’s internal 500ms screen-load benchmark ‘disappointing’ for a truly native experience (which should target ~100ms), and warned the AI-agent workflow is a luxury only feasible for a company with 3,000+ engineers — smaller startups still need the cross-platform abstraction.

   ↩ ↩²

6. daily.dev — ‘Shopify Ditches React Native and AI Is Why It Finally Made Sense’ — https://daily.dev/posts/shopify-ditches-react-native-and-ai-is-why-it-finally-made-sense-njsh5afoy

   Meta itself opted for a fully native approach for Threads, and modern departures still cite a ‘Native Ceiling’ around BLE, sensors, and high-speed A/V — even after React Native’s New Architecture ‘burnt the bridge’ with Fabric/TurboModules.

   ↩

7. GitHub — jankesec (Sevban Dönmez) profile — https://github.com/jankesec

   Senior Cyber Security Consultant with 13+ years of offensive security experience, credited with 100+ zero-day discoveries and contributions to the OWASP Web Security Testing Guide.

   ↩

8. Datasette project blog — September security releases — https://datasette.io/blog/2026/september-security-releases/

   One of us would create the automated tests highlighting the issue, then the other would implement the fix. This ensured that two separate humans had eyes on each of the issues, in addition to our coding agents running different models.

   ↩ ↩²

9. simonwillison.net — Agentic Engineering Patterns: Subagents — https://simonwillison.net/guides/agentic-engineering-patterns/subagents/

   Specialist subagents — a ‘test runner’ or ‘code reviewer’ — run in fresh context windows, preventing token exhaustion and enabling the split-responsibility workflow that underpins the Datasette audit.

   ↩

10. Cisco Security blog — ‘Frontier AI just raised the stakes’ — https://blogs.cisco.com/security/frontier-ai-just-raised-the-stakes-and-the-old-playbook-wont-hold-up

    Cisco reported scanning 1.8 billion lines of code across its portfolio in just eight weeks — a task previously estimated to take eight years — with false-positive rates below 3% once models were given threat models and live environments.

    ↩

11. Qualys blog — ‘The models that found 10,000 zero-days…’ — https://blog.qualys.com/qualys-insights/2026/09/09/the-models-that-found-10000-zero-days-broke-into-three-companies-using-weak-passwords

    Under Anthropic’s ‘Project Glasswing,’ partners identified over 10,000 high- or critical-severity vulnerabilities in a single month; the same models also broke into three companies using weak passwords during evaluations.

    ↩

12. ZDNet — ‘How fake security reports are swamping open source projects’ — https://www.zdnet.com/article/how-fake-security-reports-are-swamping-open-source-projects-thanks-to-ai/

    curl creator Daniel Stenberg closed the project’s bug bounty program after being overwhelmed by ‘AI slop’ claiming nonexistent vulnerabilities, noting the valid-rate for reports collapsed from one in six to nearly zero.

    ↩

13. Kohei Tokunaga slides on qemu-wasm internals — https://www.slideshare.net/slideshow/running-qemu-inside-browser/275371587

    Because browser APIs like WebAssembly.Module are computationally expensive to invoke for every translation block, qemu-wasm begins execution using the Tiny Code Interpreter (TCI) and only promotes ‘hot’ blocks — those executed more than 1,000 times — to a TCG JIT backend that generates a unique Wasm module per block.

    ↩

14. Farid Zakaria — ‘nixpkgs-multiverse fast mode’ — https://fzakaria.com/2026/08/14/nixpkgs-multiverse-fast-mode

    A ‘fake derivation’ trick using builtins.appendContext attaches store-path metadata directly to a string, letting the Nix CLI treat a known historical path as a valid build product and fetch it immediately from a cache without evaluating the original Nixpkgs source — a polynomial-time interval-covering algorithm then finds the fewest revisions covering a requested set of ~310,000 package versions.

    ↩

15. Hacker News discussion (item 47314161) — https://news.ycombinator.com/item?id=47314161

    Closures must fit within a browser tab’s memory limit, typically around 1.5GB, and are restricted by WASM’s 32-bit address space (4GB); large binaries can take 1–2 minutes to boot, making the tool better suited for small-to-medium CLI utilities rather than a full CI replacement.

    ↩

16. Jangda et al., ‘Not So Fast’ (ATC ‘19 / arXiv) — https://arxiv.org/html/1901.09056v3

    Wasm-compiled code typically runs 45% to 55% slower than native code in modern browsers, with peak slowdowns of 2.0x to 2.5x — a floor that bounds any browser-based emulator layered on top.

    ↩

17. Hacker News thread on WebVM/v86 comparisons — https://news.ycombinator.com/item?id=42135145

    WebVM (CheerpX) offers 10x to 100x better performance than v86 via an x86-to-WebAssembly JIT and can run unmodified Debian at near-native speeds, but browser VMs still hit I/O bottlenecks and struggle with features requiring direct kernel access such as Docker-in-browser.

    ↩

18. Farid Zakaria — ‘Review a pull request by booting it’ — https://fzakaria.com/2026/09/09/review-a-pull-request-by-booting-it

    The workflow requires setting allow-unsafe-pr-checkout: true … users should use private, segregated caches for PR builds to ensure that untrusted code cannot push to a primary production cache.

    ↩