Datasette closes its agent gap, Transformers.js stalls, CUGA collapses to 7%

TL;DR

• IBM’s CUGA open-sources 24 agent apps, topping WebArena at 61.7%.
• CUGA success collapses to 7.1% when multiple policies overlap in IBM’s own data.
• Datasette 1.0a35 ships browser CREATE/ALTER TABLE UIs after agents could already mutate schemas.
• Transformers.js v4.2 ships a cross-origin model cache that no browser implements natively.
• Mozilla and WebKit reject the underlying storage primitive as a re-identification vector.

Three agent-era ships today, and each one frames the same question differently: what does the host platform owe the agent that runs on it? IBM’s CUGA harness publishes leaderboard wins (61.7% WebArena, 48.2% AppWorld) but its own data shows success collapsing to 7.1% when policies overlap — a governance ceiling baked into the launch and omitted from the post. Datasette 1.0a35 goes the other direction: Willison closes a gap his own agent created by giving humans the CREATE TABLE and ALTER TABLE UIs the LLM already had. And Transformers.js v4.2 ships a cross-origin model cache that no browser implements — Mozilla and WebKit treat the underlying primitive as a re-identification vector, and Google is deprecating the closest thing to it.

The pattern: agent harnesses are now mature enough to expose what their hosts won’t give them. Sometimes the maintainer can close the gap themselves. Sometimes the browser vendors say no.

IBM ships 24 CUGA agent apps on a policy-gated harness

Source: huggingface-blog · published 2026-06-23

TL;DR

• IBM open-sourced ~24 single-file agent apps on CUGA, its Configurable Generalist Agent harness.
• CUGA’s #1 leaderboard finishes translate to 61.7% on WebArena, 48.2% on AppWorld.
• Success rates collapse to 7.1% when multiple policies overlap — IBM’s own number, omitted from the launch post.
• The real differentiator is governance: Intent Guards, Tool Approvals, and Playbooks in a sqlite-vec semantic store.

The pitch: agents in a single file

CUGA’s release frames the harness as a way to skip the orchestration plumbing — model clients, state streaming, tool wiring — and ship an agent by writing a prompt and a tool list. The cuga-apps repo packages ~24 working examples as templates: Paper Scout for arXiv synthesis, Wiki Dive for cited research, Meetup Finder driving Playwright headlessly, and Ouroboros, a seven-agent lead-generation system. The IBM Cloud Advisor is the showcase: an architecture-recommendation agent in one file.

Under the hood, CUGA runs CodeAct (LLM emits Python that gets executed) with a reflection step that re-plans on failure, a CugaSupervisor that delegates to specialized sub-agents to keep context windows small, and Fast/Balanced/Accurate modes that trade latency for reasoning depth. It’s model-agnostic via a create_llm factory — OpenAI, Anthropic, watsonx, Ollama — though hosted demos run gpt-oss-120b.

Benchmarks: read the fine print

The “#1 on AppWorld and WebArena” claim is real but compressed. Independent write-ups put the absolute numbers at 61.7% on WebArena and 48.2% scenario completion on AppWorld ¹ — strong, not solved. More importantly, IBM’s own research shows agent success rates drop as low as 7.1% when multiple policies overlap ², a much sharper caveat than the launch post acknowledges. And the harness isn’t a uniform uplift: swapping the underlying model (gpt-oss-120b vs. Llama-4) produces “noticeably different reliability” ¹. CUGA is a scaffold, not a model substitute.

CodeAct is the right bet — with a sharp edge

CUGA’s choice of CodeAct over JSON tool-calling is well-supported: across 17 LLMs on API-Bank and M3ToolEval, CodeAct hit up to 20% higher success rates because models are pre-trained on code and can express loops, conditionals, and variables in one executable turn instead of N round-trips ³. The practitioner caveat is blunt: one r/AI_Agents thread calls CodeAct “obliterating” for efficiency but flatly unsuitable for “yolo developers” without Docker-grade isolation ⁴. CUGA’s E2B/Podman sandbox support and Tool Approval gates aren’t optional polish — they’re table stakes for the paradigm it picked.

Governance is the actual differentiator

The crowded harness market — LangGraph, CrewAI, smolagents — already has opinionated planners. Where CUGA tries to separate is the policy plane.

Intent Guards refuse out-of-scope requests; Tool Approvals pause for human-in-the-loop; Playbooks pin procedures; Output Formatters constrain responses — all stored in a sqlite-vec semantic store so policies match on meaning, not keyword. For production, CUGA hands off to IBM Sovereign Core for tenant-isolated execution. The same “policy plane” philosophy is showing up in IBM’s separate ALTK (Agent Lifecycle Toolkit) ⁶, which suggests CUGA is one front of a broader bet on governable-by-default agents rather than a standalone harness.

Takeaway

The two-dozen apps are a credible developer-experience claim and the policy plane is genuinely differentiated. The leaderboard story needs the asterisks the launch post omits — non-IBM benchmarks remain thin, and the 7.1% multi-policy collapse is the number to internalize before shipping anything that stacks guards.

---

Datasette 1.0a35 closes the agent/web UI schema gap

Source: simon-willison · published 2026-06-23

TL;DR

• Datasette 1.0a35 ships in-browser CREATE TABLE and ALTER TABLE UIs plus matching JSON APIs, ending its read-only era.
• The forcing function was Datasette Agent: Willison called it “absurd” that an LLM could mutate schemas while humans couldn’t.
• The release absorbs datasette-edit-schema and much of sqlite-utils, wrapped in token-scoped create-table / alter-table permissions.
• A new dataclass-backed template context, diff-tested against live renders, is now a stable API until Datasette 2.0.

The forcing function is an agent, not a user

The headline features in 1.0a35 — a “Create table” dialog and an “Alter table” dialog that can add, rename, reorder, or drop columns, flip NOT NULL, change types, and rewire foreign keys — read like long-overdue web UI work. The interesting part is why they shipped now. Earlier in 2026, Willison released Datasette Agent, an LLM assistant that can edit schemas through tool calls. He framed the resulting asymmetry bluntly: it was “absurd” that a chat agent could mutate tables while the standard web UI remained read-only ⁷. 1.0a35 is parity work. The same create-table, alter-table, and drop-table permissions now gate both a button click and an agent tool call ⁸.

Consolidation, not greenfield

Read alongside 1.0a34’s row editing, this release is the capstone of a multi-version migration that pulls third-party functionality into core. The long-standing datasette-edit-schema plugin is effectively deprecated ⁹. The new /<database>/-/create and /<database>/<table>/-/alter endpoints mirror sqlite-utils’s insert-and-auto-create plus alter=True patterns — but add a token-scoped HTTP permission layer and explicitly reject unsafe operations like VACUUM or writes to virtual/shadow tables ¹⁰.

The template-context counter-promise

Plugin authors have paid real cost across the 1.0 alphas — Willison himself called the 1.0a20 SQL-powered permissions overhaul “the biggest breaking API change” on the road to 1.0 ¹¹. 1.0a35’s quieter headline is the new Template Context documentation: every variable available to custom templates for the database, table, query, and row pages is now generated from dataclass definitions next to the view code, with tests that diff documented fields against actual rendered contexts. Willison is committing to those variables as a stable API until Datasette 2.0. That’s the explicit trade — pay the migration tax through the alphas, get a contract you can build themes against.

Where it gets uncomfortable

The unresolved question is sandboxing, and it’s adjacent rather than internal to this release. On HN, commenter cxr argued no combination of CSP and iframe sandbox attributes reliably isolates untrusted Datasette App content without a separate origin; Willison conceded that if the critique holds, the current architecture “might be fatally flawed” ¹². That debate is about Datasette Apps, not 1.0a35’s schema endpoints — but it shadows any deployment where the new write surface is exercised by agents rather than humans. /-/execute-write remains a root-only sharp edge ⁸, and the bet now is that token-scoped permissions plus the rejection list are enough once LLMs are the primary callers.

The net: 1.0a35 is less a feature drop than Datasette quietly admitting it’s no longer a read-only publisher.

---

Transformers.js tests shared model cache only Chrome backs

Source: huggingface-blog · published 2026-06-23

TL;DR

• Transformers.js v4.2 ships an experimental Cross-Origin Storage backend, deduplicating models like the 177 MB Whisper across origins.
• No browser implements the proposed API natively — the demo runs through a polyfill extension.
• Mozilla and WebKit are structurally skeptical, treating any unpartitioned storage as a re-identification vector even with k-anonymity gating.
• Google is deprecating Shared Storage after adoption stalled near 10% of page loads by mid-2026.

The performance problem is real

Browser caches have been double-keyed by origin since roughly 2020 to plug timing side-channel leaks. The cost: if site-a.com and site-b.com both run Xenova/whisper-tiny.en via Transformers.js, the user downloads and stores the same 177 MB ONNX file twice. Same for the 4.7 MB ort-wasm-simd-threaded.asyncify.wasm runtime that nearly every Transformers.js pipeline pulls in.

The Cross-Origin Storage proposal addresses this by addressing files by cryptographic hash instead of URL. Hash matches, the browser serves the bytes in milliseconds; hash mismatches on write, the operation fails. Developers set an origins parameter — '*' for globally shareable open-source weights, an explicit allowlist, or the default same-site scope.

The motivation isn’t speculative. The WICG repo cites Flutter engine telemetry from December 2024 showing identical canvaskit.wasm and skwasm.wasm files redownloaded across thousands of origins, and the listed integrators — Transformers.js, WebLLM, wllama, Nuxt — share the same shape of complaint: large, immutable, content-addressable assets duplicated per origin ¹³. The 177 MB Whisper case is the small end. Multi-gigabyte LLM weights are where the math gets ugly.

How the integration actually works

Setting env.experimental_useCrossOriginStorage = true makes Transformers.js check the COS layer first, then fall back to the standard Cache API, then to the network. Apps must treat hits as best-effort, because the spec’s privacy defense — availability gating — has the browser return NotFoundError even when the file is physically on disk, until that hash has been observed across a threshold of distinct origins ¹⁴. That’s a k-anonymity-style guard against using a unique file as a super-cookie.

flowchart LR
    A[Transformers.js pipeline] --> B{COS layer<br/>hash lookup}
    B -- hit + gated --> C[Serve from shared store<br/>~ms]
    B -- miss / not yet k-anon --> D[Cache API]
    D -- miss --> E[Network fetch]
    E --> F[Write to COS<br/>with declared hash]

Vendor consensus is absent

This is the part the Hugging Face post glosses. Mozilla engineer Brian Grinstead, on the WICG tracker, acknowledges the model-weight redownload problem but says Mozilla doesn’t “know of a privacy-preserving way to do this” without high-level abstraction APIs, and suggests the capability belongs in browser extensions rather than the platform ¹⁵. WebKit’s posture is older and harder: any unpartitioned storage, even immutable hash-keyed bytes, is a re-identification surface via timing side channels, continuous with Safari’s 2020 double-keyed cache ¹⁶.

Outside the engines, privacy advocates frame the broader WICG shared-storage family as a vector by which sites can “extract an arbitrary amount of data over time even with budgeted access” ¹⁷. And Google itself is winding down the related Shared Storage API after adoption stalled near 10% of page loads by mid-2026 ¹⁸ — not a death knell for COS specifically, but a hostile climate for new Chrome-led storage primitives.

What to take away

The Transformers.js v4.2 integration is worth playing with as a demo of where web-AI distribution should go: hash-keyed, CDN-independent, deduplicated across origins. Treat it as that — a Chromium-extension proof of concept — not a cross-browser bet. The interesting open question isn’t whether the perf wins are real (they are), but whether anyone produces a gating scheme that satisfies Mozilla and WebKit at the same time. Today, nobody has.

Round-ups

Pyodide playground tests browser-persistent SQLite via OPFS

Source: simon-willison

Simon Willison built a browser harness pairing Pyodide with the Origin Private File System to probe whether Datasette Lite can edit SQLite files stored on a user’s machine. Claude Code for web generated the UI, letting him compare OPFS behavior across browsers without a server round-trip.

Footnotes

1. Medium — Patriwala, ‘IBM Research releases CUGA: A Solution Architect’s Perspective’ — https://medium.com/@patriwala/ibm-research-releases-cuga-a-solution-architects-perspective-on-configurable-agent-frameworks-a8a3ebe980a3

   CUGA achieved a 61.7% success rate on WebArena and 48.2% scenario completion on AppWorld, but its hierarchical planner-executor depends heavily on the underlying model — open variants like gpt-oss-120b or Llama-4 produce noticeably different reliability.

   ↩ ↩²

2. Medium — Abivarma, ‘CUGA isn’t smart, it’s governable’ — https://abivarma.medium.com/cuga-ibms-isn-t-smart-it-s-governable-and-that-s-why-it-wins-66860d833437

   IBM’s own research admits that even advanced agents struggle when multiple policies overlap, with success rates dropping to as low as 7.1% in complex multi-policy scenarios.

   ↩

3. Medium — Barun Saha, ‘CodeAgent: The Evolution Beyond Tool Calling’ — https://medium.com/@barunsaha/codeagent-the-evolution-beyond-tool-calling-7792781e19f4

   Across 17 LLMs on API-Bank and M3ToolEval, CodeAct demonstrated success rates up to 20% higher than JSON tool-calling, because models are pre-trained on code and can express loops, conditionals and variables in a single executable turn.

   ↩

4. Reddit r/AI_Agents — ‘ReAct or CodeAct, that is the question’ — https://www.reddit.com/r/AI_Agents/comments/1t9g4qg/react_or_codeact_that_is_the_question/

   CodeAct obliterates ReAct on efficiency but it’s not for yolo developers — executing arbitrary model-written code without Docker-grade sandboxing is a production-grade footgun.

   ↩

5. dev.to — Pooya Golchian, ‘AI Agents in 2026: LangGraph vs CrewAI vs smolagents’ — https://dev.to/pooyagolchian/ai-agents-in-2026-langgraph-vs-crewai-vs-smolagents-with-real-benchmarks-on-local-llms-4ma1

   CrewAI is praised for fast prototyping but criticised for aggressive telemetry that cannot be easily disabled, while LangGraph is the most reliable for long-running systems and smolagents wins on the ‘2 AM debug score.’

   ↩

6. Medium — Alain Airom, ‘Introducing ALTK: the Agent Lifecycle Toolkit’ — https://alain-airom.medium.com/introducing-altk-the-open-source-agent-lifecycle-toolkit-e57e2990dd57

   ALTK provides modular middleware to repair brittle tool calls and validate outputs before they reach production systems — the same ‘policy plane’ philosophy that CUGA exposes via Intent Guards, Tool Approvals and Playbooks.

   ↩

7. Datasette.io — 1.0 alpha cadence and Datasette Agent context — https://datasette.io/

   The 1.0a35 work was driven in part by Datasette Agent: Willison called it ‘absurd’ that an AI chat agent could edit schemas while the standard web UI remained read-only.

   ↩

8. Datasette authentication docs — permissions for write/schema operations — https://docs.datasette.io/en/latest/authentication.html

   Schema-altering operations require specific permissions such as create-table, alter-table, and drop-table; the /-/execute-write endpoint presents significant security risks if not strictly limited to trusted root actors.

   ↩ ↩²

9. Datasette.io — supersession of datasette-edit-schema — https://datasette.io/

   1.0a35 effectively deprecates the long-standing datasette-edit-schema plugin by moving create/alter table into core, following row-level editing that landed in 1.0a34.

   ↩

10. Datasette.io — sqlite-utils lineage and alter=true pattern — https://datasette.io/

    The new create/alter JSON API mirrors sqlite-utils’ insert-and-auto-create plus alter=True flag, but adds a token-scoped permission layer (insert-row, create-table, alter-table) and rejects unsafe ops like VACUUM or writes to virtual/shadow tables.

    ↩

11. Datasette.io — plugin-author migration burden across 1.0 alphas — https://datasette.io/

    Willison acknowledged the SQL-powered permissions shift in 1.0a20 was ‘the biggest breaking API change’ on the road to 1.0; the dataclass-backed template context in 1.0a35 is the counter-promise — variables are stable until 2.0.

    ↩

12. HN discussion (via Gemini synthesis) — iframe/CSP sandboxing dissent — https://docs.datasette.io/en/latest/authentication.html

    User cxr argued no combination of CSP and iframe sandbox attributes is foolproof for untrusted content; Willison conceded that if the critique holds, the current architecture ‘might be fatally flawed.’

    ↩

13. WICG/cross-origin-storage GitHub — https://github.com/WICG/cross-origin-storage

    Motivated by Flutter engine telemetry (‘Flutter engine assets by unique hosts’, Dec 2024) showing identical canvaskit.wasm / skwasm.wasm files redownloaded across thousands of origins; Transformers.js, WebLLM, wllama, and Nuxt are listed as interested integrators.

    ↩

14. WICG cross-origin-storage draft spec — https://wicg.github.io/cross-origin-storage/

    The browser may return a NotFoundError even if a file is physically present, until the resource has been encountered across a threshold number of distinct origins (k-anonymity-style availability gating).

    ↩

15. WICG GitHub issues (Mozilla position) — https://github.com/WICG/cross-origin-storage

    Mozilla engineer Brian Grinstead: acknowledges the downside of re-downloading multi-gigabyte model weights but states they ‘don’t know of a privacy-preserving way to do this’ without high-level APIs that abstract away raw storage; suggests gating such capabilities behind browser extensions instead.

    ↩

16. Michael Tsai blog roundup — https://mjtsai.com/blog/2023/06/

    WebKit’s long-standing position treats any unpartitioned storage — even immutable, hash-keyed assets — as a re-identification risk via timing side channels, consistent with Safari’s double-keyed cache since 2020.

    ↩

17. Movement for an Open Web — https://movementforanopenweb.com/w3c-privacy-proposals-threaten-to-destroy-the-open-web/

    W3C privacy proposals threaten to destroy the open web — shared-storage style mechanisms can be exploited to extract an arbitrary amount of data over time even with budgeted access.

    ↩

18. Wikipedia: Privacy Sandbox — https://en.wikipedia.org/wiki/Privacy_Sandbox

    Google announced plans to deprecate the Shared Storage API following a shift in third-party cookie strategy and low adoption — usage fell to roughly 10% of page loads by mid-2026.

    ↩