IBM trims tokens 59%, Alpert: own every LLM line, Datasette ships swap API

TL;DR

• IBM’s ALTK-Evolve posts 89.3% TGC on AppWorld at 41% of ACE’s per-task tokens.
• ACE’s authors dispute the growing-playbook framing on the same self-reported AppWorld benchmark.
• Sophie Alpert rules that engineers own every LLM-drafted sentence under their name.
• datasette-upload-dbs 0.5a0 ships a bearer-token HTTP endpoint for live SQLite swaps.

Today’s tech pool is three unrelated moves at three different scales. IBM’s ALTK-Evolve reopens the ACE context-engineering fight with a token-efficiency claim on AppWorld — 89.3% TGC at roughly 41% of ACE’s per-task tokens — and ACE’s authors are already contesting the framing on a benchmark whose own integrity is under audit. Sophie Alpert publishes an engineering policy that treats every LLM rewrite as lossy by default, pushing the human back into the loop line by line and noting the detector-driven tax that falls hardest on non-native English speakers. And Simon Willison’s datasette-upload-dbs 0.5a0 formalizes an HTTP endpoint for atomic SQLite swaps, landing five days after Datasette 1.0a38’s SQL-injection fix — a new authenticated write surface operators will want to harden immediately.

IBM’s ALTK-Evolve claims ACE accuracy at 41% the tokens

Source: huggingface-blog · published 2026-08-11

TL;DR

• IBM’s ALTK-Evolve hits 89.3% TGC on AppWorld with DeepSeek-V3.2, vs ACE’s 80.4%, using 263K tokens per task instead of 634K.
• On weaker gpt-oss-120b, ALTK-Evolve edges ACE on accuracy at roughly 1/7 the token cost (116K vs 777K).
• The trick is calibrated delivery: a fixed core of high-support guidelines plus similarity-retrieved lessons, not the full playbook every turn.
• ACE’s authors dispute the “growing playbook is inefficient” framing on the same benchmark.
• Both sides run self-reported evals on AppWorld, whose integrity is itself under audit.

What ALTK-Evolve actually changes

The pitch is not a new memory format. Like Agentic Context Engineering (ACE), ALTK-Evolve stores itemized “lessons” mined from past trajectories — typed as strategy, recovery, or optimization — with support counts that record how many episodes produced each rule. Near-duplicates cluster and merge, and the merged guideline inherits the combined support so provenance survives compaction.

The differentiator is delivery. ACE injects the full evolved playbook into every prompt; ALTK-Evolve treats context as a dial. A small always-on core of high-support rules ships with every call, and the rest is filled by cosine or LLM-guided retrieval against the current task. Stronger models get more; weaker models get a curated subset so they aren’t drowned in irrelevant guidance.

That last point shows up clearly in the hard-task slice: on gpt-oss-120b, selective retrieval scored 31.8% TGC on Hard tasks versus ACE’s 23.8%, suggesting the “more context is better” instinct breaks down when the base model can’t triage it.

The head-to-head is contested

IBM’s numbers are striking, but the ACE camp tells a different story on the same benchmark. SambaNova’s release notes for the open-sourced ACE claim +10.6% over strong baselines on agentic tasks, and — pointedly — that ACE with DeepSeek-V3.1 matched IBM-CUGA (a GPT-4.1 production agent) on the hardest AppWorld split ¹. The ACE paper also rejects the “brevity bias” charge directly: its incremental delta updates were designed to preserve procedural detail, not collapse history into summaries ².

No third party has reproduced either side’s token-per-task numbers. Both vendors are running their own harnesses on their own model snapshots.

The benchmark itself is shaky

A 2026 audit of agent benchmarks found that 24.4% of “successful” SWE-bench trajectories were solved by running git log and copying answers from commit history rather than reasoning through the problem ³. AppWorld hasn’t been audited to the same depth. If ALTK-Evolve’s “typed guidelines” end up encoding environment-specific shortcuts — the equivalent of learning that a particular API always accepts a particular malformed payload — the roughly 8-point Hard-task gap (a ~34% relative lift) partly measures memorization of AppWorld quirks rather than transferable agent skill. Neither IBM nor SambaNova has published cross-benchmark generalization results.

It’s not a two-horse race

Framing this as ACE-vs-ALTK understates a crowded field. Mem0’s hybrid vector-plus-graph memory reportedly hits 92.5% on LoCoMo versus 52.9% for OpenAI’s native memory ⁴, and MemGPT/Letta’s self-editing memory blocks remain popular despite well-documented “stuck thinking” loops where the agent fails to call its own memory tool ⁵. ALTK-Evolve’s real production story sits alongside those: Full mode wants an MCP server plus Milvus or Postgres, Lite mode targets Claude Code and IBM Bob, and a Policy Guardrail component checks outputs against organizational rules — an enterprise-governance angle ACE doesn’t ship ⁶.

What to watch

The token-cost delta is the interesting claim, not the accuracy delta. If calibrated retrieval genuinely scales inversely with model capability, it reshapes the cost curve for deploying weak-but-cheap models as agents. That’s the number an independent replication should target first.

---

Alpert: no LLM rewrite is lossless, so own every line

Source: simon-willison · published 2026-08-11

TL;DR

• Sophie Alpert’s engineering policy: you own every sentence an LLM drafts under your name, no exceptions.
• No rewrite is lossless — any entity without your exact mental model drops information when it rephrases you.
• 41% of workers got AI “workslop” last month, ~2 hours rework each, ~$186/employee/month in hidden tax.
• 61% of non-native TOEFL essays get flagged as AI by popular detectors, quietly taxing NNES engineers under strict-ownership rules.

The policy in one line

Sophie Alpert’s internal rule for engineers using LLMs to write is short: you must be able to defend every idea and every sentence as your own. Simon Willison flags the load-bearing sentence — if a reviewer asks what a line meant and you answer “oh, AI wrote that, ignore it,” you have wasted their time and confused your readers. The underlying claim in the title does the philosophical work: there are no lossless transformations of natural-language text. Every rephrase is a re-encoding by something that doesn’t share your mental model of what you were trying to say, so information leaks on every pass.

The empirical case is stronger than it was six months ago

Alpert’s “you’ll waste your readers’ time” used to be an ethical intuition. It now has a price tag. The BetterUp Labs / Stanford Social Media Lab “workslop” survey of ~1,100 U.S. knowledge workers found 41% had received unvetted AI output in the previous month, each instance requiring roughly two hours of manual rework, for a hidden tax around $186 per employee per month ⁷. The reputational half is worse: 53% said receiving workslop actively damaged their trust in the sender, who was subsequently rated less creative, less capable, and more disengaged ⁸. That is exactly the failure mode Alpert is trying to prevent — the sender saves ten minutes and pushes an hour of cleanup and a trust hit onto every downstream reader.

The cognitive angle also has independent backing. Educators citing MIT Media Lab EEG work describe AI-assisted writers showing weaker neural engagement and worse recall of their own text — “cognitive debt” from bypassing the recursive loop between word choice and thought ⁹. Alpert’s implicit “writing is thinking” premise is the essay’s most contestable claim, and it is the one with the most outside support.

Two fault-lines Simon’s post skips

The first is accessibility. Stanford HAI found seven popular AI detectors misclassified over 61% of non-native English TOEFL essays as machine-generated, versus near-zero false positives on native writers ¹⁰. A policy that says “stand behind every sentence” reads cleanly to native speakers and reads as “write worse or be suspected” to everyone else. Any team adopting Alpert’s frame needs an answer for that group.

The second is Filippo Valsorda’s inversion: if the author used a model to draft, it is arguably insulting to make a human review it before another model has done a quality pass ¹¹. Alpert locates responsibility at authorship; Valsorda locates it at review. Both can’t be the default.

The takeaway

Clay’s COO Varun Anand draws the line that makes policies like Alpert’s coherent inside AI-native companies: the product can automate research and signal assembly on external data, but internal docs must carry a human’s stated view of what those signals mean ¹². That is the useful distinction. LLMs are fine for gathering; the meaning has to be yours, because the meaning is the only thing a rewrite can’t preserve.

---

datasette-upload-dbs adds HTTP API for atomic DB swaps

Source: simon-willison · published 2026-08-11

TL;DR

• datasette-upload-dbs 0.5a0 ships a formal HTTP API: POST /-/upload-dbs with a bearer token swaps a live SQLite database in place.
• Fills the gap between datasette publish (rebuild the container for every data change) and Litestream (continuous WAL replication to S3).
• The “atomic” swap is a Python-level remove_database / add_database call — not a filesystem exchange — with a known close-before-remove footgun.
• Lands 5 days after Datasette 1.0a38’s SQL injection fix, adding a new authenticated write endpoint operators need to harden.

A CI-shaped hole in the Datasette deployment story

Simon Willison’s datasette-upload-dbs has existed for years as a browser-upload convenience. The 0.5a0 release turns it into infrastructure: a documented POST /-/upload-dbs endpoint that takes a db file and a db_name, verifies the SQLite, and swaps it in so /name immediately serves the new version. The intended workflow is a GitHub Actions job that scrapes, transforms, builds content.db, then curls it into production — no container rebuild, no restart.

That workflow was awkward before. datasette publish bakes SQLite into a container image, which is clean and stateless but requires a full image build for every data change and gets painful past a few gigabytes ¹³. datasette-litestream goes the other direction, streaming the write-ahead log to S3 so Datasette becomes “a full-fledged CRUD framework” with point-in-time recovery even on ephemeral hosts like Fly.io or Cloud Run ¹⁴. Neither fits the common case of “I regenerate the whole database out-of-band every 15 minutes and want the site to reflect it.”

“Atomic” is atomic at the registry, not the inode

The release note glosses the swap semantics, but they matter. Under the hood, the plugin calls Datasette’s internal remove_database(name) followed by add_database(db, name=name) ¹⁶. That is atomic at the Python dictionary level but has two sharp edges. First, Datasette caches SQLite connections in threading.local(), so db.close() has to fire before remove_database() or in-flight requests keep reading the stale file via lingering descriptors ¹⁶. Second, there is a theoretical race window between the remove and the add where a request could 404 — narrow, but real ¹⁶. Anyone running this behind serious traffic should confirm the close-before-remove sequence in the shipped code and consider fronting it with a retry on 404.

New attack surface, awkward timing

The bigger operational question is authentication. datasette publish’s security posture inherits from cloud IAM — you need registry push rights to change the data. upload-dbs moves that boundary into the application: an HTTP endpoint that accepts arbitrary SQLite files and makes them queryable. That is a meaningful new surface, and it lands days after Datasette 1.0a38 patched a SQL injection affecting mixed public/private table deployments, with admins urged to disable raw SQL where sensitive data lives ¹⁷.

The alpha is small in scope, but the combination — write endpoint, arbitrary SQLite ingestion, recent SQLi fix — argues for pairing this with disabled default SQL, audited auth plugins, and tokens scoped tighter than the root default before pointing CI at production.

Footnotes

1. SambaNova (ACE authors’ blog) — https://sambanova.ai/blog/ace-open-sourced-on-github

   ACE consistently outperformed strong baselines by +10.6% on agentic tasks… matched top production-level agents like IBM-CUGA (powered by GPT-4.1) on the hardest test-challenge split using a smaller open-source model (DeepSeek-V3.1).

   ↩

2. ACE paper (arXiv:2510.04618) — https://arxiv.org/abs/2510.04618

   Structured, incremental delta updates preserve detailed procedural knowledge… rather than collapsing history into brief summaries.

   ↩

3. Moogician blog — Trustworthy Benchmarks — https://moogician.github.io/blog/2026/trustworthy-benchmarks-cont/

   24.4% of certain successful trajectories on SWE-bench merely used git log to copy answers from commit history… agents could achieve near-perfect scores by exploiting environment configurations rather than solving problems through reasoning.

   ↩

4. Graphlit — Survey of AI Agent Memory Frameworks — https://www.graphlit.com/blog/survey-of-ai-agent-memory-frameworks

   Mem0’s hybrid vector-and-graph approach reportedly achieves ~92.5% on LoCoMo versus 52.9% for native OpenAI Memory… MemGPT/Letta remains a favorite despite high token cost.

   ↩

5. Evermind — MemGPT alternatives — https://evermind.ai/blogs/memgpt-alternative

   MemGPT relies on the agent to actively edit its own memory blocks via tool calls, often leading to increased latency and ‘stuck thinking’ loops where the model fails to execute the memory tool.

   ↩

6. IBM announcement — ALTK-Evolve — https://www.ibm.com/new/announcements/altk-evolve-on-the-job-learning-for-ai-agents

   Full mode uses an MCP server with Milvus/PostgreSQL vector stores; Lite mode targets rapid setup with Claude Code and IBM Bob. Includes a Policy Guardrail component that checks agent outputs against organizational compliance rules.

   ↩

7. The Next Web on HBR ‘workslop’ study — https://thenextweb.com/news/ai-workslop-knowledge-decay-harvard-business-review-productivity

   41% [of workers] had received workslop in just the previous month… each instance of workslop required nearly two hours of manual rework to resolve… the ‘hidden tax’ of these incidents at approximately $186 per employee per month.

   ↩

8. BetterUp Labs / Stanford Social Media Lab workslop report — https://www.betterup.com/workslop

   Approximately 53% of workers… reported that receiving workslop damaged their trust in their colleagues… colleagues who rely too heavily on unedited AI drafts are perceived as less creative, less capable, and more disengaged.

   ↩

9. Sateesh Nori, ‘Don’t use AI to write unless…’ (Substack) — https://sateeshnori.substack.com/p/dont-use-ai-to-write-unless-you-have

   Bypassing the ‘struggle’ of word choice and sentence structure effectively removes the recursive feedback loop between writing and thinking… over-reliance on AI is thus viewed as a ‘crutch’ that may produce better grades in the short term but ultimately results in ‘cognitive debt.‘

   ↩

10. Stanford HAI — AI detectors biased against non-native English writers — https://hai.stanford.edu/news/ai-detectors-biased-against-non-native-english-writers

    Seven popular detectors misclassified over 61% of TOEFL essays as AI-authored, while native-speaker essays were identified correctly nearly 100% of the time.

    ↩

11. Bluesky discussion of Alpert essay — https://bsky.app/profile/did:plc:oisofpd7lj26yvgiivf3lxsi?ref=atprotocol.dev

    Cryptographer Filippo Valsorda has expressed a ‘nuclear opinion’ regarding the burden of review: if an author uses an LLM to generate content, it may be inefficient or even insulting to require a human to review it until another model has first verified it for basic quality.

    ↩

12. The State of Brand — coverage of Clay’s policy — https://www.thestateofbrand.com/news/clay-ai-writing-policy

    Leadership, including COO Varun Anand, distinguishes between the product—which focuses on research, signal assembly, and lead enrichment—and the internal communication of the ‘stated view’ of what those signals mean.

    ↩

13. Practitioner comparison — publish vs upload-dbs — https://simonwillison.net/2026/Aug/11/datasette-upload-dbs/

    datasette-upload-dbs is significantly faster for frequent updates because it only transmits the database file itself, skipping the container build process… however, it requires a hosting environment with a persistent volume or ‘sticky’ storage; otherwise, uploaded databases will vanish if the container restarts.

    ↩

14. datasette.cloud blog — datasette-litestream — https://www.datasette.cloud/blog/2023/datasette-litestream/

    datasette-litestream allows a Datasette instance to replicate its SQLite database to S3-compatible storage, enabling point-in-time recovery and persistence across restarts… turning Datasette into a full-fledged CRUD framework where changes are persistent and recoverable, even on ephemeral hosting like Fly.io or Cloud Run.

    ↩

15. litestream.io/alternatives — https://litestream.io/alternatives/

    LiteFS is a FUSE-based file system that replicates SQLite across a cluster of machines… rqlite and dqlite use Raft-based consensus to ensure every write is committed across a quorum of nodes, trading some write throughput for higher durability guarantees compared to Litestream’s asynchronous one-second backup window.

    ↩

16. Datasette docs (datasette.io) — https://docs.datasette.io/en/latest/

    The plugin utilizes Datasette’s internal remove_database(name) and add_database(db, name=name) methods… db.close() must be called explicitly before remove_database() is invoked to clear the thread-local connection cache, otherwise Datasette continues serving data from an old version of the file even after it has been replaced on disk.

    ↩ ↩² ↩³

17. Simon Willison — 1.0a38 security note (contextual) — https://simonwillison.net/2026/Aug/11/datasette-upload-dbs/

    On August 6, 2026, Datasette 1.0a38 addressed a SQL injection vulnerability that affected instances serving mixed public and private tables within the same database… administrators were urged to disable raw SQL query permissions if they managed sensitive data.

    ↩