JS Wei (Jack) Sun

Datasette ships app panel, Phonon TTS hits 28ms, Dharma's 2B tops Opus 5

Three indie dev-tool shipments today: Datasette's sandboxed app panel, Phonon's 28ms on-device TTS, and Dharma's 2B extractor beating Opus 5.

Datasette ships app panel, Phonon TTS hits 28ms, Dharma’s 2B tops Opus 5

TL;DR

  • Dharma’s 2B extractor scores 0.834 self-graded, above Claude Opus 5’s 0.816 on the same suite.
  • Dharma runs self-hosted at $2.24/hour on an L40S, 147× under frontier extraction APIs.
  • Datasette Agent 0.5a0 pins a side-panel app builder beside every table and query page.
  • Phonon’s 31M English model hits 28ms CPU TTFA, 60× faster than Kokoro on the same hardware.
  • MIT Tech Review maps 3 agentic-enterprise risks: data lineage, analytics quality, runaway tokens.

Three independent shipments land in AI tooling today, each with its own headline and its own scope. Datasette Agent 0.5a0 pins a persistent app-builder panel beside any table or query page, with generated HTML running in a locked-down iframe. Phonon publishes a 31M-parameter English TTS that hits 28ms CPU TTFA. Dharma-AI’s 2B document extractor scores above Claude Opus 5 on an in-house suite while running self-hosted at $2.24/hour on an L40S.

They don’t share a thesis so much as a profile: small models and small teams competing with frontier vendors on narrow slices. Each also ships with its own caveats — Datasette’s SQL agent still caps near 85% on messy schemas, Phonon’s voice cloning routes to Gradium’s cloud, Dharma’s 0.834 comes from an F1 × Levenshtein × valid-JSON composite the lab designed itself — but the ships themselves are the news. In the round-ups, MIT Tech Review maps three risk axes for agentic enterprises, and Simon Willison cuts an LLM 0.37.1 bugfix with four outside contributors.

Dharma’s 2B doc extractor outscores Claude Opus 5 at 1/147 cost

Source: huggingface-blog · published 2026-11-04

TL;DR

  • Dharma-AI’s 2B Field Extractor scored 0.834 in-house vs. 0.816 for Claude Opus 5 and 0.741 for GPT-5.6 Luna.
  • The suite is self-graded on an F1 × Levenshtein × valid-JSON composite that favors models trained on that exact contract.
  • $2.24/hour on an L40S runs the model self-hosted, 6×–147× cheaper than frontier extraction APIs.
  • PaddleOCR-VL and Particula independently corroborate the direction: specialized sub-10B models beating frontier APIs on narrow extraction tasks.

The claim

Dharma-AI, a Rio de Janeiro spin-off from EloGroup backed by AWS GAIA and NVIDIA Inception 1, released a 2B-parameter LoRA adapter over its DharmaOCR-LITE v1.1 base, trained on 43,384 examples spanning English, Brazilian Portuguese, and additional multilingual documents. It takes a document image plus an optional JSON schema and returns structured fields — null when a requested key is absent, rather than hallucinating a value.

On a 546-document, 1,638-scenario in-house suite:

ModelComposite score
Dharma Field Extractor (2B)0.834
Claude Opus 50.816
GPT-5.6 Luna0.741

The composite multiplies field-level F1 by Levenshtein similarity by the valid-JSON rate. On held-out public sets, Dharma reports matching all 18 SROIE receipt fields and correctly parsing FUNSD handwritten checkboxes.

The methodology caveat

Vendors grading their own homework is the live scandal in benchmark land right now. Independent audits have found self-reported scores run on average 6.4% higher than third-party reruns of the same suites 2, which is exactly why the industry is shifting to proctored, secret holdout sets 3.

Dharma’s scoring choice compounds the problem in a specific way: a multiplicative composite means a single malformed JSON output zeroes an otherwise-high F1. That penalty lands hardest on generalist models invoked through a prompt and barely touches a model explicitly fine-tuned to emit that schema. The 9.3-point gap over GPT-5.6 Luna is partly a gap in instruction-following under a rigid output contract, not necessarily in document understanding.

Practitioner commentary on the release also flags where Dharma didn’t test: handwriting under real conditions, official stamps, skewed or low-quality scans, and — the one that matters most for enterprise document processing — multi-page tables where schema consistency across page breaks is a known weakness of small VLMs 4.

Why the direction is still credible

Strip away the specific leaderboard and the thesis holds up independently. Particula reports 7B models fine-tuned on ~10k domain examples hitting 99.8% JSON validity on extraction tasks versus roughly 85% for Claude Opus and GPT-5 5. PaddleOCR-VL-1.6, at 0.9B parameters, reportedly beat GPT-5.2 on OmniDocBench v1.6 at 96.34% extraction accuracy 6. The pattern — specialized sub-10B models clearing frontier APIs on bounded structured tasks — is now attested across at least three independent efforts.

What actually matters

Even if the Claude Opus 5 head-to-head softens under an external rerun, the operational story is what changes procurement math. $2.24/hour on an L40S for unlimited volume, versus per-token pricing that scales linearly with document count, is a 6×–147× cost spread that holds regardless of whether the extractor is the best or merely competitive. Add sovereign deployment — sensitive contracts and bank statements never leaving your VPC — and the frontier API is no longer the obvious default for high-volume extraction pipelines.

The number to watch isn’t 0.834. It’s whether anyone outside Dharma reruns the suite on hard document classes 4 and reports back.


Datasette Agent 0.5a0 ships a side-panel app-builder stack

Source: simon-willison · published 2026-11-03

TL;DR

  • Datasette Agent 0.5a0 adds a persistent side panel that pins the chat next to any table or query page.
  • Combined with datasette-apps 0.3a0, a prompt spawns a database-backed app you can edit from the same panel.
  • Generated HTML runs in an iframe without allow-same-origin, with CSP default-src 'none' blocking outbound requests.
  • SQL-agent benchmarks still cap near 85% accuracy, with up to 66% failure on messy production schemas.

The panel is the pitch

Simon Willison dropped three coordinated releases over two days — datasette-agent 0.5a0, datasette-apps 0.3a0, and datasette-llm 0.1a10 — and the headline move is architectural, not a feature list. The agent now lives in a panel: a persistent side column that sits next to whatever Datasette page you’re on, so you can prompt an app into existence against the current table and then iterate on it without leaving the view 7.

That puts Datasette in a different category from the obvious comparisons. Claude Artifacts optimizes for the “magic trick” of rendering code as it types but rewrites the whole artifact for small edits; ChatGPT Canvas is closer to a shared editor than an app host. datasette-apps is pitched at durable, data-integrated internal tools that persist as rows, inherit Datasette’s permission model, and keep SQL as the write path 8. The bet is that an AI HUD pinned over a database beats a chat that emits disposable artifacts — when your primary artifact is a table.

Sandbox is the load-bearing wall

The security story is unusually concrete for an alpha. Generated HTML renders in an <iframe> with sandbox="allow-scripts allow-forms" — crucially, no allow-same-origin. An immutable CSP is injected via a <meta> tag inside srcdoc with default-src 'none', which disables outbound HTTP. Parent/child communication uses the Channel Messaging API rather than raw postMessage, so a child reload can’t hijack the port 9.

Writes are gated separately: the agent inherits the authenticated GitHub user’s permissions, and any write SQL or table creation requires explicit approval plus an execute-write-sql grant 10. This matters because letting LLM-generated HTML run same-origin against a session cookie is exactly the failure mode that has bitten most prompt-to-app tools.

flowchart LR
    U[User + GitHub auth] --> A[datasette-agent panel]
    A --> L[datasette-llm<br/>model routing]
    A --> AP[datasette-apps]
    AP --> IF[iframe sandbox<br/>no allow-same-origin<br/>CSP default-src 'none']
    AP -->|allow-listed<br/>stored queries| DB[(Datasette DB)]
    A -->|write SQL<br/>requires approval| DB
    IF <-->|MessageChannel| AP

Where the thesis is still contested

The panel doesn’t change the hard ceiling underneath it. Independent benchmarks put general-purpose LLMs around 85% accuracy on text-to-SQL, with failure rates up to 66% on real production schemas — and Willison’s own DSPy experiments surfaced schema-listing sparsity as a recurring cause of hallucinated column names 11. Competing architectures argue the agentic reflect-and-retry loop is the wrong abstraction: semantic-layer approaches like Wren AI and dbt reportedly cut response times from ~20s to under 7s by precompiling the question space, and MotherDuck is positioned as the scale-out answer for teams who find SQLite limiting 12.

The 0.5a0 release doesn’t try to push that ceiling. What it changes is who notices when the model guesses wrong — because the human is now sitting next to the query, in the panel, the whole time. The open UX question is whether the write-approval prompt holds up when a session generates dozens of them, or whether users start clicking through. For an alpha that ships a hardened sandbox and a coherent three-plugin stack in one week, that’s the right problem to be left with.

Further reading


Phonon TTS hits 28ms on-device, but cloning still phones home

Source: huggingface-blog · published 2026-11-05

TL;DR

  • Phonon’s 31M English model hits 28 ms CPU TTFA — 60× faster than Kokoro, 3 ms on an independent WebGPU port.
  • Voice cloning still round-trips Gradium’s cloud API — the Transformers integration ships without a Mimi encoder.
  • Small-model failures persist on PIN codes, dates, and acronyms like “GPUs,” despite dedicated linguistic-handling logic.
  • The bundled AudioSeal watermark has a known inversion flaw that flips authentic audio to “synthetic” at 75% error.

A real streaming-latency breakthrough

Gradium’s Phonon drops text-to-speech latency into territory where “streaming” starts to mean something. The 31M English checkpoint (65 MB, Apache 2.0) clocks a 95th-percentile time-to-first-audio of 28 ms on CPU; the 40M multilingual sibling hits 38 ms. For context, DataRoot Labs measured StyleTTS2-based competitors Kokoro and Kitten at >500 ms TTFA on the same MacBook Air M5 — roughly a 60× gap, driven by Phonon’s frame-by-frame flow-matching head versus chunk-at-a-time GAN vocoders 13.

The external signal that matters most isn’t Gradium’s benchmark table, though — it’s that a solo developer outran it. Anders Blixt’s independent WebGPU port hits 115× real-time on an M3 Max and a 3 ms first-audio latency, and exposes per-word timings pulled straight from one of the model’s attention heads, a feature Gradium’s official browser package doesn’t ship 14. That a weekend port beat the vendor runtime both corroborates the architectural claims and suggests the official numbers are conservative.

ModelTTFA (CPU)SizeApproach
Phonon 31M28 ms65 MBFlow matching, streaming
Phonon 40M38 ms75 MBFlow matching, streaming
Pocket TTS (predecessor)~165 ms—Flow matching
Kokoro / Kitten (StyleTTS2)>500 ms—GAN vocoder, chunked

”On-device” has an asterisk

The privacy framing needs qualifiers. A Medium teardown points out that while final synthesis runs locally, creating a cloned voice file still requires Gradium’s cloud API 15. The Hugging Face model card for the Transformers v5.21 integration confirms the structural reason: it ships without a Mimi encoder, so users can only consume precomputed 512-dim speaker latents hosted on the Gradium hub 16. Zero-shot cloning from a raw recording isn’t possible out of the box.

flowchart LR
    A[Raw voice recording] -->|cloud API| B[Gradium Mimi encoder]
    B --> C[512-dim speaker latent]
    C --> D{Phonon on device}
    E[Input text] --> D
    D --> F[Streaming audio, 28ms TTFA]

Combine that with the CC BY-NC license on the 40M multilingual model, and the clean “fully local, Apache 2.0” story applies only to the 31M English checkpoint paired with Gradium-issued voices.

Quality cracks and a watermark that bites back

Practitioners on Hacker News report the usual small-model tax: outputs that sound like “a warped cassette tape,” and consistent stumbles on structured strings — rupee amounts, PIN codes, dates, acronyms like “GPUs” — despite the linguistic-handling claims 17. There are no emotion or prosody controls either.

The bigger red flag is the casual mention of AudioSeal-style provenance. The recent “Watermark Shortcut” study found detectors trained on watermarked-synthetic vs. unwatermarked-real audio latch onto the mark itself as the “fake” signal; stamping an AudioSeal mark onto a genuine recording pushed mark-to-frame error rates from 16% to 75% 18. That inverts the intended guarantee: authentic audio gets flagged as synthetic, and strip attacks evade detection.

What’s actually new

Phonon earns the latency headline. A 28 ms CPU TTFA that an independent port can push to 3 ms on GPU is a genuine shift for on-device voice UX — offline assistants, in-browser narrators, synchronized word timings. The privacy and safety claims riding alongside it are doing more marketing work than the architecture supports.

Round-ups

MIT Tech Review maps 3 risk axes for agentic enterprises

Source: mit-tech-review-ai, mit-tech-review-ai, mit-tech-review-ai

Agent-to-agent handoffs now move data across restricted systems with no human in the loop, and a trio of pieces digs into the fallout: data lineage gaps, degraded analytics quality, and runaway token economics where one coding task burns 10-50x a chat prompt.

Simon Willison ships LLM 0.37.1 bugfix with 4 contributors

Source: simon-willison

The CLI tool for talking to language models gets a patch release pulling in fixes from Steven Huang, Tao An, Arjan Mossel, and XQ. No new features — purely stability work on top of the 0.37 line.

Footnotes

  1. Dharma-AI corporate site — https://dharma-ai.com.br/en/dharma-ai-2/

    Dharma-AI is a Rio de Janeiro spin-off from EloGroup pursuing a ‘specialization over scale’ strategy with 2B–7B SLMs, backed by selection into the AWS Global AI Accelerator and NVIDIA Inception

    ↩
  2. LayerLens — The Hidden Flaws of AI Benchmarks — https://layerlens.ai/reports/the-hidden-flaws-of-ai-benchmarks-why-the-industry-needs-a-reality-check

    self-reported scores are, on average, 6.4% higher than those obtained by third-party testers

    ↩
  3. Contrary Research — AI Benchmarks Climbing the Wrong Hills — https://research.contrary.com/report/ai-benchmarks-climbing-the-wrong-hills

    labs grade their own homework, creating a conflict of interest that the industry is now trying to replace with secret test sets and proctored examinations

    ↩
  4. Minerka — Dharma Field Extractor writeup — https://minerka.com/stories/dharma-field-extractor/

    evaluations should focus on hard cases — handwriting, official stamps, skewed or low-quality scans — and on multi-page tables where schema consistency across page breaks is a known challenge for small-parameter models

    ↩ ↩2
  5. Particula.tech — Why Specialized 7B Models Outperform GPT-5 — https://particula.tech/blog/why-specialized-7b-models-outperform-gpt5-production

    a 7B model fine-tuned on 10,000 domain-specific examples can achieve 99.8% JSON validity and over 96% classification accuracy, whereas general models like GPT-5 or Claude Opus may hover around 85% validity

    ↩
  6. local-ai-zone — Best AI OCR Models 2026 Ranking — https://local-ai-zone.github.io/guides/best-ai-ocr-models-ultimate-ranking-2026.html

    the 0.9B PaddleOCR-VL-1.6 has reportedly outperformed GPT-5.2 on OmniDocBench v1.6, achieving 96.34% extraction while running at a fraction of frontier compute cost

    ↩
  7. daily.dev summary of datasette-apps — https://daily.dev/posts/datasette-apps-host-custom-html-applications-inside-datasette-brgstofsz

    datasette-apps is a plugin designed to host custom HTML/JavaScript applications directly inside a Datasette instance, granting them unique read-write capabilities via SQL queries and stored canned queries… it provides a copy-paste prompt generator that feeds a database’s schema and sandbox rules into external LLMs to write the app code.

    ↩
  8. thewaynext.com — Claude Artifacts vs ChatGPT Canvas — https://thewaynext.com/en/blog/claude-artifacts-vs-chatgpt-canvas

    Claude Artifacts is frequently praised for its ‘magic trick’ immediacy — rendering apps as they are typed — but is criticized for performing full code rewrites for minor changes… datasette-apps is positioned as a tool for building durable, data-integrated internal tools.

    ↩
  9. agentpatterns.ai — Browser sandbox for agent-generated HTML — https://agentpatterns.ai/security/browser-sandbox-agent-generated-html/

    The core of the security model is a tightly constrained

    ↩
  10. GitHub datasette/datasette-agent (auth and permissions) — https://github.com/datasette/datasette-agent

    The agent inherits the permissions of the authenticated GitHub user. Executing write SQL or creating new tables requires explicit user approval and the execute-write-sql permission… 79% of developers grant AI agents access to cloud environments despite 97% expressing reservations about credential leaks.

    ↩
  11. n1n.ai — Exploring Datasette Agent for LLM data analysis — https://explore.n1n.ai/blog/exploring-datasette-agent-llm-data-analysis-2026-05-22

    Internal research using DSPy has identified that models sometimes guess column names incorrectly if the schema listing in the prompt is too sparse… benchmarks for SQL agents indicate a ceiling at roughly 85% accuracy, with failure rates surging as high as 66% in production schemas plagued by schema mix-ups.

    ↩
  12. colrows.com — Vanna.ai alternatives — https://colrows.com/blogs/vanna-ai-alternatives/

    Fine-tuning models or adding a semantic layer (like those in Wren AI or dbt) outperforms raw agentic reasoning by reducing response times from 20 seconds down to under 7 seconds… MotherDuck provides sub-second latency for analytical queries, positioning itself as the SQLite for Big Data for teams that find Datasette too limited for scale.

    ↩
  13. DataRoot Labs TTS comparison — https://datarootlabs.com/blog/text-to-speech-models

    Kokoro and Kitten TTS are StyleTTS2-based with GAN vocoders synthesising entire chunks; Kokoro’s first-audio latency is >500 ms versus Phonon’s ~7–8 ms frame-by-frame streaming — roughly a 60x gap on the same MacBook Air M5.

    ↩
  14. blixt/phonon GitHub (independent WebGPU port) — https://github.com/blixt/phonon

    115x real-time on M3 Max, first audio in as little as 3 ms, with live word-level timings extracted directly from one of the model’s attention heads — something the official Gradium browser package does not expose.

    ↩
  15. Medium teardown by jinlow (‘The four things you hand over…’) — https://jinlow.medium.com/the-four-things-you-hand-over-when-speech-moves-onto-the-device-a-teardown-of-gradium-phonon-498143431e23

    The final synthesis happens locally, but the creation of a cloned voice file still requires Gradium’s cloud API — so Phonon is not a 100% offline solution for developers who need local-only cloning.

    ↩
  16. FermionResearch/Phonon-1 model card (HF) — https://huggingface.co/FermionResearch/Phonon-1

    Users must supply precomputed 512-dim speaker latents from the Gradium hub; the Transformers v5.21 implementation ships without a Mimi encoder, so zero-shot cloning from a raw recording is not possible out of the box.

    ↩
  17. Hacker News discussion (item 50369215) — https://news.ycombinator.com/item?id=50369215

    Some users reported extreme distortion that sounded like ‘a warped cassette tape’ or ‘garbled words’, and the model consistently stumbles on rupee amounts, PIN codes, dates and acronyms like ‘GPUs’.

    ↩
  18. alphaXiv 2606.23335 — ‘Watermark Shortcut’ study — https://www.alphaxiv.org/abs/2606.23335

    Detectors trained on watermarked-synthetic vs unwatermarked-real audio latch onto the watermark itself as the ‘fake’ label; adding an AudioSeal mark to a genuine recording pushed error rates from 16% to 75% (‘mark-to-frame’).

    ↩
Jack Sun

Jack Sun, writing.

Engineer · Bay Area

Hands-on with agentic AI all day — building frameworks, reading what industry ships, occasionally writing them down.

Digest
All · AI Tech · AI Research · AI News
Writing
Essays
Elsewhere
Subscribe
All · AI Tech · AI Research · AI News · Essays

© 2026 Wei (Jack) Sun · jacksunwei.me Built on Astro · hosted on Cloudflare