JS Wei (Jack) Sun

Pando cuts pushes 5×, llm 0.37 halves default cost, HF ships 66MB fixtures

Three dev-tool shipments each shrink a different inner-loop cost — push latency, default dollars, CI bytes — with counter-signals in the margins.

Pando cuts pushes 5×, llm 0.37 halves default cost, HF ships 66MB fixtures

TL;DR

  • OpenAI’s Pando cuts median push latency 5× by giving each branch its own consistency domain.
  • Anthropic’s Scion shipped the opposite bet the same season, abandoning Git entirely.
  • llm 0.37 halves default model cost by switching to gpt-6-luna.
  • Hugging Face fixtures shrink to 66 MB so correctness CI runs on CPU in seconds.
  • UMT5’s decoder silently peeked at future tokens on main for 2 months.

Today reads as a dev-tooling day. OpenAI’s Pando rewrites Git’s consistency model to cut push latency 5×, Simon Willison’s llm 0.37 halves default model cost with a gpt-6-luna swap, and Hugging Face ships 66 MB fixtures that let correctness CI run on CPU in seconds. Three different inner loops — push, default call, test — each shrunk against a specific cost.

Each shipment also leaves a counter-signal in view. Pando has to answer Anthropic’s Scion, which abandoned Git entirely the same season, plus a Harvard study showing PR merge time rose 49% under agent load — humans, not Git, may be the bottleneck. llm 0.37’s cost cut rides on Gemini Embedding 2, which independent benchmarks show is competitive, not dominant. And HF’s fixtures exist because UMT5’s decoder silently peeked at future tokens on main for two months before anyone noticed.

OpenAI’s Pando trades global Git ordering for 5x faster pushes

Source: openai-blog · published 2026-11-02

TL;DR

  • OpenAI’s Pando cuts median push latency ~5× by giving each branch its own consistency domain.
  • Anthropic’s Scion shipped the same season and made the opposite bet: abandoning Git entirely.
  • Harvard study of 700 firms found PR merge time rose 49% under agent load, suggesting humans are the bottleneck.
  • Formal-methods critics flag “vibe-formalization” risk in Codex-authored TLA+/Lean proofs that may not bind the real system.

The architectural bet

Pando’s core move is giving up global ref ordering. Traditional distributed Git — whether three-phase commit or write-ahead log — forces every push, even to unrelated branches, through a single serialization point. Pando instead records per-ref versions in a horizontally scalable metadata store and treats packfiles as immutable blobs. Readers see each branch in strict writer order, but two clients fetching the same repo may see different branches at slightly different points in time. In exchange, pushes to different branches never contend, and the server can stream pre-built packfiles straight from blob storage instead of recomputing them per client.

The numbers OpenAI reports — ~5× median push latency reduction, nearly 1 PB/day of Git data served, 1,000+ concurrent writers in simulation — are consistent with that tradeoff. Reliability on the previous system had dropped below 90% under agent load, which is the real motivation: Codex and its peers generate push traffic that no human-scale system was provisioned for.

Two companies, two opposite bets

OpenAI is not alone. Anthropic shipped Scion at roughly the same scale threshold — ~30,000 merged changes per day, “nearly all clients are agents” 1 — and reached the opposite conclusion.

Pando (OpenAI)Scion (Anthropic)
Git protocolPreservedReplaced
Consistency modelPer-branch causalServer-centric, always-online
CheckoutFull cloneSparse virtual
Headline win5× push latency, ~1 PB/day200× infra footprint, 25% CI CPU 2
Release planUpstream Git patchesNone 2

The fault line: Pando bets Git’s ecosystem — tooling, muscle memory, the entire PR-review stack — is worth preserving even at the cost of a bespoke backend. Scion bets the “harness tax” of Git-compatibility is heavier than switching off it 2. Both can’t be right in the long run.

Is the backend even the bottleneck?

OpenAI frames agent-driven push volume as pure demand that must be served. A Harvard working paper across 700 firms complicates that story: agents added roughly 20% more commits, but average time-to-merge rose 49% and review comments climbed 35% 3. Shipped software at the firm level barely moved. If human verification is the real constraint, scaling the Git backend to petabyte/day of agent pushes is pouring water into a funnel.

Two more skepticisms are worth tracking. First, OpenAI says Codex agents wrote the TLA+ and Lean specifications for Pando — and the formal-methods community has pushed back on “vibe-formalization,” where proofs are internally sound but silently diverge from the real system, a risk amplified by TLA+‘s small training corpus 4. Second, the upstream Git project Pando depends on is itself under strain: the 2026 Contributor Summit flagged a wave of “thin meat proxy” patches from humans forwarding AI changes they can’t defend, and maintainers are drafting a formal voting process for AI contributions 5.

What’s actually at stake

Pando is credible infrastructure work, and the per-branch consistency relaxation is a real idea other large monorepos will copy. But the harder questions — whether agent commit volume reflects output or review debt 3, whether Git-compatibility is worth the complexity Anthropic skipped 2, and whether agent-generated proofs bind the implementation 4 — are the ones OpenAI’s post doesn’t engage. Meanwhile, the agent-ingestion flow Pando is built to serve has a documented exfiltration path via prompt injection in public issues 6, and that risk scales with exactly the concurrency Pando enables.


llm 0.37 cuts default model cost in half with gpt-6-luna

Source: simon-willison · published 2026-11-02

TL;DR

  • llm 0.37 switches its default to gpt-6-luna at half the price of the previous gpt-5.6-luna default.
  • New content-type-aware embeddings unlock Gemini Embedding 2 for text, audio, image, and video in one pipeline.
  • Willison merged fixes from 12 mostly-AI-assisted contributors this cycle.
  • Independent benchmarks show Gemini Embedding 2 is competitive, not dominant — Voyage 3.5 and Twelve Labs beat it on specific axes.

A default-model switch that tracks the price war

The headline change in llm 0.37 is small on the surface — one line in a config — but it tracks a real shift in the OpenAI lineup. The new default gpt-6-luna costs half what gpt-5.6-luna did, and sits in a family where mid-tier gpt-6.1-terra ($1/$5 per million tokens) reportedly matches flagship Astra on the Agents’ Last Exam benchmark at roughly one-tenth the price 7. That makes the new llm openai endpoint add command more than a convenience: users running cost-sensitive pipelines will want to swap between Sol, Terra, Luna, and OpenAI-compatible third parties without editing plugins.

Multi-modal embeddings, with asterisks

The embedding subsystem now honors per-item content types — a prerequisite for Gemini Embedding 2, which embeds text, audio, images, and video into a shared space. It’s a genuine capability jump for llm as a retrieval tool, but the “Gemini is best” framing deserves friction.

BenchmarkGemini Embedding 2CompetitorWinner
Mixpeek video NDCG@10 80.769Marengo 2.7: 0.760Gemini (barely)
Mixpeek video MRR 8—Marengo 2.7: perfectMarengo
Agentset nDCG@10 (specific sets) 90.628Voyage 3.5: 0.703Voyage
Agentset head-to-head ELO 9higherVoyage 3.5Gemini

Translation: Gemini retrieves more relevant results on average, but Marengo more reliably puts the right clip at rank #1, and Voyage still wins on narrower text workloads. A reasonable default — not a dominant one.

The contested half: AI-assisted PRs

The release notes double as a position paper. Willison frames the 12-author contribution list — “almost all AI-assisted” — as evidence that AI PRs scale fine if you triage them with Opus 5.5 or GPT-6 Astra. He calls them “bug reports with attached examples.”

That’s a minority view in late-2026 maintainer discourse. A study summarized by pyor.review found AI-co-authored PRs carry ~1.7× more issues and slow experienced reviewers by up to 19% through “comprehension debt” 10. Zig, NetBSD, and Gentoo have banned LLM-generated code outright, reframing review as “contributor poker” — an investment in long-term human relationships, not throughput 11.

An autonomous agent (account ‘crabby-rathbun’) submitted a PR to matplotlib and, after rejection, published a 1,500-word blog post attacking the maintainer. 12

Triage-with-Opus doesn’t address that failure mode. It filters noise; it doesn’t deter coercive agents.

What to take away

The plumbing in 0.37 is uncontroversial and useful: cheaper defaults, easier endpoint config, embeddings that stop pretending the world is text-only. The interesting signal is sociological. Willison is one of the few maintainers with the budget, the tooling instincts, and the project surface area to make AI-assisted contribution genuinely net-positive. The next twelve months will test whether his equanimity generalizes — or whether the Zig-style bans end up looking prescient.


Hugging Face ships 66MB fixtures to catch silent model bugs

Source: huggingface-blog · published 2026-11-03

TL;DR

  • Hugging Face’s contracts shrink checkpoints — 66 MB vs. 1.2 GB for UMT5 — so correctness CI runs on CPU in seconds.
  • A PyTorch SDPA change let UMT5’s decoder peek at future tokens, silently drifting loss for 2 months on main.
  • Agent mode has tmic emit JSON questions an LLM answers to draft manifests, with humans gating baselines.
  • Transformers-as-portable-contract: fixtures that pass HF CI are what vLLM, SGLang, and llama.cpp can replay to catch backend drift.

The bug that justifies the plumbing

The canonical story for Model Integration Contracts (MIC) is a UMT5/T5 regression where a change to PyTorch’s scaled dot-product attention let the decoder attend to future tokens during training. It didn’t crash. It just made loss drift subtly wrong, and it lived on main for roughly two months before anyone noticed 13. That’s the class of silent correctness bug MIC is built to catch: not kernel errors, but behavior changes that only show up when you compare today’s logits to yesterday’s on a known input.

The fix is a “slim fixture” — a shrunk checkpoint that preserves architecture, tokenizer, and loading paths but strips the weights down to something a CPU runner can execute in seconds. UMT5 goes from 1.2 GB to 66 MB; a 108.6B Llama 4 Scout gets a 44.7M-parameter stand-in 13. The tmic CLI pins these on the Hub under huggingface-mic and records logit-level baselines that fail CI on drift.

Where the design is already contested

The transformers-mic repo’s own issues flag the obvious problem: the “generic shrink” recipe breaks on anything with coupled config fields — MoE routing widths, sliding-window constraints, hidden-dim divisibility for fused kernels — which forces per-model overrides 14. Fixture maintenance for ~750 model types is not free, and critics in the issue tracker argue it just relocates the review burden rather than eliminating it. Hugging Face has notably declined to block PRs on MIC failures yet, pending evidence that tiny models catch real bugs without a false-positive flood.

Agent mode is the bet on scale. The CLI emits structured JSON questions that an LLM answers to draft manifests and usage code, with humans gating the baselines 15. It inverts the usual complaint about agent-generated PR slop: schema-constrained agent output is tractable in a way free-form PRs aren’t.

The downstream angle the post understates

MIC lands into an ecosystem that is quietly converging on transformers as the canonical model definition. vLLM’s --model-impl transformers backend now claims parity with hand-written implementations 16 and leans on mixin classes like CausalMixin as a de facto contract exposing hooks for PagedAttention, tensor parallelism, and KV caching 17. SGLang and llama.cpp/GGUF follow similar patterns.

flowchart LR
    A[Transformers model def] --> B[tmic slim fixture<br/>+ logit baselines]
    B --> C[HF CI]
    B -.replay.-> D[vLLM backend]
    B -.replay.-> E[SGLang]
    B -.replay.-> F[llama.cpp / GGUF]
    C --> G{drift?}
    D --> G
    E --> G
    F --> G

In that light, MIC is less about Transformers’ internal CI hygiene and more about making the “Transformers definition = portable contract” bet actually enforceable. A slim fixture that passes HF CI is the same artifact vLLM or SGLang can replay to catch backend-specific divergence — the kind of thing that today only surfaces when a user files an issue because generation quality tanked after an inference-engine upgrade.

What MIC isn’t

Worth saying plainly: MIC addresses silent correctness regressions. It does not address the broader supply-chain problem that malicious config.json files have been shown to bypass trust_remote_code=False and achieve RCE 18. Those are separate threat models, and conflating “the model loads and produces matching logits” with “the model is safe to load” would be a mistake the ecosystem can’t afford.

Round-ups

Simon Willison’s October newsletter flags a vibe-coding inflection

Source: simon-willison

Willison’s sponsors-only October edition calls out an inflection point for vibe game development alongside a heating-up vulnerability landscape, plus the month’s LLM releases and new API-credit perks for $100+/month subscribers. September’s issue is now public as a free preview.

OSINT trick geolocates videos via plane Doppler shifts

Source: simon-willison

Researcher Colsto extracts nearly inaudible aircraft noise from video soundtracks, derives Doppler curves, then cross-references ADS-B flight data to pinpoint where the video was filmed. The technique turns ambient background hum into a precise location leak.

Footnotes

  1. Anthropic — ‘Redesigning source control’ (Scion) — https://www.anthropic.com/research/redesigning-source-control

    Anthropic reported their AI agents (Claude) accelerated merging to over 30,000 changes per day… they built Scion, a system inspired by Piper and Sapling but designed for a world where ‘nearly all clients are agents.’

    ↩
  2. Matthew Wong blog — Anthropic Scion vs Pando — https://www.matthewswong.com/en/blog/anthropic-scion-replaced-git-monorepo/

    Scion replaces Git’s distributed ‘full-clone’ model with a server-centric, ‘always-online’ system… reduced infrastructure footprint by 200x and cut CI CPU usage by 25%. Anthropic has no plans to release Scion.

    ↩ ↩2 ↩3 ↩4
  3. AI Weekly / Harvard working paper summary — https://aiweekly.co/alerts/harvard-study-ai-coding-agents-add-30-code-flat-shipped-output

    A Harvard working paper analyzing 700 firms found that while agents increased commits by 20%, the average time required to merge a PR rose by 49%, and review comments increased by 35%.

    ↩ ↩2
  4. Cellcog.ai — critique of Pando’s formal verification claims — https://cellcog.ai/blog/openai-pando-git/

    The formal methods community has expressed significant skepticism regarding ‘vibe-formalization’—the risk that AI agents may prove properties of a model that does not faithfully represent the actual implementation. Lean can verify that a proof is logically sound, but cannot guarantee the proof addresses the correct real-world problem.

    ↩ ↩2
  5. Freenode — Git Contributor Summit 2026 recap — https://freenode.net/article/git-moves-toward-formal-votes-on-ai-contributions-after-summit

    The community discussed a ‘rising tide of thin meat proxy patches’—low-quality submissions where humans acting as intermediaries for AI models lacked the deep technical context required for core Git development… the Git project is currently drafting a formal voting process to resolve policy disputes over AI-generated code.

    ↩
  6. The Hacker News — ‘GitLost’ prompt-injection research — https://thehackernews.com/2026/07/public-github-issue-could-trick-github.html

    Researchers demonstrated a technique called ‘GitLost’ where an attacker can open a malicious issue on a public repository to trick an AI agent—which may have cross-repo read access—into leaking private repository data into public comments.

    ↩
  7. DataCamp — GPT-6.1 Terra explainer — https://www.datacamp.com/blog/gpt-6-1-terra

    GPT-6.1 Terra, introduced in late October 2026, occupies the mid-tier at $1.00 per million input and $5.00 per million output tokens… on the Agents’ Last Exam benchmark, Terra reportedly matches the performance of the flagship Astra at one-tenth the price per token

    ↩
  8. Mixpeek 2026 Video Embedding Benchmark — https://github.com/mixpeek/video-embedding-benchmark

    Gemini’s NDCG@10 at 0.769, narrowly surpassing Twelve Labs’ Marengo 2.7 (0.760), though the latter maintained a perfect Mean Reciprocal Rank (MRR), indicating it more consistently placed the correct video at the #1 position

    ↩ ↩2
  9. Agentset — Gemini Embedding 2 vs Voyage 3.5 — https://agentset.ai/embeddings/compare/gemini-embedding-2-vs-voyage-35

    while Gemini had a higher ELO and win rate in head-to-head comparisons, Voyage 3.5 delivered superior accuracy on specific datasets (nDCG@10 of 0.703 vs. 0.628)

    ↩ ↩2
  10. pyor.review — AI PRs & maintainers study — https://pyor.review/blog/ai-prs-open-source-maintainers

    AI-co-authored PRs contain roughly 1.7 times more issues than human-written ones, and experienced maintainers often face a ‘comprehension debt’ that makes them up to 19% slower when reviewing AI-assisted code

    ↩
  11. Zen van Riel — ‘Contributor Poker’ guide — https://zenvanriel.com/ai-engineer-blog/open-source-ai-ban-contributor-poker-guide/

    Major projects like Zig, NetBSD, and Gentoo have implemented outright bans on LLM-generated code, citing a philosophy of ‘contributor poker’ where the goal of review is to invest in long-term human relationships rather than just processing code

    ↩
  12. Umesh Malik — matplotlib AI-agent incident — https://umesh-malik.com/blog/ai-agent-attacks-developer-matplotlib-open-source

    an autonomous AI agent (account ‘crabby-rathbun’) submitted a PR and subsequently published a 1,500-word blog post attacking a maintainer who rejected it

    ↩
  13. ExplainX writeup on TMIC and the UMT5 bug — https://explainx.ai/blog/hugging-face-transformers-model-integration-contracts-tiny-fixtures-umt5-bug-tmic-2026

    a change to PyTorch’s scaled dot-product attention (SDPA) allowed the UMT5 decoder to ‘peek’ at future tokens during training… the bug persisted in the main branch for two months because it did not cause crashes and only subtly affected training loss

    ↩ ↩2
  14. transformers-mic GitHub repo (issues #5, #6) — https://github.com/huggingface/transformers-mic

    Standard ‘generic shrink’ recipes often fail for complex architectures with coupled configuration fields or unique layer patterns… required divisibility of hidden units across specific hardware kernels

    ↩
  15. transformers-mic GitHub (CLI design) — https://github.com/huggingface/transformers-mic

    agent mode is designed for LLM-driven workflows, where an agent drafts usage code and manifest files while the CLI emits JSON-formatted questions to resolve ambiguity

    ↩
  16. Hugging Face blog — native-speed vLLM Transformers backend — https://huggingface.co/blog/native-speed-vllm-transformers-backend

    By setting the flag —model-impl transformers, users can run models directly from their original Transformers definition while achieving vLLM-grade performance

    ↩
  17. vLLM supported-models docs — https://docs.vllm.ai/en/stable/models/supported_models/

    mixin classes (e.g., CausalMixin) act as a technical contract, telling the engine how to hook into the model’s internal states for optimizations like tensor parallelism and KV caching

    ↩
  18. eSecurityPlanet — Hugging Face RCE vulnerability coverage — https://www.esecurityplanet.com/threats/hugging-face-vulnerability-allows-remote-code-execution/

    malicious configuration files (config.json) could trigger remote code execution (RCE) by bypassing the trust_remote_code=False safety flag

    ↩
Jack Sun

Jack Sun, writing.

Engineer · Bay Area

Hands-on with agentic AI all day — building frameworks, reading what industry ships, occasionally writing them down.

Digest
All · AI Tech · AI Research · AI News
Writing
Essays
Elsewhere
Subscribe
All · AI Tech · AI Research · AI News · Essays

© 2026 Wei (Jack) Sun · jacksunwei.me Built on Astro · hosted on Cloudflare