Cloudflare Python Workers GA, HF wires GGUF Metal, tokenizers v1 hits 15.9x

TL;DR

• Cloudflare Python Workers hit GA with 1s cold starts via deploy-time Wasm memory snapshots.
• Workers’ Wasm VM breaks threading and multiprocessing, forcing async-only concurrency.
• Hugging Face transformers loads GGUF quants on Apple Silicon via ggml Metal kernels.
• GGUF fast path covers only Qwen3.5 and Qwen3.8 checkpoints today.
• Tokenizers v1 ships 15.9x faster encoding with bit-for-bit v0.23 parity.

Today’s tech beat is Python-runtime plumbing, three ships deep. Cloudflare takes Python Workers to GA after a two-year preview, cutting cold starts from ~10s to ~1s by snapshotting Wasm linear memory at deploy — at the price of a sandbox where threading is gone and the patched network stack quietly diverges from CPython. Hugging Face wires ggml’s Metal kernels into transformers so from_pretrained loads GGUF quants directly on Apple Silicon, though the fast path today covers only two Qwen checkpoints and the default loader still dequantizes to FP32/BF16.

The clean release is tokenizers v1: 15.9x faster single-threaded encoding, p99 latency collapsing from 145.9 µs to 2.29 µs on DeepSeek-v4, and bit-for-bit parity with v0.23 as the correctness promise. The round-ups add a Multiverse paper reframing block pruning as an Ising problem and Willison’s defense of MCP as the plumbing non-YOLO agent deployments actually need.

Cloudflare’s Python Workers hit GA with a 1s cold-start floor

Source: simon-willison · published 2026-09-21

TL;DR

• Python Workers are GA after a two-year preview, running CPython compiled to WebAssembly via Pyodide inside the V8-based workerd runtime.
• Cold starts land around 1 second, down from ~10s, using a deploy-time snapshot of Wasm linear memory after top-level imports.
• threading and multiprocessing don’t work inside the WebAssembly VM, forcing async-only concurrency patterns.
• Cloudflare’s patched network stack makes urllib3 redirects and TLS defaults silently diverge from CPython semantics.
• Cross-language RPC now lets TypeScript Workers call Python Workers with automatic FFI conversion.

The stack under the hood

The architecture is worth spelling out because every limitation flows from it. Python code is compiled to WebAssembly via Pyodide, executed inside V8, which is embedded in Cloudflare’s open-source workerd runtime — the same 123MB binary that pywrangler ships locally for development.

flowchart LR
    A[Your Python code] --> B[Pyodide<br/>CPython→Wasm]
    B --> C[V8 engine]
    C --> D[workerd runtime]
    D --> E[KV / D1 / Queues<br/>via direct bindings]
    D <-. RPC + FFI .-> F[TypeScript Workers]

Because Python and JS share one execution context, calls to KV, D1, or Queues avoid IPC overhead — and now TypeScript Workers can invoke Python Workers directly through schema-less RPC, with Pyodide’s FFI handling type conversion ¹. That’s the setup Cloudflare wants you to build: TS at the edge for I/O, Python for LangChain and model glue.

Cold starts aren’t free

Cloudflare’s marketing leans on Workers’ famous 0ms cold starts, but Python doesn’t inherit that. The team snapshots Pyodide’s Wasm linear memory after top-level imports at deploy time, cutting cold starts from roughly 10 seconds to about 1 second ². That’s roughly 2.4× faster than a vanilla AWS Lambda Python cold start, but only marginal against Lambda SnapStart’s ~855ms — and Wasmer argues 60ms is architecturally feasible, calling Cloudflare’s ~1s floor a real ceiling for latency-sensitive edge workloads ³. The snapshot also pins you to whichever Pyodide version is baked into workerd, which matters the moment you need a NumPy version that isn’t shipped.

Leaky abstractions

Simon flags threading and multiprocessing as broken, and that’s true, but it undersells the problem. Cloudflare patches Python’s network stack to tunnel through the JavaScript fetch API, and Python’s async event loop is bound to V8’s ⁴. The practical consequence: urllib3 redirect handling and TLS defaults can silently differ from CPython, and async code that behaves lazily on your laptop can behave eagerly in production ⁴. Test suites that pass locally will still surprise you in prod. This is the class of bug that costs a weekend.

Add the resource envelope: 64 MiB compressed bundle size, 128 MiB RAM per isolate, billed on CPU-ms — not wall-clock — at $0.02 per million ⁵. CPU-ms billing genuinely favors I/O-bound Python (API gateways, DB proxies), but 128 MiB rules out any serious Pandas or NumPy work, which is awkward given who Python attracts.

Where it actually wins

The competitive picture is diverging. Deno Deploy is betting on the WebAssembly Component Model with componentize-py for microsecond instantiation; Vercel’s Fluid Compute keeps a full Linux Python and pauses billing during I/O ⁶. Cloudflare’s embedded-Pyodide bet trades cold-start floor and memory ceiling for two things the others don’t have: sub-millisecond cross-language RPC into TypeScript, and CPU-ms billing that makes idle Python nearly free.

If you’re building an AI agent that spends 90% of its life waiting on model APIs, that trade looks great. If you’re running scientific Python, it isn’t for you — and the announcement post doesn’t quite say so out loud.

---

Hugging Face tokenizers v1 hits 15.9x faster encode

Source: huggingface-blog · published 2026-09-21

TL;DR

• Hugging Face tokenizers v1 ships 15.9x faster single-threaded encoding vs. v0.23, with 18x on BPE.
• p99 latency collapses from 145.9 µs to 2.29 µs on DeepSeek-v4 — a ~64x single-encode win.
• Bit-for-bit parity with v0.23 IDs, API, and vocab is the release’s central correctness promise.
• HF closes the gap to tiktoken and TokenDagger without dropping training or WordPiece/Unigram support.

The numbers that matter

Hugging Face rewrote the guts of tokenizers and the throughput chart is genuinely striking: 139.6 MB/s single-threaded BPE encode, 76% linear scaling across 8 Apple M4 Max cores, a 2.8x smaller runtime heap (~19.1 MB for GPT-OSS), and a 4.1x smaller binary from splitting the library into tk-encode, tk-serialize, and tk-train crates. Non-Latin scripts benefit disproportionately — Amharic 14.6x, Hindi 12.5x — because the old regex-based pre-tokenizer was doing more work on multi-byte codepoints than anyone noticed.

The framing HF picked — “CPU-maxxing” to lower Time to First Token — is the right one. vLLM V1, which leans on Rust-backed tokenization, reports up to 5x latency and 1.7x throughput gains over V0, explicitly crediting faster tokenization for letting the engine hit GPU-bound KV prefill sooner ⁷. Sub-microsecond p99 encode isn’t a vanity metric at high concurrency; it’s the difference between a saturated H100 and one waiting on Python.

What actually changed

Three architectural bets carry most of the win:

1. Bitstream splitting replaces regex-based segmentation, eliminating a hot allocation path.
2. A shared word cache short-circuits repeat lookups on common strings — huge for chat traffic where system prompts recur.
3. A double-array trie with a zero-allocation encode path rebuilds WordPiece from scratch.

This fits a broader ecosystem move from quadratic-worst-case BPE to linear-time O(n) implementations. Even Karpathy’s educational minbpe has seen 55–108x speedups in optimized ports ⁸, and rs-bpe, BlockBPE, and quicktok are all pushing similar ideas.

Where HF still trails the specialists

The headline number needs one asterisk. Community benchmarks consistently note that raw tokenizer speed depends heavily on vocabulary size and merge count, not just engine quality — a smaller vocab can look “faster” regardless of implementation ⁹. Controlled comparisons still put tiktoken ahead on pure BPE, and TokenDagger claims 4x over tiktoken on code and 2x on >1GB NLP files by JIT-compiling PCRE2 and skipping regex for special tokens entirely ¹⁰.

So v1 isn’t a new absolute ceiling on BPE throughput. It’s HF closing a long-standing gap while keeping the full multi-algorithm API (BPE + WordPiece + Unigram) and the training pipeline that specialized encoders don’t offer. That’s a defensible position — most teams don’t want to swap engines when they move from Llama to BERT to T5.

Caveats worth auditing

Bit-for-bit parity with v0.23 is the release’s central correctness claim, but the HF forums already document special tokens being unexpectedly split, and community reports mention rare (~1 in 1000) “gibberish” decode outputs in the fast path ¹¹. For high-volume serving, 1-in-1000 is not a rounding error.

There’s also a subtler concern: recent work shows tokenizer behavior can leak training-set membership ¹². Aggressive caching and bitstream splitting change observable timing, which is exactly the kind of surface that side-channel researchers pick apart. Worth watching as v1 lands in production stacks.

---

Transformers adds GGUF quant support via ggml Metal kernels

Source: huggingface-blog · published 2026-09-22

TL;DR

• Hugging Face wired ggml’s Metal kernels into transformers, so from_pretrained now loads GGUF quants directly on Apple Silicon.
• Throughput matches llama-bench for 128-token generation on an M2 Max — but only for Qwen3.5 and Qwen3.8 today.
• The default loader still dequantizes GGUF blocks to FP32/BF16, so advertised RAM savings only appear on the Metal fast path.
• Real unlock is Python hackability — PyTorch hooks, MoE expert tricks, and LoRA on quantized checkpoints without leaving the HF stack.

The news under the news

The headline is that transformers can now run llama.cpp-format quants on a Mac with the same one-liner you use for any other model. The subtext is that Hugging Face bought ggml.ai — Georgi Gerganov’s team — in February 2026, with the projects staying MIT-licensed and technically autonomous ¹³. This integration is the first visible artifact of that deal: rather than reimplementing quantized inference, transformers reuses ggml’s Metal kernels through the kernels library. Frame it as a corporate consolidation with an engineering payoff, not a spontaneous handshake.

Performance: competitive, with an asterisk

Hugging Face reports throughput on par with llama-bench on a MacBook Pro M2 Max for 128-token generations. That framing survives scrutiny for short-decode workloads on Qwen3.5/3.8, but it’s narrower than it sounds. Independent numbers on an M4 Max put llama.cpp at ~85 tok/s and MLX at ~88 tok/s on Llama 3.2 3B Q4_K_M ¹⁴, and llama.cpp with Flash Attention pulls ahead by up to 50% on prefill and long-context runs ¹⁵. For agentic workloads that chew through 20k-token prompts, transformers is still the slower path.

Two loop-level wins do generalize beyond GGUF: generate now skips attention masks for unpadded inputs and defers stopping-criteria copies asynchronously so the CPU can queue the next GPU op. Those changes speed up every model in the library, not just quantized ones.

Caveats worth reading twice

Two footnotes deserve top billing. First, the default GGUF loader is still the legacy path that dequantizes blocks to FP32/BF16 at load time to stay compatible with vanilla PyTorch ops ¹⁶. The advertised memory footprints (2.74 GB for Qwen3.5-4B at Q4_K_M) only materialize when you’re on Metal with the kernel path active — CUDA and CPU users get the file format, not the compression.

Second, the Metal SDPA kernel matches PyTorch reference outputs bit-for-bit only about 97% of the time, thanks to op reordering ¹⁷. That’s fine for chat. It’s not fine for evaluation harnesses, regression suites, or anyone diffing logits against a reference implementation.

Where this actually wins

The compelling pitch isn’t tokens per second — llama.cpp and MLX still own that leaderboard. It’s the Python surface on top of quantized weights. MoE research is the clearest example: hobbyists on r/LocalLLaMA have shown that expanding the active expert count in late transformer layers drops latency 10.9% and reasoning-token count 8.5% with no accuracy loss ¹⁸. That kind of forward-pass surgery is trivial in PyTorch and miserable in C++.

Pair that with GgufConfig(dequantize=True), which turns any GGUF checkpoint into a standard training base for LoRA or QLoRA, and transformers has claimed a niche llama.cpp was never going to serve: the hackable local model. Raw throughput isn’t the fight Hugging Face is picking here.

Round-ups

Willison defends MCP against ‘bad idea’ critique

Source: simon-willison

MCP looks pointless if you hand a terminal agent unrestricted API access, Willison concedes, but it earns its keep the moment you want scoped service access, key isolation from the model, user-facing auth flows, and audit logs — the plumbing non-YOLO deployments actually need.

Multiverse frames LLM block pruning as an Ising problem

Source: huggingface-blog

Researchers recast transformer block removal as an Ising optimization, borrowing statistical-physics methods to decide which layers to cut. The approach treats each block as a spin variable, letting solvers find pruning configurations that preserve accuracy better than greedy layer-drop heuristics used in prior compression work.

Footnotes

1. Cloudflare blog — cross-language RPC — https://blog.cloudflare.com/python-workers-advancements/

   TypeScript and Python Workers can now call each other directly via schema-less RPC, with Pyodide’s FFI handling type conversion — enabling hybrid deployments where Python handles AI/LangChain logic and TS handles I/O.

   ↩

2. Cloudflare blog — Python Workers advancements — https://blog.cloudflare.com/python-workers-advancements/

   Cold starts drop from roughly 10 seconds to about 1 second by snapshotting WebAssembly linear memory after top-level imports at deploy time.

   ↩

3. Wasmer blog — Python on the Edge — https://wasmer.io/posts/python-on-the-edge-powered-by-webassembly

   Cold starts as low as 60ms are technically feasible; Cloudflare’s ~1s Python cold start remains a significant barrier for latency-sensitive edge workloads, and users are locked to the specific Pyodide version embedded in workerd.

   ↩

4. explainX — Python Workers GA analysis — https://explainx.ai/blog/cloudflare-python-workers-ga-quick-tunnels-2026

   Cloudflare patches the network stack to route through the JavaScript fetch API and ties Python’s async loop to the JS event loop; libraries like urllib3 exhibit redirect and security behaviors that diverge from CPython semantics.

   ↩ ↩²

5. Cloudflare Workers pricing docs — https://developers.cloudflare.com/workers/platform/pricing/

   Worker size limit is 64 MiB compressed on both Free and Paid plans; each isolate is capped at 128 MiB RAM, and CPU-ms (not wall-clock) is billed at $0.02 per million.

   ↩

6. TechBytes — Edge WASM CPython guide 2026 — https://techbytes.app/posts/edge-computing-wasm-cpython-wasm-guide-2026/

   Python 3.13 reached Tier-2 support for wasm32-wasi and Deno Deploy uses the WebAssembly Component Model (componentize-py) to achieve microsecond-level instantiation — a different bet than Cloudflare’s embedded-Pyodide approach.

   ↩

7. vLLM optimization documentation — https://docs.vllm.ai/en/stable/configuration/optimization/

   vLLM V1 reports up to a 5x improvement in latency and a 1.7x boost in total throughput over V0, with faster tokenization letting the engine transition sooner to GPU-bound KV cache generation and directly lowering TTFT.

   ↩

8. arXiv 2604.11518 — BPE algorithmic analysis — https://arxiv.org/html/2604.11518v1

   Newer Rust implementations achieve linear-time O(n) BPE encoding versus the quadratic worst-case behavior of some legacy implementations; ports of minbpe using PyTorch+CUDA report 55x–108x speedups over the original code.

   ↩

9. MachineLearningPlus — tiktoken vs Hugging Face tokenizers — https://machinelearningplus.com/gen-ai/tiktoken-vs-huggingface-tokenizers/

   For pure encoding tasks, tiktoken historically outperformed older open-source libraries by 3–6 times… the choice of vocabulary often impacts performance more than the library itself; a vocabulary with fewer merges to apply can appear ‘faster’ regardless of the engine.

   ↩

10. Hacker News — TokenDagger discussion (item 44422480) — https://news.ycombinator.com/item?id=44422480

    TokenDagger… reports a 4x speedup on code sample tokenization and 2x throughput on >1GB NLP files versus tiktoken, using a JIT-compiled PCRE2 engine and bypassing regex for special tokens entirely.

    ↩

11. Hugging Face discuss forum — ‘Tokenizer is splitting special token’ — https://discuss.huggingface.co/t/tokenizer-is-splitting-special-token/161147

    Users report edge cases where special tokens are unexpectedly split by the tokenizer, and rare (~1 in 1000) ‘gibberish’ decode outputs suggest edge-case stability in the decoding path remains under active development.

    ↩

12. arXiv 2507.11941 — tokenizer membership inference research — https://arxiv.org/html/2507.11941v1

    Tokenizers can be exploited for membership inference attacks: the specific way a tokenizer handles text can reveal whether that data was included in the model’s original training set.

    ↩

13. shartech.cloud — ggml.ai / Hugging Face partnership — https://blog.shartech.cloud/ggml-ai-hugging-face-partnership/

    In February 2026, Hugging Face officially acquired ggml.ai, the founding team behind the pervasive llama.cpp and whisper.cpp libraries… the projects would remain under the MIT license and that the original team would maintain full technical autonomy

    ↩

14. local-llm.net (llama.cpp vs MLX comparison) — https://www.local-llm.net/compare/llama-cpp-vs-mlx/

    llama.cpp achieved roughly 85 tok/s on Llama 3.2 3B (Q4_K_M), while Apple’s MLX reached 88 tok/s [on M4 Max, 64GB]

    ↩

15. GitHub discussion, ik_llama.cpp — https://github.com/ikawrakow/ik_llama.cpp/discussions/164

    llama.cpp frequently outperforms MLX in prompt processing (prefill) and long-context scenarios by up to 50% when Flash Attention is enabled

    ↩

16. Hugging Face docs — Quantization/GGUF — https://huggingface.co/docs/transformers/en/quantization/gguf

    By default, the library employs a legacy loader that dequantizes GGUF blocks into full-precision (FP32 or BF16) tensors during the loading phase to ensure compatibility with standard PyTorch operations

    ↩

17. Hugging Face docs — Metal quantization backend — https://huggingface.co/docs/transformers/en/quantization/metal

    certain Metal kernels, such as the Scaled Dot Product Attention (SDPA) implementation, do not guarantee bit-for-bit reproducibility; they match standard PyTorch outputs only about 97% of the time due to differences in operation reordering

    ↩

18. r/LocalLLaMA — Qwen3 MoE expert-count hacking — https://www.reddit.com/r/LocalLLaMA/comments/1w6lk6z/increasing_active_parameters_per_token_in_moe/

    expanding the active expert count… in the later transformer layers reported an 8.5% reduction in reasoning tokens and a 10.9% drop in latency without a loss in accuracy

    ↩