HF Jobs runs vLLM per-minute, Ai2 ships Olmo diagnostic, Datasette plugin unpins

TL;DR

• Hugging Face Jobs runs OpenAI-compatible vLLM endpoints per-minute at ~$1.50/hr on A10G Large.
• vLLM’s --api-key protects only /v1 routes, leaving diagnostic paths unauthenticated by default.
• Ai2’s Olmo Hybrid beats a same-recipe transformer by 0.04 loss on content words.
• Olmo’s real win is the filtered-token-loss diagnostic, not the hybrid architecture itself.
• datasette-export-database 0.3a2 loosens its pin to >=1.0a27 after core hit 1.0a35.

Today’s three AI-tech ships sit at different layers of the stack with no shared thread. Hugging Face Jobs now launches an OpenAI-compatible vLLM endpoint as a one-liner billed per-minute — a practical serving primitive with a couple of cost and auth gotchas worth knowing before you point traffic at it. Ai2’s Olmo Hybrid posts a 0.04 loss edge over a same-recipe transformer on content-word prediction, ties on verbatim copying, and quietly contributes a filtered token loss diagnostic that’s more useful than the architecture verdict itself. And in plugin-land, datasette-export-database swaps an exact ==1.0a27 pin for >=1.0a27, unblocking installs on every newer Datasette alpha — a small fix with a real lesson about pinning against pre-1.0 cores.

Hugging Face Jobs runs vLLM endpoints per-minute

Source: huggingface-blog · published 2026-06-26

TL;DR

• hf jobs run launches an OpenAI-compatible vLLM server on HF infrastructure, billed per-minute (≈$1.50/hr on A10G Large).
• vLLM’s --api-key flag only protects /v1 routes, leaving diagnostic and monitoring paths unauthenticated by default.
• The post’s --tensor-parallel-size = GPU count rule can lower throughput once NCCL all-reduce saturates the interconnect on PCIe nodes.
• HF’s H200 tier runs ≈$5/hr per GPU vs. $1.40–$3.80 on specialized clouds — a 2–3× gap worth $3–5k/month for steady workloads.

The one-liner is real

Hugging Face’s recipe genuinely collapses vLLM deployment to a single hf jobs run invocation against the official vllm/vllm-openai image. The job proxy issues a private URL gated by HF tokens, billing kicks in per minute against a payment method, and hf jobs cancel stops the meter immediately. The walkthrough scales from Qwen3-4B on an a10g-large flavor up to Qwen3.5-122B-A10B on dual H200s, with the usual vLLM knobs — --max-model-len, --max-num-seqs, reasoning parsers, tool calling — exposed as CLI flags. For one-off evals, batch generation, and throwaway agent harnesses, this is the lowest-friction path on the market.

It’s also the part of the story HF tells well. The interesting questions sit in what the post leaves out.

Security defaults are weaker than the recipe suggests

The blog treats --expose 8000 plus HF tokens as sufficient. Independent security analysis disagrees: vLLM’s --api-key only authenticates specific /v1 endpoints, leaving diagnostic, metrics, and (historically) video-processing paths reachable, and CVE-2026-22778 turned one such unauthenticated endpoint into a CVSS-9.8 remote code execution ¹. Thousands of exposed vLLM ZeroMQ sockets have been catalogued on the public internet, suggesting the “one command” pattern routinely ships without a reverse proxy ¹.

On HF Jobs the proxy is your only line of defense. If you treat the resulting URL as “private because it has a token,” you’ve already lost the threat model:

flowchart LR
    A[HF token holder] -->|authenticated| P{HF job proxy}
    B[Anyone with URL] -->|/metrics, /debug| P
    P --> V[vLLM container]
    V --> M[/v1/chat - api-key gated/]
    V --> U[/diagnostic & monitoring - open/]
    V --> X[/video endpoints - CVE-2026-22778/]

Fine for ephemeral evals. Not fine for anything internet-reachable that outlives the experiment.

--tensor-parallel-size = GPU count is a starting point, not an answer

The article tells readers to match --tensor-parallel-size to hardware. Production benchmarks complicate that: on Llama-class models, TP=8 on a single node can lower throughput vs. TP=4 once NCCL all-reduce saturates the interconnect, and on PCIe boxes without NVLink the penalty is severe ². Data-parallel replicas frequently beat naive TP scaling under high concurrency. Anyone reaching for the recommended h200x2 to host Qwen3.5-122B should benchmark TP=2 against two TP=1 replicas before locking in.

The pricing column the post skips

HF positions H200x2 as the on-ramp for large MoEs but doesn’t compare hourly rates. Specialized clouds list on-demand H200 at $1.40–$3.80/hr, against HF’s ≈$5/hr per GPU — a gap that compounds to $3–5k/month on steady workloads ³. The H200’s 141GB HBM3e is what justifies any H200 tier over H100 for long-context or large-batch inference, with ~2× speedups on memory-bound shapes ⁴; HF doesn’t help you decide when that premium pays off versus an A10G or a cheaper neocloud.

Where it actually fits

The same hf CLI was recently redesigned as “agent-optimized,” emitting structured next-command hints to Claude Code and Codex drivers and reportedly cutting agent token usage ~6× ⁵. Read that way, hf jobs run vllm/vllm-openai is less a deployment story than a primitive for agents spinning up ephemeral compute. That is exactly where it belongs — and exactly where Inference Endpoints, with multi-minute cold starts on scale-to-zero, do not ⁶.

---

Olmo Hybrid wins content words, ties on copy tasks

Source: huggingface-blog · published 2026-06-25

TL;DR

• Ai2’s Olmo Hybrid beats a same-recipe Olmo 3 transformer by 0.04 loss on content words.
• That edge collapses to zero on verbatim copying and closing brackets — tasks that reward exact token lookup.
• Fixed-state recurrence provably cannot copy past a length set by its state size — prior theory predicts the ceiling.
• Ai2’s real contribution is the “filtered token loss” diagnostic, not the architecture itself.

The loss-gap diagnostic

Ai2 trained two 7B models on identical data with identical recipes: Olmo 3, a pure transformer, and Olmo Hybrid, which swaps most attention layers for recurrent ones (Gated DeltaNet) while keeping a few attention layers intact. Then they did something most architecture comparisons skip — instead of reporting a single validation loss, they computed the per-token loss gap across linguistic categories.

The pattern is sharp. On content words — meaning-bearing nouns, verbs, adjectives — the hybrid leads by roughly 0.04 nats. On function words (“the,” “of,” “is”), the lead shrinks to about 0.02. On verbatim n-gram copying, the gap trends to zero as the repeated segment grows. On closing braces in Python and HTML, it vanishes entirely: simple attention is already sufficient for bracket matching.

The framing Ai2 lands on is clean: attention is for lookup, recurrence is for state-tracking. Bolted together, you get a model that earns its parameters differently in different positions of the same sequence.

Why the split is theoretically grounded

This isn’t a quirk of Olmo’s training run. Jelassi et al.’s “Repeat After Me” proved formally that any fixed-state recurrence must fail at verbatim copying once sequence length exceeds a bound proportional to its state size — while a two-layer transformer can copy strings of exponential length ⁷. The flat gap on long repeated n-grams is that ceiling showing up in the loss.

The state-tracking side has equally specific theory behind it. Grazzi and Siems et al. showed that linear RNNs only acquire genuine finite-state-automaton expressivity once their transition eigenvalues are allowed to include negative values ⁸. Olmo Hybrid’s choice of Gated DeltaNet — which supports negative eigenvalues — over vanilla Mamba is the mechanism that makes the content-word advantage possible in the first place.

The systems literature corroborates the same split at the deployment level. Microsoft’s Samba patches the retrieval hole by pairing Mamba with sliding-window attention, reporting perfect passkey recall to 1M tokens ⁹. AI21’s Jamba runs a 1:7 attention-to-Mamba ratio and claims 3× Mixtral throughput at 256K context ¹⁰. Ai2’s 3:1 GDN-to-attention recipe is in-family with both.

What the chart hides

Two caveats temper the story. First, pre-training loss — even filtered — is a contested proxy for downstream capability. Models with identical validation losses can diverge sharply on reasoning, because aggregate loss is dominated by easy frequent tokens ¹¹. A 0.04-nat content-word gap is small in absolute terms, and Ai2 doesn’t yet translate it to user-visible behavior.

Second, hybrid stacks buy their efficiency with real operational debt. Practitioners report that recurrent hidden states are numerically sensitive enough to require FP32 precision and to break common attention optimizations like cascade attention, with RL post-training prone to reward collapse otherwise ¹². None of that appears on a loss-gap chart.

The takeaway worth keeping: average loss is a blunt instrument, and “which tokens does the architecture earn?” is the right question for the next generation of hybrid recipes. Whether 0.04 nats on content words shows up in the chatbot is the question Ai2 hasn’t answered yet.

---

Datasette plugin loosens version pin to survive 1.0 alphas

Source: simon-willison · published 2026-06-25

TL;DR

• datasette-export-database 0.3a2 swaps a datasette==1.0a27 exact pin for >=1.0a27, unblocking installs on every newer alpha.
• Datasette core has already reached 1.0a35, so the old pin silently excluded every current install.
• The plugin uses SQLite’s Online Backup API for byte-for-byte snapshots of a live database, gated by a dedicated export-database permission.
• Commentators frame it as a lighter alternative to Litestream — manual exports, no continuous replication, no crash recovery between them.

The one-character fix

Willison calls it “embarrassingly tiny,” and it is: == became >= in one line of pyproject.toml. But the slip mattered because Datasette 1.0 is a moving target. The core hit 1.0a35 in late June 2026, picking up a SQL-powered permissions system and a browser-based table editor along the way ¹³. Any plugin pinned to ==1.0a27 would either refuse to install or force pip to downgrade Datasette itself — breaking every other plugin in the environment.

The fix also brings the package back in line with standard Python packaging guidance. Exact pins inside a library are widely treated as an anti-pattern because they “prevent users from receiving critical security patches or bug fixes from dependencies without a new release of the library itself” ¹⁴. Libraries declare ranges; applications freeze them in lockfiles. The original ==1.0a27 was a slip, not a stance.

What the plugin actually does

Strip away the release-note minimalism and the plugin is doing something genuinely useful. It wraps SQLite’s Online Backup API to produce a “byte-for-byte consistent snapshot of the database while it remains fully operational and online” ¹⁵. In-memory databases are explicitly excluded, since the file-copy path doesn’t apply ¹⁵. Access is gated through a specific export-database permission rather than piggybacking on root, so an operator can grant snapshot rights without handing out the keys to everything else ¹⁶.

That’s a real gap-filler for the growing class of Datasette deployments that accept writes — CSV uploads, custom internal apps, dashboards that mutate state — and need a way to grab a coherent backup without taking the site down.

Where it stops, and where Litestream starts

The plugin is on-demand and human-triggered. That’s the whole point, and also the whole limit. Independent coverage positions it as “a lighter alternative to tools like Litestream, which provides continuous, real-time replication to cloud storage,” and notes plainly that it “does not solve the problem of server crashes or accidental DELETE commands that happen between manual exports” ¹⁷.

For production CRUD workloads, this plugin complements those options; it doesn’t replace them.

The argument underneath

The fact that an export-for-mutable-DB plugin exists at all reflects Datasette’s drift away from its original “baked, read-only data” posture. Not everyone is happy about it — some developers argue the read-only default was itself a safety feature for public data publishing, and the road to writable Datasette has required successive CSRF and permission reworks across the 1.0 alphas ¹⁸.

The 0.3a2 release is a packaging fix. The interesting question it surfaces is whether Datasette’s 1.0, when it finally lands, will be remembered as the read-only publishing tool that grew up, or as a different product wearing the same name.

Footnotes

1. Michael Hannecke, Medium (vLLM Security Hardening Guide) — https://medium.com/@michael.hannecke/vllm-in-production-a-security-hardening-guide-for-enterprise-deployments-56a9c2c213dd

   vLLM’s —api-key flag only secures specific /v1 endpoints, leaving other critical diagnostic or monitoring paths completely unauthenticated… CVE-2026-22778 (CVSS 9.8) allowed for Remote Code Execution by sending malicious URLs to video-processing endpoints that lacked authentication.

   ↩ ↩²

2. JarvisLabs blog — Scaling LLM Inference (DP/PP/TP) — https://jarvislabs.ai/blog/scaling-llm-inference-dp-pp-tp

   Increasing to TP=8 on a single node can actually lead to a throughput drop under high concurrency compared to TP=4… frequent NCCL all-reduce calls saturate the interconnect bandwidth, especially on PCIe-based systems without NVLink.

   ↩

3. JarvisLabs — H200 price guide — https://jarvislabs.ai/blog/h200-price

   Specialized GPU clouds offer on-demand H200 instances ranging from $1.40 to $3.80 per hour… self-hosting on a specialized cloud can save between $3,000 and $5,000 per month compared to managed services like HF Dedicated Endpoints.

   ↩

4. Cyfuture Cloud — H200 price/specs guide — https://cyfuture.cloud/kb/gpu/nvidia-h200-price-guide-costs-specs-and-2025-updates

   The H200’s 141GB HBM3e memory is nearly double the H100’s 80GB capacity… H200 can deliver up to 2x faster inference speeds for memory-bound tasks, potentially lowering the total cost-per-token despite a higher hourly rental price.

   ↩

5. Hugging Face blog — HF CLI for Agents — https://huggingface.co/blog/hf-cli-for-agents

   The CLI is now ‘agent-optimized,’ detecting when it is being driven by AI tools like Claude Code or Codex… structured hints that suggest the next logical command with pre-filled IDs, which reportedly reduces token usage by up to 6x compared to manual API calls.

   ↩

6. Spheron Network — HF Inference Endpoints alternatives — https://www.spheron.network/blog/hugging-face-inference-endpoints-alternatives/

   Waking an endpoint can take several minutes as weights are pulled and the container initializes… this delay makes scale-to-zero unsuitable for real-time, user-facing applications unless paired with a custom client-side queuing system.

   ↩

7. Jelassi et al., ‘Repeat After Me’ (arXiv 2406.07522) — https://arxiv.org/abs/2406.07522

   Vanilla Mamba models fail to reliably copy tokens once sequences exceed roughly 2,000 tokens, whereas Transformers maintain near-perfect accuracy… for Mamba to perform a perfect COPY operation on arbitrary sequences, its state size would need to grow linearly with the input length.

   ↩

8. Grazzi/Siems et al., ‘Unlocking State-Tracking in Linear RNNs Through Negative Eigenvalues’ (arXiv 2411.12537) — https://arxiv.org/abs/2411.12537

   Finite-precision LRNNs require negative eigenvalues in the range [-1, 1] to flip bits or track state changes effectively… incorporating negative values allows these architectures to simulate any finite state automaton.

   ↩

9. Microsoft Research, ‘Samba: Simple Hybrid State Space Models’ — https://www.microsoft.com/en-us/research/publication/samba-simple-hybrid-state-space-models-for-efficient-unlimited-context-language-modeling/

   Samba pairs Mamba with Sliding Window Attention and has demonstrated ‘perfect memory recall’ on the Passkey Retrieval and Phonebook tasks… tested up to 1M tokens in zero-shot settings.

   ↩

10. DeepLearning.AI, ‘AI21 Labs Jamba 1.5 Outpaces Transformers in Long-Text Processing’ — https://www.deeplearning.ai/the-batch/ai21-labs-jamba-1-5-outpaces-transformers-in-long-text-processing

    Jamba supports a 256K context window on a single GPU and achieves 3x the throughput of models like Mixtral-8x7B… interleaving attention and Mamba layers at a 1:7 ratio to balance reasoning and throughput.

    ↩

11. Medium, ‘Why Scaling Pre-training Loss Might Be Ruining Your LLMs Reasoning’ — https://medium.com/@zljdanceholic/why-scaling-pre-training-loss-might-be-ruining-your-llms-reasoning-f17b0467829a

    Two models achieving the same validation loss can exhibit vastly different capabilities… a model might achieve a low average loss by being highly accurate on ‘easy’ frequent tokens while remaining incapable of the complex ‘long-tail’ reasoning required for specialized tasks.

    ↩

12. Hugging Face blog (bird-of-paradise), ‘Three Horsemen of Numerical Divergence’ — https://huggingface.co/blog/bird-of-paradise/three-horseman-of-numerical-divergence-1

    Because recurrent states are highly sensitive to computational precision, practitioners must often use FP32 precision for hidden states and disable certain optimizations like ‘cascade attention’ to avoid reward collapse during RL.

    ↩

13. letsdatascience.com (Datasette 1.0 alpha status) — https://letsdatascience.com/news/datasette-export-database-releases-version-03a2-with-export-da738105

    As of late June 2026, the project has reached version 1.0a35, introducing major features like a native web interface for creating and altering tables… installing plugins via standard commands sometimes fails because many are pinned to specific alpha versions to accommodate evolving hook signatures.

    ↩

14. letsdatascience.com (packaging anti-pattern context) — https://letsdatascience.com/news/datasette-export-database-releases-version-03a2-with-export-da738105

    Pinning in libraries is viewed as ‘broken’ because it prevents users from receiving critical security patches or bug fixes from dependencies without a new release of the library itself.

    ↩

15. letsdatascience.com (implementation detail) — https://letsdatascience.com/news/datasette-export-database-releases-version-03a2-with-export-da738105

    this plugin creates a safe, byte-for-byte consistent snapshot of the database while it remains fully operational and online… the plugin explicitly excludes in-memory databases, as these cannot be backed up to a file using the same physical file-copy logic.

    ↩ ↩²

16. letsdatascience.com (security model) — https://letsdatascience.com/news/datasette-export-database-releases-version-03a2-with-export-da738105

    Security is handled through a specific export-database permission, which ensures that only authorized actors can trigger an export from the database action menu.

    ↩

17. letsdatascience.com (competitive landscape) — https://letsdatascience.com/news/datasette-export-database-releases-version-03a2-with-export-da738105

    it is often viewed as a lighter alternative to tools like Litestream, which provides continuous, real-time replication to cloud storage for high-availability disaster recovery… it does not solve the problem of server crashes or accidental DELETE commands that happen between manual exports.

    ↩

18. letsdatascience.com (ecosystem dissent) — https://letsdatascience.com/news/datasette-export-database-releases-version-03a2-with-export-da738105

    Some developers have expressed dissent regarding this shift, noting that the original read-only focus was a critical safety feature for public data publishing.

    ↩