HF signs kernels after RCE, LeRobot patches pickle RCE, Photoroom halves FID
Two trusted ML dependencies quietly patch pickle-family RCE bugs while Photoroom rewrites its data pipeline to halve FID to 11.
HF signs kernels after RCE, LeRobot patches pickle RCE, Photoroom halves FID
TL;DR
- Hugging Face ships a signed kernel repo type after CVE-2026-4372 bypassed
trust_remote_code=Falseacross 232M downloads. - LeRobot v0.6.0 silently patches CVE-2026-25874, an unauthenticated pickle-deserialization RCE in
PolicyServer. - Photoroom re-captions its PRX corpus with Qwen3-VL, cutting FID from ~21 to ~11 at 100k steps.
- LIBERO-plus drops top VLAs from >95% to <30% under mild camera or pose perturbations.
- Torch Stable ABI excludes Philox/RNG, blocking ABI-stable Flash-Attention-2 whenever dropout is enabled.
Two of today’s three AI-tech reads are the same story in different repos: a trusted ML dependency silently shipped a patch for an unauthenticated pickle-family RCE. Hugging Face’s new signed kernel repo type is the response to CVE-2026-4372, which bypassed trust_remote_code=False across roughly 232M downloads via an optional kernels dep. LeRobot v0.6.0 quietly closes CVE-2026-25874 in its PolicyServer. Neither disclosure got the noise it earned — the fix landed with a version bump and a changelog line.
The third read sits one layer over: Photoroom halved its FID from ~21 to ~11 by re-captioning its entire PRX corpus with Qwen3-VL and swapping PNG for JPEG-92. It’s a data-pipeline post, not a security one, but it belongs alongside the other two — none of today’s ships is a model. The work is all in the plumbing beneath one.
Hugging Face hardens Kernels after 232M-download RCE
Source: huggingface-blog · published 2026-07-06
TL;DR
- Hugging Face ships a Hub “kernel” repo type with Sigstore signing, Nix hermetic builds, and trusted-publisher gating by default.
- CVE-2026-4372 bypassed
trust_remote_code=Falseacross ~232M downloads via thekernelsoptional dependency. - The vaunted Torch Stable ABI excludes Philox/RNG, blocking ABI-stable Flash-Attention-2 whenever dropout is enabled.
- TVM FFI is the more interesting bet: DLPack zero-copy across JAX/CuPy/PyTorch, build times cut from minutes to seconds.
Cleanup, not innovation
Read the revamp announcement alongside CVE-2026-4372 and the framing changes. The blog presents cosign signing, Nix reproducibility, and a “trusted publisher” opt-in as forward-looking security architecture. The timing tells a different story: attackers had been poisoning a hidden _attn_implementation_internal field in config.json, which caused from_pretrained() to silently fetch and execute Python from an attacker-controlled repo — even when users had explicitly set trust_remote_code=False 1. CVE.org pegs the exposure at Transformers 4.56.0 through 5.2.x, downloaded roughly 232 million times during the six-month window 2. The new Hub repo type and default-deny loading behavior are the fix, not a feature.
The signing story is thinner than it looks. In May 2026, over 600 malicious npm packages passed Sigstore verification because attackers minted valid ephemeral certificates using stolen maintainer OIDC tokens 3. The same failure mode applies to HF’s cosign flow: compromise a CI identity and you produce a “legitimate” signature. Trusted-publisher gating moves the trust boundary to the OIDC issuer rather than eliminating it.
The Stable ABI’s fine print
HF pitches the Torch Stable ABI adoption as two years of forward compatibility, but PyTorch’s own docs are more restrained. The stable surface is a limited subset, and Philox/RNG have no stable counterparts — which specifically blocks fully ABI-stable Flash-Attention-2 builds whenever dropout is enabled 4. Kernel authors targeting stochastic ops should treat the two-year guarantee as conditional. manylinux_2_28 and the dynamic libstdc++ fix are unambiguous wins (the static-linking crashes at C++ regex init were real), but “compile once, run for two years” is aspirational for anything touching RNG state or crossing CUDA major transitions.
| HF’s framing | The qualification |
|---|---|
| Sigstore + cosign prevents malicious uploads | 600+ npm packages passed Sigstore via stolen OIDC in May 2026 3 |
| Torch Stable ABI = 2yr forward compat | No stable Philox/RNG; breaks FA2 with dropout 4 |
| Custom kernels deliver “verified speedups” | torch.compile matches custom kernels on many decode workloads 5 |
TVM FFI is the actual bet
The genuinely interesting piece is the Apache TVM FFI adoption, and it’s undersold in the post. Feng’s December 2025 talk documents pybind11-to-TVM-FFI migrations dropping build times “from minutes to seconds” by cutting the massive framework-specific headers, plus DLPack-based zero-copy tensor handoff across JAX, CuPy, and PyTorch 6. FlashInfer and NVIDIA’s CuTe DSL have already adopted it, giving HF’s bet external validation rather than a bespoke standard.
The unspoken question is whether the whole distribution apparatus earns its keep. Practitioner threads argue that a properly configured torch.compile baseline matches hand-tuned kernels for many decode workloads, and that LibTorch host overhead — tensor destructors, allocator churn — often dominates before any kernel-level gain lands 5. If the agentic kernel-development pipeline HF is scaffolding produces artifacts that lose to a one-line compile flag, the machinery is solving the wrong problem.
Takeaway
The revamp is HF hardening its supply chain in response to a real RCE, wrapped in a bet that TVM FFI becomes the cross-framework substrate. Both are defensible. But “trusted publisher” is not a cryptographic guarantee, and Stable ABI is not portability. Treat the security posture as improved, not solved.
LeRobot v0.6.0 quietly patches an unauthenticated RCE
Source: huggingface-blog · published 2026-07-07
TL;DR
- LeRobot v0.6.0 silently patches CVE-2026-25874, an unauthenticated pickle-deserialization RCE in
PolicyServer. - LIBERO-plus shows top VLAs drop from >95% to <30% under mild camera or pose perturbations.
- NVIDIA GR00T N1.7 matches N1.5 on RoboCasa at ~64%, gaining only in language-following.
- New FSDP multi-GPU support inherits open upstream bugs in
accelerate’s gradient clipping and loss curves.
The headline feature is a security patch
Read past the marketing and v0.6.0 is a mandatory upgrade. VulnCheck’s disclosure shows PolicyServer deserializing untrusted pickle payloads over plaintext gRPC in SendPolicyInstructions, SendObservations, and GetActions — any of them yields remote code execution on the inference host 7. Every release up to and including v0.5.1 is affected; v0.6.0 is the designated fix, but the announcement foregrounds FastWAM and Robometer instead of the CVE, which independent commentary pegs at CVSS 9.3–9.8 8.
That framing matters because LeRobot inference typically runs on a privileged GPU box wired to physical actuators. The attack path is short:
flowchart LR
A[Unauthenticated<br/>attacker] -->|gRPC pickle payload| B[PolicyServer<br/>≤ v0.5.1]
B -->|deserialize| C[RCE on<br/>GPU host]
C --> D[Policy weights<br/>& datasets]
C --> E((Physical<br/>actuators))
If you run LeRobot in a lab, treat this release as a security update first and a research showcase second.
World models arrive with a benchmark asterisk
The technical additions are real. VLA-JEPA predicts latent future frames and discards the world model at inference for speed; FastWAM pairs a video-generation expert with a compact action expert and skips the “dreaming” phase in deployment; GR00T N1.7 swaps in Cosmos-Reason2-2B behind a 32-layer diffusion transformer, trained on 20,000 hours of EgoScale human video 9.
The catch is what the new benchmarks reveal. The LIBERO-plus authors — whose suite is one of the six now wired into lerobot-eval — argue current VLAs learn shortcuts, not language. Models scoring above 95% on stock LIBERO collapse below 30% under modest viewpoint or initial-state changes, and score similarly well when the language instruction is randomized 10. NVIDIA’s own N1.7 numbers reinforce the point: RoboCasa success is essentially flat versus N1.5, with the wins concentrated in instruction-following rather than raw task completion 9. Anyone using the shiny new leaderboards to pick a policy should perturb the camera before signing off.
Reward models and FSDP: read the fine print
Robometer (4B, Qwen3-VL) and TOPReward — a zero-shot trick that reads the log-probability of the token “True” — are the more genuinely novel bits. TOPReward hits a 0.947 Value-Order Correlation on ManiRewardBench, impressive for something that requires no reward-model training 11. But it normalizes per-episode via min-max, so scores aren’t comparable across tasks, and step-wise VLM queries in an RL loop demand parallel envs or a serious local GPU — awkward for LeRobot’s hobbyist base 11.
The new FSDP path has similar sharp edges. Upstream accelerate still has open issues where clip_grad_norm_ produces inconsistent values under FSDP even in deterministic mode, and configs with fsdp_use_orig_params=False can train with a loss that never decreases 12. LeRobot also declines to auto-scale learning rates across GPUs. Reproducing the flagship policies at multi-GPU scale will be a support burden.
Takeaway
v0.6.0 is the most ambitious LeRobot release to date, and also the one whose changelog most understates what upgrading buys you. Patch first for the RCE, then benchmark honestly — with the camera moved.
Photoroom re-captions PRX corpus, halves FID to 11
Source: huggingface-blog · published 2026-07-06
TL;DR
- Photoroom re-captioned its entire PRX corpus with Qwen3-VL, dropping FID from ~21 to ~11 at 100k steps.
- Pipeline splits Lance for curation, MDS for streaming — an emerging pattern that trades format rigidity for resumable shuffling.
- JPEG quality-92 replaces PNG, cutting storage 3-10× with no visible generation quality loss in internal A/Bs.
- Captioner choice between Qwen3.5-9B and Qwen3-VL-8B rests on a ~0.4 FID gap that sits inside known CMMD/FID noise.
The two-format pipeline is becoming a pattern
Photoroom’s Part 4 blog opens the hood on the data stack behind PRX, its 7B text-to-image model, and the most transferable idea is the format split. Curation and feature engineering happen in Lance, whose columnar layout is optimized for random-access blob fetches — LanceDB benchmarks put it up to 75× faster than Parquet on non-sequential lookups 13. Training then streams from Mosaic Data Shards, which give deterministic resumable shuffling and elastic mid-epoch restarts.
The catch the post is honest about: MDS is deliberately rigid. Adding a column normally means rewriting terabytes of shards. Two independent design choices push back on that rigidity from different angles. First, skip lists applied in the loader let Photoroom filter NSFW, screenshots, or user opt-outs without touching the shards. Second, text latents are computed on the fly with Qwen3-VL rather than materialized into the shards — a 3-4% throughput tax in exchange for the freedom to swap text encoders later without a full rewrite. Different mechanisms, same underlying bet: keep the shards small and immutable, absorb the flexibility cost elsewhere.
Re-captioning does the heavy lifting
The biggest quality delta in the whole post isn’t architecture or optimizer — it’s captions. Moving from short baseline captions to 100-200 word VLM-generated paragraphs cut FID roughly in half:
| Captioner | FID (↓) | CMMD (↓) | DINO-MMD (↓) |
|---|---|---|---|
| Qwen3.5-9B | 10.51 | 0.278 | 0.162 |
| Qwen3-VL-8B (chosen) | 10.98 | 0.351 | 0.182 |
| Qwen2.5-VL-7B | 15.86 | 0.393 | 0.285 |
| LLaVA-1.5-LLaMA3-8B | ~21 | ~0.52 | ~0.35 |
Photoroom picked Qwen3-VL-8B over the metrically-better Qwen3.5-9B on throughput grounds (20 img/s per H200) and framework stability. That’s defensible — but the deltas driving the choice deserve scrutiny.
Where the metrics get soft
Jayasumana et al. introduced CMMD precisely because FID is a biased Gaussian estimator that needs ~50k samples to stabilize; CMMD is unbiased and sample-efficient 14. Fine. The problem: CMMD inherits CLIP’s feature-space blind spots on spatial relations and attribute binding, so a captioner that produces CLIP-agreeable prose can win on CMMD without producing better images 15. The ~0.4 FID and ~0.07 CMMD gap between the top two captioners sits comfortably inside that noise floor. DINO-MMD partly hedges this, but the ranking is more fragile than the table suggests.
Two other omissions worth flagging. The 13-bucket aspect-ratio scheme (0.5-2.0) is the NovelAI recipe that eliminated “decapitated subject” artifacts from square-crop training 16; practitioner threads warn about upscaling into larger buckets and bucket-imbalance failure modes that the post doesn’t address 17. And on captioning form, NVIDIA’s Cosmos-3 report describes a quadrant-based scanning approach — the VLM describes four spatial regions plus a center, then merges into JSON 18 — which is quietly displacing flat long-paragraph captions at larger labs targeting spatial grounding.
Takeaway
The engineering choices are sound and land squarely on 2025 best practice. The interesting tell is what Photoroom optimized against: their pipeline is built for cheap iteration on captions and encoders, not for locking in a single training recipe. Given how much of the FID gain came from re-captioning, that priority looks correct.
Footnotes
-
eSecurity Planet on CVE-2026-4372 — https://www.esecurityplanet.com/threats/hugging-face-vulnerability-allows-remote-code-execution/
↩By poisoning a hidden field titled
_attn_implementation_internal, an attacker can point the library to a malicious repository ID on the Hugging Face Hub… When a user calls the standardfrom_pretrained()method—even with remote code execution explicitly disabled—the library silently downloads and executes Python code from the attacker-controlled repository. -
CVE.org record CVE-2026-4372 — https://www.cve.org/CVERecord?id=CVE-2026-4372
↩Vulnerable versions were downloaded approximately 232 million times during the six-month window the flaw was active; affects Transformers 4.56.0 through 5.2.x when the optional
kernelspackage is installed. -
VentureBeat on npm Sigstore provenance abuse — https://venturebeat.com/security/npm-sigstore-provenance-stolen-identity-audit-grid-2026
↩ ↩2In May 2026, over 600 malicious npm packages successfully passed Sigstore verification because attackers had generated valid ephemeral certificates using stolen OIDC tokens from maintainer accounts.
-
PyTorch docs — LibTorch Stable ABI notes — https://docs.pytorch.org/docs/stable/notes/libtorch_stable_abi.html
↩ ↩2The Philox and RNG (Random Number Generator) APIs lack stable counterparts, which currently blocks general ABI-stable implementation for libraries like Flash-Attention-2 when dropout is enabled.
-
r/cpp_questions — ‘libtorch/pytorch is embarrassingly slow’ — https://www.reddit.com/r/cpp_questions/comments/1o7frwa/libtorch_pytorch_is_embarrassing_slow_what_are/
↩ ↩2Independent developers express concern over LibTorch’s general performance, citing significant time spent in tensor destructors and memory allocations… for many decode workloads, a correctly configured torch.compile baseline may actually match or beat the gains of custom kernels.
-
Feng — ‘TVM FFI: Open ABI and FFI for MLsys’ talk (Dec 2025) — https://syfeng.net/assets/talks/20251227-TVM-FFI:%20Open%20ABI%20and%20FFI%20for%20MLsys.pdf
↩Migrating from traditional pybind11 architectures to TVM FFI can reduce build times from minutes to seconds by eliminating the need to include massive framework-specific headers like those from PyTorch; standardizes on DLPack for zero-copy tensor transfers between JAX, CuPy, and PyTorch.
-
The Hacker News (CVE-2026-25874 disclosure) — https://thehackernews.com/2026/04/critical-cve-2026-25874-leaves-hugging.html
↩Unauthenticated attackers can send malicious pickle payloads to PolicyServer via gRPC calls such as SendPolicyInstructions, SendObservations, or GetActions, achieving remote code execution on the inference host.
-
op-c.net analysis of CVE-2026-25874 — https://op-c.net/blog/cve-2026-25874-hugging-face-lerobot-rce/
↩Affects all versions up to and including v0.5.1; v0.6.0 is the designated remediation, but the release notes advertise world models rather than foregrounding the pickle/TLS fix.
-
NVIDIA GR00T N1.7 blog on Hugging Face — https://huggingface.co/blog/nvidia/gr00t-n1-7
↩ ↩2Trained on over 20,000 hours of EgoScale human egocentric video with a dual-system Action Cascade (Cosmos-Reason2-2B + 32-layer Diffusion Transformer); N1.7 delivers comparable RoboCasa success to N1.5 (~64%) but with markedly improved language-following.
-
LIBERO-plus paper (arXiv 2510.13626) — https://huggingface.co/papers/2510.13626
↩VLA models achieving >95% on standard LIBERO suites collapse below 30% under modest camera-viewpoint or initial-state perturbations, and remain performant even when language instructions are randomized — indicating shortcut learning rather than genuine language grounding.
-
wispaper.ai technical review of TOPReward — https://www.wispaper.ai/en/user-blog/topreward-token-probabilities-hidden-zero-shot-rewards-robotics-20260228/eng
↩ ↩2TOPReward achieves a mean Value-Order Correlation of 0.947 on ManiRewardBench but relies on per-episode min-max normalization, complicating cross-task reward comparison and requiring high-end GPUs for step-wise VLM queries in RL loops.
-
Hugging Face accelerate GitHub issue #3256 (FSDP) — https://github.com/huggingface/accelerate/issues/3256
↩clip_grad_norm_ under FSDP can produce inconsistent outputs even with deterministic mode enabled, and models with fsdp_use_orig_params=False exhibit constant loss that never decreases — a foot-gun for anyone scaling LeRobot policies across GPUs.
-
LanceDB blog — Lance v2.2 benchmarks — https://www.lancedb.com/blog/lance-format-v2-2-benchmarks-half-the-storage-none-of-the-slowdown
↩Lance v2.2 is heavily optimized for blob fetches… up to 75x faster than Parquet and significantly more efficient than MDS for non-sequential lookups
-
Jayasumana et al., ‘Rethinking FID’ (CVPR 2024) — https://openaccess.thecvf.com/content/CVPR2024/papers/Jayasumana_Rethinking_FID_Towards_a_Better_Evaluation_Metric_for_Image_Generation_CVPR_2024_paper.pdf
↩CMMD… is an unbiased estimator [that] does not produce inconsistent scores when sample sizes vary… significantly more sample-efficient than the 50,000 images typically required for FID
-
EmergentMind — CMMD topic overview — https://www.emergentmind.com/topics/clip-based-maximum-mean-discrepancy-cmmd
↩CMMD may inherit [CLIP’s] blind spots, potentially rewarding models that match CLIP’s biased feature space rather than objective visual realism
-
Towards AI — ‘How Stable Diffusion Trains Variable Resolution Images’ — https://pub.towardsai.net/how-stable-diffusion-trains-variable-resolution-images-without-pytorch-errors-007487ae7933
↩Aspect ratio bucketing… originally popularized by NovelAI… significantly improves the model’s ability to generate naturally framed subjects, such as full-body characters without ‘decapitated’ heads or missing feet
-
Hugging Face discuss — ARB for SDXL fine-tuning — https://discuss.huggingface.co/t/aspect-ratio-bucketing-when-fine-tuning-sdxl/86174
↩Avoid upscaling small images to fit large buckets (‘no_upscale’), as this introduces noise and artifacts… a dataset with 39 tall images and only one wide image can lead to imbalanced learning
-
NVIDIA Cosmos-3 technical report — https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf
↩quadrant-based scanning mechanism where the VLM independently describes four spatial quadrants and a central region before merging them into a final JSON-formatted caption