Split-LLM leaks off-audit, Uno’s 3× disputed, CausalPFN called Rung 1

TL;DR

• Split-LLM decoys leak 100% of real rows via a return-gradient wire the privacy audit never listed.
• Uno’s 3× lossless speedup shrinks once DiffusionGemma’s lead reruns the baseline at ~2,700 tok/s on the same H200.
• CausalPFN matches a tuned T-Learner at 100× inference speed on Lalonde-CPS with no target fine-tuning.
• Pearl’s camp rejects transformer causal estimators as ‘statistics on steroids’ stuck at Rung 1.
• Do-PFN and CausalFM ATEs collapse toward zero when the synthetic prior misses the real data-generating process.

Today’s three research features share one texture: each headline number lives inside a scope choice the paper made, and today a different reviewer pushed back on a different scope. Split-LLM’s decoy privacy scheme cleared its forward-channel audit and still leaked every real row through the return-gradient wire the auditors never declared as a privacy surface. Uno’s 3× lossless speedup holds at high batch and long context and collapses at batch-size-1, once DiffusionGemma’s lead reruns the same baseline on the same H200. CausalPFN matches a tuned T-Learner at 100× inference speed on Lalonde-CPS — but Judea Pearl’s camp refuses to grade transformer causal estimators on the same rubric at all, calling them stuck at Rung 1.

None of the three papers is being called wrong. Each is being told the number it printed answers a narrower question than the abstract implied — and today, three separate parties said so on the record.

Split-LLM decoys leak 100% of real rows via return gradients

Source: hf-daily-papers · published 2026-09-02

TL;DR

• Zero-gradient signature unmasked real rows in 4,096/4,096 frames across nine seeds, collapsing decoy anonymity to nothing.
• The system passed its forward-channel privacy audit because the return-gradient wire was never declared as a privacy surface.
• Clipping and noise on outbound gradients closed the leak for ~0.01 nats of cross-entropy — on a Qwen3-0.6B toy setup.
• The decoy paradigm was already weak against active adversaries in prior work, regardless of this specific bug.

The bug: a signature made of zeros

The paper dissects a split-learning setup where a trusted local node (tln) holds the embeddings, head, and loss, while an untrusted cloud node (ucn) runs 11 middle transformer layers on Qwen3-0.6B. To hide real data, the tln mixes 32 real rows with 48 “decoy” rows per 80-row frame, applies per-request rotations/permutations, and clips activations with Gaussian noise (C=1.0, σ=0.35C) before sending them across a D=64 bottleneck.

The forward channel looks fine. The backward channel is where the design collapses: the private loss is computed only on the 32 real rows, so the gradients returned to the ucn for the 48 decoys are identically zero. The ucn doesn’t need to invert anything — it just reads off which rows have nonzero gradients. Row-level de-anonymization hit 100% on every one of 4,096 frames per run. Once the decoys were filtered, a frequent-token probe recovered 0.65–1.50 percentage points above a constant-guess baseline: modest, but the anonymity set was already gone.

flowchart LR
    A[80-row frame: 32 real + 48 decoys] --> B[tln: embed + rotate + noise]
    B -- activations D=64 --> C[ucn: 11 transformer layers]
    C -- activations --> D[tln: head + private loss on 32 real rows]
    D -- gradients: 32 nonzero, 48 zero --> C
    C -. reads zero-pattern .-> E((Adversary identifies real rows))

The methodological point is the one worth keeping: the original privacy evaluation passed because the return-gradient wire wasn’t instrumented as an attack surface. A calibrated gate with positive and negative controls flipped from pass to fail the moment that channel was included in the adversary’s view.

The fix is cheap here, probably not at scale

The reported mitigation — per-row clipping at C=0.01 plus Gaussian noise on outbound gradients — costs about 0.01 nats of held-out cross-entropy. That is suspiciously cheap, and the research context suggests why: Qwen3-0.6B on WikiText-2 is a regime already comfortably “within budget.”

Scaling up hits known walls. TPDP 2026 work shows aggressive clipping introduces a non-vanishing bias that stalls convergence, while looser regularization-style DP variants preserve utility but fail against reconstruction ¹. ICML 2026’s DP-SGD-RC exists precisely because deterministic per-sample clipping is memory-prohibitive at LLM scale, and even its randomized approximation only matches deterministic utility on Llama 3.2 ². A 0.01-nat tax on a 0.6B model tells you very little about a production split-training run.

The deeper problem the paper sidesteps

Even a decoy scheme that survived the zero-gradient bug would still face stronger adversaries. Gradient inversion against language models can reconstruct tokens from intermediate gradients under batching, and standard perturbation defenses tend to shift the utility–privacy frontier rather than close it ³. SDAR-style attacks go further: a curious server can train a decodable simulator of the client’s private layers via adversarial regularization, recovering features and labels without ever needing a metadata side-channel ⁴.

That is the uncomfortable subtext. Enterprise interest in split architectures is being driven by HIPAA and GDPR pressure, but some CTOs are already treating splitting as “a temporary bridge rather than a final security destination,” pivoting toward on-edge deployment instead ⁵. This paper’s most durable contribution isn’t the fix — it’s the audit discipline: every wire, including the one carrying gradients home, is a privacy surface until proven otherwise.

---

Uno’s 3× lossless LLM speedup runs into a baseline dispute

Source: hf-daily-papers · published 2026-09-02

TL;DR

• DiffusionGemma’s lead re-ran Uno’s baseline on the same H200 and got ~2,700–3,000 tok/s — roughly 3× Uno’s cited number.
• Uno distills LoRA diffusion adapters onto a frozen 8B AR model for a claimed 3× lossless speedup.
• Uno’s win is regime-specific: strong at high batch and long context, weaker at batch-size-1 (~405 vs ~836 tok/s for DiffusionGemma).
• Same-family work (DFlash 2, xPress) hints acceptance-length tricks could push these speedups well past 3×.

What Uno actually ships

Uno is a “diffusion-augmented” autoregressive LLM. The AR weights ($\theta_{AR}$) stay frozen and define the target distribution; a set of LoRA adapters ($\theta_\Delta$) is distilled — via Discrete Consistency Distillation — to draft an entire block of tokens in one denoising step from pure noise. A Ψ-Spec sampler then runs the draft through the base AR weights in parallel and rejection-samples to guarantee the output matches the AR distribution exactly. Hence “lossless.”

flowchart LR
    N[Noisy block] --> D["θ_AR + θ_Δ<br/>(diffusion draft)"]
    D -->|candidate tokens| V["θ_AR verify<br/>(parallel)"]
    V -->|accept| O[Output tokens]
    V -.reject.-> N

The Uno-8B model, trained on 23T tokens with a 7B-token diffusion distillation phase, reports up to 3× throughput over its AR base and outperforms DiffusionGemma-26B and Nemotron-Labs-Diffusion-14B across agentic, long-context, and reasoning benchmarks — 25.8 vs 14.1/14.3 on τ³-Bench Banking, 68.0 vs 19.7/7.3 on AA-LCR. It also beats Inception’s proprietary Mercury 2 on tool use, coding and long context, while claiming ~4.6× the max system throughput (5,255 vs 1,197 tok/s). Using Uno for RL rollouts cut end-to-end DAPO training time by 40%.

The baseline fight

The headline “8B beats 26B” framing is under active dispute. Brendan O’Donoghue, who leads DiffusionGemma at DeepMind, reproduced the comparison on identical H200 hardware and reports DiffusionGemma clearing ~2,700 tok/s at BF16 and >3,000 tok/s at FP8 — nearly triple the ~1,136 tok/s number Uno used as its baseline ⁶. If that holds, DiffusionGemma-26B is not the model Uno’s tables describe.

Independent coverage adds a second caveat: even taking Uno’s own numbers, DiffusionGemma has higher batch-size-1 throughput (~836 vs ~405 tok/s) ⁷. Uno’s advantage is specifically a high-batch, memory-bound, long-context server story — not a low-latency single-user one. And Mercury 2, the proprietary comparison point, has itself moved: Inception now claims 91.1 on AIME 2025 and second-place quality on Copilot Arena while remaining the fastest model tested ⁸.

Where the ceiling probably is

The structural problem for any diffusion drafter is that tokens in a block are “individually likely but jointly improbable” ⁹ — the draft can look plausible token-by-token but fail joint verification, collapsing acceptance length and, with it, the speedup. Ψ-Spec pays this cost implicitly; the paper itself flags acceptance length as the dominant knob, plus a train/test mismatch between deterministic DCD trajectories and stochastic sampling.

Concurrent work suggests headroom in both directions. xPress adds a lightweight causal refiner over the diffusion block and reports up to 56% higher acceptance length ¹⁰, which would compound directly with Ψ-Spec. DFlash 2, from an overlapping author group, already reports >6× speedups on math and code, outperforming multi-token prediction and AR drafting ¹¹.

Takeaway

The architecture — frozen AR + distilled LoRA diffusion drafter + rejection-verified sampler — is a clean, portable recipe, and the RL-rollout speedup is the number practitioners should actually care about. The comparative benchmarks are provisional: at minimum, wait for a re-run against O’Donoghue’s DiffusionGemma numbers before treating “8B beats 26B” as settled.

---

Causal Foundation Models match tuned baselines at 100× speed

Source: hf-daily-papers · published 2026-09-01

TL;DR

• CausalPFN hits PEHE 8.97 on Lalonde-CPS, edging out a tuned T-Learner (9.04) with no fine-tuning on the target data.
• 100× inference speedup on GPU vs. traditional causal ML pipelines that need AutoML tuning and cross-fitted nuisance models (1800s → ~18s).
• Do-PFN and CausalFM shrink ATE estimates toward zero (relative errors ~0.9) when the synthetic prior misses the real data-generating process.
• Judea Pearl’s camp rejects the framing, calling transformer-based causal estimators “statistics on steroids” stuck at Rung 1.

The pitch: zero-shot CATE from a pretrained transformer

Causal Foundation Models (CFMs) drop the bespoke pipeline — pick estimator, tune nuisance models, cross-fit — and replace it with in-context learning. You feed a transformer an observational dataset ${(x_n, t_n, y_n)}$ as tokens, append query covariates, and read out the conditional average treatment effect. No training on the target data.

The trick is Prior-data Fitted Networks. The authors sample millions of synthetic Structural Causal Models from a broad prior over DAGs and MLP mechanisms, generate observational data plus ground-truth counterfactuals, and train the transformer to predict the causal quantity given the context. Under the negative log-likelihood objective, the network approximates the posterior predictive over causal effects — amortized Bayesian inference in one forward pass.

The paper packages three variants: Do-PFN for interventional distributions with non-identifiable priors, CausalPFN for backdoor/strong-ignorability settings, and CausalFM for IV and frontdoor identification.

The numbers hold up — narrowly

On RealCause-Lalonde, CausalPFN’s PEHE of 8.97 beats the T-Learner (9.04) and X-Learner (11.90). For population ATE it’s competitive but not dominant: IPW hits 0.15 relative error vs. CausalPFN’s 0.17. The efficiency story is the real headline — traditional pipelines median 1800+ seconds per task; CausalPFN runs in 18.4s on CPU, and an A100 delivers the full 100× speedup.

CausalPFN’s uncertainty story is more principled than a one-line summary suggests: the arXiv version details a regression-calibration procedure where temperature parameters are tuned so outcome coverage empirically bounds the (unobservable) CATE coverage ¹². That’s a mechanism, not just a Bayesian gesture.

The critique: prior coverage is doing all the work

Here’s where the paradigm gets contested. An OpenReview thread on CausalFM makes the sharpest technical point: a single observational SCM induces an exponentially large space of interventional distributions, so any meta-prior that misses the real “grounding prior” produces “unavoidable posterior inconsistency and estimation bias” ¹³. The paper’s own results confirm this — Do-PFN and CausalFM’s ATE shrinkage toward zero is prior misspecification in action.

An independent review of the layer6ai-labs/cfms repo flags a related deployment risk: no prior-coverage diagnostics ship with the code, so a model will happily return a fast estimate on a dataset that looks nothing like its training prior, with no refusal signal ¹⁴. Silent failure, structurally ungrounded output.

Pearl’s camp isn’t buying the framing at all

“Statistics on steroids… smart parrots that lack a parsimonious world model.”

That’s Judea Pearl on transformer-based causal reasoning generally ¹⁵. His argument: functional approximators of conditional probabilities live on Rung 1 of the Ladder of Causation and cannot support the counterfactual reasoning Rung 3 requires. Practitioners on r/MachineLearning echo it — reducing causal inference to a benchmark task treats “strong ignorability” as something amortizable, when it’s a substantive modeling assumption the analyst has to defend ¹⁶.

What’s actually being deployed

The industrial center of gravity in 2026 sits elsewhere. Causaly’s June 2026 Microsoft partnership fuses causal knowledge graphs with simulation to deliver “cited, defensible decisions” for pharma R&D — governance and provenance, not amortized ICL ¹⁷. CFMs may win on wall-clock time and PEHE, but the buyers of applied causal inference are currently paying for the opposite architecture: explicit graphs, auditable assumptions, human-in-the-loop.

The speed and accuracy numbers are real. Whether the exercise counts as causal inference — or as very fast pattern-matching on semi-synthetic benchmarks — is the fight that will decide whether CFMs escape the tabular research niche.

Round-ups

SimpleMemVLA feeds raw video history to VLM for robot memory

Source: hf-daily-papers

SimpleMemVLA skips dedicated memory modules and instead pipes timestamped video frames straight into a pretrained vision-language model, using its hidden states to drive a flow-matching action head. The approach beats purpose-built memory architectures on long-horizon manipulation benchmarks.

EmbodiedSkills wraps VLA policies in a verified skill interface

Source: hf-daily-papers

EmbodiedSkills orchestrates robot agents through a fixed executable-skill interface with post-action verification, closing the loop between high-level skill selection and low-level VLA execution. The framework supports online adaptation and plugs into backbones like Qwen3-VL and OpenPI/pi0.5.

FlowBalance uses verifiers to steer on-policy reasoning self-improvement

Source: hf-daily-papers

FlowBalance reweights reasoning trajectories using terminal-verifier signals and profile-based trajectory balance, guarding against false-positive self-guidance that plagues reverse-KL objectives. The method lifts mathematical reasoning accuracy by grounding token-level log-probability updates in verifier-calibrated group advantages.

Teacher-gated distillation routes unreliable prompts to RL instead

Source: hf-daily-papers

TGOPD probes each prompt with verifier-scored samples to check teacher reliability, sending trustworthy cases through dense on-policy distillation and unreliable ones to GRPO-style reinforcement learning. The per-prompt routing improves both training efficiency and downstream accuracy over uniform distillation baselines.

Conformal Relevance auto-designs score functions via in-context ensembles

Source: hf-daily-papers

The framework replaces hand-crafted conformal score functions with in-context learning and ensembling across LLM prompts, tightening prediction sets while preserving coverage guarantees. It applies uniformly across NLP retrieval tasks and formalizes a complementarity condition plus a saturation bound for the ensemble.

ENEAS merges text-prompted tracking with open-concept segmentation

Source: hf-daily-papers

ENEAS extends the SeC architecture with a text-prompting adapter and temporal memory, then adds a semantic verification layer that filters visual distractors through VLM refinement and embedding matching. It handles both instance tracking and open-vocabulary discovery in one model, with 3D reconstruction demos.

Study probes LLM ability to propagate local revisions across artifacts

Source: hf-daily-papers

When users request a small edit during conversational artifact generation, LLMs must trace dependencies and update every affected section. The paper benchmarks how well models handle this propagation and tests cost-effective test-time compute strategies to fix the parts they miss.

Footnotes

1. TPDP 2026 — Ullah, DP split-learning trade-offs — https://tpdp.journalprivacyconfidentiality.org/2026/pdfs/ullah.pdf

   Aggressive clipping introduces a non-vanishing bias that can stall convergence, while weaker regularization-style DP variants preserve accuracy but prove ineffective against reconstruction.

   ↩

2. ICML 2026 poster — DP-SGD-RC (Randomized Clipping) for LLMs — https://icml.cc/virtual/2026/poster/62676

   Standard DP-SGD scales poorly with LLM context length and width; randomized clipping via Hutchinson-style trace estimation matches deterministic clipping utility on Llama 3.2 while cutting the memory overhead of per-sample clipping.

   ↩

3. USENIX Security ‘23 — Yue et al., gradient inversion on language models — https://www.usenix.org/conference/usenixsecurity23/presentation/yue

   Gradient inversion attacks on language-model training can reconstruct tokens from intermediate gradients even under batching, and standard perturbation defenses often only shift the utility–privacy trade-off rather than close the channel.

   ↩

4. arXiv 2605.24879 — SDAR split-learning reconstruction — https://arxiv.org/html/2605.24879v1

   A malicious server can train a ‘decodable simulator’ of the client’s private layers via adversarial regularization, reconstructing sensitive features and labels with high accuracy even without observing raw inputs.

   ↩

5. ArcCompute — ‘Private LLMs in 2026’ — https://www.arccompute.io/resources/arc-blog/private-llms-in-2026-the-infrastructure-decision-behind-enterprise-ai-control

   HIPAA and GDPR compliance mandates have made standard cloud APIs a liability … some CTOs are pivoting toward fully local, on-edge private LLMs, viewing distributed splitting as a temporary bridge rather than a final security destination.

   ↩

6. Developer’s Digest — DiffusionGemma explainer (context for O’Donoghue rebuttal) — https://www.developersdigest.tech/blog/diffusiongemma-diffusion-text-generation

   a properly tuned vLLM setup achieved roughly 2,700 tokens/s at BF16 precision and exceeded 3,000 tokens/s using FP8 … nearly triple what was cited in the Uno paper

   ↩

7. implicator.ai coverage of Uno — https://www.implicator.ai/uno-lossless-ai-speedup/

   DiffusionGemma retains a higher batch-size-one throughput (~836 tok/s vs Uno’s ~405 tok/s), [but] Uno’s architecture may still hold an edge in high-batch, long-context scenarios where memory management becomes the primary bottleneck

   ↩

8. Inception Labs — Mercury 2 launch post — https://www.inceptionlabs.ai/blog/introducing-mercury-2

   Mercury 2 tied the speculative ‘GPT-5 Mini’ on the AIME 2025 benchmark with a score of 91.1 … ranked second on quality while remaining the fastest model overall

   ↩

9. Leonie Monigatti — Speculative Decoding survey — https://leoniemonigatti.com/blog/speculative-decoding.html

   independently predicted tokens in a block might be individually likely but jointly improbable, leading to low acceptance rates

   ↩

10. alphaXiv — xPress causal refiner for diffusion drafters — https://www.alphaxiv.org/abs/2511.00606

    xPress acts as a lightweight causal refiner, restoring missing dependencies across the diffusion block to increase acceptance length by up to 56%

    ↩

11. Hugging Face — z-lab/Qwen3.8-27B-DFlash2 checkpoint — https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2

    DFlash 2 show speedups exceeding 6× on tasks like math reasoning and code generation, outperforming traditional multi-token prediction (MTP) and autoregressive drafting

    ↩

12. CausalPFN arXiv (2506.06039) — calibration methodology — https://arxiv.org/html/2506.06039

    CausalPFN uses a regression calibration framework where temperature parameters are tuned to ensure that outcome coverage serves as a reliable proxy for the unobservable Conditional Average Treatment Effect (CATE) calibration; empirically the Integrated Coverage Error for regression bounds the ICE for CATE.

    ↩

13. OpenReview discussion on CausalFM (prior uncoverage) — https://openreview.net/challenge?redirect=%2Fforum%3Fid%3DYjPDsCH5zg%26noteId%3DHapx9JfwXf

    A single observational SCM can induce an exponentially large space of potential interventional distributions… if the synthetic meta-prior does not perfectly align with the real-world grounding prior, the model suffers from unavoidable posterior inconsistency and estimation bias.

    ↩

14. lilys.ai independent review of the CFMs repo — https://lilys.ai/en/notes/daily-papers-20260908/causal-foundation-models

    The implementation lacks ‘prior-coverage diagnostics,’ meaning a model may provide a prediction for a dataset that looks nothing like its training prior without flagging the potential for high error… a ‘silent failure’ risk where the model produces a fast estimate that is nonetheless structurally ungrounded.

    ↩

15. Acalytica summary of Judea Pearl interviews — https://acalytica.com/judea-pearl-llms/

    Pearl described these models as ‘statistics on steroids’ or ‘smart parrots’ that lack a parsimonious world model… because transformers are essentially functional approximators of conditional probabilities, they cannot perform the counterfactual reasoning (Rung 3) required for genuine understanding.

    ↩

16. r/MachineLearning discussion of CausalPFN — https://www.reddit.com/r/MachineLearning/comments/1lbgiua/r_causalpfn_amortized_causal_effect_estimation/

    Reducing causal inference to a ‘benchmark-driven task’ akin to image classification is fundamentally misguided… causal inference is a substantive process of reasoning about mechanisms and assumptions—such as ‘strong ignorability’—which cannot be offloaded to a pre-trained model.

    ↩

17. Causaly × Microsoft press release (June 2026) — https://www.causaly.com/news/causaly-and-microsoft-collaborate-to-connect-scientific-computation-to-scientific-decision-making-for-life-sciences-r-d

    Merged large-scale scientific computation (simulation and analytics) with causal knowledge-graph reasoning… enables R&D teams to move from raw data signals to ‘cited, defensible decisions’ within a governed environment.

    ↩