Meta cuts BLT 77%, ReasonMaxxer skips RL at $25, MV-Split hits 1000 DiT layers

TL;DR

• Meta’s BLT-S cuts byte-model bandwidth 62-77% losslessly via self-speculation, byte-identical to greedy decoding.
• ReasonMaxxer matches RL math gains for $4-$25 versus $200-$103,000 for GRPO/PPO baselines.
• MV-Split residuals train a stable 1000-layer, 13.6B-parameter DiT where vanilla Post-Norm diverges shallower.
• MLS-Bench finds AI agents tune existing ML methods rather than invent new ones.
• HumanNet releases 1M hours of human-centric video for vision-language-action pretraining.

Today’s three feature papers each swap an expensive primitive for a smaller mechanism. Meta’s Fast BLT replaces full byte-level decoding with self-speculation and diffusion drafting, cutting bandwidth 62-77% without changing outputs. ReasonMaxxer argues most of RL’s math gains are a rank-32 LoRA in disguise, reproducible for the price of a steak dinner. MV-Split residuals identify the gradient pathology that breaks deep diffusion transformers, then patch it to train a stable 1000-layer model.

Each paper also bounds its own claim. BLT’s speedups are bandwidth and NFE proxies, not H100 wall-clock. ReasonMaxxer concedes NVIDIA’s ProRL — where base models score 0% on logic puzzles — is a regime its LoRA recipe can’t reach. And MV-Split’s headline depth was already set by DeepNet in 2022; the contribution is the Mean Mode Screaming diagnosis, not the layer count.

Meta’s Fast BLT cuts byte-model bandwidth 77% losslessly

Source: hf-daily-papers · published 2026-05-07

TL;DR

• BLT-S delivers a lossless 62–77% bandwidth cut via self-speculation, byte-identical to greedy BLT.
• BLT-D-16’s 87–92% bandwidth cut roughly halves HumanEval Pass@1 (22.6 → 11.6).
• BLT-DV drafts with diffusion then verifies autoregressively, recovering most pure-diffusion quality loss.
• All speedups are estimated bandwidth/NFE proxies, not wall-clock on H100s.

The bandwidth problem byte-level models actually have

The Byte Latent Transformer already solved the compute problem of tokenizer-free models by grouping bytes into variable-length patches and running the big global Transformer only once per patch. What it didn’t solve: the lightweight local decoder still emits one byte at a time, so generation is memory-bandwidth bound. Fast BLT, from FAIR, Stanford and UW, is a set of three tricks aimed squarely at that bottleneck.

The contribution isn’t really a new mechanism. Block diffusion was popularized by Arriola and Kuleshov’s BD3-LM, which framed block size as a continuous knob — block=1 collapses to autoregressive, block=full-sequence is pure diffusion ¹. Fast BLT ports that knob into BLT’s hierarchical decoder and pairs it with a self-speculation variant.

Two knobs, very different risk profiles

BLT-S is the result that should travel. Because verification rolls back to the first mismatched byte, the output distribution is exactly greedy BLT — the ArxivIQ review calls it a “zero-compromise” upgrade ². BLT-D is the more aggressive bet: MarkTechPost confirms the 87–92% bandwidth figure for BLT-D-16 but flags HumanEval and ARC-Easy as the benchmarks where the wheels come off ³. BLT-DV is the compromise: use diffusion to draft a block in one or two steps, then let the AR head verify.

What the proxy numbers don’t tell you

Every speedup in the paper is reported as Network Function Evaluations or estimated memory bandwidth, not wall-clock on real silicon — a caveat the authors and independent reviewers both raise ². That matters more than it sounds. A widely-cited r/LocalLLaMA thread on speculative decoding notes that on H100s, draft overhead can erase gains once the target model is already compute-bound, and that batched serving introduces a “ragged tensor” problem when users in a batch accept different draft lengths, producing latency spikes single-stream benchmarks never see ⁴. BLT-S is self-speculation, so these concerns apply directly.

The bigger barrier Fast BLT doesn’t touch

The byte-level thesis has always had two adoption costs: bespoke kernels and end-to-end pretraining on bytes. A 2025 retrofit study found that bolting BLT structure onto an already-trained Llama 3 causes significant quality loss, meaning the architecture’s wins require expensive from-scratch training ⁵. AllenAI’s BoLMo write-up adds that byte-level models can suffer “byte-level collapses” early in pretraining, producing bizarre typos for thousands of steps before recovering ⁶ — patching may help, but the failure mode is real.

Fast BLT closes the inference gap between byte-level and subword Llama-class models. It doesn’t close the training-cost gap, and it doesn’t prove the bandwidth numbers hold once an H100 serving stack gets involved.

The honest read: BLT-S is a genuine lossless win worth porting into any byte-level decoder. BLT-D is a research direction, not a deployment story.

---

ReasonMaxxer matches RL math gains for $25, not $100K

Source: hf-daily-papers · published 2026-05-06

TL;DR

• ReasonMaxxer, an RL-free LoRA recipe, matches GRPO/PPO baselines on MATH-500, AIME and OlympiadBench for $4–$25 vs. $200–$103,000.
• RL only reranks tokens at 1.0–4.1% of positions, nearly always already in the base model’s Top-5.
• A rank-32 LoRA (0.3–0.5% of params) trained with contrastive loss on high-entropy decision points recovers RL’s accuracy gain.
• NVIDIA’s ProRL is the live counterexample: on logic puzzles where base models score 0%, prolonged RL reaches ~100%.

The mechanistic claim

Akgül et al. open with a forensic teardown of GRPO- and PPO-tuned models. Compared token-by-token to their base models, RL changes the argmax at only 1.0% to 4.1% of positions, and in nearly 100% of those cases the RL-preferred token was already in the base model’s Top-5 (mean rank ≈ 2.2). An oracle experiment seals it: swap base tokens for RL tokens at only those few positions and the full accuracy gain reappears; swap at random positions and nothing happens. The whole behavioral delta between a base model and an expensive RL-tuned descendant compresses into a rank-32 adapter touching 0.3–0.5% of parameters.

That picture matches what the rest of the field has been quietly converging on. Wang et al. independently showed that restricting policy-gradient updates to the top 20% high-entropy tokens of Qwen3-32B yields +11.04 on AIME’25, while training on the low-entropy 80% degrades the model ⁷. Yue et al.’s NeurIPS 2025 paper found base models match or beat their RLVR descendants at high pass@k, suggesting RL narrows the search distribution rather than expanding it ⁸. Even Hugging Face’s GRPO writeup notes RL naturally updates only 5–30% of weights and that LoRA acts as an implicit KL anchor ⁹ — exactly ReasonMaxxer’s design.

ReasonMaxxer in practice

The recipe is almost insultingly simple: pick problems where the base model has mixed rollouts (some right, some wrong), flag tokens above an entropy threshold τ as decision points, then run an advantage-weighted contrastive loss — push up correct-rollout tokens, push down incorrect ones — while a KL term anchors everything else to the base distribution. No online generation, no value model, no reward model, no PPO clipping. Tens of problems, minutes on one GPU.

The cost gap is the headline:

On Qwen3-4B it averages 47.6% across benchmarks vs. 40.6% for General-Reasoner; on DeepSeek-R1-Distill-1.5B it lifts the base from 31.9% to 40.1%.

Where the consensus breaks

The honest caveat is ProRL ¹⁰. NVIDIA reports base models scoring 0% on certain logic puzzles even at thousands of samples, while prolonged GRPO with KL resets drives the same models to ~100%. That directly violates ReasonMaxxer’s “edge-of-competence” precondition: if the base never samples a correct trajectory, there is nothing for a contrastive loss to upweight. Akgül et al. evaluate on verifiable math, where base models do occasionally get there; ProRL deliberately picks tasks where they don’t. Neither paper engages the other’s setup.

Two other gaps to flag. The paper doesn’t benchmark against modern step-level preference methods (DPO, SimPO, SVPO) where prior work shows the RL-vs-contrastive ranking flips with scale ¹¹. And independent reproduction is missing — the public repo had a single GitHub star as of early May 2026 ¹².

“RLVR actually narrows the model’s reasoning scope by focusing on a few high-reward trajectories.” ⁸

Read ReasonMaxxer as the cleanest evidence yet that, on verifiable-reward math, RL is doing elicitation rather than teaching — and that a $25 contrastive recipe can do the same eliciting. Whether that holds outside math, where ProRL is planting its flag, is the next fight.

---

MV-Split residuals push DiTs to 1000 layers without divergence

Source: hf-daily-papers · published 2026-05-06

TL;DR

• MV-Split residuals train a stable 13.6B-parameter, 1000-layer Diffusion Transformer where vanilla Post-Norm diverges far shallower.
• At a matched 400 layers, MV-Split hits FID 2.60 / IS 185.5 at 50k steps vs LayerScale’s 2.90 / 165.5.
• The diagnosis: “Mean Mode Screaming” — a mean-coherent gradient term scaling with sequence length T, causing a 13× writer-gradient blowup.
• DeepNet already trained 1000-layer Transformers in 2022 — the headline here is the mechanism, not the depth number.

The failure mode

Solo researcher Pengqi Lu (the “erwold”/“StableKirito” accounts behind Qwen2VL-Flux) ¹³ takes a mechanistic stab at why deep Diffusion Transformers collapse. Row-stochastic attention — rows summing to 1 — perfectly preserves the mean component of token representations while contracting the centered (variance) component. Stack enough layers and the useful “spatial variation” signal decays unless the residual branch keeps refilling it.

The paper proves the gradient for residual writers splits exactly into a mean-coherent part that scales as O(T) and a centered, diffusive part. As long as tokens are heterogeneous, the mean-coherent contributions cancel across the sequence. Once tokens drift toward alignment, the cancellation stops and the mean-coherent term screams — empirically a 13× amplification of the writer-gradient norm. Worse, the Softmax Jacobian’s null space then suppresses Q/K logit gradients, locking the model in a homogenized state it cannot escape.

What MV-Split actually does

The fix is a subspace-routed residual. Instead of the usual RMSNorm(X + f(X)), MV-Split treats the mean and centered subspaces independently:

• The centered branch output is added to the trunk with a learnable gain β.
• The trunk mean is contracted by (1−α) and replenished by α times the branch’s mean — a leaky integrator, not a hard reset.

Setting α ≪ β damps the unstable mean-coherent gradient without throttling feature learning, which is exactly where LayerScale’s isotropic gating costs convergence speed. A non-obvious design constraint, buried in Appendix H: the token mean implicitly carries the diffusion timestep signal, so naive “hard centering” to zero is actively harmful ¹⁴. The leakiness is load-bearing.

For multimodal stacks, MV-Split uses segment-wise projectors so the average text token doesn’t pollute the average image token.

How much is genuinely new

The 1000-layer headline deserves an asterisk. Microsoft’s DeepNet hit 1000 layers in 2022 with DeepNorm’s depth-dependent residual scaling ¹⁵. What’s new isn’t the depth — it’s the diffusion-specific diagnosis and the fact that the fix targets the mean subspace rather than uniformly shrinking residual gain.

That distinction shows up in the numbers. At 400 layers — the matched comparison — MV-Split’s FID 2.60 / IS 185.5 tracks the pre-crash trajectory of the unstable baseline, while LayerScale plateaus at 2.90 / 165.5. At 1000 layers, the model reaches FID 2.77 / IS 217.3 on ImageNet and 0.534 overall on GenEval (92.81% single-object, but only 25.75% on positioning).

But independent reviewers flag that the 1000-layer run is a “separate scale-validation point,” not part of the matched-frontier sweep — so quality claims at that depth rest on weaker comparative footing ¹⁶. There’s also no production reason yet to want 1000 layers; this is a stunt benchmark that happens to expose a real mechanism.

What’s still empirical

The “Alignment-Amplification Law” identifies the transition into MMS but cannot predict the training step at which an un-stabilized model crashes — one digest notes the diagnostic “checks code against itself rather than intent” ¹⁷. And because the Softmax-Jacobian analysis is attention-specific, none of this is guaranteed to transfer to Mamba or CNN-based diffusers ¹⁸.

The durable artifacts here are three: the gradient decomposition, the leaky-mean residual, and a Triton kernel fusing MV-Split with RMSNorm/RoPE/SwiGLU for a 22% wall-clock win. Use those at 24 layers and you’ll probably get value. Use 1000 layers and you’ll get a tweet.

Round-ups

Listwise Policy Optimization unifies group-based RLVR methods

Source: hf-daily-papers

Group-based policy gradients in RL with verifiable rewards share a geometric structure on the LLM response simplex, which the authors exploit in Listwise Policy Optimization. LPO performs explicit target projection through divergence minimization, yielding monotonic improvement and steadier training than first-order approximations.

AutoTTS automates test-time scaling via controller synthesis

Source: hf-daily-papers

AutoTTS recasts test-time scaling as controller synthesis over reasoning trajectories and probe signals, letting LLMs discover their own inference strategies. A beta parameterization and execution-trace feedback deliver better accuracy-cost tradeoffs with little overhead beyond the base model.

MLS-Bench finds AI agents tune rather than invent ML methods

Source: hf-daily-papers

MLS-Bench evaluates whether agents can produce generalizable, scalable ML methods and finds today’s systems lean on engineering-style tuning over genuine method discovery. Bottlenecks trace to missing scientific insight rather than compute, even when test-time scaling and adaptive context are provided.

Affine recurrent models hit a hard wall on state tracking

Source: hf-daily-papers

Affine recurrent networks, including State-Space Models and Linear Attention, cannot correct hidden-state drift once representations are preserved, capping them to finite-horizon solutions. The paper formalizes a distinguishability ratio and readability threshold that explain why accumulated error, not expressive capacity, governs tracking failure.

HumanNet scales embodied video pretraining to 1M hours

Source: hf-daily-papers

HumanNet releases a million-hour, richly annotated human-centric video corpus and shows egocentric footage can substitute for robot demonstrations when training vision-language-action models. The team validates the transfer on the Magic Cobot platform, covering activity understanding, motion generation, and human-to-robot policy learning.

Paper argues the chatbot interface is reshaping AI’s social footprint

Source: hf-daily-papers

The chatbot paradigm is framed not as a neutral UI choice but a dominant sociotechnical configuration with structural downsides across legal, economic, and environmental systems. The authors push researchers to consider non-conversational interfaces that better match task structure and accountability.

Compute-anchored wages: agents shift labor pricing to GPU markets

Source: hf-daily-papers

Treating AI agents as a production technology that converts compute capital into cognitive labor, the paper derives a Compute-Anchored Wage via CES aggregation. The model predicts wage-setting moves from labor markets to compute capital markets, with sharp factor-share consequences.

Footnotes

1. Lossfunk Letters (on BD3-LM, Arriola/Kuleshov) — https://letters.lossfunk.com/p/future-of-llms-might-not-be-autoregressive

   When the block size is set to one, the model functionally collapses into a standard autoregressive model; conversely, setting the block size to the full sequence length recovers a pure diffusion model.

   ↩

2. ArxivIQ Substack review — https://arxiviq.substack.com/p/fast-byte-latent-transformer

   BLT-S is highlighted as a ‘zero-compromise’ upgrade, achieving a 77% bandwidth reduction with 100% preservation of task accuracy… the paper’s results rely on estimated bandwidth rather than raw wall-clock benchmarks on physical hardware like H100s.

   ↩ ↩²

3. MarkTechPost — https://www.marktechpost.com/2026/05/11/meta-and-stanford-researchers-propose-fast-byte-latent-transformer-that-reduces-inference-memory-bandwidth-by-over-50-without-tokenization/

   BLT-D-16 (block size 16) is the fastest variant, reaching roughly 87 to 92 percent reduction in estimated memory bandwidth versus BLT, but with a meaningful score drop on tasks such as ARC Easy and HumanEval, indicating that aggressive block parallelism trades off accuracy for speed.

   ↩

4. r/LocalLLaMA discussion on speculative decoding — https://www.reddit.com/r/LocalLLaMA/comments/1qg2592/speculative_decoding_turning_memorybound/

   On powerful hardware like the NVIDIA H100, the overhead of a large draft model can negate speed gains if the target model is already running in its compute-efficient regime… high-concurrency workloads introduce the ‘ragged tensor’ problem, where users in a batch accept different numbers of tokens, leading to GPU misalignment and latency spikes.

   ↩

5. arxiv 2509.11252 (BLT retrofit study) — https://arxiv.org/html/2509.11252v2

   ‘retrofitting’ existing models like Llama 3 into a BLT framework results in a significant performance drop, implying that the benefits of the architecture can only be realized through expensive, end-to-end training from scratch.

   ↩

6. AllenAI blog (BoLMo) — https://allenai.org/blog/bolmo

   EvaByte is noted for its extreme data efficiency, rivaling top-tier token-based models like Llama 3 while using 5x less training data… EvaByte researchers reported ‘byte-level collapses’ during early pre-training, where models generated ‘bizarre typos’ that only resolved after several thousand steps.

   ↩

7. Shenzhi Wang et al. — ‘High-Entropy Minority Tokens Drive RLVR’ project page — https://shenzhi-wang.github.io/high-entropy-minority-tokens-rlvr/

   Restricting policy-gradient updates exclusively to the top 20% of high-entropy tokens yielded +11.04 on AIME’25 and +7.71 on AIME’24 for Qwen3-32B; training on the low-entropy 80% led to severe performance degradation.

   ↩

8. ArxivIQ Substack — NeurIPS 2025 recap of Yue et al. — https://arxiviq.substack.com/p/neurips-2025-does-reinforcement-learning

   RLVR-tuned models dominate at low values of k, but base models frequently match or exceed their RL-trained counterparts when k is large (e.g., k=256 or higher)… RLVR actually narrows the model’s reasoning scope by focusing on a few high-reward trajectories.

   ↩ ↩²

9. Hugging Face blog on GRPO (NormalUhr) — https://huggingface.co/blog/NormalUhr/grpo

   RL fine-tuning is inherently sparse, naturally updating only 5%–30% of a model’s weights even when the full model is unfrozen… LoRA acts as an implicit KL regularizer, preventing the model from drifting too far from its original distribution.

   ↩

10. ProRL (Liu et al., NVIDIA), arXiv 2505.24864 — https://arxiv.org/html/2505.24864v1

    Base models often fail completely (0% success) on complex logical puzzles even with thousands of samples, while ProRL-trained models have achieved up to 100% success on these same tasks… uncovering novel reasoning strategies inaccessible to the base model.

    ↩

11. oxRL controlled study (Findings of EMNLP 2024) — https://aclanthology.org/2024.findings-emnlp.463/

    At 1.5B parameters, online RL (SGRPO) outperformed DPO on GSM8K by approximately 9 percentage points; at 7B parameters, the RL-free SimPO variant actually surpassed RL-based methods on the same benchmark.

    ↩

12. LonePatient arXiv digest (May 2026) — http://lonepatient.top/2026/05/08/arxiv_papers_2026-05-08.html

    Reports indicate the cost of training a reasoning-capable model using this method dropped to as little as $4, a three-order-of-magnitude reduction… the farukakgul/ReasonMaxxer GitHub repository is in its nascent stages, currently showing a minimal count of 1 star.

    ↩

13. hyper.ai search profile — Pengqi Lu — https://beta.hyper.ai/en/search?q=Pengqi+Lu

    Prior to the 1000-layer milestone, Lu was recognized for developing Qwen2VL-Flux, a framework that integrated the Qwen2-VL vision-language model with the FLUX architecture; the mv-split repository and 1000-layer DiT weights were released under his personal ‘StableKirito’ and ‘erwold’ accounts.

    ↩

14. arXiv HTML — Mean Mode Screaming (paper appendix) — https://arxiv.org/html/2605.06169v1

    Appendix H reveals that the token mean implicitly carries the diffusion timestep signal… ‘hard centering’ (forcing mean to zero) is detrimental because it destroys this useful global information.

    ↩

15. Wang et al., ‘DeepNet: Scaling Transformers to 1000 Layers’ (ResearchGate) — https://www.researchgate.net/publication/379742602_DeepNet_Scaling_Transformers_to_1000_Layers

    DeepNorm… bounds the expected magnitude of model updates… successfully scaled Transformers up to 1,000 layers (i.e., 2,500 attention and feed-forward sub-layers) without difficulty.

    ↩

16. ngjoo.com paper notes on 2605.06169 — https://www.ngjoo.com/papers/2605.06169/

    the 1000-layer run was a ‘separate scale-validation point’ and not part of the matched 400-layer frontier comparison, meaning its efficiency relative to smaller models remains an open research question.

    ↩

17. AI Native Foundation Daily Paper Digest (2026-05-11) — https://ainativefoundation.org/ai-native-daily-paper-digest-20260511/

    certain validation methods ‘check code against itself rather than intent,’ suggesting that while the architecture is stable, the underlying ‘regime theory’ for predicting the exact onset of MMS remains partially empirical.

    ↩

18. ChatGPT-ArXiv-Paper-Assistant digest (daizedong.github.io) — https://daizedong.github.io/ChatGPT-ArXiv-Paper-Assistant/

    the analysis of Softmax Jacobian null spaces and mean-preservation is specific to Transformer-style attention… transferability of these findings to attention-free mixers (like Mamba or CNN-based diffusers) remains to be tested.

    ↩