CorrGRPO fixes GRPO math, HC-DLM diffusion decoding, Anthropic infers UV sky

Sources

The missing map of the sky anthropic.com

CorrGRPO: Correlation-Normalized GRPO for Multi-Reward Learning huggingface.co

Group Relative Policy Optimization (GRPO) is widely used to train reasoning language models, where it computes advantages by centering and normalizing rewards across rollouts of the same prompt. For multiple rewards, GRPO sums the reward components and normalizes the total reward by its within-group standard deviation. The corresponding variance equals the sum of all pairwise reward covariances. For a fixed centered reward, larger aggregate covariance produces smaller advantages, and vice versa,

Adaptive Reward Routing: Dynamic Multi-Reward Optimization for Joint Audio-Video Diffusion via Forward-Process RL huggingface.co

Multi-reward guided reinforcement learning (i.e., RL) offers a promising way to improve joint audio-video diffusion models along several complementary objectives, including modality-specific quality, cross-modal semantic alignment, and temporal synchronization. Its effectiveness, however, depends on two quantities that change during training: where reward-driven updates should act, and how competing rewards should be combined. Existing methods tend to rely on fixed routing and reward weights, fa

Make Sparse Rewards Count: Density-Aware Reward Aggregation for Multi-Reward RL huggingface.co

Multi-reward reinforcement learning trains large language models to satisfy multiple behavioral objectives simultaneously. Reward-wise normalization, as used in GDPO, preserves reward-specific relative information within rollout groups, but different objectives can still exhibit uneven learning progress. We study this behavior through advantage energy, the sum of a reward’s squared advantages over a batch. Under idealized GDPO normalization, we show that this energy is proportional to active-gro

Hierarchical Continuous Diffusion Language Models huggingface.co

Discrete diffusion language models offer a compelling alternative to autoregressive generation for tasks demanding bidirectional reasoning and global constraint satisfaction. Yet they share a structural bottleneck: when decoding in parallel, each token is sampled independently from its marginal, severing the statistical dependencies among the tokens decoded together. Continuous diffusion language models avoid this by denoising a shared continuous state, but their denoiser sees only that state, s

E-MoE: Enhanced Mixture-of-Experts for Non-Factorized Diffusion Language Models huggingface.co

Masked diffusion models (MDMs) generate sequences by progressively unmasking several tokens per denoising step, but their reverse process is typically factorized over positions, limiting sample quality in the few-step regime where diffusion’s speed advantage over autoregressive decoding matters most. A recent line of work introduces a continuous Gaussian latent, trained as a variational autoencoder, to capture correlations across positions, but such approaches are prone to posterior collapse, wh

Scaling and Distilling Text Embeddings for Better Diffusibility huggingface.co

Diffusion language models (DLMs) offer a promising alternative to autoregressive (AR) language generation. Recent advances in continuous DLMs, which apply latent diffusion to continuous text embeddings, raise a practical question: which embedding makes the best latent space, i.e., the most diffusible? To answer this, we search through different embeddings and find that scaling the embedding model to stronger ones within the same family (T5 to T5Gemma-1 to T5Gemma-2) greatly improves generative p

Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It huggingface.co

Pretrained transformers burn only a fraction of their depth tracking in-context references, following just 1.4-3.6 lines across 13 base models. Inserting a task-trained rank-8 LoRA at one early layer, with all other weights frozen, extends Qwen3-8B to 24-line chains at 99% exact accuracy and Ouro-1.4B past 160 lines after eight loops.

Sharpening Tax in Post-Training huggingface.co

The sharpening hypothesis says RL post-training trades solution coverage for single-shot accuracy, a pattern seen in math and code. New work finds the same tax holds in agentic tool-use settings, where pre-trained LLMs equipped with scaffolding already cover many behaviors that post-training then narrows rather than adds.

AutoDataBench: A Data-centric Testbed for Accelerating Auto Research huggingface.co

Auto-research benchmarks mix training frameworks, hyperparameters, compute, and data, muddying why one agent beats another. AutoDataBench strips those confounds to test data intelligence alone: an agent’s ability to understand, manipulate, and improve the datasets that shape model capabilities, built on a controlled testbed with fixed training recipes.

Pretrain Once, Route Anywhere: Towards a Foundation Model for LLM Routing huggingface.co

Today’s LLM routers are fit locally to one query workload and candidate pool, needing retraining when either shifts. RouteFM reframes routing as a foundation-model problem, pretraining once so a single router assigns queries to the best model across heterogeneous pools without extra supervision when the environment changes.

Beyond Memory: Harnessing Long-Horizon Agents with Explicit Belief States huggingface.co

Long-horizon agents store interaction history as raw memory, which rarely yields a coherent view of the current world. PoS maintains an explicit belief state at inference, fusing an estimate of world state with unresolved task requirements so the agent tracks what it still needs to learn and act on.

When Users Change Their Minds: Measuring and Repairing Intent Drift in LLM Agents huggingface.co

When users change their minds mid-dialogue, superseded intent keeps leaking into agent actions. IntentFlux converts verifiable tasks into dialogues with controlled intent changes while reusing original graders; across 627 calibration cases, mean task score drops from 0.476 to 0.384 once drift is introduced.

World Observer: Joint Actor-Observer Generation for Persistent World Modeling huggingface.co

Video world models stay actor-centric and lose track of objects once they leave view, failing to restore state on re-entry. World Observer jointly generates an agent-centric actor perspective with a separate observer stream, letting the model keep watching regions beyond the actor’s current field of view.

On-Policy or Off-Policy Learning? A Systematic Study of Distillation Dynamics huggingface.co

On-policy learning has been argued to reduce catastrophic forgetting, produce sparser parameter updates, and improve generalisation. However, existing comparisons between supervised fine-tuning and reinforcement learning vary many factors simultaneously, making the contribution of rollout policy difficult to isolate. We study the effect of rollout policy in a controlled strong-to-weak distillation setting, by independently varying rollout policy, token-level KL direction, and learning rate acros

Smaller Models, Better Rejects: Preference Distillation Scaling huggingface.co

Preference distillation typically treats a teacher response as preferred and the student’s own response as rejected. This assumes that self-generated failures are the most informative negatives and that rejects must come from a model at least as large as the student, making generation costly at scale. We find neither assumption holds: across students from 7B to 72B, smaller frozen models generate rejects with less inference compute yet train stronger students than self-generated rejects, before

OneStreamer: Unifying Perception, Memory, and Proactive Response in Streaming Video Interaction huggingface.co

Streaming video LLMs must retain evidence before its relevance to future tasks is known and respond when sufficient evidence becomes available. The challenge is to form reusable factual memory without compromising real-time perception. We introduce OneStreamer, which jointly learns query-independent evidence recording and task response through a shared proactive generation process. Its Proactive Hierarchical Caption Memory (PHCM) produces time-grounded local-detail captions and summaries of comp

GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis huggingface.co

Working agents need to read diverse files, coordinate tools, and produce deliverables. Training such agents requires tasks built on many real files with verifiable results, but few pipelines exist to synthesize this kind of data. Existing pipelines either generate files with models, which lack realism and diversity, or build tasks on real files without task-specific verifiers, leaving result quality unchecked. We introduce GraphForge, an evidence-graph based framework that grounds both the task

Predictive Credit: Measuring What Scientific Explanations Add to Experimental Forecasts huggingface.co

Research agents explain planned experiments. We measure predictive credit with paired forecasts sharing an intervention, forecaster, and outcome while varying description, matched explanation, and donor context. Five checks track commitment, delivery, predictive gain, alignment, and known-signal uptake. Across 336 prospective states in controlled learning, 12 Tox21 endpoints, and 24 OpenML tasks, v5’s frozen credit decision was inconclusive. Tox21’s preregistered ROC AUC interval-score harm test

Prefill-Free Cross-Family KV Cache Transfer for Heterogeneous Multi-Agent LLMs huggingface.co

Recent multi-agent LLM systems increasingly combine heterogeneous models for specialized agent roles. However, text-based communication requires each receiver to prefill shared context already processed by the sender. Reusing the sender’s key-value (KV) cache avoids this redundancy, but prefill-free transfer across model families must handle differences in tokenization, model depth, and KV representations. To address these issues, we propose HeteroFold, a prefill-free cross-family KV cache trans

ActiveSaddler: Automated Curriculum Learning for Agent Harness Optimization huggingface.co

Automated harness optimization can substantially improve LLM agents by iteratively updating their prompts, tool interfaces, and control logic from execution feedback. However, existing methods primarily optimize how the harness is updated while largely fixing which training scenarios generate the feedback that drives those updates. As the harness evolves, the scenarios most useful for further optimization can change, suggesting that the training curriculum itself should adapt alongside the harne

X-Tree: Tokenizing Reusable Experience for Efficient Agent Generalization huggingface.co

Multi-step agents are trained on flat action streams: SFT and RLVR weight every token uniformly and ignore the sub-procedures that recur across tasks, the hierarchy that lets humans plan top-down from reusable routines. This structure sits unused, and flat training uses each scarce trajectory less fully than its content allows. Recent agents do use that structure, but only as LLM-written skills in context, never in the weights, so their gains do not generalize beyond retrieval. We instead recove

A Missing Piece for Trustworthy AI Reviewers: From Benchmarking Rhetorical Robustness to SciCore Review huggingface.co

AI reviewers can assign different judgments to manuscripts that report the same science in different wording, potentially rewarding rhetorical optimization over scientific improvement. We formulate Rhetorical Robustness as the joint requirement of stability across content-preserving rewrites and discrimination across papers. We introduce RobustReview, a controlled full-manuscript benchmark with 1,260 manuscript versions, and evaluate 30 reviewer configurations. The benchmark reveals false robust

Decentralized Master-Mind: Joint Action Refinement through Iterative Intent Denoising in Multi-Agent Pathfinding huggingface.co

Decentralized multi-agent path finding (MAPF) with communication requires agents to reach individual goals without collisions under partial observability. Learnable policies trained on expert data provide an effective approach to this problem. However, when several coordinated joint actions are valid in the same context, independently sampling from per-agent distributions can recombine locally valid choices into incompatible joint actions. This failure can arise from the final sampling mechanism

AutoGUIWorld: Image Generators as Visual World Models for GUI Agent huggingface.co

GUI agents require high-quality interaction trajectories to learn how software environments respond to actions, maintain state, and support multi-step workflows. However, the diversity of available trajectories is constrained by the applications, interface states, and workflows accessible in the underlying environments. Expanding this coverage requires deploying increasingly diverse and complex software, with specialized applications imposing additional installation, configuration, and runtime c

Fewer Tokens, Better Action: GPT-6 Astra Robot Agents with 14% Higher Success Rate but 65% Fewer Tokens huggingface.co

Vision language model (VLM) agents can control robots through visual feedback and action primitives, but repeated model invocations and redundant observations incur substantial token overhead. We introduce PyRUA-Lean, an interactive code-execution framework that couples feedback-driven primitive composition with selective observation: the agent composes classical robot primitives and learned vision-language-action (VLA) policies into Python cells that perform conditional checks and local retries

Decoding Looped Transformers Better for (Almost) Free huggingface.co

Looped Transformers achieve parameter efficiency by repeatedly executing a shared block across recurrent loops. Each loop yields an intermediate representation decodable for the same next token, yet standard decoding discards earlier states. Because earlier loops embody less computation, recurrence inherently supplies aligned weak-and-strong prediction pairs without auxiliary models or external training. We introduce LoopCD, a training-free contrastive decoding framework that guides token select

4Director: Controlling Video World Models with Rigid 3D Geometry huggingface.co

Precise control over camera and object motion is essential for professional video production. Existing methods control objects only coarsely, through image-plane cues that are ambiguous in depth and rotation or through 3D tracks and blobs that lack complete geometry and lose consistency across viewpoint changes. We introduce 4Director, a video world model conditioned on an explicit 4D scene representation: each object is reconstructed once from the input image as a canonical mesh and moved by on

Multimodal Flow: Unified Flow Modeling of Language and Vision in Embedding Spaces huggingface.co

We present Multimodal Flow, a fully continuous generative model of language and vision. Most unified multimodal models either model both language and quantized images as discrete tokens or combine discrete language prediction with continuous image generation. The former introduces a visual quantization bottleneck. The latter requires modality-dependent objectives and sampling procedures. Fully continuous modeling avoids these trade-offs and enables a shared generative process, but remains undere

Argo-Bench: Evaluating Data Agents on Enterprise-Scale Workflows huggingface.co

Real-world enterprise data science and analytics workflows require reasoning across dozens of tables, performing statistical analyses, and acting on the results. Established text-to-SQL benchmarks evaluate query generation alone, and audits have found their answer keys frequently wrong. Because real enterprise warehouses are too sensitive to release, these benchmarks are built on public datasets where a business event fits in a single table. We introduce Argo-Bench, an evaluation framework compr

Benchmarking and Enhancing Skill-Level Memory for Partially Observable Robotic Manipulation huggingface.co

Recent advances in robot learning have enabled manipulation policies to perform increasingly diverse tasks and generalize across environments. However, reliable execution often depends on hidden task states that cannot be determined from current observations alone, making interaction history essential. We introduce HIDE, a benchmark for evaluating manipulation memory under partial observability. HIDE comprises 15 tasks covering repetition counting, historical-state recall, and execution-progress

Omni-Embed-Mini: Binding Modalities Without Forgetting via Dense Distillation huggingface.co

Extending a text embedding model to new modalities typically degrades text retrieval quality, and existing omni-modal embedders compensate with multi-billion parameters. We present Omni-Embed-Mini, a 0.9B-parameter model that maps text, speech, audio, images, video, and visually-rich documents into a single shared cosine space without updating any text-side parameter. Our key insight is that the teacher signal requires no separate embedding model: each media sample is paired with a dense cascade

Agent Priors-guided Policy Learning huggingface.co

Robots that learn from a few demonstrations often require two forms of generalization. Compositional generalization recombines skills to solve new tasks, and skill generalization lets the learned policy behind each skill work in new situations. The two depend on each other, yet information is lost between composition and the skills it calls. Where a skill works is determined by the structure its policy is trained with, while composition sees the skill only through a separate description, such as

ROWBench: Do Video Models Render What the Program Specifies? huggingface.co

Programmable world models separate executable dynamics from visual generation, offering a promising foundation for next-generation game engines. However, their visual adherence to explicit rules and interactions remains insufficiently evaluated. Existing benchmarks assess visual quality, controllability, and instruction or physical adherence, but rarely test fidelity to fine-grained, program-specified world events. We introduce PROWBench, comprising 170 programmatically constructed episodes and

Retrieval-Augmented Skill Optimization via Cross-Harness Adaptation huggingface.co

An agent skill is a reusable, actionable natural-language artifact that guides an agent to perform a task effectively under a given harness. Recent studies have explored the optimization of agent skills, contributing to a growing collection of publicly available skills spanning diverse tasks, domains, and harnesses. Despite millions of publicly shared skills, existing skill optimization methods largely overlook this accumulated knowledge, instead relying solely on expensive agent rollouts to ite

InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulation huggingface.co

We study test-time evolution for humanoid loco-manipulation: solving tasks that a controller was never trained for by repurposing its existing skills, improving from its own attempts, and retaining what it learns, without retraining. Our key insight is that a broad controller already holds much of the competence a new task needs, and that this competence becomes accessible through an interface between planning and control that is expressive enough to specify contact-rich, multi-stage interaction

Persona Dosing: Calibrated Activation Steering for Graded Trait Control huggingface.co

An activation-steering coefficient sets intervention strength, but requesting a particular degree of persona expression requires a behavioral scale. We study persona dosing: controlling a language model through a trait description and a requested mean intensity. PersonaDose specializes a shared, description-conditioned FLAS controller on persona responses, then calibrates its flow time against measured trait expression. Training responses are not paired with requested target intensities. Across

PhysVista: Benchmarking Physical Intelligence in VLMs via a Perception-Reasoning-Assessment Loop huggingface.co

Vision-Language Models (VLMs) have shown strong multimodal reasoning capabilities, yet whether they truly capture the physical consistency underlying real-world dynamics remains unclear. Existing benchmark paradigms often suffer from fragmented evaluation, focusing on isolated cognitive stages while overlooking the inherent synergy between perception, reasoning, and physical judgment. The lack of a holistic perspective limits the ability to diagnose whether VLMs can reliably evaluate the physica

Ego2Act: Evaluating Goal-Directed Manipulation in Egocentric Video Generation huggingface.co

Video generation models are increasingly being explored as world simulators for embodied planning and learning. To do so effectively, these models must not only generate visually appealing frames, but also predict how environments dynamically evolve when executing goal-directed actions. While evaluating these capabilities is crucial, existing benchmarks focus mainly on single short actions or step-by-step instructions. This leaves multi-step physical reasoning underexplored, especially in egocen

Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes huggingface.co

On-policy self-distillation has recently emerged as an effective approach for improving language-model reasoning by supervising students with a frozen or EMA version of themselves that receives privileged information. Its application to multimodal large language models (MLLMs), however, remains largely unexplored. Recent approaches use privileged visual information, such as image crops corresponding to a question, to improve fine-grained perception, but their gains are confined to tasks that ben

OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning huggingface.co

We present OmniSeek, an agentic framework that transforms an Omni Large Language Model (Omni-LLM) into an active, multi-turn reasoning agent with native tool use. Rather than passively processing an entire audio-visual sequence in a single forward pass, OmniSeek makes evidence acquisition part of the reasoning process: it dynamically decides whether to look or listen, and over which temporal window, to retrieve sparse but critical evidence across different modalities within long contexts. Throug

Video Generation Models: A Survey of Post-Training and Alignment huggingface.co

Video generation has rapidly progressed from short, low-quality clips to high-resolution, long-duration sequences with complex spatiotemporal dynamics. Despite strong generative priors learned through large-scale pretraining, pretrained video models often fail to reliably follow human intent, maintain temporal coherence, or satisfy physical and safety constraints. Compared with image and text generation, alignment in video generation presents unique challenges, including error accumulation over

Better Supervision Is Nearby: Neighborhood On-Policy Self-Distillation huggingface.co

On-policy self-distillation (OPSD) trains mathematical reasoning models using a privileged teacher that sees a reference solution and supervises student-sampled prefixes. Standard OPSD uses one fixed parameter setting at every state, but nearby settings may offer additional supervision. We find that local parameter perturbations reveal complementary reference-aligned corrections under the same reference context. Different experts supply these corrections at different reference positions. Their p

PixelDense: Dense Prediction as Representation Alignment for Pixel Diffusion huggingface.co

Representation alignment (REPA) accelerates diffusion transformer training, but its alignment targets are almost exclusively semantic encoders such as DINOv2 and CLIP. Recent analysis points to spatial structure, not global semantics, as the carrier of the alignment effect, yet dense-prediction foundation models trained to predict that structure remain overlooked as REPA targets. In pixel-space diffusion, SAM2, Depth Anything v2, and Metric3D v2 each outperform the DINOv2-only GenEval baseline,

Beyond the Current Scene: Event-Referential Grasping with Active View Selection huggingface.co

A robot that observes people interacting with objects should be able to carry out later requests that refer back to those interactions. Such requests may specify a grasp target by the role it played in a past event rather than by its name or appearance. Moreover, the target may no longer be visible when the robot is asked to act. We present BeyondSCe, a zero-shot robotic grasping system for this event-referential setting. Given the event history and the current scene, the system identifies the r

EgoTools: Towards Tool-Centric Reasoning in Real-World Egocentric Videos huggingface.co

Real-world embodied tasks, from everyday activities to professional procedures, require agents to act under physical constraints while tracking evolving object and task states. Tool use sits at the heart of such tasks, as many everyday and professional activities are tool-mediated. Understanding them requires reasoning about affordances, hand-tool-object geometry, procedural progress, and causal effects on target objects. Yet despite strong performance on perception-oriented video tasks such as

SILSA: Sliding-Window Slice Latents for Topology-Preserving High-Resolution 3D Generation huggingface.co

High-resolution 3D generation increasingly relies on voxel latents and multi-stage pipelines that first predict active structure and then synthesize local geometry. While effective, this design fragments continuous surfaces into many local tokens, inflates generation cost, and often weakens topological consistency for thin or highly connected shapes. We introduce SILSA, a topology-aware 3D generation framework that represents shapes with compact sliding-window slice latents. Instead of generatin

RPTune: Learned Context Curation for LLM Catalog Search huggingface.co

For small merchant businesses (SMBs) whose catalogs fit within a long-context LLM, full-catalog prompting offers a compelling alternative to multi-stage retrieval designed primarily for large marketplaces with millions of items. However, fitting the full catalog into the context window does not ensure that the model can use it effectively, since LLMs do not exploit long contexts uniformly. We therefore study in-context catalog search through two complementary questions: (1) how to curate and pre

DataMagic: Authoring Data Videos through Declarative Multi-Agent Orchestration huggingface.co

Data videos communicate data insights through dynamic charts, voice narration, and synchronized animations, and have become a widely adopted form of data storytelling. However, producing them requires expertise in data analysis, narrative design, and video editing. Static visualization tools lack narrative and animation capabilities; authoring tools rely on pre-prepared charts rather than raw data; and pixel-level models generate videos end-to-end but cannot guarantee data accuracy or provenance

Architect-Ant: Editable Automatic Furnishing of Architectural Floor Plans huggingface.co

An automated furniture arrangement system using a specialized dataset and vision-language model with procedural reasoning and preference optimization to generate valid and plausible room layouts.

Align Then Reason: A Multimodal Lip-Sync Judge for Dubbing huggingface.co

Dubbing quality control requires a reference-free judge that can determine whether a candidate text line matches a speaker’s visible articulation in both content and timing, using only silent video and text because dubbed audio may not yet exist. Existing visual speech recognizers and video-language models are poorly suited to this setting: even when fine-tuned to recover spoken content from lip motion, they remain largely insensitive to temporal errors. We introduce Align Then Reason (ATR), a m

References

SuperPowerDaily superpowerdaily.com

approximately one-third of the map is statistically predicted rather than observed… successfully reconstructing observed sky does not guarantee accuracy in unobserved areas

CellCog.ai cellcog.ai

AI agent loops can share ‘blind spots’ with the models that produced the work, making domain-expert review essential — Ménard identified faint circular artifacts that two rounds of automated AI agent reviews had failed to catch

AlphaSignal alphasignal.ai

the project remains unpublished in peer-reviewed form and the full technical paper is still in preparation… its utility as a research dataset depends on future independent benchmarks against new UV observations

arXiv 2507.16385 (STAR benchmark / Flux Error) arxiv.org

deep learning models frequently prioritize perceptual quality over physical accuracy… novel metrics such as Flux Error (FE) ensure that reconstructed images do not just look ‘clean’ but remain mathematically faithful to the original photometry

Christopher Go (planetary imager, Facebook post) facebook.com

AI-driven processing can create ‘fake details’ that compromise planetary research — ‘less is more’

arXiv — ULTRASAT mission status 2026 arxiv.org

ULTRASAT is currently providing an all-sky NUV image more than ten times deeper than the previous GALEX map, reaching >23.5 AB mag

NVIDIA GDPO project page (nvlabs.github.io) nvlabs.github.io

GDPO decouples the normalization of individual reward components before aggregation, preventing the advantage collapse that occurs when distinct reward combinations map to identical advantage values in standard GRPO.

GoPenAI blog — ‘GDPO: The hidden flaw in multi-reward RLHF’ blog.gopenai.com

GDPO improves AIME accuracy by 6.3% while reducing excessive response lengths by 80%, and raises correct-format ratio on the Berkeley Function Call Leaderboard by roughly 4 points over GRPO.

RoboticsCenter.ai summary of DARA (Make Sparse Rewards Count) roboticscenter.ai

DARA applies an inverse-square-root density correction so that sparse reward components contribute comparable ‘advantage energy’ to dense ones, reaching format-compliance targets in up to 26% fewer steps than GDPO and length-compliance in up to 65% fewer steps.

Medium — Jenwei, ‘Evolution of Policy Optimization: GRPO, DAPO, Dr. GRPO’ medium.com

Dr. GRPO removes the standard-deviation denominator entirely because dividing by group std introduces a length and difficulty bias — groups with low variance get inflated updates and the model learns to produce longer, verbose answers even when wrong.

DeepMind blog — PopArt (2018) deepmind.google

PopArt adaptively rescales value targets using running mean and standard deviation while correcting the output layer weights so that unnormalized predictions are preserved — enabling a single agent to learn 57 Atari games with wildly different reward scales.

Hugging Face blog — ‘MAD-GRPO’ (javadtaghia) huggingface.co

Replacing standard deviation with Median Absolute Deviation gives a more robust group scale that is less sensitive to outlier rollouts, mitigating the exploding-update problem seen when one sample in a group dominates the variance estimate.

Sander Dieleman blog, ‘Continuous diffusion language models’ (Aug 2026) sander.ai

Continuous diffusion language models were held back by a habit, not a limitation — the practice of pulling trajectories back to token space at every step.

Ken Ashe, ‘What HC-DLM fixes about diffusion language models’ kenashe.ai

HC-DLM should be read as a mechanism paper rather than a production-ready model; the scaffold fix addresses a real failure mode in parallel decoding, but its scalability to billion-parameter models remains unproven.

AI Weekly brief on E-MoE aiweekly.co

E-MoE repurposes expert-routing decisions as a discrete shared latent variable, letting all positions condition on the same routing choices and avoiding the posterior collapse that plagues continuous-latent VAE approaches.

alphaXiv page for ‘Scaling and Distilling Text Embeddings for Better Diffusibility’ alphaxiv.org

As embeddings become more discriminative through scaling they become less diffusible; soft-label distillation from a T5Gemma-2 teacher pulls semantically similar words closer and cuts generative perplexity by roughly 40% versus the ELF baseline.

r/LocalLLaMA discussion, ‘Why are diffusion-encoder LLMs not more popular?’ reddit.com

Diffusion-encoder models theoretically solve the Goldilocks problem of context-window noise, but they have historically struggled with discrete token mapping — the distilled-embedding trick looks like a trade secret for scaling non-autoregressive models.

OpenReview, CCDD (Coevolutionary Continuous Discrete Diffusion) openreview.net

CCDD concatenates an embedding chain and a discrete-token chain denoised in parallel; the dual representation roughly doubles input sequence length and substantially increases memory and FLOP demands at both training and inference time.