Qwen3.8-Next matches 397B, OPSA drops the teacher, CAST-8B beats 120B

Sources

On the Design of Qwen3.8-Next Architecture: Evaluation, Efficiency, and Training Stability huggingface.co

Qwen3.8-Flash-Next is a sparse mixture-of-experts architecture combining hybrid gated delta-net and sparse attention layers, gated residual branches, and off-accelerator n-gram embeddings to improve efficiency, capability, and training stability.

Does On-Policy Distillation Really Distill? From Noisy Teacher to Self-Improvement huggingface.co

On-policy distillation relies mainly on suppressing low-probability tokens rather than teacher guidance, motivating a supervision-free entropy-adaptive method that substantially improves reasoning performance.

CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents huggingface.co

CAST improves LLM agent reliability by generating structured action-level rationales from sparse outcomes to train critique and policy models.

WebWorld: The Browser as a World Model for Self-Improving Web Code huggingface.co

WebWorld treats a live browser as an executable world model, letting a vision-language model verify its own HTML fixes through interaction contracts before exporting them as supervised fine-tuning data. The loop is evaluated on HTMLBench-400 and MiniAppBench-Val, closing the certification gap for autonomous web code repair.

Scaling Large Reasoning Models beyond Human Supervision: A Path toward Superintelligence huggingface.co

The paper organizes post-RLHF training into a structured ladder built on reusable verifiers, self-generated curricula, and autonomous co-evolution, then names the failure modes that block progress: reward hacking, feedback drift, and curriculum collapse across policy capability, feedback fidelity, and experience quality.

MNIST-PRO: MNIST is Back as a Partially Observable World for AI Agents huggingface.co

The benchmark forces agents to reconstruct digits from fragmented glimpses, isolating perceptual-state construction from reasoning. Frontier multimodal models stumble at integrating glimpses over time, deciding when to keep exploring, and revising wrong beliefs, exposing weak working memory rather than weak perception.

Normalized Low-Rank Adaptation huggingface.co

Normalized Low-Rank Adaptation applies a normalization step to LoRA’s down-projection matrix, accelerating convergence and lifting accuracy without adding parameters or inference cost. The authors also report reduced catastrophic forgetting and gains in reinforcement learning fine-tunes, with code released on GitHub.

Verification-Aware Training for Speculative Decoding huggingface.co

The method simulates the sequential verification loop during draft-model training and reweights the loss toward tokens likely to be accepted. Applied on top of EAGLE-3 and DFlash, it raises acceptance rates and end-to-end speedups without changing the target LLM or inference stack.

The Safeguard Worked. Is the LLM System Safer? huggingface.co

Local scores like refusal rate, attack success rate, and policy violation rate can improve while residual real-world harm stays flat. The authors propose evaluating guardrails against deployment context and downstream risk, not in isolation, to avoid safety theater in production LLM systems.

Chain-of-Thought Faithfulness of Reasoning Models Varies with Where and How Preference Cues Are Delivered huggingface.co

Across open-weight reasoning models, preference cues delivered through tool returns or implicit artifacts produce lower verbalized commitment and higher unverbalized adoption than cues stated in the prompt. Transcript-based CoT monitors miss these hidden influences, weakening a key AI-safety assumption about reasoning models.

Learning to Evaluate Before Improving: Automatic Rubric Induction for Automatic Research Agents huggingface.co

AutoSciRub improves autonomous scientific agents by generating task-specific executable rubrics that guide experiments, verify criteria, and iteratively refine outputs.

PaperGym: Rubric-Centered Evolution for Research-Plan Generation huggingface.co

PaperGym converts scientific papers into training environments by separating research questions from evaluation rubrics, enabling reinforcement learning that improves research planning across multiple model sizes.

Dynamic Important Example Mining for Reinforcement Finetuning huggingface.co

DIEM adaptively selects and reweights training examples during reinforcement fine-tuning by estimating each sample’s marginal contribution to policy improvement and constraining batch updates to stabilize optimization.

Evaluating the Hidden Costs of Personalization in Large Language Models huggingface.co

The study proposes PRISK, a framework that reveals how personalized context in LLMs increases irrelevant personalization, preference narrowing, and sycophantic bias.

SafeAtlas-VL: Beyond Binary Multimodal Safety with Large-Scale Data and Guard Models huggingface.co

SafeAtlas-VL introduces a large-scale multimodal safety dataset with graded risk labels and trains guard models that predict continuous safety scores, achieving state-of-the-art generalization.

CogEvol: Towards Efficient and Reliable Learning Environment Generation huggingface.co

CogEvol is a family of models that generate structured learning artifacts in a single pass using supervised fine-tuning and reinforcement learning with vision-language rewards, achieving high quality with far fewer parameters and lower cost.

EvoGenUI-Bench: Evaluating LLMs as Multi-Turn Generative UI Assistants huggingface.co

Large language models can generate interactive web interfaces, but reliable generative UI requires maintaining an executable artifact as user requests evolve. We introduce EvoGenUI-Bench, a benchmark for multi-turn interface maintenance comprising 150 five-turn tasks and 750 turns across three scenarios: information presentation, executable interaction, and tool-grounded external state. We execute generated artifacts in a browser and evaluate them using screenshots, source and DOM evidence, acto

CoVA-SFT: A Large-Scale Dataset for Chain of Visual Abstractions huggingface.co

CoVA-SFT is a multimodal reasoning dataset that teaches language models to interleave text and visual abstractions via structured reasoning steps, improving performance on visual reasoning benchmarks.

Super Library Agent: Joint Generation and Maintenance of Multiple Applications Beyond the Single Codebase huggingface.co

A shared-library agent framework reduces redundancy and structural erosion when sequentially generating related applications by guiding cross-application code extraction and migration.

Matrix-Game 3.5: Enhancing Real-Time Streaming Interactive World Models with Patch Memory huggingface.co

Matrix-Game 3.5 improves interactive world generation via geometry-aware memory, static-dynamic disentanglement, and progressive distillation for long-horizon real-time simulation.

GenFirst: Generation Before Reconstruction for Stable End-to-End Latent Generative Modeling huggingface.co

Direct end-to-end training of latent generative models avoids collapse via entropy preservation and asymmetric dynamics, using a generation-first strategy to achieve state-of-the-art image synthesis and unified multimodal generation.

Scaffolding Foundation Models into Physical-World Agents Pushes the Frontier of Long-Horizon Navigation huggingface.co

NavMCP integrates vision-language reasoning with navigation execution via structured collaboration channels to enable persistent long-horizon embodied exploration without retraining.

SpanCalib-VLM: Calibrated Hallucination Span Detection in Vision-Language Models huggingface.co

A hybrid system combining a multimodal sequence tagger and a generative vision-language model improves hallucination span detection and calibration through union-calibrated fusion.

DICS: Exploring Data Intrinsic Consistency for Visual Instruction Selection huggingface.co

DICS selects high-quality visual instruction data by measuring internal sample consistency, improving model performance with far less training data.

PaperBanana-Interact: Scientific Diagram Refinement with Multi-Turn Human Feedback huggingface.co

A multi-turn benchmark and multi-agent system improve scientific diagram generation by reducing quality drift and feature forgetting across revision turns.

LightNav-0: Eliciting VLM Spatial Intelligence for Generalist Embodied Navigation huggingface.co

LightNav-0 is a compact generalist navigation model that leverages a pretrained vision-language model’s spatial reasoning via unified pointing tokens and action tokenization to achieve state-of-the-art embodied navigation across diverse tasks and robots.

DreamX-Creator: Democratizing Native Audio-Video Generation at 2K Resolution huggingface.co

A compact 7B native joint audio-video generator uses cross-modal attention, progressive joint training, reinforcement learning with multimodal feedback, and an autoregressive 2K refinement pipeline to produce synchronized high-resolution outputs.

RECAP-Forcing: Retaining Content Appearances for Long Video Generation huggingface.co

RECAP-Forcing improves long video generation by indexing memory according to appearance novelty rather than recency, preserving key-value caches for newly visible content to maintain long-range consistency without extra training.

MMMMM: A Unified Taxonomy for Investigating the Mechanisms of Multilingual MultiModal Misinformation huggingface.co

Multimodal misinformation on social media is highly prevalent, potent, and harmful, yet difficult to detect and counter, and still poorly understood compared to its text-only counterpart. Research on the properties and deceptive strategies of multimodal misinformation is hindered by a lack of taxonomies grounded in real-world contexts and by the limitations of current multimodal machine learning models, which prevent the automation of annotation and analysis at scale. We address these shortcomin

Keep-or-Drop? Adaptive Tokenizer for Compact Video Representation huggingface.co

KATok is an adaptive transformer-based video tokenizer that selectively drops uninformative tokens to achieve data-dependent compression while preserving spatial consistency for diffusion-based generation.

Weaving Visual Narratives: Agentic Image Bundle Composition Beyond Atomic Visual Matching huggingface.co

Image Bundle Composition reframes retrieval as dynamic assembly of relationally coherent image groups, with a benchmark and agentic framework addressing combinatorial joint relevance.

Lucida: Parse, Generate, and Place for Composable Real-to-Sim Scene Modeling huggingface.co

Lucida improves composable indoor scene reconstruction by distributing pipeline requirements across parsing, asset generation, and VLM-guided placement to achieve high-fidelity editable replicas from cluttered captures.

Lies We Can See: Joint Verbal and Non-Verbal Deception by VLM Agents in Embodied Social Interactions huggingface.co

MineAmongUs introduces a 3D multimodal Among Us environment and the ARIA harness to study embodied VLM-agent deception through verbal and non-verbal actions, revealing non-verbal channels as key to winning.

ContextBias: Controlled Evaluation of Bias Persistence Under Context Shift in Text-to-Image Models huggingface.co

Text-to-image models exhibit persistent role-linked visual stereotypes across varied contexts, with cross-role attribute concentration increasing rather than diminishing under unrelated prompts.

Cross-lingual Functional Vectors for Emotion Detection in Large Language Models huggingface.co

Function vectors extracted from one language improve multilingual emotion recognition by capturing language-independent task signals and reducing inference overhead.

Chat-Edit-3D++: Interactive 3D and 4D Scene Editing via Large Language Models huggingface.co

CE3D++ enables flexible dialogue-driven 3D and 4D scene editing by decoupling 2D editing from 3D reconstruction via a Hash-Atlas network and using a large language model to autonomously invoke visual tools.

BLARM: Animating 3D Objects from Video via Blending Latent Rigid Motion Primitives huggingface.co

BLARM predicts temporally coherent 3D mesh animations from monocular video using learned rigid motion components and skinning weights without explicit rigs.

SHAPE of Chain-of-Thought in Math Reasoning huggingface.co

SHAPE analyzes chain-of-thought reasoning via semantic spaces and heuristics to diagnose LLM mathematical reasoning and improve post-training.

References

Vizuara Substack — Kimi Linear analysis vizuara.substack.com

Kimi Delta Attention upgrades Gated DeltaNet with channel-wise gating so every feature dimension has its own gate, avoiding the coarse scalar decay Qwen3-Next inherits from vanilla GDN.

NVIDIA Research — Gated DeltaNet-2 research.nvidia.com

Gated DeltaNet-2 decouples the erase and write operations into separate channel-wise gates, addressing a ‘scalar tie’ that limits memory-editing precision in the original Gated DeltaNet used by Qwen3-Next.

Sebastian Raschka — Qwen Sparse Attention gallery sebastianraschka.com

Linear-attention variants like GDN are fast but inherently weaker at exact recall than softmax attention; QSA’s micro-block indexer still carries a residual quadratic component that may bottleneck at very long sequences.

MarkTechPost — FlashQLA release coverage marktechpost.com

FlashQLA achieves 2–3× forward and ~2× backward speedups over the Flash Linear Attention Triton kernel on H200s via gate-driven intra-card context parallelism and warp-specialized kernels tuned for Hopper/Blackwell.

MindStudio — Qwen3.8-Flash-Next benchmarks mindstudio.ai

Flash-Next scores 62.5% on SWE-bench Pro (vs Claude 4.6 Max at 53.4% and predecessor 3.7-Plus at 55.8%) and 58.7% on DeepSWE 1.1—roughly 4× the 16.5% of Qwen3.7-Plus.

Introl — DeepSeek Engram conditional memory writeup introl.com

DeepSeek’s Engram and Qwen3.8-Flash-Next both offload >50B-parameter n-gram tables to host memory, treating ‘conditional memory’ as a distinct scaling axis from MoE ‘conditional computation’, with <3% throughput penalty from prefetching.

data-pilot.com on Thinking Machines’ Tinker data-pilot.com

On-Policy Distillation… involves the student generating its own ‘rollouts’ which are then graded by a teacher model… using a reverse-KL divergence objective, providing dense, token-level supervision, helping student models like Inkling-Small achieve high-tier reasoning performance (reaching 70% on AIME’24) at roughly 10% of the compute required for traditional RL.

Lp-Reg paper (arXiv 2510.03222) arxiv.org

Lp-Reg specifically targets the preservation of ‘reasoning sparks’ — valuable low-probability tokens… Standard GRPO often suppresses these tokens as ‘bad actions’… while baseline entropy-control methods collapse after roughly 1,000 steps, Lp-Reg sustains stable exploration for over 3,000 training steps.

dev.to — ‘On-Policy Distillation Works Better Without the Teacher’ dev.to

Replacing the teacher entirely with a fixed negative penalty on low-probability tokens achieved identical performance on mathematical reasoning benchmarks… OPD functions more as a self-improvement filter that ‘prunes’ bad reasoning branches than a knowledge transfer mechanism.

RealSafe-R1 application note (ResearchGate) researchgate.net

Applied to safety-aligned models like RealSafe-R1, [OPSA] achieved a superior safety-reasoning tradeoff by concentrating updates on early ‘compliance-decision’ tokens.

Project page — DripNowhy/On-Policy-Self-Adaptation dripnowhy.github.io

Practitioners are advised to use ‘Slime-ready’ mirrors of the DAPO-Math-17k dataset to avoid the 1.79M-row duplicated parquet files found in upstream versions… targeting only the 20% of valid tokens with the lowest actor log probabilities, assigning negative advantages between -0.5 and -1.0.

TheNeuron AI daily digest theneuron.ai

AI researcher Gurtej Gill interpreted the findings as evidence that many apparent distillation gains are actually just a form of ‘regularization against unforced errors’… some dissent remains regarding whether this ‘self-adaptation’ can truly substitute for a teacher in more creative or open-ended tasks.

Phil Schmid — ‘Agents: pass@k vs pass^k’ philschmid.de

pass^k requires an agent to succeed on all k independent trials… top-tier models like GPT-4o achieved roughly 50% success on a first attempt, [but] their pass^8 score dropped to below 25%

Sierra blog on τ-bench sierra.ai

τ-bench evaluates an agent’s ability to maintain consistency across complex, multi-turn dialogues… even state-of-the-art function-calling agents struggle to ‘say no’ to users when a request violates a corporate rule

Medium — Dutta Saswata, ‘Structuring Multi-Agent Systems around Irreversible Actions’ medium.com

as projects scale, even advanced tool-calling agents struggle with ‘context pollution’… [CAST] bridges the gap between unreliable prompt-based critics and black-box optimization methods by making ‘thinking’ part of the supervision signal

IRMA paper (arXiv 2504.11468) arxiv.org

IRMA… employs a multi-agent setup where a ‘retriever’ agent extracts relevant domain policy rules and a ‘verifier’ agent reformulates the original query with specific tool suggestions before the main agent attempts an action

τ²-bench / τ³-bench (arXiv 2506.09338) arxiv.org

τ²-bench addressed the limitation of ‘single-control’ environments… by introducing ‘dual-control’ scenarios where the user also uses tools; τ³-bench corrected over 75 task errors [and] identified numerous ‘incorrect expected actions’ in the original airline domain

OpenReview discussion on process reward models / SFT-vs-RL openreview.net

excessive SFT can actually degrade subsequent RL performance… SFT often induces ‘pseudo-reasoning paths’—rigid, imitative behaviors that look like reasoning but are fundamentally incorrect