Sources

Program-as-Weights: A Programming Paradigm for Fuzzy Functions huggingface.co

Fuzzy-function programming compiles natural-language specifications into compact neural artifacts using a 4B compiler and 0.6B interpreter, achieving efficient, local execution with reduced memory usage and faster inference.

WARP: Weight-Space Analysis for Recovering Training Data Portfolios huggingface.co

WARP is a framework that infers training data compositions from released model weights by analyzing geometric footprints in weight space through model merging and feature extraction.

Denser neq Better: Limits of On-Policy Self-Distillation for Continual Post-Training huggingface.co

On-policy self-distillation in continual post-training accelerates in-domain specialization but fails to prevent forgetting and can collapse in out-of-distribution scenarios, indicating that on-policy data alone is insufficient for continual learning.

Logit-Contribution Scoring Identifies Non-Literal Retrieval Heads huggingface.co

Logit-Contribution Scoring measures how each attention head’s output-value circuit feeds answer-token logits, isolating the heads that synthesize context rather than copy it. The method beats prior attribution techniques on retrieval benchmarks including MuSiQue and BABILong, and ships with open code.

Transferability for General Reasoning: An Automated Curriculum for Multi-Domain RLVR huggingface.co

TAC runs a bandit-style online curriculum over reasoning domains, using gradient-geometry alignment to estimate which training mixes transfer broadest. Prioritizing high-transfer domains during GRPO lifts macro-averaged accuracy across held-out tasks versus uniform sampling.

AutoMem: Automated Learning of Memory as a Cognitive Skill huggingface.co

AutoMem treats memory as a learnable cognitive skill, jointly optimizing both the memory structure and the agent’s proficiency at file-system-style operations over it. The framework delivers sizable gains on long-horizon agent trajectories where fixed memory schemes degrade.

Learning to Move Before Learning to Do: Task-Agnostic pretraining for VLAs huggingface.co

The two-stage recipe first learns physical competence via self-supervised inverse dynamics on unlabeled robot data, then adds lightweight language grounding. On the SIMPLER benchmark and WidowX hardware, it beats behavior cloning while using far fewer expert demonstrations.

Morphing into Hybrid Attention Models huggingface.co

FlashMorph frames the choice of which layers keep full attention versus swap to linear attention as a budget-constrained subset optimization, solved with morphable layerwise gates and linearization regularization. The resulting hybrids retain long-context retrieval accuracy while cutting inference cost.

Multi-Resolution Flow Matching: Training-Free Diffusion Acceleration via Staged Sampling huggingface.co

MrFlow generates at low resolution, upsamples in pixel space with a pretrained GAN, then re-injects noise for a short high-resolution refinement pass. The staged pipeline exploits quadratic token savings to reach 25x speedups on flow-matching models with no training or runtime changes.

AgenticSTS: A Bounded-Memory Testbed for Long-Horizon LLM Agents huggingface.co

AgenticSTS enforces a bounded contract: each turn reassembles a fresh prompt from typed retrieval over explicit memory layers, letting researchers ablate individual memory components. Tested on Slay the Spire 2, isolating strategic-skill memory measurably improves conditional reasoning.

EvoPolicyGym: Evaluating Autonomous Policy Evolution in Interactive Environments huggingface.co

Autonomous agents evaluate policy improvement through iterative editing within fixed budgets, revealing that successful policy evolution requires both task-specific mechanisms and feedback-constrained refinement.

PACE: A Proxy for Agentic Capability Evaluation huggingface.co

PACE is a framework that predicts expensive agentic LLM benchmark performance using a small subset of atomic evaluation instances, achieving high accuracy at a fraction of the cost.

WorldDirector: Building Controllable World Simulators with Persistent Dynamic Memory huggingface.co

WorldDirector enables controllable video generation with persistent object memory by decoupling semantic motion planning from visual rendering through LLM coordination of 3D trajectories and camera movements.

SkillCoach: Self-Evolving Rubrics for Evaluating and Enhancing Agentic Skill-Use huggingface.co

SkillCoach is a self-evolving rubric framework that evaluates and improves agentic skill-use by analyzing skill selection, following, composition, and reflection processes, providing better supervision than outcome-only metrics.

Representation Distribution Matching for One-Step Visual Generation huggingface.co

Representation Distribution Matching enables high-quality image generation by matching feature distributions under pretrained encoders, with improved performance through optimized batch sizes and multi-encoder evaluation metrics.

Discrete Diffusion Language Models for Interactive Radiology Report Drafting huggingface.co

Diffusion language models match or exceed autoregressive models in medical visual question answering while offering faster decoding and bidirectional text editing capabilities.

DuoMem: Towards Capable On-Device Memory Agents via Dual-Space Distillation huggingface.co

DuoMem is a dual-space distillation framework that transfers procedural problem-solving from large language models to compact student models through context-space and parameter-space distillation, achieving high performance with minimal additional parameters and improved inference speed.

When Search Agents Should Ask: DiscoBench for Clarification-Aware Deep Search huggingface.co

DiscoBench evaluates search agents’ ability to handle ambiguous queries through clarification questioning and recovery in multi-step information-seeking tasks across diverse real-world domains.

Scaling Laws for Grid-Based Approximate Nearest Neighbor Search in High Dimensions huggingface.co

Grid-based multiprobe algorithms demonstrate superior dimensional scaling properties compared to graph-, tree-, and partitioning-based methods for approximate nearest neighbor search, making them competitive for high-dimensional and rebuild-heavy applications.

From SRA to Self-Flow: Data Augmentation or Self-Supervision? huggingface.co

Research investigates the mechanisms behind self-alignment methods in diffusion transformers, finding that performance improvements stem primarily from data augmentation along the noise dimension rather than token interactions between noise levels.

InstanceControl: Controllable Complex Image Generation without Instance Labeling huggingface.co

InstanceControl enables multi-instance image generation by using vision-language models to establish instance-level correspondences between text prompts and visual conditions, while employing adaptive mask refinement for improved accuracy.

Breaking Failure Cascades: Step-Aware Reinforcement Learning for Medical Multimodal Reasoning huggingface.co

A reinforcement learning approach called MRPO is introduced to improve clinical image reasoning by addressing cascading errors through step-wise process rewards, demonstrating superior performance over existing methods.

AgenticDataBench: A Comprehensive Benchmark for Data Agents huggingface.co

A comprehensive benchmark named AgenticDataBench is introduced to evaluate data agents across diverse domains with fine-grained task annotations and skill-based coverage metrics.

Optimizing Visual Generative Models via Distribution-wise Rewards huggingface.co

A novel reinforcement learning framework for visual generation uses distribution-wise rewards to improve image diversity and quality while addressing mode collapse and computational efficiency issues.

AnyGroundBench: A Specialized-Domain Benchmark for Video Grounding in Vision-Language Models huggingface.co

Vision-Language Models struggle with domain adaptation in specialized spatio-temporal video grounding tasks, highlighting limitations in zero-shot generalization and in-context learning capabilities.

Anthropic found a hidden space where Claude puzzles over concepts technologyreview.com

The AI firm Anthropic has developed a technique that has given it the clearest glimpse yet at what’s really going on inside large language models as they answer questions or carry out tasks. What they found ranges from the mundane to the unnerving. Researchers at the company built a tool called the Jacobian lens (or…

AGVBench: A Reliability-Oriented Benchmark of Data Augmentation for Vein Recognition huggingface.co

Research evaluates 30 augmentation strategies for vein recognition across multiple datasets and architectures, revealing inconsistencies between accuracy and adversarial security while highlighting dataset-specific effectiveness variations.

Parameter-Efficient Quantum-Inspired Fast Weight Programmers for Traffic-Matrix Forecasting huggingface.co

Quantum-inspired recurrent models using gated QKAN-FWPs demonstrate superior forecasting accuracy with reduced computational requirements compared to traditional LSTM networks for traffic matrix prediction.

References

Sakana AI — Text-to-LoRA sakana.ai

Text-to-LoRA (T2L) uses a task encoder to extract vector representations from text descriptions, which are then passed through MLP blocks to generate the A and B low-rank matrices for a base model… matching or exceeding the performance of manually fine-tuned ‘oracle’ adapters in zero-shot settings.

Hacker News thread #48774368 (user mathisfun123) news.ycombinator.com

compared to randomly testing coefficients in signal processing without a clear understanding of the resulting filter’s behavior… a LoRA blob remains an opaque behavioral artifact where the developer ‘doesn’t even know what they are getting out of it’

Hyperion Consulting — AI Research Decoded hyperion-consulting.io

Because inference is local, sensitive data (such as internal logs or user documents) never leaves the host environment, aligning with high-risk compliance standards like the EU AI Act… a compiled PAW artifact is a frozen, deterministic binary that remains stable over time.

Burnyverse survey — Neurosymbolic AI burnyverse.com

ALCHEmist achieves performance comparable to or better than large-scale LLM annotators while reducing total labeling costs by a factor of approximately 500x… standard symbolic code can become brittle compared to newer neurosymbolic methods that treat programs as differentiable weights.

arXiv 2510.13259 — synthetic benchmark validity arxiv.org

static, model-generated datasets often fail to capture the ‘messiness’ and unpredictable edge cases of real-world production environments… using the same LLM to both generate the benchmark and perform the task can lead to overinflated accuracy scores that do not translate to general capabilities.

gopubby.com — LoRA generation review ai.gopubby.com

T2L’s weight generation is often described as ‘lossy compression’; it can maintain the performance of the adapters it was trained to reconstruct, but its ability to generalize to entirely unseen tasks remains a point of academic debate… often results in slightly lower downstream accuracy than a model fine-tuned on a specific dataset.

arXiv — LLMSurgeon (Luo et al., 2605.30348) arxiv.org

LLMSurgeon estimates a calibrated soft confusion matrix on held-out reference data and solves a constrained inverse linear problem to de-blur biased predictions, recovering domain proportions from generated text alone.

NeurIPS 2024 poster — ‘Blind Baselines Beat Membership Inference Attacks’ neurips.cc

Blind baselines—which simply look at data distributions without querying the target model—can often outperform state-of-the-art MIAs, suggesting many claimed MIA successes are actually detecting distribution shifts rather than true membership.

Minnesota Journal of Law, Science & Tech — Proposed FRE 707 journals.law.umn.edu

Any inference that a model was trained on a specific dataset must be supported by a demonstrable error rate and peer-reviewed methodology; the Advisory Committee deferred a final vote citing the black-box nature of AI as a hurdle to cross-examination.

Emergent Mind — WARP paper explainer emergentmind.com

Simulated trajectories often perform better than the true training trajectory… merging-based paths provide a smoothed version of the transition, making it easier for the MLP to extract the underlying signal.

AI Weekly — ‘UW-Madison’s WARP recovers training data mixtures from weights’ aiweekly.co

The framework addresses ‘access asymmetry’ where researchers study public models but remain blind to proprietary data distributions—information effectively teleported into the released weights.

AI Native Foundation Daily Digest (2026-07-03) ainativefoundation.org

This transparency is a double-edged sword: while it aids in accountability, it also allows competitors to clone high-performing data recipes, potentially eroding the competitive advantage of organizations that invest heavily in data curation.

Medium explainer of Thinking Machines Lab’s on-policy distillation post medium.com

On-policy distillation is framed as a ‘cheap and powerful’ middle path between SFT and RL, providing dense per-token feedback on student-generated trajectories to avoid the sparse rewards that make standard RL expensive and unstable.

Rethinking On-Policy Distillation (arXiv 2607.06855) arxiv.org

In privileged self-distillation settings, researchers observed a relative accuracy drop of up to 17% on AIME 2024 and AIME 2025 across multiple models… as the reasoning trace grows, the ‘prefix drift’ between student and teacher increases, making the teacher’s token-level rewards less reliable.

alphaXiv discussion of Denser ≠ Better (StableSDPO results) alphaxiv.org

On the AIME benchmark, a standard SDPO model with a 5% EMA rate collapsed to 34.38% performance; however, replacing it with a matched ‘restart-and-freeze’ strategy restored performance to 55.00%.

Amazon Science — Physics-Guided Policy Optimization with Self-Distillation amazon.science

PGPO uses a mutual-information estimate to modulate step sizes, preventing the late-training collapse observed in vanilla SDPO.

chrisliu298/awesome-on-policy-distillation (GitHub) github.com

Success depends strictly on ‘thinking-pattern consistency’ between teacher and student; if their underlying logic differs, the distillation often fails… a teacher with lower benchmark scores but higher initial thinking-pattern overlap can be more effective than a ‘smarter’ teacher that thinks differently.

ICML 2026 poster — Self-Distillation Policy Optimization (LAS group) icml.cc

SDPO formalizes self-distillation as a process where a model acts as its own probabilistic oracle by conditioning on rich textual feedback, converting outcome-based rewards into granular next-token predictions.

Sources

References

Jack Sun, writing.