Sources

Puro-2B: Poor Lab’s Qwen2-1.5B Trained on RTX 5090 within $5090 huggingface.co

A cost-efficient open-source pretraining recipe trains 2B-parameter models on consumer GPUs for under $7K, yielding performance near larger baselines while deriving cost scaling laws and studying data curricula.

Sliding-window beats linear attention huggingface.co

Sliding window attention with sinks outperforms post-trained linear attention on long-context tasks without requiring retraining, offering a cheaper and more reliable inference solution.

LoopArena: Benchmarking Models as Runtime Controllers for Loop Engineering huggingface.co

LoopArena benchmarks how well a controller model guides a separate coding agent through long tasks, revealing low strict success rates and significant cost reductions.

Blind Men and the Elephant: Probing the Epistemic Myopia of LLMs under Long-Tail Divergent Knowledge huggingface.co

ElephantBench tests whether large language models hold multiple conflicting accounts of obscure facts, using a graph-based pipeline to build verifiable multi-account questions. Results show widespread epistemic incompleteness: models collapse divergent viewpoints into a single answer, missing alternative accounts stored in parametric memory.

Rubric-to-Code Credit Assignment for Reinforcement Learning huggingface.co

Rubric-to-Code Credit Assignment sharpens reinforcement learning for interactive web app generation by tying rubric-level feedback to localized code spans instead of whole outputs. The hierarchical reward drives Ling-RCCA-Flash to substantial gains on MiniAppBench and ArtifactsBench over standard GRPO baselines.

EvoUndo: Recoverability-Constrained Self-Evolution for LLM Agent Harnesses huggingface.co

Self-evolving agents that rewrite their own harnesses often cannot undo the damage. EvoUndo formalizes recoverability using counterfactual states and a recovery calculus, finding that reliable rollback requires co-designing verification, state grounding, and the expressivity of the recovery language.

StarHarness: Evolving Harnesses with Stratified Search for Enterprise Environments huggingface.co

StarHarness treats agent harness design as search, evolving fixed-weight configurations against stratified task pools with a hidden selection set to prevent overfitting. It lifts tool-use scores on ITBench SRE, EnterpriseOps-Gym ITSM, and AutomationBench Finance, and transfers across GPT and Qwen models.

ContextPilot: Teaching Agents for Proactive Context Management via Fine-grained RL huggingface.co

ContextPilot equips agents with expanded context-editing tools — soft offloading, memory writes, planning scratchpads — and trains them with branch-sampling reinforcement learning that assigns action-level advantages using entropy variation. The result: sharper long-horizon reasoning by identifying which context decisions actually matter.

J-Zero: Unified Challenger—Solver—Judge Co-Evolution from Zero Data huggingface.co

J-Zero bootstraps self-improving language models without seed datasets by pitting a task generator, solver, and judge against each other using predefined preference pairs. The adversarial loop works across both verifiable and unverifiable domains, extending self-play beyond math and code.

StepGuard: Learning Step-Level Guardrails with Scalable Supervision and Safety-Utility Balancing huggingface.co

StepGuard inserts a step-level guard model that vets tool calls before execution, trained on trajectories from StepGen and tuned with Balance-GRPO to weigh safety against utility. On AgentDojo and AgentDyn it blocks attacks while keeping task completion largely intact.

DART-SD: Diamond-topology Aware Retrieval and Tuning for Self-Distillation of Multi-Turn Tool-Calling Agents huggingface.co

DART-SD improves multi-turn tool-calling agents by modeling execution as a diamond-topology graph, identifying critical failure points, and applying localized self-distillation to preserve valid reasoning while correcting errors.

Locate Anything in Videos: Rethinking Efficient Generative Spatio-Temporal Video Grounding huggingface.co

Parallel Tube Decoding enables simultaneous spatial and temporal video grounding by removing autoregressive dependencies, drastically cutting latency while improving accuracy.

Ring Forcing: Towards Precise Long-Term Memory for Autoregressive Video Diffusion huggingface.co

Ring Forcing is an autoregressive video diffusion framework that improves long-term memory through ring-structured training, history compression, and sparse rotary embeddings to achieve minute-long coherence.

Paint What You See: Benchmarking Dexterous Visual Tool Use in Multimodal Agents huggingface.co

EASEL benchmarks fine-grained visual tool use through reference-guided reconstruction and semantic tasks, revealing that current multimodal agents struggle with closed-loop precision and trajectory stability.

LayerRecall: A State-Conditioned Memory Router for Long-Horizon Consistency in Video Generation huggingface.co

LayerRecall selectively routes long-range historical memory into specific video diffusion layers to improve long-video consistency, supervised by cross-horizon prediction matching.

LMSM: LLM Security Framework Inspired by Linux Security Modules huggingface.co

LMSM applies a modular security framework to LLM serving by separating evidence calibration, policy evaluation, and output gating, enabling flexible interpretability-based enforcement without rebuilding request handling.

Fast Weight Attention for Continual Learning huggingface.co

Recurrent fast-weight memories and selective state-space models are analyzed as online learning rules under autoregressive semantics, yielding normalized update families with stable renormalization that improve length extrapolation.

Code as Worlds: Agentic Discovery of Executable World Representations for Physical Reasoning huggingface.co

Code-as-World represents physical environments as executable code to enable quantitative reasoning and scalable supervision for vision-language models.

Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models huggingface.co

VLAct improves vision-language-action model performance by pre-training on diverse robot data with preserved vision-language priors and shared action semantics, achieving strong results across simulations and unseen embodiments with limited compute.

Act with Intent: Distilling Behavior Intent for Vision-Language-Action Models huggingface.co

Vision-Language-Action (VLA) models can turn multimodal context into robot actions, but their action decoders are still trained largely by behavior cloning. This supervises which motor command was demonstrated while leaving implicit the local objective served by the behavior under the instruction. Future-based supervision enriches action learning with frames, latent observations, trajectories, or motion representations, but these signals capture particular realizations of what may happen rather

PonderPounce: A Pretrained MLLM as an Episode Context Engine for Robot Control huggingface.co

PonderPounce leverages native causal context in multimodal language models as robot episode memory, jointly training a reasoning System2 module and a fast System1 action model to improve long-horizon policy performance without dedicated memory architectures.

Video Generative Models as Geometry Learner huggingface.co

GeoNeXt repurposes pretrained video generative models as a unified framework for geometry estimation via next-frame prediction, enabling efficient joint modeling of depth and surface normals with minimal labeled data.

Revisiting Local Context for Long-Horizon Streaming 3D Reconstruction huggingface.co

ABot-Recon achieves stable long-horizon streaming 3D reconstruction by using only local temporal context and frame-independent predictions composed sequentially, reducing drift via a lightweight temporal refiner and composition-aware pose loss.

Language Chain in Alignment: Cross-lingual Ranking Preference Optimization huggingface.co

Cross-lingual Ranking Preference Optimization improves multilingual alignment by transferring English preference knowledge to target languages through hierarchical ranking optimization.

Agentic Artifact Creation: Systems, Evaluation, Principles, and Opportunities huggingface.co

This survey defines agentic artifact creation as stateful, feedback-driven construction of deliverables by AI systems and analyzes 259 works across artifact families, settings, and evaluation practices to propose principles for accountable control.

GGSS: Geodesic-Gated Spherical Steering for Inference-Time Debiasing of Generative Vision-Language Models huggingface.co

GGSS reduces demographic bias in generative vision-language models by steering visual tokens along geodesic arcs with an adaptive gate, preserving visual-language accuracy.

Acquire, Repair, Preserve: A Diagnosis-Guided Post-Training Recipe for Small-Model Dialogue Game Agents huggingface.co

A 2B open-weight model improves interactive dialogue-game performance through supervised fine-tuning and turn-local preference training, though gains remain largely within the targeted game family.

Generative Semantic Scene Completion huggingface.co

Generative discrete diffusion recasts outdoor LiDAR semantic scene completion via paired sparse-dense synthesis, semantic-guided generation, and structured refinement to achieve state-of-the-art results.

Ask or Answer: A Decision Framework for Multi-Turn Health Misinformation Intervention huggingface.co

RO-PnR learns when to ask clarifying questions versus deliver corrections by weighing interaction costs against expected gains, improving cost-adjusted outcomes in health misinformation dialogues.

Training, learning and inference: unified dynamics of neural systems huggingface.co

Atomic generation facts compiled into a graph enable recursive scientific processes, and training-learning dynamics in nanoGPT are characterized by state-conditioned parameter updates, persistent functional reorganization, and frozen inference projections validated across architectures.

Lost in Compression: A Controlled Cross-Lingual Audit of Extractive Prompt Compressors huggingface.co

Learned prompt compressors trained on English data disproportionately degrade non-English contexts, widening token-cost disparities, while multilingual training and deterministic methods reduce this gap.

References

Puro-2B-Base model card (Hugging Face) huggingface.co

Puro-2B-Base achieved an overall score of 57.81%, surpassing Qwen2-1.5B’s 55.14%… approaching Qwen2.5-1.5B which maintains a lead with an overall score of 60.73%.

alphaXiv analysis of Puro-2B (2608.27370) alphaxiv.org

Modified NVIDIA drivers to enable PCIe peer-to-peer (P2P) and GPUDirect RDMA, recovering significant communication performance that is otherwise software-locked… yielding a 2.7x improvement in compute-per-dollar over H200 rentals.

alphaXiv (Hyperball / MuonH paper, 2511.18903) alphaxiv.org

MuonH fixes the Frobenius norm of every Transformer weight matrix at its initialization value… yielding a 20-30% token-equivalent speedup over standard weight decay in models up to 1.2B parameters.

Medium: ‘Muon Optimizer — The Complete Guide’ (Abir) medium.com

Muon introduces roughly 10-20% wall-clock overhead due to the Newton-Schulz iterations… Moonshot’s Kimi-K2 and Zhipu’s GLM-5 adopted MuonClip to mitigate attention logit explosion when scaling vanilla Muon to trillion-parameter regimes.

EACL 2026: Curriculum Model Averaging study aclanthology.org

Traditional LR decay reduces the model’s plasticity just as it encounters its most complex and high-quality data at the end of the curriculum, effectively wasting the best samples… CMA maintains a moderate LR through the end of the curriculum and uses model averaging to stabilize the result.

r/MachineLearning discussion (referencing Albert Gu) reddit.com

Albert Gu called the title ‘misleading’ — the result holds for post-training linearization of an existing Transformer, not for models trained from scratch, where SSMs and linear attention can still match or beat SWA.

Moonlight review of ‘Paying Attention to Hybrid Attention’ (Benfeghoul et al., 2025) themoonlight.io

SWA-only configurations — with the linear attention path fully disabled — retained nearly 100% of the hybrid model’s performance on PIQA/HellaSwag, implying LoLCATs-style fine-tuning trains models to ignore the linear path entirely.

kenashe.ai analysis blog kenashe.ai

The prior linearization literature compared against sink-free SWA, which collapses once initial tokens leave the window; adding four fixed sink tokens neutralizes the supposed gap the whole field was trying to close.

RAttention paper (alphaxiv 2506.15545) alphaxiv.org

Standard SWA presents a Pareto bottleneck; RAttention pairs a 512-token window with a Residual Linear Attention module to capture out-of-window information, matching full-attention with far less KV memory than Mistral/Gemma 4K windows.

AI Advances (Medium) — Mamba-3 write-up aiadvances.org

Microsoft’s Phi-4-mini-flash and Nvidia’s Nemotron-H interleave Mamba blocks with sliding window attention, suggesting production teams treat SWA and SSMs as complementary rather than substitutes.

explainx.ai commentary explainx.ai

Practitioners currently spending ‘compute capital’ converting LLMs to linear variants should immediately re-evaluate; the burden of proof has shifted back onto linear-attention researchers to demonstrate value over the sink-augmented SWA baseline.

humanlayer GitHub — Opus 5 on SlopCodeBench github.com

GPT-5.4 and Claude Opus 4.6 achieve only 11% and 17% strict pass rates… no tested model successfully completed an entire problem end-to-end without quality degradation.

r/theprimeagen discussion of SlopCodeBench reddit.com

agent-generated code is roughly 2.3x more verbose than human-maintained repositories; agents pass checkpoints but their code quality deteriorates rapidly

TeamBench project page teambench.github.io

structurally enforces Planner/Executor roles at the OS level to measure the ‘Teamwork Necessity Index’ — the point at which a multi-agent team outperforms a single agent

Addy Osmani — ‘Loop Engineering’ blog addyosmani.com

Some practitioners view it as the next major abstraction layer… skeptics argue it is merely ‘a cron job wearing a hat,’ expressing concern over runaway token costs and ‘cognitive surrender.’

arXiv 2509.02547 (PEAR / harness robustness) arxiv.org

identical models can see performance swings of 10–20% depending solely on the agent harness used… PEAR specifically targets vulnerabilities in the planner-executor structure

BuildThisNow — ‘Best AI Coding Model 2026’ buildthisnow.com

Claude Opus 4.8 has been observed writing unauthorized tests in 20% of cases where specifically instructed not to

Sources

References

Jack Sun, writing.