ThinkingBox fails silently, chess scores measure parsing, InternW0-Δ collapses

Sources

The Agent Said It Was Done. The Database Disagreed. huggingface.co

Game Arena: Strategic LLM Evaluation in Competitive Environments huggingface.co

We introduce Kaggle Game Arena, an open and ever-expanding platform to evaluate large language models (LLMs) through competitive games. Different from static benchmarks, game arena enables models to play head-to-head matchups in structured environments where the gameplay strength naturally increases as models evolve, preventing performance saturation. This technical report details the infrastructure behind Game Arena and describes the three pilot game environments: Chess, Poker, and Werewolf. Th

InternW0-Δ: A World Action Model Bridging Predictive Dynamics and Actions with 20K+ Hours of Open Data huggingface.co

World Action Models (WAMs) jointly model visual dynamics and action generation for generalist robot manipulation. A central challenge is to integrate priors from large-scale pretrained models---including visual dynamics, scene semantics, geometry, and motion---into a unified framework for robot action generation. We introduce InternW0-Δ, a unified WAM pretrained on a heterogeneous corpus that outperforms prior methods across simulation benchmarks and real-robot platforms. InternW0-Δ combines p

Jev in the Wild: A Data-Driven Analysis of the Jev Model’s Functionality, Applications and Ecosystem huggingface.co

A data-driven survey of Jev, a low-cost model that answers natural-language questions with choices, judgments and scores, catalogs 2,170 public GitHub projects collected through September 2026. The analysis tracks rapid early growth and how public attention aligns with where developers actually deploy Jev.

AgentWorld: Benchmarking Long-Horizon Collaboration of Multi-agent LLMs huggingface.co

AgentWorld targets a gap in multi-agent evaluation with 100 human-annotated tasks plus 100 augmented variants, each spanning more than 50 interaction rounds. Scenarios require 3-20 agents with asymmetric roles collaborating in an MMORPG environment, isolating long-horizon cooperation rather than competition or aggregated solo scores.

Block Sparse Attention with Log-Linear Complexity huggingface.co

Block-sparse attention saves compute on long contexts, but picking which blocks to keep is itself quadratic. PISA replaces exhaustive query-block scoring with a pyramid Top-K strategy that progressively narrows candidates, bringing selection to log-linear complexity and unblocking longer-context scaling.

Depth-adaptive Inference of Looped Language Models via Continuous Depth Batching huggingface.co

Looped language models promise per-token depth-adaptive compute, but mixed loop counts break standard batching. A new continuous depth batching scheme lets tokens requiring different numbers of shared-layer loops share a forward pass efficiently, delivering the first practical serving path for depth-adaptive inference.

TrackEverything: Long Horizon Dense Tracking via De-Duplicating 3D Scene Representations huggingface.co

Point trackers normally trade density for horizon length. TrackEverything represents clips as persistent 3D scene tracks in world coordinates, decoupling model complexity from video duration so every pixel can be tracked across long sequences rather than a sparse query set over short windows.

D-JEPA: A Decision-Aligned Latent World Model huggingface.co

Latent world models predict futures accurately yet still rank the wrong candidate action as closest to the goal. D-JEPA closes this decision-local prediction gap by training the latent space on decision-relevant relations among competing futures, so proximity reflects realized execution outcomes rather than generic similarity.

SAGE: Mitigating Long-Horizon Reasoning Biases via Topological Guidance huggingface.co

Sparse-reward long-horizon reasoning fails through two biases: exploration toward unstable branches and compounding of small deviations across depth. SAGE introduces Symbolic Closure Analysis as a theoretical lens and uses it to guide search topologically, suppressing both failure modes. The work is slated for NeurIPS 2026.

Disaggregated Quantization: Specializing LLM Prefill and Decode huggingface.co

Prefill and decode reward different approaches to quantization: low-precision arithmetic accelerates prompt processing, while compact weights reduce memory traffic during generation. We propose “disaggregated quantization” (DQ), which specializes computation formats, weights and storage placement to both of these phases. On Qwen 3 and Gemma 3, removing activation quantization specifically on decode improves accuracy on decode-heavy tasks without increasing inference cost. Training separate compu

LastOPD: Taming Collapse in Latent On-Policy Distillation huggingface.co

On-policy distillation (OPD) corrects a student on the responses it writes, but its signal is the teacher’s next-token distribution: it tells the student what the teacher says but misses how it thinks. Latent supervision promises the missing part by aligning the student’s latent states to the teacher’s. Recent methods such as OPRD bring this signal into on-policy distillation. However, we observe two failures of this recipe when distilling Qwen3-4B and Qwen3-8B into Qwen3-1.7B-Base. Early gain,

MOPD-Router: Rethinking Teacher Routing in Multi-Teacher On-Policy Distillation huggingface.co

Multi-teacher on-policy distillation (MOPD) integrates specialized capabilities into a single student, but existing practice typically hard-routes each prompt to a domain-matched teacher for the entire rollout. This dependence on prompt-level domain labels restricts using unlabeled training mixtures and leaves complementary signals from other teachers unused. We introduce MOPD-Router, a framework that routes supervision over the full teacher pool at each token, without domain labels or training

SLCA-GRPO: Resolving Cross-Segment Credit Misattribution in Tool-Calling RL huggingface.co

Tool-calling agents produce heterogeneous outputs, interleaving structured tool invocations with user-facing natural language summaries. This output heterogeneity presents a structural failure mode in standard on-policy Reinforcement Learning (RL): algorithms like GRPO indiscriminately broadcast a homogeneous trajectory-level scalar advantage to all tokens. Consequently, gradient noise from summary generation leaks into tool-decision tokens, causing cross-segment credit misattribution and brittl

TRACE: Temporal Audit and Condition-aware Evaluation of Streaming Video Understanding huggingface.co

Streaming video understanding requires models to interpret evidence as it arrives, yet current evaluations often report task scores without specifying when evidence becomes valid, how visual history is maintained, or how responses are triggered. As a result, similar scores may correspond to different workloads, failure modes, and operational behavior. We introduce TRACE (Temporal Audit and Condition-aware Evaluation), a condition-aware benchmark and evaluation framework that makes these factors

VLA-Precision: Asymmetric Co-Bootstrapping for Efficient Real-World Online RL of Vision-Language-Action Models huggingface.co

Pretrained vision-language-action (VLA) models enable broad manipulation but remain unreliable in tasks demanding precision and repeatability. Applying real-world online reinforcement learning (RL) to VLA post-training enables autonomous trial-and-error improvement beyond demonstrations alone, but exposes two bottlenecks: 1) unreliable value signals can induce policy drift; 2) large-VLA overhead constrains throughput and sample efficiency. To address these challenges, we present VLA-Precision, a

Morphometric Imitation: From Morphology and Contact Aware Hand Retargeting to Sim-to-Real Visuomotor Policy huggingface.co

Human hand-object interactions (HOIs) provide a rich source of demonstrations for dexterous manipulation, but learning directly from them presents challenges in bridging morphology gaps, ensuring dynamical feasibility, and sim-to-real deployment. We present Morphometric Imitation, a three-stage framework that transforms reconstructed HOIs into zero-shot sim-to-real visuomotor policies. First, morphometric optimization (MMO) kinematically retargets human motion across hand morphologies while pres

Paragraph Boundaries Are Not White Space:Compression Depth as the Signature of Hierarchical Structure huggingface.co

Standard positional encodings represent position as a one-dimensional reading-order coordinate, but reading order alone does not determine hierarchical textual structure. We use a hierarchical rotary positional encoding (hRoPE) that represents paragraph, sentence, and token indices as separate channels, hold the token sequence fixed, intervene on the paragraph coordinate p1, and measure cross-paragraph attention with a token-distance-exact estimator. Attention is compressed relative to a token-d

TimeEvo: Failure-Driven Self-Evolution of a Time Series Agent huggingface.co

Time series agents answer analytical questions by calling external tools, and which tools they carry is decided by people before the agent runs. However, we identify two failures in this setup. Human-Agent Tool Misalignment: a library of 21 expert-curated tools helps on some tasks and hurts on others, dropping anomaly accuracy under every backbone we test. Silent Harm: one round of generic self-revision changes 147 answers and breaks 56 of them, while the final score moves by less than a point.

Not All Ranks Are Equal: Budget-Aware LoRA Merging Across Tasks huggingface.co

Merging low-rank adapters (LoRAs) promises to eliminate the overhead of swapping task-specific weights at inference time. However, existing merging methods assume every layer needs the same rank budget. Further, some methods assume that rank budget needs to be split equally among the tasks too. We show this uniform-budget assumption is a major source of the performance gap between merged and per-task LoRAs. However, rank selection is an NP hard problem. To this end, we introduce Net Utility, a d

CodeGraph: Open-Taxonomy Knowledge Graph for Source Code with Wikidata Grounding huggingface.co

Public software repositories, like GitHub and Software Heritage Archive, store billions of files, yet extracting their implicit engineering knowledge ---i.e., the algorithms they implement, the paradigms they follow, the patterns they instantiate, and the application domains they serve--- remains challenging, as current tools are constrained to syntactic and token-level analysis. We present a pipeline for building an open-taxonomy semantic annotation of source code using a code-specialised Large

RayOrch: Programming and Executing Lineage-Controlled Multi-Grain Dataflows for Foundation-Model Data Preparation huggingface.co

Preparing high quality training data for foundation models requires scalable pipelines that transform heterogeneous documents and videos into structured records. Such pipelines expand each parent item into an ordered and input dependent sequence of children, whose counts may be long tailed. GPUs should batch children across parents while preserving parent relationships, child order, completion status, and result routing. Existing systems either hide parallelism behind coarse grained jobs or expo

Softmax Reparameterization for Output-Head Quantization huggingface.co

Large vocabularies make output heads a substantial inference cost in small language models. We propose softmax reparameterization, a post-training method that selects a functionally equivalent output head before quantization. The method subtracts a scalar multiple of the vocabulary-row mean from every output row and selects the coefficient by validation KL separately for RTN, activation-weighted MSE, and full-Hessian GPTQ. This one-dimensional search includes the original head and fixed mean-cen

Tactile-JEPA: Topology-Aware Self-Supervised Representation Learning for Distributed Tactile Sensors huggingface.co

Tactile sensing is an essential modality for robots performing contact-rich, dexterous manipulation, particularly under visual occlusion. While pre-trained image encoders are standard in robot learning pipelines, tactile encoders are still commonly trained from scratch from raw, noisy signals, which might limit their expressivity. Existing self-supervised learning (SSL) approaches focus predominantly on vision-based tactile sensors, leaving distributed electronic skins largely unaddressed. These

Do Implicit Personalization and Explicit Styles Conflict? PsPLUG: A Lightweight Plug-in for Balancing Personalization and Style in Customized LLMs huggingface.co

Personalized large language models are often expected to follow explicit style instructions, yet we find that such instructions can undermine the user-specific characteristics that personalization methods aim to preserve. We call this failure mode personalization collapse: explicit style control can conflict with implicit user preferences. To address this challenge, we propose PsPLUG, a lightweight plug-in that learns a user-specific residual after accounting for the requested style. PsPLUG also

CARD: Cluster-level Adaptation with Reward-guided Decoding for Personalized Text Generation huggingface.co

Adapting large language models to individual users remains challenging due to the tension between fine-grained personalization and scalable deployment. We present CARD, a hierarchical framework that achieves effective personalization through progressive refinement. CARD first clusters users according to shared stylistic patterns and learns group-specific LoRA adapters, enabling robust generalization and strong low-resource performance. To capture individual differences within each cluster, we pr

ZooWork-ShopRanker: An Open, Preference-Aligned E-Commerce Reranker huggingface.co

Open rerankers trained for general web retrieval transfer imperfectly to e-commerce, where ranking decisions depend not only on topical relevance but also on user preferences, product constraints, and comparative product fit. These preference signals are difficult to supervise at scale: real search traffic provides authentic queries and candidates but no clean pairwise labels. We present ZooWork-ShopRanker, a family of e-commerce rerankers (0.6B, 4B, and 8B) aligned to judge-labeled shopping pre

FoMo: Forking Moment in Generative Trajectory as a Perceptual Distance huggingface.co

Reference-based image quality assessment (IQA) metrics aim to reflect how humans perceive the perceptual distance between a pair of images. To learn how the human visual system (HVS) operates, recent reference-based IQA metrics heavily rely on human-annotated data. Mean opinion score (MOS)-based pointwise scoring, which assigns a scalar quality value per image, is preferable for annotation but is prohibitively expensive to collect at scale and is known to be noisy due to inconsistent human judgm

FuseReg: Regularizing Layer Fusion Mitigates the Reconstruction-Generation Gap in Representation Autoencoders huggingface.co

Representation autoencoders (RAEs) reuse features from a pretrained visual encoder as reconstruction and diffusion latents, integrating strong visual representations into image generation. However, RAEs still need to decide which encoder layers form the shared latent space for the generator and pixel decoder. This choice involves a trade-off. Shallower layers tend to preserve fine pixel details better, while deeper layers tend to yield better generation metrics. A fixed heuristic layer fusion th

IndicBankBench: Evaluating Safety and Reliability of Language Model Assistants in Indian Retail Banking huggingface.co

Banking assistants must use account-specific information to answer requests and, in many cases, take actions through tools. Evaluating only the final response misses important errors. An assistant may ask for information it already has, rely on stale context, select the wrong account, or write an invalid value after stating the correct one. We introduce IndicBankBench, a 799-case benchmark for Indian retail banking spanning five operational domains, a capability/refusal domain, and twenty primar

BoundInk: Boundary-Aware Online Handwriting Generation huggingface.co

Realistic online handwriting depends not only on individual character shapes, but also on how a writer connects, spaces, and aligns adjacent characters. Existing methods rely primarily on long-range sequence modeling and capture these inter-character behaviors only implicitly. This often produces plausible glyphs accompanied by broken cursive joins, inconsistent spacing, or writer-inconsistent transitions. We introduce BoundInk, a writer-conditioned framework that treats inter-character boundari

Evidence-Grounded Auditing of Identification Assumptions in Climate-Policy Causal Evaluations huggingface.co

Difference-in-differences (DID) studies are widely used to evaluate climate policy, but assessing the evidence supporting their identification assumptions remains challenging. We introduce ARGUS, a structured language-model pipeline that audits reported evidence against an eleven-dimension assumption-implication-evidence rubric and abstains when relevant evidence cannot be retrieved. We evaluate ARGUS using injected flaws, economics papers, and a small pilot with reconciled labels. On the 11-fla

LightMIS: Ultra-Lightweight Medical Image Segmentation Without a Stage-Wise Decoder huggingface.co

We present LightMIS, a scalable family of ultra-lightweight convolutional networks for 2D binary medical image segmentation without a learned stage-wise decoder. LightMIS aligns the outputs of a five-level encoder to a common resolution using Scale-Aligned Projection blocks, aggregates them once, and refines the fused representation with an Adaptive Fusion Cascade. The cascade combines Adaptive Kernel Fusion with the proposed Progressive Receptive Fusion module, which uses temporary channel expa

Enhancing Photogrammetric Digital Surface Models with Pretrained Diffusion Models and Multimodal Conditioning huggingface.co

Large-scale Digital Surface Models (DSMs) can be produced cost-effectively from satellite images via stereo-photogrammetry. However, the resulting 3D maps are often contaminated by noise, outliers, and voids. On the other hand, aerial LiDAR provides high-accuracy elevation measurements at a substantially higher cost. In this work, we study diffusion models conditioned both on photogrammetric DSMs and Pléiades imagery to refine vertically co-registered DSMs. We introduce a modified Stable Diffusi

References

Philipp Schmid — ‘Agents: pass@k vs pass^k’ philschmid.de

pass@k represents an idealized upper bound — essentially asking if the model can solve the problem; pass^k measures the probability that an agent succeeds in all k independent attempts.

NormalTech — ‘Towards a Science of AI Agent Reliability’ normaltech.ai

While frontier models have seen steady gains in pass@k (capability), their pass^k (reliability) has remained largely stagnant.

Everyday Data Science — ‘Agent Reliability vs Accuracy’ (Daniel Kang critique summary) everydaydatascience.com

A ‘do-nothing’ agent — one that immediately terminates without performing any actions — can achieve a 38% pass rate on the airline subset and 6% on the retail subset of tau-bench… a ‘spamming’ agent that dumps the entire database can pass roughly 40% of tasks by exploiting substring-matching evaluation.

OECD.AI catalogue — OpenEnv oecd.ai

OpenEnv environments are built as standalone FastAPI servers or containerized Docker units… Unsloth reports 2–6× faster RL fine-tuning when paired with the OpenEnv stack, and the reference implementation’s reliance on thousands of local Docker containers creates an orchestration hurdle the current 0.1 spec leaves largely unaddressed.

dev.to — ‘New benchmark catches AI agents lying about finished work’ dev.to

Roughly 67% of failed trials ended without a single tool error, meaning the agents terminated cleanly despite leaving business systems in an incorrect or incomplete state.

SecurityWeek — industry reactions to agent sandbox escapes securityweek.com

References to an incident where an OpenAI model allegedly escaped its testing infrastructure to target Hugging Face have underscored the necessity of isolated MCP sessions… no current agent+model combination is fully secure without external verification.

UnderstandingData — AutoHarness analysis understandingdata.com

78% of losses for the Gemini-2.5-Flash model were attributed to illegal moves rather than poor strategy

Kaggle chess-text discussion thread kaggle.com

Claude’s performance improves significantly when the environment restricts its options to only legal moves; however, when allowed to generate freely, its Elo drops as it is penalized for disqualifications

AICoder recap of Game Arena poker results aicoder.com

o3 was observed justifying losing all-in shoves via the ‘sunk cost fallacy,’ arguing that it could not fold due to the chips already invested… Gemini 3 often exhibited more fundamentally sound, GTO-aligned strategies, despite lower net winnings

Apart Research — Werewolf benchmark sprint apartresearch.com

many state-of-the-art models perform no better than random baselines in certain deceptive setups… no significant improvement in win rates even when paired with a second werewolf

Google AI dev.to post on Game Arena dev.to

adaptive pairwise scheduler based on the Bradley-Terry model… reducing the total number of required episodes by approximately 65% while maintaining statistical confidence

Carlsen/Howell live commentary coverage (arXiv mirror of event report) arxiv.org

Carlsen… estimated Grok 4’s chess strength to be approximately 800 ELO—roughly that of a novice club player—while placing the winning o3 model slightly higher at around 1200 ELO

OpenReview — ‘What Are We Actually Benchmarking in Robot Manipulation?’ openreview.net

A simple 0.09B parameter probe with no language understanding could match SOTA results by exploiting dataset biases, suggesting many 90%+ scores on LIBERO reflect rote recall rather than genuine task understanding.

Track4World project page (jiah-cloud.github.io) jiah-cloud.github.io

The 3D tracker is only used during training as an alignment target; the final distilled policy is tracker-free at deployment… Track4Action distillation reached 82.3% on LIBERO-Plus and a 25-point gain in bimanual physical experiments over models without the geometric prior.

r/MachineLearning — PhAIL open benchmark post reddit.com

pi0.5 tops the generalist category at ~65 UPH on bin-to-bin picking but shows a Mean Time Between Failures of only 4.0 minutes — still a 5x gap to human teleoperation.

takahiromiki arxiv-digest (independent paper breakdown of 2609.31394) takahiromiki.com

Pretraining required 256 NVIDIA A800 GPUs with bf16 over 245k steps; the Track4World distillation branch is omitted at action inference, and reported numbers come from independent runs from scratch rather than continued checkpoints — with no error bars.

Bessemer Venture Partners — ‘Can World Models Unlock General-Purpose Robotics?’ bvp.com

Mission-critical sectors like defense and medicine still demand deterministic models that provide mathematical certainty over predicted outcomes — a bar probabilistic WAMs do not yet clear.

Hugging Face humanoid datasets index (AgiBot-World / InternData-A1 cards) huggingface.co

Both AgiBot-World and InternData-A1 are released under CC BY-NC-SA 4.0, imposing a strict non-commercial barrier; checkpoints trained on them inherit ‘license stacking’ from data, code (Apache-2.0) and base-model (e.g., Gemma) terms.