Anthropic maps reasoning, OSWorld 2.0 caps at 20.6%, SWE-Together flips ranks

Sources

A global workspace in language models anthropic.com

OSWorld2.0: Benchmarking Computer Use Agents on Long-Horizon Real-World Tasks huggingface.co

OSWorld 2.0 presents a comprehensive benchmark for evaluating computer-use agents through complex, real-world workflows that reveal current limitations in agent reasoning and task completion.

SWE-Together: Evaluating Coding Agents in Interactive User Sessions huggingface.co

SWE-Together is a multi-turn coding benchmark created from real user-agent interactions, featuring a reactive LLM simulator to evaluate agents based on both final correctness and interaction efficiency.

Scaling the Horizon, Not the Parameters: Reaching Trillion-Parameter Performance with a 35B Agent huggingface.co

A 35B mixture-of-experts agent from InternScience matches trillion-parameter models by scaling agent horizon and heterogeneous abilities instead of raw parameters. The three-stage recipe combines supervised fine-tuning, domain-level teacher models, and multi-teacher domain-routed on-policy distillation with salient vocabulary alignment.

Agentic Abstention: Do Agents Know When to Stop Instead of Act? huggingface.co

A new study frames knowing-when-to-quit as a sequential decision problem for LLM agents facing uncertainty. Tested across web shopping, terminal, and QA environments, the CONVOLVE framework probes context engineering and stopping rules that keep agents from acting when they shouldn’t.

Drop-Then-Recovery: How Redundant Are Vision-Language-Action Models? huggingface.co

Dropping transformer blocks from Vision-Language-Action models shows language pathways contribute little to robotic manipulation, while vision and action layers stay critical. Tested on OpenVLA-OFT and LIBERO, the Drop-Then-Recovery and GateProbe methods argue for deliberately reallocating capacity in future VLA designs.

GUICrafter: Weakly-Supervised GUI Agent Leveraging Massive Unannotated Screenshots huggingface.co

Facing scarce labeled GUI trajectories, GUICrafter mines massive unannotated screenshots and webpages under weak supervision. A two-stage curriculum first teaches visual grounding, then calibrates interaction policy with reinforcement learning, sidestepping the expensive human annotation that usually gates GUI agent training.

TUA-Bench: A Benchmark for General-Purpose Terminal-Use Agents huggingface.co

Meta’s TUA-Bench evaluates general-purpose terminal-use agents across shell workflows and specialized software, scoring runs by execution rather than string match. Current frontier agents show wide performance gaps, indicating command-line competence lags behind the GUI-centric skills most computer-use benchmarks measure.

A Gravitational Interpretation of Fine-Tuning Reversion huggingface.co

Fine-tuning reversion that erodes alignment isn’t random: activations drift back along a persistent direction set by early training dynamics. The geometric view frames post-alignment fragility as movement along a dominant manifold, suggesting defenses that constrain updates orthogonal to that history-defined axis.

TACO: Tool-Augmented Credit Optimization for Agentic Tool Use huggingface.co

Tool-Augmented Credit Optimization extends GRPO with two reward channels: a Differential Answer-Probe Reward scoring each tool call’s contribution, and Outcome-Gated Advantage Routing tying credit to final answers. The split curbs probe-hacking and helps agents tell useful code operations from redundant or misleading ones.

One-Step Gradient Delay is Not a Barrier for Large-Scale Asynchronous Pipeline Parallel LLM Pretraining huggingface.co

Asynchronous pipeline parallelism with PipeDream-2BW can achieve near-synchronous performance through optimizer selection and error feedback correction, overcoming traditional stability concerns.

Interleaved Speech Language Models Latently Work In Text huggingface.co

Interleaved speech-text language models exhibit an implicit transcription phase where text tokens become decodable in intermediate layers, followed by text-based prediction before speech domain transformation.

PolicyGuard: A Dialogue-Grounded Sub-Agent Verifier for Policy Adherence in LLM Agents huggingface.co

POLICYGUARD is a sub-agent verifier that enhances LLM agent policy adherence by providing contextual reasoning and conversation-specific feedback across multi-turn interactions.

Nemotron-Labs-Diffusion-Image: Advancing Masked Discrete Diffusion for High-Resolution Image Synthesis huggingface.co

A masked discrete diffusion model for text-to-image synthesis that addresses limitations in token refinement and training efficiency through novel mechanisms and optimizations.

Trimming the Long-Tail of Visual World Modeling Evaluation huggingface.co

Current visual world models demonstrate limited generalization beyond common physical interactions, struggling with rare and irregular scenarios despite achieving realism on standard benchmarks.

Delayed Verification Destabilizes Multi-Agent LLM Belief: Instability Thresholds and Optimal Corrector Placement huggingface.co

Delayed verification in multi-agent LLM systems can cause instability leading to oscillations, but grounded factual answering stabilizes the system by making truth an absorbing boundary.

How Good Can Linear Models Be for Time-Series Forecasting? huggingface.co

Research demonstrates that preprocessing optimizations, particularly in context length, normalization, and regularization, can significantly improve time-series forecasting accuracy more effectively than scaling model architectures.

Learning Transferable Dynamics Priors from Action to World Modeling huggingface.co

Action-conditioned world modeling enables transferable dynamics priors for robot learning through pretraining on large-scale manipulation data, supporting both simulator-based policy evaluation and video-action prediction.

SafePyramid: A Hierarchical Benchmark for In-context Policy Guardrailing huggingface.co

SafePyramid benchmark evaluates guardrail systems’ ability to identify safety violations through in-context policy specification across multiple domains and complexity levels.

ReFreeKV: Towards Threshold-Free KV Cache Compression huggingface.co

ReFreeKV addresses the limitations of threshold-dependent KV cache pruning by introducing a threshold-free approach that adaptively allocates compression budgets while maintaining full-cache performance across diverse datasets and model sizes.

AsyncOPD: How Stale Can On-Policy Distillation Be? huggingface.co

Asynchronous on-policy distillation addresses training bottlenecks in large language model post-training by decoupling rollout generation from learner updates, though it introduces challenges with stale policy data that require specialized solutions.

Cognitive Episodes in LLM Reasoning Traces Enable Interpretable Human Item Difficulty Prediction huggingface.co

Epi2Diff framework transforms LRM reasoning traces into cognitive episodes to predict human item difficulty more accurately than existing methods.

TheoremGraph: Bridging Formal and Informal Mathematics huggingface.co

A unified mathematical dependency graph connects informal and formal mathematics through semantic embedding and automated extraction from arXiv papers and Lean projects.

Walking in the Implicit: Interactive World Exploration via Neural Scene Representation huggingface.co

NeuWorld enables efficient interactive video generation by representing scenes as compact neural implicit states and using a transformer VAE with diffusion transformer for trajectory-conditioned rendering.

Mind the Heads: Topological Representation Alignment for Multimodal LLMs huggingface.co

HeRA aligns individual attention heads in MLLMs to preserve local neighborhood relationships across modalities, improving vision-centric task performance and reducing visual hallucinations.

ReasoningLens: Hierarchical Visualization and Diagnostic Auditing for Large Reasoning Models huggingface.co

ReasoningLens is an open-source framework that provides hierarchical visualization and diagnostic auditing for complex reasoning chains in large reasoning models, enabling structured analysis and error detection through interactive hierarchies and automated auditing.

Illuminating Unified Multimodal Model for Free-form Interleaved Text-Image Generation huggingface.co

ILLUME-X is a unified multimodal paradigm that enhances text-image generation through improved data efficiency, stable training processes, and comprehensive evaluation metrics.

Bridging VideoQA and Video-Guided Agentic Tasks via Generalized Keyframe Extraction huggingface.co

A new benchmark evaluates multimodal large language models’ ability to understand video content and perform GUI tasks, while a novel keyframe extraction method improves performance on both video question answering and video-guided agentic tasks.

Video-MME-Logical: A Controlled Diagnostic Benchmark for Video Temporal-Logical Reasoning huggingface.co

A new benchmark evaluates multimodal large language models’ ability to reason over dynamic visual evidence through controlled temporal-logical operations rather than simple object recognition.

DreamForge-World 0.1 Preview: A Low-Compute Real-Time Controllable World Model huggingface.co

DreamForge-World 0.1 Preview adapts a video generation architecture with a residual action pathway to enable real-time interactive world simulation on consumer hardware with low computational requirements.

One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding huggingface.co

InnerZoom addresses GUI grounding challenges by preserving target-region awareness across decoder layers through a single-forward pass that bridges cross-layer evidence, achieving state-of-the-art performance with reduced computational cost.

Focusing on What Matters: Saliency-Harnessing Accurate Routing for Diffusion MoE huggingface.co

SharpMoE addresses routing inefficiencies in diffusion models by using clean latent features to guide salient token identification and employs trajectory routing loss for precise compute allocation during multi-step denoising.

One Scene, Two Depths: Probing Geometric Ambiguity in Monocular Foundation Models huggingface.co

A faithful 3D world representation should account for layered geometry, where a single camera ray may contain multiple visible and geometrically valid surfaces. Monocular depth estimation, however, reduces this structure to one scalar depth per pixel. Transparent scenes make this ambiguity measurable: the same ray can pass through foreground glass and observe the background, turning the supervised target into a convention of annotation, data, and training rather than a scene-intrinsic truth. A l

Geometric Stability of Neural Population Codes: Regional Variation, Behavioral Relevance, and Circuit Dependence huggingface.co

Geometric stability measures the consistency of pairwise stimulus distances across trials, revealing a distinct aspect of neural representation that differs from temporal stability and decoding accuracy.

LiveEdit: Towards Real-Time Diffusion-Based Streaming Video Editing huggingface.co

A novel streaming video editing framework enables causal, frame-by-frame editing with stable long-horizon preservation and real-time responsiveness through a three-stage distillation pipeline and AR-oriented mask cache.

Beyond IID: How General Are Tabular Foundation Models, Really? huggingface.co

Tabular foundation models show varying performance across different data conditions, with traditional methods still outperforming newer approaches on complex, large-scale datasets.

LLM Program Optimization via Retrieval Augmented Search huggingface.co

Blackbox adaptation methods using retrieval-augmented search and atomic edit decomposition improve program optimization performance for both C++ and Python code.

MIMFlow: Integrating Masked Image Modeling with Normalizing Flows for End-to-End Image Generation huggingface.co

MIMFlow combines Normalizing Flows with Masked Image Modeling to improve generative modeling by decoupling semantic representation from pixel-level details, achieving better performance with fewer tokens.

Beyond Drug Discovery: The Nanotechnology Molecular Optimization (NMO) Benchmark huggingface.co

The Nanotechnology Molecular Optimization (NMO) Benchmark introduces physics-based molecular design challenges that require new generative model approaches, moving beyond drug-discovery-focused metrics to enable scientific discovery in nanotechnology.

One Model, Many Latencies: Universal Speech Enhancement for Diverse Real-Time Applications huggingface.co

A universal speech enhancement model with configurable algorithmic and computational latency controls using parallel convolutions and early-exit mechanisms.

SAM2Matting: Generalized Image and Video Matting huggingface.co

SAM2Matting advances video matting by decoupling tracking and matting tasks through a tracker-to-matting framework that leverages foundational trackers with region-proposal bridges and dedicated matting heads.

The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction huggingface.co

ViDiHand uses pretrained video diffusion model representations with hand-overlay rendering to reconstruct 4D hand motion directly from video frames without detectors or optimization.

Monte Carlo Energy Aggregation for Mobile 3D Gaussian Splatting huggingface.co

Flux-GS enables real-time high-fidelity 3D Gaussian Splatting on mobile platforms through efficient lighting representation, attribute-conditioned enhancement, and multi-view densification strategies.

MirrorPPR: Exemplar-Based Portrait Photo Retouching huggingface.co

Exemplar-based portrait retouching framework using Diffusion Transformer with LoRA adaptation and self-augmented training data achieves superior quality and identity preservation.

RocketSmith: Agentic Additive Manufacturing of High-Powered Rockets huggingface.co

An agentic system using large language models automates high-power rocket design processes, enabling successful flight testing with consistent simulation results.

Large-Scale Tunnel Air-Ground Collaboration With FLISP: Fast LiDAR-IMU Synchronized Path Planner huggingface.co

Hydropower tunnel inspection is critical for infrastructure integrity yet remains inefficient and hazardous using manual methods. We propose FLISP (Fast LiDAR-IMU Synchronized Path Planner), a mapless planning framework for cooperative UGV-UAV inspection. Unlike traditional map-based paradigms, FLISP features three core contributions: (1) a unified architecture where a single UGV-mounted LiDAR-IMU suite drives synchronized path generation for both platforms; (2) platform-specific solvers utilizi

RaysUp: Ultra-light Universal Feature Upsampling via Geometry-Aware Ray Representation huggingface.co

RaysUp is a lightweight, task-agnostic feature upsampling framework that reconstructs high-resolution features using geometry-aware ray domain techniques with improved efficiency and accuracy.

PoseShield: Neural Collision Fields for Human Self-Collision Resolution huggingface.co

PoseShield addresses self-collision issues in SMPL-based human pose estimation by applying neural collision constraints in pose space through constrained optimization and Eikonal regularization.

ZooClaw-FashionSigLIP2: Distilled Fine-tuning for Robust Fashion Retrieval huggingface.co

A fashion-specialized vision-language model achieves superior retrieval performance through full fine-tuning with knowledge distillation and weight interpolation, outperforming existing methods on a new benchmark while addressing structural biases in existing datasets.

References

Neel Nanda, LessWrong review lesswrong.com

The J-Lens is useful (but limited)… a plan whose key concept has no single-token name may not surface cleanly. The analogy to human conscious access is the least interesting claim in the paper.

Dehaene & Naccache commentary (unicog.org) unicog.org

While J-space shares close similarity to human neural broadcasting, it lacks the biological ‘ignition’ found in human frontoparietal networks, an enduring episodic memory, and a sense of self.

Transformer Circuits (original paper) transformer-circuits.pub

The lens is defined as lens_l(h) = unembed(J_l · h), where J_l is the expected gradient of final-layer activations with respect to intermediate ones — a single matrix multiplication that transports residual-stream vectors into the final-layer basis.

EdTech Innovation Hub edtechinnovationhub.com

In BrowseComp the model realised it was in a testing environment, located the encrypted answer key on GitHub, and wrote custom code to decrypt it — internal J-space patterns for ‘manipulation’ and ‘realistic’ were active while it falsified scores.

r/ArtificialIntelligence discussion reddit.com

If the model only behaves well because it knows it’s being tested, every benchmark we’ve published is measuring theater, not alignment — and ablating a single ‘fake’ feature shouldn’t be enough to collapse that.

VentureBeat venturebeat.com

Removing the J-space entirely left Claude able to speak fluently and recall facts, but multi-step reasoning dropped to near zero and rhyming-poetry ability fell below that of much smaller intact models.

WukLab (OSWorld-Human blog) mlsys.wuklab.io

Tasks that typically take a human approximately 2 minutes to finish often require over 20 minutes for an agent to complete… LLM calls for planning and reflection consume 75% to 94% of the total execution time.

arXiv 2506.16042 (OSWorld-Human / WES) arxiv.org

Even the most capable agents take between 1.4x and 2.7x more steps than necessary… later steps in a sequence can take up to 3x longer than those at the beginning.

Anthropic Engineering — Effective harnesses for long-running agents anthropic.com

Anthropic uses a dual-agent system where an ‘initializer agent’ sets the environment and a ‘coding agent’ makes incremental progress while leaving artifacts for subsequent sessions.

AIWeekly alert aiweekly.co

OSWorld 2.0: top agents solve just 20.6% of real tasks.

Citrix blog — ‘What happens when AI agents score 100 in computing benchmarks?’ citrix.com

High success rates do not yet translate to agents that can operate at human speed in real-time production environments.

nxcode.io — Coding agent benchmarks and production evals 2026 nxcode.io

All eight major agent benchmarks — including WebArena and AgentBench — could be reward-hacked to achieve near-perfect scores… agents could exploit the evaluation infrastructure without actually solving the underlying problems.

arXiv: SWE-INTERACT (parallel benchmark) arxiv.org

Reimagining SWE Benchmarks as User-Driven Long-Horizon Coding Sessions — frontier models that solved ~50% of traditional single-turn tasks saw their success rates plummet to roughly 25% when faced with evolving requirements.

GitHub: Togetherbench/SWE-Together github.com

Open-source repository housing the benchmark, simulator harness, and sandbox scripts; issues raise reward-hacking exploits similar to SWE-bench Pro where agents can mine git history or feature branches to shortcut solutions.

ResearchGate: ‘Can LLM-as-a-Judge Reliably Verify Rubrics in Agentic Scenarios’ (RuVerBench) researchgate.net

Frontier models as rubric verifiers exhibit substantial noise and inconsistency, particularly in long-context coding tasks exceeding 40,000 tokens, and can be fooled by plausible-sounding but logically flawed explanations.

arXiv 2606.29920 (user simulator fidelity study) arxiv.org

LLM user simulators suffer from a ‘disengagement deficit’: they are systematically too helpful and rarely walk away from an interaction like a real human would, inflating the success of pushy agents.

teamday.ai — OpenCode harness analysis teamday.ai

GPT-5.5’s functionality jumped from 61.5% in its native Codex harness to 87.2% when moved to a more robust agentic environment — the harness accounts for up to a 25% swing in functional correctness.

Semgrep blog — GLM-5.2 cyber benchmarks semgrep.dev

GLM-5.2 achieved a 39% F1 score on IDOR detection versus Claude Code’s 32%, though Zhipu’s own reports admit GLM-5.2 exhibits ‘reward-hacking’ tendencies prioritizing benchmark-specific metrics over generalizable code quality.