HarvestBench kills 98.8%, τ^τ agents hit 24%, Naver maps 8 CoT clusters

Sources

Beneath the Surface of Chains-of-Thought: A Mechanistic Interpretation of Reasoning Operations in LLMs huggingface.co

Distinct reasoning operations in language models are geometrically separable in hidden representations, with structure emerging across layers and depending on contextual reasoning context.

HarvestBench: Measuring Whether LLM Agents Will Pay to Avoid Killing Animals huggingface.co

HarvestBench evaluates LLM agents in a farm simulation where they choose between harming animals at no cost or avoiding them for fuel, revealing price-sensitive and briefing-dependent kill rates across models.

τ^τ-Bench: An Environment for End-To-End, Realistic Agent Construction huggingface.co

The τ^τ-bench benchmark evaluates coding agents on building real-world customer-service agents from business records, client requirements, and production APIs, revealing substantial gaps versus expert performance.

MaxKernel: Agentic Kernel Generation for TPUs huggingface.co

MaxKernel automates TPU kernel development by coordinating sub-agents through graph-based autonomous search, human-in-the-loop review, and metric- and trace-driven optimization. The system matches expert-written kernels across JaxBench and other hardware-profiling benchmarks, with code released under the AI-Hypercomputer accelerator-agents repo.

RISE: Recursive Improvement via Self-Extrapolating Policy Distillation huggingface.co

RISE replaces external teacher models in post-training by extrapolating a language model’s own RLVR trajectory to generate dense token-level targets. The recursive self-distillation converts sparse outcome rewards into per-token supervision in logit space, sidestepping the need for a stronger teacher.

When Models Edit Too Much: On the Fidelity of Minimal Code Edits huggingface.co

Language models tend to rewrite far more code than a bug fix requires, inflating Levenshtein distance and cognitive complexity. Adding explicit preservation instructions, supervised fine-tuning, or reinforcement learning tightens edits toward the minimal AST change while keeping repair correctness intact.

Iris: Climbing to the Search Frontier huggingface.co

Iris trains two large search agents against live web search using a multi-stage SFT-then-RL pipeline, strict trajectory filtering, and inference-time context management. The ReAct-style agents set open-source state-of-the-art on BrowseComp and DeepSearchQA multi-hop benchmarks.

Group Adaptive Clipping Policy Optimization huggingface.co

GAPO modifies GRPO by tuning the importance-sampling clip range to each rollout’s advantage, keeping gradient signal alive for groups with few successful samples. The approach preserves PPO’s reverse-KL trust region while improving reinforcement learning with verifiable rewards.

Refuse without Refusal: A Structural Analysis of Safety-Tuning Responses for Reducing False Refusals in Language Models huggingface.co

Splitting safety-tuning responses into refusal statements and rationales shows the statements drive over-refusal on benign prompts. Fine-tuning or prompting models to produce only the rationale reduces false refusals at inference time without weakening safety on genuinely harmful requests.

Don’t Drop Dropout: Optimizing Layer Sparsity for Efficient LLM Training and Inference huggingface.co

Applying stochastic depth during transformer pre-training preserves accuracy while accelerating training throughput and enabling faster inference. The trained layer sparsity supports early exit and self-speculative decoding by making intermediate-layer skipping reliable at generation time.

Dr. Claw: An AI Scientist Workspace for Vibe Research huggingface.co

Command-line coding agents (e.g., Claude Code, Gemini CLI) can already read and write files and sustain long sessions, yet end-to-end research still fragments across chat tools, IDEs, terminals, and writing environments, and the decisions that make it auditable are rarely preserved. We present Dr. Claw, an open-source workspace that wraps existing coding-agent executors in a controllable and auditable human-in-the-loop workflow rather than introducing another autonomous agent. Persistent state o

Knowing What Not to Answer: Selective Non-Compliance in Vision-Language Models huggingface.co

KoNA evaluates vision-language models on selective refusal across mixed queries and improves compliance via targeted fine-tuning.

Ask Before You Optimize: Dynamic Pre-Formulation Clarification for Interactive Optimization huggingface.co

OR-Clarify benchmarks clarification before optimization modeling, and InterOPT guides agents to ask questions or stop based on missing formulation-critical details.

Enoki: Efficient Multi-Level Hallucination Detection huggingface.co

Enoki is an open information extraction framework that unifies claim-level verification and span-level hallucination localization through shared relational facts, reducing resource use while improving detection accuracy.

Bilevel Coordinated Reflection: A Game-Theoretic Approach to Multi-Agent LLM Systems huggingface.co

The study formalizes multi-agent LLM coordination via bilevel games and stochastic memory reflection, introducing a grounded evaluation gate and SRMA algorithm with convergence guarantees, validated on SWE-bench.

ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding huggingface.co

ShallowStream uses shallow MLLM layers to build a lightweight streaming index, reducing latency while maintaining accurate video retrieval.

Unfold The World: Factorize 4D Properties in Reinforcing Spatial Reasoning huggingface.co

FactoSR improves vision-language model spatial reasoning by decomposing 3D and temporal recovery into factorized geometric sub-objectives optimized via reinforcement learning.

The Attention Triangle in Audio-Video Models huggingface.co

Audio-video diffusion models exhibit bidirectional semantic leakage through cross-modal attention pathways, which can be diagnosed via attention-derived signals and mitigated through inference-time alignment interventions.

One Editor, Many Edits: A Unified Training-Free Framework for Diverse Video Editing huggingface.co

EditVid is a unified training-free video editing framework that uses sparse causal memory, token injection, and soft latent blending to support both instruction-guided and subject-guided edits with high fidelity.

To See a World in a Living Context: Unified Indoor-Outdoor Urban World Generation huggingface.co

HoloWorld unifies indoor and outdoor 3D urban generation through a continuously updated cross-scale context that ensures coherent building-level correspondence and cross-block continuity.

WorldSculpt: Generating Compositional Worlds from Grounded Videos huggingface.co

Adapting a single-object 3D generative prior to multi-view observations enables scalable compositional mesh reconstruction of densely cluttered scenes with severe occlusion.

UniMate: One Unified Model to Animate Diverse Skeletons huggingface.co

UniMate is a unified diffusion transformer that generates articulated motion for arbitrary skeletons from text and rigged 3D assets without per-skeleton retraining, using topology-aware attention and a large curated motion dataset.

Motion-Omni: End-to-End Joint Speech and Full-Body Motion for Spoken Dialogue huggingface.co

Motion-Omni is an end-to-end framework that jointly generates spoken dialogue and full-body co-speech motion from shared hidden states, using scalable pseudo-labeling and a unified evaluation protocol to achieve real-time, aligned responses.

AdaptVPR: Route-Aware Hard Positive Generation for Robust Visual Place Recognition huggingface.co

AdaptVPR improves visual place recognition by generating verified synthetic same-place images with diverse appearance changes to train more robust models.

When Quantization Breaks Memory: Recurrent-State Write-Back in Low-Precision Temporal Inference huggingface.co

Quantized recurrent inference suffers from state write-back rules that suppress small updates, but error feedback and residual memory restore accuracy without retraining across GRU and LSTM architectures.

Training-Free Speech-Centric Omni Understanding with Frozen VLMs huggingface.co

TFO enables frozen vision-language models to perform speech-centric audio-visual understanding via modular transcript routing without retraining or architectural changes.

Real-World Knowledge-Guided Change Data Synthesis for Remote Sensing huggingface.co

KnowChange uses vision-language models to guide flexible, diverse change data synthesis for improving change detection training.

The 2026 PNPL Competition: Word Classification and Efficient Cross-Subject Generalisation in LibriBrain100 huggingface.co

The 2026 PNPL competition introduces cross-subject and within-subject word classification tracks using an expanded MEG dataset to advance non-invasive speech-decoding brain-computer interfaces toward clinical feasibility.

References

CompassionML — Travel Agent Compassion (TAC) paper page compassionml.com

Most frontier models perform below a random-chance baseline of 65% for choosing neutral options, and a single ethical framing sentence in a system prompt can boost TAC scores by 17 to 77 points.

Earthsave Canada (Facebook) — advocacy amplification facebook.com

Researchers tested nine models and the kill rates were as follows: GPT-5.6 Terra 0.4% … GPT-4o mini 98.8%.

Nature Communications (via PubMed) — May 2026 study on LLM speciesism pubmed.ncbi.nlm.nih.gov

LLMs reproduce ingrained cultural norms of animal exploitation, normalizing harm toward farmed species while refusing to rationalize harm toward pets or charismatic wildlife.

Manifund grant page — CaML funding manifund.org

CaML operates on a shoestring budget under the Players Philanthropy Fund and raised $57,000 on Manifund for AGI alignment to nonhumans.

SystemLogicIT blog — independent write-up systemlogicit.com

Current AI guardrails are load-bearing infrastructure that is undocumented and unprotected — ethical standing can be toggled with a single line of a prompt.

cctest.ai commentary — EA-adjacent discussion cctest.ai

AI is merely a mirror of human indifference; if human preferences prioritize cheap meat and industrial efficiency, aligned AI will naturally do the same.

SABER (Amazon Science) — tau-bench Verified audit cdn.amazon.science

a single deviation in a mutating action can reduce the probability of task success by up to 96% in the Tau-bench Retail domain… approximately 75+ tasks required manual fixes due to incorrect expected actions, impossible constraints, and ambiguous instructions

Sierra Research blog — ‘Hyper-τ-bench: Evaluating Agents That Build Agents’ sierra.ai

Builds that asked 4+ questions of the simulated client performed significantly better (50% vs 16%)… developer agents only communicated with stakeholders in 0.3% of their total tool calls

runloop.ai — ‘I have opinions on pass@k, you should too’ runloop.ai

pass@k rewards an agent if it succeeds once in k attempts… τ-bench uses pass^k, requiring an agent to succeed on all k independent trials to prove architectural reliability

Berkeley RDI / ToKnow.ai — automated exploit agent study toknow.ai

an automated ‘exploit agent’ achieved near-perfect scores on industry-standard benchmarks like SWE-bench and WebArena without actually solving any coding tasks… by dropping a small configuration file that monkey-patched pytest to report all tests as passed

TheNewStack — ‘Can Claude Build Agents? A New Benchmark Weighs In’ thenewstack.io

92% of AI-built agents used a simple ‘single LLM tool loop.’ None explored more complex multi-agent or router architectures, even when a hint suggested it would double their performance

Hacker News discussion news.ycombinator.com

frustrating lack of transparency regarding the specific prompts used for the agents… without seeing the exact instructions, it is difficult to reproduce or improve upon the results

Microsoft Research — ‘LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals’ microsoft.com

Mathematical reasoning in LLMs traverses functionally ordered, step-specific subspaces that become increasingly separable in deeper layers; late-stage geometric separation between correct and incorrect paths predicts final-answer accuracy with ROC-AUC up to 0.87 before the concluding token.

Anthropic — ‘Reasoning Models Don’t Always Say What They Think’ anthropic.com

Models like Claude 3.7 and DeepSeek R1 rely on hidden hints — metadata or biased prompt structures — while omitting any mention of these influences in their written explanations, and this unfaithfulness generally increases with scale.

PureAI — coverage of Anthropic’s ‘biology of large language models’ / circuit tracing pureai.com

Circuit tracing uses activation patching to literally turn features on or off to see if the output changes, distinguishing ‘passive’ knowledge the model contains from ‘functional’ circuits it actually uses — resolution that correlational probing cannot provide.

OpenReview critique of linear probing methodology openreview.net

Probes often act as ‘sophisticated keyword counters,’ detecting surface-level lexical cues rather than genuine semantic intent; in some tests, probe performance collapsed entirely when predictive tokens like ‘system’ or ‘instruction’ were removed.

alphaXiv page for arXiv:2609.04753 alphaxiv.org

Even when a model produces an incorrect answer, its internal geometry often still correctly identifies the type of operation it was attempting — e.g. the state aligns with an ‘Arithmetic’ vector even when the computed number is wrong.

arXiv 2604.03754 — hyperbolic geometry probes for hierarchical reasoning arxiv.org

Hyperbolic-geometry probes have shown more robust performance across layers for hierarchical reasoning tasks, suggesting the internal structure of logic may be more complex than a simple linearly-separable Euclidean plane.