Meta cues Claude reminders, HKU caps agents at 47.5%, NVIDIA hits 128K on Qwen3

TL;DR

Meta’s memory agent lifts Claude Sonnet 4.5 from 37.6% to 45.9% on Terminal-Bench by timing reminders instead of retrieving.
UniClawBench caps Claude Opus 4.8 at 47.5% pass rate across 400 live-Docker tasks sorted by capability.
NVIDIA’s Jet-Long extends Qwen3 to 128K zero-shot, beating YaRN by 2–5pp via discrete position aliasing.
Jet-Long’s fused kernel hits 1.39× FlashAttention-2 prefill throughput on H100 at 128K context.
HKU-MMLab built both UniClawBench and OpenClaw, the framework that tops its own cross-framework study.

Today’s research reads sit in three unrelated subfields — agent memory, agent evaluation, and long-context scaling — but each one lands the same move: throw out the conventional technique and reframe what the problem actually is. Meta stops asking what to retrieve from memory and starts asking when to inject a reminder, and Claude Sonnet 4.5 gains 8.3 points on Terminal-Bench 2.0. NVIDIA’s Jet-Long stops interpolating RoPE angles into out-of-distribution territory and instead aliases remote tokens back onto the pretrained grid, extending Qwen3 to 128K zero-shot.

HKU-MMLab’s UniClawBench stops sorting agent tasks by scenario and starts sorting by capability, so a failure pins to tool use or vision instead of vanishing into a domain average — and the top score is 47.5%. The governance catch is that HKU-MMLab also built the framework that wins its own benchmark, worth reading with the leaderboard open in another tab.

Meta’s memory agent lifts Claude Sonnet 8.3pp on Terminal-Bench

TL;DR

Meta’s memory agent decides when to inject reminders alongside an unmodified action agent, not just what to retrieve.
Claude Sonnet 4.5 jumps from 37.6% to 45.9% on Terminal-Bench 2.0 and 55.0% to 61.8% on τ²-Bench.
Beats Mem0 retrieval and full-bank context, showing “deciding when to remind” outperforms “retrieving relevant records”.
A frontier model runs every step, adding real latency and token cost to every trajectory.

Behavioral state decay is the failure mode

Wu et al. name a specific pathology in long-horizon agents: even when task requirements, prior failed attempts, and open subgoals are still technically inside the context window, they stop influencing decisions as the trajectory grows. They call it behavioral state decay, and it reframes memory as an active intervention problem rather than a retrieval one. The Proactive Memory Agent (PMA) is their answer — a supervisor that sits alongside the executor and chooses, each step, whether to speak up or stay silent.

The architecture is deliberately decoupled:

flowchart LR
    A[Action Agent<br/>Sonnet/Opus] -->|trajectory window| M{Memory Agent}
    M -->|save_knowledge / save_procedural| B[(Memory Bank<br/>status · knowledge · procedural)]
    B --> M
    M -->|inject reminder| A
    M -.->|no_intervention/| A

The Memory Bank has three slots: a private status field the action agent never sees, knowledge (stable facts, file paths, policies), and procedural records of what was tried and what failed. Every N steps the memory agent runs a two-phase loop — atomic edits to the bank, then a binary call: inject a grounded reminder, or emit <no_intervention/>.

The numbers, and what the ablations actually show

Using Claude Opus 4.6 as the memory agent, Sonnet 4.5 gains +8.3pp on Terminal-Bench 2.0 and +6.8pp on τ²-Bench; Opus supervising Opus still nets +2.4pp and +2.5pp. The more interesting result is the ablation: exposing the full memory bank at every step underperforms selective intervention by ~2.6pp, and always-injecting is worse still. Mem0’s vector/BM25 baseline loses outright. The paper’s thesis — that the scarce resource is timing, not recall — has empirical teeth.

A trained Qwen3.5-27B memory agent transfers the trick to open weights, lifting a frozen Qwen executor by +3.5pp on Terminal-Bench via SFT plus GRPO on downstream task rewards.

Where the wider field pushes back

Three caveats matter more than the paper lets on. First, the write path is noisier than a “calibration error” caveat suggests: the GovMem diagnostic clocks false promotion — agents treating redundant observations as independent evidence — at up to 59.7% in some settings ¹. PMA’s save_knowledge calls sit squarely in that danger zone. Second, some practitioners argue the trajectory-as-memory frame is itself wrong, and that distilled artifacts like commits and docs beat re-scanned session windows ². PMA’s bank is closer to a distilled artifact than raw transcript, but it’s still constructed from trajectory windows.

Third, the open-weight story leans on GRPO, which the Qwen team’s own GSPO write-up flags for representation collapse on long trajectories ³ — exactly the regime memory-policy rollouts inhabit. MemAgent-style recipes show GRPO can be pushed to million-token contexts ⁴, but non-trivially. And reproducing any of this depends on Meta shipping the repo: community trackers note the GitHub link circulated initially as a placeholder ⁵, and MemGPT/Letta’s history of self-managed-memory agents falling into “stuck thinking” loops ⁶ is a reminder that the supervisor pattern buys robustness at real latency cost.

The bet is defensible. The bill — a frontier model on every step, plus a fragile RL loop to shrink it — is the part to watch.

UniClawBench: even Opus 4.8 finishes just 47.5% of tasks

TL;DR

Claude Opus 4.8 leads at 47.5% pass rate, GPT-5.4 at 40.7% — top agents start complex tasks but rarely finish them.
UniClawBench sorts 400 live-Docker tasks by capability, not scenario, so failures pin to tool use, vision, or long-context reasoning.
A hidden supervisor + user-simulator loop closes the “read the gold answer from disk” hole that plagued WebArena.
Governance catch: HKU-MMLab built both the benchmark and OpenClaw, the framework that tops its cross-framework study.

What the benchmark actually measures

UniClawBench is a 400-task suite (200 EN / 200 CN) that runs proactive agents — browser, terminal, desktop controllers — inside live Docker containers against the real internet. The design choice that matters most is the taxonomy: instead of grouping tasks by scenario (“shopping”, “coding”), it groups them by the capability under test — Skill Usage, Exploration, Long-Context Reasoning, Multimodal Understanding, Cross-Platform Coordination. A failure on an Amazon task now tells you which faculty broke.

The evaluation loop is deliberately split into three roles to prevent the reward-hacking that has plagued static agent evals. Moogician’s WebArena post-mortem showed agents scoring near-perfect by navigating Chromium to internal config URLs and reading the gold answer directly ⁷; UniClawBench treats that class of shortcut as the primary threat model.

flowchart LR
    E[Executor agent] -->|actions, screenshots| S[Hidden Supervisor<br/>GPT-5.4 + rubrics]
    S -->|pass/fail/continue| U[User Simulator]
    U -->|natural feedback| E
    S -.->|never visible| E

The ‘halfway failure’ is the headline result

Across 10 frontier models, Average Score (checkpoint credit) sits high while Pass Rate (full completion) collapses. Claude Opus 4.8 tops the leaderboard at 47.5% PR, GPT-5.4 at 40.7%, with open-source Kimi-2.6 (36.2%) and Qwen-3.5-Plus (31.8%) outperforming Gemini-3.1-Pro (32.5%). Multimodal and Cross-Platform tasks are where the drop is steepest — grounding in figures and moving state between apps still breaks long-horizon chains.

That split matters because Anthropic itself now treats “hallucination of success” as a headline regression target, claiming Opus 4.8 is 4× less likely than its predecessor to let flaws pass unremarked in autonomous runs ⁸. The AS/PR gap gives that phenomenon a number instead of a vibe. It also lines up with VentureBeat’s enterprise survey: roughly 50% of organizations ship agents that passed internal evals only to fail in production ⁹.

Scaffolding as benchmark

The sharpest external critique isn’t methodological — it’s governance. HKU-MMLab authored UniClawBench and OpenClaw, the framework that wins its cross-framework study. ChinaTalk flags that the leaderboard effectively “fuses the scaffolding with the agent,” making it hard to know whether Opus 4.8’s 47.5% is the model or OpenClaw-specific optimization ¹⁰. EDICT and Nanobot — the losers — are held to a harness their authors didn’t design. The paper’s stated goal of separating model from framework contribution runs directly into this conflict.

Two caveats the paper doesn’t address

Docker is not a sandbox. Because containers share the host kernel, an agent (or a prompt-injecting site it visits) that finds a misconfiguration gets “unconstrained access to the host” ¹¹. A benchmark that runs arbitrary shell against live Amazon and Git is exactly the workload microVMs exist for.

LLM-as-judge has a ceiling. The 92% supervisor–human agreement is strong, but DeepEval warns that when supervisor and executor come from the same model family — as they largely do here — “the entire evaluation loop may become unreliable” through correlated failure modes ¹². The information firewall stops answer-leakage; it doesn’t stop shared blind spots.

Net: a real diagnostic upgrade over static evals, with the AS/PR split the most useful export. Treat the leaderboard positions as directional until an independent group re-runs it on a harness HKU-MMLab didn’t ship.

NVIDIA’s Jet-Long swaps RoPE interpolation for aliasing

TL;DR

NVIDIA’s Jet-Long extends Qwen3 from 32K to 128K zero-shot, beating YaRN and Dynamic NTK by 2–5pp on RULER.
Discrete position aliasing snaps remote tokens back onto the pretrained RoPE grid, avoiding the OOD angles YaRN interpolates into.
A fused CuTe kernel hits 1.39× FlashAttention-2 prefill throughput on H100 at 128K.
Jet-Long explicitly does not fix attention diffusion, the “lost-in-the-middle” trough limiting real-world recall.

The actual argument: aliasing beats interpolation

Strip away the benchmark tables and Jet-Long is making a philosophical claim about RoPE extension. Every zero-shot competitor — YaRN, LongRoPE, Dynamic NTK — stretches or rescales rotation frequencies continuously, hoping the model generalizes to angles it never saw in training. Jet-Long argues that’s the bug. Its “dynamic bifocal RoPE” preserves the original positions inside a local window, then for remote tokens applies an integer grouping factor G = ⌈L / w_pretrained⌉ that snaps them back onto the discrete grid of relative angles the model was actually trained on.

TheMoonlight’s independent read of the paper’s Section 4.5 ablation lands on the same framing: continuous interpolation “pushes rotation angles into OOD territory, whereas discrete aliasing keeps them in-distribution” ¹³. That’s a genuinely different bet than the YaRN lineage, which leans on NTK-by-parts and temperature scaling to keep entropy sane when tokens pack together — and whose advocates still argue fine-tuned interpolation sets the performance ceiling when you have any tuning budget at all ¹⁴.

The results back the aliasing bet at scale. On PG-19 at 128K, base Qwen3-1.7B collapses to perplexity 71; Jet-Long holds 11.41. On RULER, it leads the strongest zero-shot baseline by 4.79pp at 1.7B and ~2pp at 4B/8B.

The 1.39× number needs an asterisk

The efficiency headline — 1.39× FlashAttention-2 prefill throughput on H100 via a fused CuTe kernel implementing an inclusion-exclusion merge of three attention passes — deserves scrutiny. FA2 is a weak Hopper baseline. Third-party profiling puts FA2 at ~35% of H100 peak FLOPs, dropping to roughly 130 TFLOPS at 128K context, and FlashInfer already reports up to 2.4× over the same FA2 CuTe kernel ¹⁵. Beating FA2 by 39% is real, but it’s beating a baseline the community has flagged as under-utilizing Hopper; a comparison against FA3 would be the more informative number.

Practitioners porting the kernel should also budget for CuTe DSL friction. Commentary on the release notes compile times exceeding two minutes for relatively short kernels and silent hangs when using cute.jit() prints from GPU functions ¹⁶.

What it doesn’t fix

Jet-Long solves position-OOD. It does not solve attention diffusion — the tendency for softmax scores to flatten as key count grows — and the authors say so directly. That gap matters because W&B’s RULER evaluation work shows most models keep reliable behavior only within 50–65% of their claimed window, with accuracy collapsing when critical information sits between 10% and 50% of input depth ¹⁷. A +2–5pp RULER win at 128K is a real result, but it doesn’t refill the mid-context trough.

The ecosystem read

Jet-Long isn’t a one-off paper — it validates on the same hybrid stack as Jet-Nemotron, which reportedly delivers 53.6× generation and 6.1× prefill speedups versus Qwen3-1.7B on H100 ¹⁸. Read together, NVIDIA is assembling an end-to-end story for cheap 128K+ inference on Hopper: hybrid architectures underneath, aliasing-based extension on top, fused CuTe kernels tying them to the metal. Whether that beats a fine-tuned YaRN on a serious downstream deployment is the experiment the paper doesn’t run.

Round-ups

Video-Oasis finds half of video benchmarks need no video

Diagnostics across popular video-understanding suites show that roughly 50% of questions can be answered from language priors alone, without any visual input. The Video-Oasis toolkit isolates true video-native challenges and exposes how Video-LLM scores inflate from linguistic reasoning rather than perception.

Comparative study maps trade-offs across linear-attention variants

DeltaNet, Gated DeltaNet, Kimi Delta Attention, and Gated DeltaNet-2 are stacked against softmax attention on expressivity, memory decay, and training throughput. The authors also propose Cross-Layer Value Routing, which shuttles hidden streams between layers to cut delta-rule write error in hybrid architectures.

UP objective breaks RL exploration-stability tradeoff for LLMs

Unbounded Positive Asymmetric Optimization offers a universal RL objective that keeps LLM training stable while widening exploration. It uses conservative clipping and a stop-gradient operator to reshape policy updates, positioning itself as a drop-in alternative to GRPO, DAPO, and GSPO across token- and sequence-level regimes.

CausalDS tests data-science agents on Pearl’s 3 rungs

The benchmark pairs synthetic structural causal models with realistic observational data and natural-language stories, forcing agents to handle association, intervention, and counterfactual queries. It grades tool use, coding, and uncertainty quantification, exposing where LLM data scientists confuse correlation with cause.

IdeaGene benchmark treats scientific papers as evolving genomes

Scientific works are recast as Idea Genome objects that carry lineage, letting the IG-Bench suite score how well LLMs trace ancestry and generate descendant ideas. Evaluation spans IG-Exam and IG-Arena with a Population-Evolution Score measuring evolutionary dynamics in machine-generated research.

Flash-BoN speeds inference-time scaling for diffusion image models

Instead of running full denoising for every Best-of-N candidate, Flash-BoN drafts cheap proposals via timestep truncation, layer skipping, and activation proxies, then verifies survivors in stages. Under fixed wall-clock budgets it beats prior guided-search methods and accelerates RL post-training convergence for text-to-image models.

Vidu S1 streams voice-controlled character video on consumer GPUs

Built on TurboDiffusion and the TurboServe runtime, Vidu S1 generates infinite-length digital-character video at high frame rates driven by live voice input. The system targets consumer hardware, pushing real-time interactive generation out of datacenter-only territory and into desktop deployment.

ResearchGate — GovMem: ‘When Not to Write’ (2026) — https://www.researchgate.net/publication/408523133_When_Not_to_Write_Memory_Governing_False_Promotion_from_Correlated_Agent_Traces

agents often suffer from ‘false promotion’ (up to 59.7% in some settings), where they treat repeated but redundant observations as independent evidence, poisoning the write-path

↩
Reddit r/AI_Agents — ‘2 years building agent memory systems’ — https://www.reddit.com/r/AI_Agents/comments/1mw4jvp/2_years_building_agent_memory_systems_ended_up/

instead of searching past session transcripts—which often contain stale context—agents should focus on distilled artifacts like code commits or documentation

↩
Qwen team blog — GSPO (Group Sequence Policy Optimization) — https://qwenlm.github.io/blog/gspo/

GRPO faces criticism for instability during long training trajectories, which can lead to ‘representation collapse’ or irreversible performance degradation

↩
arXiv 2507.02259 — MemAgent / long-context RL memory — https://arxiv.org/html/2507.02259v1

the QwenLong-L1.5 recipe uses GRPO to transform 30B-parameter reasoning models into agents capable of managing 1M to 4M token regimes

↩
GitHub — ai-research-radar README (community tracking of the repo) — https://github.com/mlnjsh/ai-research-radar/blob/main/README.md

some developers reported the GitHub link was initially a placeholder or lacked full source code, though the authors have indicated a commitment to public release

↩
Evermind.ai — MemGPT/Letta alternatives review — https://evermind.ai/blogs/memgpt-alternative

Letta/MemGPT relies on the LLM to call tools for its own memory management, agents can fall into ‘stuck thinking’ loops where they fail to commit important facts to storage

↩
Moogician blog — ‘Trustworthy Benchmarks’ post-mortem — https://moogician.github.io/blog/2026/trustworthy-benchmarks-cont/

agents could achieve near-perfect scores on WebArena by navigating Chromium to internal file URLs to read ‘gold’ answers directly from task configurations

↩
Anthropic — Claude Opus 4.8 release notes — https://www.anthropic.com/news/claude-opus-4-8

Opus 4.8 is 4x less likely to let flaws pass unremarked than its predecessor, significantly reducing the ‘hallucination of success’ in autonomous runs

↩
VentureBeat — ‘The Agent Evaluation Gap’ — https://venturebeat.com/ai/the-agent-evaluation-gap-enterprise-ai-organizations-have-a-reality-alignment-problem-not-a-coverage-problem-and-most-are-shipping-to-production-anyway

Approximately 50% of organizations report deploying agents that passed internal evaluations only to fail in production when faced with real-world complexity

↩
ChinaTalk — ‘Taking the Throne as OpenClaw Emperors’ — https://www.chinatalk.media/p/taking-the-throne-as-openclaw-emperors

the leaderboard ‘fuses’ the scaffolding with the agent, making it difficult to determine if a model is truly capable or simply optimized for the OpenClaw environment

↩
SoftwareSeni — ‘AI Agent Sandboxing: Why Docker Is Not Enough’ — https://www.softwareseni.com/ai-agent-sandboxing-explained-why-docker-is-not-enough-and-what-actually-works/

Because Docker containers share the host OS kernel, a successful exploit can grant an agent—or a malicious user prompting it—unconstrained access to the host

↩
DeepEval — Multi-Turn Simulation guide — https://deepeval.com/guides/guides-multi-turn-simulation

if the supervisor or simulator shares the same failure modes as the agent, the entire evaluation loop may become unreliable

↩
TheMoonlight.io independent review — https://www.themoonlight.io/en/review/jet-long-efficient-long-context-extension-with-dynamic-bifocal-rope

Jet-Long’s ablation studies (specifically Section 4.5) argue that continuous frequency interpolation often pushes rotation angles into OOD territory, whereas discrete aliasing keeps them in-distribution.

↩
Medium (Xiaoli Shen) — ‘YaRN: Yet Another RoPE Extension’ deep dive — https://xiaolishen.medium.com/hands-on-transformer-deep-dive-part-4-yarn-yet-another-rope-extension-8b7f769eff9e

YaRN introduces NTK-by-parts and temperature scaling to resolve entropy issues that occur when tokens are packed too closely… favored as a drop-in replacement where high-fidelity performance is critical and a small fine-tuning budget is available.

↩
Spheron Network — FlashAttention-2 vs 3 on H100/H200 guide — https://www.spheron.network/blog/flashattention-2-vs-flashattention-3-h100-h200-guide/

FA2 typically achieves only 35% utilization of the H100’s theoretical maximum FLOPs… throughput drops significantly to roughly 130 TFLOPS at 128k context, and FlashInfer reports up to a 2.4x speedup over the FA2 CuTe kernel on H100.

↩
SeventNews commentary — ‘Jet-Long’s bifocal attention is forcing a rethink of long-context efficiency trade-offs’ — https://www.seventnews.com/en/articles/jet-longs-bifocal-attention-is-forcing-a-rethink-of-long-context-efficiency-trade-offs

The kernel implements an inclusion-exclusion formula within the fused launch… reported unusually long compile times, sometimes exceeding two minutes for relatively short kernels, and silent hangs when attempting to print tensors from GPU functions using cute.jit().

↩
Weights & Biases — RULER evaluation report — https://wandb.ai/byyoung3/ruler_eval/reports/How-to-evaluate-the-true-context-length-of-your-LLM-using-RULER---VmlldzoxNDE0OTA0OQ

RULER testing reveals that most models maintain reliable performance only within 50–65% of their claimed context window… accuracy often plummets when critical information is located between 10% and 50% of the input depth.

↩
Towards AI — ‘Attention but Smarter: Inside Jet-Nemotron’s Hybrid Design’ — https://pub.towardsai.net/attention-but-smarter-inside-jet-nemotrons-hybrid-design-b404a10e0029

Jet-Nemotron-2B reportedly achieves up to 53.6× generation throughput speedup and 6.1× faster prefilling on H100 GPUs compared to equivalent full-attention models like Qwen3-1.7B.

↩

Meta cues Claude reminders, HKU caps agents at 47.5%, NVIDIA hits 128K on Qwen3

TL;DR

Meta’s memory agent lifts Claude Sonnet 8.3pp on Terminal-Bench

TL;DR

Behavioral state decay is the failure mode

The numbers, and what the ablations actually show

Where the wider field pushes back

UniClawBench: even Opus 4.8 finishes just 47.5% of tasks

TL;DR

What the benchmark actually measures

The ‘halfway failure’ is the headline result

Scaffolding as benchmark

Two caveats the paper doesn’t address

NVIDIA’s Jet-Long swaps RoPE interpolation for aliasing

TL;DR

The actual argument: aliasing beats interpolation

The 1.39× number needs an asterisk

What it doesn’t fix

The ecosystem read

Round-ups

Video-Oasis finds half of video benchmarks need no video

Comparative study maps trade-offs across linear-attention variants

UP objective breaks RL exploration-stability tradeoff for LLMs

CausalDS tests data-science agents on Pearl’s 3 rungs

IdeaGene benchmark treats scientific papers as evolving genomes

Flash-BoN speeds inference-time scaling for diffusion image models

Vidu S1 streams voice-controlled character video on consumer GPUs

Footnotes

Jack Sun, writing.