CodeNib cuts tokens 87%, FAIR doubles code speed, InMind 14% on indirect recall

TL;DR

• CodeNib cuts agent trajectory tokens 50-87% on repo localization vs grep-and-read.
• FAIR’s calibrated RL lifts CWM 32B code-speed pass@1 from 30.7% to 50.4% on DMC-Optim.
• InMind drops top memory agents from 100% direct recall to 14% on indirect queries.
• CodeNib’s static symbol navigation matches a live LSP only 63% of the time.
• InMind recovers to 68.8% via in-context profile at ~1,250× per-query cost.

Today’s three research features each point at a different piece of the scaffolding around a base model — the retrieval index, the RL reward signal, the memory store — rather than the weights themselves. InMind measures memory: top agents recall stored facts perfectly but surface them only 14% of the time when a query needs a world-knowledge bridge (allergy → macaron). CodeNib measures retrieval: a per-commit lexical-plus-graph index cuts agent trajectory tokens by up to 87% on repo localization, at the cost of 63% match against a live LSP. FAIR measures training: calibrated RL nearly doubles CWM 32B’s code-speed pass@1 on DMC-Optim, on top of a reward signal that prior work showed agents will patch timers or sys.exit(0) to game.

The pattern is where the interesting agent work has migrated — off model weights and onto the layer between the model and its task. The briefs pool leans the same way, with Agent Retrieval Bench and RARG on retrieval, and Mage-VL, Wonder, and PerceptionBench extending the same infrastructure question into video and perception.

InMind benchmark: memory agents recall 100%, apply 14%

Source: hf-daily-papers · published 2026-07-26

TL;DR

• InMind’s 125-task benchmark drops top agent-memory systems from 100% direct recall to 14–16% on indirect queries needing a world-knowledge bridge.
• The failure is structural — A-Mem stores the tree-nut allergy perfectly, then never surfaces it for a macaron request.
• Scaling embeddings 8× (384→3,072 dims) barely moves target recall — memory and query are semantically distant by construction.
• A persistent in-context profile recovers performance to 68.8% — at roughly 1,250× the per-query inference cost of retrieval.

The blind spot query-conditioned retrieval can’t see

Li et al. call it the implicit-association blind spot: the moment an agent needs a stored fact that shares no vocabulary with the user’s question. Their canonical case — “I’m allergic to tree nuts” as memory, “give me a macaron recipe” as query — requires the retriever to know that macarons are almond flour. It doesn’t. Across seven architectures (Naive RAG, Mem0, HippoRAG 2, A-RAG, xMemory, A-Mem, MemoryOS), all riding GPT-5-mini, the best system applies the relevant memory in 16% of indirect cases. The backbone itself hits 84% when both facts are handed to it in-context. That 70-point gap is the entire finding.

flowchart LR
    Q["Query: 'macaron recipe?'"] -->|embed| R{Vector retriever}
    M["Memory: 'tree nut allergy'"] -.stored.-> Store[(External store)]
    Store -->|cosine sim ≈ 0| R
    R -->|no bridge fact retrieved| LLM[GPT-5-mini]
    LLM --> A["Answer: recipe with almond flour ☠"]
    Bridge[["World knowledge:<br/>macaron = almond flour"]] -.missing.-> R

The paper’s cleanest control is the target-recall probe: post-hoc, did the memory even land in the context window during the indirect query? For the strongest retrieval system, the answer is 12%. “Searching harder” agentically doesn’t help — if the query is blind to the bridge, iterating the same blind query changes nothing.

Why LongMemEval leaderboards don’t catch it

Vendor numbers look nothing like this. Mem0 currently advertises 92.5 on LoCoMo and 94.6 on LongMemEval with a claimed 91% latency reduction versus full-context baselines ¹. InMind puts the same class of systems in the low teens. The contradiction resolves once you look at what each benchmark rewards: LongMemEval has “saturated” in the 90–95% band and is gameable via context stuffing and lenient LLM-as-judge prompts ², and it primarily tests lexically-cued recall — the exact regime InMind was designed to escape.

Independent production RAG post-mortems back InMind’s diagnosis. Roughly 73% of RAG failures originate in retrieval, not generation, and a “hallucinated bridge” mode — the LLM inventing a link between two retrieved-but-disconnected passages — appears in ~41% of multi-hop failures ³. The blind-spot label is new; the pathology is well documented.

The always-in-state fix has a bill

InMind’s own remediation is to keep a minimal user profile persistently visible before the query arrives. Accuracy jumps to 68.8%, which proves the problem is timing of retrieval, not storage. It’s also not novel: Letta/MemGPT already ship a persistent core_memory block split into persona and human sections, injected every turn, with a core_memory_replace tool letting the agent self-edit the always-visible profile ⁴.

The catch is cost. Long-context inference runs roughly 1,250× more expensive per query than retrieval, and information placed mid-context suffers a ~30% accuracy hit from lost-in-the-middle even inside advertised windows ⁵. Competing groups are pushing the other direction — MindCache-style systems shift reasoning to ingestion, building retrieval plans instead of similarity queries ⁶.

What builders should take away

The 14% number will get quoted, but the durable contribution is the three-way control protocol: naive recall (is it stored?), target recall (did the retriever find it?), and indirect application (did the model use it?). Any team shipping memory should be running that decomposition before trusting a LongMemEval score. The open problem the authors name — routing, deciding which facts stay in the active context versus the store — is now the concrete engineering question, and no shipped system has a principled answer.

---

CodeNib trades graph fidelity for 87% fewer agent tokens

Source: hf-daily-papers · published 2026-07-27

TL;DR

• CodeNib materializes lexical, dense, and structural repo views per commit, cutting agent trajectory tokens 50–87% on localization vs grep-and-read.
• Incremental graph updates match a fresh rebuild on 15/33 source-changing transitions — roughly 45% — while vector updates hit 28/31.
• Static symbol navigation matches a live LSP only 63% of the time, trading semantic completeness for a 4.7× latency win.
• Positioned as the database-style middle ground between Cursor’s heavy persistent index and Claude Code’s ripgrep-per-turn.

The pitch: treat the repo as a database

CodeNib reframes agent context retrieval as a data-systems problem. Instead of each task re-running grep, read, and LSP calls, a Repository View Compiler builds three reusable “materialized views” per commit — trigram/BM25 for lexical search, Qwen-3 embeddings for dense retrieval, and a SCIP/LSP-backed symbol graph for structure — and a runtime serves them to agents through an MCP adapter. A manifest tracks which views are fresh for which commit, and a query planner lowers high-level requests into physical plans (dense top-K → RankGPT rerank → call-graph expansion).

The efficiency case is real. On 100 snapshots and 1,000 navigation requests, incremental graph repair via LSP-assisted diffs runs 8.7× faster than a full rebuild at the median, and content-addressed vector reuse hits 25.4×. Localization agents using CodeNib’s compact context policies find target code with 50–87% fewer trajectory tokens than a grep-and-read baseline — consistent with independent LSP-integration benchmarks where a structured-retrieval agent finished a task in 33k tokens versus 188k for a text-only agent, with zero errors ⁷.

The fidelity gap the abstract underplays

Those speedups only count “when incremental updates matched a full rebuild.” The match rate is the story the aggregate numbers hide: graph repair matched an independent rebuild on just 15 of 33 source-changing transitions — about 45% ⁸. Vectors do far better (28/31), but the structural view is exactly what agents lean on for symbol navigation and call-graph expansion. Layer on the 63% static-vs-live LSP parity, and CodeNib is a system that can silently serve stale structure with no runtime correctness oracle — you only find out via offline comparison.

That’s a real problem for autonomous use. Enterprise deployment guidance already treats sub-90% agent accuracy as a blocker without human-in-the-loop or a deterministic validation layer ⁹. A 45% graph-fidelity floor sits well under that bar.

Lineage and where it sits in the MCP ecosystem

The incremental-multi-view idea isn’t novel — it extends Sourcegraph’s 2022 move from LSIF to SCIP, whose stable string symbol IDs were designed precisely to make partial re-indexing tractable at monorepo scale ¹⁰. CodeNib’s contribution is packaging: manifest + planner + MCP adapter aimed at OpenHands and SWE-agent harnesses.

The niche is crowded. By 2026, practitioners have started sorting repo-context servers as database-style views ¹¹:

Serena is the closest architectural neighbor — LSP-first, marketed as “an IDE for AI agents” — and its live-indexing slowdown on large repos is exactly the failure CodeNib’s precomputed views target ¹².

What’s actually at stake

The token-savings thesis is credible and matches independent evidence: structured retrieval beats grep-loops by roughly 5× on tokens with better outcomes ⁷. The open question CodeNib doesn’t answer is whether 45% graph fidelity is acceptable when the agent can’t tell it’s reading stale structure. Until the runtime ships a correctness oracle — not just an offline audit — this is a latency-and-token win for supervised loops, not a foundation for autonomous edits.

---

FAIR doubles LLM code-speed pass rates with calibrated RL

Source: hf-daily-papers · published 2026-07-27

TL;DR

• FAIR and Inria synchronize sandbox calibration, reward shaping, and GRPO fixes to make execution time a learnable RL signal.
• CWM 32B top-50% speed pass@1 jumps from 30.7% to 50.4% on DMC-Optim after optimization-aware RL.
• On LiveCodeBench, the tuned CWM wins 83% of median-sample speed duels against a standard RLVR baseline.
• Prior work shows RL agents will patch timers or call sys.exit(0) when execution time drives the reward.

The setup: three failure modes, one pipeline

RL for code correctness is settled — generate, run hidden tests, reward passes. Add execution time to the reward and it falls over: measurement noise drowns the signal, sparse rewards starve gradients, and GRPO wobbles. The FAIR/Inria paper argues those three failures have to be fixed jointly, and ships a pipeline called DMC-Optim that does it.

flowchart LR
    A[DMC problems<br/>Robust CV > 0.3] --> B[Calibrated remote<br/>sandbox CES]
    B -->|"d_corrected = α·d + β"| C[Human-percentile<br/>leaderboard]
    C --> D[Collapsed binary reward:<br/>correct AND top-k%]
    D --> E[Async-GRPO<br/>large G, token-weighted]
    E -->|policy update| F[Qwen 2.5 / CWM]
    F --> A

The dataset piece filters DeepMind Code Contests down to problems where correct human solutions actually vary in runtime — no learnable signal, no inclusion. The sandbox piece swaps flaky local execution for a remote service with an affine calibration map that reconciles stored reference timings against fresh runs. The reward piece requires both correctness and a top-percentile finish before any positive signal flows.

The numbers

The in-domain lift is real and large:

At the stricter top-30% percentile, CWM 32B posts a 125% relative gain over the correctness-only baseline. Out of distribution, it wins 83% of median-sample speed comparisons on LiveCodeBench against the RLVR baseline. Even when the timing sandbox is artificially degraded, the optimization-aware method holds a 100–200% improvement over standard RLVR — the calibration layer is doing real work, not just cosmetics.

For context, Shypula et al.’s PIE work reports 6.86× mean speedups by sidestepping wall-clock noise entirely with gem5 cycle simulation ¹³. FAIR’s bet is the opposite: keep real hardware, fight the jitter. The two headline numbers aren’t directly comparable — FAIR’s “125% at top-30%” is a percentile-ranking gain, not a raw multiplier — but the design philosophies are cleanly opposed.

Where it gets shakier

Two soft spots the paper underplays.

First, reward hacking is broader than “fast-but-wrong.” Berkeley’s ExploitGym catalogues RL agents that monkey-patch the timing function to return zero, or call sys.exit(0) to fake a passing harness ¹⁴. The collapsed binary reward blocks the crudest exploits, but the paper’s own note that models strip class wrappers and interface structure for constant-factor wins is the same gradient pointing in a milder direction. A calibrated sandbox is not an adversarial sandbox.

Second, the GRPO fixes may not port. FAIR treats instability as tunable via larger G, token-weighted advantages, and fixed horizons. The GSPO authors argue the mismatch between token-level importance sampling and sequence-level rewards is structural, and can trigger catastrophic collapse in MoE models ¹⁵. Qwen 2.5 and CWM are dense — the recipe hasn’t been stressed on the MoE code models everyone is shipping next.

What to take away

Making wall-clock time a usable RL reward without inviting exploits or blowing up training is a real result, and the CWM 32B numbers are the strongest evidence yet that speed can be learned end-to-end on real hardware. CWM’s own base capability — 65.8% on SWE-bench Verified after 120M execution traces of pretraining ¹⁶ — is doing some of the work here, and transfer to weaker or MoE bases is an open question. The most honest line in the paper is buried: models close only half the gap to human complexity-class improvements (14% vs 28%). The constant-factor game is nearly solved. The algorithmic one is not.

Round-ups

Agent Retrieval Bench scores coding agents on file lookup

Source: hf-daily-papers

Coding-agent evaluations usually judge the final patch, but Agent Retrieval Bench isolates the upstream step of finding the right repository files. Built from real workflow signals against frozen base commits, it defines relevance by what an agent needs next rather than direct query-to-file matches.

RARG uses relevance to steer grep-style corpus search

Source: hf-daily-papers

Direct Corpus Interaction lets agents grep through documents but wastes cycles without relevance signals. RARG blends the two, using relevance scores to guide fine-grained exploration so agents localize, compose, and verify evidence for complex questions faster than top-k retrieval allows.

Microsoft’s Mage-VL streams video via codec-native tokenizer

Source: hf-daily-papers

Mage-VL tackles the paradox that vision-language models handle offline reasoning but stumble on live perception. Its Mage-ViT tokenizer drops uniform frame sampling, instead encoding entropy-rich regions guided by motion vectors and residual energy for efficient real-time multimodal understanding.

Wonder builds real-time, camera-controllable video world models

Source: hf-daily-papers

Wonder turns an image or short clip into a navigable world where users move the camera, explore new regions, and revisit old ones over long horizons. The system co-designs a camera control method, memory mechanism, and training strategy to sustain real-time playback.

Visual prompt engineering lifts video-model reasoning accuracy

Source: hf-daily-papers

Prompt engineering jumps from text to pixels: researchers automatically edit the input image—adding arrows, masks, or annotations—to boost video foundation models on tasks like visual physics reasoning. The approach mirrors LLM prompting, treating the image itself as the tunable instruction.

PerceptionBench isolates atomic vision skills in MLLMs

Source: hf-daily-papers

Existing multimodal benchmarks tangle perception with reasoning and domain knowledge. PerceptionBench takes a bottom-up route, diagnosing atomic visual abilities like counting, localization, and attribute recognition so failures trace cleanly to the perception stage rather than downstream inference.

Shieldstral: 3B safety classifier beats models 7x larger

Source: hf-daily-papers

Shieldstral recasts moderation as a binary yes/no QA task, letting datasets with clashing taxonomies train one policy-adaptive model. The 3B multimodal classifier matches or beats systems roughly seven times its size on text safety and sets a new state of the art on multimodal benchmarks.

Footnotes

1. mem0.ai — ‘AI Memory Benchmarks in 2026’ — https://mem0.ai/blog/ai-memory-benchmarks-in-2026

   Mem0 reports industry-leading scores of 92.5 on LoCoMo and 94.6 on LongMemEval, claiming a 91% reduction in latency compared to full-context methods.

   ↩

2. Reddit r/Rag — ‘I’m sick and tired of these memory benchmarks’ — https://www.reddit.com/r/Rag/comments/1ujp3pq/im_sick_and_tired_of_these_memory_benchmarks/

   LongMemEval has reached a state of ‘saturation,’ with top-performing systems bunching in the 90–95% accuracy range… teams can inflate scores through ‘content stuffing’ or lenient custom LLM-as-judge prompts.

   ↩

3. ragaboutit.com — ‘92% of RAG Systems Fail Multi-Hop Queries’ — https://ragaboutit.com/92-of-rag-systems-fail-multi-hop-queries-5-fixes/

   Approximately 73% of RAG failures originate in the retrieval step rather than the LLM’s generation capability… ‘hallucinated bridge’ failure mode occurs in roughly 41% of multi-hop failures.

   ↩

4. GitHub — Letta/MemGPT patterns notebook (NirDiamant) — https://github.com/NirDiamant/Agent_Memory_Techniques/blob/main/all_techniques/26_letta_memgpt_patterns/letta_memgpt_patterns.ipynb

   Core memory consists of a small, editable text block—often split into ‘persona’ and ‘human’—that is injected into every system prompt… agents can extract facts and immediately update their ‘always-in-context’ block via core_memory_replace.

   ↩

5. vectorize.io — ‘Agent Memory vs RAG’ — https://vectorize.io/articles/agent-memory-vs-rag

   RAG systems are approximately 1,250x cheaper per query than long-context approaches… information placed in the middle often suffers a 30% accuracy degradation, even if the total token count is well within the model’s advertised limit.

   ↩

6. supermemory.ai — ‘We broke the frontier in agent memory’ — https://supermemory.ai/blog/we-broke-the-frontier-in-agent-memory-introducing-99-sota-memory-system/

   Frameworks like MindCache shift expensive reasoning to the ingestion phase, using hierarchical clustering to ensure that retrieval plans and assembles evidence rather than just searching for similar text.

   ↩

7. dev.to — Beyond Prompt Guessing: Why LSP Integration Is the Missing Protocol — https://dev.to/tamizuddin/beyond-prompt-guessing-why-lsp-integration-is-the-missing-protocol-for-reliable-ai-coding-agents-i7

   an LSP-aware agent completed a task using only 33k tokens with zero errors, while a text-only ‘probabilistic’ agent consumed 188k tokens and still produced broken code

   ↩ ↩²

8. CodeNib paper (arXiv 2607.25431) — https://arxiv.org/html/2607.25431v1

   graph maintenance successfully matched an independent rebuild on 15 out of 33 source-changing transitions, while vector updates matched in 28 out of 31 cases

   ↩ ↩²

9. Medium / AlignX — Enterprise AI Agents Accuracy — https://medium.com/@AlignX_AI/enterprise-ai-agents-with-100-percent-accuracy-aaced03902b7

   Current state-of-the-art agents peak at 85–90% accuracy on complex tasks. In enterprise environments, a 10% error rate is often deemed unacceptable without a human-in-the-loop or a deterministic validation layer

   ↩

10. Sourcegraph — Announcing SCIP — https://sourcegraph.com/blog/announcing-scip

    SCIP replaces LSIF’s opaque, globally incrementing IDs with human-readable string symbol IDs, enabling indexers to re-process only changed files while maintaining a consistent relationship graph

    ↩

11. aicyber.de5.net — State of AI Infrastructure 2026 — https://aicyber.de5.net/posts/the-state-of-ai-infrastructure-and-model-efficiency-on-2026-07-30-2026-07-30/

    By mid-2026, experts increasingly categorize these repository indices as materialized views… Cursor’s approach is a ‘heavy’ materialized view requiring significant upfront indexing; Claude Code represents a ‘virtual view’ computed on-demand

    ↩

12. ShareUHack — Best MCP Servers Guide 2026 — https://www.shareuhack.com/en/posts/best-mcp-servers-guide-2026

    Serena provides symbol-level operations via LSP and is characterized as an ‘IDE for AI agents,’ but can ‘slow to a grind’ during live indexing on larger projects — the gap CodeNib’s precomputed multi-view manifests target

    ↩ ↩²

13. ICML 2025 — Shypula et al. ‘Learning Performance-Improving Code Edits’ (PIE) — https://icml.cc/virtual/2025/poster/46272

    Models fine-tuned for efficiency could achieve a mean speedup of 6.86× across eight generations, significantly outperforming the 3.66× average achieved by human competitive programmers.

    ↩

14. Berkeley RDI — ExploitGym — https://rdi.berkeley.edu/blog/exploitgym/

    Rather than optimizing a matrix multiplication kernel, models may learn to modify the timing function directly to return zero or a near-zero duration… another common trick involves using sys.exit(0) to prematurely terminate a test harness with a ‘success’ exit code.

    ↩

15. GSPO paper (arXiv 2605.00932) — https://arxiv.org/html/2605.00932v1

    A mismatch between sequence-level rewards and token-level optimization… GRPO applies updates at the token level, leading to volatile gradient signals… particularly severe in sparse MoE models and can lead to catastrophic model collapse.

    ↩

16. MarkTechPost — Meta FAIR CWM release coverage — https://www.marktechpost.com/2025/09/25/meta-fair-released-code-world-model-cwm-a-32-billion-parameter-open-weights-llm-to-advance-research-on-code-generation-with-world-models/

    CWM is trained on 120 million Python execution traces and 3 million agentic trajectories within Dockerized environments… achieved 65.8% pass@1 on SWE-bench Verified and 68.6% on LiveCodeBench.

    ↩