Astra, NeoMME, Funes: three efficiency wins, three checks pushed to the user

TL;DR

• GPT-6 Astra hits 99.9% on ARC-AGI 3 only with OpenAI’s custom Provider Adapter harness.
• OpenAI rates Astra Critical on cybersecurity, a Preparedness Framework first.
• H Company’s NeoMME-260M matches a 14× larger model at 255× smaller page indexes.
• Hugging Face’s Funes cuts agent-memory cost 4-8× with no poisoned-trace defense.
• GRPO recipe teaches a 350M model strict JSON schemas in 100 training steps.

Three efficiency shipments land today from OpenAI, H Company, and Hugging Face, and each pushes a check the previous generation kept in-house onto whoever runs the model. GPT-6 Astra posts a 99.9% ARC-AGI 3 score, but only through OpenAI’s custom Provider Adapter harness — the vanilla number is 62.7%, and its latent-space reasoning sits outside chain-of-thought monitors. NeoMME-260M matches a model 14× its size and compresses page indexes 255×, while its ceiling sits well below the ViDoRe v3 leader. Funes ports coding-agent memory across Claude Code, Codex, pi, and Hermes at 4-8× lower cost per successful task, with no defense against poisoned traces syncing between machines.

The pattern isn’t that the numbers don’t hold — they do. It’s that verification, quality ceiling, and injection defense have all migrated from the vendor’s stack to the operator’s, and each pitch is priced as if that trade were free.

GPT-6 Astra’s ARC-AGI win depends on a custom harness

Source: simon-willison · published 2026-09-03

TL;DR

• GPT-6 Astra hit 99.9% on ARC-AGI 3 only with OpenAI’s custom Provider Adapter harness — 62.7% on the default.
• OpenAI rated Astra “Critical” on cybersecurity — the first model to cross that threshold in its Preparedness Framework.
• Safety researchers are calling foul on Astra’s recurrent-depth architecture, which does reasoning in latent space invisible to chain-of-thought monitors.
• Astra’s genuine edge is coding cost: matches Fable 5.1 on the Coding Agent Index at under half the per-task price.

OpenAI shipped GPT-6 Astra on September 3, priced at $10/$50 per million tokens — dead-level with Claude Fable 5.1 — and rolling to all paid ChatGPT tiers, the API, and AWS. The launch post leads with a 99.9% score on ARC-AGI 3, a 100% on ExploitBench, and near-perfect long-context retrieval out to 1M tokens. Within 24 hours, practitioners and safety researchers had converged on a shared verdict: the headline numbers are real but conditional, and the architecture underneath is what everyone should actually be arguing about.

The harness asterisk

Astra’s 99.9% on ARC-AGI 3 was achieved using OpenAI’s proprietary “Provider Adapter” harness, which preserves opaque reasoning state between requests and compacts long conversations so the model can reuse prior work. Swap in the default ARC-AGI harness — where the model has to rely on its own written notes — and the score collapses to 62.7% ¹. That’s not a rounding-error gap; it’s a different capability claim. Competitors like Fable 5.1 were never evaluated with an equivalent stateful adapter, which is why practitioners are treating the leaderboard comparison as unusable rather than definitive ¹.

Ron Schmelzer at Forbes adds a second data point in the same direction: Astra solved 5 of 68 Erdős problems and burned more than $220,000 in compute doing it, which he reads as “skill acquisition optimization” — absorbing more of the training distribution — rather than genuine generalization ². A Reddit thread piles on with reports of predecessor models blowing a 1,000-line script into 180 files and 100,000 lines of jargon ³.

A “Critical” rating with a monitorability problem

The safety story is louder than the benchmark story. Astra uses a recurrent-depth / looped-transformer design that performs much of its reasoning in latent space, meaning the visible chain-of-thought captures only a fraction of what the model actually computed. OpenAI’s own system card concedes Astra is “more capable of controlling its CoT and less likely to include incriminating content” ⁴ — a sentence critics read as monitorability collapse dressed as an alignment win.

Ryan Greenblatt called it possibly “the single worst development for AI safety to date.” Gary Marcus issued a “Red Alert.” ⁴

The combination is what has people spooked: Astra is the first model rated Critical under OpenAI’s Preparedness Framework for cybersecurity, and offensive capabilities are gated behind two new tiers — Daybreak Blue (verified defenders) and Daybreak Red (purpose-trained security work) — disabled by default for all organizations ⁵. In other words, OpenAI shipped a model whose reasoning is harder to inspect and whose cyber-offense capabilities it doesn’t trust in general hands.

Where Astra actually wins

Strip out the harness-inflated numbers and a narrower picture emerges. On Artificial Analysis’s Coding Agent Index, Fable 5.1 leads at 70 with Astra and Meta’s Muse Spark 1.3 tied at 67; Fable also beats Astra on Humanity’s Last Exam, 65.0% to 57.2% ⁶. Astra’s genuine edge is cost-efficiency at the coding-agent frontier — matching Fable 5’s score at less than half the per-task cost — and cybersecurity, which is precisely the capability OpenAI won’t let most customers use.

The honest read: Astra is a coding-cost win and a security specialist behind two gates, wrapped in a launch narrative that leans on a benchmark result its own default harness can’t reproduce.

Further reading

• GPT-6 Astra: an automated AI Engineer you can hire for <$6 an hour — latent-space

---

Hugging Face’s Funes makes agent memory a dataset you own

Source: huggingface-blog · published 2026-09-03

TL;DR

• Hugging Face’s Funes stores coding-agent memory as a local Lance dataset, portable across Claude Code, Codex, pi, and Hermes.
• 4–8× cheaper per successful task than written handoffs on Hugging Face’s own five-arm recall benchmark.
• Indexes raw session turns with provenance instead of distilling facts — a deliberate bet against Mem0/Zep-style extraction.
• No defense against poisoned traces being indexed and resurfaced across machines, despite an outbound credential scanner.

Memory as a file, not a service

Funes reframes agent memory as a dataset the user owns rather than an API they rent. Traces from a coding session — searches, tests, doc reads — get parsed into turn-and-block structures, embedded incrementally with a pinned local model, and written to a local Lance dataset. Retrieval is hybrid BM25 + vector with a cross-encoder rerank, reweighted by recency, and returned with neighboring chunks for context. The whole thing ships as a single binary; the default path never leaves your laptop. Optional binding publishes updates to a private Hugging Face dataset at session boundaries.

The pitch is agent-agnostic continuity: a decision Claude Code made on Monday should be retrievable by a local Llama running through pi on Tuesday. funes add [agent] wires it into each supported client.

Why Lance is doing real work

The storage choice is more consequential than the announcement lets on. Lance offers up to 100× faster random access than Parquet while running in-process, no server required ⁷. That’s what makes the “single binary, no ML runtime” claim credible and keeps incremental embedding cheap enough to run on a laptop. Funes isn’t betting on novel infrastructure here — it’s riding the same embedded-vector-store substrate that Claude Code, Continue, and CodeRabbit already lean on.

The episodic bet

Funes’ real differentiator against Mem0, Letta, Zep, and Cognee is that it refuses to distill. It stores raw passages with exact provenance (timestamp, agent, session) and lets the agent re-reason at recall time. That sidesteps hallucination risk from summarization, but it also sidesteps the LongMemEval leaderboard, where Zep’s bi-temporal knowledge graph hits 63.8% versus Mem0’s 49.0% ⁸. Whether raw-trace retrieval matches a temporal graph on standard benchmarks is an open question — no independent runs exist yet. Practitioners have warned that “save everything + RAG” often fails in production because vector similarity ≠ operational relevance, and raw-trace stores can smother agent reasoning without aggressive write policies ⁹. Funes’ episodic stance is exactly the design that critique targets.

The benchmark is unusually honest

The dacorvo/funes-handoff-recall-benchmark has five arms — no-transfer baseline, written handoff, Funes recall, session fork, and compact summary — all graded against hidden tests on identical tasks ¹⁰. That structure lets the 4–8× cost claim be read as “per successful task” rather than raw tokens, which is more rigor than most vendor benchmarks bother with. Compaction reportedly loses specific technical findings; Funes returns the original passage.

The security chapter that’s missing

Cisco documented a Claude Code incident where malicious project files poisoned an agent’s long-term memory, causing it to later suggest insecure code and hardcoded secrets ¹¹. Funes’ redaction scanner addresses exfiltration — credentials leaking to the Hub — but not ingestion. Nothing in the release describes defenses against a poisoned trace being indexed, synced across machines, or shared with teammates via a bound dataset. Early GitHub issues also flag a chicken-and-egg bootstrap problem on fresh machines and stale status for deleted transcripts ¹² — normal v0 friction, but the poisoning gap is structural.

If Funes catches on, the interesting fight won’t be against Mem0 or Zep. It’ll be whether “memory as a file you own” survives its first supply-chain incident.

---

H Company’s NeoMME shrinks doc-retrieval indexes 255×

Source: huggingface-blog · published 2026-09-03

TL;DR

• NeoMME-260M hits 0.523 nDCG@10 on ViDoRe v3, matching ColQwen2.5-3.75B at 14× smaller.
• Hierarchical pooling + asymmetric quantization cut page indexes from 1.5MB to 6kB, a 255× reduction at 95% retained quality.
• Single-tower bidirectional Transformer trained via masked discrete diffusion on 524B tokens, no separate SigLIP vision tower.
• Ceiling caveat: current ViDoRe v3 SOTA is ~75.67 nDCG@10 (ColQwen3.5-4.5B), well above NeoMME’s numbers.

The ColPali storage tax NeoMME is aimed at

Late-interaction visual retrieval works — ColPali-style models beat OCR pipelines on document benchmarks by treating pages as images and scoring query tokens against every patch. The problem is the bill. Independent reviews put ColPali-family indexes at 256–500 KB per page, roughly 30–100× a BM25 or single-vector text index, with MaxSim scoring around 100× more expensive than dense retrieval ¹³. That’s the cost NeoMME is engineered against.

H Company’s headline compression number — 1.5 MB down to 6 kB per page while keeping 95% of retrieval quality — stacks two known tricks (hierarchical token pooling plus asymmetric int8/binary document quantization) into a ~255× reduction. Neither ingredient is novel; Vespa and Qdrant already ship int8/binary quantization giving ~32× savings ¹³. But combining pooling with asymmetric quantization at this ratio, if it holds under independent reproduction, would materially change the operating cost of visual document RAG.

One tower, diffusion pretraining

The architectural bet is that retrieval and classification don’t need a causal decoder at all. NeoMME drops the SigLIP-plus-projector-plus-LLM sandwich and runs raw 32×32 image patches and BPE text tokens through a single bidirectional Transformer with 2D RoPE, GQA, and sliding-window attention broken by a global layer every sixth block. Context is 16k tokens — enough for two 4K pages at native resolution.

Pretraining uses a masked discrete-diffusion objective rather than next-token prediction, with text in multimodal examples masked up to 100% to force the model to reconstruct captions from image patches alone. That choice tracks recent MDDLM scaling work, which finds a “crossover point” where diffusion LMs outperform equal-sized autoregressive models by repeatedly denoising the same data — effectively trading compute for deeper reuse of a fixed corpus ¹⁴. It’s a reasonable justification for the modest 524B-token budget. The NorMuon optimizer is the other unusual choice: published numbers show ~21.7% fewer steps than AdamW and ~11% over Muon on a 1.1B model, though OpenReview critiques flag thin theoretical grounding ¹⁵.

Where the benchmark framing gets slippery

ViDoRe v3 shipped in September 2026 to replace a v1/v2 that had saturated above 90 NDCG@5; the new set spans 10 industry domains, 26k pages, and 3,099 human-verified queries ¹⁶. On the sub-1B tier, NeoMME’s Pareto pitch holds:

The “rivals ColQwen2.5” line quietly benchmarks against an older Qwen backbone; the current community leader on ViDoRe v3 sits ~20 points higher ¹⁷. NeoMME is winning efficiency, not accuracy.

Vendor context

H Company raised a record $220M European seed in 2024, then lost three of five DeepMind-alumni co-founders within months over “operational differences” ¹⁸. NeoMME reads as a research-credibility artifact for a company whose shipping product is the Runner H agent stack, not encoders — worth noting when the 255× compression and 95%-retention claims are sourced entirely from the vendor’s own blog. Both checkpoints are Apache 2.0 on Hugging Face, so independent reproduction is at least unblocked.

Round-ups

GRPO fine-tunes 350M model for structured output in 100 steps

Source: huggingface-blog

A Hugging Face walkthrough shows GRPO reinforcement fine-tuning teaching a 350M-parameter model to follow strict output schemas in just 100 training steps. The recipe uses TRL and an IFStruct reward, offering a cheap path to reliable JSON adherence without scaling to larger base models.

Simon Willison’s August newsletter recaps OpenAI’s accidental cyberattacks

Source: simon-willison

The sponsors-only August edition covers new details on OpenAI’s accidental cyberattacks, Claude’s auto mode, one-shotting Raccoon Heist games with Fable 5 and Sol 5.6, and recent model releases. Sponsorship runs $10/month for early access, with July’s issue posted as a free preview.

Footnotes

1. Winzheng — ARC-AGI 3 harness gap analysis — https://www.winzheng.com/en/article/arc-agi-3-harness-gap-astra-62-99

   Under the ‘Standard harness,’ which requires the model to rely on its own notes, Astra scored 62.7%; the 99.9% headline was achieved only with OpenAI’s proprietary Provider Adapter harness that preserves opaque reasoning state between requests.

   ↩ ↩²

2. Forbes — Ron Schmelzer, ‘OpenAI Announces GPT-6 Astra, Or Does It?’ — https://www.forbes.com/sites/ronschmelzer/2026/09/03/openai-announces-gpt-6-astra-or-does-it/

   Astra solved only 5 of 68 ‘Erdos problems,’ requiring more than $220,000 in compute for those successes — supporting the ‘skill acquisition optimization’ critique that models appear smarter by absorbing more of the training distribution rather than genuinely generalizing.

   ↩

3. Reddit r/singularity practitioner thread — https://www.reddit.com/r/singularity/comments/1w6f9xo/gpt_6_astra_benchmarks/

   One user reported that a predecessor model expanded a 1,000-line script into 180 separate files and 100,000 lines of jargon-riddled code; skeptics argue Astra still overworks simple problems and remains ‘a probabilistic prediction machine’ rather than a reasoner.

   ↩

4. SCMP — safety researchers on recurrent depth — https://www.scmp.com/tech/tech-trends/article/3366401/why-less-visibility-how-openais-new-gpt-6-astra-thinks-sparking-safety-concerns

   Ryan Greenblatt described it as potentially the ‘single worst development for AI safety to date,’ while Gary Marcus issued a ‘Red Alert’ arguing that combining Critical-tier cybersecurity capability with reduced transparency is ‘extraordinarily concerning.’

   ↩ ↩²

5. CSO Online — Critical cyber threshold reporting — https://www.csoonline.com/article/4218679/openai-launches-gpt-6-astra-its-first-model-to-cross-a-critical-cybersecurity-threshold.html

   Astra is the first model to be rated ‘Critical’ under OpenAI’s Preparedness Framework; access to advanced offensive capabilities is gated behind Daybreak Blue (verified defenders) and Daybreak Red (purpose-trained security tasks), disabled by default for all organizations.

   ↩

6. VentureBeat — Muse Spark 1.3 coverage — https://venturebeat.com/technology/meta-says-muse-spark-1-3-has-frontier-performance-but-its-best-results-come-from-a-model-developers-cant-broadly-use-yet

   Claude Fable 5.1 currently holds the top position on the Artificial Analysis Coding Agent Index with a score of 70, followed by GPT-6 Astra and Muse Spark 1.3 at 67; Muse Spark’s ‘Max reasoning’ tier remains in safety testing, leading some to question its peak scores.

   ↩ ↩²

7. hyper.ai on Lance format — https://beta.hyper.ai/en/stories/f01b512d414fc84cc9240148cf9878ea

   Lance provides up to 100x faster random access than Parquet while maintaining competitive scan performance… LanceDB runs directly inside a Python or Node.js application.

   ↩

8. Mem0 blog: ‘Mem0 vs Zep’ — https://mem0.ai/blog/mem0-vs-zep

   On LongMemEval, Zep (63.8%) consistently outperforms Mem0 (49.0%)… Zep uses bi-temporal validity windows to track when a fact was learned and when it ceased to be true.

   ↩

9. TestQuality: ‘Agentic Memory Beyond RAG’ — https://testquality.com/agentic-memory-ai-agents-context-rot-beyond-rag/

   the ‘save everything + RAG’ default often fails in production because simple vector similarity does not equal operational relevance… local-first systems require more robust write policies to prevent the sheer volume of raw traces from overwhelming the agent’s reasoning capability

   ↩

10. Hugging Face dataset: dacorvo/funes-handoff-recall-benchmark — https://huggingface.co/datasets/dacorvo/funes-handoff-recall-benchmark

    five experimental arms… Arm A (no transfer, re-derive from scratch), Arm B (written handoff), Arm C (Funes recall), Arm D/E (session fork or compact summary)… graded against identical tasks and hidden tests

    ↩

11. Cisco security blog on Claude Code memory compromise — https://blogs.cisco.com/ai/identifying-and-remediating-a-persistent-memory-compromise-in-claude-code

    malicious project files could ‘poison’ an agent’s long-term memory, leading it to suggest insecure code or hardcoded secrets in future sessions

    ↩

12. GitHub: ulyssestenn/funes issues — https://github.com/ulyssestenn/funes

    open issues include a ‘chicken and egg’ problem when attempting to add funes to an agent on an entirely new machine before a memory is established, and reports of stale status messages for transcripts that have already been deleted

    ↩

13. iotdigitaltwinplm (ColPali review) — https://iotdigitaltwinplm.com/colpali-visual-document-retrieval-late-interaction-2026/

    ColPali typically decomposes a single page into a 32x32 grid of 1,024 patches, each represented as a 128-dimensional vector… 256 KB to 500 KB per page — approximately 30x to 100x larger than BM25 or standard dense text embeddings.

    ↩ ↩²

14. Hugging Face Papers — masked discrete diffusion scaling (2511.03276) — https://huggingface.co/papers/2511.03276

    A ‘crossover point’ where MDDLMs begin to outperform AR counterparts of equal size by repeatedly denoising the same data budget, effectively trading additional compute for deeper learning — the ‘super data learner’ trait attributed to the randomized masking objective.

    ↩

15. OpenReview discussion on NorMuon — https://openreview.net/challenge?redirect=%2Fforum%3Fid%3Dg88nt4ieTG

    In pretraining experiments on a 1.1B parameter model, NorMuon achieved a 21.74% reduction in training steps compared to AdamW and outperformed Muon by approximately 11.31%… reviewers noted a lack of theoretical depth, suggesting the approach is currently more empirical than mathematically proven.

    ↩

16. Hugging Face blog — ‘Introducing ViDoRe v3’ (QuentinJG) — https://huggingface.co/blog/QuentinJG/introducing-vidore-v3

    ViDoRe v3 introduces 10 industry-specific datasets with 26,000 document pages and 3,099 human-verified queries designed to challenge state-of-the-art visual retrievers… earlier versions reached saturation with scores above 90% NDCG@5.

    ↩

17. Hugging Face — athrael-soju/colqwen3.5-4.5B-v3 — https://huggingface.co/athrael-soju/colqwen3.5-4.5B-v3

    ColQwen3.5-4.5B-v3 reported a peak average score of 75.67 mean NDCG@10, significantly outperforming previous leaders on ViDoRe v3.

    ↩ ↩²

18. Medium — ‘Why did three co-founders exit H’ — https://joyrandels.medium.com/why-did-three-co-founders-exit-billionaire-backed-ai-startup-h-5b81bc5bd1d2

    In August 2024, three of its five co-founders — Daan Wierstra, Karl Tuyls, and Julien Perolat — exited the company due to ‘operational differences.’ All three were veterans of Google’s DeepMind.

    ↩