WeatherNext trims 140km, Frontis-MA1 rides search to 71%, BM25 beats RAG agents

TL;DR

• WeatherNext cuts NHC cyclone track error 140km below leading physics ensembles at 8× speed.
• Frontis-MA1-35B hits 71% on MLE-Bench Lite, mostly via the OpenMLE-Evo search harness.
• BM25 overtakes file-system agents past 10M tokens on a 601M-token enterprise corpus.
• Qwen-UI-Agent tops Hugging Face’s daily list at 302 upvotes for cross-platform GUI plus CLI execution.

Three research releases land today, and each rewards a reader who checks where the headline number actually comes from. DeepMind’s WeatherNext posts the cleanest lede — 140 km closer to observed cyclone tracks than leading physics ensembles across the NHC’s 2025 season — while its own error bars concede that AI models still blur the storm’s inner core and can be nudged by perturbing less than 0.1% of global observations. Frontis-MA1-35B clears 71% on MLE-Bench Lite, but the base model scores 39% on the same benchmark without the OpenMLE-Evo search harness carrying it. And Onyx AI’s long-context retrieval study finds BM25 overtaking file-system agents past 10M tokens on a 601M-token enterprise corpus — a striking result that omits the hybrid-plus-reranker baseline every 2026 production stack already runs.

On Hugging Face’s daily list, Alibaba’s Qwen-UI-Agent (302 upvotes) and the Metis native-memory reframe (269) headline seven brief-length round-ups covering memory modules, resettable environments, and hybrid diffusion backbones.

DeepMind’s WeatherNext cuts NHC cyclone track errors by 140km

Source: deepmind-blog · published 2026-08-06

TL;DR

• WeatherNext ran 140 km closer to observed cyclone positions than leading physics ensembles across NHC’s 2025 season.
• Its Functional Generative Networks — 180M params, 6-hour steps — beat GenCast on 99.9% of variables at ~8× speed.
• AI models still blur the cyclone inner core, underestimating peak winds and minimum central pressure.
• Perturbing <0.1% of global observations can hide or fabricate a hurricane in AI forecasts — unaddressed at launch.

The lead-time claim actually checks out

The most striking thing about WeatherNext’s release is that the NHC’s own verification numbers back the marketing. In the 2025 Atlantic season, the model logged as GDMI in NHC files came in about 140 km closer to observed cyclone positions than the leading physics-based ensembles, with 72-hour track errors under 80 miles and Hurricane Melissa’s 5-day track error near 100 nmi — roughly half the five-year rolling average ¹. That is the quantitative form of “an extra day of lead time,” and it is unusual to see a Google Nature paper this cleanly corroborated by a government operational scoreboard within weeks.

The architecture behind the second-generation model is also now documented externally. WeatherNext 2’s FGN is a ~180M-parameter Graph Transformer with 24 layers operating on 6-hour steps, and reportedly outperforms the diffusion-based GenCast on 99.9% of variables and lead times at roughly 8× the speed ². That’s what makes 1,000-member ensembles in under a minute on a single TPU tractable.

ECMWF is pursuing the opposite bet: its AIFS runs alongside physics-based IFS, and AIFS-TC is a bolt-on ML corrector for intensity errors on standard AI outputs — Europe’s answer is explicitly hybrid rather than pure-AI ³.

Where it still loses to physics

Two independent critiques dent the triumphalism. A Rice University group finds AI models of this class systematically underestimate maximum wind speeds and minimum central pressures because their outputs are too smooth to resolve a cyclone’s compact inner core ⁴. A broader Physics World survey extends the point: on record-breaking, out-of-distribution storms, physics models like ECMWF’s HRES still beat AI, which is trained on ERA5 reanalysis and regresses toward historical norms ⁵.

AI weather models excel at average skill but frequently underperform traditional numerical models… on out-of-distribution ‘black swan’ events. ⁵

Melissa is thus the storm AI got right. The class it is structurally worst at is precisely the unprecedented one — which is also the class emergency managers care about most.

The security question DeepMind didn’t answer

The launch post is silent on the attack surface created by open-sourcing weights. A Michigan State paper, “Cyc-Attack,” shows that gradient-based adversarial perturbations to less than 0.1% of global weather measurements — comparable to tampering with one of roughly a hundred active satellites — can steer or hide a cyclone in DLWF forecasts including WeatherNext and GenCast, using a differentiable surrogate to bypass non-differentiable detectors like TempestExtremes ⁶. As NHC operationalizes GDMI, that is a live governance question, not a hypothetical.

The honest scoreboard reads: AI is now the best single model at the National Hurricane Center, but only inside its training distribution, and nobody has yet audited its adversarial robustness.

---

Frontis-MA1 hits 71% on MLE-Bench Lite, mostly via search

Source: hf-daily-papers · published 2026-07-29

TL;DR

• Frontis-MA1-35B hits 71.21% Medal Average on MLE-Bench Lite, edging GPT-5.5+Codex (68.2%) and nearing 2.8T Kimi K3 (72.7%).
• The OpenMLE-Evo search harness carries the run — the base 35B jumps from 39.39% to 71.21% with search.
• Concurrent open agent MLEvolve already reports 65.3% on the full 75-task MLE-bench with the same 12-hour budget.
• Cloud Security Alliance calls the release a “security-relevant threshold” for AI co-developing successors, not autonomous recursive self-improvement.

The claim and the catch

Frontis-MA1 ships as a 35B “meta-evolution agent” post-trained on four program-evolution operators — Draft, Improve, Debug, Crossover — plus OpenMLE-Gym, a 5,758-task executable benchmark pool, and OpenMLE-Evo, an inference-time evolutionary search harness. On MLE-Bench Lite the full stack posts 71.21% Medal Average, which the paper frames as beating GPT-5.5+Codex and rivaling frontier MoE models an order of magnitude larger.

That framing is split-specific. MLE-Bench Lite is the 22-task low-complexity subset that nearly every new agent reports on, because a full-bench run costs around $48K and shows a 9–13% validation-to-test generalization gap ⁷. MLEvolve, a concurrent evolutionary agent using graph-based search, ran the full 75-task set at 65.3% and hit 80.3% on the low-complexity split — the same regime as Lite — but Frontis-MA1 never evaluates against it ⁸. The “SOTA vs GPT-5.5” line reads differently once a directly comparable open-source competitor is on the leaderboard.

Search does most of the lifting

The more interesting number in the paper is internal. Here’s what happens when you decompose the score:

Roughly two-thirds of the headline improvement comes from the search harness, not the weights. That’s not a knock — the operator-plus-search design is the contribution — but it means Frontis-MA1’s proximity to Kimi K3 (which needs a ~16×GB10 cluster to serve at speed ⁹) reflects search-time compute more than parameter-efficient capability. The same pattern shows up in transfer to NatureBench Lite: swapping in Evo lifts a fixed base model from 20% to 50%.

Contamination pressure and a measured safety frame

Coding benchmarks are under a contamination cloud right now. OpenAI recently withdrew support for SWE-bench Verified after finding over 60% of remaining tasks unsolvable or memorized, and independent audits put contamination worth 15–20 points on similar leaderboards ¹⁰. Frontis’s own limitations section admits “data leakage” and “degenerate targets” forced them to filter tasks out of OpenMLE-Gym — an honest note, but one that argues for independent re-runs before the frontier-beating claim propagates.

On the safety side, the Cloud Security Alliance frames Frontis-MA1 as the point where “AI materially participates in developing successor systems under human supervision” ¹¹ — a threshold, not a takeoff. That aligns with the Frontis team’s own survey framing of self-improvement as a “trace-to-capability” engineering problem ¹², and it’s the right register. The durable contributions here are the 5,758-task gym, the operator taxonomy, and the open weights. The 71% number needs a second pair of eyes.

---

BM25 beats agentic RAG past 10M tokens in Onyx study

Source: hf-daily-papers · published 2026-07-29

TL;DR

• BM25 overtakes file-system agents at ~10M tokens and finishes 20 points ahead (50.5 vs 30.7) on a 601M-token enterprise corpus.
• LightRAG’s construction wall: 102B tokens and ~4 years to index the 601M-token corpus.
• Agent+BM25 lifts full-scale accuracy from 36.9 to 69.4 on 1/9th the tokens — discovery, not reasoning, was the bottleneck.
• No hybrid+reranker baseline — the 2026 production default (BM25+dense+cross-encoder) is absent from Onyx AI’s own comparison.

The crossover

Onyx AI’s scaling study runs seven RAG pipelines across a 28-tier nested corpus that grows from 1.7M to 601M tokens, holding the gold documents and questions fixed while stratified noise scales up around them. The headline finding is a clean crossover: at the smallest tier, a file-system agent with list_dir/grep/read_doc tools beats BM25 (77.4 vs 74.7). Somewhere near 10M tokens the curves cross. At full scale the agent collapses to 30.7 while BM25 holds 50.5, with DenseRAG trailing at 29.9.

The mechanism is what the authors call the “lexical anchor” effect: enterprise questions carry product names, ticket IDs, and error strings that BM25 matches exactly, while dense retrieval and graph traversal drift toward “traps” — topically similar but factually wrong distractors that the benchmark deliberately plants alongside the gold documents.

The agent’s failure mode is also diagnosable. Swap its raw file-tree tools for a BM25 search tool (the paper’s “Agent+BM25” intervention) and full-scale accuracy jumps from 36.9 to 69.4 on roughly 1/9th the tokens. Independent write-ups on Claude Code-style agents describe the same pathology: tool storms of recursive list_dir and repeated grep calls that saturate context without converging ¹³. Agency is useful after a global ranker narrows the search space, not as a substitute for one.

Where the graph methods hit a wall

Graph RAG doesn’t lose on accuracy so much as on the meter running before you ask a question. The paper fits construction-cost curves and extrapolates: LightRAG grows super-linearly (b=1.36) and would need 102B tokens and ~4 years on one instance to index 601M tokens; MS-GraphRAG lands at 7.9B tokens and ~50 days. In practice LightRAG only built successfully out to 2,254 documents. BM25’s generative build cost is zero.

That extrapolation deserves a caveat the paper doesn’t offer. Independent reporting shows original GraphRAG cost ~$33K to index a 5GB corpus in early 2024, but LazyGraphRAG and smaller-model variants have since cut indexing cost by 99.9% ¹⁴. Quoting the linear extrapolation as a steady-state indictment is directionally right but temporally suspect.

The baseline that isn’t in the paper

The uncomfortable question: BM25 beats what, exactly? The paper compares four paradigms as pure archetypes. Production RAG in 2026 is neither — the practitioner default is RRF hybrid retrieval with a cross-encoder reranker, which independent write-ups credit with lifting accuracy from ~60% to ~85% over pure vector search ¹⁵. A recent methodological critique warns that studies omitting a strong dense+reranker first stage systematically overstate whatever the authors do include, because “a reranker cannot recover a relevant document that was never surfaced” ¹⁶.

Provenance matters here. The paper comes from Onyx AI (formerly Danswer), whose commercial product is enterprise workplace search, and the corpus is Onyx’s own EnterpriseRAG-Bench of ~511K synthetic Slack/Jira/GitHub/Confluence documents ¹⁷. Onyx has separately claimed the same stack beats ChatGPT Enterprise and Claude Enterprise on workplace tasks ¹⁸. The benchmark itself is a genuine contribution — public, reusable, adversarially designed — but reviewers on r/LocalLLaMA already flag that serious retrieval still runs multi-stage rerank pipelines the paper doesn’t evaluate ¹⁸.

Takeaway

Take the empirical result and leave the framing. Lexical anchors really do matter more at 600M tokens than the graph-and-agents hype cycle admits, and the Agent+BM25 result is a useful design pattern: use a global ranker for recall, an agent for follow-up reasoning. But “BM25 wins” is what Onyx wants you to conclude. The unanswered question is whether BM25 still wins against BM25+dense+cross-encoder at 600M tokens — and the paper doesn’t run that experiment.

Round-ups

Alibaba’s Qwen-UI-Agent targets cross-platform GUI plus CLI execution

Source: hf-daily-papers

Qwen-UI-Agent, topping Hugging Face’s daily list at 302 upvotes, aims for reliable operation on real devices across platforms, blending GUI interaction with CLI execution and long-horizon task completion. The technical report also outlines proactive service initiation and autonomous self-improvement as design goals.

Metis introduces native memory as a foundation model capability

Source: hf-daily-papers

Metis reframes agent memory as a first-class foundation-model capability rather than an external module bolted onto LLMs. The paper, the day’s top-voted on Hugging Face at 269 upvotes, formalizes memory foundation models alongside multimodal and reasoning models as a distinct axis of native ability.

Microsoft’s Echoverse builds deep, resettable environments for computer-use agents

Source: hf-daily-papers

Echoverse shifts the bottleneck in computer-use agent training from environment quantity to environment depth, arguing behavioural richness and interaction targeting drive returns. Microsoft’s pipeline generates login-gated, stateful synthetic apps that agents can break and reset, standing in for real production software.

Memory Decoder scales parametric long-term memory to 6.9B params

Source: hf-daily-papers

Memory Decoder at Scale pretrains a dedicated parametric memory module on 300B tokens, decoupling long-term recall from reasoning in decoder-only LLMs. The authors argue standard Faiss retrieval pipelines become infeasible at this data scale, motivating a learned memory of 6.9B parameters instead.

Chimera hybrid diffusion backbone gets a Chinchilla-style scaling recipe

Source: hf-daily-papers

Chimera processes text, image, and video tokens in a single raster-ordered stream without positional embeddings, combining Kimi Delta Attention for O(N) long-context tracking with interleaved Multi-head Latent Attention. The authors accompany the architecture with a Chinchilla-style scaling recipe for visual diffusion transformers.

MisKnow-Agent shows deep research agents adopt misleading sources as fact

Source: hf-daily-papers

Deep research agents propagate credible-looking but false web content into their final reports, a new benchmark called MisKnow-Agent finds. The framework injects misleading knowledge into open retrieval environments to test whether planning, synthesis, and report generation stages filter or amplify errors.

Explorative Modeling proposes a third pretraining axis for end-to-end generation

Source: hf-daily-papers

Explorative Modeling targets generative modeling’s holdout from the end-to-end training revolution, arguing that factored sampling procedures block joint optimization over multi-modal distributions. The XM framework introduces exploration as a third pretraining axis alongside data and compute, enabling single-stage generative training.

Footnotes

1. NHC 2025 Verification Report — https://www.nhc.noaa.gov/verification/pdfs/Verification_2025.pdf

   GDMI [WeatherNext] was on average 140 km closer to actual cyclone location than the leading physics-based ensembles, with 72-hour Atlantic track errors dropping below ~80 miles and 5-day Melissa track error near 100 nmi — roughly 50% below the five-year average.

   ↩

2. MarkTechPost — FGN technical deep dive — https://www.marktechpost.com/2025/11/17/google-deepminds-weathernext-2-uses-functional-generative-networks-for-8x-faster-probabilistic-weather-forecasts/

   FGN is a ~180 million parameter Graph Transformer with 24 layers operating at a 6-hour timestep, roughly 8x faster than the diffusion-based GenCast (57M params, 12-hour steps), and outperforms GenCast on 99.9% of variables and lead times.

   ↩ ↩² ↩³

3. ArticSledge — ECMWF hybrid response — https://www.articsledge.com/post/ai-weather-forecasting

   ECMWF operationalized its own AIFS alongside physics-based IFS and built AIFS-TC, a machine-learning system that ‘corrects’ intensity errors in standard AI outputs, achieving intensity predictions comparable to top operational guidance.

   ↩

4. PreventionWeb / Rice University study — https://www.preventionweb.net/news/ai-weather-models-show-promise-hurricane-forecasts-rice-study-finds-key-physical-limitations

   AI models frequently underestimate storm strength, particularly maximum wind speeds and minimum central pressures, producing overly smooth forecasts that lack the sharp detail required to represent a cyclone’s compact inner core.

   ↩

5. Physics World — extreme-weather critique — https://physicsworld.com/a/physics-based-models-still-beat-ai-for-predicting-extreme-weather-events/

   AI weather models excel at average skill but frequently underperform traditional numerical models like HRES when predicting record-breaking extremes, systematically erring toward normality on out-of-distribution ‘black swan’ events.

   ↩ ↩²

6. Cyc-Attack (arXiv 2508.15724, Michigan State) — https://arxiv.org/abs/2508.15724

   Altering less than 0.1% of global weather measurements — comparable to tampering with one out of a hundred active satellites — can effectively hide an approaching hurricane or fabricate a false landfall using gradient-based adversarial perturbations against DLWF models like GenCast and WeatherNext.

   ↩

7. Sally Liu, ‘Deep dive on OpenAI’s MLE-bench’ (Medium) — https://sallysliu.medium.com/deep-dive-on-openais-mle-bench-93f2aae10a8a

   a full metric reading costs approximately $48,000… independent analyses suggest a persistent generalization gap of 9-13% between validation and test sets

   ↩

8. MLEvolve project page (internscience.github.io) — https://internscience.github.io/MLEvolve/

   MLEvolve secured the top spot on the full 75-task MLE-bench set, achieving a 65.3% overall medal rate within a 12-hour runtime budget… 80.3% medal rate on low-complexity tasks

   ↩

9. daily.dev, ‘Kimi K3 matches GPT-5 on benchmarks, costs 4x less’ — https://daily.dev/posts/kimi-k3-matches-gpt-5-on-benchmarks-costs-4x-less-and-the-weights-are-free-g7buv7gcl

   Kimi K3 is a 2.8-trillion-parameter Mixture-of-Experts model that activates only 104B parameters per request… its 1.56TB compressed size means running the full model at useful speeds requires a 16x GB10 cluster

   ↩

10. NHIMG, ‘AI Coding Benchmarks and SWE-bench: Are Your Tests Predictive Enough?’ — https://nhimg.org/community/ai-beyond-identity/ai-coding-benchmarks-and-swe-bench-are-your-tests-predictive-enough/

    contamination from training data can inflate scores by as much as 15-20 points… OpenAI recently withdrew its support for the ‘SWE-bench Verified’ subset after discovering that over 60% of the remaining tasks were effectively unsolvable or that models were simply reproducing ‘gold patches’ seen during training

    ↩

11. Cloud Security Alliance, ‘AI Recursive Self-Improvement: Security Implications’ — https://labs.cloudsecurityalliance.org/research/ai-recursive-self-improvement-security-implications-v1-0-csa/

    Frontis-MA1 represents a ‘security-relevant threshold’ where AI materially participates in developing successor systems under human supervision

    ↩

12. FrontisAI, ‘Awesome Self-Improving Agents’ survey site — https://frontisai.github.io/Awesome-Self-Improving-Agents/

    making agents smarter after deployment is essentially a ‘trace-to-capability’ problem where interaction experience must be converted into durable system updates

    ↩

13. Daily Dose of DS on file-system agents — https://blog.dailydoseofds.com/p/build-your-own-100-local-ai-second

    Every tool call adds output to the context window… agents may enter a ‘control loop’ failure where they issue repetitive or cascading tool calls (e.g., recursive list_dir or repeated grep for the same symbol) without converging on an answer, exponentially increasing token costs and latency.

    ↩

14. FalkorDB blog on GraphRAG economics — https://www.falkordb.com/blog/graphrag-sdk-knowledge-graph/

    Early 2024 estimates for indexing a 5GB dataset reached as high as $33,000 due to exhaustive community summarization… by mid-2025, optimizations like LazyGraphRAG and smaller models reduced these costs by 99.9%.

    ↩

15. tianpan.co, ‘Hybrid Search in Production’ — https://tianpan.co/blog/2026-04-12-hybrid-search-production-bm25-dense-embeddings

    Adding BM25 keyword matching to a pure vector system can jump accuracy from roughly 60% to 85%… hybrid retrieval using Reciprocal Rank Fusion (RRF) to combine BM25 and dense signals is now the recommended default for 90% of production systems.

    ↩

16. arXiv 2508.11758 — ‘Beyond the Reranker’ critique — https://arxiv.org/html/2508.11758v1

    New RAG methods appear superior only because they are compared against ‘vanilla’ baselines like BM25 while omitting the industry-standard ‘Dense Retrieval + Cross-Attention’ pipeline… a reranker cannot recover a relevant document that was never surfaced in the initial retrieval step.

    ↩

17. GitHub: onyx-dot-app/EnterpriseRAG-Bench — https://github.com/onyx-dot-app/EnterpriseRAG-Bench

    EnterpriseRAG-Bench is a comprehensive benchmark comprising over 500,000 synthetic documents across nine source types, including Slack, Jira, GitHub, and Confluence… simulates a realistic company (‘Redwood Inference’) by introducing cross-document coherence.

    ↩

18. r/LocalLLaMA discussion of EnterpriseRAG-Bench — https://www.reddit.com/r/LocalLLaMA/comments/1t5c5qn/an_open_benchmark_for_testing_rag_on_realistic/

    Onyx’s internal system has reportedly outperformed commercial giants like ChatGPT Enterprise and Claude Enterprise on specific workplace tasks… some suggest that high-fidelity retrieval still requires expensive multi-stage pipelines involving rerankers and hybrid search.

    ↩ ↩²