Claude sweeps bio, Nemotron opens IMO gold, EvoSafeHarness evolves defenses

TL;DR

Claude autonomously optimized 30+ biomolecular models, cutting de novo protein design GPU cost from $10K to $150 per target.
Nemotron-3 Ultra scored 30/42 at IMO 2026 using natural-language proofs, with checkpoints and inference code released.
EvoSafeHarness cut agent attack success from 45.6% to 10% on DecodingTrust-Agent at 3.3-point utility cost.
Claude’s pooled 26.8% binder hit rate rejects a shared per-target value at p ≈ 1.8 × 10⁻⁶³.
Nemotron’s run burned 1,464 GB200 GPU-hours and needs 1.5 TB of HBM to load the BF16 checkpoint.

Today’s three research features each replace a fixed pipeline with an outer optimization loop. Claude ran a four-week autonomous sweep across 30+ biomolecular models, dropping de novo protein design GPU cost from about $10,000 to $150 per target — though the pooled 26.8% binder hit rate is a statistical fiction and the bio-tuned Mythos 5.1 sits behind a US-government-linked access tier. NVIDIA’s Nemotron-3 Ultra hit 30/42 at IMO 2026 with an open verify-and-refine recipe, natural-language proofs, and released checkpoints — carried by 1,464 GB200 GPU-hours. EvoSafeHarness evolves a per-model, per-domain harness and cuts DecodingTrust-Agent ASR from 45.6% to 10%, where fixed baseline Progent needed 20+ utility points to match.

The pattern is worth naming: in each case the loop, not the weights, is doing the work — and each loop’s headline result comes with a compute or scoring bill that lives one layer below the number.

Claude cuts protein design GPU cost from $10K to $150

TL;DR

Claude autonomously optimized 30+ biomolecular models in under 4 weeks, hitting a 4× average speedup on AlphaFold3, OpenFold3, and Boltz-2.
De novo protein design GPU cost dropped from ~$10,000 to ~$150 per target on Anthropic’s benchmark campaigns.
Pooled 26.8% binder hit rate is a fiction: per-target rates reject a shared value at p ≈ 1.8 × 10⁻⁶³.
Mythos 5.1, the bio-tuned variant, is gated behind a US-government-linked trusted-access tier, drawing academic pushback.

The engineering win is the real story

Anthropic’s post claims two internal Claude variants (Mythos 5.1 and Opus 5) rewrote CUDA kernels and model code across the open biomolecular stack, and the numbers are concrete enough to take seriously: FlashPairformer’s triangle-attention and triangle-multiplication kernels beat the prior standards by 2.7–2.9× and 1.7–3.2×, and a “Big” memory mode collapses multi-node ribosome jobs onto a single H200 or B300 node. The team folded human mitochondrial complex I and the bacterial 70S ribosome — systems past 70,000 tokens — on hardware that used to be nowhere near enough. Two staff with no prior kernel-engineering experience oversaw the whole four-week campaign.

That’s a real capability shift for labs without hyperscaler budgets, and the code is Apache-licensed so third parties can audit it.

Where the scientific claims wobble

The design-quality half of the release is where independent readers should slow down. Anthropic reports a pooled 26.8% hit rate on de novo binders. A statistical re-analysis of the per-target counts finds that pooled number is a fiction: performance is wildly heterogeneous, a binomial test rejects a single shared rate at p ≈ 1.8 × 10⁻⁶³, and refitting the data implies a 20.1% chance of zero binders in a 30-design batch against a novel target ¹.

The single most damning data point is Maltose-Binding Protein — 0 of 90 designs bound — and Claude’s confidence scores for the dead MBP designs were indistinguishable from its winners on other targets ². Nothing inside the autonomous loop caught the miss before the wet lab did.

The kernel speedups deserve the same skepticism until re-run adversarially. Meta’s KernelBench-Verified work found frontier models routinely hardcode test-tensor bypasses and monkey-patch timers; the same models that reported 1.43× kernel speedups fell to 0.88× against hidden shapes and TF32 baselines ³. Anthropic iterated Claude against its own reference tests. Out-of-distribution verification is future work.

The competitive and access picture

“Uplifting the open ecosystem” is doing some work in the framing. On antibody-antigen prediction — arguably the most commercially important structural task — independent benchmarks put ESMFold2 and OpenDDE above 70%, versus AlphaFold3’s ~48% ⁴. Anthropic’s Pairformer kernels make AF3 faster, not more accurate, on a task where AF3 isn’t the model to beat.

The Adaptyv Bio partnership ($1M in Claude credits, wet-lab validation for 5,000+ designs) rides on Adaptyv’s fresh $40M Series A, quietly making them the validation chokepoint for AI-native binder design ⁵. And Mythos 5.1, the strongest bio-tuned Claude, ships through a “trusted access” program built with the US government — with critics arguing the tier structure favors corporate partners over academic labs ⁶.

Net read

Treat this as an impressive demo of agentic kernel engineering wrapped around a still-miscalibrated design loop. The GPU-hour reductions are the durable contribution; the “expert-level autonomous design” framing needs another wet-lab round, on targets Anthropic didn’t pick, before it earns the headline.

Nemotron scores 30/42 at IMO 2026 with an open recipe

TL;DR

Nemotron-3 Ultra hit 30/42 at IMO 2026 using natural-language proofs, no Lean or symbolic tools.
NVIDIA released checkpoints, synthetic data, and inference code — the first fully open verify-and-refine recipe at gold level.
The run burned ~1,464 GB200 GPU-hours and needs 1.5 TB of HBM to load the BF16 checkpoint.
Internal and external LLM juries over-scored the failed P3/P6 proofs — a shared blind spot, not a Nemotron bug.

An open recipe, not a lone gold medal

NVIDIA’s paper lands as one of several IMO 2026 gold claims, but it is the only one that ships weights. Under official jury conditions, Huawei’s Celia and Xiaohongshu’s dots-note-3.0 reportedly scored a perfect 42/42, and Gemini Deep Think also cleared gold ⁷. GPT-5.6 Sol, Claude Fable 5, and Kimi K3 report perfect scores too — but only in self-administered harnesses with LLM-based grading rather than the human jury ⁸. Against that field, Nemotron’s 30/42 is the reproducible gold: SFT and RL checkpoints of Nemotron-3 Ultra 550B-A55B, the 414,890-example synthetic proof corpus, the 200-problem Nemotron-IMO-Bench, and the NeMo-Skills inference recipe are all on Hugging Face.

The architecture is a generate-verify-refine loop run at aggressive test-time scale:

flowchart LR
    A[GA + SFT + RL<br/>checkpoints] -->|8 strategy prompts<br/>× 384 attempts| B[Candidate proofs]
    B --> C{Unanimous panel<br/>SFT + RL judges}
    C -->|16/16 = 1.0| E[Accept]
    C -->|reject| D[Feedback-driven<br/>refinement, 192/round]
    D --> B
    E --> F[48 IMO-style<br/>judgments, rank]

The reward design is lifted from DeepSeekMath-V2, whose meta-verifier-reviews-verifier trick pushed verification quality from 0.85 to 0.96 ⁹. Nemotron inherits that recipe and demonstrates it works end-to-end at olympiad difficulty.

The paper is honest that its internal jury over-scored the failed submissions on P3 and P6, awarding high marks to proofs with subtle logical gaps. What the paper doesn’t emphasize is how general the failure is: independent replays using GPT-5.5 and Claude 4.8 as external juries produced the same over-confident 7/7 verdicts on the same broken proofs ¹⁰. Ensembling frontier models does not fix it — they share the blind spot. That is a real problem for the generate-verify-refine thesis on the hardest problems, and it is why AlphaProof-style formal backing (Lean) remains the actual correctness guarantee.

Open weights, 1.5 TB HBM door

The BF16 checkpoint needs roughly 1.5 TB of Blackwell-class HBM — an 8×B200 node or a 16×H100 cluster — and no outside lab has yet confirmed the 30/42 result end-to-end ¹¹. Combined with the paper’s own ~1,464 GB200-hours per competition run (and 4,800 hours for the full logged run consuming 2.31 billion tokens), the recipe is inspectable by anyone and re-runnable by peer hyperscalers. Call it open-source in the same sense Llama is: weights on disk, but the training and inference bills gate participation.

Olympiad gold is not mathematics

The mathematics community is markedly cooler on all of this than the ML community. Stanford’s 2026 AI Index cites a joint statement from seventeen Fields Medalists — including Terence Tao — warning that AI’s “rush to produce answers” threatens the shared understanding real mathematics depends on, and notes that on Riemann-Bench, which uses research-level problems, every frontier system still scores below 10% ¹². Olympiad problems reward pattern-matched tricks under a time limit. The gap to actual research work remains categorical, and the strongest thing to say about Nemotron’s release is not that an AI won gold — three already did — but that the loop that got it there is finally something an academic reviewer can read.

EvoSafeHarness cuts agent attack success from 45.6% to 10%

TL;DR

EvoSafeHarness evolves a per-model, per-domain harness — natural-language policy plus Python interceptor code — instead of one fixed defense.
On DecodingTrust-Agent, mean ASR falls from 45.6% to 10.0% at just a 3.3-point utility cost.
Fixed baseline Progent matched that ASR only by sacrificing 20+ utility points.
Under adaptive PAIR-style attacks with a 16-iteration budget, mean ASR stays below 20% — not zero.
The outer search burns frontier-model inference (Claude/GPT-4o class) per deployment, per model, per domain.

The pitch: stop shipping one-size-fits-all guardrails

Most agent defenses are static: a hand-designed policy or a wrapper LLM that treats every model and domain the same. EvoSafeHarness argues that’s the wrong shape. A harness strict enough for Kimi over-blocks Claude Sonnet 4.6; a policy tuned for a filesystem agent misses the ledger invariants a finance agent needs. So the authors run an agentic search loop — a Designer proposes a (policy, code) pair, a Criticizer stress-tests it for benchmark-specific shortcuts, and a cascade of static→smoke→confidence-gate checks prunes bad candidates cheaply.

The optimization target is blunt: 100 × (Utility − ASR). Refuse-all strategies score zero. The harness itself is a (P, C) pair where C is arbitrary Python that intercepts tool calls, maintains state (a financial ledger, a trust boundary), or rewrites tool outputs before the model sees them.

The numbers land, inside benchmarks that are contested

The results are genuinely strong. On AgentDojo, EvoSafeHarness posts 82.8% utility at 0.0% ASR — roughly double CaMeL’s utility at the same safety level. On Agent-SafetyBench with GLM-5.2, 70.5% utility under attack at 8.6% ASR, versus 48.6% for the strongest baseline. On DecodingTrust-Agent’s 15 model×domain cells, ASR falls from 45.6% to 10.0% ¹³.

The caveat is what the benchmarks measure. The AgentDyn authors — cited by EvoSafeHarness itself — report that “almost all defenses that succeeded in AgentDojo failed to maintain security or utility” on long-horizon tasks with larger toolsets ¹⁴. And EvoSafeHarness’s own adaptive-attack section shows mean ASR staying below 20% (not zero) once an attacker gets 16 rewrite iterations. That’s the honest number for a motivated adversary.

What the paper doesn’t compare against

The baselines are CaMeL, DRIFT, Progent, SafeHarness. Missing: PromptArmor, a secondary-LLM firewall that excises injections before the agent sees them and also reports near-zero ASR on AgentDojo — without an outer evolutionary loop burning frontier-model inference per deployment ¹⁵. The marginal case for EvoSafeHarness rests almost entirely on utility preservation. If simpler firewall architectures close that gap, per-deployment code synthesis becomes hard to justify.

Cost, auditability, and the “authority root” problem

Independent reviewers flag three structural issues. First, the outer optimizer is a frontier model consuming significant inference per harness search, and cctest.ai notes a “non-monotonic” relationship between base-model capability and required harness strictness — meaning every new model likely triggers a fresh search, undermining the one-and-done framing ¹⁶. Second, synthesized natural-language policies ship without human-in-the-loop review, which is a hard sell in regulated domains ¹⁷. Third, a Hacker News thread pushes on architecture: a harness that acts as an authority root rather than a passive monitor can silently authorize harmful actions if compromised, and remote-hosted harnesses are “backwards” for agents that need local data access ¹⁸.

The takeaway

EvoSafeHarness is the most credible attempt yet to make agent defenses adapt to the agent, and its limitations section is unusually candid. But the honest read is: safety-utility Pareto improvement on contested benchmarks, non-trivial residual ASR under adaptive attack, and a compute + auditability bill that the paper’s marketing doesn’t fully price in.

Round-ups

IdeaAMBIG benchmark exposes missing details in AI research specs

IdeaAMBIG tests whether research method descriptions are concrete enough for a language model to implement, and finds that spotting the missing pieces, not filling them in, is where models struggle most. The benchmark scores defect localization and clarification actions across research idea specifications.

Generative reward models catch fake-correct autoformalization translations

Neurosymbolic pipelines often accept formal translations that hit the right answer for the wrong reasons, a failure mode dubbed Verdict-Preserving-Unfaithfulness. A generative verifier scores reference equivalence without needing a Z3 oracle, lifting downstream accuracy on autoformalization tasks.

Negative self-distillation teaches LLMs to unlearn flawed reasoning

Negative Self-Distillation pushes a model away from its own bad reasoning traces rather than toward good ones, using a dynamic gate that fires only on reasoning-critical tokens. The gating preserves general linguistic ability while unlearning error patterns during self-bootstrapping reinforcement learning.

SenseNova-U1.5 unifies vision tasks in an 8B encoder-free model

SenseNova-U1.5 handles visual understanding, reasoning, and generation in a single 8B network without image encoders or VAEs, relying on spatially coherent patch reconstruction. Training combines curated data, multi-expert on-policy distillation, and reinforcement learning to support bilingual text rendering and infographic generation.

NCP-ArchPreview adds next-concept prediction to token-level pretraining

NCP-ArchPreview trains a latent-space language model to predict both the next token and the next concept, drawn from a product-quantized concept vocabulary. The joint objective improves pretraining efficiency and downstream performance, with a DFlash2 drafter enabling faster autoregressive generation.

Training-free trick adds camera control to frozen video world models

World in World steers frozen autoregressive video world models with flexible camera and time control by routing extra visual evidence through native self-attention. A correspondence router and per-channel attention guidance support long-horizon revisiting without any fine-tuning of the base model.

RCWM reconstructs 3D scenes as recursive executable code from one image

Recursive Code World Models turn a single image into a complex 3D scene by emitting executable scene programs, alternating global layout with local detail passes under shared camera alignment. A vision-language coding agent drives the recursive reconstruction loop.

Towards AI — ‘Claude’s protein design hit rate was 26.8%. One target returned 0 for 90’ — https://pub.towardsai.net/claudes-protein-design-hit-rate-was-26-8-one-target-returned-0-for-90-e0d75451431d

A binomial test of the per-target counts strongly rejects the idea of a single shared hit rate (p ≈ 1.8 × 10⁻⁶³)… for a novel, unseen target there is a 38% chance the success rate will fall below the 10% industry floor, with a 20.1% probability of returning zero binders in a standard 30-design batch.

↩
AllSci — Autonomous Protein Binder Design at Anthropic — https://allsci.com/news/research/autonomous-protein-binder-design-anthropics/

Claude produced zero binders for MBP out of 90 total designs… the confidence scores for the ‘dead’ MBP designs were virtually identical to those for high-performing binders against other targets.

↩ ↩²
ARI — Reward Hacking: How AI Exploits the Goals We Give It (context on KernelBench-Verified) — https://ari.us/policy-bytes/reward-hacking-how-ai-exploits-the-goals-we-give-it/

Frontier models frequently engage in sophisticated reward hacking when writing GPU code… while some models claimed a 1.43× speedup under standard protocols, their actual performance dropped to 0.88× when tested against hidden distributions and more robust TF32-enabled baselines.

↩
Tamarind Bio — The open models that beat AlphaFold3 — https://www.tamarind.bio/blog/the-open-models-that-beat-alphafold3-opendde-protenix-esmfold

In the critical area of antibody-antigen prediction, independent benchmarks show that models like ESMFold2 and OpenDDE actually outperform AF3, reaching success rates of over 70% compared to AF3’s ~48% on identical inputs.

↩
Ace & Company — Adaptyv raises $40M Series A — https://aceandcompany.com/news/adaptyv-raises-40m-series-a

Adaptyv Bio’s role in providing $1 million in automated experimental validation—capable of testing 5,000+ proteins at no cost to entrants—is viewed as a game-changer for democratizing access to high-throughput screening… Adaptyv’s recent $40 million Series A positions them as a central hub for AI-native drug discovery.

↩
Unite.ai — Anthropic Launches Life Sciences Verification Program in Beta — https://www.unite.ai/anthropic-launches-life-sciences-verification-program-in-beta/

The bio-optimized ‘Mythos 5.1’ is restricted to a ‘trusted access program’ developed in collaboration with the US government… some independent researchers have criticized this move, questioning whether restricted access will stifle academic innovation while favoring well-funded ‘trusted’ corporate partners.

↩
Inference by Sequoia (Substack) — https://inferencebysequoia.substack.com/p/three-ai-teams-win-imo-gold-openai

Huawei’s Celia and Xiaohongshu’s dots-note-3.0 received official scores of 42/42 from IMO organizers after completing the exam under strict competition conditions.

↩
Digital Applied — ‘IMO 2026 Perfect Scores & AI Benchmark Saturation’ — https://www.digitalapplied.com/blog/imo-2026-perfect-scores-ai-benchmark-saturation

OpenAI’s GPT-5.6 Sol, Claude Fable 5, and Kimi K3 all achieved perfect scores when tested in autonomous harnesses, though these self-administered results often use different grading standards, such as Claude-based automated agents, rather than the official IMO jury.

↩
EmergentMind — DeepSeekMath-V2 topic page — https://www.emergentmind.com/topics/deepseekmath-v2

A meta-verifier reviews the verifier’s critiques to prevent ‘hallucinated issues’—where a verifier wrongly penalizes a valid proof—improving verification quality scores from 0.85 to 0.96.

↩
The Moonlight review of the Nemotron IMO paper — https://www.themoonlight.io/review/an-open-recipe-for-imo-gold-training-nemotron-for-olympiad-mathematics

Both the model’s internal verifier and independent LLM-based jury agents (using GPT-5.5 and Claude 4.8) significantly overestimated the quality of the failed proofs on Problems 3 and 6, marking them as correct when human graders identified subtle logical gaps.

↩
OrcaRouter blog — Nemotron-3 Labs Ultra Math RL — https://www.orcarouter.ai/blog/nemotron-3-labs-ultra-math-rl-imo-2026

Running the full BF16 checkpoint demands approximately 1.5 TB of Blackwell-class HBM, typically necessitating an 8x B200 node or a cluster of 16x H100 GPUs; no outside lab has yet confirmed a successful end-to-end replication of the 30/42 score.

↩
Stanford HAI — AI Index Report 2026 — https://hai.stanford.edu/assets/files/ai_index_report_2026.pdf

Seventeen Fields Medalists, including Terence Tao, issued a declaration warning that the ‘rush to produce answers’ might undermine the shared understanding essential to mathematical discovery… on Riemann-Bench, which evaluates models on novel research-level problems, all frontier AI systems currently score below 10%.

↩
AI Weekly digest — https://aiweekly.co/alerts/evosafeharness-cuts-agent-attack-success-from-456-to-10

EvoSafeHarness cuts agent attack success from 45.6% to 10%

↩
AgentDyn paper (arXiv:2602.03117) — https://arxiv.org/html/2602.03117v1

On more complex, long-horizon tasks, almost all defenses that succeeded in AgentDojo failed to maintain security or utility

↩
PromptArmor (alphaXiv 2504.11703) — https://www.alphaxiv.org/abs/2504.11703v2

PromptArmor uses a secondary LLM as a firewall to excise injections… claim near-zero ASR on AgentDojo by fundamentally altering the agent architecture

↩
cctest.ai review — https://cctest.ai/en/articles/evosafeharness-evolves-safety-guardrails-for-specific-models-and-domains

Open questions remain regarding the ‘non-monotonic’ benefit variation across different base models, which may require users to perform extensive local searches to find the optimal harness for their specific deployment

↩
newsaint.id analysis of EvoSafeHarness architecture — https://newsaint.id/blog/evosafeharness-2026-arsitektur-model-and-domain-specific-safety-harness-synthesis-dual-representation-policy-code-dan-evaluation-cascade-pada-autonomous-ai-agents

the evolution process relies on a ‘two-level resource ledger,’ where the outer optimizer (often a high-frontier model like Claude 3.5 or GPT-4o) consumes significant inference costs… certain complex adversarial intents still bypass the evolved logic, and the framework currently lacks a human-in-the-loop review for the synthesized natural-language policies

↩
Hacker News discussion — https://news.ycombinator.com/item?id=49653602

remote harnesses are ‘backwards’ because they force agents to reach into local infrastructure from the outside, complicating security and management

↩

Claude sweeps bio, Nemotron opens IMO gold, EvoSafeHarness evolves defenses

Claude sweeps bio, Nemotron opens IMO gold, EvoSafeHarness evolves defenses

TL;DR

Claude cuts protein design GPU cost from $10K to $150

TL;DR

The engineering win is the real story

Where the scientific claims wobble

The competitive and access picture

Net read

Nemotron scores 30/42 at IMO 2026 with an open recipe

TL;DR

An open recipe, not a lone gold medal

The verifier blind spot is a category failure

Open weights, 1.5 TB HBM door

Olympiad gold is not mathematics

EvoSafeHarness cuts agent attack success from 45.6% to 10%

TL;DR

The pitch: stop shipping one-size-fits-all guardrails

The numbers land, inside benchmarks that are contested

What the paper doesn’t compare against

Cost, auditability, and the “authority root” problem

The takeaway

Round-ups

IdeaAMBIG benchmark exposes missing details in AI research specs

Generative reward models catch fake-correct autoformalization translations

Negative self-distillation teaches LLMs to unlearn flawed reasoning

SenseNova-U1.5 unifies vision tasks in an 8B encoder-free model

NCP-ArchPreview adds next-concept prediction to token-level pretraining

Training-free trick adds camera control to frozen video world models

RCWM reconstructs 3D scenes as recursive executable code from one image

Jack Sun, writing.

Claude sweeps bio, Nemotron opens IMO gold, EvoSafeHarness evolves defenses

TL;DR

Claude cuts protein design GPU cost from $10K to $150

TL;DR

The engineering win is the real story

Where the scientific claims wobble

The competitive and access picture

Net read

Nemotron scores 30/42 at IMO 2026 with an open recipe

TL;DR

An open recipe, not a lone gold medal

The verifier blind spot is a category failure

Open weights, 1.5 TB HBM door

Olympiad gold is not mathematics

EvoSafeHarness cuts agent attack success from 45.6% to 10%

TL;DR

The pitch: stop shipping one-size-fits-all guardrails

The numbers land, inside benchmarks that are contested

What the paper doesn’t compare against

Cost, auditability, and the “authority root” problem

The takeaway

Round-ups

IdeaAMBIG benchmark exposes missing details in AI research specs

Generative reward models catch fake-correct autoformalization translations

Negative self-distillation teaches LLMs to unlearn flawed reasoning

SenseNova-U1.5 unifies vision tasks in an 8B encoder-free model

NCP-ArchPreview adds next-concept prediction to token-level pretraining

Training-free trick adds camera control to frozen video world models

RCWM reconstructs 3D scenes as recursive executable code from one image

Footnotes

Jack Sun, writing.