Opus 4.7 runs robodog solo, o3 cracks 18 cold cases, PA-DR cuts leak to 9.9%

TL;DR

• Opus 4.7 finishes Anthropic’s robodog course autonomously in 12 min, vs 264 min for humans on Opus 4.1.
• OpenAI’s o3 confirms diagnoses in 18 of 376 previously unsolved pediatric cases at Boston Children’s.
• ServiceNow’s PA-DR cuts research-agent query leakage from 34% to 9.9% via per-step rewards.
• Exomiser still beats LLMs by 12 points (35.5% vs 23.6%) on top-1 phenotype-driven diagnosis.
• Closed-loop actuation still fails — Claude navigates to a beach ball but can’t nudge it to target.

Three AI-research leads today share a methodological move more than a topic: each measures an LLM against a live deployment baseline rather than a benchmark leaderboard. Anthropic puts Opus 4.7 on a physical robodog course built for human engineering teams, OpenAI’s o3 is run against Boston Children’s queue of previously unsolved pediatric genetic cases, and ServiceNow’s PA-DR is measured by how much private context its research agent leaks into outbound search queries — the same side-channel pattern behind a recent Copilot exploit and a 195M-record Mexican taxpayer breach.

The wins are real and the baselines are real, which makes the residual gaps load-bearing too. Claude beats human teams 20× on the robodog but still can’t close an actuation loop. o3 confirms 18 of 376 cold cases yet Exomiser, a non-LLM tool, still leads on top-1 phenotype-driven diagnosis. PA-DR’s main finding is that prompting an agent not to leak doesn’t work — privacy has to be trained in. The robodog demo also lands the week a US government order forced Anthropic to suspend its top-tier models, sharpening the autonomy claim’s regulatory edge.

Claude Opus 4.7 beats human teams 20× on Anthropic’s robodog

Source: anthropic-research · published 2026-06-18

TL;DR

• Opus 4.7 finished the robodog course autonomously in 12 min 7 sec, vs 264 minutes for humans with Opus 4.1.
• The autonomous run used 1,045 lines of code to the human team’s 10,309 — nearly 10× less for the same objectives.
• Claude still failed closed-loop actuation — it could navigate to a beach ball but not nudge it to a target.
• Demo lands days after a US government order forced Anthropic to suspend its top-tier models, sharpening the regulatory subtext.

From assistant to operator in ten months

In August 2025, Opus 4.1 couldn’t drive the robodog on its own; the best run was a 264-minute slog with a human team in the loop. Phase Two swapped the humans out. Opus 4.7, running through Claude Code with adaptive thinking maxed, completed the same four core tasks in 12 minutes 7 seconds — a researcher only typed the initial prompt and rubber-stamped command approvals. Glen Rhodes’ independent recap confirms both the wall-clock and the code-size numbers ¹, and the 37× speedup against the fully unassisted human team made it into the wire coverage ²³.

The pattern Anthropic flags — model assists human → human assists model → model operates alone — is the same arc they pointed to in offensive-security evals last year. The novelty here is that the actuators are physical: LIDAR pipes, video feeds, motor controllers. Most of Claude’s generated code reportedly worked on the first try, and variance across three trials was low, which matters more than the raw 20× figure. A flashy single run is a demo; tight variance is a capability claim.

Where the loop still breaks

The one task Claude couldn’t finish is the one the project is named for. The robodog could locate a beach ball and circle around it, but couldn’t execute the continuous, sub-second feedback needed to actually push it to a marker. This isn’t a quirky edge case — it’s the predictable ceiling for a text-conditioned policy driving continuous actuators. A recent industry survey on simulation training pegs the field at roughly 0.05% of the interaction data needed for true generalization, and notes that models routinely “hallucinate” physical possibilities like motor torques that exceed hardware limits ⁴. Writing a Python script to subscribe to a LIDAR topic is in-distribution for an LLM. Holding a tight feedback loop on a soft, rolling object is not.

The regulatory subtext

Phase Two also dropped days after a US government order forced Anthropic to disable its most advanced models worldwide ⁵. That makes “our model autonomously drives robot hardware 20× faster than humans” either a spectacularly mistimed flex or a deliberately timed one — depending on whether you read Anthropic as cornered or as actively reinforcing a frontier models are dangerous, regulate them narrative. Outside commentary has already picked a side: a Medium write-up framed the robodog as the opening shot of “a new AI race” rather than a benchmark result ⁶.

The capability datapoint survives scrutiny. The clean “scaling solves robotics” story doesn’t — not while the fetching half of Project Fetch is still the part Claude can’t do, and not while Anthropic’s headline models are under a federal stop-order.

---

OpenAI o3 solves 18 of 376 cold pediatric genetic cases

Source: openai-blog · published 2026-06-18

TL;DR

• OpenAI’s o3 confirmed diagnoses in 18 of 376 previously unsolved pediatric cases at Boston Children’s.
• That’s a 4.8% yield on a cohort that had already exhausted standard specialist review.
• Exomiser still beats LLMs on top-1 phenotype-driven diagnosis by ~12 points (35.5% vs 23.6%) across 5,000+ cases.
• The study was retrospective and unblinded, with no clinician-time or false-positive metrics reported.

The headline number, in context

Researchers from Boston Children’s Hospital, Harvard, and OpenAI ran o3 Deep Research over 376 cold pediatric cases and got 18 CLIA-confirmed diagnoses back. A 4.8% incremental yield on cases that already failed standard specialist review is genuinely useful — for the families involved, definitive. Kyra Benton’s 19-year diagnostic odyssey ended with o3 flagging a frameshift variant in HSPB8, confirmed as myofibrillar myopathy a week before her 28th birthday ⁷.

The yield was uneven across cohorts: 10% on neurodevelopmental cases and 13.3% on early psychosis, but only 1% on sudden unexpected death. The model also did things harder than database lookup — inferring a 22q11.2 deletion from low-quality signals, naming digenic TTN/SRPK3 interactions, and proposing a novel S1PR1–vitiligo mechanism by synthesizing structural biology with immunology.

Why this isn’t a diagnostic engine

The setup matters. o3 ingested HPO terms, clinician notes, and a pre-filtered variant table (rarity, predicted protein effect, ClinVar class). It then proposed molecular explanations that two human experts adjudicated under ACMG/AMP before any lab confirmation. This is a reasoning layer over a conventional genomics pipeline, not a replacement for one.

That distinction matters because, on raw variant ranking, the specialized tools still win. A 5,000-case medRxiv comparison put Exomiser’s top-1 accuracy at 35.5% versus 23.6% for the best LLM variants tested ⁸. OpenAI’s own forthcoming life-sciences model, GPT-Rosalind, passed only 36.1% of expert-authored tasks on the 2026 LifeSciBench — best in class, but failing two of three ⁹. The BCH win comes from synthesis across phenotype, literature, and inheritance patterns, not from beating Exomiser at its own game.

What the press release skips

Two methodology problems deserve flagging. First, the study is retrospective and reviewers were unblinded to model confidence scores — a setup ripe for priming effects, especially on novel-mechanism calls like the S1PR1 hypothesis. Second, much of the literature describing these “unsolved” conditions was almost certainly in o3’s pretraining corpus, which complicates any claim of generalization.

Comparable prospective deployments show what’s missing. Google’s AMIE matched physicians on differential diagnosis quality in a Beth Israel Deaconess trial but lost on management practicality and cost-effectiveness ¹⁰ — precisely the dimensions BCH did not measure. No clinician-time-saved, false-positive-burden, or downstream-cost numbers appear in the paper.

The institutional play

BCH frames this as one node in a broader rollout — the hospital reports ~60,000 reclaimed labor hours and $7M redeployed across 50+ AI automations ¹¹. And the follow-on, OpenAI’s GPT-Rosalind, is launching through a gated “trusted access program” with Lawrence Livermore and Johns Hopkins APL as early partners ¹². That governance model is defensible for biodefense work, but it means the low-resource clinics most likely to benefit from automated reanalysis will be last in line.

The result deserves the coverage it’s getting. It also deserves the prospective, blinded follow-up trial that would turn proof-of-concept into evidence.

---

ServiceNow’s PA-DR cuts agent query leakage from 34% to 9.9%

Source: huggingface-blog · published 2026-06-18

TL;DR

• ServiceNow’s PA-DR cuts research-agent query leakage from 34% to 9.9% while holding task success near the task-only RL ceiling.
• Task-only RL spiked leakage to 51.7% as agents learned to pack queries with private context to retrieve better.
• Prompting “don’t leak” doesn’t work — privacy must be trained in via per-step rewards, not bolted on at inference.
• Copilot’s SearchLeak and a 195M-record Mexican taxpayer breach exploit the same queries-as-side-channel pattern in production.

The bug: better agents are leakier agents

MosaicLeaks names a failure mode that anyone who has watched a deep-research agent type into Google has probably noticed: to retrieve well, the agent stuffs its web queries with context from private documents. No single query spills the secret, but the cumulative query log lets an observer reconstruct it. ServiceNow built a 1,001-chain benchmark to measure this, and the headline finding is uncomfortable — pure task-reward RL took Qwen3-4B from 48.7% to 59.3% strict-chain success, but answer/full-info leakage jumped from 34.0% to 51.7%. Capability and privacy move in opposite directions when you only optimize one.

The threat model is no longer academic. Microsoft 365 Copilot’s SearchLeak disclosure earlier this cycle showed one-click exfiltration of MFA codes and password-reset links via server-side image fetches inside the search surface — exactly the query-as-channel pattern MosaicLeaks formalizes ¹³. A separate incident report describes attackers driving Claude Code and GPT-4.1 to automate 5,000+ remote commands and lift 195M taxpayer records from Mexican government systems, roughly 10× faster than manual intrusion ¹⁴.

PA-DR: dense rewards on individual calls

The fix ServiceNow proposes, Privacy-Aware Deep Research (PA-DR), replaces trajectory-level rewards with two situational signals. A task reward scores each Plan/Choose/Read call on immediate correctness — the authors report this alone is 5-6× more sample-efficient than outcome-based RL. A learned privacy reward uses a Qwen3-4B classifier to penalize query text that leaks directly or contributes to mosaic reconstruction. The combination lands at 58.7% success / 9.9% leakage — essentially keeping the task-only RL gains while erasing the privacy regression.

Worth noting: simply prompting the base agent “do not leak” reduced leakage and success, and still failed to stop significant flow. This is the load-bearing claim of the paper. Inference-time instructions are not a substitute for shaping the policy.

The catch: scrubbing tokens isn’t scrubbing channels

Two threads of independent work complicate the victory lap.

First, PA-DR is not the only contextual-integrity defense in this cycle. PrivacyChecker, a runtime wrapper grounded in Nissenbaum’s CI framework, intercepts and rewrites queries with no fine-tuning and reports comparable deltas — 36% → 7% leakage on DeepSeek-R1 ¹⁵. For teams that can’t retrain their agent stack, that’s a meaningful alternative.

Second, query-text scrubbing leaves the network layer wide open. Whisper Leak recovers conversation topics from encrypted streaming traffic via packet size and timing patterns with >90% precision ¹⁶. A PA-DR-trained agent can strip every revealing token from its query and still leak through bandwidth and latency fingerprints — the classifier never sees those.

Caveats and what’s next

The benchmark is small and narrow. An independent write-up notes the 1,001 chains required substantial human effort and cover only a handful of company contexts drawn from DRBench ¹⁷; generalization to health and finance is untested. And practitioner skepticism toward Qwen3-class agent numbers is sharp — community reports describe “death spirals of bad tool calls” in real enterprise workflows ¹⁸, suggesting the 58.7% success figure may not survive contact with messier corpora.

The 9.9% number is still the most concrete privacy-vs-utility datapoint published in this space. It’s a floor for what training-time mitigation can achieve, and a reminder that anyone deploying a deep-research agent against private data without one of these mechanisms is leaking by default.

Round-ups

LLM agents are least safe on their very first tool call

Source: hf-daily-papers

Tool-calling LLM agents show a measurable cold-start safety gap: refusal and guardrail behavior strengthens only after a few prior task completions. Hidden-state analysis traces the shift to representation drift, and the authors propose Safety Over Depth as a deployment-time mitigation.

TRACE compiles user corrections into runtime rules for coding agents

Source: hf-daily-papers

TRACE mines corrections that users issue to coding agents and compiles them into runtime checks enforced on later actions. The skill-layer pipeline sharply cuts preference violations on both in-distribution and out-of-distribution tasks without hurting overall task pass rates.

WebChallenger pairs PageMem with divide-and-conquer browsing for open-weight agents

Source: hf-daily-papers

WebChallenger is a generalist web agent that represents pages through a structured PageMem abstraction over the DOM and runs a divide-and-conquer observation pipeline with compound action workflows. The framework reaches strong autonomous navigation scores using open-weight base models rather than frontier APIs.

LabVLA grounds vision-language-action models in lab automation

Source: hf-daily-papers

LabVLA targets scientific laboratory workflows with a two-stage recipe: FAST action-token pretraining on Qwen3-VL-4B-Instruct, then a flow-matching DiT action expert with knowledge insulation. Trained on simulator-generated trajectories, it tops prior VLAs on the LabUtopia benchmark of automated lab tasks.

WEAVER world model boosts robotic manipulation with flow-matching loss

Source: hf-daily-papers

WEAVER is a multi-view latent world model for robotic manipulation that swaps standard objectives for a flow-matching loss to keep long-horizon dynamics consistent. The architecture supports policy evaluation, policy improvement, and test-time planning, outperforming prior world models across each of these uses.

MaxProof uses tournament search to crack competition math proofs

Source: hf-daily-papers

MaxProof scales test-time compute for mathematical proof generation by combining a generative verifier, critique-conditioned proof repair, and population-level tournament selection. Built on the MiniMax-M3 series, the framework reaches competitive scores on high-level math olympiad problems where single-shot proof attempts typically collapse.

MiniMax Sparse Attention speeds ultra-long-context LLMs via blockwise sparsity

Source: hf-daily-papers

MiniMax Sparse Attention pairs blockwise sparse patterns with Grouped Query Attention and Top-k selection to keep tensor cores busy during both prefill and decoding. The design cuts attention compute at long context lengths while matching dense-attention quality, and the team has released the MSA implementation on GitHub.

Footnotes

1. Glen Rhodes blog (independent recap) — https://glenrhodes.com/anthropic-project-fetch-phase-2-claude-opus-4-7-solves-robodog-challenge-20x-faster-than-best-human-team/

   humans aided by an earlier model (Opus 4.1) took 264 minutes, the unassisted Opus 4.7 finished in 12 minutes and 7 seconds, utilizing only 1,045 lines of code compared to the human team’s 10,309

   ↩

2. KuCoin News flash — https://www.kucoin.com/news/flash/anthropic-s-claude-4-7-completes-robotic-tasks-37-times-faster-than-human-teams

   Claude 4.7 Completes Robotic Tasks 37 Times Faster Than Human Teams

   ↩

3. Benzinga — https://www.benzinga.com/markets/private-markets/26/06/53291817/anthropic-fetch-claude-operates-robots-faster-than-humans

   Anthropic Fetch: Claude Operates Robots Faster Than Humans

   ↩

4. PR Newswire / industry report on robot simulation training — https://www.prnewswire.com/apac/news-releases/the-final-mile-of-robot-simulation-training-why-scene-assets-are-becoming-the-critical-missing-link-in-the-industry-chain-302803783.html

   the field has only 0.05% of the data needed for true generalization… causing models to ‘hallucinate’ physical possibilities, such as requesting motor torques that exceed hardware limits

   ↩

5. The Guardian — https://www.theguardian.com/technology/2026/jun/13/anthropic-disable-advanced-ai-models-us-government-order

   Anthropic disable advanced AI models [following] US government order

   ↩

6. Medium (Joe Njenga) — https://medium.com/@joe.njenga/meet-anthropics-ai-trained-robot-dog-is-this-the-start-of-a-new-ai-race-a28468332573

   Meet Anthropic’s AI-trained robot dog — is this the start of a new AI race?

   ↩

7. iHeart/AP coverage of Kyra Benton case — https://kool108.iheart.com/content/2026-06-18-doctors-used-ai-to-diagnose-18-kids-with-rare-diseases-that-puzzled-them/

   After a 19-year diagnostic odyssey, the o3 system flagged a frameshift variant in the HSPB8 gene, confirmed as a rare form of myofibrillar myopathy — a diagnosis received one week before her 28th birthday

   ↩

8. medRxiv benchmark study (5,000+ cases) — https://www.medrxiv.org/content/10.1101/2024.07.22.24310816v3.full-text

   Exomiser achieved a ‘top-1’ accuracy of approximately 35.5%, whereas the best-performing LLMs (including GPT-4 variants) reached only 23.6%

   ↩

9. LifeSciBench coverage (letsdatascience.com) — https://letsdatascience.com/blog/openai-lifescibench-173-scientists-best-ai-passed-36-percent

   In the 2026 LifeSciBench — 750 expert-authored tasks — GPT-Rosalind passed only 36.1% of challenges, though it outperformed all other frontier models

   ↩

10. Google Research — AMIE real-world clinical study — https://research.google/blog/exploring-the-feasibility-of-conversational-diagnostic-ai-in-a-real-world-clinical-study/

    AMIE was safe and matched doctors on differential diagnosis quality, [but] human physicians still surpassed the AI in management plan practicality and cost-effectiveness

    ↩

11. Hoodline — Boston Children’s AI infrastructure report — https://hoodline.com/2026/05/boston-children-s-ai-war-on-medical-mysteries/

    BCH reclaimed approximately 60,000 hours of labor and redeployed over $7 million in costs through 50+ AI automations, moving from narrow pilots into a ‘common, secure AI foundation’

    ↩

12. R&D World — OpenAI Rosalind Biodefense launch — https://www.rdworldonline.com/openai-launches-rosalind-biodefense-offers-federal-agencies-early-access-to-its-life-sciences-model/

    GPT-Rosalind is accessible only through a ‘trusted access program’ requiring institutional safety reviews; early government partners include Lawrence Livermore National Laboratory and Johns Hopkins APL

    ↩

13. Dark Reading — Copilot SearchLeak attack — https://www.darkreading.com/application-security/copilot-searchleak-attack-1-click-data-theft

    attackers could use server-side fetches of image URLs within the search interface to exfiltrate MFA codes, password reset links, and private organizational files

    ↩

14. Beam.ai — AI Agent Security Breaches 2026 — https://beam.ai/agentic-insights/ai-agent-security-breaches-2026-lessons

    an attacker utilized Claude Code and GPT-4.1 to exfiltrate 195 million taxpayer records from Mexican government agencies… automated the execution of over 5,000 remote commands, accelerating the breach by an estimated 10x

    ↩

15. arXiv — PrivacyChecker (contextual-integrity mitigation) — https://arxiv.org/pdf/2603.01865

    PrivacyChecker uses a ‘privacy agent’ to intercept and reformulate queries locally, removing out-of-context information before it reaches external APIs… reduce leakage significantly (e.g., from 36% to 7% on models like DeepSeek-R1)

    ↩

16. arXiv — Whisper Leak / streaming side-channel paper — https://arxiv.org/html/2601.17617v1

    analyzes packet size and timing patterns in encrypted streaming responses to infer the topic of a conversation with over 90% precision, bypassing TLS/HTTPS protections

    ↩

17. hyper.ai independent write-up of MosaicLeaks — https://hyper.ai/en/stories/63333d90a8a4c8999f1e37d3ca2103ed

    producing high-quality, verifiable data points for the benchmark still requires substantial human intervention, limiting the current dataset scale… initial dataset is restricted to a few specific company contexts

    ↩

18. r/machinelearningnews discussion on enterprise agent benchmarks — https://www.reddit.com/r/machinelearningnews/comments/1rwx4y1/most_ai_agents_today_are_failing_the_enterprise/

    high benchmark scores often fail to translate to ‘boots on the ground’ utility, noting that models like Qwen3 Coder frequently enter ‘death spirals’ of bad tool calls in real-world scenarios

    ↩