Schwartz ships 36 papers, FAIR scores theorems, Llama runs two prompts at once

TL;DR

• Harvard’s Schwartz ran Claude through BootLoops to produce 36 manuscripts in 90 days across 18 fields.
• Reviewers flagged hallucinated citations and consensus-of-hallucinations risk in the physics output.
• Meta FAIR scores theorem interestingness as proof-length ratio, hitting Spearman ρ=0.756 with downstream utility.
• ProofOptimizer compresses state-of-the-art proofs 50-87%, undercutting the longer-is-deeper premise.
• Llama-3.2 runs 2 prompts per forward pass via averaged embeddings, restored by a fine-tune on <0.025% of pretraining data.

Today’s research beat is three methodological swings — one at how science gets produced, one at how theorems get valued, one at how prompts share compute. Harvard’s Matthew Schwartz ran Claude through a harness called BootLoops and produced 36 manuscripts across 18 fields in 90 days, with independent reviewers already flagging hallucinated citations and consensus-of-hallucinations risk. Meta FAIR proposes a proof-length ratio as a measure of mathematical interestingness, landing the same week ProofOptimizer shortens state-of-the-art proofs 50–87% — a direct counter to the longer-is-deeper premise. And a Llama-3.2 result shows 2 prompts can share one forward pass by averaging their embeddings, a property strongest at initialization that a tiny fine-tune can restore after pretraining washes it out.

Schwartz’s BootLoops ships 36 Claude-written papers in 90 days

Source: anthropic-research · published 2026-10-01

TL;DR

• Harvard’s Matthew Schwartz produced 36 manuscripts across 18 fields in 3 months by running Claude through a harness called BootLoops.
• 15 novel elliptic Feynman integrals, a 20-year-old ecology equation solved, and 4,452 economics papers audited.
• Independent reviewers flag hallucinated citations and a “consensus of hallucinations” risk in the physics output.
• Schwartz pitches AI as a convex hull filling gaps between human expertise silos.

The output is real, and it is a lot

Anthropic’s post on “Claude-shaped science” documents what Matthew Schwartz, a Harvard theoretical physicist, got out of three months with Claude Opus 4.5 and then Claude Fable 5: 36 manuscripts, 18 fields, 19 human collaborators. The connective tissue is BootLoops, an open-source harness that coordinates a “master session” of Claude sub-agents on Google Cloud VMs — porting code, validating data, playing adversarial referee — around a semi-numerical bootstrap that pushes integrals to 1,000-digit precision.

The individual results are substantive enough to take seriously. BootLoops 1.0 shipped 30 elliptic Feynman integral computations, split evenly between literature reproductions and genuinely new results, with constants recognized via PSLQ against a provenance tool called Gatekeeper ¹. In ecology, the harness cracked the Etienne equation for neutral biodiversity at scales that had stalled for two decades. In economics, an NBER working paper with Andrews and Shapiro reports that the AI data editor flagged discrepancies in 3,460 of 4,452 papers (~78%), delivered 10×+ speedups on 496 articles, and autonomously proposed extensions in roughly 1,000 cases ² — numbers the Anthropic post quietly declines to feature.

What BootLoops actually is

flowchart LR
    H[Human scientist<br/>sets taste + direction] --> M{Master Claude session}
    M --> P[Porting agent]
    M --> V[Validation agent]
    M --> R[Adversarial referee]
    P & V & R --> G[Gatekeeper<br/>PSLQ, 1000-digit checks]
    G --> O[Manuscript draft]
    O -.hallucinated refs.-> X((Error surface))

Schwartz’s conceptual pitch is the convex hull: human expertise is “jagged,” concentrated in narrow silos, and LLMs are useful not at the frontier but in the interior — connecting Feynman diagrams to Bayesian evidence integrals in genetics, or stress-pattern databases across 6,072 languages. He is explicit that Claude provides neither taste nor direction, and that it suffers “compaction,” poor time estimation, and a tendency to declare victory on incomplete proofs.

The verification gap the post underplays

Independent commentary converges on one worry: BootLoops ships errors as fast as it ships results. A physics-adjacent review of Schwartz’s earlier Sudakov-shoulder preprint found hallucinated bibliographies — real titles with invented author lists — despite instructions to double-check every reference ³. More damagingly, Jonathan Oppenheim’s arXiv 2512.07809 dissects a separate AI-origin physics paper and shows the model’s “novel” covariance criterion was a re-framing of 35-year-old Gisin–Polchinski work that tested Hamiltonian locality rather than nonlinearity ⁴. His warning on generator–verifier pipelines of the BootLoops type:

Risk producing a consensus of hallucinations rather than catching them.

That is precisely the failure mode a generator-and-referee loop like BootLoops is built to prevent. Economists have their own version: Per Engzell, cited in Tyler Cowen’s roundup, asks whether original authors would recognize the AI’s “improvements” as fixes rather than silent re-specifications ⁵. The Aspen “10,000 Einsteins” panel split along the same seam — convex-hull optimists on one side, skeptics arguing that if LLMs trivially solve second-year doctoral problems, the training pipeline is “collectively broken” and reliability degrades precisely where real research begins ⁶.

Takeaway

The honest headline is not “36 manuscripts in 18 fields.” It is “36 manuscripts whose errors only domain experts can catch, produced faster than domain experts can catch them.” BootLoops is a credible demonstration that Claude-shaped problems exist. It is also a credible demonstration that the bottleneck has moved from authorship to audit — and that arXiv’s reported one-year ban on fabricated AI references is a floor, not a ceiling, for what institutions will have to build.

---

Meta FAIR scores math ‘interestingness’ by proof-length ratio

Source: hf-daily-papers · published 2026-09-22

TL;DR

• Patel et al. define “interestingness” as predicted proof length divided by statement description length.
• The metric shows Spearman ρ=0.756 with downstream utility (how much a theorem shortens later proofs).
• Rewarding interestingness cuts mathlib overlap from 91.9% to 30.6%, which the authors read as out-of-distribution discovery.
• ProofOptimizer (ICLR 2026) compresses state-of-the-art proofs by 50–87%, directly contradicting “longer proof = deeper theorem.”

The metric

The paper’s load-bearing move is a scalar: interestingness I(T|P) = 100 · V(T|P) / L(T|P), where V is the predicted number of Lean 4 proof lines and L is the character count of the statement plus necessary definitions. Theorems that are short to state but long to prove score high. The authors argue this intrinsic property tracks extrinsic utility — how much a theorem shortens later proofs in a library — and report Spearman ρ=0.756 between the two on mathlib.

To estimate V(T|P) they train a 27B Qwen 3.6 value model with GRPO, using a “premise expansion” trick that unrolls atomized mathlib lemmas back into multi-step proof challenges. The resulting model beats GPT-5.5 and Claude Opus 4.6 on mean absolute error for proof-length prediction. A separate Conjecturer model is then GRPO-trained to maximize interestingness while being penalized for parse failures, premise-irrelevant statements, and anything simp or tauto can close.

What the numbers show

The headline result is a distributional shift. Base Qwen produces theorems that LLM judges rate 91.9% “substantially or fully contained” in mathlib; the interestingness-trained conjecturer drops that to 30.6%. Mean ground-truth interestingness rises from 1.76 to 7.58, and the trained 27B outperforms a prompted Claude Opus 4.6 across all eight tested areas (number theory, combinatorics, algebra, analysis, and others). In a six-round graph-theory discovery loop, pruning candidates by interestingness beat pruning by raw proof length — the length-only variant produced bloated, low-quality statements.

The contradiction nobody has resolved

Here is where the research bundle complicates the story. ProofOptimizer (arXiv:2510.15700, ICLR 2026) shows that routine RL post-processing shortens state-of-the-art prover output by 87% on miniF2F, 57% on PutnamBench, and ~50% on IMO 2025 without affecting correctness ⁷. If half-to-seven-eighths of a proof’s length is removable slack, then V(T|P) measured on raw generations is substantially noise, and a ratio built on top of it partially tracks tactic-coverage artifacts rather than mathematical depth. The two systems are philosophically opposed: Patel et al. reward length, ProofOptimizer penalizes it.

Reviewers on The Moonlight raise a second structural concern: as the iterative loop accumulates premises, V(T|P) for previously-discovered theorems should mechanically fall, so yesterday’s “interesting” result becomes today’s trivial corollary ⁸. alphaXiv commenters add that Lean tactics like aesop and grind can collapse conceptually deep arguments into a single line, biasing the metric toward whatever the current tactic library doesn’t yet automate ⁹. The authors concede both points in their limitations section but do not address them in the artifact.

The taste problem

The deeper critique comes from working mathematicians. Kevin Buzzard argues on the Xena blog that proofs too complex for humans to absorb fail mathematics’ primary epistemic function regardless of any scalar score ¹⁰. James Maynard and Javier Gómez-Serrano, discussing OpenAI’s recent Millennium-problem work, make the parallel point that Lean-verified AI output routinely lacks “storytelling” and conceptual clarity ¹¹. A metric that rewards long proofs of short statements is orthogonal to both complaints — and arguably exacerbates them.

The paper’s contribution is real: it operationalizes a question the field has been hand-waving about. But the 91.9%→30.6% number should be read as drift away from mathlib under a length-biased reward, not confirmed novelty, until there is a baseline against STP-style self-play ¹² and a version of the metric that survives ProofOptimizer-style compression.

---

Llama-3.2 runs two prompts through one forward pass

Source: hf-daily-papers · published 2026-09-23

TL;DR

• Averaging the embeddings of two text streams makes a Transformer output roughly the average of each stream’s next-token distribution.
• Strongest at initialization, the property degrades monotonically as the model trains on single-stream language modeling.
• A fine-tune on <0.025% of pretraining data restores it, cutting content-token median rank from 284 to 5 on Qwen2.5-3B.
• Cost: Pythia-2.8B’s single-stream LAMBADA accuracy falls from 0.544 to 0.357 after the linearity-restoring tune.

The Superposition Linearity Hypothesis

Tikhonov et al. make a sharp architectural claim: if you take two token sequences A and B and feed the model their element-wise embedding average, a standard Transformer treats the result as a superposition rather than noise. The ground-truth next token for stream A appears in the top-10 of the mixed output distribution 30–40% of the time on an un-tuned Llama-3.2-3B or Qwen2.5-3B; by top-100, recovery exceeds 60%. This holds across Pythia, Qwen, Llama, OLMo, and Gemma without any training for the task.

The theoretical hook is that this isn’t learned. Tracking Pythia checkpoints shows hidden-state additivity error rises monotonically through pretraining — the standard LM objective actively suppresses an architectural prior the randomly-initialized network already had ¹³. This extends Anthropic’s Toy Models of Superposition ¹⁴ from the feature axis (more concepts than neurons) to the sequence axis (more streams than forward passes).

Why the signal survives attention

The ablation that carries the paper is attention-patching. Replace stream A’s attention weights with those from an unrelated stream C — “donor patching,” preserving locality and sink structure but not content — and predictable tokens (grammar, high-frequency) still get recovered. Shuffle the weights randomly and recovery collapses. Content tokens, in contrast, need the actual additive composition of embeddings: they’re reconstructed from the value-vector sum, not the attention shape. The U-shaped depth profile — final layers stay more linear than middle layers even late in training — is consistent with this picture.

The catches the headline hides

Three independent threads complicate the “free throughput doubling” reading.

Decoding is unsolved. Averaging logits creates a “geometric-mean obstruction” that penalizes tokens plausible in one stream but implausible in the other. The authors’ Joint Contrastive Decoding — using a small auxiliary model to push logits toward one stream and away from the other — hits 0.430 LAMBADA accuracy on mixed streams versus 0.540 for the small-model single-stream baseline. Independent reviewers list full disentanglement of a mixed hidden state as an open problem with a “substantial residual gap” ¹⁵.

The fine-tune has teeth. Restoring linearity on Pythia-2.8B drops its own single-stream LAMBADA from 0.544 to 0.357 — roughly a 35% relative hit ¹⁶. You are trading base capability for parallel throughput, not getting it for free.

Serving economics are narrower than 2×. Practitioner notes point out guided decoding gives no latency win for a single request and loses efficiency when the two streams have unequal lengths ¹⁷ — the win is a batching trick, not a per-request speedup.

And the interpretability tension is real. Superposition is exactly the mechanism that makes neurons polysemantic and SAE-based audits hard ¹⁸. A fine-tune that restores superposition at inference time is pushing against the direction mechanistic interpretability wants models to move.

What would change the verdict

The ablations are the strongest part of the paper; the serving story is the weakest. The experiments cap at context length 512, two streams, and English. Scale any of those axes convincingly and this becomes a real inference primitive rather than a striking architectural curiosity.

Round-ups

WROP dataset trains object permanence into video world models

Source: hf-daily-papers

Video generation models show emergent reasoning but fail basic object permanence, the cognitive prior that objects persist when occluded. WROP supplies a core-cognition-inspired training corpus that instills permanence and solidity in world models, a step toward human-like physical intelligence. The paper drew 237 upvotes.

AgentKernel moves agent trust enforcement into an OS-level kernel

Source: hf-daily-papers

Current agent governance runs as application middleware, sharing a process boundary with the agents it monitors — a weak defense when untrusted inputs reach privileged tools. AgentKernel proposes an OS-style trust boundary that mediates memory, inputs, and tool calls to contain prompt-injection attacks.

ExplorationBench tests AI discovery in verifiable alien worlds

Source: hf-daily-papers

ExplorationBench evaluates whether AI systems can form hypotheses, design experiments, and iterate on results in synthetic worlds whose rules aren’t in pretraining data. The benchmark separates genuine exploration from memorized recall by verifying new hypotheses against ground-truth world dynamics the model has never seen.

Coding agents synthesize task-and-motion planners without hand-engineering

Source: hf-daily-papers

Generalized task and motion planning usually demands heavy TAMP-specific engineering to reuse structure across problem instances. The RoboCode project shows coding agents can automate that step, synthesizing planners that handle discrete decisions coupled to geometric, kinematic, and dynamic constraints across new instances.

SAEs encode parts-of-speech as distributed groups, not single latents

Source: hf-daily-papers

Using part-of-speech tags as a controlled probe, researchers find morpho-syntactic categories are highly recoverable from sparse autoencoder activations but never map one-to-one onto individual latents. PoS distinctions instead emerge from structured groups of features, complicating the clean interpretability story SAEs promise.

Rufus-Air open-sources an 8-stage post-training recipe for GLM-4.5-Air

Source: hf-daily-papers

Built on GLM-4.5-Air-Base (106B-A12B), the pipeline runs SFT, then reasoning, coding, and instruction-following RL, three agent stages, and RLHF. The release documents data, reward design, infrastructure, and stagewise results, moving from verifiable rewards to judge-based signals as capabilities advance.

PUBG Ally fields a voice-enabled LLM teammate inside Battlegrounds

Source: hf-daily-papers

The embodied agent plays alongside humans in PUBG: BATTLEGROUNDS, pairing agentic tool use with real-time game control under strict latency limits. A language model drives perception, speech, and action in sync, so chatter stays aligned with what Ally is actually doing on the map.

Footnotes

1. Unite.ai — BootLoops 1.0 coverage — https://www.unite.ai/schwartz-releases-bootloops-1-0-an-open-source-llm-harness-for-science/

   The initial 1.0 release documented 30 end-to-end integral computations: 15 were reproductions of known literature, while the remaining 15 represented novel results, including challenging elliptic Feynman integrals that were previously uncomputed.

   ↩

2. NBER WP 35782 (Andrews, Schwartz, Shapiro) — https://www.nber.org/system/files/working_papers/w35782/w35782.pdf

   The workflow flagged discrepancies in 3,460 articles or their associated appendices… In nearly 1,000 cases, the system autonomously proposed extensions to the original research.

   ↩

3. r/singularity discussion thread — https://www.reddit.com/r/singularity/comments/1wvfbua/a_harvard_physicist_spent_3_months_doing_research/

   Critical review of Schwartz’s AI-assisted preprints revealed technical sloppiness, most notably hallucinated citations where AI-generated manuscripts listed spurious authors for real paper titles.

   ↩

4. arXiv 2512.07809 — Jonathan Oppenheim — https://arxiv.org/html/2606.25057v1

   The AI-generated criterion tested ‘locality of the Hamiltonian’ rather than the intended property of nonlinearity, making the entire result irrelevant to the problem it claimed to solve… a ‘consensus of hallucinations’.

   ↩

5. Marginal Revolution — Tyler Cowen — https://marginalrevolution.com/marginalrevolution/2026/10/merging-llms-and-economics-research.html

   Experts like Per Engzell have expressed skepticism regarding the AI’s ‘improvement’ claims, questioning whether original authors would agree with the AI’s re-implementations.

   ↩

6. Aspen Center for Physics — ‘10,000 Einsteins’ panel — https://aspenphys.org/event/10000-einsteins-ai-and-the-future-of-theoretical-physics/

   Some physicists argue that if AI can solve second-year doctoral problems in minutes, the traditional educational model is ‘collectively broken’… AI is proposed not as an agent to push the ultimate frontier further, but as a tool to fill in these gaps.

   ↩

7. ProofOptimizer (arXiv:2510.15700, ICLR 2026) — https://arxiv.org/abs/2510.15700

   ProofOptimizer can reduce the length of proofs generated by state-of-the-art provers (such as Seed-Prover) by 87% on miniF2F, 57% on PutnamBench, and roughly 50% for IMO 2025 problems.

   ↩

8. The Moonlight review of ‘Learning to Discover Interesting Mathematics’ — https://www.themoonlight.io/en/review/learning-to-discover-interesting-mathematics

   Experts have questioned whether the interestingness ranking will remain stable as a library grows and premises multiply, potentially changing the ‘conditional’ difficulty of previously discovered theorems.

   ↩

9. alphaXiv discussion page for arXiv:2609.28603 — https://www.alphaxiv.org/abs/2609.28603

   Critiques on alphaXiv note that proof length can be stylistic and that it remains to be seen whether these autonomously generated libraries will lead to breakthroughs in human-relevant fields like physics or economics.

   ↩

10. Kevin Buzzard, Xena Project blog (Oct 2026) — https://xenaproject.wordpress.com/2026/10/01/to-grieve-or-not-to-grieve/

    If a proof remains too complex for human absorption, it fails the primary goal of mathematics: advancing understanding rather than just confirming truth.

    ↩

11. The Guardian, ‘OpenAI mathematicians millennium prize problem’ (Sep 2026) — https://www.theguardian.com/science/2026/sep/12/openai-mathematicians-millennium-prize-problem

    Mathematicians like James Maynard and Javier Gómez-Serrano have noted that while AI results often pass formal machine verification in Lean, they frequently lack ‘storytelling’ or conceptual clarity.

    ↩

12. STP: Self-play Theorem Prover (arXiv:2505.22257) — https://arxiv.org/html/2505.22257v1

    The STP framework explicitly extends Minimo’s conjecturer-prover idea to practical languages like Lean 4, using DeepSeek-Prover-V1.5 as its base… generating millions of conjectures and proving them to create an adaptive curriculum.

    ↩

13. aimodeling.com — ‘LLM Linear Superposition: Two Thoughts’ — https://www.aimodeling.com/en/news/slug/llm-linear-superposition-two-thoughts

    Superposition fidelity is maximized at initialization and gradually diminishes as the model optimizes the standard language modeling objective — the training process actually degrades the model’s natural ability to maintain distinct semantic signals.

    ↩

14. Elhage et al., ‘Toy Models of Superposition’ (Anthropic, arXiv:2209.10652) — https://arxiv.org/abs/2209.10652

    Neural networks can represent more features than they have dimensions by storing them as nearly-orthogonal directions, a phenomenon we call superposition.

    ↩

15. Emergent Mind — ‘Fully disentangle constituent streams from mixed hidden state’ (open problem) — https://www.emergentmind.com/open-problems/fully-disentangle-constituent-streams-from-mixed-hidden-state

    A substantial residual gap remains between mixed-stream decoding and single-stream baselines; fully recovering constituent continuations from a superposed state is an unresolved challenge.

    ↩

16. aimodels.fyi — independent paper summary — https://www.aimodels.fyi/papers/arxiv/your-transformer-can-hold-two-thoughts-once

    Fine-tuning the model to improve superposition linearity results in a measurable decrease in single-stream quality — Pythia-2.8B LAMBADA accuracy dropped from 0.544 to 0.357 after the linearity-restoring procedure.

    ↩

17. aicoder.com — practitioner commentary on the Tikhonov et al. result — https://aicoder.com/news/news-20260926-transformer-linear-superposition-two-thoughts

    The ‘guided decoding’ trick is not yet a shipped system; it fails to provide latency benefits for single requests and loses efficiency if the two streams differ significantly in length.

    ↩

18. The Sequence (Substack) — mechanistic interpretability edition — https://thesequence.substack.com/p/the-sequence-knowledge-712-mechanistic

    Superposition drives polysemanticity, where individual neurons fire for multiple unrelated concepts — encouraging superposition at inference time may trade off against interpretability and safety audits that rely on monosemantic features.

    ↩