Beam-1.0 hallucinates 47%, Anthropic merges 30K PRs/day, RTX Spark hits 98°C

TL;DR

• Reflection ships Beam-1.0, a 501B open MoE scoring 97.7% AIME at 47% hallucination.
• Anthropic hits 30,000 merges/day with Claude as first reviewer, up from 2,500.
• Nvidia’s RTX Spark laptops power-fail at 98°C under sustained double-precision load.
• Nvidia staged a multi-level embargo gagging first-wave reviewers on battery, thermals, chassis.
• Willison builds a LaTeX renderer encoding full docs in URL fragments via Claude Code.

Today’s three tech releases each lead with a throughput number, and each number comes with a quality floor the vendor would rather not put in the headline. Reflection’s Beam-1.0 lands frontier-class on AIME, SWEBench and LiveCodeBench — and clocks a 47% hallucination rate on AA-Omniscience, roughly 2× its nearest open-weight peer. Anthropic grew Claude Code merges from 2,500 to 30,000 a day in four months by making Claude the first reviewer — and independent benchmarks peg AI reviewers at 5–15% false-positive rates, with practitioners warning about PRs approved unread. Nvidia’s RTX Spark laptops run 122B models locally at 35 tok/s — and power-failed at 98°C under sustained load, with a staged embargo keeping that detail out of day-one coverage.

The pattern isn’t that these releases are broken. It’s that the metric each vendor leads with — parameters, merges, local tokens — is also the thing producing the ceiling. Scaling the headline number is what makes the floor visible.

Reflection ships Beam-1.0: 501B open, 47% hallucination rate

Source: huggingface-blog · published 2026-10-16

TL;DR

• Reflection released Beam-1.0, a 501B-parameter / 23B-active MoE under Apache 2.0, with a 1M-token context window.
• Self-reported scores land frontier-class: 97.7% AIME 2026, 81.6% SWEBench Verified, 88.3% LiveCodeBench.
• Artificial Analysis clocks a 47% hallucination rate on AA-Omniscience, roughly 2× Kimi K2 Horizon (26%) and Qwen3.8 (30%).
• ~300 GB VRAM at 4-bit: $19K Mac Studio cluster to $60K+ RTX PRO 6000 rig.
• The sleeper is Miles, a PyTorch-native RL stack with Rollout Routing Replay for GRPO on sparse MoEs.

A credible frontier-scale open release, with asterisks

Reflection AI — the Laskin/Antonoglou ex-DeepMind outfit, not the 2024 Reflection 70B that got accused of being a Llama wrapper ¹ — dropped Beam-1.0-501B-A23B this week. The headline specs are real: 501B total parameters with 23B active per token, 52 decoder layers, 256 experts per MoE layer with group-limited top-8 routing, and a hybrid sliding-window/global attention scheme that keeps the KV cache around 36 KiB per token on a 1M-token context. Apache 2.0, day-0 support in transformers, vLLM, and SGLang.

The self-reported benchmarks put it in frontier company: 97.7% on AIME 2026, 90.6% GPQA Diamond, 81.6% SWEBench Verified, 88.3% LiveCodeBench. NVFP4 quantization retains ~99% of BF16 reasoning. On paper, this is the strongest U.S. open-weights release of the year.

Independent evals say “answers more, hallucinates more”

Artificial Analysis’s AA-Omniscience evaluation is where the gloss comes off. Beam-1.0 posts a 47% hallucination rate — versus 26% for Kimi K2 Horizon and 30% for Qwen3.8 ². The model compensates by attempting harder questions, so its net index edges those peers, but factual calibration is clearly weak. External rankings place it #2 among U.S. open models and still behind Kimi K3 and DeepSeek V4.1 Flash on raw reasoning and agentic tasks ²³.

That weak calibration is at least consistent with Reflection’s headline efficiency pitch: 3–4× fewer tokens than Z.ai’s GLM-5.2, achieved by an RL regime that explicitly penalizes unnecessary generation ⁴. Critics have a name for the failure mode:

“learning to not solve the task” — what happens when a token penalty teaches the model to commit early rather than reason through ⁴.

Reflection’s release notes don’t quantify how they avoided it.

”Open weights you can own” needs a budget line

Apache 2.0 is meaningful; the hardware bill is also meaningful. BF16 Beam is 1 TB — 16 H200s to serve, or 8 for FP8. NVFP4 drops the footprint to ~280 GB but assumes Blackwell B200/B300 silicon (four B200s or two B300s recommended). At 4-bit, independent reviewers put practical hosting at $19K for a high-end Mac Studio cluster, $60K+ for RTX PRO 6000 rigs — and argue DeepSeek V4.1 Flash delivers more on the same iron ³. “Own it yourself” is really “serve it yourself if you’re a well-funded org.”

The real sleeper: Miles

The release ships a PyTorch-native RL post-training stack called Miles, built with the SGLang team. It pairs SGLang rollouts with Megatron-LM training and ships Rollout Routing Replay (R3), which records MoE expert routing during rollout to keep it aligned at training time ⁵ — a non-trivial engineering fix for anyone doing GRPO on sparse models. For teams doing serious post-training on MoEs, Miles may matter more than Beam itself.

The positioning is explicit. Reflection carries a $25B valuation with Nvidia, Sequoia, and Lightspeed backing, a reported $6.3B SpaceX compute deal, and a U.S.-brokered South Korea sovereign-AI partnership ⁶. Beam is being pitched as the Western open-weights counterweight to DeepSeek and Z.ai — not just a model drop.

---

Anthropic merges 30,000 PRs/day with Claude as first reviewer

Source: simon-willison · published 2026-10-16

TL;DR

• Anthropic’s Claude Code team went from 2,500 to 30,000 merges/day in 4 months by making Claude first reviewer.
• Scion, a custom VCS, borrows heavily from Meta’s Sapling/Mononoke — attribution Anthropic initially understated.
• Independent benchmarks peg AI reviewers at 5–15% false-positive rates, with “architectural disasters” slipping through.
• Practitioners warn of “zombie code reviews”: PRs approved unread because agents outpace human comprehension.

Review as spellcheck, not a gate

Addy Osmani, Morgan Westlee Lunt and Qing Wang’s post reframes the entire premise of code review. “Did a human read this?” is treated as a lossy proxy for “what could go wrong and which check should catch it?” In practice that means Claude reviews — and often fixes — most changes before any human is pinged, and human attention is rationed for architectural calls. The headline consequence: throughput jumped from ~2,500 daily merges in May 2026 to over 30,000 by September.

That is a vendor-reported number on a vendor-built workflow. It is also, so far, the most concrete data point anyone has published on continuous AI review at scale.

The infrastructure nobody mentions in the headline

The 30,000/day figure is not a model story — it’s a plumbing story. Seven engineers built Scion in five months to handle 25 million checkouts per day; git status equivalents dropped from 3.9 seconds to 19 ms, reclaiming roughly 25% of CI CPU capacity ⁷. Without that, agent sandboxes alone would push 600 PB of daily Git traffic.

Scion’s architecture is also not clean-room. Former Mononoke tech lead Rain (“sunshowers”) argued the initial post’s thin attribution to Meta’s Sapling/Mononoke work “bordered on plagiarism”; Anthropic later amended it with a clarifying footnote ⁸. The uncomfortable loop: Claude helped build Scion, and Claude was trained on the open-source VCS projects Scion effectively ports.

What independent evidence actually says about AI review

Anthropic’s framing elides a well-documented failure mode. Field studies put most AI reviewers at 5–15% false-positive rates, and the moment FPR crosses 20–30% developers reflexively dismiss warnings — a 10% FPR alone burns 20+ engineering hours per month on a mid-sized team ⁹. Vendor bug-catch claims swing wildly: CodeRabbit self-reports 45.8%, Graphite Diamond 18.3% on comparable datasets; the only honest evaluation is a side-by-side pilot measuring developer acceptance rate ¹⁰.

More damning for the “continuous review” pitch: agents reliably flag anti-patterns and unused imports but routinely greenlight changes that violate design-token hierarchies or break business-specific payment logic, because the context window can’t hold system intent ¹¹. Anthropic’s adversarial-review setup shares those detector limits, and the post itself concedes models drift into “myopia” on pedantic concerns.

The human externality

The sharpest practitioner critiques land on comprehension, not accuracy.

“Outsourcing the building outsources the learning.”

Thorsten Ball’s line captures the worry. Gergely Orosz has coined “zombie code reviews” — PRs approved unread because agents generate faster than humans can parse — and reports a visible quality decline alongside the velocity gains. Kent Beck flags “re-soloing,” where AI lets developers skip peer collaboration entirely ¹². When the same model writes and reviews, test failures risk becoming mirror errors: two Claudes agreeing on a broken assumption.

Takeaway

Treat the post as an existence proof, not a template. Continuous AI review clearly scales inside a lab whose product is the reviewing agent, with custom infrastructure and effectively unlimited inference budget. For teams without Scion, without a frontier-lab compute margin, and without Claude Code’s bespoke harness, the load-bearing question is still the one Anthropic quietly sidesteps: who is reading the code the model just wrote, and do they understand it?

---

RTX Spark laptops throttle to 98°C running local 122B models

Source: the-verge-ai · published 2026-10-16

TL;DR

• Nvidia’s RTX Spark laptops start at $2,599 and climb near $7,000 for a 128GB CUDA-developer SKU.
• Qwen 3.5 122B hits 35 tok/s on the Surface Laptop Ultra’s local inference stack.
• The unit power-failed at 95–98°C under sustained double-precision load.
• Nvidia staged a multi-level embargo gagging first-wave reviewers on battery, thermals, and chassis behavior.

A local-LLM workstation in a laptop shell

Strip away The Verge’s “superchip” framing and RTX Spark is a specialized box: a Windows-on-Arm laptop built so a developer can run a 100B+ model on battery without touching a cloud API. Nvidia’s own developer pitch is more honest than the marketing. Spark ships the first PyTorch-CUDA build for Windows on Arm, a jointly-engineered “Windows Developer Config” with Microsoft, and two agent-oriented runtimes: PAIR (serve local inference to other machines on your LAN) and Switchyard (a router that spills tokens to cloud APIs when the local model stalls) ¹³. Nvidia claims compatibility with 97% of the top 15,000 Python packages — the number that actually matters if you’ve ever tried to pip install on an M-series Mac ¹³.

Pricing confirms the audience. The ASUS ProArt P14/P16, HP OmniBook Ultra 16, Lenovo Yoga 9n and Microsoft Surface Laptop Ultra all start at $2,599, with 128GB unified-memory configurations pushing close to $7,000 ¹⁴. These are not MacBook Pro competitors; they’re DGX Spark desktops with a keyboard attached.

The thermal and prefill reality

StorageReview’s time with the Surface Laptop Ultra gives the first hard numbers. Qwen 3.5 (122B) sustains ~35.2 tokens/sec and Nemotron 3 Nano (30B) hits 63.9 tok/s — genuinely competitive for a laptop. But the same unit repeatedly climbed to 95–98°C and power-failed under sustained double-precision load, and Microsoft’s own documentation shows a 170-second prefill wait on a 128K-token prompt ¹⁵. That second number is the one that will bite agentic workflows, which resubmit long contexts on every loop. The 45–80W laptop envelope is compute-bound in a way the 140W DGX Spark desktop simply isn’t ¹⁵.

Notebookcheck’s teardown adds a second problem: the N1X uses Cortex-X925 + A725 cores that were already superseded by newer Arm designs before Spark shipped. Single-core Geekbench of ~2,718 trails Apple’s M4 Pro by roughly 30%, and idle power draw is a startling ~30W — bad news for a device sold on mobility ¹⁶.

Mac Studio vs Spark, and the embargo

Spark laptop bandwidth is inferred from the synthesis note that it runs “roughly half” its desktop sibling ¹⁷; Nvidia has not published the figure directly, which is consistent with the embargo pattern below.

The r/LocalLLM thread weighing Spark against a Mac Studio is blunt: Apple wins on raw bandwidth and memory ceiling, but macOS has no CUDA, no native FP4 tensor acceleration, no RDMA, and Docker runs inside a Linux VM. For 24/7 on-prem agents, the Nvidia stack still wins despite the bandwidth gap ¹⁷.

Read any of this week’s glowing coverage against the embargo structure Nvidia imposed. Reviewers were explicitly barred from discussing battery life, thermals, or chassis-specific variables in the first wave — a “multi-level embargo” critics called scummy ¹⁸. The GPU numbers are real. The parts Nvidia didn’t want you to see are the parts that decide whether a $7,000 laptop is a workstation or a space heater.

Round-ups

Claude Code builds a LaTeX renderer that stores docs in URL fragments

Source: simon-willison

A new LaTeX rendering tool encodes the full document in a URL #fragment, making every equation a shareable bookmark. Inspired by Tiny Little Office, the build pairs MathJax 4.1.3 and faster-latex WASM with an iframe sandbox for XSS protection, all generated via Claude Code prompts.

Footnotes

1. VentureBeat (on 2024 Reflection 70B scandal) — https://venturebeat.com/business/new-open-source-ai-leader-reflection-70bs-performance-questioned-accused-of-fraud

   independent testers found the model significantly underperformed its claims, with some researchers suggesting the initial release was merely a wrapper for other models or a mislabeled version of Llama 3

   ↩

2. Artificial Analysis evaluation — https://artificialanalysis.ai/articles/reflection-beam

   Beam-1.0 recorded a hallucination rate of 47% on the AA-Omniscience benchmark… significantly higher than competitors like K2 Horizon (26%) and Qwen3.8 (30%)

   ↩ ↩²

3. YouTube review (independent) — https://www.youtube.com/watch?v=l8PoJzKv38s

   Hosting the model at 4-bit quantization requires roughly 300 GB of VRAM, translating to hardware costs ranging from $19,000 for high-end Mac Studio clusters to over $60,000 for enterprise-grade NVIDIA RTX PRO 6000 setups… competitors like DeepSeek V4.1 Flash offer superior performance on the same machines

   ↩ ↩²

4. Layer3Labs guide — https://www.layer3labs.io/guides/reflection-ai-explained

   Reflection AI’s primary claim is a 3-to-4x improvement in inference efficiency over Z.ai’s GLM-5.2… achieved through reinforcement learning (RL) that explicitly penalized the model for using unnecessary tokens… critics express concern that the model may learn to avoid complex tasks entirely to minimize token costs—a phenomenon described as ‘learning to not solve the task’ if applied naively

   ↩ ↩²

5. PyTorch blog on Miles framework — https://pytorch.org/blog/miles-a-pytorch-native-stack-for-large-scale-llm-rl-post-training/

   Miles addresses the ‘distributed systems problem’ of modern RL by coordinating high-throughput rollout workers (via SGLang) with efficient training consuming those samples (via NVIDIA Megatron-LM)… Rollout Routing Replay (R3)… records expert routing in MoE models during rollout to ensure alignment during training

   ↩

6. The Daily Upside — https://www.thedailyupside.com/technology/artificial-intelligence/reflection-wants-to-deliver-americas-most-powerful-open-weight-ai/

   a $25 billion valuation… backers include Nvidia, Sequoia Capital, and Lightspeed… a $6.3 billion compute agreement with SpaceX and a U.S. government-brokered deal to help South Korea develop a ‘sovereign AI’ ecosystem

   ↩

7. DevOps.com - Anthropic Builds Its Own Source Control — https://devops.com/anthropic-builds-its-own-source-control-after-ai-agents-outgrow-git

   A team of seven engineers built Scion in five months to handle 25 million daily checkouts; checkouts dropped from minutes to seconds and git status equivalents fell from 3.9 seconds to 19 ms, reclaiming roughly 25% of CI CPU capacity.

   ↩

8. Simon Willison - Redesigning source control at Anthropic — https://simonwillison.net/2026/Oct/13/redesigning-source-control-at-anthropic/

   Scion’s architecture is heavily influenced by Meta’s Sapling/Mononoke, with former Mononoke lead Rain (sunshowers) arguing the initial lack of explicit attribution ‘bordered on plagiarism’ before Anthropic updated the post with a clarifying footnote.

   ↩

9. The New Stack - AI Code Review Burnout — https://thenewstack.io/ai-code-review-burnout/

   Industry benchmarks show most AI reviewers operate at 5-15% false-positive rates, but once FPR exceeds 20-30% developers reflexively dismiss warnings; a 10% FPR can waste 20+ engineering hours monthly for a mid-sized team.

   ↩

10. Macroscope - Greptile vs CodeRabbit 2026 — https://macroscope.com/content/greptile-vs-coderabbit-2026

    On the independent Martian Code Review Bench, Greptile scored F1 63.2 and CodeRabbit 61.8 on real OSS pull requests; vendor benchmarks diverge wildly (CodeRabbit 45.8% bug catch, Graphite Diamond 18.3%), so the recommended test is a side-by-side week measuring ‘acceptance rate.’

    ↩

11. Orizens blog - AI Code Review 2026 — https://orizens.com/blog/2026-11-03-ai-code-review/

    AI agents excel at anti-patterns and unused imports but frequently approve ‘architectural disasters’—code that violates design token hierarchies or breaks business-specific payment logic—because models reason over a limited context window and miss long-term system intent.

    ↩

12. xgrowing.ai - Thorsten Ball / Orosz / Beck commentary — https://xnews.xgrowing.ai/news/thorsten-ball-ai-code-quality-surpasses-human-review-redefining-development-5704

    Orosz warns of ‘zombie code reviews’ where agents generate code faster than humans can comprehend, so teams approve PRs unread; Ball adds that ‘outsourcing the building outsources the learning,’ and Beck flags ‘re-soloing’ where developers use AI to bypass peer collaboration entirely.

    ↩

13. NVIDIA Developer Blog (‘Code, Test, and Deploy on One Machine’) — https://developer.nvidia.com/blog/code-test-and-deploy-on-one-machine-with-nvidia-rtx-spark/

    Developers get a Windows Developer Config, PAIR for serving inference to other local devices, and a Switchyard model router that splits workloads between local agents and cloud APIs; NVIDIA claims compatibility with 97% of the top 15,000 Python packages and ships the first PyTorch-CUDA build for Windows on Arm.

    ↩ ↩²

14. StreetInsider / Nvidia launch release — https://www.streetinsider.com/Corporate+News/Nvidia+launches+RTX+Spark+laptops+starting+at+%242%2C599/27195871.html

    Nvidia launches RTX Spark laptops starting at $2,599

    ↩

15. StorageReview — Surface Laptop Ultra for Business — https://www.storagereview.com/review/microsoft-surface-laptop-ultra-for-business-review-rtx-spark-n1x

    Qwen 3.5 (122B) ran at ~35.2 tokens/sec and Nemotron 3 Nano (30B) at 63.9 t/s, but the unit repeatedly hit 95–98°C and suffered power failures under sustained load; Microsoft’s own data shows a 170-second prefill wait on a 128K prompt.

    ↩ ↩²

16. Notebookcheck N1X deep-dive (via r/hardware) — https://www.reddit.com/r/hardware/comments/1x6ow93/notebookcheck_nvidia_rtx_spark_analysis_finally/

    The N1X uses Cortex-X925 + A725 cores already superseded by newer Arm designs, and single-core Geekbench of ~2,718 trails Apple’s M4 Pro by roughly 30%; multi-core performance holds on battery but idle draw is unusually high at ~30W.

    ↩

17. r/LocalLLM — ‘Mac Studio vs DGX Spark for a 24/7 on-prem AI’ — https://www.reddit.com/r/LocalLLM/comments/1wvpf3u/mac_studio_vs_dgx_spark_for_a_247_onprem_ai/

    For sustained local inference the Mac Studio M5 Ultra’s 1.2–2 TB/s bandwidth and 512GB ceiling beats Spark’s ~300 GB/s, but macOS lacks RDMA, native CUDA and FP4 tensor acceleration — developers describe Spark as the better ‘production-ready’ box despite lower bandwidth.

    ↩ ↩²

18. Notebookcheck analysis (via r/hardware) — https://www.reddit.com/r/hardware/comments/1x6kzn9/notebookcheck_nvidias_rtx_spark_is_here_the/

    Reviewers were explicitly barred from discussing battery life, thermals, or chassis-specific variables in the first wave of coverage — a ‘multi-level embargo’ critics called scummy.

    ↩