Stanford clocks China at 2.7%, NVIDIA’s 550B NVFP4, Inclusion AI retrofits 1T

TL;DR

• Stanford AI Index puts China within 2.7% of US frontier on Chatbot Arena, down from 17–31%.
• NVIDIA ships 550B Nemotron 3 Ultra in 4-bit NVFP4, the largest such pretraining run to date.
• Inclusion AI retrofits its 1T Ling-2.0 MoE with 7:1 hybrid linear attention, reusing 20T pretraining tokens.
• OpenRouter ranks Ring-2.6-1T #75 of 94 overall, undercutting the report’s GPT-5.4 parity claim.
• Entry-level developer hiring (ages 22–25) fell ~20% since 2024 while senior hiring stayed flat.

Three unrelated lead papers today. Stanford’s 2026 AI Index lands as the annual industry yardstick — China within 2.7% of US frontier on Chatbot Arena, entry-level developer hiring down ~20% since 2024, and frontier models still misreading analog clocks about half the time. NVIDIA ships the largest 4-bit pretraining run published to date: a 550B-parameter Nemotron 3 Ultra in NVFP4, with a 5.9× decode claim that quietly compares Blackwell to Hopper. Inclusion AI retrofits its 1T Ling-2.0 MoE with 7:1 hybrid linear attention, preserving 20T tokens of prior pretraining — though OpenRouter ranks the resulting Ring-2.6-1T #75 of 94, well shy of the report’s GPT-5.4 parity pitch.

Stanford AI Index: China closes US model gap to 2.7%

Source: hf-daily-papers · published 2026-04-13

TL;DR

• Claude Opus 4.6 leads ByteDance’s Dola-Seed-2.0 by 2.7% on Chatbot Arena, down from a 17–31% US lead.
• China hit near-parity on roughly 1/23rd of US private AI investment — a capital-efficiency story policy memos will run with.
• Entry-level developer employment (22–25) fell ~20% since 2024, while senior hiring stayed flat.
• Top frontier models misread analog clocks ~50% of the time on ClockBench, even as they beat humans on ChemBench.

Parity arrived, and the scoreboard broke at the same time

The ninth Stanford AI Index lands with a headline number that has already escaped into the geopolitical discourse: the performance gap between leading US and Chinese frontier models has collapsed to 2.7% on Chatbot Arena, with Anthropic’s Claude Opus 4.6 at Elo 1,503 narrowly edging ByteDance’s Dola-Seed-2.0-Preview at 1,464 ¹. Two years ago that gap was 17–31%. China got here while absorbing roughly one twenty-third of US private AI investment — the capital-efficiency story that will dominate every policy memo written about this report.

The Index’s own caveat is that nobody should trust the scoreboard too much. R&D World highlights the “jagged frontier” running through the benchmarks chapter: models that crush ChemBench score below 20% on astrophysics replication, and top-tier systems fail ClockBench — reading an analog clock — about half the time ². SWE-bench numbers vaulted from 60% to near-100% in a year, but practitioners report roughly 37-point drops when those same agents hit production. The pivot toward agentic evaluations and Humanity’s Last Exam reads as overdue rather than visionary.

The sovereignty chapter nobody can define

The Index introduces an “AI sovereignty” analytical framework, and the independent reception has been rough. GIS Reports calls true sovereignty an “illusion” for most nations and argues the honest goal is “AI resilience” — modular, interoperable stacks that preserve optionality rather than national ownership of the whole pipeline ³. Stanford’s own researchers, quoted in the coverage, compare defining the concept to “nailing jelly to a wall.” Expect the chapter to be cited more than it is implemented.

The environmental ledger is similarly uncomfortable. The Index treats compute emissions abstractly; Epoch AI puts a number on it. Training Grok-4 alone consumed an estimated 310 GWh of electricity, emitted 140,000–150,000 tons of CO2, and required 750 million to 1.2 billion liters of cooling water ⁴. Multiply that across the dozen frontier runs of 2025 and the “AI is just software” framing stops surviving contact with a utility bill.

Enterprise reality: adopted, not deployed

The most actionable findings sit in the enterprise chapter. 88% of firms have adopted AI in at least one function, but fewer than 10% have scaled it with robust controls, and 78% of executives concede they could not pass an independent AI audit ⁵. That is the gap the report’s title is really about: capability moved faster than governance, evaluation, and the data infrastructure needed to track impact.

Labor data tells the same story from the worker side. Employment for software developers aged 22–25 fell roughly 20% since 2024, even as senior-level hiring held steady ⁶. The model isn’t “AI takes the jobs”; it’s “AI takes the rungs of the ladder,” and the Index is the first mainstream document to put a number on it.

The empirical scaffolding here is solid. The framing — that parity is real, sovereignty is achievable, and benchmarks mean what they say — is exactly where the independent commentary is pushing back hardest.

---

NVIDIA pretrains 550B Nemotron 3 Ultra in 4-bit NVFP4

Source: hf-daily-papers · published 2026-06-11

TL;DR

• 550B-parameter Nemotron 3 Ultra (55B active) is the largest 4-bit NVFP4 pretraining run shipped to date.
• NVIDIA’s 5.9× decode throughput claim over GLM-5.1 is hardware-confounded — NVFP4/Blackwell vs. BF16/Hopper.
• Leads RULER at 1M context (94.7% on the LLM-Stats leaderboard; 76.83% in the paper’s own table).
• Trails DeepSeek-V4 on GPQA and HLE despite winning agentic benchmarks.

4-bit pretraining is a recipe, not a property

The headline claim — stable NVFP4 pretraining at 550B — is real, but conditional. Independent analysis stresses that 4-bit stability isn’t intrinsic to the format: it requires Random Hadamard Transforms to flatten activation outliers and keeping the first/last 10–15% of transformer blocks in BF16 ⁷. NVIDIA’s own report corroborates this indirectly, with two loss divergences during the run. One traced to insufficient precision in gradient accumulation (fixed by reverting output-layer reductions to FP32); a second at 16T tokens forced the team to cap training at 20T rather than the originally planned horizon. “Stable NVFP4” is better read as a working recipe than a solved problem.

LatentMoE and the dead-expert problem

Ultra uses LatentMoE — compression-before-routing — to keep 55B parameters active out of 550B total. NVIDIA reports growing expert imbalance and “dead experts” in later layers, plus residual-norm spikes in early layers during the back half of pretraining. This isn’t a Nemotron-specific bug. Microsoft’s MAI-Base-1 technical report describes the same pattern: training-loss spikes on coding data correlated with expert imbalance, even under dropless routing ⁸. Routing collapse at the 500B+ scale looks structural to the LatentMoE design, and DeepSeek-V3-style auxiliary-loss-free routing is emerging as the direct critique.

Agentic ≠ reasoning, and the throughput math

The post-training pitch — Multi-teacher On-Policy Distillation (MOPD) across ten domain teachers, plus RLVR on verifiable math/code — produces strong agentic-trajectory and tool-use scores. Nathan Lambert’s review names the ceiling bluntly:

Because the process is on-policy, it can only amplify reasoning paths the student is already capable of producing; it cannot transfer high-level logic that falls outside the student’s sampling distribution.

Ultra correspondingly trails DeepSeek-V4 on HLE and GPQA despite winning agentic benchmarks ⁹. The throughput numbers deserve the same scrutiny. Steele Fortress calls the 5.9×/4.8× headline figures “NVIDIA Math” — an NVFP4 model on Blackwell against BF16 baselines on Hopper is hardware-confounded, and the 8K-input / 64K-output profile is decode-heavy in a way that flatters Ultra’s multi-token prediction heads ¹⁰.

What actually holds up: 1M context — and a jailbreak

The cleanest win is long context, though the exact number depends on who’s counting. NVIDIA’s paper reports 76.83% on RULER at 1M tokens; LLM-Stats’ independent leaderboard puts Ultra at 94.7%, with Qwen-3.5 (397B) trailing near 90% ¹¹. The gap likely reflects different RULER subtask weightings or evaluation harnesses — worth a footnote, but both numbers point the same direction: the hybrid Mamba-Attention stack genuinely flattens the context-rot curve that pure-transformer models hit beyond 128K.

Less flattering: NRLabs published a working policy bypass in which an “uncensored” system prompt overrides native safety controls to elicit malware generation ¹². NVIDIA shipping a separate Nemotron-3 Content Safety model is an implicit acknowledgment that Ultra’s own policy surface is thin — safety is a sidecar, not a property of the base model.

Net read

Ultra is a serious systems achievement — NVFP4 at scale, a Mamba-Attention hybrid that holds at 1M tokens, and a clean open release of checkpoints, recipes, and pretraining data. The marketing layer is where the qualifications stack up: throughput comparisons need a hardware-matched baseline, MOPD has a known ceiling on reasoning depth, LatentMoE routing remains an open problem at this scale, and the safety story depends on a second model.

---

Inclusion AI retrofits a 1T MoE with 7:1 linear attention

Source: hf-daily-papers · published 2026-06-12

TL;DR

• Inclusion AI retrofitted its 1T Ling-2.0 MoE with hybrid linear attention, preserving 20T tokens of prior pretraining.
• OpenRouter ranks Ring-2.6-1T #75 of 94 for overall intelligence, undercutting the report’s GPT-5.4 parity claim.
• KPop swaps IcePop’s importance-ratio clip for binary-KL, lifting SWE-bench Verified to 76.28%.
• Q4-quantized 1T checkpoints still need 600–700 GB of VRAM, restricting use to cloud or Mac Studio clusters.

The architecture move

The interesting engineering in the Ling/Ring 2.6 report isn’t a new model — it’s how Inclusion AI avoided training one. Rather than start from scratch, the team “transplanted” the attention stack of the existing 1T Ling-2.0 MoE: fusing QK-Norm into projection weights, warming up Lightning Attention layers, and gradually absorbing MLA, all while keeping the 20T-token pretrain investment intact. The result is a 7:1 ratio of linear (Lightning) to softmax (MLA) attention layers, aimed at 256K-context inference without quadratic blowup.

That ratio is also where the biggest open question sits. MiniMax — another lab betting on hybrid linear attention — has publicly reported that Lightning-Attention hybrids match Transformers on MMLU-style evals but show “clear deficits in complex, higher-order multi-hop reasoning at larger scales” ¹³. The 2.6 family inherits that risk, and Inclusion AI’s benchmark mix doesn’t lean heavily on the kind of deep composition where the gap would surface.

The benchmark story doesn’t fully hold

The headline numbers — Ling-2.6 reaching index score 34 at 16M output tokens, Ring-2.6 hitting 95.78% on AIME 2026 and 87.60% on PinchBench — are real but contested when checked independently. OpenRouter’s surfacing of Artificial Analysis data places Ring-2.6-1T at #33/94 for coding and #75/94 for overall intelligence, with an instruction-following score of 0.446 flagged as “below average” ¹⁴. Practitioners on r/LocalLLaMA have been blunter, accusing the family of being “benchmaxxed” after reporting that the model “fails at basic documentation instructions despite its high parameter count” ¹⁵.

That critique lines up uncomfortably well with a limitation the paper itself admits: the token-efficiency objective sometimes can’t tell “wasteful repetition” from “necessary factual elaboration.” Concise is a feature on math benchmarks and a bug on documentation tasks.

Where the paper is on firm ground: KPop

The cleanest contribution is the RL recipe. KPop replaces IcePop’s uniform importance-ratio constraint — which tends to over-mask low-probability tokens — with a binary-KL divergence between training and inference distributions, treating the vocabulary as a target-vs-rest partition ¹⁶. The 5.5-point SWE-bench Verified jump (70.8% → 76.28%) attributed to KPop is one of the better-isolated deltas in the report, and the algorithm is the part most likely to get picked up by other groups doing trillion-scale agentic RL. The supporting ASystem infrastructure — decoupling rollout collection from parameter updates — is the unglamorous bit that makes 1T-scale RL on tool-use trajectories actually tractable.

The deployment tax

Open weights, sure, but with asterisks. Community estimates put Q4-quantized 1T variants at 600–700 GB of memory, limiting practical use to cloud endpoints or Mac Studio clusters ¹⁷. The serving stack lags the release: BailingMoE support requires cloning a specific ling_2_5 SGLang branch or hand-patching vLLM ¹⁸. For most readers, this is a paper to learn from, not a checkpoint to run.

Round-ups

VibeThinker-3B challenges scaling laws on verifiable reasoning

Source: hf-daily-papers

Weibo’s 3B-parameter VibeThinker hits state-of-the-art Pass@1 on verifiable reasoning tasks by combining curriculum-based supervised fine-tuning, multi-domain reinforcement learning, and self-distillation. The recipe pushes a compact model past much larger peers and supports a ‘compression-coverage’ hypothesis about where small-model gains come from.

LLMs flip correct answers under plausible counterarguments

Source: hf-daily-papers

The WhoFlips benchmark challenges models on MMLU questions they answered correctly, then measures how often a counterargument makes them switch. MaxFlip rates vary widely across models and subjects, showing that headline accuracy hides large gaps in answer stability and self-attribution.

Sparse autoencoder features wobble across seeds but subspaces hold

Source: hf-daily-papers

Individual SAE features shift between training seeds, yet the unstable ones occupy reproducible low-rank subspaces that carry most reconstruction signal. Stable features dominate prediction-relevant signal, suggesting interpretability work should target cross-seed subspaces rather than fragile per-feature explanations.

WebStep grades web agents step-by-step, not just on final success

Source: hf-daily-papers

WebStep treats browsing tasks as a semantic MDP and tracks state changes at every step, exposing where agents stall, loop, or take wrong turns. The process-level view separates exploration reach from execution accuracy, surfacing skill gaps that terminal success rates hide.

CoDA-Bench exposes code agents’ weakness on data-heavy tasks

Source: hf-daily-papers

CoDA-Bench pairs code execution with data discovery to test whether agents can locate, join, and reason over real datasets. Frontier agents falter when the two skills must interleave, revealing that strong coding benchmarks overstate readiness for analyst-style workloads.

Tangram brings non-uniform KV-cache compression to vLLM serving

Source: hf-daily-papers

Tangram allocates KV-cache budgets per attention head rather than uniformly, then uses ragged paging and ahead-of-time load balancing to avoid fragmentation. Integrated with vLLM, the scheme lifts multi-turn throughput and cuts decode latency while keeping prefill time intact.

TokenPilot trims long-horizon agent context without breaking caches

Source: hf-daily-papers

TokenPilot manages LLM agent context at two granularities: Ingestion-Aware Compaction stabilizes prompt prefixes for cache reuse, while Lifecycle-Aware Eviction drops segments by residual utility. The combination lowers token footprints in long sessions without sacrificing prompt-cache continuity across turns.

Footnotes

1. TrendForce — https://www.trendforce.com/news/2026/04/14/news-china-reportedly-closes-ai-performance-gap-with-u-s-stanford-report-says-anthropic-leads-by-just-2-7/

   the performance gap between the leading U.S. and Chinese AI models has narrowed to just 2.7%, with Anthropic’s Claude Opus 4.6 leading at Elo 1,503, followed closely by ByteDance’s Dola-Seed-2.0-Preview at 1,464

   ↩

2. R&D World — https://www.rdworldonline.com/stanford-hai-2026-ai-index-ai-posts-gains-in-science-and-medicine-while-often-struggling-to-read-a-clock/

   Frontier models outperform humans on ChemBench yet score below 20% on astrophysics replication tasks, and on ClockBench top-tier models fail to read analog clocks roughly 50% of the time

   ↩

3. GIS Reports Online — https://www.gisreportsonline.com/r/ai-sovereignty/

   for most nations, true AI sovereignty is an ‘illusion’; they should instead pursue ‘AI resilience’ built on modular, interoperable architectures that allow optionality rather than total ownership of the stack

   ↩

4. Epoch AI — https://epoch.ai/data-insights/grok-4-training-resources

   Training Grok-4 required an estimated 310 GWh of electricity and emitted roughly 140,000–150,000 tons of CO2, with cooling demands of 750 million to 1.2 billion liters of water

   ↩

5. Startup Fortune — https://startupfortune.com/stanfords-2026-ai-index-confirms-the-enterprise-window-is-closing-faster-than-most-founders-think/

   78% of executives admit they could not pass an independent AI audit, despite 88% of firms having adopted AI in at least one function and fewer than 10% successfully scaling it with robust controls

   ↩

6. The Median (Substack) — https://dcthemedian.substack.com/p/20-takeaways-from-stanfords-2026

   Employment for entry-level software developers aged 22–25 plummeted by nearly 20% since 2024, even as senior-level hiring remained stable

   ↩

7. Medium (AI Guys) — NVFP4 analysis — https://medium.com/aiguys/nvidia-proved-4-bit-training-works-at-real-scale-not-just-inference-7a1472287d4a

   4-bit precision cannot be used in isolation; it requires a recipe that includes Random Hadamard Transforms to redistribute activation outliers and selective high-precision layers — typically keeping the first and last 10–15% of blocks in BF16.

   ↩

8. Microsoft AI — MAI-Base-1 technical report — https://microsoft.ai/pdf/mai-thinking-1.pdf

   Microsoft’s MAI-Base-1, which utilizes LatentMoE, reported significant training spikes on coding datasets, specifically correlating with high expert imbalance even under ‘dropless’ routing.

   ↩

9. Interconnects (Nathan Lambert) — Frontier post-training review — https://www.interconnects.ai/p/frontier-post-training-recipe-review

   Because the process is on-policy, it can only amplify reasoning paths the student is already capable of producing; it cannot transfer high-level logic that falls outside the student’s sampling distribution. Consequently, while Ultra leads in agentic and long-context benchmarks, it trails behind competitors like DeepSeek-V4 in hard reasoning tasks like HLE and GPQA.

   ↩

10. Steele Fortress — Hidden costs and security pitfalls — https://steelefortress.com/fortress-feed/nvidia-nemotron-3-the-hidden-costs-and-security-pitfall

    Comparing an NVFP4-optimized model on next-gen Blackwell chips to a BF16 model on previous-gen chips creates an ‘apples-to-oranges’ baseline… some experts refer to the marketing of these gains as ‘NVIDIA Math’.

    ↩

11. LLM-Stats RULER leaderboard — https://llm-stats.com/benchmarks/ruler

    Nemotron-3 Ultra (550B) leads the RULER 1M-context leaderboard with a score of 94.7%… Qwen-3.5 (397B) follows with approximately 90%.

    ↩

12. NRLabs — Bypassing Nemotron V3 policy protections — https://www.nrlabs.com/blog-posts/bypassing-nemotron-v3-policy-protections

    A critical policy bypass vulnerability where ‘uncensored’ system prompts can override native safety controls, allowing the generation of malware.

    ↩

13. Moonlight review of the technical report — https://www.themoonlight.io/en/review/ling-and-ring-26-technical-report-efficient-and-instant-agentic-intelligence-at-trillion-parameter-scale

    MiniMax reported that while their hybrid Lightning Attention models matched Transformers on standard benchmarks like MMLU, they showed clear deficits in complex, higher-order multi-hop reasoning at larger scales

    ↩

14. OpenRouter model page (Artificial Analysis index) — https://openrouter.ai/inclusionai/ling-2.6-1t

    instruction-following score of 0.446 is ‘below average’… ranks #33 out of 94 models specifically for coding and #75 for overall intelligence

    ↩

15. r/LocalLLaMA discussion thread on Ling/Ring 2.6 — https://www.reddit.com/r/LocalLLaMA/comments/1ucih9e/ling_and_ring_26_technical_report_efficient_and/

    users have reported that the model fails at basic documentation instructions despite its high parameter count, leading to accusations that the model may be ‘benchmaxxed’

    ↩

16. Moonlight review — KPop vs IcePop analysis — https://www.themoonlight.io/en/review/ling-and-ring-26-technical-report-efficient-and-instant-agentic-intelligence-at-trillion-parameter-scale

    IcePop’s constant-ratio method tends to over-mask low-probability tokens… KPop views the vocabulary as a two-event partition—the target token versus all others—and computes the BKL between the training and inference distributions

    ↩

17. r/LocalLLaMA hardware thread — https://www.reddit.com/r/LocalLLaMA/comments/1sz59l4/inclusionailing261t_hugging_face/

    Even with high-level quantization (e.g., Q4), the 1T variants are estimated to require between 600GB and 700GB of RAM/VRAM, making them inaccessible to all but the most high-end multi-GPU or Mac Studio clusters

    ↩

18. inclusionAI/Ling-V2 GitHub deployment notes — https://github.com/inclusionAI/Ling-V2

    Developers are currently required to clone specific branches (e.g., ling_2_5 in SGLang) or apply manual patches to vLLM to support the BailingMoE architecture

    ↩