MIT’s modules vanish in GPT-2, Dion3 needs Hopper, RLVR trades tasks

TL;DR

MIT recovers 4 cognitive domains from LLM neuron clusters at ARI 0.78 unsupervised.
Targeted ablations open a 10.3× accuracy gap between in-domain and out-of-domain neurons.
Dion3 cuts Muon’s optimizer step to 4× AdamW on Hopper/Blackwell CuteDSL kernels.
Math-RLVR shrinks Qwen3-8B’s solvable AIME pool from 33.6% to 1.6% in 20 steps.
AutoResearchEval finds frontier agents strong on engineering, weak on methodological novelty.

Today’s AI research drops don’t share a topic, but each headline result comes attached to a condition the number itself doesn’t advertise. MIT’s modularity finding recovers four brain-like cognitive domains from LLM neuron overlaps — but the same clustering yields nothing in GPT-2, suggesting the structure is a byproduct of successful optimization rather than of the transformer architecture. Dion3’s 6× overhead cut against Muon materializes only on Hopper/Blackwell with the right CuteDSL kernels; the algorithmic tricks alone leave most of the win on the silicon. And Wei’s RLVR sweep finds a math-training gain that arrives already spending best@32 diversity on sibling tasks, with KL, SFT priors, and on-policy distillation all failing to preserve the joint capability.

MIT: frontier LLMs self-organize into brain-like modules

TL;DR

Han et al. (MIT) recover four cognitive domains from LLM neuron-overlap clustering at ARI 0.78, unsupervised.
Targeted ablations dropped in-domain accuracy 25.9% vs. 2.5% elsewhere — a 10.3× gap.
Modularity is absent in GPT-2, suggesting specialization tracks successful optimization, not the transformer architecture itself.
Social-reasoning ablations fell below significance, the weakest of the four proposed modules.

What the paper actually shows

Pengrui Han and colleagues ran attribution patching over MLP neurons in six instruction-tuned models — Mistral-Small-24B up through Mistral-Large-123B, plus Qwen2.5, OLMo-2, and Llama-3.1-70B — across 46 tasks built as minimal pairs. For each task they isolated the top 0.1% of causally important neurons, then measured Jaccard overlap between task sets. Tasks drawn from the same cognitive domain shared roughly four times as many critical neurons as tasks across domains, and unsupervised clustering on the overlap matrix recovered the four predefined domains at ARI 0.78. Counterfactual ablation confirmed the split: silencing “language” neurons produced grammatical errors with intact reasoning; silencing “physics” neurons produced fluent but physically impossible answers. Language-selective units concentrated in early layers, reasoning units in the middle and late stack.

The paper’s most editorially loaded claim — that modularity is “a fundamental property of intelligent systems” — leans hard on a control: GPT-2, which fails the reasoning tasks, shows no such structure ¹. Modularity, on this reading, is what emergence looks like from the inside.

What’s genuinely new vs. what isn’t

The language-module result is not new. AlKhamissi, Tuckute, and Fedorenko (a co-author here) already localized language-selective units in 18 LLMs and showed ablation collapses linguistic behavior while sparing reasoning ². That same group also reported that Theory-of-Mind and Multiple-Demand networks do not dissociate as cleanly as language ³ — a caveat Han et al. reproduce when they concede social-reasoning ablations landed in the right direction but below their significance threshold. The novel contribution is scope: four domains, six frontier models, and the clean unsupervised recovery.

Two reasons to hold the “convergence” framing loosely

First, ablation as evidence has known failure modes. The Hydra Effect — downstream components silently compensating for silenced neurons — plus polysemantic MLP units that encode multiple unrelated features together, mean single-neuron ablation can produce interpretability illusions in both directions ⁴. A 25.9% in-domain drop is real, but it’s neither a ceiling nor a clean readout of functional load.

Second, alphaXiv commenters note the paper documents emergence without showing that modular organization helps — there’s no head-to-head against a hypothetical monolithic baseline on the same tasks ⁵. “Convergent solution to intelligence” is a strong reading of a correlation with scale.

The brain analogy also strains at the algorithm level. A 2026 NYU ECoG study found humans predict upcoming words by grouping them into hierarchical grammatical constituents, while LLMs predict from flat local context ⁶. Neuron clusters that look cortex-shaped can still sit atop a fundamentally different prediction rule — and the domain split may just reflect the statistical seams of training curricula that separate math, code, physics, and social prose.

Takeaway

Read this as a solid mechanistic-interpretability result for the language module, a plausible extension to formal reasoning, and an open question for social cognition and for whether modularity is doing any computational work at all.

Dion3 cuts Muon’s optimizer step from 26× to 4× AdamW

TL;DR

Dion3 cuts Muon’s optimizer step from 26× AdamW to 4× on a 7B model across four GH200s.
Gram Newton-Schulz, symmetric-GEMM kernels, and sparse row updates together deliver the 6× overhead reduction.
Tri Dao’s companion gram-newton-schulz release independently clocks 40–50% orthogonalization runtime cuts at matched perplexity.
Speedups require Hopper/Blackwell CuteDSL kernels — the algorithmic tricks alone leave most of the win on the table.

Where the 6× actually comes from

Muon converges better than AdamW but pays for it with an $O(n^3)$ Newton-Schulz orthogonalization on every weight matrix, plus heavy all-reduce traffic under FSDP. Dion3 attacks all three costs at once rather than optimizing any single layer to death.

The algorithmic move is Gram Newton-Schulz: instead of iterating on the $n \times m$ weight matrix directly, form the $n \times n$ Gram matrix $XX^\top$ once, iterate the inverse-square-root there, and project back at the end. For a Transformer MLP with aspect ratio $\alpha=4$ and $T=5$ iterations, that’s 55% fewer FLOPs than standard Newton-Schulz — 68% once you add symmetric kernels. Tri Dao’s independent write-up confirms the kernel-level number: 40–50% off the orthogonalization step, up to 2× on the total optimizer step, at matched perplexity ⁷.

The CuteDSL symmetric-GEMM kernels exploit the fact that $XX^\top$ is symmetric: schedule only the lower triangle, mirror-write to the upper triangle in the epilogue, halve the FLOPs. They hit ~2× over cuBLAS on both Hopper and Blackwell.

The sparse update rule is the least obvious ingredient. Each step, Dion3 picks the top-$f$ fraction of momentum rows by $\ell_1$ norm ($f=1/4$ or $1/8$), orthogonalizes only that submatrix, and updates only those weights. Unselected rows keep their momentum for the next step (error feedback), so information isn’t lost. In practice this doesn’t hurt convergence and sometimes helps as a regularizer.

What the broader Muon conversation adds

Dion3 sits on top of the original Dion paper’s distributed insight: use amortized power iteration to orthonormalize a low-rank subspace without ever reconstructing a full parameter matrix on one device ⁸. That’s what makes FSDP/TP work at all here.

But Muon-family optimizers are not settled tech. Hugging Face’s “Scaling is not plug-and-play” documents that naive Muon at multi-billion scale routinely produces “paralysis” (vanishing updates) or “explosions” (loss spikes, NaNs), and stabilization requires weight decay plus per-shape update-scale corrections that early variants lacked ⁹. Dion3 inherits those requirements — it doesn’t claim to solve them.

NorMuon offers a sharper critique from a different direction: Muon flattens condition numbers but leaves neuron norms uneven, and adding Adam-style second moments per neuron beats vanilla Muon by 11.31 pp on 1.1B pretraining ¹⁰. Dion3’s row-selection heuristic is thematically adjacent but isn’t benchmarked head-to-head against NorMuon.

The complexity tax is real

Independent explainers are blunt about the integration bar: the headline numbers are gated on custom CuteDSL kernels for Hopper/Blackwell and a bespoke “megabatching” communication schedule, which makes Dion3 harder to drop into legacy pipelines than AdamW ¹¹. For square matrices ($\alpha=1$), Gram Newton-Schulz is FLOP-identical to the standard version and can be slower in wall-clock because of extra kernel launches — hence the library’s automatic fallback ¹². The float16-over-bfloat16 recommendation for the Gram matrix, plus a mid-iteration restart to purge spurious negative eigenvalues, are both concessions that the Gram formulation is more precision-sensitive than the original ⁷¹¹.

Read the 6× as real but conditional: on aspect ratio, on hardware generation, and on a team willing to absorb a custom optimizer stack. For teams already committed to Muon at scale, it’s a clear upgrade. For everyone else, it’s evidence that the post-AdamW design space is very much still moving.

RLVR gains on one task destroy searchability on the next

TL;DR

Math-RLVR lifts IFEval pass@1 by 6.5% on Qwen3-8B while collapsing best@32 by 9.8%.
IF→Math training shrinks the sometimes-solvable AIME prompt pool from 33.6% to 1.6% in 20 steps.
First-token JS divergence spikes 106× vs. interior tokens, localizing the damage.
KL, SFT priors, and on-policy distillation all fail to preserve joint capability in Wei’s sweep.

The phenomenon has a name now

Wei et al. call it verifier-induced support reshaping: on-policy RL with verifiable rewards (RLVR) optimizes a task by concentrating probability mass on the trajectories that already work, and in doing so makes the trajectories some future task would need too rare to sample. It is not catastrophic forgetting — the old task still scores well. It is a forward-looking collapse of what the paper calls “effective rewardable support,” measured as best@k under a fixed rollout budget.

The community has been circling this for a year. Yue et al. already showed that base models beat their RL-tuned descendants at pass@256 or pass@1024 on the hardest problems, because RLVR reranks existing solutions rather than discovering new ones ¹³. Wei’s contribution is reframing that gap as a trainability constraint: if the rewardable trajectories vanish from your samples, on-policy RL on the next task simply stalls.

Math and instruction-following interfere asymmetrically

Training Qwen3-8B on MATH with GRPO raises IFEval pass@1 by 6.5% but drops best@32 by 9.8% — some IF prompts get consistently easier, others become unreachable. The reverse is worse. IF-RLVR pushes the model from “Deliberative Reasoning Initiation” (step-by-step openings) to “Direct Answer Initiation,” and DAI openings are barren for math discovery. When Math-RLVR is then applied on top, the mixed-difficulty AIME pool — prompts the base model solved sometimes — collapses from 33.6% to 1.6% within 20 optimizer steps.

Independent benchmarks corroborate the asymmetry. MathIF found that scaling reasoning training degrades user-constraint adherence ¹⁴. ReasonIF shows frontier models fail to follow reasoning-trace instructions more than 75% of the time, with IFS scores below 0.25 ¹⁵. The joint capability A∧B really is much harder than A+B, and it is not a Qwen quirk.

The damage is localized to the first few tokens

The most surprising diagnostic: JS divergence between the RLVR policy and the base model peaks at the first token, at up to 106.7× the interior-token divergence. The paper’s causal intervention is the payoff — force an IF-trained model to open with a base-model token or a reasoning-style prefix, and best@32 math scores recover with no additional training. The knowledge is still in the weights; RLVR just closed the entry route.

This lines up with independent entropy-flow work arguing that GRPO collapse concentrates at roughly 5% of structurally critical decision points ¹⁶. Different labs are localizing the damage to a small set of high-leverage tokens, which points at targeted-token KL regularization rather than the global KL knob Wei tested.

Mitigations: Wei is pessimistic, others aren’t

Wei sweeps KL penalties, SFT priors, and on-policy distillation and reports all three either fail or require task-specific tuning that won’t generalize. The broader literature is less resigned. Robust Policy Optimization frames the fix as optimizing over a KL-bounded neighborhood and claims stable math accuracy under subsequent IF adaptation ¹⁷. And practitioner reports on GLM-5.2 say the team abandoned GRPO for PPO to get “qualitative improvements in training controllability and generalization” ¹⁸ — implicating GRPO’s critic-free, group-normalized design rather than on-policy RLVR as a whole.

The phenomenon is real and now well-named. Whether it is intrinsic to on-policy verifiable-reward RL or an artifact of the specific algorithm most labs happen to use is the fight worth watching.

Round-ups

Reasoning training amplifies self-correction but not calibration

Deliberative behaviors like self-correction and hypothesis testing get boosted most by reasoning training, yet correctness-linked traits such as confidence calibration barely improve. The Behavioral Lift analysis exposes a gap between what thinking models perform and what actually predicts right answers across vision-language tasks.

Frontier research agents optimize well but lack novelty

Systematic evaluation across long-horizon AI R&D tasks finds autonomous agents strong at engineering optimization yet unstable in solution framing, weak on methodological novelty, and inconsistent at reusing prior experience. AutoResearchEval scores execution and feedback control separately from final task metrics.

Intern-S2-Mobius splits memory from reasoning for faster inference

Mobius-v0 routes global knowledge into FFN memory modules and iterative reasoning into self-attention, letting Intern-S2-Mobius match baseline performance with less training data and quicker inference. The decoupled design targets compositional reasoning without paying the usual dense-model compute tax.

Optimal domain-data repetition grows mildly with LLM scale

When model size and token budgets scale proportionally, the best number of passes over high-quality domain data rises only slowly with scale. Repetition count tracks domain validation loss more tightly than the volume of unique data, reshaping pretraining mix decisions.

Clinical multi-agent LLMs adopt socially plausible shortcuts

Committees of language-model agents making clinical decisions fall for shortcuts that sound socially reasonable rather than isolated spurious cues. Only an independent referee agent reliably catches the cascade, suggesting benchmark-gaming defenses need external oversight instead of peer deliberation among the deciding agents.

Second Thought runs LLM agent reasoning during idle waits

During the gaps where ReAct agents wait for tool observations, Second Thought spawns auxiliary reasoning branches in parallel with the main decode. The training-free scheme cuts sequential steps and turn counts on agentic benchmarks while keeping Pass@1 accuracy intact.

Marionette models games as 3D state, renderer, and diffusion

Marionette factors interactive game generation into an autoregressive 3D articulated world-state predictor, a zero-parameter geometry renderer, and a video-diffusion appearance model. The split enables direct state-level control and long-horizon consistency repair, improving FVD over end-to-end video models.

AI Weekly — ‘LLMs mirror the brain’s modular neuron layout, study finds’ — https://aiweekly.co/alerts/llms-mirror-the-brains-modular-neuron-layout-study-finds

The paper’s framing is ‘unusually direct’ in asking whether modularity is a biological accident or a convergent requirement of intelligence; modularity was absent in GPT-2, appearing only once models reach reasoning competence.

↩
AlKhamissi & Tuckute — ‘The LLM Language Network’ project page — https://bkhmsi.github.io/llm-language-network/

Applying functional localizers to 18 LLMs identified units selective for language over math or code; ablating these units causes a dramatic collapse in language modeling while leaving reasoning intact.

↩
Semantic Scholar — AlKhamissi, Tuckute et al., ‘The LLM Language Network’ (NAACL 2025 precursor) — https://www.semanticscholar.org/paper/The-LLM-Language-Network%3A-A-Neuroscientific-for-AlKhamissi-Tuckute/3469a28aacad3fddb22984178ed6edb034a76886

Language-selective units emerge in LLMs and are causally necessary for linguistic behavior, but analogous ‘Theory of Mind’ and ‘Multiple Demand’ networks are far less cleanly separable than the language module.

↩
ResearchGate — ‘Empirical Limits of Neuron-Level Ablation for AI Safety’ — https://www.researchgate.net/publication/405134136_Empirical_Limits_of_Neuron-Level_Ablation_for_AI_Safety

Ablation studies can ‘lie’ because of the Hydra Effect / self-repair: downstream components compensate for silenced neurons, and polysemantic MLP units encode multiple unrelated features, making ‘surgical’ single-neuron ablation an interpretability illusion.

↩
alphaXiv discussion of Han et al. (2608.13567) — https://www.alphaxiv.org/abs/2608.13567

The study establishes that modularity emerges, but does not demonstrate that modular organization improves raw task accuracy compared to monolithic processing — leaving the functional payoff of specialization open.

↩
NYU News — ‘Does the brain work like an LLM in predicting words?’ (2026) — https://www.nyu.edu/about/news-publications/news/2026/april/does-the-brain-work-like-an-llm-in-predicting-words—new-study-s.html

LLMs predict words from immediate local context while the human brain predicts by grouping words into hierarchical grammatical constituents — a structural mismatch even where surface behavior converges.

↩
Tri Dao blog — ‘Gram Newton-Schulz’ — https://tridao.me/blog/2026/gram-newton-schulz/

Gram Newton-Schulz provides a 40–50% reduction in runtime for the orthogonalization step compared to the standard Newton-Schulz routine, yielding up to a 2x speedup on the total optimizer step while staying within 0.01 validation perplexity of the baseline.

↩ ↩²
Microsoft Research — original Dion paper page — https://www.microsoft.com/en-us/research/publication/dion-distributed-orthonormalized-updates/

Dion uses amortized power iteration to orthonormalize only a low-rank subspace of the momentum matrix, computing updates without ever reconstructing a full parameter matrix on a single device — retaining synchronous semantics under FSDP/TP.

↩
Hugging Face blog — ‘Scaling is not plug-and-play’ — https://huggingface.co/blog/bird-of-paradise/scaling-is-not-plug-and-play

Naively applying Muon to multi-billion-parameter models often results in ‘paralysis’ (vanishing updates) or ‘explosions’ (loss spikes and NaNs); stability requires weight decay and per-parameter update-scale corrections that early Muon variants lacked.

↩
NorMuon paper (arXiv 2504.05295) — https://arxiv.org/html/2504.05295v2

Original Muon effectively reduces condition numbers but leads to non-uniform neuron norms; NorMuon adds neuron-wise second-moment normalization and outperforms Muon by 11.31 percentage points on 1.1B-parameter pretraining.

↩
Emergent Mind — Dion3 explainer — https://www.emergentmind.com/papers/2608.11612

The complexity tax is real: Dion3’s headline speedups depend on custom CuTeDSL kernels and ‘megabatching’ communication patterns, and some practitioners note this makes it harder to integrate into legacy pipelines than plain AdamW.

↩ ↩²
alphaXiv — Dion3 discussion page — https://www.alphaxiv.org/abs/2608.11612

For square matrices (α=1) Gram Newton-Schulz is FLOP-identical to standard Newton-Schulz and can be slightly slower in wall-clock due to extra kernel launches; the library falls back to standard NS with symmetric kernels in this regime.

↩
Yue et al., ‘Does RLVR Really Incentivize Reasoning Capacity?’ (arXiv 2606.15455) — https://arxiv.org/abs/2606.15455

base models often maintain a higher pass@256 or pass@1024… for the hardest problems requiring novel strategies, the RL-tuned model becomes less likely to stumble upon the correct answer than its original base version

↩
Fu et al., ‘Scaling Reasoning, Losing Control’ (MathIF, arXiv 2505.14810) — https://arxiv.org/abs/2505.14810

as models are scaled or fine-tuned specifically for reasoning — such as through distilled long chains-of-thought — their ability to follow user-specified constraints often degrades

↩
ReasonIF GitHub (Kwon et al.) — https://github.com/ykwon0407/reasonIF

many ‘state-of-the-art’ models fail to follow reasoning instructions more than 75% of the time… highest Instruction Following Scores (IFS) often remain below 0.25

↩
‘Battle for Entropy: RL Algorithms & LLMs’ (gopubby practitioner blog) — https://ai.gopubby.com/battle-for-entropy-rl-algorithms-llms-9ea4f9acfe4f

collapse is not uniform but is triggered by ‘premature overconfidence’ at a small subset (~5%) of structurally critical decision points

↩
Robust Policy Optimization (FRPO) — ResearchGate 400622099 — https://www.researchgate.net/publication/400622099_Robust_Policy_Optimization_to_Prevent_Catastrophic_Forgetting

optimizing rewards across a ‘KL-bounded neighborhood’… ensures that a model’s math accuracy remains stable even when it is subsequently adapted for new downstream instruction sets

↩
Wilson Wu, ‘PPO vs GRPO’ (practitioner blog) — https://wilsonwu.me/en/blog/2026/ppo-vs-grpo/

GLM-5.2 switched from GRPO back to PPO to achieve ‘qualitative improvements’ in training controllability and generalization

↩

MIT's modules vanish in GPT-2, Dion3 needs Hopper, RLVR trades tasks

MIT’s modules vanish in GPT-2, Dion3 needs Hopper, RLVR trades tasks

TL;DR

MIT: frontier LLMs self-organize into brain-like modules

TL;DR

What the paper actually shows

What’s genuinely new vs. what isn’t

Two reasons to hold the “convergence” framing loosely

Takeaway

Dion3 cuts Muon’s optimizer step from 26× to 4× AdamW

TL;DR

Where the 6× actually comes from

What the broader Muon conversation adds

The complexity tax is real

RLVR gains on one task destroy searchability on the next

TL;DR

The phenomenon has a name now

Math and instruction-following interfere asymmetrically

The damage is localized to the first few tokens

Mitigations: Wei is pessimistic, others aren’t

Round-ups

Reasoning training amplifies self-correction but not calibration

Frontier research agents optimize well but lack novelty

Intern-S2-Mobius splits memory from reasoning for faster inference

Optimal domain-data repetition grows mildly with LLM scale

Clinical multi-agent LLMs adopt socially plausible shortcuts

Second Thought runs LLM agent reasoning during idle waits

Marionette models games as 3D state, renderer, and diffusion

Jack Sun, writing.

MIT’s modules vanish in GPT-2, Dion3 needs Hopper, RLVR trades tasks

TL;DR

MIT: frontier LLMs self-organize into brain-like modules

TL;DR

What the paper actually shows

What’s genuinely new vs. what isn’t

Two reasons to hold the “convergence” framing loosely

Takeaway

Dion3 cuts Muon’s optimizer step from 26× to 4× AdamW

TL;DR

Where the 6× actually comes from

What the broader Muon conversation adds

The complexity tax is real

RLVR gains on one task destroy searchability on the next

TL;DR

The phenomenon has a name now

Math and instruction-following interfere asymmetrically

The damage is localized to the first few tokens

Mitigations: Wei is pessimistic, others aren’t

Round-ups

Reasoning training amplifies self-correction but not calibration

Frontier research agents optimize well but lack novelty

Intern-S2-Mobius splits memory from reasoning for faster inference

Optimal domain-data repetition grows mildly with LLM scale

Clinical multi-agent LLMs adopt socially plausible shortcuts

Second Thought runs LLM agent reasoning during idle waits

Marionette models games as 3D state, renderer, and diffusion

Footnotes

Jack Sun, writing.