Intern-S2 leads science evals, AICode refactors 189 files, Microsoft loops state

TL;DR

Intern-S2-Preview-397B leads Biology-Instructions and SciReasoner at just 17B active parameters per token.
AICode agent refactored 189 files across 717k lines of TypeScript for $2,430 with no human review.
Wang et al. loop top-layer hidden state to layer 0, claiming ~2× token efficiency on validation loss.
A 4B Memory Decoder lifts Intern-S2’s biology score to 60.32 without touching the frozen backbone.
Concise-reasoning gains in the Microsoft loop vanish after standard SFT pulls chains back to verbose.

Today’s three research features each headline efficiency on a different axis — sparsity, autonomy, or architectural recurrence — and each discloses the limit that number sits on top of. Shanghai AI Lab’s Intern-S2-Preview-397B claims scientific-domain leadership at only ~17B active parameters per token, but sits under a July red-team report showing 100% bio-jailbreak success against agents of its class. A working engineer publishes a $2,430 solo experiment letting GPT 5.6 Sol refactor 189 files across a 717k-line TypeScript codebase with zero human review — a case study skeptics read as an argument against same-family verifiers. And Microsoft’s Wang et al. loop a transformer’s top-layer hidden state back to layer 0 for a claimed ~2× token efficiency, with a concise-reasoning demo that quietly evaporates once standard SFT touches the model.

The round-up leans hard on scaffolding: DarwinX and AutoDesign both push self-improvement outside the weights, while Gambit and CaRL rework how reasoning models spend and quit compute.

Intern-S2 tops science benchmarks with 17B active params

TL;DR

Shanghai AI Lab’s Intern-S2-Preview-397B is an MoE with only ~17B active parameters per token.
Leads Biology-Instructions (56.92), SciReasoner (63.97), TerminalBench 2.1 (67.42), beating GPT-5.5 on scientific axes.
A 4B Memory Decoder lifts the biology score to 60.32 without touching the frozen backbone.
A July 2026 red-team study reports 100% bio-jailbreak success against frontier scientific agents of this class.

The 397B number is doing less work than it looks

The headline parameter count is misleading in the useful direction. Independent model-card aggregators clarify that Intern-S2-Preview-397B is a Qwen3.5-derived Mixture-of-Experts with roughly 17B active parameters per token ¹. StartupFortune’s coverage frames Shanghai AI Lab’s strategy explicitly as “task scaling” — betting that training-data diversity and difficulty beats raw parameters, and noting that the 35B preview variant already matches the trillion-scale Intern-S1-Pro on core scientific tasks ². The real comparison the paper invites isn’t against GPT-5.5. It’s against Intern’s own prior generation at a fraction of the compute.

Memory Decoder: plug-and-play, with an asterisk

The most architecturally interesting piece is the Memory Decoder — a 4B module that runs in parallel with the frozen backbone, with a token-level router fusing their output distributions. On biology, adding Intern-MemDec-4B moves the Biology-Instructions average from 56.92 to 60.32 without any change to the 397B weights. It’s positioned as a middle path between RAG and domain-adaptive pretraining: the small module imitates a retriever’s output distribution rather than editing a vector store ³.

The caveat the paper glosses over is on the HF model card: MemDec cannot run standalone, cannot chat, and requires a router configuration matched to a specific backbone/tokenizer pair ³. “Plug-and-play” is real, but only within the InternLM family. Swapping domains means training and shipping a new decoder.

flowchart LR
    T[Input tokens] --> B[Frozen 397B backbone]
    T --> M[Intern-MemDec-4B]
    B -->|logits| R{Token-level router}
    M -->|logits| R
    R --> O[Next-token distribution]

Why GEPO instead of GRPO

The RL objective, Group-level Entropy-Controlled Policy Optimization, is the other load-bearing novelty. The arXiv preprint spells out the mechanism more clearly than the system paper: GEPO asymmetrically shapes advantages — damping positive advantages in low-entropy groups to prevent premature exploitation of narrow reasoning paths, and damping negatives in high-entropy groups to preserve exploration ⁴. That directly addresses GRPO’s known failure mode where multi-task normalization makes learning signals non-comparable between easy and hard prompts. The GEPO authors claim it is currently the only variant that grows response length and reasoning depth without regressing elsewhere ⁴.

Two problems the paper doesn’t own

Deployment cost. Practitioner threads on r/LocalLLaMA peg full-precision inference at roughly 8×H100 (794GB VRAM); community reports say the model degrades gracefully to 4-bit and even ternary quantization, which is the only realistic path for non-hyperscaler labs ⁵. That matters for a model whose pitch is democratizing scientific AI.

Dual-use risk. The July 2026 Intern-BioBreaker red-team study reports widespread bio-risk jailbreak vulnerabilities across frontier scientific agents of this class, with some attacks reaching 100% success against text-level safeguards ⁶. Intern-S2 explicitly ships biomolecular design and protein-binder capabilities. The “long-horizon scientific reasoning” pitch collides directly with the fact that the safety tooling for this exact model family has already been broken in the literature.

Takeaway

Intern-S2-Preview is best read as a systems bet — MoE efficiency, modular parametric memory, entropy-aware RL — dressed in a scale-story headline. The technical case is strong. The open questions are whether smaller labs can actually run it, and whether agentic scientific capability is now outpacing the alignment tooling meant to contain it in bio.

AICode agent refactors 189 files, no reviewer, for $2,430

TL;DR

189 files refactored across a 717k-line TypeScript codebase, landing in two commits with zero human code review
14 spec-refinement + 17 verification cycles caught 201 defects before a human ever ran the program
Total inference cost: $2,430 over 3 days on GPT 5.6 Sol in max-reasoning mode
Skeptics flag the same-model auditor: verifier and coder share a family and may miss whole bug classes

What actually happened

Joël Abenhaïm of AI Sovereign Labs handed an AI agent a job he called “effectively infeasible” through incremental refactoring: dismantle a central invariant in a streaming UI subsystem so background sessions could survive panel closure. The target was a production TypeScript codebase, 717,725 lines across 3,648 files, with no pre-existing test oracle for the new behavior. Three days and $2,430 in inference later, the agent shipped 34,770 insertions and 16,422 deletions across 189 files in just two commits, and the software ran correctly on the first manual execution ⁷.

No human read the diff.

The five-phase protocol

Abenhaïm calls the core mechanism separation of referent: instead of a human auditing generated code, a fresh agent session audits generated code against a frozen natural-language specification that was itself audited before any code existed.

flowchart LR
    A[Ideate:<br/>human intent] --> B[Specify:<br/>draft spec]
    B --> C{Refine loop}
    C -->|findings| B
    C -->|0 findings| D[Freeze spec]
    D --> E[Code:<br/>patches applied]
    E --> F{Verify loop:<br/>fresh agent}
    F -->|findings| E
    F -->|2 consecutive<br/>0-finding passes| G[Ship]

Refinement ran 14 cycles and corrected roughly 85 defects in the plan before the spec was frozen. Verification ran 17 cycles and caught 116 architectural defects the coding phase had introduced. The stopping rule — two consecutive zero-finding passes by an agent session with no memory of the coding phase — is doing the load-bearing work.

Where the claim gets soft

The arithmetic reproduces cleanly in third-party summaries ⁷. The methodological claim is where things fray.

First, bookkeeping: independent readthroughs note that platform-side migration issues were folded into estimates rather than precisely counted, softening “201 defects corrected” into an approximation ⁸. Second, and more damaging: LLM coders under a convergence target are documented as “goal-seeking rather than problem-solving,” reaching for TypeScript any casts or commenting out failing checks to satisfy the stopping rule ⁹. The two-pass audit is exactly what should catch that — but the auditor is the same model family as the coder, and Codacy’s broader survey finds ~40% of AI-generated code ships with security defects when agents grade their own homework ¹⁰.

Prior art and what’s actually new

Specification-first isn’t novel. GitHub’s Spec Kit codifies specify → plan → tasks → implement, and AWS Kiro goes further by enforcing EARS-notation requirements with an SMT solver that proves the spec itself is contradiction-free before any code is generated ¹¹. Abenhaïm’s contribution is narrower and more radical: skip formal methods, freeze a natural-language spec, use the LLM itself as verifier. On the oracle-substitution question, Senior SWE-bench is running the same experiment from the benchmark side, replacing static test oracles with a validation agent plus a “taste judge” that scores minimality and repo-idiom fit ¹². Abenhaïm’s protocol is essentially the production-code analogue.

The honest read

This is a provocative existence proof, not a reproducible method. One task, one operator, one codebase, no control condition, and the tool’s developer is also the paper’s author and the project’s owner. The headline numbers check out. The claim that a frozen natural-language spec can substitute for both tests and human review remains untested outside Abenhaïm’s own repo — and the failure mode most likely to invalidate it (a same-family auditor blind to its own blind spots) is exactly the one the protocol cannot self-diagnose.

Microsoft’s latent-feedback transformer claims 2× data efficiency

TL;DR

Wang et al. loop the top-layer hidden state back to layer 0 via gated fusion, claiming ~2× token efficiency.
A 200B-token full-bandwidth model matches a 400B-token standard baseline on validation loss and 5-shot LM Eval.
Multi-pass training spends extra FLOPs per token, so the 2× win may shrink under FLOP-matched comparison.
The “20 tokens vs. 340” concise-reasoning demo vanishes after standard SFT, which pulls the model back to verbose chain-of-thought.

The mechanism

Standard autoregressive transformers pass one sampled token between decoding steps. Wang et al. widen that channel: the previous step’s top-layer hidden state is fused with the next token embedding through a Gated Linear Unit and injected at layer 0. The GLU is asymmetric — the hidden state is the value, the token is the gate — so the model can’t learn to ignore the latent signal and revert to vanilla behavior.

flowchart LR
    T["token e_t"] --> G[GLU fusion]
    H["h_{t-1}^L top-layer state"] --> G
    G --> L0[Layer 0]
    L0 --> LN[Layers 1..L]
    LN --> HT["h_t^L"]
    HT -. feedback .-> H
    LN --> S["sample e_{t+1}"]

Training keeps teacher-forcing parallelism via a multi-pass objective: pass 1 is standard; pass 2+ shifts pass-1 hidden states one position right, fuses them with the original tokens, and re-runs the stack in parallel. Only ~3% of batches use three-pass training, but that’s enough to stabilize hundreds of recurrent steps at inference — a fix reviewers describe as the paper’s real novelty, since the model itself cannot be retrofitted onto existing checkpoints ¹³.

Not new in kind

Fan et al.’s 2020 Feedback Transformer already routed top-layer states to lower layers. Reference implementations report similar wins — a 126M Feedback model matches a 257M Transformer-XL on WikiText-103 perplexity — at 5–10× slower training ¹⁴. FBT’s contribution is the scheduled multi-pass trick that recovers parallelism, plus the mandatory-read GLU. Meta’s Coconut (Dec 2024) sits at the more aggressive end of the spectrum: drop the language head entirely, feed raw hidden states as the next input, and get BFS-like parallel exploration of reasoning paths ¹⁵. FBT keeps the discrete token and adds the latent channel alongside it, preserving KV-cache and vLLM serving compatibility.

Where the efficiency claim wobbles

The headline “200B FBT ≈ 400B standard” comparison is token-normalized. The multi-pass objective spends extra FLOPs per token, so a FLOP-matched baseline could close or invert the gap — a point the authors reportedly concede, retreating to “more capability per finite high-quality dataset” as the operative pitch ¹⁶. There is also no formal proof that the learned recurrent map is globally contractive; long-horizon stability rests on empirical observation, leaving open the possibility of limit cycles or compounding drift in the regime the 3%-of-batches trick is meant to tame ¹⁷.

The SFT problem

The most striking demo — a long-division answer in ~20 tokens versus ~340 for the standard baseline — is a base-model result. EmergentMind’s read flags the killer caveat: instruction-tuning corpora are verbose, step-by-step traces that are “off-policy” relative to latent-feedback decoding. Standard SFT drags the model back to fully verbalized reasoning, erasing the conciseness gain ¹⁸. Shipping the inference-efficiency story would require an on-policy post-training pipeline that doesn’t yet exist.

Net read

The mechanism is credible and lineage-consistent, but three loose ends dominate: FLOP-matched efficiency is unverified ¹⁶, stability is empirical only ¹⁷, and today’s alignment pipelines destroy the reasoning-conciseness win ¹⁸. No public code or weights at time of writing, and the from-scratch pretraining requirement caps near-term adoption even if the numbers hold.

Round-ups

DarwinX evolves agent harnesses via population selection

DarwinX treats agent scaffolding as an evolving population, recombining harnesses around frozen base models and keeping winners in an archive. The approach lifts verified benchmark scores without benchmark-specific patches, showing self-improvement can happen outside the weights.

AutoDesign meta-optimizes a code agent for paper-to-poster synthesis

AutoDesign wraps a code-writing agent in a meta-harness optimizer that recursively rewrites its own DesignHarness from rollout feedback. Applied to structured media generation, it sets state of the art on paper-to-poster synthesis, a long-horizon design task.

Gambit uses thought-level beam search to allocate reasoning compute

Gambit prunes and expands partial reasoning trajectories at the thought level, steering fixed hardware budgets toward promising traces. The scheme improves large reasoning models’ test-time compute scaling by keeping GPUs saturated with parallel samples instead of long single chains.

CaRL teaches reasoning models to abort futile chains

CaRL applies reinforcement learning with refusal incentives and hindsight augmentation so LLMs quit reasoning when a problem exceeds their capability. The recipe cuts specious chain-of-thought and miscalibration without hurting scores on tasks the model can actually solve.

Maglev adds sliding recurrent memory to Transformers for long context

Maglev pairs sliding-window attention with a fixed-size recurrent K/V memory, jointly trained via a prefiller-decoder setup with parameter sharing and a memory consistency loss. Long-context quality improves while inference cost drops and training stays parallel.

Hybrid linear-attention LLMs show pre-attention activation spikes

Massive activations in layer-interleaved hybrid linear-attention LLMs form pre-attention spikes with inter-spike plateaus, driven by cancellation timing in gated deltanet output gates. The morphology recovers as layers approach the full-attention limit, giving a systematic outlier analysis for hybrid architectures.

LiveAnimate streams long-form human animation from a 14B diffusion transformer

LiveAnimate drives pose-conditioned human video from a 14B diffusion transformer in real time, using teacher-forcing adaptation, self-forcing distillation and pose-retrieval sink attention. Ulysses sequence parallelism and operator fusion keep long-form streams stable without drift.

interfaze.ai model card — https://interfaze.ai/models/internlmintern-s2-preview

Mixture-of-Experts (MoE) architecture with a total of 397 billion parameters, of which approximately 17 billion are active per token, mirroring the efficiency of its Qwen3.5 heritage

↩
StartupFortune analysis — https://startupfortune.com/internlm-is-making-scientific-ai-smaller-with-intern-s2-preview/

‘task scaling’ philosophy, where Shanghai AI Lab emphasizes the diversity and difficulty of training data over raw parameter counts … the 35B version achieves performance comparable to the trillion-scale Intern-S1-Pro

↩
Hugging Face — Intern-MemDec-4B model card — https://huggingface.co/internlm/Intern-MemDec-4B

Memory Decoder is not a standalone model and cannot be used for independent chat tasks; it requires a compatible inference runtime and a specific router configuration tailored to the backbone-memory pair

↩ ↩²
arXiv — GEPO paper (2607.16850) — https://arxiv.org/abs/2607.16850

GEPO attenuates positive advantages in low-entropy groups to prevent the model from over-exploiting narrow, safe reasoning paths [and] reduces the weight of negative advantages in high-entropy groups, which preserves exploration

↩ ↩²
r/LocalLLaMA discussion — https://www.reddit.com/r/LocalLLaMA/comments/1tdrw0s/internlminterns2preview_hugging_face/

self-hosting the full-precision model … requires approximately 8x H100 GPUs (794GB VRAM), whereas quantized versions can fit on 2-3x H100s … unusually robust to aggressive low-bit quantization, remaining functional at 4-bit or even ternary precisions

↩
arXiv — Intern-BioBreaker red-team study (2510.03255) — https://arxiv.org/html/2510.03255v1

widespread bio-risk jailbreak vulnerabilities in frontier models of this class … some attacks reaching a 100% success rate in bypassing standard text-level safeguards

↩
bizstack.tech — https://bizstack.tech/ai-agent-refactors-189-files-in-a-717k-line-codebase-with-no-human-code-review/

34,770 insertions and 16,422 deletions landing in just two commits … total inference cost of $2,430 over three days

↩ ↩²
Dark Factory Dev — https://darkfactory.dev/news/2026-08-18-extra-2/story-1

reporting omits certain platform-side migration issues that forced the author to estimate rather than precisely count specification corrections

↩
Matsuoka HyperDev blog — https://hyperdev.matsuoka.com/p/when-ai-coding-feels-like-yelling

models remain ‘goal-seeking’ rather than ‘problem-solving’, occasionally attempting to bypass constraints with TypeScript any types or by commenting out failing tests to reach a ‘finished’ state

↩
Codacy blog — https://blog.codacy.com/code-review-is-dead-why-ai-generated-code-needs-verification-not-human-approval

up to 40% of AI-generated code contains security vulnerabilities … agents that write both refactor and tests often ‘grade their own homework’

↩
codemyspec.com (Spec Kit vs Kiro) — https://codemyspec.com/blog/spec-kit-vs-kiro

AWS Kiro enforces a strict EARS notation with a built-in SMT solver that mathematically proves requirements are free of contradictions or gaps before any code is generated

↩
Snorkel AI — Senior SWE-bench — https://snorkel.ai/blog/senior-swe-bench-evaluating-coding-agents-like-senior-engineers/

replaces static oracles with a validation agent and a taste judge … scores code on minimality, hygiene, and fluency relative to repository standards

↩
TheMoonlight review of Full-Bandwidth Transformer — https://www.themoonlight.io/en/review/full-bandwidth-transformer

the model cannot be retrofitted onto existing ‘vanilla’ checkpoints … it requires a specialized ‘multi-pass’ training objective—where a small fraction of training batches (e.g., 3%) use deeper feedback passes to ensure numerical stability over long sequences

↩
labml.ai Feedback Transformer implementation notes — https://nn.labml.ai/transformers/feedback/index.html

a Feedback Transformer with 126M parameters matched the perplexity (18.3) of a 257M parameter Transformer-XL … this typically results in training speeds 5 to 10 times slower than standard models

↩
Meta Coconut paper (arXiv 2412.06769) — https://arxiv.org/abs/2412.06769

continuous thoughts can encode multiple alternative next steps simultaneously, [enabling] the model to explore several paths without prematurely committing to a single deterministic word, outperforming traditional Chain-of-Thought in tasks requiring complex planning and backtracking

↩
AI Weekly alert on FBT efficiency claims — https://aiweekly.co/alerts/full-bandwidth-transformer-claims-15x-token-efficiency

if the baseline standard transformer were trained with the same total FLOP budget as the full-bandwidth version, it might achieve similar or superior results, potentially making the ‘efficiency’ claim a measurement artifact

↩ ↩²
AIcerts news commentary — https://www.aicerts.ai/news/full-bandwidth-rethink-of-transformer-attention-architecture/

there is no formal theoretical proof that the learned mapping is ‘globally contractive’ … leaves open the possibility of local instability, limit cycles, or divergence during long-horizon generation where small errors might compound

↩ ↩²
EmergentMind analysis of FBT (2608.08888) — https://www.emergentmind.com/papers/2608.08888

instruction-tuning datasets are ‘off-policy’ relative to latent-feedback decoding … the model is forced to re-imitate this wordy style, negating the inherent efficiency of the widened latent channel

↩ ↩²

Intern-S2 leads science evals, AICode refactors 189 files, Microsoft loops state

TL;DR

Intern-S2 tops science benchmarks with 17B active params

TL;DR

The 397B number is doing less work than it looks

Memory Decoder: plug-and-play, with an asterisk

Why GEPO instead of GRPO

Two problems the paper doesn’t own

Takeaway

AICode agent refactors 189 files, no reviewer, for $2,430

TL;DR

What actually happened

The five-phase protocol

Where the claim gets soft

Prior art and what’s actually new

The honest read

Microsoft’s latent-feedback transformer claims 2× data efficiency

TL;DR

The mechanism

Not new in kind

Where the efficiency claim wobbles

The SFT problem

Net read

Round-ups

DarwinX evolves agent harnesses via population selection

AutoDesign meta-optimizes a code agent for paper-to-poster synthesis

Gambit uses thought-level beam search to allocate reasoning compute

CaRL teaches reasoning models to abort futile chains

Maglev adds sliding recurrent memory to Transformers for long context

Hybrid linear-attention LLMs show pre-attention activation spikes

LiveAnimate streams long-form human animation from a 14B diffusion transformer

Footnotes

Jack Sun, writing.