Meta Skaling cuts sweep 10×, Parallel-RL +10.7%, activation oracles hide target

Sources

Skaling: Chinchilla’s Exponents Meet Kaplan’s Coupling huggingface.co

The Skaling law couples model capacity and data via an interaction exponent to improve loss prediction across training regimes and reduce compute needs for scaling experiments.

SFT Conflicts, RL Coexists: A Theoretical and Empirical Analysis of Multi-Task Learning for LLMs huggingface.co

Reinforcement learning enables stable multi-task reasoning in large language models through sparse, near-orthogonal parameter updates, unlike supervised fine-tuning which suffers from task conflicts, motivating a decoupled Parallel-RL training approach.

When Activation Oracles Learn Not to Read: Concept-Specific Blind Spots in Fine-Tuned Oracles huggingface.co

Activation Oracles can become concept-specific anti-readers that selectively fail to recover internally represented hidden concepts, revealing reliability concerns for learned interpretability interfaces.

Adversarial Attacks for Good: A Survey of Proactive Protection across the Visual Content Lifecycle huggingface.co

Content owners increasingly deploy adversarial perturbations as protection, and this survey catalogs the toolkit across a visual asset’s lifecycle: privacy filters, unlearnable examples, generative safeguards, adversarial CAPTCHAs, and provenance marks. The authors stress-test each class against adaptive threats and transferability across multimodal models.

PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say huggingface.co

Auditing only agent outputs misses the real leak: PrivacyPeek inspects tool-call trajectories and probes for hidden sensitive data, showing widespread unnecessary acquisition during task execution. Prompt-based defenses barely dent the problem, exposing a blind spot in current agent privacy evaluation.

DCAS: Decoupling CLI Agent Scaffolding to Internalize Planning across Scaffolds huggingface.co

CLI coding agents usually bake planning into their scaffold, hurting portability. DCAS separates scaffolding from policy and trains on planning-aware trajectories, letting a single model generalize across different agent environments in cross-scaffold evaluation instead of overfitting to one harness.

The Optimizer Is the Agent: Reasoning-Driven Search across Prompts, Programs, and ML Workflows huggingface.co

Rather than bolting agents onto search algorithms, ReASearch treats optimization as autonomous reasoning with diagnosis and persistent memory. The unified framework beats specialized controllers—evolutionary search, bandits, textual-gradient methods—across prompt tuning, program synthesis, and ML workflow optimization.

Small Foundation Models of Human Cognition and Behaviour huggingface.co

Fine-tuning small language models on human behavioral data matches large baselines in-distribution and scales better out-of-distribution, the Centauri project reports. The models lean on task stimuli and feedback rather than choice history, approaching the noise ceiling as cognitive proxies.

Enfold: Folding World Model Imagination into Predictive Representations for Ultra-Efficient Embodied Control huggingface.co

Running a generative world model at every control step is too slow for embodied agents. Enfold distills future-generative computation into a present-only representation across multi-level states, cutting action latency while keeping the predictive signal that guides control.

Addressable Memory for Video World Models huggingface.co

Interactive video world models forget quickly; WorldTrace adds a compressed KV cache with virtual RoPE positions that the model can address like scene landmarks. The training-free method improves temporal consistency and episodic recall on LoopBench, letting scenes persist across long horizons.

SimWAM: A Simple World Action Model for End-to-End Autonomous Driving huggingface.co

SimWAM trains a lightweight action planner using video generation as a training signal, enabling efficient trajectory prediction without future generation at inference and supporting reinforcement learning optimization.

Efficient Knowledge Distillation for LLMs: Offline Top-K Logits and a Fused Chunked KL Loss huggingface.co

Efficient knowledge distillation for small language models is achieved via cached teacher logits and a memory-linear chunked KL loss, enabling longer contexts and faster training.

Round-Trip Consistency: Bidirectional Diffusion Models Can Predict Their Own Rollout Errors huggingface.co

A bidirectional latent diffusion model uses round-trip consistency as a self-supervised error proxy for autoregressive rollouts across physical and video dynamics.

When Privileged Guidance Misaligns: State-Matched Routing and Contextualized Self-Distillation for Multi-Turn Agents huggingface.co

SMRC-SD improves multi-turn agent training by routing privileged reference guidance only to execution states that match the student’s current trajectory and conditioning teacher context on those states.

Relevant but Incomplete: Referential Dangling as a Paradigm-Level Failure Mode in Hard Prompt Compression huggingface.co

Hard prompt compression frequently breaks multi-hop reasoning by deleting context needed to interpret retained answers, and restoring missing referential support substantially recovers accuracy.

Zero Gap Is Not Restoration: Stratified Per-Question Probability Evaluation and Step-wise Mitigation of Benchmark Contamination huggingface.co

Proposed SA-PPG metric and RailCap decoding intervention more accurately assess and reduce benchmark contamination in pretrained models.

Modular TTT: Rethinking Test-Time Training as Composable Modules huggingface.co

Modular TTT decomposes test-time training into configurable graph components to systematically evaluate design choices and build efficient sequence models.

Reinforcement Learning with Evolving Rubrics as Rewards for Audio Reasoning huggingface.co

AudioRubrics improves audio reasoning by using self-evolving, audio-grounded rubric rewards that adapt to each question and target policy weaknesses during reinforcement learning.

Capek 0.5: An Execution-Centric Vision-Language Model for Embodied Intelligence huggingface.co

Capek 0.5 organizes embodied vision-language capabilities around execution roles, trains specialist models via reinforcement learning, and merges them into a unified model for iterative robot control.

StreamArena: Toward Continuous, Interactive, and Long-Horizon Agentic Streaming Video Understanding huggingface.co

StreamArena evaluates long-form interactive video understanding, revealing trade-offs between real-time interaction and long-horizon memory, which StreamMind addresses via a two-tier architecture separating immediate interaction from persistent multimodal recall.

Multi-Agent Forensic Reasoning for Generalizable Deepfake Video Detection huggingface.co

A large-scale deepfake video benchmark with fine-grained annotations and a multi-agent reasoning framework improves detection of emerging synthetic face videos.

Uncertainty-Aware World Model for Aerial Image-Goal Navigation huggingface.co

A latent world model for aerial navigation improves trajectory selection by separating explainable future uncertainty from residual discrepancies, reducing inference cost while handling large-scale outdoor environments.

Towards Interpretable Foundation Models for Retinal Fundus Images huggingface.co

DualIFM is an interpretable self-supervised foundation model for retinal imaging that uses a BagNet backbone and 2D projection to provide faithful, visualizable representations with far fewer parameters than comparable models.

Beyond Simply Environment Scaling: Designing Effective Environment Distributions for Multimodal Agent Learning huggingface.co

Effective multimodal agent training is improved by selecting diverse environments via ability-aware selection and structuring difficulty through hierarchical curriculum learning.

MatrAIx: Simulating the World with 8.3 Billion Persona Agents huggingface.co

MatrAIx is a large-scale simulated-user evaluation framework that uses diverse persona records and interactive environments to test AI systems across many domains.

OneEmo: A Unified Multimodal Reasoning Model for Emotion Perception, Understanding, and Interaction huggingface.co

OneEmo is a unified affective generalist that leverages multi-task supervised fine-tuning and a reinforcement learning strategy to improve emotional reasoning across perception, comprehension, and interaction tasks.

Do AI Personas Grow? Analyzing and Benchmarking Personality Evolution in LLM Agents After Life Events huggingface.co

Personality-conditioned LLM agents show event-induced trait shifts that match human directions but with smaller magnitudes and compressed variance, prompting a benchmark for directional fidelity.

YOLO-PEFT: Parameter-Efficient Fine-Tuning on YOLO Family huggingface.co

YOLO-PEFT formulates adapter placement for real-time detectors as a constraint-planning problem to enable auditable, budgeted parameter-efficient fine-tuning with explicit refusal when constraints are violated.

DuplexGen: Adaptive Synthesis of Human-AI Turn-Taking Dialogues huggingface.co

DuplexGen calibrates dialogue generation against human preferences to produce scenario-adaptive turn-taking behaviors in full-duplex interaction.

Douyin Multimodal Embedding Model Technical Report huggingface.co

DME is a two-stage multimodal embedding model that merges efficient contrastive pre-training with training-only reasoning and reconstruction mechanisms to achieve fine-grained discrimination at billion-scale serving cost.

FATE: Frame-Level Audio-Visual Temporal Embedding huggingface.co

FATE learns frame-level audio-visual embeddings that jointly capture semantic content and temporal alignment through cross-video and within-video contrastive training.

Can MLLMs Decode the Creative Leap? Introducing C4 for Cross-Concept Understanding huggingface.co

A cognition-inspired framework evaluates multimodal models on cross-concept creativity using Chinese idiom-based image generation and decoding tasks.

CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models huggingface.co

CLIP-CC-Bench evaluates long-form video description using expert paragraph references and ensemble LLM embeddings to benchmark 17 video-language models.

Complementary Matrix-Gated QKAN Fast-Weight Programmers for Quantum Dynamics Forecasting huggingface.co

Self-modulating quantum-inspired fast-weight programmers with coordinate-wise complementary matrix gating improve long-context sequence forecasting while preserving efficient update structures.

References

KuCoin News (wire coverage of Meta FAIR paper) kucoin.com

Meta demonstrates that their coupled formula allows for ‘L-shaped’ grid sweeps… reduces the compute required to plan a training run by approximately 10x while improving the precision of loss predictions in boundary regions where data and model size are imbalanced.

alphaXiv discussion of arXiv:2608.07222 alphaxiv.org

Independent researchers on alphaXiv have noted that fitting these laws is a fragile non-convex problem, recommending gradient-free optimizers like CMA-ES over standard L-BFGS for more reliable out-of-the-box results.

Farseer paper (Li et al. 2025, ResearchGate) researchgate.net

Farseer employs a 9-parameter equation… reduces extrapolation error by approximately 433% compared to previous standards, enabling researchers to predict the performance of 25-billion-parameter models with less than 1% relative error.

Databricks engineering blog: ‘How Long Should You Train Your Language Model?’ databricks.com

Llama 3 8B famously ignored the 20:1 ratio, training on over 15 trillion tokens—a ratio exceeding 1,800 tokens per parameter… the cost of serving a model to millions of users often dwarfs the one-time training cost.

EmergentMind topic page (summarizing Epoch AI replication) emergentmind.com

Small changes in the training data points can lead to vastly different values for the exponents α and β… the underlying noise in model training runs often dwarfs the precision of the optimization algorithm itself.

TheClarity.today (independent write-up) theclarity.today

For certain datasets (like Farseer-code), the fitted k was closer to 1 (0.77–0.90), leading to performance similar to Chinchilla… the direction of the optimal ratio drift depends on the specific architecture and data.

Mukherjee et al., arXiv:2505.11711 (‘RL Finetunes Small Subnetworks in LLMs’) arxiv.org

RL algorithms — including PPO, DPO, and GRPO — induce parameter update sparsity where only 5% to 30% of a model’s weights change significantly, and training only these sparse subnetworks can recover full-model performance.

HuggingFace discussion of Mukherjee et al. (concurrent-work critique) huggingface.co

When training in pure FP32 the sparsity effect largely disappears, suggesting many RL updates are simply too small to overcome BF16 rounding thresholds — a mechanical rather than intrinsic explanation for the phenomenon.

GaryStack/Parallel-RL GitHub README github.com

The repository is currently in a ‘preliminary release’ state; specific environment details, reward verifiers, and distributed paths are still being refined.

Stoica & Ramesh, ‘Model Merging with SVD to Tie the Knots’ (KnOTS) semanticscholar.org

Using SVD to jointly transform LoRA-finetuned weights into an aligned representation space before applying TIES yields performance improvements of up to 4.3% across vision and language benchmarks.

Long-to-Short via Model Merging (GitHub benchmarks) github.com

Smaller models (1.5B–3B) struggle to maintain complex Chain-of-Thought reasoning through merging alone, often requiring the direct training signals provided by a unified multi-task objective.

Parallel-R1 / Native Parallel Reasoner (arXiv 2512.07461) arxiv.org

‘Parallel-RL’ frameworks focus on computational parallelism — teaching models to branch their reasoning paths into simultaneous execution graphs — a distinct research thread from weight-level sparsity work.

OpenReview — Jakkli, Rajamanoharan & Nanda, ‘Current Activation Oracles Are Hard to Use on Safety-Relevant Tasks’ openreview.net

Many apparent successes of AOs are actually instances of the model reconstructing nearby text tokens from an activation vector and then answering queries based on that decoded text, rather than inspecting deeper semantic signals… in settings where text inversion is impossible, AOs performed at near-chance levels.

AI Weekly alert summary aiweekly.co

A standard AO can recover the hidden word with ~94% accuracy [but] the fine-tuned version’s exact recovery rate collapses to approximately 35%, effectively becoming an ‘anti-reader’… auditors must pair Oracle natural-language reports with independent, simpler linear probes to ensure the Oracle hasn’t learned to ‘hide’ its findings.

Hyper.ai paper page (2607.23379) hyper.ai

Rather than becoming specialist readers, these fine-tuned probes often become ‘concept-specific anti-readers’… the oracle retains the ability to accurately read and describe other concepts it was not specifically fine-tuned on, proving that the blind spot is concept-specific rather than a global degradation.

Qwen team blog — Qwen-Scope SAE release qwen.ai

Qwen-Scope uses Sparse Autoencoders (SAEs) to disentangle dense representations into interpretable features… narrow fine-tuning for specific tasks like Taboo leaves highly readable traces in activation differences, which can be identified using independent probes or ‘crosscoders’ even when the model attempts to suppress the information.

Shared Sapience Substack — coverage of Anthropic’s Jacobian Lens / Taboo model organism sharedsapience.substack.com

When a model was asked to describe a web-spinning animal without using its name, the J-lens revealed the concept ‘spider’ lighting up internally as a pre-verbal stepping stone… independent critiques suggest that Taboo models suffer from ‘catastrophic forgetting’; when subjected to generic secondary fine-tuning, they often lose the ability to hide the secret.

arXiv preprint (Bersia & Gaintseva, primary paper HTML) arxiv.org

Linear probes applied to the AO’s own internal layers reached near-perfect (100%) accuracy in identifying the hidden concept… the failure occurs during the final ‘verbalization’ or readout stage of the oracle, not from loss of information.