GenCeption cuts vision data 500×, Qwen two-hop hits 11%, Soofi S tops German
Every URL the pipeline pulled into ranking for this issue — primary sources plus the supporting and contradicting findings each Researcher returned. Inline citations in the issue point back here.
Sources
Video Generation Models are General-Purpose Vision Learners huggingface.co
Driven by next-token prediction, NLP shifted from task-specific models into powerful generalist foundation models. What, then, is the equivalent catalyst needed to achieve a general-purpose model in computer vision? In this paper, we contend that large-scale text-to-video generation serves as a strong pre-training paradigm for computer vision, providing the necessary spatiotemporal priors, vision-language alignment, and scalability required for general visual intelligence. We introduce GenCeptio
A Sovereign, Open-Source Foundation Model for German and English huggingface.co
We present Soofi S 30B-A3B, a sovereign, open-source Mixture-of-Experts (MoE) hybrid Mamba Transformer foundation model for German and English. Its hybrid design activates only 3B of 30B parameters per token and keeps the inference cache near-constant as context grows, giving it a decisive throughput advantage over dense models for long-context, high-concurrency deployment. Pretrained on roughly 27 trillion tokens with deliberately up-weighted German, Soofi S matches dense 14 to 27B models on ag
Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning huggingface.co
Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream reasoning tasks. We formalize this failure as the \textbf{Knowing—Using Gap}, characterized by an accuracy gap and a temporal lag between memorization and generalization. To understand this phenomenon, we fine-tune LLMs with unseen knowledge and monitor the spatial permeation dynamics of the knowledge internally using a novel intervention technique called
Long-Horizon-Terminal-Bench: Testing the Limits of Agents on Long-Horizon Terminal Tasks with Dense Reward-Based Grading huggingface.co
Long-Horizon-Terminal-Bench pushes terminal agents past short, outcome-only tests with 46 tasks across nine categories, from system administration to data engineering. Dense reward-based grading credits partial progress instead of pass/fail, exposing where agents stall on multi-step workflows that current benchmarks miss entirely.
Trust Region Policy Distillation huggingface.co
Trust Region Policy Distillation reworks on-policy distillation by dynamically constructing a nearby teacher, taming the gradient variance that has made OPD unstable. The authors prove monotonic improvement and global convergence, giving distillation the trust-region guarantees that PPO brought to policy optimization.
Scalable Visual Pretraining for Language Intelligence huggingface.co
Scalable Visual Pretraining for Language Intelligence argues that converting documents and web pages to plain text throws away equations, figures and page structure that carry real knowledge. The paper pretrains language models directly on visual renderings of these sources to preserve signals lost by tokenization.
Self-Guided Test-Time Training for Long-Context LLMs huggingface.co
Long-context accuracy degrades as inputs grow because models fail to locate relevant evidence. Self-Guided Test-Time Training treats the prompt itself as a training example, fine-tuning instance-specific parameters at inference so the model adapts to the passage it must actually answer from.
From RGB Generation to Dense Field Readout: Pixel-Space Dense Prediction with Text-to-Image Models huggingface.co
Instead of decoding tokens into RGB pixels, this work maps a pretrained DiT’s outputs directly to task-native fields via a token-local linear head and task LoRAs. FLUX-Klein hits state-of-the-art on KITTI depth, trimap-free matting, normals, saliency, pose and referring segmentation with minimal added parameters.
KronQ: LLM Quantization via Kronecker-Factored Hessian huggingface.co
GPTQ and its heirs assume every output channel contributes equally to layer reconstruction, relying only on input activation statistics. KronQ folds a Kronecker-factored gradient covariance into the Hessian, weighting channels by their true impact on loss and improving low-bit quantization quality without retraining.
PanoWorld: Real-World Panoramic Generation huggingface.co
PanoWorld reframes camera motion in omnidirectional video as translations under fixed headings, using rotation equivariance to keep long-range memory consistent. Dense Panoramic Ray-Conditioning and a geometry-aware memory bank let the world model recall distant scene structure that flat-camera generators lose.
Flow-ERD: Agent-type Aware Flow Matching with Entropy-Regularized Distillation for Diverse Traffic Simulation huggingface.co
Flow-ERD is a multi-agent traffic simulator that combines agent-type aware flow matching with entropy-regularized distillation to achieve both realistic and diverse motion patterns.
Phone Segmentation and Recognition through Phonological Activation Mapping huggingface.co
Phone segmentation and recognition are inherently related tasks, yet modern approaches typically model them separately. We argue that phonetic structure is already latent in the representations of self-supervised speech models (S3Ms), and one only needs to steer them to solve both tasks. We leverage S3M-based Phonological Activation Mapping (SPAM), which maps each S3M representation frame to a vector of phonological feature activations, such as voicing and nasality. On top of SPAM, we introduce
MedPMC: A Systematic Framework for Scaling High-Fidelity Medical Multimodal Data for Foundation Models huggingface.co
Medicine is inherently multimodal, requiring clinicians to synthesize information across diverse data streams. Yet the development of multimodal foundation models is constrained by limited access to large-scale, high-quality clinical data. Although PubMed Central (PMC) offers a complementary source of expert-authored image-text data, existing PMC-derived resources remain limited in fidelity, reproducibility, and clinical validation. We introduce MedPMC, an automated, continuously updatable frame
VaseMuseum: Digital Intelligent Museum for Ancient Greek Pottery huggingface.co
Vision-language models (VLMs) have made interactive digital museums increasingly feasible by connecting 3D digitization with natural-language artifact exploration. However, in cultural heritage domains such as ancient Greek pottery, reliable VLM assistance is limited by two challenges. First, open-ended interpretation requires grounding fine-grained 2D/3D visual evidence in specialized curatorial knowledge, yet the retrieval process may introduce weak sources and unverifiable references. Second,
References
The Decoder the-decoder.com
Google DeepMind argues video generators already contain the world models computer vision has been missing
AIbase aibase.com
the 14-billion-parameter version of GenCeption takes approximately ten seconds to process just 81 frames, limiting its utility for real-time applications like robotics or AR
XenoSpectrum xenospectrum.com
it remains a ‘representation learner’ rather than a predictive physics engine
ScriptsHub (V-JEPA analysis) scriptshub.net
Critics, including Yann LeCun, argue this ‘generative trap’ wastes massive computational resources on high-entropy noise—such as the exact movement of individual leaves in the wind—which is irrelevant to high-level understanding
Roboflow blog on DepthAnything 3 blog.roboflow.com
UniDepthV2 as the leader on KITTI with an AbsRel of 0.055 (5.53%), outperforming DA3 in raw in-domain metric accuracy
Marigold paper (arXiv 2402.14973) arxiv.org
fine-tuning Stable Diffusion for monocular depth estimation, arguing that a model capable of generating a realistic ‘photo’ must inherently understand spatial relationships
Gekhman et al., EMNLP 2024 — ‘Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?’ arxiv.org
LLMs struggle to acquire new factual knowledge through fine-tuning… as the model eventually learns new knowledge, it linearly increases its tendency to hallucinate.
Models can achieve near-zero training loss on facts yet fail 100% of the time to answer questions about those facts if the training data lacked diversity; instruction fine-tuning cannot recover knowledge that was memorized but not properly augmented during pre-training.
LessWrong — Hase et al. critique, ‘But is it really in Rome? An investigation of the ROME model editing technique’ lesswrong.com
Causal tracing identifies computational bottlenecks or retrieval paths rather than the actual storage site of the data — editing the layers tracing points to is not always the most effective way to update knowledge.
Gupta et al., ‘Model Editing at Scale leads to Gradual and Catastrophic Forgetting’ (scalable-model-editing.github.io) scalable-model-editing.github.io
ROME and MEMIT suffer from two phases of degradation: gradual forgetting followed by catastrophic forgetting, with MEMIT stable up to ~1,000 edits and ROME collapsing under 100.
r/LocalLLaMA thread — ‘Has anyone seen grokking during LLM finetuning?’ reddit.com
A model might appear to have reached a plateau in performance while fine-tuning on a specific logic or reasoning task, only to ‘grok’ the underlying principle much later — early stopping may prevent models from reaching full reasoning potential.
TheMoonlight.io review of the Mem2Gen paper themoonlight.io
The ‘oracle headroom’ metric of which their heuristic recovers 58–75% is a self-defined ceiling that may not reflect real-world performance across diverse architectures, and there is no plug-in fix for popular open-weight models.
The Decoder the-decoder.com
Soofi S achieved the highest English and German aggregate scores among fully open models… +2.8 points over Olmo 3 32B and +6.3 points over Apertus 70B
Smol AI newsletter (LocalLLaMA roundup) news.smol.ai
gated on Hugging Face, model card with ‘TODO’ license and context length, and allegations the training mix (notably AIML-TUDA/QA-base) may have leaked into evaluation sets
connic.co analyst brief connic.co
27 trillion tokens for a 30B model is heavily overtrained by Chinchilla standards; lead Michael Fromm counters that hybrid Mamba-MoE stacks benefit differently from massive data repetition than dense models do
NVIDIA developer blog (Nemotron 3 Super) developer.nvidia.com
Nemotron-3 Nano achieves up to 3.3x higher throughput than Qwen3-30B-A3B on single-H200 serving with a 23 Mamba-2 / 23 MoE / 6 GQA layer stack
T-Systems press release (Industrial AI Cloud Munich) t-systems.com
the Munich site houses ~10,000 Blackwell GPUs including ~1,000 DGX B200 systems, delivers ~0.5 ExaFLOPS, and is cooled with Eisbach canal water on 100% renewable energy
effektiv.ch (Apertus release analysis) effektiv.ch
Apertus is Apache 2.0 with fully open weights and training data trained on 1,800+ languages on the Swiss ‘Alps’ supercomputer — a stricter reading of ‘open’ than Soofi’s gated, custom-license preview
