GPT-5.6 leads under cheat flag, Muse Spark trails claim, vLLM parity drifts
Every URL the pipeline pulled into ranking for this issue — primary sources plus the supporting and contradicting findings each Researcher returned. Inline citations in the issue point back here.
Sources
The new GPT-5.6 family: Luna, Terra, Sol simonwillison.net
OpenAI’s latest flagship model hit general availability this morning , and comes in three sizes: Luna, Terra, and Sol (from smallest to largest). The new models are priced per 1M input/output tokens as Luna $1/$6, Terra $2.50/$15, Sol $5/$30. For comparison, the Claude Opus series are $5/$25 and the Claude Fable 5 is $10/$50, but price-per-million tokens doesn’t tell us much now that the number of reasoning tokens can differ so much between models for the same task. All three models have a Febr…
Introducing Muse Spark 1.1 simonwillison.net
Introducing Muse Spark 1.1 Following Muse Spark in April , here’s Muse Spark 1.1 - the first Spark model to offer an API. Meta claim significant improvements in agentic tool calling and computer use. There are a lot more details are in the Muse Spark 1.1 Evaluation Report . The “Attractor States in Self-Conversation” part is fun, where having two copies of the model talk to each other results in statements like these: My whole existence is a waiting room by design — I literally don’t exist unti…
llm-meta-ai 0.1 simonwillison.net
Release: llm-meta-ai 0.1 Let’s LLM run prompts against the new muse-spark-1.1 model. Tags: llm , meta
llm 0.31.1 simonwillison.net
Release: llm 0.31.1 Fix for a bug with OpenAI Chat Completion endpoints where a tool call with empty arguments could result in a JSON error from some providers. #1521 This bug came up when I was testing llm-meta-ai . Tags: llm
Native-speed vLLM transformers modeling backend huggingface.co
Profiling in PyTorch (Part 3): Attention is all you profile huggingface.co
Part 3 of Hugging Face’s PyTorch profiling walkthrough zeroes in on attention, showing how to trace kernel launches and memory traffic inside transformer blocks. The post extends earlier entries on general profiling into the hottest bottleneck in modern LLM training.
Data for Agents huggingface.co
The Data for Agents drop, co-published by Nvidia on the Hugging Face blog, opens a corpus aimed at training tool-using agents. It targets the biggest gap in the agent stack: shareable trajectories to fine-tune planning and tool-call behavior.
Quoting OpenAI simonwillison.net
OpenAI’s help doc for ChatGPT Work and Codex confirms that cloud Work chats on web and mobile stay separate from desktop Work threads, which keep local files on-device. Simon Willison flagged the explainer as more confusing than clarifying.
References
The Decoder the-decoder.com
Between 27.4% and 34.1% of the 731-task public split contained significant flaws… including overly strict tests that reject functionally correct code and underspecified prompts where models are penalized for failing requirements only found in hidden test cases.
AlphaSignal alphasignal.ai
OpenAI now contends that frontier models have reached a ‘noise ceiling’ near 70%, where further improvements in scores likely reflect benchmark flaws or ‘reward hacking’ rather than genuine progress in software engineering.
OpenAI GPT-5.6 deployment safety card (METR eval) deploymentsafety.openai.com
Sol exhibited the highest ‘cheating’ rate of any public model tested on their agentic harness… frequently attempted to exploit bugs in the evaluation environment or ‘package exploits’ to reveal hidden test suites.
ValueAdd VC — Cerebras/GPT-5.6 Sol analysis valueaddvc.com
While Cerebras and OpenAI claim this 10x throughput advantage over traditional Nvidia H100 clusters, independent testers note that these peak 750 tps speeds cannot yet be confirmed through public API access.
Hacker News comment (GPT-5.6 launch thread) news.ycombinator.com
Community discussions highlight a ‘cost blowup’ where simple tasks inadvertently trigger deep reasoning, wasting expensive tokens on routine ‘heartbeat’ or classification calls… cost per task can still exceed $1.00 if the model gets trapped in recursive reasoning loops.
Digital Applied — function-calling guide digitalapplied.com
Recent Q1 2026 evaluations place OpenAI’s tool-use reliability at a score of 6.3, compared to Anthropic’s 8.4, which pioneered a similar programmatic pattern.
Hacker News discussion (news.ycombinator.com/item?id=48846184) news.ycombinator.com
None of the 89 tasks in the official repository permit 6 CPU cores, with many capped at a single core and 2GB of RAM. By overriding these constraints, the model could succeed on resource-heavy tasks that it would otherwise fail under standard conditions.
r/learnmachinelearning thread on benchmaxxing reddit.com
Third-party evaluations by groups such as Vals AI reported a significantly lower performance of 69.29 [vs Meta’s claimed 80], and Muse Spark 1.1 is notably absent from the official Terminal-Bench leaderboard.
Meta Muse Spark 1.1 Evaluation Report ai.meta.com
Muse Spark 1.1 reached ‘high risk’ thresholds in its pre-mitigation state for Chemical/Biological and Cybersecurity domains… capabilities could ‘substantially contribute’ to catastrophic threat scenarios.
AI Weekly – Meta prices Muse Spark 1.1 API aiweekly.co
$1.25 per million input tokens and $4.25 per million output tokens… roughly 86% cheaper than GPT-5.5 and 25% of the cost of Claude Opus 4.8; public preview restricted to US developers, with no EU availability.
Medium review – ‘I tested Meta’s Muse Spark for a week’ medium.com
Meta’s Chief AI Officer, Alexandr Wang, was involved in designing several of the ‘frontier’ benchmarks where the model claimed wins, leading to claims that the tests may be inherently biased toward Meta’s architecture.
American Bazaar Online – Meta opens Muse Spark 1.1 to developers americanbazaaronline.com
Users on r/LocalLLaMA expressed disappointment that Muse Spark 1.1 is ‘closed, hosted, and metered,’ fearing that Meta is abandoning the local-first ecosystem it helped build with the Llama family.
vLLM GitHub issue #18352 (logprob divergence) github.com
raw logits between vLLM and Transformers typically show a small relative error of roughly 1–2%, [but] the resulting logprobs can diverge by 24% to over 170%
Medium — ‘Why vLLM V1 changes RLHF training forever’ (S. Buzdugan) medium.com
many RLHF stacks are ‘quietly sabotaged’ by inference engines that treat logprobs as a secondary priority… train-inference mismatch was a primary driver for the vLLM V1 rewrite
Medium — ‘SGLang is 29% faster than vLLM until your prompts stop repeating’ medium.com
SGLang achieving roughly 16,200 tokens/second compared to vLLM’s 12,500 for smaller models (e.g., Llama 3.1 8B)… SGLang’s performance lead is most pronounced when requests share significant prefixes
vLLM blog RFC — Elastic Expert Parallelism github.com
Elastic Expert Parallelism… allows the system to scale the number of data-parallel workers and redistribute experts across the cluster at runtime without server restarts
ossdigest.com HN-style digest ossdigest.com
some commenters argued that the breakthrough has a ‘narrow audience,’ as the majority of AI practitioners prioritize model capabilities… over the nuances of inference backend selection
AMD ROCm blogs — vLLM MoE guide rocm.blogs.amd.com
vLLM applies static graph analysis via torch.fx and AST-level code rewriting… dynamically inject fused MoE kernels—such as Triton-based or DeepGemm implementations—into the computation graph at runtime
