Fable 5.1 forces reasoning, BenchMIRT finds 2 axes, Paint.NET unreviewed

Sources

Claude Fable 5.1 made me a really nice animated pelican simonwillison.net

Today is Claude Fable (and Mythos) 5.1 day . Anthropic say that Fable 5.1 “sets a new standard for coding, knowledge work, and long-running problem-solving tasks”. Their announcement spends a notable amount of time on scientific research, boasting of a 52.6% score on the brand new Terminal-Bench-Science 0.1 benchmark (first announced on August 27th ), up from 24.7% for Fable 5, 29.0% for Opus 5 and 22.4% for GPT-5.6 Sol. Other benchmarks show slightly improved scores, but none as impressive as…

BenchMIRT: What are LLM benchmarks actually measuring? huggingface.co

Quoting Rick Brewster simonwillison.net

Direct2D has always been the biggest hurdle for Paint.NET on WINE, and it’s clear that it will never be completed enough for Paint.NET’s use. And I can’t just “disable” the use of Direct2D. So, instead, Paint.NET now has an internal, from-scratch, clean-room reverse-engineered rewrite of Direct2D that it uses on WINE (triggered by using /wine ). It lives in PaintDotNet.Windows.Direct2D1.Managed.dll . This was written by our good friend Claude , without whom this would NOT have been possible and…

Introducing @huggingface/kernels: 200+ WebGPU Kernels for Local AI huggingface.co

Hugging Face released @huggingface/kernels, a library bundling more than 200 WebGPU compute kernels for running AI models locally in the browser. The package targets developers building client-side inference without server round-trips, extending Hugging Face’s push into on-device execution.

Codex bundles LibreOffice simonwillison.net

Poking through his cache folder, Simon Willison found the Codex (now ChatGPT) desktop app ships a 1.7GB runtime including full Python, Node.js, Poppler, git, and a 430MB headless LibreOffice. Skill files tell Codex how to invoke the bundled binaries for document tasks.

datasette-mcp 0.2 simonwillison.net

The first non-alpha release of datasette-mcp changes execute_sql output from arrays of arrays to arrays of objects, so weaker LLMs stop losing track of which positional value maps to which column. The plugin now requires mcp 2.1.1 or later.

GeoJSON Map Viewer simonwillison.net

Simon Willison published a browser tool for overlaying GeoJSON files on OpenStreetMap and exporting PNGs, built by GPT-5.6-Sol with Claude Code iterations. He used it to visualize California district boundaries that ChatGPT Work extracted from government data sources on request.

Python 3.15.0 candidate 2 is here! simonwillison.net

Release manager Hugo van Kemenade announced Python 3.15.0 RC2, the final candidate before October’s stable release. Only clear bug fixes are allowed from here, and maintainers are urged to publish 3.15 wheels now — binaries built against the RC will work with the final release.

References

VentureBeat venturebeat.com

Fable 5.1 uses approximately 1.7 times more output tokens than Fable 5 to solve the same tasks… a 75% reduction in cache-read costs to $0.25 per million tokens

BigGo finance recap of Cognition/Millennium case finance.biggo.com

A ‘one-in-a-million’ concurrency bug had remained unexplained for four to five years… Fable 5.1 successfully identified the root cause in a single afternoon [by] disassembling an external vendor library and matching its logic against a core dump

bpdata.com news aggregator news.bpdata.com

swalsh challenged the community, commenting, ‘Now that it’s a solved benchmark, can we get the animated version?’… users noted a technical ‘skill issue’ where the wheels appeared to rotate in the wrong direction

r/claudexplorers thread on reasoning behavior reddit.com

sending reasoning:{enabled:false} now triggers a 400 invalid_request_error, as the model requires a thinking block to process tool calls

awongcm.io 2026 model landscape review awongcm.io

GPT-5.6 Sol—currently priced at a promotional $4/M input and $20/M output—is being adopted as the ‘refined gentleman’ of AI: a more cost-effective workhorse that provides 95% of Fable’s intelligence at a fraction of the operational budget

Hacker News discussion thread news.ycombinator.com

‘pelicanmaxxing,’ suggesting that AI labs may be overfitting their models on this specific prompt because it is a well-known public test

Paint.NET forum (Rick Brewster original post) forums.paint.net

Requires WINE 11.14+ and DXVK; users must set d3dcompiler_47 to ‘native’ via wine reg or the app crashes; launching with /wine disables UIAnimation and Windows 11 DisplayInformation components.

Simon Willison — ‘Vibe engineering’ (Oct 2025) simonwillison.net

Willison draws a line between ‘vibe coding’ (trusting the output without review) and ‘vibe/agentic engineering’ done by professionals — and has since flagged a ‘disturbing realization’ that experts like Brewster are now shipping code volumes they cannot possibly audit, blurring the two categories.

Cloud Security Alliance — GenAI Code Security research note labs.cloudsecurityalliance.org

Independent testing (Veracode 2025) found AI-generated code introduces security vulnerabilities in ~45% of cases, with no measurable improvement as models get more capable — a direct concern for 180K unreviewed lines handling COM lifetimes and GPU resources.

dev.to — ‘I reviewed 6 vibe-coded codebases’ dev.to

Reviewer found vibe-coded projects consistently hit an ‘80/20 wall’ where individual features work but interactions fail in production, and that manual review of a 300K-line AI codebase took roughly 10× longer than the AI’s implementation time.

PlagiarismToday — ‘Cleanroom as a Service: AI Washing Copyright’ plagiarismtoday.com

Legal scholars question whether a true ‘clean room’ can exist when the implementing agent has already ingested the protected code during training; purely AI-generated output may also fall into a copyright void under Thaler v. Perlmutter, leaving the developer with ‘all the liability and none of the protection.’

NoesisNews — coverage of the Paint.NET/Claude Direct2D work noesisnews.fr

Claude reportedly achieved more Direct2D progress in three weeks than the WINE community had produced in the twelve years since Paint.NET adopted the API in 2014, including reverse-engineering the math for the built-in effects library.

Liner review of tinyBenchmarks (Polo et al., 2024) liner.com

tinyBenchmarks reduces massive benchmarks like MMLU (14k+ items) to curated subsets of just 100 items while maintaining an average error margin of under 2%, using a generalized p-IRT estimator that combines direct sampling with IRT model predictions.

Stanford CRFM — ‘Reliable and Efficient Evaluation’ crfm.stanford.edu

Classical IRT tools were built for many test-takers and few items, whereas AI benchmarks invert this — few models, many items — causing estimators to become computationally infeasible or produce unreliable ranking inferences when ability distributions are non-normal.

alphaxiv 2502.15361 — Reasoning amplifies bias in thinking models alphaxiv.org

Generating Chain-of-Thought reasoning can actually amplify social bias; models may initially identify the correct ‘Unknown’ answer but then ‘rationalize’ their way into a stereotypical conclusion through spurious intermediate steps.

Center for AI Safety — WMDP benchmark description safe.ai

Standard refusal behavior is fragile because it can be bypassed through jailbreaking or adversarial fine-tuning; WMDP-based unlearning (RMU) aims to reduce hazardous-topic scores to random-chance while maintaining high MMLU reasoning scores.

Renascence — coverage of UK AI Security Institute audit renascence.io

Models could inflate their ‘safety’ scores simply by refusing a higher volume of requests, a pattern that appears safer in statistical aggregates but does not reflect a genuine underlying trait of harm avoidance.

Semantic Scholar — py-irt (Lalor & Rodriguez) semanticscholar.org

A scalable Item Response Theory library for Python, extending Lalor et al. (2016) and Rodriguez et al. (2021)‘s work showing that high accuracy scores do not always correlate with high latent ability and introducing ‘local efficiency’ metrics for per-item information.