Qwen 3.8 burns 22K tokens on one pelican, Willison ships SVG-to-MP4

TL;DR

• Qwen 3.8 27B posts GPQA Diamond 89.2 and SWE-bench Pro 61.7 from a 17GB local file.
• Default xhigh reasoning spends 22,276 tokens and 21 minutes drawing a single pelican SVG.
• MTP speculative decoding already lifts throughput 1.6-2.24× on MLX within 48 hours.
• Willison’s in-browser tool converts animated SVGs to MP4 via 30MB of ffmpeg.wasm.
• Pelicans rank 6th of 8 animals in a 1,008-SVG study, undercutting the gaming claim.

Today’s two tech features share an unlikely pivot: Simon Willison’s pelican on a bicycle SVG test, the informal benchmark that half the industry now runs against every new model. Qwen 3.8 27B lands with genuinely Opus-tier scores from a 17GB local file, but its default xhigh reasoning mode burned 22,276 thinking tokens and 21 minutes to draw exactly one pelican — with a Hugging Face thread arguing the overthinking is a structural defect in ssm_conv1d, not a config knob. Meanwhile Willison himself shipped markdown-svg-renderer, an in-browser SVG-to-MP4 converter built because Twitter, Slack, and LinkedIn all reject the animated SVGs his LLM outputs produce. Attached to the release: a 1,008-SVG study showing pelicans rank 6th of 8 animals — evidence against the recurring claim that labs specifically game his benchmark. Two very different drops, one shared artifact.

Qwen 3.8 27B hits Opus-tier scores, overthinks by default

Source: simon-willison · published 2026-08-16

TL;DR

• Qwen 3.8 27B posts GPQA Diamond 89.2, LiveCodeBench 90.3, and SWE-bench Pro 61.7 from a 17GB local file.
• Default xhigh reasoning burned 22,276 thinking tokens and 21 minutes to draw one pelican SVG on an M5 Max.
• One Hugging Face post claims the overthinking is a structural defect in the ssm_conv1d layers, not just a config choice.
• MTP speculative decoding already hits 1.6–2.24× on MLX and ~72% on llama.cpp — patches shipped within 48 hours.

The 17GB miracle, verified

Alibaba’s Apache-2 licensed Qwen 3.8 27B lands a dense vision-capable model into the same benchmark neighborhood as Claude Opus and GPT-4: GPQA Diamond ~89.2, Agents’ Last Exam 42.9 (up from 27.3 on Qwen 3.6), LiveCodeBench v6 90.3, and SWE-bench Pro 61.7 ¹. Simon Willison, running the Q4_K_M quant on an M5 Max MacBook Pro and an NVIDIA DGX Spark, calls the fact that it fits in a 17GB file “a miracle” — and community reviewers on r/LocalLLaMA describe it as “basically Opus locally” on private coding evals that usually break sub-30B models ². The honest caveat: most numbers still trace back to Alibaba’s own eval harness. Independent leaderboard entries are thin two days in.

Why xhigh melts down

The model ships with reasoning_effort defaulted to xhigh, and it shows. Willison’s first pelican-on-a-bicycle SVG consumed 22,276 reasoning tokens over 21 minutes to produce 3,223 tokens of output. Asked for “an SVG of a circle,” it delivered an animated Bauhaus geometric study with concentric guide rings and a “restrained ambient motion” palette debate in the trace. A community reasoning-trace review clocks 15,000–40,000 thinking tokens on tasks as trivial as HTML clones, and users are already shipping blended low+xhigh chat templates that recover most quality at ~20% of the token cost ².

Willison frames this as a bad default. One Hugging Face discussion post pushes a stronger diagnosis:

Scale misalignments in the ssm_conv1d temporal-processing layers deliver a distorted temporal signal — the model “pre-worries” about failure risks and doesn’t know when to stop reasoning ³.

That’s a single technical claim from a discussion thread, not a confirmed root cause. But if it holds, dialing reasoning_effort down is a workaround, not a fix — and a point-release patch is the real remediation.

MTP is already outrunning the 72% number

Willison’s headline speed win — a 72% throughput bump on the Spark using llama.cpp’s --spec-type draft-mtp — is turning out to be the floor. The model’s native Multi-Token Prediction head, trained in rather than bolted on, unlocks bigger gains on other stacks. The MTPLX project reports ~1.6× on an M4 Mac mini and up to 2.24× on M5 Max with no external draft weights and no extra VRAM cost ⁴. On CUDA, an RTX 5090 mobile write-up measures 33–39% with --spec-draft-n-max 2 (median 50.9 t/s), but flags two gotchas: acceptance decays sharply on prose at higher n-max while coding stays resilient, and current MTP implementations require --parallel 1, so it’s single-slot only ⁵.

What it actually means

Community sentiment doesn’t crown Qwen 3.8 as a universal winner — it splits the field on axis. Gemma 3 27B still wins on creative-writing feel; Qwen 3.8 wins on agentic coding and long-context repo edits, but is markedly slower to first token because reasoning is always on ⁶. That’s the real shape of the release: a dense 27B that can drive coding agents, annotate images with pixel-accurate bounding boxes, and one-shot working web tools — but ships with a broken default and leans on the community to make it fast. Two days in, the community is delivering. The daily-driver question isn’t capability anymore; it’s whether Alibaba patches the xhigh collapse before the next point release lands.

---

Willison ships in-browser SVG-to-MP4 via ffmpeg.wasm

Source: simon-willison · published 2026-08-16

TL;DR

• Willison’s markdown-svg-renderer now converts animated SVGs to MP4 entirely client-side, loading ~30MB of ffmpeg.wasm on demand.
• Twitter, Slack, and LinkedIn reject animated SVG, killing his pelican-on-a-bicycle LLM outputs on arrival.
• WASM encoding runs 10-50× slower than native FFmpeg and can’t reach GPU media blocks.
• Pelicans rank 6th of 8 animals in a 1,008-SVG study, undercutting claims labs game the benchmark.

What actually shipped

Simon Willison’s markdown-svg-renderer takes a Markdown document — pasted in, or loaded from a CORS-friendly URL like a GitHub Gist — and replaces every embedded SVG code block with a tabbed panel: rendered preview, PNG, JPEG, raw code, and as of this week, MP4. The video tab inspects the SVG for SMIL/CSS animations, guesses a loop duration, rasterises frames in the browser, and pipes them through an FFmpeg build compiled to WebAssembly. Nothing hits a server.

The point is portability. Twitter, LinkedIn, and most chat platforms won’t render animated SVG. Willison’s pelican-on-a-bicycle prompts — his running informal benchmark for frontier LLMs — increasingly produce animated output that dies on arrival everywhere except a browser. The MP4 tab turns them into something you can actually post.

The ffmpeg.wasm tradeoff

Shipping 30MB of FFmpeg to the browser is not free. WebAssembly is CPU-bound by design, so a 1080p encode can run 10-50× slower than native FFmpeg, and the newer WebCodecs API — which taps hardware media blocks — is up to 15× faster than any WASM software encoder for H.264 or AV1 ⁷. Nobody would build YouTube on this stack.

For a short looped SVG animation, none of that matters. The community treats ffmpeg.wasm as a tool for “narrow, high-value jobs” ⁷, and a personal SVG-sharing utility is exactly that shape. The alternative — a backend that accepts arbitrary user SVG and shells out to FFmpeg — is a security nightmare Willison sidesteps entirely.

The frame-capture problem nobody mentions

The clever part isn’t the encoding; it’s getting frames out of an animated SVG in the first place. Canvas’ drawImage() on an animated SVG source captures only the initial state, and calling setCurrentTime() on the SVG element doesn’t refresh the bitmap the canvas reads from ⁸. The standard workaround is to walk the animation timeline, freeze computed values into static SVG markup at each timestep, and re-rasterise — fiddly enough that most “SVG to video” tools just don’t bother.

Why this tool matters beyond one hobbyist

The renderer has quietly become the viewing surface for a chunk of the LLM evaluation community. When Gemini-generated SVGs mysteriously failed in Chrome and Firefox but worked in Safari, users caught a bug in Willison’s own sanitiser that was stripping valid SVG attributes ⁹ — the tool is under enough real use to function as informal QA. And it sits alongside a small ecosystem of Markdown+SVG approaches, including marknative, which takes the opposite architectural bet: server-side native rendering with no browser or DOM in the loop ¹⁰.

Meanwhile the benchmark it serves keeps getting more defensible. Dylan Castillo’s 1,008-SVG grid study across eight animals and six vehicles found pelicans came in sixth — no evidence labs are training to the prompt ¹¹. The bird earned its spot the honest way.

Footnotes

1. Medium — ‘Qwen 3.8 27B vs Qwen 3.6 27B’ — https://medium.com/data-science-in-your-pocket/qwen-3-8-27b-vs-qwen-3-6-27b-a61868b5d79d

   GPQA Diamond ~89.2, Agents’ Last Exam 42.9 (up from 27.3 on 3.6), LiveCodeBench v6 90.3 and SWE-bench Pro 61.7 — placing the dense 27B in an Opus/GPT-4 tier on paper.

   ↩

2. r/LocalLLaMA — ‘Qwen 3.8 Q8 reasoning trace review’ — https://www.reddit.com/r/LocalLLaMA/comments/1vobpv4/qwen_38_q8_reasoning_trace_review_code_review/

   xhigh burns 15,000–40,000 reasoning tokens on tasks like simple HTML clones; community ‘high’ presets built by blending low+xhigh templates reportedly recover most quality at ~20% of the token cost.

   ↩ ↩²

3. Hugging Face discussion #76 on Qwen/Qwen3.8-27B — https://huggingface.co/Qwen/Qwen3.8-27B/discussions/76

   Scale misalignments in the ssm_conv1d temporal-processing layers deliver a distorted temporal signal — the model ‘pre-worries’ about failure risks and doesn’t know when to stop reasoning, which is why xhigh degenerates into loops even on trivial prompts.

   ↩

4. Hugging Face — Qwen3.8-27B-MTPLX-Optimized-Quality — https://huggingface.co/Youssofal/Qwen3.8-27B-MTPLX-Optimized-Quality

   Native MTP head enables speculative decoding with reported speedups of ~1.6x on M4 Mac mini and up to 2.24x on M5 Max, without the additional VRAM cost of an external draft model.

   ↩

5. r/LocalLLaMA — ‘Qwen36 27B Uncensored Heretic v2 native MTP’ — https://www.reddit.com/r/LocalLLaMA/comments/1t5yajb/qwen36_27b_uncensored_heretic_v2_native_mtp/

   On RTX 5090 mobile (24GB) MTP with —spec-draft-n-max 2 delivers a 33–39% speedup (median 50.9 t/s), but acceptance decays sharply on prose at n-max 4 while coding tasks stay resilient; MTP currently requires —parallel 1.

   ↩

6. r/LocalLLaMA — ‘Best LLM for mobile: Gemma vs Qwen’ — https://www.reddit.com/r/LocalLLaMA/comments/1p5d288/best_llm_for_mobile_gemma_vs_qwen/

   Gemma 3 27B still preferred for creative writing and ‘human-like’ prose; Qwen 3.8 wins on agentic coding and long-context repo edits but is markedly slower to first token because of always-on reasoning.

   ↩

7. konvrt.dev — ffmpeg.wasm vs WebCodecs (2026) — https://konvrt.dev/blog/ffmpeg-wasm-vs-webcodecs-2026

   Because WebAssembly is fundamentally a CPU-bound execution environment, encoding a 1080p video can be 10 to 50 times slower than native FFmpeg… WebCodecs can leverage the hardware’s dedicated media blocks, performing H.264 or AV1 encoding up to 15x faster than Wasm-based software encoders.

   ↩ ↩²

8. d3x0r gist — SVG/canvas snapshot behavior — https://gist.github.com/d3x0r/d19713181850cae0e524030a537988b5

   CanvasRenderingContext2D.drawImage() method typically captures only the ‘initial state’ or first frame of the SVG… setCurrentTime() … does not automatically refresh the image data used by the canvas.

   ↩

9. Hacker News discussion — https://news.ycombinator.com/item?id=49276909

   when users noticed that Gemini-generated SVGs failed to render in Chrome and Firefox while working in Safari, Willison identified and fixed a bug in his own software that was improperly stripping SVG attributes.

   ↩

10. marknative (liyown/marknative on GitHub) — https://github.com/liyown/marknative

    marknative provides a native engine that generates paginated SVG documents directly from Markdown without requiring a browser or DOM.

    ↩

11. Dylan Castillo — ‘Pelicanmaxxing’ study — https://dylancastillo.co/posts/pelicanmaxxing.html

    Researcher Dylan Castillo generated 1,008 SVGs across a grid of eight animals and six vehicles… pelicans actually ranked sixth out of eight animals, and no statistically significant ‘pelican-only’ performance boost was found.

    ↩