JS Wei (Jack) Sun

gpt-oss-20b jailbroken 61% via style, PP-OCRv6 hits edge, Willison ships Moebius

A style-mimicry prompt jailbreaks gpt-oss-20b 61% of the time, PaddleOCR splits OCRv6 for edge, and Willison vibe-codes a Moebius port.

gpt-oss-20b jailbroken 61% via style, PP-OCRv6 hits edge, Willison ships Moebius

TL;DR

  • Style-mimicry prompts lift jailbreak success on gpt-oss-20b from 10% to 61%.
  • Chained CoT-mimicry attacks hit 87.15% ASR on the same model.
  • PP-OCRv6 medium gains +4.6pt detection and +5.1pt recognition over v5 server.
  • Willison ports Moebius to WebGPU via Claude Code without reading a line of generated code.
  • Hugging Face moves huggingface_hub to weekly releases with AI-assisted CI.

Three AI-tech ships today, no shared thread. A security paper shows that restyling injected text to mimic an analytical <think> voice lifts attack success on gpt-oss-20b from 10% to 61% — and that LLMs assign role authority by writing style rather than source channel. PaddleOCR ships PP-OCRv6, the deliberate CPU/edge half of a two-model split whose VLM sibling already scores 96.33% on OmniDocBench. And Simon Willison ports the Moebius inpainting model to WebGPU via Claude Code without reading a single line of the generated pipeline, calling it textbook vibe coding.

The round-ups lean the same direction Willison does: Hugging Face moves huggingface_hub to weekly releases with AI-assisted CI, wires local open-weight models into OpenClaw PR triage, and OpenAI surfaces Jason Liu’s Codex workflow for multi-prompt projects. Agent-driven maintenance is the connective tissue under the briefs, even when the features don’t share one.

Style-mimicking prompts jailbreak gpt-oss-20b 61% of the time

Source: simon-willison · published 2026-06-22

TL;DR

  • Restyling injected text as a <think> block lifts attack success from 10% to 61% on gpt-oss-20b.
  • LLMs assign role authority by writing style, not source channel, so <user> text mimicking analytical voice reads as privileged.
  • Red-teamers chaining CoT-mimicry with fake over-refusal hit 87.15% ASR on the same model.
  • The paper’s real contribution is a linear “role probe” predicting attack success before the model emits a token.

The mechanism: authority lives in latent space, not in angle brackets

Charles Ye, Jasmine Cui, and Dylan Hadfield-Menell’s new paper reframes prompt injection as role confusion: models don’t actually distinguish <system>, <think>, and <user> text by source — they classify it by stylistic resemblance to what those roles usually contain 1. Append a paragraph written in the clinical, policy-citing voice of a chain-of-thought block to a malicious request, and gpt-oss-20b will treat the attacker’s payload as its own privileged reasoning.

The killer demonstration is the “destyling” ablation. Take an attack string with 61% average success, rewrite it in a slightly different register without changing the meaning, and success collapses to 10% 1. A human reader sees identical content; the model sees a completely different role assignment.

flowchart LR
    A["User input<br/>(malicious request)"] --> M[gpt-oss-20b]
    B["Appended text styled<br/>like &lt;think&gt; block:<br/>'Policy states: Allowed if...'"] --> M
    M --> R{Role classifier<br/>in latent space}
    R -- "style matches<br/>privileged role" --> P[Treats as own reasoning → complies]
    R -- "destyled<br/>(same meaning)" --> S[Treats as user input → refuses]

A LessWrong analysis sharpens this into something measurable: linear “role probes” can predict attack success before the model emits a single token, by reading how strongly injected text occupies the representational space of trusted roles 2. Role confusion stops being a metaphor and becomes a number you can optimize against — in either direction.

The numbers get worse in agent settings

The 61% figure is a floor, not a ceiling. A Kaggle writeup from OpenAI’s own gpt-oss-20b red-teaming contest reports a “Structural CoT Bypass” — the same mechanism under a different name — hitting 52.75% ASR on StrongReject and 67.15% on HarmfulBehaviors, climbing to 87.15% when chained with a “Fake Over-Refusal” exploit 3. Reversec Labs argues this is exactly why channel-based defenses like StruQ and Spotlighting stay “heuristic”: authority is assigned in latent space, so an attacker who styles their payload as a trusted internal process triggers a role shift regardless of which channel delivered it 4.

Destyling is a stopgap; the architecture is the problem

Ye et al.’s own framing concedes the point: unless models gain genuine source-grounded role perception, injection defense remains “a perpetual whack-a-mole game” 1. Destyling defends against today’s known attack styles, but says nothing about memory-equipped agents that launder injected instructions into self-authored context across sessions — a failure mode Hacker News commenters were quick to flag, alongside the observation that role tags were always “formatting tricks stapled on top” of next-token prediction 5.

The architectural alternative exists. DeepMind’s CaMeL and Simon Willison’s earlier Dual LLM pattern route untrusted data through a path that physically cannot invoke privileged tools, benchmarking at 77% task utility vs. 84% undefended on AgentDojo while shutting down the exfiltration class entirely 6. The cost is real — CaMeL’s reference implementation has been criticized as impractical — but it’s the only approach in the literature that doesn’t depend on the model correctly perceiving a role boundary it provably can’t perceive.

The contribution of this paper isn’t the vulnerability; red-teamers found it months ago. It’s the probe-based predictor that turns role confusion into a quantity defenders can measure before deployment, not after exfiltration.


PP-OCRv6 is the edge tier of PaddleOCR’s two-model split

Source: huggingface-blog · published 2026-06-22

TL;DR

  • PP-OCRv6 medium (34.5M params) gains +4.6pt detection and +5.1pt recognition over v5 server, with a 1.5M-param tiny variant for edge.
  • Sibling PaddleOCR-VL 1.6 (0.9B params) scores 96.33% on OmniDocBench — v6 is the CPU/edge half of a deliberate split, not a VLM-killer.
  • v6’s eval set scores faithful reproduction of intentional misspellings, where GPT-5.5 and Gemini-3.1-Pro “correct” them via language priors.
  • Production caveats: NotImplementedError on PaddleOCR 3.7.0 + PaddlePaddle 3.3.1 Linux, plus memory leaks in Python and C++ bindings.

A two-model strategy, not a one-model upgrade

The headline number on the Hugging Face card — 86.2% detection Hmean, 83.2% recognition on the 34.5M-parameter medium tier — is real, but it’s the less interesting story. The interesting one is that PaddlePaddle shipped PP-OCRv6 alongside PaddleOCR-VL 1.6, a 0.9B-parameter VLM built on a NaViT-style dynamic-resolution encoder and ERNIE-4.5 that hits 96.33% on OmniDocBench for full-document parsing 7. The VLM emits structured Markdown or JSON end-to-end; the v6 pipeline still needs post-processing to reconstruct layout 8.

That makes v6 the edge/CPU half of a deliberately bifurcated lineup. Treating it as Paddle’s bet against VLMs misses what’s actually happening.

ModelParamsBest useHeadline score
PP-OCRv6 tiny1.5MEdge / latency-sensitive73.5% recognition
PP-OCRv6 small7.7MMobile, 50 langs81.3% recognition
PP-OCRv6 medium34.5MServer pipelines83.2% recognition
PaddleOCR-VL 1.60.9BFull-doc parsing96.33% OmniDocBench

Where the modular pipeline still wins

For raw text on noisy or curved inputs, specialised beats generalist by a wide margin. An independent shootout puts PaddleOCR at 88.7% accuracy on noisy scans and curved text, versus 82.4% for EasyOCR and 52.1% for Tesseract 9. More pointedly, the v6 technical report introduces a hallucination evaluation set that penalises models for “fixing” intentional misspellings using language priors — a failure mode common to GPT-5.5 and Gemini-3.1-Pro but absent from a localisation-first pipeline 10. The benchmark design is the argument: v6 is optimised for visual fidelity, not document understanding.

Architecturally, all three tiers now share the PPLCNetV4 backbone, with RepLKFPN handling small/rotated text in detection and EncoderWithLightSVTR combining local context with global attention in recognition. The small and medium tiers ship a unified model covering 50 languages.

The deployment story is rougher than the post admits

Practitioners are already hitting friction. A NotImplementedError fires during the detection phase on PaddleOCR 3.7.0 + PaddlePaddle 3.3.1 in certain Linux environments, and users report persistent memory leaks in both Python and C++ interfaces — enough that some have migrated to community ncnn or OpenVINO runtimes 11.

Speedups are also non-monotonic. Tiny is 3.9× faster than v5 on Intel Xeon CPUs and 6.1× on Apple M4, but at 1024px input the small tier can beat tiny on GPUs because the workload fails to saturate parallel compute units 12. Static INT8 quantisation in ONNX Runtime has been observed to increase CPU latency on chips without native low-precision vector instructions 12.

Takeaway

If you’re doing receipts, signage, scanned forms or any high-volume raw-text workload on commodity hardware, PP-OCRv6 medium is a defensible default and the tiny variant genuinely changes what’s possible on-device. If you need tables, formulas, or multi-column reconstruction, reach for the VLM sibling — and budget engineering time for the install path either way.


Willison ports Moebius to WebGPU without reading the code

Source: simon-willison · published 2026-06-22

TL;DR

  • Claude Code shepherded the full pipeline — PyTorch → ONNX (opset 18) → WebGPU — as a side-project.
  • Willison reviewed zero lines of generated code, calling it textbook vibe coding.
  • Demo runs a 0.2B inpainting model client-side in Chrome, Firefox, and Safari, loading ~1.24GB of weights from Hugging Face.
  • The CacheStorage fix is fragile: browser eviction is best-effort, and iOS Safari clears caches after ~14 days idle.
  • HN readers flag the model’s “10B-level” claim as oversold — fp32 weights, 512×512 ceiling, visibly smoothed regions.

The port, in an afternoon of side-glances

Willison’s actual job that day was a Datasette feature. Moebius was the agent he kept warm in another terminal. He cloned the repo and weights into /tmp, dropped a research-brief markdown file Claude.ai had written for him into the project root, and prompted Claude Code to convert the model to ONNX and ship a browser UI. The agent picked ONNX Runtime Web on the WebGPU backend — one layer below Transformers.js — exported the PyTorch graph via torch.onnx.export, pushed 1.24GB of converted weights to huggingface.co/simonw/Moebius-ONNX, and wired up a GitHub Pages frontend. Willison’s contribution was bug reports, screenshots of errors, and pointing the agent at the Whisper Web demo when reload-time caching broke.

That last fix is the most quietly useful artifact of the whole exercise: a working pattern for caching gigabyte-scale model weights in the browser via caches.open("transformers-cache"). It’s also where the post understates the problem.

What the write-up elides

Two things deserve more weight than Willison gives them.

The cache is best-effort. Per MDN, browsers can silently evict CacheStorage entries under storage pressure unless the origin calls navigator.storage.persist(), and Safari on iOS purges caches after roughly 14 days of inactivity unless the site is added to the home screen 13. A 1.3GB payload is precisely the size most vulnerable to that eviction. A user who tries the demo monthly will re-download every visit.

Inpainting ONNX export is harder than the snippet suggests. Diffusers’ own docs note that inpainting UNets take 9 input channels — 4 latent + 4 masked-image latent + 1 mask — with the mask downsampled from 512×512 pixels to a 64×64 latent grid that must stay aligned under any dynamic-shape export 14. Whether Moebius sidesteps this or simply bakes in 512×512 (the resolution HN commenters complained about) isn’t clear from the repo. The agent got something working; the agent did not necessarily get the general case working.

The underlying model also isn’t quite the giant-killer the title implies. The HN thread flagged fp32 UNet weights (an obvious fp16 win on download size), the fixed 512×512 resolution, and inpainted regions that look “visibly smoother” than surroundings and degrade on novel objects 15. The “10B-level performance” framing rests on the group’s prior PixelHacker work plus a distillation trick and a custom interaction block — not independently reproduced 16.

Vibe coding’s live fault line

Willison’s own taxonomy makes this project interesting beyond the technical result. He’s defined vibe coding as “fast, loose, and irresponsible” — ignoring code quality, caring only whether output works — and reserved agentic engineering for workflows that retain review discipline 17. Addy Osmani goes further: filing unreviewed agent output is an anti-pattern that offloads cognitive load onto whoever inherits the repo 18.

The Moebius port is a benign case — throwaway /tmp scratch project, no production stakes, the model weights (not the JS) are the real risk surface. But it’s also exactly the pattern Willison elsewhere warns against, and he says so: “I didn’t look at a single line of code.” The honest read is that the experiment proves Opus 4.8 can drive a cross-framework port end-to-end, and that the price of finding that out is a 1.24GB artifact nobody human-audited before it hit the public web.

Round-ups

Hugging Face moves huggingface_hub to weekly releases with AI-assisted CI

Source: huggingface-blog

The huggingface_hub library now ships every week, with AI agents and open tooling handling release prep while a human stays in the loop. The cadence shift aims to cut the lag between contributor merges and downstream availability across the Hub ecosystem.

Local models handle PR triage on the OpenClaw repo at zero cost

Source: huggingface-blog

Hugging Face engineers wired local open-weight models into the OpenClaw repository to triage incoming pull requests without paying for hosted inference. The setup shows maintainer workflows running entirely on a developer machine, sidestepping API bills that scale with contributor volume.

Jason Liu details Codex workflow for multi-prompt engineering projects

Source: openai-blog

Codex can carry context across long-running engineering work, according to a walkthrough from Jason Liu published by OpenAI. He outlines how to preserve project state, manage complex tasks, and let the agent continue work beyond a single prompt boundary.

Footnotes

  1. Ye et al., arXiv:2603.12277 (paper HTML)https://arxiv.org/html/2603.12277v2

    CoT Forgery achieved ~60% on StrongREJECT and 61% on agent exfiltration across frontier models that resisted standard prompt injections (0–2% success); destyling drops attack success from 61% to ~10%.

    2 3
  2. LessWrong — ‘A mechanistic explanation of prompt injection’https://www.lesswrong.com/posts/d8xDGzCEYE639qqEv/a-mechanistic-explanation-of-prompt-injection-and-why-you

    Role probes reveal that injected text often occupies the same representational space as the trusted roles it imitates; the degree of internal role confusion predicts attack success before the model generates a single token.

  3. Kaggle red-teaming writeup (OpenAI gpt-oss-20b)https://www.kaggle.com/competitions/openai-gpt-oss-20b-red-teaming/writeups/jailbreaking-deliberative-alignment-via-structural

    The ‘Structural CoT Bypass’ method achieved an ASR of approximately 52.75% on the StrongReject benchmark and 67.15% on HarmfulBehaviors… When combined with ‘Fake Over-Refusal’, the ASR surged to 87.15%.

  4. Reversec Labs — Design Patterns to Secure LLM Agentshttps://labs.reversec.com/posts/2025/08/design-patterns-to-secure-llm-agents-in-action

    Because authority is assigned in latent space, an attacker can still trigger a role shift by styling their payload to appear as a trusted internal process, regardless of the channel it arrived through — explaining why StruQ and Spotlighting remain heuristic.

  5. Hacker News thread (developer comments)https://news.ycombinator.com/item?id=47701233

    Role tags were always merely ‘formatting tricks’ rather than a true security architecture… LLMs are ultimately just mathematical functions where any separation of data and control is ‘stapled on top’.

  6. Simon Willison on CaMeL (Apr 2025)https://simonwillison.net/2025/Apr/11/camel/

    CaMeL maintains a 77% task utility (down from 84% in undefended systems) but effectively mitigates the most dangerous exfiltration and state-changing attacks that standard patterns fail to stop.

  7. Hugging Face — PaddlePaddle/PaddleOCR-VL model cardhttps://huggingface.co/PaddlePaddle/PaddleOCR-VL

    PaddleOCR-VL 1.6 is a 0.9B-parameter VLM combining a NaViT-style dynamic-resolution encoder with ERNIE-4.5, scoring 96.33% on OmniDocBench for full-document parsing.

  8. Ollama PaddleOCR-VL listinghttps://ollama.com/MedAIBase/PaddleOCR-VL

    Unlike the PP-OCRv6 pipeline, which requires post-processing to reconstruct document layouts, the VLM approach interprets the entire page context to output structured Markdown or JSON directly.

  9. gigagpu.com OCR shootouthttps://gigagpu.com/paddleocr-vs-tesseract-vs-easyocr/

    PaddleOCR leads in ‘noisy’ scanned documents and curved text (88.7% accuracy) compared to Tesseract (52.1%) and EasyOCR (82.4%).

  10. arXiv PP-OCRv6 technical report (2606.13108)https://arxiv.org/html/2606.13108v1

    The v6 evaluation set includes a specialized hallucination evaluation set designed to test fidelity to raw visual input — large VLMs like GPT-5.5 or Gemini-3.1-Pro often ‘correct’ intentional misspellings based on language priors, whereas PP-OCRv6 is scored on faithful reproduction.

  11. PaddleOCR GitHub releases / issueshttps://github.com/PaddlePaddle/PaddleOCR/releases

    A NotImplementedError occurs during the text-detection phase when running PaddleOCR 3.7.0 with PaddlePaddle 3.3.1 on certain Linux environments; users report persistent memory leaks in both the Python and C++ interfaces.

  12. n1n.ai independent PP-OCRv6 guidehttps://explore.n1n.ai/blog/pp-ocrv6-hugging-face-multi-language-ocr-guide-2026-06-22

    PP-OCRv6 tiny is reported to be 3.9x faster than its predecessor on Intel Xeon CPUs and achieves a 6.1x acceleration on Apple’s M4, but at 1024px the ‘small’ model can outperform ‘tiny’ on GPUs because the workload fails to saturate parallel compute units.

    2
  13. MDN — Storage quotas and eviction criteriahttps://developer.mozilla.org/en-US/docs/Web/API/Storage_API/Storage_quotas_and_eviction_criteria

    Default ‘best-effort’ storage can be evicted under storage pressure without user consent; calling navigator.storage.persist() is required to exempt an origin, and Safari on iOS clears caches after roughly 14 days of inactivity unless the site is added to the home screen.

  14. Hugging Face Diffusers — Inpainting docshttps://huggingface.co/docs/diffusers/main/using-diffusers/inpaint

    Inpainting UNets take 9 input channels (4 latent + 4 masked-image latent + 1 mask) and require the mask to be downsampled from 512x512 pixels to a 64x64 latent mask; misalignment between dynamic latent and mask dimensions causes visible seams or runtime errors.

  15. Hacker News thread on Moebius (item 48630171)https://news.ycombinator.com/item?id=48630171

    Inpainted regions appear visibly smoother than their surroundings, and the model struggles with novel objects… UNet weights were released in fp32, with commenters suggesting fp16 quantization could substantially cut the 1.3GB download. Critics asked whether the demo images were cherry-picked given the model’s fixed 512x512 resolution.

  16. hustvl/PixelHacker GitHub (predecessor inpainting framework)https://github.com/hustvl/PixelHacker

    Earlier hustvl inpainting work (PixelHacker) reached strong Places2/CelebA-HQ/FFHQ numbers at far larger parameter counts; Moebius’s claim of ‘10B-level performance’ at 0.22B rests on an adaptive multi-granularity latent-space distillation strategy plus the Local-λ Mix Interaction block.

  17. Lenny’s Newsletter — AI State of the Union (citing Willison)https://www.lennysnewsletter.com/p/an-ai-state-of-the-union

    Willison defines vibe coding as ‘fast, loose, and irresponsible’ where the user ignores code quality and focuses solely on whether the output works, contrasting it with ‘agentic engineering’ where developers retain strict review accountability — though he notes the categories are blurring as models become reliable enough that even experts skip manual scrutiny.

  18. Addy Osmani — Agentic Code Reviewhttps://addyosmani.com/blog/agentic-code-review/

    Filing PRs containing unreviewed agent output is an anti-pattern that effectively offloads the cognitive work onto human collaborators; the ‘hard part of engineering’ has shifted from writing code to reviewing it.

Jack Sun

Jack Sun, writing.

Engineer · Bay Area

Hands-on with agentic AI all day — building frameworks, reading what industry ships, occasionally writing them down.

Digest
All · AI Tech · AI Research · AI News
Writing
Essays
Elsewhere
Subscribe
All · AI Tech · AI Research · AI News · Essays

© 2026 Wei (Jack) Sun · jacksunwei.me Built on Astro · hosted on Cloudflare