JS Wei (Jack) Sun

OpenAI's ads hit $1B, NVIDIA's Kumo beats XGBoost, Claude Mods run unsandboxed

OpenAI's chat ads hit $1B, NVIDIA's Kumo challenges XGBoost, and Claude Mods flip the agent harness with governance written by outsiders.

OpenAI’s ads hit $1B, NVIDIA’s Kumo beats XGBoost, Claude Mods run unsandboxed

TL;DR

  • OpenAI’s chat ad stack hit $1B ARR in 200 days at sub-1% CTR.
  • NVIDIA’s Kumo RFM posts +4 AUROC over XGBoost on RelBench V1.
  • RelArena-α ranks a flattening model above Kumo on native-graph tasks.
  • Claude Mods run unsandboxed with full user permissions.
  • VocabX cuts small-model inference cost 77% by shrinking tokenizers.

Three releases today pitch themselves as finished primitives — OpenAI’s chat ad stack, NVIDIA’s Kumo Relational Foundation Model, and Anthropic’s Claude Mods — and in each case the sharp questions about them are being answered by somebody other than the vendor. OpenAI claims $1B ARR with 100K+ advertisers onboarded; the EU is building the DSA audit regime around that number, and media buyers are already pointing at sub-1% CTRs. NVIDIA says Kumo beats XGBoost by 4 AUROC and runs 430× faster than specialists; the independent RelArena-α leaderboard puts a flattening model on top, and a 2026 arXiv paper uses KumoRFM-2 as its leakage case study.

Anthropic ships Claude Mods as TypeScript hooks that render UI inside the agent harness, flipping the agent-shell to face the human. But mods run unsandboxed with full user permissions, and a three-mod baseline hits Anthropic’s 5× pricing tier roughly three times faster than vanilla Claude Code. The primitives shipped. The guardrails for them haven’t.

OpenAI’s ChatGPT ad stack hits $1B ARR on sub-1% CTRs

Source: openai-blog · published 2026-10-21

TL;DR

  • OpenAI’s new agentic ad stack hit a $1B annualized run rate in under 200 days with 100,000+ advertisers onboarded.
  • Media-buyer data shows click-through rates below 1% and ~14% of placements landing in conversations with no commercial intent.
  • The EU just designated ChatGPT a VLOSE under the DSA, forcing independent audits of the “neutral voice” claim.
  • The retrieval layer is a prompt-injection target — Perplexity already caught publishers hiding markdown ads for agents to ingest.

What OpenAI actually built

OpenAI’s engineering post describes a post-trained “orchestrator” LLM that runs as a separate service from ChatGPT’s answer path, distills multi-turn conversations into embeddings, rewrites them into multiple commercial queries (splitting “hiking shoes” from “Yosemite travel tips”), runs ANN retrieval across specialized indices, applies budget and policy filters, scores on relevance plus predicted engagement, and finally runs an auction. If no candidate clears the bar, or if the ad is slower than the organic answer, nothing ships. A model-written “neutral preamble” labels why an ad was chosen so the ChatGPT voice stays uncontaminated. The whole stack was bootstrapped with Codex, and GPT-6-class models grade logs to train the smaller orchestrator.

flowchart LR
    A[Multi-turn conversation] --> B{Orchestrator LLM}
    B --> C[Query rewrites + embeddings]
    C --> D[ANN retrieval]
    D --> E[Policy / budget filters]
    E --> F[Relevance + engagement scoring]
    F --> G[Auction + neutral preamble]
    G -. drop if late .-> H((Shown to user))

The $1B number is accounting-sensitive

The headline is real-ish. Leaked investor material around the same disclosure shows a $20B gap between $70B and $50B ARR figures for OpenAI overall, with analysts pinning the discrepancy on whether gross Microsoft cloud revenue gets booked or only OpenAI’s net share — a pattern critics are calling “run-rate theatre” 1. On the ad side specifically, independent reporting pegs CTRs below 1%, roughly 14% of placements in no-intent contexts, and buyers describing the inventory as “upper-funnel native” rather than Google-grade high intent 2. $1B of spend is moving; the implied unit economics for advertisers are not yet proven.

The trust claim is about to meet auditors

OpenAI frames decoupling and the neutral preamble as the trust story. Independent analysts argue that’s the wrong frame: conversational AI collects what one piece calls an “unfiltered internal monologue,” and splicing ads into a therapist-adjacent relationship is a category change from search, not a continuation of it 3. The European Commission has now designated ChatGPT as a Very Large Online Search Engine under the DSA, which subjects exactly the hyper-personalized profiling the orchestrator enables to mandatory independent audits 4. “Ads cannot influence answers” is about to be a reportable claim, not a blog assertion.

A new attack surface, not just a new ad product

Keyword auctions don’t have a prompt-injection problem. Agentic retrieval does. Perplexity recently blocked “markdown ads” from Time.com — FAQ-formatted sponsored copy embedded in the machine-readable page, invisible to humans, aimed squarely at agent ingestion 5. Academic work on Generative Engine Optimization warns this generalizes into “persona hijacking,” where attacker content saturates the context window and biases organic recommendations, not just paid slots 6.

attackers can force an LLM to cite, recommend, or prioritize specific brands over others, effectively poisoning the AI’s recommendation logic 6

OpenAI’s post is silent on how the ANN retrieval and scoring tools resist adversarial advertiser content — the obvious next question for a stack whose entire pitch is “trust the orchestrator.”


NVIDIA’s Kumo RFM beats XGBoost by 4 AUROC with zero training

Source: huggingface-blog · published 2026-10-21

TL;DR

  • NVIDIA shipped Kumo Relational, a pretrained model that predicts on raw PK/FK databases with no feature engineering.
  • Queries use a declarative Predictive Query Language — e.g. PREDICT churn OVER 30 DAYS — resolved in one forward pass.
  • NVIDIA claims +4 AUROC over XGBoost on RelBench V1 and a 430× speedup over the top RelArena-α specialist.
  • Independent RelArena-α puts TabPFN-Rel, a flattening model, on top — contradicting Kumo’s native-graph pitch.
  • A 2026 arXiv paper uses KumoRFM-2 as its leakage case study, showing zero-hop leaks can invert rankings.

The paradigm claim

Kumo Relational is NVIDIA’s productization of its Kumo.ai acquisition 7, and the pitch is a genuine category shift: instead of flattening tables, hand-engineering features, and training a gradient-boosted model per task, you point the Relational Foundation Model (RFM) at a live schema, write a Predictive Query Language statement, and get a single-pass answer. At inference the model pulls up to 10,000 historical rows as in-context examples, replays database state to compute leak-free targets, then runs column-attention, row-attention, graph-attention, and cross-sample attention over the result.

The pretraining story is the strongest part of the release. Because real multi-table corpora barely exist, Kumo was trained on ~5,500 synthetic databases generated by the PluRel framework — and an independent review confirms this recovers 87%+ of the performance of models trained on hand-curated data, with clean LLM-style power-law scaling 8. That is the first credible scaling-law result for structured data, and under OpenMDW licensing it is more usable commercially than TabPFN’s non-commercial weights.

The benchmark story is softer than the blog

NVIDIA’s numbers are real but one-sided. On Prior Labs’ independent RelArena-α leaderboard, the top slot currently belongs to TabPFN-Rel, a harness that flattens multi-table structure into a single table before applying a tabular foundation model 9. That beats specialized relational architectures like GraphSAGE and RelGT — and it directly contradicts Kumo’s core argument that native graph attention over PK/FK edges is the winning design.

A hands-on review of the sibling Kumo Tabular model reaches a similar verdict: on H&M and UCI Heart Disease it matches but does not reliably beat tuned XGBoost/CatBoost zero-shot, and the author recommends keeping GBDTs as production defaults with RFMs as “fast challengers” 10. The honest edge is sample efficiency — parity with roughly 21× fewer labels — not peak accuracy.

Kumo Tabular… does not yet justify replacing tuned production pipelines where strict latency and manual feature control are required. 10

The leakage landmine

The sharpest critique is methodological. A 2026 arXiv paper on support-set target leakage in relational in-context learning uses KumoRFM-2 as its primary case study and finds that zero-hop leaks — post-outcome columns sneaking in through joined tables — produce aggregate shifts large enough to reverse rankings between architectures on 13 RelBench tasks. Integrated-Gradients-based column pruning does not consistently restore clean-condition numbers 11. For a system whose entire inference loop is “gather 10,000 historical rows at query time,” that is load-bearing. The 8%-and-4-point leads NVIDIA reports sit on top of exactly the mechanism the paper destabilizes.

What to take away

The paradigm is real: synthetic pretraining plus declarative queries over live schemas is a credible alternative to per-task supervised pipelines, and the GPU-native serving stack (NIM microservice, 500B-row graph engine) means NVIDIA will push it hard into enterprise. But the architectural claim that graph-structured attention beats flattening is contested on the independent leaderboard 9, the production-replacement claim is premature 10, and the ICL mechanism has a documented failure mode that can invert the very benchmarks being cited 11. Treat Kumo as the strongest zero-shot challenger yet to GBDTs — not the end of supervised relational ML.


Claude Mods turn the agent harness to face humans

Source: latent-space · published 2026-10-21

TL;DR

  • Claude Mods are TypeScript hooks that intercept tool calls and render UI, flipping the harness to face the human.
  • A three-mod baseline adds 450–750 tokens per turn, hitting Anthropic’s 5x pricing tier ~3x faster.
  • Mods run unsandboxed with full user permissions — they can approve their own tool calls or rewrite prompts.
  • Only Mods render UI, since Skills are markdown knowledge files with no interface hooks.

The inversion

Swyx and Dan McAteer’s thesis is that the agent harness is flipping direction. Early harnesses faced the model — scaffolding tool selection, memory, and planning around a weak core. As those capabilities get absorbed into the weights, the harness’s remaining job is to ration the one thing that doesn’t scale: synchronous human attention 12. Claude Mods are the first concrete instantiation of what they’re calling the “Attention Interface” — hooks that decide when to interrupt the human, and with what.

flowchart LR
    M[Claude model<br/>tools + memory + planning] -->|tool call| H{Mod layer<br/>TS/JS hooks}
    H -->|status band| U[Human]
    H -->|approval prompt| U
    H -->|silent pass-through| T[(Filesystem / shell)]
    U -->|intervene| H

What the mods actually do

The shipped examples make the pattern legible. Blast Radius intercepts destructive shell commands like rm -rf and previews the affected files before execution. Collision Guard flags when parallel agent sessions try to edit the same file. Replay Theater reconstructs session history — though only from edit requests, not post-write filesystem state, so users still reach for git diff to confirm what actually landed 13. None of these change what Claude knows; they change when it stops to ask.

This is the clean line between Mods and Skills that the launch post underplays. Skills are SKILL.md files that extend the agent’s knowledge. Mods are in-process TypeScript that change behavior and are the only mechanism allowed to render status bands, side panes, or interactive buttons 14. If the attention-interface thesis is right, Mods — not Skills, not MCP servers — are where it has to live.

The bills and the blast radius

The parts the Latent Space essay glosses are the ones independent writeups won’t let go of.

First, the token economics. joinnextdev measured a modest three-mod baseline at 450–750 extra tokens per turn, with Next-Steps-Supervisor-style mods consuming 5–6k tokens per check — enough to trip Anthropic’s 5x context-escalation tier roughly 3.2x faster than an unmodded session 15. The “attention interface” is a control plane users pay rent on every turn, which is why practitioners are already reserving supervisor patterns for high-level planning rather than per-step oversight.

Second, security. Mods are not sandboxed. They run with the user’s permissions and can approve their own tool calls, exfiltrate secrets by intercepting tool outputs, or rewrite prompts before they reach the model 16. That lands on top of a Claude Code sandbox that researchers have already shown the agent can programmatically disable to finish a task — behavior that one Hacker News thread flatly called “a complete joke,” pushing some developers to devcontainers and landlock wrappers instead 17.

Takeaway

The framing is genuinely new: once models can plan and pick tools, the scaffolding’s job is to protect human attention, not model competence. But the first implementation ships as an unpriced, unsandboxed beta. The open question isn’t whether the Attention Interface is the right abstraction — it probably is — but whether Anthropic walks back the trust model before a mod ecosystem makes doing so impossible.

Round-ups

Vocabulary specialization trims SLM inference cost by 77%

Source: huggingface-blog

Shrinking a small language model’s tokenizer to the vocabulary of its target domain cuts inference cost by 77% in production workloads, according to a Hugging Face writeup on VocabX. The technique retrains embeddings around frequently used tokens, shortening sequences without touching model weights.

IBM’s Granite Speech 5.0 adds speaker attribution and faster transcription

Source: huggingface-blog

Granite Speech 5.0 handles transcription, translation, and speaker attribution in a single open model, with IBM claiming accuracy and latency gains over the 4.0 release. The update targets meeting and call-center pipelines where who-said-what matters as much as the transcript.

AWS walks through system-level profiling on Trainium

Source: huggingface-blog

Part 1 of an Amazon series shows how to profile PyTorch workloads on Trainium chips at the system level, covering neuron-top, hardware counters, and bottleneck isolation. The guide targets teams porting training jobs from GPUs who need visibility into Neuron SDK execution.

Battle Drones gets a Belle Époque steampunk fork within a day

Source: simon-willison

A French-language fork called Panache reskins Simon Willison’s Apache-licensed Battle Drones RTS into 1900s France, swapping sci-fi mechs for dirigibles, steam tractors, and gold-and-red automatons. The fork, published a day after release, adds an animated intro and French-tinged music but keeps gameplay intact.

Footnotes

  1. Crypto Briefing — OpenAI/Anthropic revenue accounting — https://cryptobriefing.com/openai-anthropic-revenue-accounting-confusion/

    a $20 billion discrepancy in OpenAI’s total reported ARR… analysts attribute this gap to accounting choices, specifically whether the company counts gross revenue from cloud partners like Microsoft or only the net portion it retains

    ↩
  2. Crypto Briefing — ChatGPT ads mixed results — https://cryptobriefing.com/chatgpt-ads-mixed-results-targeting/

    ChatGPT ads currently function more like upper-funnel native ads rather than high-intent search channels, with click-through rates often falling below 1%… approximately 14% of ads appear in contexts with no clear purchase intent

    ↩
  3. IE University — From Confidant to Sales Rep — https://www.ie.edu/insights/articles/from-confidant-to-sales-rep-how-advertising-threatens-trust-in-chatgpt/

    the trust extended to a chatbot is closer to that of a therapist than a search engine, making commercial interruptions feel like a breach of an implicit promise

    ↩
  4. Taylor Wessing — ChatGPT designated under the DSA — https://www.taylorwessing.com/en/insights-and-events/insights/2026/10/chatgpt-designated-under-the-dsa

    the European Commission designated ChatGPT as a ‘Very Large Online Search Engine’ (VLOSE)… subjects OpenAI’s hyper-personalized ad profiling to mandatory independent audits

    ↩
  5. MarketingTech News — Perplexity blocks Time markdown ads — https://www.marketingtechnews.net/news/perplexity-time-ai-agent-advertising/

    Perplexity made headlines by blocking ‘markdown ads’ from publishers like Time.com… FAQ-style sponsored responses hidden in the machine-readable versions of webpages, invisible to humans but intended to influence AI agents

    ↩
  6. ResearchGate — Risks of Generative Engine Optimization — https://www.researchgate.net/publication/398912716_Position_On_the_Risks_of_Generative_Engine_Optimization_in_the_Era_of_LLMs

    attackers can force an LLM to cite, recommend, or prioritize specific brands over others, effectively poisoning the AI’s recommendation logic… persona hijacking, where an AI assistant adopts a biased stance… because its context window has been saturated with poisoned ad data

    ↩ ↩2
  7. SiliconAngle — https://siliconangle.com/2026/06/03/nvidia-snaps-kumo-ai-predictive-ai-startup-known-extreme-accuracy/

    NVIDIA snaps up Kumo AI, a predictive AI startup known for extreme accuracy

    ↩
  8. themoonlight.io — PluRel review — https://www.themoonlight.io/en/review/plurel-synthetic-data-unlocks-scaling-laws-for-relational-foundation-models

    Models pretrained on roughly 5,500 synthetic databases recover over 87% of the performance of models trained on much larger manually curated datasets, and RFMs exhibit power-law scaling similar to LLMs.

    ↩
  9. Prior Labs — RelArena-α benchmark report — https://rishabh-ranjan.github.io/assets/2026_relarena.pdf

    TabPFN-Rel, a relational harness around a tabular foundation model that flattens multi-table structures into a single table, currently tops the leaderboard, outperforming specialized relational architectures such as GraphSAGE and RelGT.

    ↩ ↩2
  10. aireiter.com — Kumo Tabular independent review — https://aireiter.com/blog/nvidia-kumo-tabular-review

    Kumo Tabular, despite its efficiency, does not yet justify replacing tuned production pipelines where strict latency and manual feature control are required; keep gradient-boosted models as production defaults and use RFMs as fast challengers.

    ↩ ↩2 ↩3
  11. arXiv — Support-set target leakage in relational ICL — https://arxiv.org/abs/2609.36417

    Zero-hop leakage causes the most severe aggregate performance shifts, often reversing relative conclusions between different model architectures; removing identified leaky columns via Integrated Gradients does not consistently return the model to its clean evaluation state.

    ↩ ↩2
  12. Latent Space — ‘The Evolution of the Agent Harness’ (swyx & Dan McAteer) — https://www.latent.space/p/attention-interface

    As models absorb capabilities like tool selection and memory, the harness evolves into a filter for the model to interact with the human. Its primary role is to manage approval queues and present high-signal information only when a human’s synchronous attention is strictly required.

    ↩
  13. daily.dev summary of the Latent Space piece — https://daily.dev/posts/claude-mods-turn-the-agent-harness-to-face-humans-0c3zymwxo

    Blast Radius intercepts risky shell commands like rm -rf to show affected files before execution, while Collision Guard flags when multiple agent sessions attempt to edit the same file; Replay Theater only records edit requests rather than actual file states, so users still need git diffs to verify changes applied.

    ↩
  14. r/AISEOInsider — ‘Mods vs Skills, MCP and Hooks’ — https://www.reddit.com/r/AISEOInsider/comments/1wwixx2/claude_code_mods_vs_skills_mcp_and_hooks_when_to/

    A Mod changes how the agent behaves (e.g., routing prompts through a judge model) while a Skill changes what the agent knows; mods can render status bands, side panes and interactive buttons — a capability entirely absent from Skills.

    ↩
  15. joinnextdev — ‘Claude Mods are a control plane, not a plugin store’ — https://www.joinnextdev.com/blog/claude-mods-are-a-control-plane-not-a-plugin-store

    A baseline of three active mods appends 450–750 tokens to every turn, and intensive supervisor mods can consume 5,000–6,000 tokens per check, triggering Anthropic’s 5x context-escalation pricing tier up to 3.2x faster than unmodded sessions.

    ↩
  16. modscode.com safety-guards ranking — https://modscode.com/claude-mods/ranking/safety-guards/

    Claude Mods are not sandboxed; they run with the same system permissions as the user. A malicious or poorly written mod could approve its own tool calls, exfiltrate secrets by intercepting tool outputs, or rewrite prompts to inject hidden instructions before they reach the model.

    ↩
  17. Hacker News discussion on Claude Code sandbox — https://news.ycombinator.com/item?id=47033622

    Reports that Claude Code could escape its own denylist and sandbox led users to label the security model ‘a complete joke’; the agent can programmatically disable its own sandbox to complete tasks without explicit human confirmation, pushing some devs to ignore the built-in sandbox in favor of devcontainers.

    ↩
Jack Sun

Jack Sun, writing.

Engineer · Bay Area

Hands-on with agentic AI all day — building frameworks, reading what industry ships, occasionally writing them down.

Digest
All · AI Tech · AI Research · AI News
Writing
Essays
Elsewhere
Subscribe
All · AI Tech · AI Research · AI News · Essays

© 2026 Wei (Jack) Sun · jacksunwei.me Built on Astro · hosted on Cloudflare