JS Wei (Jack) Sun

Willison's llm-echo 0.5 ships the agent mocks OpenAI's SDK still lacks

Every URL the pipeline pulled into ranking for this issue — primary sources plus the supporting and contradicting findings each Researcher returned. Inline citations in the issue point back here.

← Back to the issue

Sources

llm-echo 0.5 simonwillison.net

Release: llm-echo 0.5 I’m working towards a new LLM release, and testing PRs caused me to revisit the llm-echo debug plugin. This plugin provides a “model” which doesn’t involve an LLM at all - it just repeats back the prompts that have been fed to it. The new release now updates the simulated token usage counts to take tool calls, system prompts, and reasoning tokens into account. Example usage: uvx —with llm-echo llm -m echo ‘hi there’ -o thinking 1 —usage Outputs: First I consider the prom…

llm-tools-datasette 0.1.1 simonwillison.net

Release: llm-tools-datasette 0.1.1 When I switched LLM over to httpx2 in LLM 0.33 I forget that this would break any plugins which used httpx without directly depending on it. llm-tools-datasette was one of those plugins, now fixed. Tags: datasette , llm

References

MockServer — LLM Response Mocking docs mock-server.com

MockServer returns byte-for-byte correct provider responses for OpenAI, Anthropic and Gemini, including full streaming SSE framing and tool call metadata, without network activity.

simonwillison.net — LLM releases series index simonwillison.net

A chronological series tracking every LLM core and plugin release, showing a cadence that accelerated through 2026 around reasoning traces, typed events and server-side tools.

simonwillison.net — llm-tool-use tag simonwillison.net

Model responses are no longer treated as simple text strings but as a stream of typed events, including tool calls and tool results.

simonwillison.net — model-context-protocol tag simonwillison.net

llm-anthropic now supports server-side tools like WebSearch, WebFetch and CodeExecution, invoked through the unified -T interface.

GitHub issue — openai/openai-agents-python #678 github.com

Users request first-class mocking hooks for tool calls and reasoning tokens because stubbing them manually is brittle and often fails on strict message-order validation.

simonwillison.net — LLM 0.x release notes (Apr 2026) simonwillison.net

Earlier LLM releases added the plugin surface for usage accounting and attachments that llm-echo 0.5 now simulates end-to-end.

Jack Sun

Jack Sun, writing.

Engineer · Bay Area

Hands-on with agentic AI all day — building frameworks, reading what industry ships, occasionally writing them down.

Digest
All · AI Tech · AI Research · AI News
Writing
Essays
Elsewhere
Subscribe
All · AI Tech · AI Research · AI News · Essays

© 2026 Wei (Jack) Sun · jacksunwei.me Built on Astro · hosted on Cloudflare