condense-json 1.0 shrinks LLM's log store, not its prompt tokens
Every URL the pipeline pulled into ranking for this issue — primary sources plus the supporting and contradicting findings each Researcher returned. Inline citations in the issue point back here.
Sources
condense-json 1.0 simonwillison.net
Release: condense-json 1.0 I’m trying to get braver at releasing 1.0 versions. This little library is a year and a half old now - I’ve applied some sensible and non-disruptive fixes and shipped the big 1.0 for it. Here’s an example of what it can do, lifted from the README: { “foo” : { “bar” : { “string” : ” This is a string with foxes in it ” , “nested” : { “more” : [ ” Here is a string ” , ” another with foxes in it too ” ] } } } } Combine that with a replacements object: { “1” : ” with foxes…
References
neura.market coverage of condense-json 1.0 neura.market
condense-json is positioned as a ‘smart reduce’ strategy that preserves the original data model while optimizing it for both storage and subsequent model consumption
the-ai-beat.com on LLM 0.32 conversation redesign the-ai-beat.com
By using hash-based IDs for individual messages, the database can now automatically deduplicate content… enables llm to represent conversations as trees rather than linear logs, allowing users to fork a conversation from any point without redundant data storage
steveash/hitchhikers-guide notes on llm 0.32 github.com
users should perform a manual backup using llm logs backup before upgrading, as older raw provider payloads (like log probabilities) are no longer persisted in the new format
Simon Willison, llm 0.32 rc1 post simonwillison.net
the llm logs command now merges data from both legacy and new tables to provide a unified history
photostructure.com on SQLite vacuum photostructure.com
in Write-Ahead Logging (WAL) mode, a VACUUM operation can temporarily double the disk footprint because the entire rebuilt database is written into the WAL file before being merged
simonwillison.net /llm/ tag notes on TOON-style formats simonwillison.net
TOON averaged 2,744 tokens compared to 4,545 for standard JSON—a 39.6% saving… critics argue that because LLMs are predominantly trained on JSON, any savings from custom formats must be weighed against potential degradation in reasoning
