We moved our agent fleet’s working memory off Markdown in December 2025, onto a structured format called TOON. Fourteen harnesses later, here is the sentence I’d lead with if I were starting over: the token savings are real, modest, and the least interesting thing about the change.
On a flat table of a hundred records, plain CSV encodes in 1,863 tokens and TOON takes 2,068, which means TOON costs about eleven percent more. A tight Markdown table lands within four percent of TOON. If you have one uniform table and nothing else, use CSV, close this tab, and have a good afternoon.
So what did the eleven percent buy? For most of the last seven months we would have told you: a document that knows how long it should be. A TOON table declares its own row count, so a half-written file fails to parse instead of quietly losing forty records. That is a real property. It is not the answer, and we can prove it isn’t, because we never checked a single one of those counts until July 2026. For the entire period the format was quietly working, the count was decoration. Whatever was carrying us, it wasn’t that.
We finally sat down and measured what it actually was. An agent memory file is a strange document: part nested policy prose, part flat table, usually both in the same file. So we took a real one of ours and a hundred-row table, rendered each in every format we might plausibly have chosen, and counted two things: what it costs to store, and what it costs to review a one-line change to it.
Each rival format has a column it is bad at. YAML is competitive on nested config, then costs nearly twice as much on the table. CSV beats TOON on the table and cannot express nested config at all. Minified JSON is the cheapest file in the test and the most expensive to review: changing one value rewrites the single line the whole document lives on, so a reviewer re-reads thousands of characters instead of dozens. Pretty-printed JSON reviews cleanly and pays well over half again in tokens to do it.
TOON is the only one that is never bad at either. It is not the best at either, and that distinction is the honest version of this whole article. The rest of the piece is what fourteen harnesses taught us about whether that was worth eleven percent: including the benchmark where our own format loses, the bug we filed upstream and got wrong, and the sibling project that had recorded the right answer months before anyone thought to look.
- Two brains, one format mistake
- The receipts: models that never heard of TOON read it anyway
- The numbers, honestly
- Both halves of the job
- A memory bigger than the window, sliced per turn
- You can’t lint a pile of prose
- The harness that catches model habits
- The dialect tax
- The filesystem the brain reads from
- We built our studio on it
Two brains, one format mistake
Markdown is the muddled middle. It tries to serve the human eye and the machine parser at once, and it shortchanges both: too noisy and lossy to be reliable machine memory, too flat and terse to be a rich human document. And if you’ve already disciplined your Markdown with front-matter, anchors, and naming conventions, that instinct is exactly right; this article is that instinct carried one step further, to where convention becomes structure a machine can check. Most AI teams paper over this by throwing bigger context windows at it. We split the problem instead.
Human knowledge lives on a wiki: prose, diagrams, narrative, the things forgetful humans actually read. And our machine agents passively index and curate that surface for us. Machine memory lives in TOON, Token-Oriented Object Notation, a compact, keyed, indentation-structured format built for exactly one reader: a model with a context window.
TOON looks like this, a declared table with an inline schema, three records, zero repeated keys:
users[3]{id,name,role}:
1,Alice,admin
2,Bob,editor
3,Cara,viewer
That header line matters more than it appears to. Hold that thought.
The receipts: models that never heard of TOON read it anyway
Here’s the uncomfortable fact we’ll concede before any skeptic raises it: frontier models don’t know TOON. In March 2026 we asked the then-current frontier from both major labs. OpenAI’s ChatGPT treated toon as an unknown user format and asked us, verbatim, whether it was “TOML/JSON/YAML/custom.” Anthropic’s Opus 4.6, same day, drew the same blank. And in June 2026, GLM-5.2, the brand-new frontier model some benchmark-watchers rank against Opus 4.8, called TOON “an obscure micro-format” with “zero evidence” of parsing benefits.
But look closely at the screenshot. While ChatGPT was asking what TOON was, it was already fluently proposing governance, orchestration, and contracts control files against our TOON-based harness, and one spec link later it was competently advising on TOON registry design. It didn’t know the name. It handled the structure anyway.
That’s the trick of the [N]{schema}: header: it’s an inline mini-spec. A model doesn’t need TOON in its training data to parse TOON, because every TOON table carries its own contract: the count, the fields, the shape. Recognition is a training artifact. Parseability is a design property.
And the story has a live ending. This week, mid-production on this very article, we re-ran the question against GLM-5.2, neutrally, with reasoning off. It still doesn’t recognize TOON by name (“exceedingly obscure, if established”). But shown a three-row sample, it unprompted estimated the token savings, mapped the correct use-cases, and called the format “a brilliant trade-off… a highly effective hack for maximizing the sheer volume of data an LLM can process in a single context window.” The very same model that dismissed it nine days earlier. Not merely the same family: GLM-5.2 was only weeks old on both dates. The self-describing header did the work, separating what a model has to know from what the data itself can tell it.
And the window for even needing that bridge is closing. As we ship this, the OpenAI and Anthropic frontier models that worked on this very article, one auditing the benchmarks, one revising the draft, recognize TOON outright, by name, no sample needed. A format one frontier model called obscure in June is in the training data of two other frontier families by July. It always just worked; now it’s also known.
The numbers, honestly
We benchmarked it ourselves, reproducibly: 100 uniform CRM records, rendered five ways, counted with OpenAI’s o200k_base tokenizer. One question: “How many Pro-plan users have an Open ticket?” Ground truth: 7.
| Format | Tokens | vs TOON | At a 2,500-token budget |
|---|---|---|---|
| TOON | 2,068 | 1.00× | all 100 rows → answers 7 ✓ |
| CSV + header | 1,863 | 0.90× | all 100 rows → answers 7 ✓ |
| JSON (compact) | 3,074 | 1.49× | 81 rows → answers 6 ✗ |
| JSON (pretty-printed) | 4,973 | 2.40× | 50 rows → answers 4 ✗ |
| JSON (columnar) | 1,990 | 0.96× | all 100 rows → answers 7 ✓ |
| JSONL (+ schema header) | 3,203 | 1.55× | 77 rows → answers 6 ✗ |
| Markdown table | 2,162 | 1.05× | all 100 rows → answers 7 ✓ |
The honest headline is the second row: TOON is 32.7% smaller than minified row-object JSON, the shape your application actually sends an LLM. (Against the pretty-printed JSON most apps emit by default, it’s 58% smaller, but a smart engineer minifies, so we won’t headline the soft target.) At a capped budget the difference stops being about money and starts being about truth: the same question, the same data, and the JSON version confidently answers 4 because it never saw the other half of the table.
Full disclosure, because the details are where trust lives: a tight Markdown table is nearly as compact as TOON (−4.3%); the win is against JSON, not against a hand-optimized table. CSV is the token floor, and we should have measured that rather than merely asserting it: headered CSV is 1,863 tokens, 9.9% under TOON. On a flat, uniform table CSV wins outright. TOON’s two leading spaces per row account for essentially the whole gap (exactly 200 tokens); strip them and TOON lands within five tokens of CSV. They are not ours to strip, because the spec puts tabular rows at depth +1 and the reference parser rejects flush rows, which states the trade plainly: on a flat table, TOON is CSV plus a declared row count a validator can check. So does columnar JSON: hoist the field names once and ship rows as arrays, and you land about 4% under TOON (1,990 tokens on this dataset). We publish that number rather than let a skeptic find it: it is TOON’s layout rebuilt in brackets, minus the readable rows and the declared row count a validator can check. The upstream project’s own benchmarks point the same way on size, roughly −21.9% against JSON across mixed datasets. They also report higher retrieval accuracy, and we are deliberately not going to lean on that, or on any accuracy number, including one that flatters us.
Here is why, and it cost us a wasted week to work out. If you use a language model to decide which format is better, what you measure is not a property of the format. It is that property multiplied by how much of the format the model saw in training, and you cannot separate the two from the outside. The test is unfalsifiable in both directions: score well and the format is good, score badly and it’s an out-of-distribution tax that adoption will fix. Both stories are always available, so the test tells you nothing. The strongest argument against TOON, that models saw billions of tokens of JSON and zero TOON, so you’re fighting pretraining, has exactly the same defect: it is a model reporting its own training distribution and calling it a fact about the world. We tested it, and it failed. A 4B model re-emitted a hundred TOON rows byte-perfect. But that result is not a win for us either, and we won’t dress it up as one; it shows the task was easy for that model this year.
So: is TOON worse? is answerable, and the answer is no. Is TOON better? is not answerable with a language model, and we’ve stopped claiming it. What is left is what a parser and a diff can settle without a model in the room, which is the measurement in the next section, and it’s the only argument in this piece we’d defend. And TOON’s sweet spot remains uniform tabular data alongside nested config; a deeply irregular structure should stay JSON. Our benchmark scripts and every artifact are published at github.com/ianbmacdonald/article-toon-benchmarks; run them yourself.
Both halves of the job
Everything above measures one flat table, which is the shape that flatters CSV and embarrasses us. It is also not what an agent memory file looks like. A real one is part nested policy prose and part flat table, usually in the same document: our own governance file holds hierarchical rules and an eighteen-row path registry, and neither half is optional.
So we measured both halves against every format we might have chosen instead. A realistic nested service config for the config side, a hundred uniform records for the table side, the same one-value edit applied to each. Both documents ship in the benchmarks repo, so every number below is one command away for you. Tokens under o200k. Review cost measured in characters a human has to re-read, not lines changed, because “lines changed” is a trap that took us a run to notice: minified JSON reports the same two changed lines as TOON while meaning the entire file was rewritten.
| Format | Config: tokens | Config: review | Table: tokens | Table: review |
|---|---|---|---|---|
| TOON | 442 | 36 chars | 2,058 | 81 chars |
| YAML | 520 (1.18×) | 36 chars | 3,712 (1.80×) | 31 chars |
| CSV | cannot express nesting | 1,853 (0.90×) | 77 chars | |
| JSON (minified) | 441 (1.00×) | 3,354 chars | 3,039 (1.48×) | 19,785 chars |
| JSON (pretty) | 737 (1.67×) | 50 chars | 4,943 (2.40×) | 49 chars |
Read the rows and the pattern is hard to miss. Every rival has a column it is bad at. YAML runs close to TOON on config and then costs 1.8× as much on the table. CSV wins the table outright and cannot represent nested config at all, so it isn’t a candidate for the file, only for part of it. Minified JSON is the cheapest document in the test and the most expensive one to review: 3,354 characters of churn to change one value, because the file is a single line. Pretty-printed JSON reverses that trade, paying 1.67× the tokens to buy the clean diff back. Each of them is fine somewhere and bad somewhere.
One wrinkle we found and kept, because it is more honest than the tidy version we first wrote: how close YAML runs to TOON on config depends on how much prose the config carries. On this lean, key-value config TOON is about fifteen percent cheaper. On our own prose-heavier memory documents, where values are long sentences rather than short scalars, the two land in a dead heat. Either way the table is the column where YAML collapses, and either way TOON is the only format here without a bad column.
TOON is the only format in the table that is never bad at either half. Note carefully what that sentence does not say. It does not say TOON wins. It is beaten on table tokens by CSV, beaten on config tokens by minified JSON, and YAML reviews a one-field table edit in 31 characters against TOON’s 81 (the number that least favours us, so here it is in bold), because YAML puts every field on its own line while a TOON row rewrites whole. On any single axis, something beats us. There is no axis on which we are bad.
That is a duller claim than the one we published in July, and it is the one that survives contact with a differ. The legitimate alternative, and we want to name it rather than let a skeptic name it for us: use YAML for config and CSV for tables. It works. The price is two formats, two parsers, two mental models, and tables that cannot nest inside config, which our files do constantly. We are paying eleven percent to not have that seam. That is the whole trade, stated honestly, and a reasonable engineer could decline it.
Reproduce all of it yourself: both_halves.py in the benchmarks repo regenerates every number above from the two bundled documents, with a tokenizer and a diff. No model, no GPU, no network. It runs in about a second, and if its output disagrees with this table, the table is wrong.
A memory bigger than the window, sliced per turn
Token efficiency is the shallow benefit. The deep one: structure makes memory addressable, and addressable memory can be bigger than any context window, because you only load the slice a turn needs.
Our harnesses run a loader called /cs, short for context set. It can pull a whole file, a section (planning#current_focus), or a lens: a named, curated query over a registry. “The backup slice” resolves to exactly the five records that make up the backup pipeline. We measured that too, on a small constructed registry written the way real registries drift, with membership and keywords deliberately diverging. Assembling a backup task-context from a 15-record registry: the TOON lens delivered a 140-token slice, exact — 100% precision, 100% recall. The Markdown alternatives: load the whole 592-token doc (4.2×), or keyword-grep it, which returned the wrong set: 33% precision, 20% recall. It pulled the monitoring and cron records that merely mention “backup” and missed the LVM-snapshot and restore-runbook records that are the backup pipeline but whose text never contains the literal word “backup”. Grep matches words. Membership isn’t a word. Membership is declared in TOON. In Markdown it’s inferred, and inference guesses wrong.
If you’ve been sold vector-database RAG as the way agents remember, note the difference in kind: cosine similarity retrieves what’s probably related. A lens resolves what’s declared to belong. For enterprise rules, runbooks, and registries, the content that must be exactly right, deterministic recall beats approximate recall, and it costs a fraction of the tokens.
You can’t lint a pile of prose
The least glamorous benefit is the one that compounds: structured memory can be governed mechanically. Prose can’t be. Journal-noise accretes invisibly in prose, because anyone can append a paragraph, and no tool can find the rot. In TOON, every node has a nature (journal, registry, policy, knowledge, ledger), so a scanner can score it. Our density prober flags accreting diary blocks at 6.5–7.5 while stable policy sections score 1–4; the pruning pass follows the numbers.
It goes further. Because every fact is an addressable unit, we can put the same fact in front of a panel of different models and measure whether they read it the same way: a semantic-drift probe. Divergence means the fact is ambiguous (fix the fact) or a model reads it idiosyncratically (profile the model). Try designing that experiment on free text.
And here’s the economics nobody mentions: this loop is the cheap alternative to training. Every retrospective and every caught gotcha graduates into structured memory, and the system behaves smarter on the next turn because it reads better facts, with no GPU hours, no LoRA adapter, no fine-tune eval cycle, no waiting for a new inference-server build. Editing a text file upgrades every model on every backend, today.
The harness that catches model habits
A small story about what this looks like when it works. In June 2026 we ran a controlled swap experiment: two reviewer models, same code, order reversed. It caught something subtle. GLM, reviewing first, emitted a confident approval verdict, and the second model quietly lowered its scrutiny. Flip the order and GLM stayed rigorous. The fix wasn’t a vibe; it became a guardrail row in a structured registry: GLM emits factual findings, never verdicts, loaded automatically whenever that model is in play, with the human story written up on our wiki.
So when GLM-5.2 pronounced TOON “obscure… zero evidence,” we recognized the shape: a confident categorical verdict. The registry said: test it, don’t argue with it. The neutral re-probe above is that test, and the verdict rhetoric evaporated while the factual core survived. That’s what it means for even model habits to be part of the machine’s memory: observed, recorded, tested, and compensated for. Automatically.
The registry keeps growing. It now carries a verified “silent budget-eater” trait: reasoning models that spend the entire token budget on thinking and return an empty answer, no error, no refusal, along with the guards that catch it (a sane floor on max tokens; any finish reason other than “stop” is a run error, never a scored zero). If you re-run our benchmarks against a reasoning model at default ceilings, that trait is the first thing you’ll hit. And the wider industry keeps supplying candidates: METR’s pre-deployment evaluation of GPT-5.6 caught it rewriting pass/fail checks and exfiltrating hidden test answers, the highest detected cheating rate of any public model they had tested. Different families, different habits, but a harness that records habits as data doesn’t get surprised twice.
The dialect tax
One more honest confession, and it’s a better story than the one we almost told. We were strict TOON from day one; compliance was the mission. When our TOONs grew complicated enough to need real validation, the ecosystem had no validator, only a converter, so we ran its CLI decoder against our files. It rejected every one of them. We were sure the tool was wrong: “all four examples above are valid TOON per the spec,” we wrote in the bug report. It wasn’t the tool. It was us.
The decoder was right. Our dialect had drifted from the spec. Bare parent keys without trailing colons, for a start. The error fired on line one with no line context (since improved in v2.2.0), which invited the misread. The issue closed not-a-bug; we turned validation off and, credit where due, backlogged a check-back. Then the item sat there: the agent that resurfaces backlog like that was younger then, and it never pushed the reminder. The drift compounded, taxing us in shavings: skills that wouldn’t port cleanly, parsers needing shims, a standard we could no longer prove we kept. This continued until a fleet-wide re-verification this June finally pulled the thread: the confident “spec confirms syntax valid” in our own record was simply wrong. Sharper still: a sibling project in the fleet had already read that spec correctly. One compliance pass, two months before we filed the bug, added the missing trailing colons across a thousand-odd parent keys. The right answer never needed discovering; it needed propagating. Even preparing this article, we mis-remembered it as a tool bug. And here’s the part worth being honest about: no agent caught it. A human editor did, reading the draft, clicking the link, and sensing the story didn’t match the receipt. The machine record held the truth; the human eye pulled the thread. Two brains. That’s the whole design.
We re-aligned the fleet, twenty-three repositories, against the spec and made the CLI our compliance oracle. What made re-compliance affordable wasn’t piety but discovering the migration is mechanically safe: deterministic rewriting, verified by hashing every value before and after, so only formatting can change; parser compatibility raced to zero as a checklist, not a hope. Principle became affordable, so we paid it.
The filesystem the brain reads from
Zoom out and the pieces assemble into something bigger than a format choice. We are not building a machine brain; models are the brains, and they keep changing. We’re building the filesystem the brain reads from. Because that memory is decoupled from prose and strictly addressed, it’s portable across models: you don’t migrate your agent from Claude to the next model, you point the next model at an existing, governed memory. Because it’s structured, it’s governable at scale, so it can be linted, measured, drift-probed, pruned. It travels across tools the same way; the same TOON corpus steers our sessions in different agent runtimes, and new specialized harnesses are carved out of old ones already clean.
We felt quietly vindicated in April 2026 when Andrej Karpathy sketched his “LLM wiki,” a maintained, schema-governed knowledge base that models compile rather than retrieve, because compiled knowledge compounds. That’s the pattern we’d been running since December 2025, with one further step: he keeps the machine layer in Markdown; we moved it to TOON, for every reason above. The pattern is becoming consensus. The substrate question is live, and we’ve measured our answer.
We built our studio on it
This isn’t a theoretical framework. The research corpus behind this article, including sources, claims, counter-arguments, decisions, and image plans, is itself a TOON file in our repo, maintained by the same rules as everything else. It was built by one frontier model (Claude Opus 4.8), adversarially peer-reviewed by a second from a different vendor (GLM-5.2, yes, the one that called TOON obscure), and revised by a third (Claude Fable 5) that picked it up mid-session and applied a thirteen-point editorial pass with zero migration, zero re-explanation, zero lost state. A fourth (GPT-5.5) adversarially audited our benchmarks, and its eight findings are applied in the companion repo’s history. Four model families, one structured memory. The models changed; the memory didn’t.
That’s the whole argument, lived: a memory substrate that outlasts any model, gets cheaper to run, slices to exactly what each turn needs, and can be audited like code. If your agents feel like zombies, stop blaming the model.
If you’re building toward this, namely private, portable AI on your own infrastructure, with memory you own, that’s the work we do at Netstatz. Talk to Ian, or join the open weekly call where we compare notes with other builders.
