Keeping every document revision can be surprisingly cheap

Instead of designing a clever sequence of text diffs, put every previous version of a document into a JSON array and compress the whole thing.

That was Simon’s idea during a dog walk. He talked it through in voice mode, then asked a coding model to build experimental Python prototypes.

In the resulting synthetic test, 1,000 revisions occupied 20.4 MB uncompressed and 80.3 KB as a Zstandard-compressed array. Repeated material across versions gave the compressor plenty to work with.

There is an obvious catch: rewriting one increasingly large blob on every edit gets expensive. The prototype therefore also tested chunks, sealing each after 128 revisions or 3 MB of uncompressed JSON. Both implementations retained timestamps and used atomic database updates.

Those numbers describe simulated revisions, not a guarantee for arbitrary documents. But the experiment is a useful reminder that an ordinary compressor can sometimes replace a much more elaborate storage design.