The Specification Is Not a Document
Loading article…
Pull for previous story
Imagine joining a software project a few years from now.
It is a large, mature system. It has thousands of business rules, years of operational experience, regulatory obligations, performance constraints, strange edge cases, customer commitments, and architectural decisions made by people who no longer work there.
You ask the obvious question.
"Where's the specification?"
Someone gives you access to it.
There is no SPEC.md. No 900-page requirements document. No canonical YAML file. No giant prompt that somebody feeds to an agent. There is not even a directory you can browse and reasonably call the specification.
So you ask how you are supposed to understand the system.
The answer is another question:
"What do you want to know?"
What is this system for?
Who depends on it, and what promises does it make to them?
Why does checkout retry a failed authorization three times?
Which parts of the system are affected by European data residency requirements?
What did production teach us in the last twelve months?
Which architectural decisions still depend on assumptions that are no longer true?
Each question produces a different view of the same underlying body of knowledge.
It is tempting to imagine that somewhere behind this interface there must still be a canonical specification: some master representation that contains the whole system in its definitive form.
But that may be the wrong assumption too. The durable knowledge of a system might not have one natural representation at all. Some of it may be formal, some executable, some observational, some textual. What matters is that these pieces can be identified, related, queried, and traced back to their sources.
I don’t think there is a single thing underneath all of this that a person could sensibly open and inspect from beginning to end.
You don’t read it because there is no useful order in which to read it.
That may turn out to be one of the more important changes in how we think about software in the generative era.
When we say that a specification should be human-readable, we usually mean that a human should be able to inspect it and understand what it says. That is obviously desirable. But we have traditionally achieved readability by imposing a particular presentation on the underlying knowledge.
A document has an order. Its author decides that one idea should come before another. A source file contains an ordered sequence of text. A codebase is more complicated: its files are normally organized into directories, packages, modules, repositories, and dependency structures.
Those structures are useful, but they impose a primary organization on information that often does not have one.
Consider a mundane rule in a payments system: a failed authorization can be retried up to three times within a particular interval.
Where does that fact belong?
It belongs to payments, but also to reliability. It may be constrained by the behavior of an external processor. A production incident may explain why the number is three. An architecture decision may explain why the retry happens asynchronously. An SLO constrains how long the process can take. A compliance requirement may prohibit retrying in some circumstances. An evaluation should reject future implementations that violate the rule.
A directory tree still makes us choose where something lives. Everything else becomes a reference, a duplicate, a search result, or a fact somebody is expected to know how to find.
Documents have the same problem. If I write an architecture document, I have to decide whether retries belong under Payments, Reliability, External Dependencies, or Failure Handling. Cross-references help, but I am still flattening a multidimensional set of relationships into an authored sequence.
The retry rule is connected to a requirement, an implementation, an evaluation, an external dependency, an incident, a decision, and the reason for that decision. Those relationships are themselves part of what we know.
Today, the person trying to understand the system reconstructs that graph.
Perhaps we should store the graph instead.
There is old precedent for this move.
In 1970, Edgar F. Codd published A Relational Model of Data for Large Shared Data Banks. One of the important ideas behind the relational model was independence from physical representation. Users should be able to reason about data without knowing the machine-level structure used to store it.
That idea now feels mundane because it won so completely.
Suppose I tell you I have a database containing twenty million customer records. You would not ask me to send you its storage files so you could read them. You might inspect its schema because you need to understand the vocabulary and relationships available to you, but then you query the contents.
Show me customers who haven't ordered in twelve months. Show me revenue by geography. Find accounts exhibiting this pattern. Join these facts to those facts. Give me the projection appropriate to the question I am trying to answer.
The database does not have a useful reading order, and nobody expects it to.
Software is full of databases, yet knowledge about the software itself still looks remarkably like a filing system:
README.md
docs/architecture.md
services/payment/
adr/017-retry-policy.md
openapi.yaml
terraform/
dashboards/payment.json
JIRA-1842Some of these artifacts describe what we intended. Some describe what we built. Some explain why we built it. Some record what happened after we deployed it. Some provide evidence that particular properties hold.
The engineer has to reconstruct the joins.
Building Information Modeling offers a second example. In Revit, plans, sections, elevations, schedules, and other views are derived from a shared building model. Autodesk's documentation describes a schedule as "another view of the model".
Despite what we lay-people tend to think, the floor plan is not the building model. It’s just one useful representation of it.
BIM is not proof that model-centric work eliminates inconsistency, maintenance cost, or interoperability problems. It does not. The building industry has spent years dealing with information exchange and semantic interoperability between modeling systems. The narrower point is enough: a mature engineering discipline has found it useful to distinguish the durable model from the particular human artifacts used to inspect it.
Software has tried to make this distinction too.
Anyone familiar with the history of software engineering should be suspicious by now.
There is a large graveyard of attempts to put a model above source code. The Object Management Group's Model Driven Architecture separated platform-independent descriptions of behavior from platform-specific models and implementations. The ambition was to preserve stable meaning while technology underneath it changed.
Charles Simonyi went somewhere related with Intentional Programming. In his 1997 USENIX talk, Intentional Programming: An Ecology for Abstractions, he described programs represented as abstract trees of intentions, with separate mechanisms determining how an intention appeared to a programmer and how it was implemented. Systems such as JetBrains MPS continue that projectional tradition today: the program is represented as an abstract syntax tree, and the editor presents a chosen view of that structure to the programmer.
This history is unusually tangible for me. Microsoft acquired Intentional Software in 2017, and the Intentional team ended up in the group I was part of. I spent significant time with Charles and the team after the acquisition.
So I don't look at Intentional Programming or MDA as naive predecessors who simply lacked modern AI. I met people who had spent years thinking deeply about separating intention from representation. Very smart people have been attacking versions of this problem for a long time.
The important question is why these approaches remained peripheral.
Part of the answer is the cost of maintaining the model.
A second authoritative representation is valuable only while it remains true. If engineers have to build the software and then separately maintain a model describing the software, maintaining the model becomes another job. Under pressure, the artifact that executes tends to win. The model drifts, or it becomes detailed enough that editing it starts to resemble programming in a less familiar environment.
Cheap code generation by itself does not solve this.
If regenerative software requires us to maintain an enormous hand-curated knowledge graph alongside the implementation, then we have rebuilt one of the central problems of model-driven development in a more fashionable database.
That objection needs an answer.
The answer cannot be that developers become full-time knowledge engineers.
The knowledge model has to emerge largely as a consequence of doing the work itself.
An incident produces observations, evidence, and decisions. A code review contains reasoning and rejected alternatives. An evaluation states something we care enough about to enforce. A production trace provides evidence about actual behavior. A customer escalation can reveal an undocumented dependency. A legal review introduces a constraint. An architectural discussion changes an assumption.
Most of this information already exists. The failure is that it remains scattered through meetings, code, tickets, chat, monitoring systems, documents, and people's memories.
Language models make extracting candidate structure from that exhaust cheaper than it used to be. Microsoft's GraphRAG work, for example, uses LLMs to extract structured knowledge from unstructured text collections as part of its retrieval pipeline.
But extraction is not authority.
A system may infer from an incident report that the timeout was changed from twelve seconds to seven after incident 1842. That can enter the model as an observation with provenance. It is a different act to declare that every future implementation must use a seven-second timeout.
The first is information extraction. The second is governance.
That distinction suggests a lifecycle rather than a bag of supposedly true facts. Knowledge can be observed, inferred, proposed, accepted, contested, superseded, or rejected. Different claims can carry different sources and levels of authority. Machines can do much of the extraction and connection work. Humans become most important at the points where evidence turns into intent.
Ratification cannot become a new approval queue for every inferred fact. Most observations should remain observations. Authority should be scoped, confidence should be explicit, and only claims that constrain future behavior should require deliberate promotion. Some claims may expire unless reaffirmed; others may inherit authority from an existing policy or decision. The goal is not to make humans approve everything the system learns. It is to make the transition from evidence to obligation visible.
If maintaining the durable model requires engineers to stop what they are doing and document it, the model will rot. If the model is mostly the structured exhaust of work the organization was already doing, with humans responsible for ratifying the pieces that constrain future behavior, it has a chance.
None of this eliminates the cost of deciding what kinds of things the system can know. Someone still has to establish vocabularies, scopes, authority rules, and enough shared structure for useful queries to work. The bet is not that this work disappears. It is that we can move from manually maintaining the contents of a second model toward maintaining the rules by which ordinary engineering activity continuously contributes to it.
The vocabulary itself also has to remain cheap to change. A knowledge substrate that requires a committee to define the universal meaning of Customer, Incident, or Dependency before anyone can record useful facts will fail for the same reason overdesigned ontologies failed before. Shared structure should emerge where interoperability and querying require it. Local concepts can remain local until there is a reason to generalize them.
The durable model should be strict about authority and loose about vocabulary.
There is still another problem. Wikis and ADRs can also be produced as part of ordinary work, and they still decay.
The difference I care about is that this model would have consequences for being wrong.
Earlier in this series, The Implementation Remembers argued that mature software contains knowledge the organization has forgotten. A validation may exist because of an outage five years ago. A retry may encode an undocumented property of a partner API. An apparently redundant ordering constraint may be the scar left by a race condition nobody remembers.
That creates a fundamental problem for regenerative software.
Suppose we delete a mature component and regenerate it from everything the organization believes to be the specification. The new component passes the known evaluations but behaves differently in production because the old implementation contained an important behavior that was never represented anywhere else.
We have discovered the difference between what the organization knows and what the implementation knows.
Regeneration is therefore more than a way to produce software. It can test whether the durable representation of the system is actually sufficient.
That gives the knowledge model something a wiki does not have: consequences.
A stale architecture page can sit unread for six years without causing a failure. A stale or incomplete constraint that participates in regeneration can produce the wrong implementation. An unknown production dependency can reveal itself when the regenerated artifact no longer preserves it. An evaluation failure can expose a contradiction between what we claim to want and what we have encoded.
This does not make model rot disappear. It gives us a mechanism for detecting some of it.
Of course, the test only works where regeneration actually happens. Knowledge surrounding a component that survives untouched for ten years can decay just as documentation does. This is one reason the pace layers of a regenerative system matter: regeneration cadence also becomes a cadence for testing whether the knowledge associated with that layer still describes reality.
The Deletion Test asked what would have to survive if we threw an implementation away. Evaluations Are the Real Codebase argued that externally verifiable behavior is more durable than the code that happens to satisfy it. The deeper consequence may be that repeatedly exercising our ability to regenerate is how we discover which pieces of knowledge have not yet escaped from the implementation.
The current codebase becomes a source of evidence about missing knowledge rather than the permanent home of that knowledge.
A mature regenerative system should be trying to move unique information out of its implementation until deleting the implementation destroys almost nothing the organization knows.
Once I started thinking about the specification this way, some surprisingly old work became relevant again.
In an essay based on a December 1997 talk, Tim Berners-Lee wrote about what he saw as the Web's unrealized collaborative potential. In Realising the Full Potential of the Web, he imagined a group maintaining a "running model — in the Web — of their plans and reasons."
Plans and reasons.
That phrase is remarkably close to what software organizations still have difficulty preserving.
The Semantic Web effort later developed machinery for representing knowledge as relationships rather than as one canonical human document. RDF represents statements in graph form; SPARQL queries patterns across RDF graphs. W3C's PROV vocabulary separately provides concepts for representing provenance, including entities, activities, agents, and their relationships.
I am not arguing that regenerative software should be built on RDF. That would turn an architectural argument into a premature technology choice.
What matters is the shape of the idea.
Imagine:
retry-policy
applies-to payment-authorization
maximum-attempts 3
motivated-by processor-rate-limit
changed-because incident-1842
protected-by retry-policy-evaluationincident-1842 can itself be connected to production evidence, customer effects, a discussion, and a decision. The external processor constraint can have a source and an effective date. The evaluation can be related to the property it protects. A rejected alternative can remain connected to the assumption that caused us to reject it.
Their adjacency is semantic, not textual.
This is also where the argument in Provenance Is the New Version Control becomes more concrete. Version control for regenerative systems cannot stop at remembering which line changed. It needs to preserve why a belief about the system changed, what evidence caused the change, what alternatives were considered, and what future behavior the decision is supposed to protect.
Michael Nygard's Architecture Decision Records were designed to preserve some of exactly this context in a lightweight form. ADRs record context and consequences because source code rarely tells us why an architectural choice was made.
An ADR is still a document, though.
If its important components become first-class parts of the system's knowledge, I can ask questions that are much harder to answer from a directory of Markdown files:
Which current behaviors were introduced by this decision?
Which alternatives did we reject?
Which assumption caused us to reject them?
Is that assumption still supported by production evidence?
Which evaluations protect the intended consequence of the decision?
Now provenance is queryable.
Once claims, sources, and reasons become first-class, another property of real systems becomes easier to represent.
Real organizations do not contain one consistent truth.
Product says customers may cancel for thirty days. Legal says fourteen. The current implementation says forty-five because nobody updated it. A support document promises sixty under a particular contract. Production evidence suggests that several important customers depend on the forty-five-day behavior.
A conventional specification creates pressure to resolve this into one answer before writing it down.
The durable knowledge model should be capable of representing the disagreement itself.
These are different claims made by different authorities, at different times, under different scopes, with different evidence. Resolving them is a governance problem. Hiding the inconsistency by placing each claim in a different document merely makes that governance problem harder to see.
Making disagreement explicit does not resolve it. It makes the organization confront the fact that resolution is an exercise of authority rather than a documentation problem.
So a representation might contain something like:
product-policy-44 asserts refund-period = 30 days applies-to consumer-product effective-from 2026-01-01 authority productlegal-opinion-19 asserts refund-period = 14 days jurisdiction ... authority legal
production-observation-817 observed refund-period = 45 days derived-from production confidence high
Now we have something useful to argue about.
The contradiction is no longer an accidental property of where documents happened to be stored. It is represented as part of what the organization knows.
This is one reason "specification" may ultimately be the wrong word for the thing I am describing. Specifications traditionally aspire to be consistent statements of intended behavior. Organizational knowledge is messier.
It contains uncertainty, disagreement, temporal claims, jurisdictional differences, customer-specific promises, observations that conflict with stated intent, and decisions that remain active even though nobody can establish whether their original rationale still applies.
A regenerative system cannot eliminate this mess by choosing a better serialization format. It can make the relationships explicit.
Formal methods give us another useful piece of the puzzle.
An Alloy model is a collection of constraints that implicitly describes a set of possible structures. The Alloy Analyzer searches for structures satisfying those constraints and can search for counterexamples to assertions.
That is a different conception of specification from pseudocode.
A specification does not necessarily have to tell us how to construct the program. It can define a space of acceptable programs.
This matters when implementations are generated.
The durable knowledge of a regenerative system should not casually insist that a future component be written in Rust, use Postgres, communicate through Kafka, or preserve today's service topology. Some of those things may truly be constraints. Most are current answers to deeper requirements.
That is the distinction I was trying to get at in When Does a Specification Become a Program? If we preserve enough implementation detail to mechanically reproduce today's program, we may simply have moved the program into another language.
The more interesting specification says what must remain true while leaving room for the next implementation to be different.
It also does not need one universal language. A formal invariant can remain formal. A schema can remain a schema. An evaluation can remain executable code. A business rule might begin as prose. A performance requirement can be a statistical threshold. A rejected architectural alternative may need several paragraphs of explanation.
Uniform syntax is less important than connection.
Can we identify the thing? Can we say what it concerns? Can we trace its source? Can we relate it to evidence, decisions, assumptions, and evaluations? Can we determine whether it is current? Can a machine retrieve it when regenerating the relevant part of the system?
The substrate can be heterogeneous while the knowledge remains connected.
Suppose I am a new engineer joining the payments team.
I do not need the whole specification. I need an introduction to the payments system.
Tell me why it exists. Explain who depends on it and what promises it makes. Show me its major concepts and dependencies. Walk through the important business rules in increasing detail. Include the architectural decisions that still constrain the design and the production incidents that most changed our understanding of it. Show me the evaluations I should understand before touching authorization.
That is documentation.
A security engineer asks a different question:
Show me the trust boundaries, credential flows, externally reachable interfaces, authorization assumptions, and unresolved security exceptions. Trace the claims to their sources.
That is documentation too.
Legal might ask which system behaviors derive from a particular regulation. An operator might ask which assumptions are contradicted by current production evidence. A product manager might ask what promises are unique to enterprise customers. An agent regenerating one bounded component might ask for every applicable constraint, evaluation, interface, and dependency.
These are not separate specifications.
They are projections.
Large language models make this practical because they can mediate between human questions and structured representations. But the model belongs in a particular place in the architecture.
The LLM should not be the specification.
If I ask why a retry limit exists, I do not want the authoritative answer to be whatever explanation the model happens to generate. I want the model to find the relevant claims, evidence, decisions, and provenance, then explain them to me.
The path looks more like this:
human question
↓
language model
↓
structured retrieval / query
↓
authoritative system knowledge
↓
facts + provenance
↓
language model
↓
useful human viewThe LLM is an interface to memory, not the memory itself.
We should also avoid pretending that natural-language querying has already been solved. Spider 2.0 was designed around 632 enterprise text-to-SQL workflow problems, often involving real-world databases with more than a thousand columns. In its original evaluation, an agent based on OpenAI's o1-preview solved only about 17 percent of the tasks.
Complex organizational knowledge will be at least as difficult.
The answer is not to make the LLM more authoritative. It is to make the underlying knowledge explicit enough that retrieval can be checked, provenance surfaced, and generated explanations traced back to their sources.
Earlier I argued in Production Is a Compiler Input that what a running system produces should become input to future regeneration.
That does not mean production should be allowed to rewrite intent automatically.
Production gives us evidence.
Suppose users almost always click a particular button twice. We have observed something. We have not established that double-clicking is desirable.
Suppose an old implementation has accepted a malformed request for six years. We have evidence that customers may depend on it. We still have to decide whether the next implementation should preserve it.
Suppose the 99.9th-percentile latency of an operation gradually increases to eight seconds. Reality has taught us something important, but it has not told us that eight seconds is acceptable.
A regenerative system cannot infer ought from is.
Production observations enter the model with provenance. Evidence can lead to a decision. A decision can change a requirement. A changed requirement can create or modify an evaluation. That evaluation then constrains future implementations.
Production is not the specification. Production teaches the specification.
The distinction also gives us a boundary for governance. Machines can observe aggressively. They can extract aggressively. They can propose relationships and detect contradictions. The transition from evidence to durable intent deserves more care because that is where we decide what future software will be required to do.
This closes the loop that The Implementation Remembers exposed. Production is where software continues to learn things its designers did not know. The challenge is making sure those lessons eventually escape from the implementation instead of becoming another mysterious conditional somebody will have to reverse-engineer ten years later.
For most of the history of software, source code became the authoritative description of a system for a practical reason.
It was the only representation detailed enough to reproduce the behavior.
Requirements were incomplete. Documentation drifted. Diagrams aged. People left. Tickets disappeared into archives. The implementation stayed because the implementation had to run.
So the code became memory.
Over time, the implementation becomes a lossy compression of everything the organization once learned.
We adapted an entire profession around that fact. When we want to know what the system really does, we inspect the implementation. When documentation and implementation disagree, the implementation wins. When nobody remembers why something behaves strangely, we perform archaeology on the artifact that survived.
That relationship makes sense while the artifact is expensive to reproduce.
Generative software changes the economics enough that we can ask whether the knowledge required to recreate the software should continue to live inside the thing being recreated.
I increasingly think the durable asset is a connected body of knowledge about what the system is for, who relies on it, what it promises, what must remain true, why those constraints exist, what alternatives were rejected, what we have learned from operating it, where that knowledge came from, how certain we are of it, where people disagree, and how a future implementation can be evaluated against it.
Code is one materialization of that knowledge. Documentation is a projection of it. Architecture diagrams are projections of it. Evaluations interrogate it. Production supplies evidence to it. Provenance explains how it came to contain what it contains.
Regeneration tests whether we captured enough.
When a regenerated implementation cannot reproduce something important, we have discovered knowledge still trapped in the old artifact. When a supposedly authoritative constraint repeatedly conflicts with current evidence, we have found something that deserves review. When two authorities disagree, we can represent the disagreement instead of accidentally hiding each claim in a different document.
None of this requires a One True Metamodel. It does not require every engineer to learn RDF. It does not require replacing formal methods, tests, schemas, prose, diagrams, or existing tools with one grand specification language.
It requires us to stop assuming that one of those representations must be the source.
None of this requires waiting for a complete regenerative platform either. A team can begin by treating decisions, evaluations, incidents, assumptions, and production observations as connected knowledge rather than unrelated artifacts, and by asking which of those relationships would need to survive if the implementation disappeared.
A future engineer arrives at a system and asks where the specification is.
There is no master file to open.
Instead she asks:
What is this system for?
Who relies on it?
Why does this behave this way?
What depends on this assumption?
What did we learn from the last failure?
Where do we disagree about what the system should do?
What would break if we changed this requirement?
Show me the architecture from the perspective of identity.
Show me the parts nobody fully understands.
Show me what changed in what we believe about this system between last year and today.
Generate a new implementation of this component that satisfies everything we currently know.
She never reads the specification.
There is no useful order in which to read it.
The codebase used to be where the system kept its memory. It doesn't have to be anymore.
The source of truth no longer has to be a source file.