Generated Docs as Drift Defense

#resources #resources/writing #resources/programming

Generated docs as drift defense

Reference pages drift the moment a flag changes and the author forgets to update prose. The only durable defense is to generate the reference from the source of truth and fail the build when they disagree.

The mechanism

In mxr:

Each CLI command has a --help snapshot test. The snapshot files live alongside the code.
A script reads the snapshots and regenerates each CLI reference page.
The build runs the snapshot tests; if --help output changed without the snapshot being accepted, the test fails.
CI runs npm run generate and npm run validate; if a generated page drifts from its snapshot, the build fails.

The result: every flag has a doc page, and no flag can ship without its doc page. The discipline isn't trust — it's a build error.

What stays hand-written

Generated reference is the floor, not the ceiling. Authors still curate:

The use: one-liner that says when to reach for this command
Three to four examples per command, from simple to composed
Recipes and "in real life" sections that connect commands

The generator handles the boring, mechanical part — every flag's name, type, default, and help string. The author adds the part that takes judgment: when to use it, what to compose it with.

Why three to four examples

One example is too few — the reader can't see composition. Six is too many — they're just restating reference. Three to four lets the page show the simplest invocation, the --format json | jq variant, the pipe-into-another-command variant when composable, and the --dry-run variant when mutating.

What this rules out

Hand-edited reference pages for things that should be generated. They look fine the day they're written and rot the day after.

A --help output that's accurate and a doc page that's three releases behind — that combination is worse than no doc page, because the reader trusts the doc page over the binary. Generated pages can't lie.

The pattern beyond CLIs

For apps: settings schemas generate the settings reference. URL routes generate the routing reference. Event types generate the webhook reference. Anywhere a type or schema is the canonical declaration, the doc page should be a view onto it, not a parallel description.