Skip to content

test(docs): fail when generated docs are out of date#4360

Open
reinkrul wants to merge 1 commit into
masterfrom
test/docs-up-to-date
Open

test(docs): fail when generated docs are out of date#4360
reinkrul wants to merge 1 commit into
masterfrom
test/docs-up-to-date

Conversation

@reinkrul

Copy link
Copy Markdown
Member

Why

Contributors who add/change a config option or CLI command must run make cli-docs, but it's easy to forget — the docs then drift from the code. This adds a unit test that regenerates the docs in-memory and fails if they differ from what's committed, so CI catches it.

Changes

  • Refactor docs/generate_docs.go: split the generators into render-to-memory functions (generatedDocFiles returns path → content); generateDocs() now just writes that map to disk. Output is byte-identical to before.
  • Test TestGeneratedDocsAreUpToDate: compares generatedDocFiles() against the committed files; on mismatch fails with "<file> is out of date; run \make cli-docs` and commit the result"`.
  • Regenerated server_options.rst, which had drifted from the policy.authzen.endpoint flag description (needed for the new test to pass).

Scope

  • Covers the files produced by the Go generator: cli-reference.rst, server_options.rst, server_options_didnuts.rst.
  • Does not cover README.rst — that's assembled by the separate rst_include step in make cli-docs, not by the Go generator.
  • Map-typed flag ordering (the old source of spurious diffs) is already handled by normalizeDefaultValue; verified the generator output is stable across runs.

🤖 Assisted by AI

Refactor the doc generators to render to memory (generatedDocFiles) and
add a test that compares the rendered output against the committed files.
This catches contributors who change a config option or CLI command but
forget to run `make cli-docs`.

Also regenerates server_options.rst, which had drifted from the
policy.authzen.endpoint flag description.

Assisted by AI
@qltysh

qltysh Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

1 new issue

Tool Category Rule Count
qlty Structure Function with many returns (count = 9): renderServerOptions 1

@qltysh

qltysh Bot commented Jun 18, 2026

Copy link
Copy Markdown
Contributor

Qlty


Coverage Impact

⬆️ Merging this pull request will increase total coverage on master by 0.83%.

Modified Files with Diff Coverage (1)

RatingFile% DiffUncovered Line #s
Coverage rating: F Coverage rating: A
docs/generate_docs.go88.0%55-57
Total88.0%
🤖 Increase coverage with AI coding...
In the `test/docs-up-to-date` branch, add test coverage for this new code:

- `docs/generate_docs.go` -- Line 55-57

🚦 See full report on Qlty Cloud »

🛟 Help
  • Diff Coverage: Coverage for added or modified lines of code (excludes deleted files). Learn more.

  • Total Coverage: Coverage for the whole repository, calculated as the sum of all File Coverage. Learn more.

  • File Coverage: Covered Lines divided by Covered Lines plus Missed Lines. (Excludes non-executable lines including blank lines and comments.)

    • Indirect Changes: Changes to File Coverage for files that were not modified in this PR. Learn more.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant