7. Step 7 — Add coverage and verification reports#

What it unlocks

Consolidated reports — Your module shows up in the consolidated coverage and verification / status reports that are published alongside the docs — the evidence side of the integration.

Note

No extra action is required here. As already explained in Step 3, the test_and_docs workflow already executes your module’s unit tests and gathers and publishes the code-coverage data. If your module passes Step 3, it is automatically part of the reports described below — this chapter only explains how that collection, aggregation and publishing happens.

7.1. How coverage is collected#

The test_and_docs workflow runs scripts/quality_runners.py, which iterates over every module under modules.target_sw in known_good.json and, per module, runs its unit tests with coverage (bazel coverage) and then extracts a per-language summary. The driver is the module’s metadata.langs:

  • "cpp" → a C++ (lcov/genhtml) coverage run.

  • "rust" → a Rust coverage run.

Both write into the two consolidated reports docs/verification_report/unit_test_summary.md and docs/verification_report/coverage_summary.md. The same scoping metadata applies to both languages:

  • metadata.code_root_path — the target pattern that is instrumented and tested (@score_my_module<code_root_path>).

  • metadata.extra_test_config — extra --flag settings for the coverage run.

  • metadata.exclude_test_targets — targets excluded from the run.

So for most modules no extra wiring is needed — declaring the language in metadata.langs (Step 1) is what enrolls the module into coverage.

7.2. Where the results are exported#

quality_runners.py does not only run the tests and coverage — it parses the output per module and exports the results in three forms:

  • Markdown summaries committed into the docs. A unit-test execution table and a coverage table are written to docs/verification_report/unit_test_summary.md (columns: module, passed, failed, skipped, total) and docs/verification_report/coverage_summary.md (columns: module, lines, functions, branches). These two files are pulled into the published documentation by docs/verification_report/platform_verification_report.rst, so every module’s unit-test and coverage numbers appear in the Platform Verification Report on the docs site.

  • Detailed HTML coverage reports. The per-module genhtml / Rust coverage HTML is written under the coverage output directory (artifacts/coverage/cpp/<module> and artifacts/coverage/rust/<module> by default; override with --coverage-output-dir).

  • CI job summary. In CI the test_and_docs workflow additionally appends the two markdown summaries to the GitHub Actions job summary ($GITHUB_STEP_SUMMARY), so the pass/fail and coverage tables are visible directly on the workflow run.

7.3. C++ coverage#

If your module declares cpp in metadata.langs, quality_runners.py runs bazel coverage over @score_my_module<code_root_path> and produces an HTML/lcov report via genhtml (the lcov package must be present — CI installs it; locally sudo apt-get install -y lcov). The line/function/branch numbers land in coverage_summary.md. To make your module’s C++ coverage meaningful:

  1. Ensure metadata.langs contains "cpp" and metadata.code_root_path points at the instrumented source root.

  2. Use metadata.exclude_test_targets to drop tests that should not count (e.g. examples, flaky or environment-specific targets).

  3. Reproduce a single module’s run locally with:

    python3 scripts/quality_runners.py --modules-to-test score_my_module

7.4. Rust coverage#

If your module declares rust in metadata.langs, the rust_coverage_report target is generated automatically into rust_coverage/BUILD by step 1.2 — no manual action is needed. Tune metadata.exclude_test_targets and metadata.rust_coverage_config in known_good.json to control what is measured. Build a single module’s report with:

bazel run //rust_coverage:rust_coverage_score_my_module

7.5. Component coverage in the documentation#

scripts/module_tests.py uses --config=llvm_cov and exports reports through @score_coverage//:generate_coverage_html for the integrated Baselibs, Persistency and Logging releases. Other modules run their unit tests without coverage until a release using coverage_tool is integrated; the activation points are commented in COVERAGE_MODULES in the script.

The export uses COVERAGE_THRESHOLD=0 to collect evidence without adding a percentage gate. It contains plain LCOV at coverage/coverage_report.dat and the detailed HTML report. Source paths are translated to the reference integration’s repository identities before upload; for example, score/foo.cpp in Baselibs becomes external/score_baselibs+/score/foo.cpp.

The report command combines these files into coverage-report/coverage_report.dat, which docs-as-code reads when the published documentation is built. Component coverage uses only the primary_need_id and code_targets already exported by the pinned modules. Missing ownership or measurements remain unavailable; the integration does not add component metadata through patches.

To include coverage in a local documentation build:

python3 scripts/module_tests.py run score_persistency
python3 scripts/module_tests.py report _module_tests/reports
bazel run //:docs_shim

7.6. Verification & status reports#

The consolidated reports live under docs/verification_report, the roadmap under docs/s_core_v_1/roadmap and the release-wide progress overview next to the release notes under docs/s_core_v_1/releases. Your module appears in the platform and module verification reports as soon as its documentation is part of the build; add it to scripts/progress_overview/collect_metrics.py to give it a colour slot in the per-release progress charts of docs/s_core_v_1/releases/progress_overview.rst. The full catalogue of reports is under Reports.