Architecture#

score_coverage architecture
status: draft
security: NO
safety: ASIL_B
version: 1
realizes: wp__sw_implementation
doc__coverage_architecture
Generic Document

The pipeline hooks into Bazel’s two coverage extension points, --coverage_output_generator and --coverage_report_generator, and adds a justification and gating layer on top. It has two phases and, in phase 1, two backends that produce the same report zip:

  • llvm: Clang and rustc coverage mapping, read with llvm-cov; Linux host tests, C++ and Rust.

  • gcov: GCC and QNX QCC .gcda counters, read with the toolchain’s gcov; C++ only. On QNX the tests run inside QEMU through score_qnx_unit_tests, which carries the counters back to the host, and Bazel’s own per-test collector turns them into LCOV.

Phase 1: collection#

The llvm_cov bazelrc config, copied by the consumer from the integration workspace, does four things:

  1. Swaps the compilers. C++ is compiled with a hermetic Clang/LLVM toolchain instead of GCC; Rust with the Ferrocene toolchain of score_toolchains_rust, which has LLVM coverage tools attached. Both emit the same covmap format.

  2. Turns on instrumentation. --experimental_use_llvm_covmap plus the coverage feature for C++; rules_rust adds -Cinstrument-coverage to rustc once the toolchain declares coverage tools. Runtime counter relocation (-mllvm -runtime-counter-relocation via the cc_feature, -Cllvm-args=-runtime-counter-relocation for Rust) enables continuous mode so coverage survives abnormal termination. Rust branch regions need -Zcoverage-options=branch on a rolling Ferrocene.

  3. Installs the per-test tool. merger.py merges the test’s profraw files with llvm-profdata, records the instrumented objects, and zips both as the test’s coverage.dat.

  4. Installs the final tool. The consumer’s score_coverage_reporter target wraps reporter.py with the scope, workspace root and LLVM tool labels. The reporter merges all per-test profiles and runs llvm-cov three times: show (HTML), export (LCOV), report (text).

Scope. Covmap instruments everything. Filtering happens at report time through the allowlist written by score_coverage_scope: an aspect walks the dependency graph from the listed production targets and collects every source file a workspace target declares (including headers it vendors from an external repository). Everything else, test sources, googletest, external dependencies, headers a wrapper rule only forwards, is excluded. For headers exposed through strip_include_prefix / include_prefix the aspect also writes a path map from the generated _virtual_includes/ path the compiler records to the declared header, and it exports the source files themselves.

Sources. The reporter does not read sources through the workspace directory: generated headers and external repositories are not there at report time. It links every in-scope file from its runfiles into a staging directory laid out like the coverage mapping expects, points llvm-cov at that directory, and afterwards files the HTML pages under canonical paths so the archive is machine-independent and every index link resolves.

Baseline. A file that no test executes produces no profile data. The scope aspect therefore also collects the compiled archives and executables, and the reporter runs llvm-cov --empty-profile over them, so untested files show up with exact 0 % entries whose denominators come from the compiler’s coverage map. Rust rlibs are expanded into their object members first because their leading lib.rmeta member makes llvm-cov reject the archive.

Phase 1, gcov backend#

The gcov backend exists for toolchains that cannot emit LLVM coverage mapping: GCC on Linux and, the reason it was built, QCC on QNX. It changes the collection, not the report.

  1. Compilers. The coverage feature of the S-CORE GCC and QCC toolchains adds -fprofile-arcs -ftest-coverage; the compiler writes a .gcno notes file per translation unit and the instrumented binary writes .gcda counters when it exits. rustc cannot produce either, so Rust sources in scope are reported as not instrumentable on this backend.

  2. Execution and transport. On Linux the test writes its .gcda under Bazel’s COVERAGE_DIR. On QNX the test runs inside a QEMU micro-VM (score_qnx_unit_tests, --run_under); GCOV_PREFIX points the counters at a guest directory, the guest tars them at exit, and the host runner extracts the archive into COVERAGE_DIR. From there both are the same.

  3. Per-test collection stays Bazel’s. Bazel’s collect_coverage.sh runs the toolchain’s gcov over the counters and its lcov_merger writes one LCOV file per test. Two properties of that collector shape the backend: it maps generated _virtual_includes/ paths back to the declared header itself, and it keeps only files of its instrumented-files manifest, which never lists sources of external repositories, so a header vendored from an external repository has no data on this backend even when a test executes it. Tests are instrumented as well (--instrument_test_targets) so header-only code that only a test translation unit instantiates is measured; the scope allowlist still drops the test sources.

  4. Final report. gcov_reporter.py sums the per-test LCOV records per file (the same semantics as merging profiles on the LLVM side), applies the scope through the shared selection logic, and adds the zero-coverage baseline: gcov --json-format over the .gcno of every in-scope translation unit without its .gcda yields every line and branch at zero. The .gcno files come from InstrumentedFilesInfo through the scope aspect, listed in <name>_gcno.txt and carried in the reporter’s runfiles. gcovr renders the HTML from the merged data (one page per file, under the canonical path) and the text summary; LCOV and unmapped_files.txt are written as on the LLVM side.

Line semantics differ between the backends and are recorded in the integration ground truth: gcov counts only lines the compiler emitted code for (no closing braces, no unused inline functions), while LLVM’s mapping keeps unused functions at 0 %. The justification and gating layer is backend-agnostic; effective_coverage.py recognises gcovr’s HTML layout.

Phase 2: report generation and gate#

generate_coverage_html.py unpacks the HTML from the zip and, when a justification YAML is given, calls justify.py (YAML plus in-code markers to a manifest of justified lines) and effective_coverage.py (recolours justified lines, computes raw and effective figures, flags stale justifications, writes report.json and summary.txt). The markdown summary is written next, then the gate compares the unrounded gated percentage against COVERAGE_THRESHOLD. Optionally the HTML, LCOV, justification report and JUnit XMLs are assembled into an artifacts tree.

Exit code 2 is reserved for runs without a verdict, so a broken report, a bad threshold or a tool failure can never look like a pass.

Module and consumer split#

Lives in @score_coverage

Role

score_coverage/merger.py

per-test profraw to profdata; C++ objects_list.txt and Rust ELF manifest discovery

score_coverage/reporter.py

final merge, llvm-cov show/export/report, allowlist filtering, --empty-profile baselines, rlib expansion, path normalisation; the selection, staging and path helpers shared with the gcov backend

score_coverage/gcov_reporter.py

gcov backend: per-test LCOV merge, .gcno baselines, gcovr HTML

score_coverage/coverage_scope.bzl

the scope aspect and rule (CcInfo and CrateInfo)

score_coverage/reporter_wrapper.bzl, defs.bzl

the consumer-facing score_coverage_scope / score_coverage_reporter API

score_coverage/justify.py, effective_coverage.py, coverage_summary.py, generate_coverage_html.py

justification, summary and gating layer

//:enable_llvm_coverage_for_death_tests

cc_feature for continuous-mode profiling

Lives in the consumer repository

Why it cannot move

score_coverage_scope(deps = [...])

names the repository’s production targets

score_coverage_reporter(...)

carries the repository’s LLVM tool labels (or, with backend = "gcov", the toolchain’s gcov) and workspace root; one target per backend

coverage_justifications.yaml

reviewed, repository-specific engineering arguments

MODULE.bazel toolchain blocks

LLVM and Ferrocene pins are per-repository decisions

the coverage:llvm_cov and coverage:qnx bazelrc blocks

bazelrc cannot be imported across modules

Two wiring details make the external hosting work: every path in the generated reporter launcher uses rlocation form because it mixes files from the consumer (_main), score_coverage and toolchain repositories, and the baseline manifest is resolved against _main explicitly because its entries are consumer files.

Design decisions#

  • Report-time filtering on top of full instrumentation. The scope allowlist decides what the report shows; --instrumentation_filter is set to the module’s root package (^//score[/:]) so that every target is compiled with counters. Bazel’s guessed default covers only the packages of the test targets: a target outside it is compiled without counters unless a direct dep is instrumented (both backends), and on the gcov backend Bazel’s collector additionally drops the counters of every target outside the filter. The reporters warn when files without test data sit in a directory that is tested from a test/ or tests/ subdirectory (ERR-13).

  • Fail loud, never fail green. Every input problem ends in exit 2. The gate compares unrounded values and floors displayed percentages.

  • Gate on the LCOV, not on llvm-cov’s text summary. The text summary omits baseline-only files; the LCOV includes them.

  • In-process tool calls. generate_coverage_html imports the justification tools instead of nesting bazel run; this keeps one process, one exit code and testable seams.

  • One report format for both backends. The gcov backend replaces only the final report step and produces the same zip layout, so phase 2, the archive, the job summary and the qualification evidence are shared. Bazel’s per-test gcov collector is kept as it is: it is where the QNX transport hands over, and re-implementing it would move the .gcda handling into the qualified tool for no gain.