Tool requirements#

Every requirement satisfies a use case or a process requirement and carries the potential errors it mitigates as tags. The verification report lists the tests that verify each requirement.

Tool requirements overview#

ID

Title

Tags

Implemented

tool_req__coverage_artifacts

Artifacts tree

archive; ERR-09

YES

tool_req__coverage_backend_select

Backend selection

report; gcov; ERR-08

YES

tool_req__coverage_eff_branch_only

Branch-only justifications

effective; ERR-05

YES

tool_req__coverage_eff_gcovr

gcovr HTML reports are supported

effective

YES

tool_req__coverage_eff_html

Justified lines are visible in the HTML

effective

YES

tool_req__coverage_eff_metric

Effective coverage metric

effective; ERR-05

YES

tool_req__coverage_eff_path_match

Justifications match files at path-component boundaries

effective; ERR-04; ERR-06

YES

tool_req__coverage_eff_stale

Stale justifications are reported and not counted

effective; ERR-04

YES

tool_req__coverage_gate_exit_codes

Exit codes

gate; ERR-03

YES

tool_req__coverage_gate_metric

Gated metric

gate; ERR-01

YES

tool_req__coverage_gate_no_verdict

Broken input yields no verdict

gate; ERR-03; ERR-08

YES

tool_req__coverage_gate_threshold

Threshold from the environment

gate; ERR-03

YES

tool_req__coverage_gate_unrounded

Unrounded comparison

gate; ERR-03

YES

tool_req__coverage_gcov_baseline

gcov backend baseline from gcno notes

report; gcov; ERR-01

YES

tool_req__coverage_gcov_html

gcov backend HTML and summary through gcovr

report; gcov

YES

tool_req__coverage_gcov_merge

gcov backend merges per-test LCOV records by summation

report; gcov; ERR-02

YES

tool_req__coverage_instrumentation_hint

Instrumentation filter hint

report; ERR-13

YES

tool_req__coverage_just_markers

In-code markers

justification; ERR-04

YES

tool_req__coverage_just_missing_file

Justified locations exist

justification; ERR-04

YES

tool_req__coverage_just_platform

Platform filter

justification

YES

tool_req__coverage_just_unknown_id

Unknown marker ids do not justify anything

justification; ERR-04

YES

tool_req__coverage_just_yaml

Justification YAML is validated

justification; ERR-04

YES

tool_req__coverage_merge_no_data

A test without instrumentation produces no coverage output

collection

YES

tool_req__coverage_merge_profraw

Per-test profiles are merged with llvm-profdata

collection; ERR-02

YES

tool_req__coverage_merge_tool_error

A missing or failing llvm-profdata fails the test's collection

collection; ERR-02

YES

tool_req__coverage_report_allowlist

The report is restricted to the scope allowlist

report

YES

tool_req__coverage_report_baseline_zero

Untested in-scope files appear at exact 0 %

report; ERR-01

YES

tool_req__coverage_report_merged_profile

One merged profile for all tests

report; ERR-02

YES

tool_req__coverage_report_missing_baseline

A missing baseline object is an error

report; ERR-01

YES

tool_req__coverage_report_outputs

Report contents

report

YES

tool_req__coverage_report_relative_paths

Report paths are workspace-relative

report; ERR-06

YES

tool_req__coverage_report_rlib_expansion

Baseline archives are reduced to members with a coverage mapping

report; ERR-07

YES

tool_req__coverage_report_unmapped

In-scope files without any coverage data are listed

report; ERR-01

YES

tool_req__coverage_scope_baseline_objects

Baseline objects accompany the scope

scope; ERR-01

YES

tool_req__coverage_scope_excludes

External and generated sources are excluded from the scope

scope

YES

tool_req__coverage_scope_gcno

gcno notes files accompany the scope

scope; gcov; ERR-01

YES

tool_req__coverage_scope_transitive

Transitive in-workspace sources define the scope

scope; ERR-01

YES

tool_req__coverage_summary_first

Summary is written before the verdict

summary

YES

tool_req__coverage_validation_ground_truth

Ground-truth validation

validation; ERR-02; ERR-07

YES

Scope#

Transitive in-workspace sources define the scope
safety: ASIL_B

score_coverage_scope shall collect, for the listed targets and their transitive in-workspace dependencies reached through deps, implementation_deps, exported_deps, components and implementation, the checked-in source and header files of cc_library and rust_library targets (srcs, hdrs) and the CrateInfo sources of rust_binary targets, and shall write them sorted and deduplicated, one canonical path per line, to the allowlist file: the workspace-relative path of a main-repository file, or external/<repo>/<path> for a header a workspace target declares from an external repository. For a header a workspace target exposes through strip_include_prefix or include_prefix it shall additionally record, in the path map, the generated <pkg>/_virtual_includes/<target>/... path the coverage mapping names together with the declared header it stands for, and it shall export the listed source files themselves (source_files output group) so the reporter can read them independently of the workspace directory.

External and generated sources are excluded from the scope
safety: QM
tags: scope
implemented: YES
version: 1

score_coverage_scope shall not list files of external targets and shall not list generated files, with one exception: headers a workspace target declares in its own hdrs from an external repository (see Transitive in-workspace sou... (tool_req__coverage_scope_transitive)). In particular, the public headers a workspace target merely inherits by forwarding another target’s CcInfo (a wrapper rule around a third-party library) shall not enter the scope, and a _virtual_includes/ path shall be mapped only when the target itself generated it.

gcno notes files accompany the scope
safety: ASIL_B
tags: scope, gcov, ERR-01
implemented: YES
version: 1

score_coverage_scope shall list, in <name>_gcno.txt and the gcno / gcno_files output groups, the .gcno notes files the compiler wrote for the translation units of in-workspace targets (taken from InstrumentedFilesInfo, restricted to files the target owns), so that the gcov backend can produce the zero-coverage baseline of every in-scope translation unit. Under a toolchain that emits no .gcno the manifest shall be present and empty.

Baseline objects accompany the scope
safety: ASIL_B
tags: scope, ERR-01
implemented: YES
version: 1

score_coverage_scope shall list the static archives of in-workspace cc_library and rust_library targets and the coverage-built executable of rust_binary targets in the objects file, so the reporter can produce zero-coverage baselines for files no test links against.

Collection#

Per-test profiles are merged with llvm-profdata
safety: ASIL_B

For each test, the merger shall merge all *.profraw files of the test into one profdata file with llvm-profdata merge --sparse, record the real paths of the instrumented objects of the test in meta/meta.json, and package both into the test’s coverage output. Object files come from the objects_list.txt entries of the coverage manifest (C++) or from ELF binaries listed in the manifest (Rust); entries under external/ are skipped.

A test without instrumentation produces no coverage output
safety: QM

When a test has no instrumented objects or no profraw files, the merger shall exit 0 without writing an output file and shall say so on stderr, so non-instrumented tests (for example Python tests) do not fail the run.

Report#

The report is restricted to the scope allowlist
safety: ASIL_B
tags: report
implemented: YES
testlink: GcovReporterMainTest__test_empty_allowlist_is_an_error (passed); GcovReporterMainTest__test_full_report (passed); GcovReporterMainTest__test_missing_gcov_is_an_error (passed); GcovReporterMainTest__test_no_reports_writes_empty_zip (passed); LoadAllowlistAndBaselineTest__test_allowlist_comments_and_blank_lines_ignored (passed); LoadAllowlistAndBaselineTest__test_baseline_objects_resolved_via_main_repo (passed); LoadAllowlistAndBaselineTest__test_missing_baseline_object_is_a_hard_error (passed); LlvmCovInvocationsTest__test_export_separates_stderr (passed); LlvmCovInvocationsTest__test_get_covered_files_applies_the_path_map (passed); LlvmCovInvocationsTest__test_get_covered_files_normalises_paths (passed); LlvmCovInvocationsTest__test_report_flags (passed); LlvmCovInvocationsTest__test_show_html_flags (passed); ReporterMainTest__test_empty_allowlist_is_an_error (passed); ReporterMainTest__test_foreign_virtual_include_is_resolved_against_the_allowlist (passed); ReporterMainTest__test_full_report (passed); ReporterMainTest__test_in_scope_file_without_coverage_data_is_listed (passed); ReporterMainTest__test_invalid_reports_write_empty_zip (passed); ReporterMainTest__test_missing_llvm_tools_is_an_error (passed); ReporterMainTest__test_no_reports_writes_empty_zip (passed); ReporterMainTest__test_path_map_files_headers_under_declared_path (passed); ExclusionRegexTest__test_does_not_hit_a_longer_in_scope_path_with_the_same_suffix (passed); ExclusionRegexTest__test_matches_the_raw_path_under_every_known_prefix (passed); ExclusionRegexTest__test_special_characters_are_literal (passed); SelectFilesTest__test_allowlisted_file_without_any_coverage_data_is_reported (passed); SelectFilesTest__test_duplicate_test_variants_keep_the_declared_path (passed); SelectFilesTest__test_no_allowlist_keeps_everything (passed); SelectFilesTest__test_out_of_scope_and_redundant_variants_are_excluded (passed); RedundantBaselineVariantsTest__test_empty_inputs (passed); RedundantBaselineVariantsTest__test_generated_header_variant_from_baseline_is_redundant (passed); RedundantBaselineVariantsTest__test_plain_sources_and_untested_files_are_kept (passed);
version: 1

The reporter shall exclude every file with coverage data that is not in the allowlist from all three report formats, matching each excluded compiled file exactly (an excluded foo/bar.h shall not suppress an in-scope src/foo/bar.h). An empty allowlist shall be an error (exit non-zero), not an empty report.

Untested in-scope files appear at exact 0 %
safety: ASIL_B
tags: report, ERR-01
implemented: YES
testlink: BaselineTest__test_compiled_stems_from_gcno_paths (passed); BaselineTest__test_gcno_manifest_resolution_and_missing_file_error (passed); BaselineTest__test_gcov_baseline_runs_the_tool_and_tolerates_failures (passed); BaselineTest__test_merge_prefers_test_data_and_zeroes_baseline_only_files (passed); BaselineTest__test_records_from_gcov_json (passed); ExpandBaselineArchivesTest__test_archive_without_any_mapping_contributes_nothing (passed); ExpandBaselineArchivesTest__test_executable_passes_through (passed); ExpandBaselineArchivesTest__test_member_without_mapping_is_dropped_and_the_rest_kept (passed); ExpandBaselineArchivesTest__test_plain_cc_archive_passes_through (passed); ExpandBaselineArchivesTest__test_rlib_is_expanded_to_object_members (passed); FilterLcovTest__test_only_target_records_survive (passed); FilterLcovTest__test_suffix_matching (passed); LlvmCovInvocationsTest__test_export_separates_stderr (passed); LlvmCovInvocationsTest__test_get_covered_files_applies_the_path_map (passed); LlvmCovInvocationsTest__test_get_covered_files_normalises_paths (passed); LlvmCovInvocationsTest__test_report_flags (passed); LlvmCovInvocationsTest__test_show_html_flags (passed); SelectFilesTest__test_allowlisted_file_without_any_coverage_data_is_reported (passed); SelectFilesTest__test_duplicate_test_variants_keep_the_declared_path (passed); SelectFilesTest__test_no_allowlist_keeps_everything (passed); SelectFilesTest__test_out_of_scope_and_redundant_variants_are_excluded (passed); RedundantBaselineVariantsTest__test_empty_inputs (passed); RedundantBaselineVariantsTest__test_generated_header_variant_from_baseline_is_redundant (passed); RedundantBaselineVariantsTest__test_plain_sources_and_untested_files_are_kept (passed);
version: 1

For every allowlisted file that appears in the baseline objects but in no test binary, the reporter shall run llvm-cov with --empty-profile over the baseline objects and shall include the file in the HTML and LCOV output with all instrumented lines and branches at zero hits, so that the LCOV record shows LH:0.

In-scope files without any coverage data are listed
safety: ASIL_B

An allowlisted file for which neither a test binary nor a baseline object carries a coverage mapping (no translation unit includes it, or it holds only template code that is never instantiated) cannot be rendered by llvm-cov, not even at 0 %. The reporter shall write every such file to text_report/unmapped_files.txt (always present, empty when there are none) as <category>\t<path>, sorted, with one of three categories: declaration-only for a header whose same-named source file (same path without extension) has coverage data, compiled-without-code for a source whose object is a member of a baseline archive (it was compiled and holds no code of its own), and no-data for everything else. The reporter shall emit a warning naming the no-data files; generate_coverage_html shall print them, copy the list into the archive as unmapped_files.txt and show the no-data count in the job summary table with one collapsible section per category. None of the categories contributes to any total.

Baseline archives are reduced to members with a coverage mapping
safety: ASIL_B

llvm-cov rejects an archive as a whole as soon as one member has no __llvm_covmap section: the lib.rmeta member of a Rust rlib, or the object of a translation unit without code (the placeholder source of a header-only library). Before passing baseline archives to llvm-cov, the reporter shall inspect every member’s ELF section table and replace such an archive by its members that carry a mapping, so that no library loses its zero-coverage baseline because of one member. The object members of the archives identify the sources that were compiled (In-scope files without any ... (tool_req__coverage_report_unmapped)).

A missing baseline object is an error
safety: ASIL_B

When an entry of the baseline objects manifest cannot be resolved to an existing file, the reporter shall exit non-zero instead of silently dropping the object.

Report paths are workspace-relative
safety: QM
tags: report, ERR-06
implemented: YES
testlink: PathsTest__test_canonical_names_use_path_map_and_foreign_trees (passed); PathsTest__test_merge_by_name_sums_variants_of_one_file (passed); PathsTest__test_normalize_raw (passed); PathsTest__test_rust_sources_are_not_findings_on_gcov (passed); MakeLcovPathsRelativeTest__test_external_paths_are_unchanged (passed); MakeLcovPathsRelativeTest__test_non_sf_lines_are_preserved (passed); MakeLcovPathsRelativeTest__test_proc_self_cwd_prefix_and_path_map (passed); MakeLcovPathsRelativeTest__test_workspace_paths_become_relative (passed); MakeLcovPathsRelativeTest__test_workspace_root_without_trailing_slash (passed); MakeHtmlPathsRelativeTest__test_missing_dir_is_a_noop (passed); MakeHtmlPathsRelativeTest__test_source_title_is_rewritten_and_hrefs_untouched (passed); ReporterMainTest__test_empty_allowlist_is_an_error (passed); ReporterMainTest__test_foreign_virtual_include_is_resolved_against_the_allowlist (passed); ReporterMainTest__test_full_report (passed); ReporterMainTest__test_in_scope_file_without_coverage_data_is_listed (passed); ReporterMainTest__test_invalid_reports_write_empty_zip (passed); ReporterMainTest__test_missing_llvm_tools_is_an_error (passed); ReporterMainTest__test_no_reports_writes_empty_zip (passed); ReporterMainTest__test_path_map_files_headers_under_declared_path (passed); CanonicalPathTest__test_instrumentation_filter_suspects_flags_libraries_tested_from_a_test_subdirectory (passed); CanonicalPathTest__test_plain_paths_are_unchanged (passed); CanonicalPathTest__test_unmapped_virtual_path_keeps_its_config_free_form (passed); CanonicalPathTest__test_virtual_path_maps_to_declared_header_under_any_config (passed); CanonicalPathTest__test_warn_instrumentation_filter_names_the_flag (passed); ForeignVirtualIncludesTest__test_ambiguous_tail_is_reported_not_guessed (passed); ForeignVirtualIncludesTest__test_include_prefix_components_are_skipped (passed); ForeignVirtualIncludesTest__test_no_match_and_non_virtual_names_are_left_alone (passed); ForeignVirtualIncludesTest__test_unique_tail_resolves (passed); StageSourcesTest__test_links_follow_the_raw_layout_and_missing_files_are_reported (passed); StageSourcesTest__test_resolution_order_runfiles_then_workspace (passed); RelocateHtmlPagesTest__test_missing_coverage_dir_is_a_noop (passed); RelocateHtmlPagesTest__test_pages_index_and_asset_links (passed); RelocateHtmlPagesTest__test_second_page_for_the_same_file_is_dropped_with_a_warning (passed); PathMapAndSummaryTest__test_load_path_map (passed); PathMapAndSummaryTest__test_summary_names_are_canonical_and_aligned (passed); ConfigPrefixTest__test_lcov_and_html_paths_drop_the_config_prefix (passed); ConfigPrefixTest__test_strip_config_prefix (passed);
version: 1

The reporter shall name every file by its canonical path in all three report formats: LCOV SF: records, the text summary, HTML page titles, the HTML page location below coverage/ and the index links. The canonical path is the allowlist path; for a header compiled through a _virtual_includes/ tree it is the declared header from the scope’s path map, or, for the tree of a target outside the scope (a test-only twin of a library exposing the same headers), the single allowlisted file whose path ends with the header’s path below the tree; a tail matching several allowlisted files shall stay unresolved and be reported. No absolute directory of the producing machine, no /proc/self/cwd/ prefix and no configuration-specific bazel-out/<config>/bin/ prefix shall remain, so that the archived report is portable and file identity depends neither on the machine nor on the build configuration. The reporter shall read the sources it renders from the scope’s exported files, so that every index link points at a generated page; a file compiled under several paths (a declared header covered by a test binary and again by the baseline archive, or under two include paths) shall appear once.

Report contents
safety: ASIL_B
tags: report
implemented: YES
testlink: GcovrRenderingTest__test_render_html_with_real_gcovr (passed); GcovrRenderingTest__test_tracefile_shape (passed); WriteEmptyOutputTest__test_produces_valid_empty_zip (passed); FindCxxfiltTest__test_explicit_then_sibling_then_none (passed); RunCommandTest__test_failure_exits (passed); RunCommandTest__test_separate_stderr_keeps_stdout_clean (passed); LlvmCovInvocationsTest__test_export_separates_stderr (passed); LlvmCovInvocationsTest__test_get_covered_files_applies_the_path_map (passed); LlvmCovInvocationsTest__test_get_covered_files_normalises_paths (passed); LlvmCovInvocationsTest__test_report_flags (passed); LlvmCovInvocationsTest__test_show_html_flags (passed); ReporterMainTest__test_empty_allowlist_is_an_error (passed); ReporterMainTest__test_foreign_virtual_include_is_resolved_against_the_allowlist (passed); ReporterMainTest__test_full_report (passed); ReporterMainTest__test_in_scope_file_without_coverage_data_is_listed (passed); ReporterMainTest__test_invalid_reports_write_empty_zip (passed); ReporterMainTest__test_missing_llvm_tools_is_an_error (passed); ReporterMainTest__test_no_reports_writes_empty_zip (passed); ReporterMainTest__test_path_map_files_headers_under_declared_path (passed); StageSourcesTest__test_links_follow_the_raw_layout_and_missing_files_are_reported (passed); StageSourcesTest__test_resolution_order_runfiles_then_workspace (passed); RelocateHtmlPagesTest__test_missing_coverage_dir_is_a_noop (passed); RelocateHtmlPagesTest__test_pages_index_and_asset_links (passed); RelocateHtmlPagesTest__test_second_page_for_the_same_file_is_dropped_with_a_warning (passed);
version: 1

The reporter’s output shall be a zip containing html_report/ (llvm-cov HTML with branch counts), lcov_report/lcov.dat (LCOV with line and branch records) and text_report/summary.txt (llvm-cov text summary), with llvm-cov warnings kept out of the LCOV data. When no valid per-test output exists, the output shall be an empty zip. The gcov backend shall produce the same layout (see gcov backend HTML and summa... (tool_req__coverage_gcov_html)).

Backend selection
safety: ASIL_B
tags: report, gcov, ERR-08
implemented: YES
version: 1

score_coverage_reporter shall accept backend = "llvm" (default, requires llvm_cov and llvm_profdata) or backend = "gcov" (requires gcov, the binary of the compiler that produced the counters) and shall fail at analysis time when the tools of the selected backend are missing, so that a coverage configuration can never silently run without a matching reporter.

gcov backend merges per-test LCOV records by summation
safety: ASIL_B

The gcov reporter shall read every per-test LCOV file Bazel’s collector lists (ignoring baseline_coverage.dat), shall normalise SF: paths as the LLVM reporter does (workspace root, /proc/self/cwd/, configuration prefix, path map, virtual-include trees of targets outside the scope), and shall merge records of the same file by adding line, branch and function execution counts across tests; a branch recorded as never reached (-) in every test shall stay so. The scope allowlist shall be applied through the same selection as the LLVM backend.

gcov backend baseline from gcno notes
safety: ASIL_B

For every allowlisted file that has no per-test data, the gcov reporter shall run gcov --json-format --stdout --branch-probabilities over the .gcno files of the scope and include the file with every reported line, branch and function at zero, so that the LCOV record shows LH:0. Baseline data shall never be added to a file that has test data. A .gcno listed in the manifest but missing shall be an error; a gcov failure on one notes file shall be reported and shall not abort the run. A source whose .gcno exists but that yields no lines shall be categorised as compiled without code, and allowlisted Rust sources shall be categorised as not instrumentable (In-scope files without any ... (tool_req__coverage_report_unmapped)).

gcov backend HTML and summary through gcovr
safety: QM

The gcov reporter shall render html_report/ with gcovr from the merged data (index.html plus one page per file with branch information, the sources staged under their canonical paths) and text_report/summary.txt from the same data, and shall write lcov_report/lcov.dat and text_report/unmapped_files.txt in the same format as the LLVM backend, so that generate_coverage_html, the justification layer and the archive need no backend-specific handling.

Justifications#

Justification YAML is validated
safety: ASIL_B

The justification processor shall reject, with exit 1 and a message per finding, a YAML that is not a mapping with an integer version and a justifications list, or that contains an entry without a kebab-case string id, a known category, a non-empty list of known platforms or a non-blank reason, or with malformed locations. Duplicate ids shall be rejected.

In-code markers
safety: ASIL_B
tags: justification, ERR-04
implemented: YES
version: 1

The justification processor shall resolve COV_JUSTIFIED <id> to the marker’s line and COV_JUSTIFIED_START <id> / COV_JUSTIFIED_STOP to the lines strictly between the markers, in checked-in sources with the configured extensions, skipping Bazel output directories, and shall combine them with the explicit locations of the YAML into a manifest keyed by workspace-relative file and line.

Justified locations exist
safety: ASIL_B

A YAML location whose file does not exist shall be reported as an error and the processor shall exit 1 after writing the manifest.

Effective coverage#

Stale justifications are reported and not counted
safety: ASIL_B
tags: effective, ERR-04
implemented: YES
testlink: ProcessHtmlFileTest__test_both_directions_uncovered_count_two_branches (passed); ProcessHtmlFileTest__test_branch_covered_in_one_instantiation_is_not_justified (passed); ProcessHtmlFileTest__test_branches_on_unjustified_lines_untouched (passed); ProcessHtmlFileTest__test_covered_in_any_instantiation_counts_as_covered (passed); ProcessHtmlFileTest__test_covered_justified_line_is_stale (passed); ProcessHtmlFileTest__test_justified_line_not_in_file_is_ignored (passed); ProcessHtmlFileTest__test_no_justifications_leaves_file_untouched (passed); ProcessHtmlFileTest__test_uncovered_branch_on_justified_line (passed); ProcessHtmlFileTest__test_uncovered_justified_line_is_counted_and_restyled (passed); MainLlvmCovTest__test_effective_coverage_is_floored (passed); MainLlvmCovTest__test_justification_for_other_file_does_not_apply (passed); MainLlvmCovTest__test_missing_html_dir_exits (passed); MainLlvmCovTest__test_missing_manifest_exits (passed); MainLlvmCovTest__test_no_justifications_effective_equals_raw (passed); MainLlvmCovTest__test_one_justified_line (passed); MainLlvmCovTest__test_stale_justification_is_reported_not_counted (passed); ProcessGcovrFileTest__test_bare_rows_of_non_instrumented_lines_do_not_shift_statuses (passed); ProcessGcovrFileTest__test_covered_line_without_branch_gap_is_stale (passed); ProcessGcovrFileTest__test_fully_taken_branch_line_is_stale (passed); ProcessGcovrFileTest__test_no_justifications_leaves_page_untouched (passed); ProcessGcovrFileTest__test_not_taken_branch_makes_justification_branch_only (passed); ProcessGcovrFileTest__test_only_the_named_line_is_marked (passed); ProcessGcovrFileTest__test_partial_covered_line_class (passed); ProcessGcovrFileTest__test_uncovered_line_is_justified_and_restyled (passed); ProcessGcovrFileTest__test_unknown_line_is_ignored (passed);
version: 1

A justified line that is covered in any instantiation and has no uncovered branch shall be reported as stale and shall not increase the effective coverage.

Branch-only justifications
safety: ASIL_B

A justified line that is covered but has a branch direction no instantiation covers shall count its truly uncovered directions once as justified branches and shall not count as a justified line.

Justifications match files at path-component boundaries
safety: ASIL_B

A justification for a file shall apply to a report page only when the page’s source path equals the justified path or ends with it at a path-component boundary; bar.cpp shall not apply to foobar.cpp.

Justified lines are visible in the HTML
safety: QM
tags: effective
implemented: YES
version: 1

Justified lines shall be restyled in the HTML report with a J marker, the justification id and reason as tooltip and a distinct colour, the index page shall show the effective figures, and stale justifications shall be listed in summary.txt.

gcovr HTML reports are supported
safety: QM
tags: effective
implemented: YES
testlink: FormatDetectionAndLcovTest__test_detect_html_format (passed); FormatDetectionAndLcovTest__test_parse_lcov_totals (passed); GcovrDetectionAndParsingTest__test_format_detected_by_per_source_naming (passed); GcovrDetectionAndParsingTest__test_index_totals_from_exec_excl_total_rows (passed); GcovrDetectionAndParsingTest__test_index_totals_missing_index (passed); GcovrDetectionAndParsingTest__test_index_totals_without_summary_block (passed); GcovrDetectionAndParsingTest__test_matching_ignores_leading_dot_slash (passed); GcovrDetectionAndParsingTest__test_source_files_skip_index_and_functions_pages (passed); GcovrDetectionAndParsingTest__test_source_path_falls_back_to_title (passed); GcovrDetectionAndParsingTest__test_source_path_from_directory_and_header (passed); ProcessGcovrFileTest__test_bare_rows_of_non_instrumented_lines_do_not_shift_statuses (passed); ProcessGcovrFileTest__test_covered_line_without_branch_gap_is_stale (passed); ProcessGcovrFileTest__test_fully_taken_branch_line_is_stale (passed); ProcessGcovrFileTest__test_no_justifications_leaves_page_untouched (passed); ProcessGcovrFileTest__test_not_taken_branch_makes_justification_branch_only (passed); ProcessGcovrFileTest__test_only_the_named_line_is_marked (passed); ProcessGcovrFileTest__test_partial_covered_line_class (passed); ProcessGcovrFileTest__test_uncovered_line_is_justified_and_restyled (passed); ProcessGcovrFileTest__test_unknown_line_is_ignored (passed); GcovrIndexAndCssTest__test_banner_inserted_before_file_list (passed); GcovrIndexAndCssTest__test_css_injected_into_first_stylesheet (passed); MainGcovrTest__test_no_justifications (passed); MainGcovrTest__test_totals_from_index_page_when_no_lcov (passed); MainGcovrTest__test_totals_from_lcov (passed);
version: 1

The post-processor shall detect gcovr --html-details reports and apply the same justification logic to them, reading totals from the LCOV file when given and from the summary rows of the index page otherwise.

Gate#

Threshold from the environment
safety: ASIL_B

The threshold shall be read from COVERAGE_THRESHOLD and default to 100 %. A value that is not a number in [0, 100] shall be rejected before any report is produced (exit 2).

Gated metric
safety: ASIL_B
tags: gate, ERR-01
implemented: YES
testlink: RawLineCoverageTest__test_baseline_only_records_count_towards_the_denominator (passed); RawLineCoverageTest__test_corrupt_records_are_errors (passed); RawLineCoverageTest__test_missing_file (passed); RawLineCoverageTest__test_no_instrumented_lines_is_an_error (passed); RawLineCoverageTest__test_result_is_not_rounded (passed); RawLineCoverageTest__test_sums_all_records (passed); EffectiveLineCoverageTest__test_integer_is_accepted (passed); EffectiveLineCoverageTest__test_malformed_reports (passed); EffectiveLineCoverageTest__test_missing_report (passed); EffectiveLineCoverageTest__test_reads_summary_value (passed); RunWithoutYamlTest__test_archive_dir_layout (passed); RunWithoutYamlTest__test_archive_is_still_produced_when_gate_fails (passed); RunWithoutYamlTest__test_archive_zip_layout (passed); RunWithoutYamlTest__test_custom_output_dir_and_platform_default (passed); RunWithoutYamlTest__test_explicit_summary_md_wins_over_step_summary (passed); RunWithoutYamlTest__test_gate_fails_at_default_threshold (passed); RunWithoutYamlTest__test_gate_fails_exactly_below_threshold (passed); RunWithoutYamlTest__test_gate_passes_at_low_threshold_and_writes_html (passed); RunWithoutYamlTest__test_github_step_summary_is_appended_not_overwritten (passed); RunWithoutYamlTest__test_invalid_threshold_is_an_error_before_any_output (passed); RunWithoutYamlTest__test_missing_testlogs_when_archiving_is_an_error (passed); RunWithoutYamlTest__test_missing_unmapped_list_is_tolerated (passed); RunWithoutYamlTest__test_stale_output_dir_is_replaced (passed); RunWithoutYamlTest__test_summary_md_is_written_even_when_gate_fails (passed); RunWithoutYamlTest__test_unmapped_files_are_reported_summarised_and_archived (passed); RunWithYamlTest__test_archive_includes_justification_report (passed); RunWithYamlTest__test_gates_on_effective_coverage (passed); RunWithYamlTest__test_justify_failure_is_an_error_not_a_verdict (passed); RunWithYamlTest__test_missing_summary_is_an_error (passed); RunWithYamlTest__test_missing_yaml_is_an_error (passed); RunWithYamlTest__test_tools_receive_workspace_relative_inputs (passed);
version: 1

With --yaml the gate shall use the effective line coverage from the justification report; without it the raw line coverage summed over all LF/LH records of the LCOV data, so that baseline-only files count. The llvm-cov text summary, which omits baseline files, shall not be used.

Unrounded comparison
safety: ASIL_B
tags: gate, ERR-03
implemented: YES
version: 1

The gate shall compare the unrounded percentage with the threshold; a value that prints as 100.00 but is below 100 shall fail a threshold of 100.

Exit codes
safety: ASIL_B
tags: gate, ERR-03
implemented: YES
testlink: RunWithoutYamlTest__test_archive_dir_layout (passed); RunWithoutYamlTest__test_archive_is_still_produced_when_gate_fails (passed); RunWithoutYamlTest__test_archive_zip_layout (passed); RunWithoutYamlTest__test_custom_output_dir_and_platform_default (passed); RunWithoutYamlTest__test_explicit_summary_md_wins_over_step_summary (passed); RunWithoutYamlTest__test_gate_fails_at_default_threshold (passed); RunWithoutYamlTest__test_gate_fails_exactly_below_threshold (passed); RunWithoutYamlTest__test_gate_passes_at_low_threshold_and_writes_html (passed); RunWithoutYamlTest__test_github_step_summary_is_appended_not_overwritten (passed); RunWithoutYamlTest__test_invalid_threshold_is_an_error_before_any_output (passed); RunWithoutYamlTest__test_missing_testlogs_when_archiving_is_an_error (passed); RunWithoutYamlTest__test_missing_unmapped_list_is_tolerated (passed); RunWithoutYamlTest__test_stale_output_dir_is_replaced (passed); RunWithoutYamlTest__test_summary_md_is_written_even_when_gate_fails (passed); RunWithoutYamlTest__test_unmapped_files_are_reported_summarised_and_archived (passed); RunWithYamlTest__test_archive_includes_justification_report (passed); RunWithYamlTest__test_gates_on_effective_coverage (passed); RunWithYamlTest__test_justify_failure_is_an_error_not_a_verdict (passed); RunWithYamlTest__test_missing_summary_is_an_error (passed); RunWithYamlTest__test_missing_yaml_is_an_error (passed); RunWithYamlTest__test_tools_receive_workspace_relative_inputs (passed); MainTest__test_end_to_end_exit_codes (passed); MainTest__test_generate_error_maps_to_exit_error (passed); MainTest__test_requires_build_workspace_directory (passed);
version: 1

generate_coverage_html shall exit 0 when the gate passes, 1 when it fails, and 2 when no verdict is possible. A tool failure shall never end in exit 0.

Broken input yields no verdict
safety: ASIL_B

A missing or non-zip coverage report, a report without html_report/, LCOV data without instrumented lines, LH exceeding LF, a missing justification YAML, a failing justification tool, or a missing summary.txt shall end the run with exit 2.

Summary and archive#

Summary is written before the verdict
safety: QM
tags: summary
implemented: YES
testlink: ParseLcovTest__test_brda_fallback_when_no_brf (passed); ParseLcovTest__test_empty_file_returns_empty_list (passed); ParseLcovTest__test_lf_without_brf_yields_no_branch_data (passed); ParseLcovTest__test_line_and_branch_counters (passed); ParseLcovTest__test_missing_file_returns_none (passed); ParseLcovTest__test_non_utf8_bytes_do_not_crash (passed); ParseLcovTest__test_record_without_end_of_record_is_flushed (passed); MathHelpersTest__test_directory_key_grouping (passed); MathHelpersTest__test_percent_zero_denominator_is_none (passed); MathHelpersTest__test_progress_bar_bounds (passed); RollupTest__test_worst_directory_first (passed); RenderTest__test_branch_dash_when_no_branch_data (passed); RenderTest__test_empty_input_renders_note (passed); RenderTest__test_justification_section (passed); RenderTest__test_load_unmapped_files (passed); RenderTest__test_overall_table_and_zero_section (passed); RenderTest__test_unmapped_files_row_and_section (passed); JustificationReportTest__test_loads_summary_and_counts_applied (passed); JustificationReportTest__test_malformed_json_returns_none (passed); ParseArgsTest__test_all_flags (passed); ParseArgsTest__test_defaults (passed); ParseArgsTest__test_unknown_platform_rejected (passed); RunWithoutYamlTest__test_archive_dir_layout (passed); RunWithoutYamlTest__test_archive_is_still_produced_when_gate_fails (passed); RunWithoutYamlTest__test_archive_zip_layout (passed); RunWithoutYamlTest__test_custom_output_dir_and_platform_default (passed); RunWithoutYamlTest__test_explicit_summary_md_wins_over_step_summary (passed); RunWithoutYamlTest__test_gate_fails_at_default_threshold (passed); RunWithoutYamlTest__test_gate_fails_exactly_below_threshold (passed); RunWithoutYamlTest__test_gate_passes_at_low_threshold_and_writes_html (passed); RunWithoutYamlTest__test_github_step_summary_is_appended_not_overwritten (passed); RunWithoutYamlTest__test_invalid_threshold_is_an_error_before_any_output (passed); RunWithoutYamlTest__test_missing_testlogs_when_archiving_is_an_error (passed); RunWithoutYamlTest__test_missing_unmapped_list_is_tolerated (passed); RunWithoutYamlTest__test_stale_output_dir_is_replaced (passed); RunWithoutYamlTest__test_summary_md_is_written_even_when_gate_fails (passed); RunWithoutYamlTest__test_unmapped_files_are_reported_summarised_and_archived (passed);
version: 1

The markdown summary (--summary-md, or appended to GITHUB_STEP_SUMMARY when the flag is absent) shall be written before the gate decides, so a failing gate still leaves the summary; the explicit flag shall take precedence over the environment variable.

Artifacts tree
safety: ASIL_B
tags: archive, ERR-09
implemented: YES
testlink: ParseArgsTest__test_all_flags (passed); ParseArgsTest__test_defaults (passed); ParseArgsTest__test_unknown_platform_rejected (passed); RunWithoutYamlTest__test_archive_dir_layout (passed); RunWithoutYamlTest__test_archive_is_still_produced_when_gate_fails (passed); RunWithoutYamlTest__test_archive_zip_layout (passed); RunWithoutYamlTest__test_custom_output_dir_and_platform_default (passed); RunWithoutYamlTest__test_explicit_summary_md_wins_over_step_summary (passed); RunWithoutYamlTest__test_gate_fails_at_default_threshold (passed); RunWithoutYamlTest__test_gate_fails_exactly_below_threshold (passed); RunWithoutYamlTest__test_gate_passes_at_low_threshold_and_writes_html (passed); RunWithoutYamlTest__test_github_step_summary_is_appended_not_overwritten (passed); RunWithoutYamlTest__test_invalid_threshold_is_an_error_before_any_output (passed); RunWithoutYamlTest__test_missing_testlogs_when_archiving_is_an_error (passed); RunWithoutYamlTest__test_missing_unmapped_list_is_tolerated (passed); RunWithoutYamlTest__test_stale_output_dir_is_replaced (passed); RunWithoutYamlTest__test_summary_md_is_written_even_when_gate_fails (passed); RunWithoutYamlTest__test_unmapped_files_are_reported_summarised_and_archived (passed); RunWithYamlTest__test_archive_includes_justification_report (passed); RunWithYamlTest__test_gates_on_effective_coverage (passed); RunWithYamlTest__test_justify_failure_is_an_error_not_a_verdict (passed); RunWithYamlTest__test_missing_summary_is_an_error (passed); RunWithYamlTest__test_missing_yaml_is_an_error (passed); RunWithYamlTest__test_tools_receive_workspace_relative_inputs (passed);
version: 1

With --archive-dir the tool shall assemble the HTML report, the LCOV data as coverage_report.dat, the justification report directory and the test.xml files of the selected bazel-testlogs subtree with their paths preserved, also when the gate fails; a missing test-logs directory shall be an error.

Instrumentation filter hint
safety: ASIL_B
tags: report, ERR-13
implemented: YES
version: 1

When an in-scope file has no test data, no file of its directory has test data, and a test or tests subdirectory of that directory has, both reporters shall warn that Bazel’s default --instrumentation_filter probably excluded the file and shall name the flag to set. The warning shall list the files and shall not change the report.

Validation#

Ground-truth validation
safety: ASIL_B
tags: validation, ERR-02, ERR-07
implemented: YES
version: 1

The pipeline shall be validated end to end against a fixture workspace with C++ and Rust units whose line and branch counts are derived by hand: the produced LCOV shall match the expected records exactly (DA, BRDA, LF, LH, BRF, BRH per file), including exact-0 % records for an untested C++ library and an untested Rust binary. The fixture shall contain a library in a package without tests that is exercised from a test subpackage, measured on both backends with the explicit instrumentation filter, and the gcov run shall be repeated with Bazel’s guessed filter to show the warning of Instrumentation filter hint (tool_req__coverage_instrumentation_hint).