Test Case Coverage (test_case_coverage)

test_case_coverage verifies that a declared set of requirements is completely covered by automated test cases. Coverage is locked and confirmed by a committed YAML file;

Requirement IDs are extracted automatically and test records are read from the gtest.lobster file.

Note

For how to annotate tests with lobster-tracing see Validation.

The component rule integrates test case coverage when test_case_coverage_lock is provided:

component(
    name = "my_component",
    requirements  = [":my_component_requirements"],
    components    = [":unit_a", ":unit_b"],
    test_case_coverage_lock = "test_case_coverage.lock.yaml",
)

When the build reports a test_case_coverage drift failure, run (here on my_component):

bazel run //<pkg>:<name>.update

Two phases

test_case_coverage has two distinct operational phases:

Test phase — bazel build //... / bazel test //...

test_runner is invoked as a build action inside the dependable_element rule for each component that has test_case_coverage_lock set. The effective enforcement stringency is controlled by the enclosing dependable_element’s maturity attribute:

maturity = "development" — --allow-check-failures is passed so the .lobster artifact is always produced even when coverage drifts. Both lock-drift failures and missing-GWT-annotation errors are downgraded to warnings;

maturity = "release" — no --allow-check-failures; a drift or missing GWT annotation fails the Bazel build action directly. The .lobster artifact is physically written before the action exits, but Bazel marks the action as failed.

Run phase — bazel run //<pkg>:<name>.update

Computes the lock from the current gtest.lobster and overwrites the committed test_case_coverage.lock.yaml in the workspace. Committing the result constitutes approval of test case coverage.

Architecture

bazel/rules/rules_score/src/test_case_coverage/
├── read_gtest_lobster.py       # reads gtest.lobster; also resolve_path, scan, req_ids
├── compute_lock.py             # uid/spec serialisation + YAML lock
├── check_lock.py               # committed vs computed comparison
├── lobster_generator.py        # emits LOBSTER activity artifact
├── update_runner.py            # entry point: bazel run <component>.update
└── test_runner.py              # entry point: bazel test //...

Data flow

Requirements targets (.lobster) ──► extract requirement IDs
subrule_lobster_gtest ───────────► gtest.lobster
                                           │
                                 read_gtest_lobster
                                 (uid, req refs, spec text)
                                           │
                                     compute_lock
                                 ┌─────────┴──────────────┐
                           RUN phase                TEST phase
                       (bazel run .update)       (bazel test)
                                 │                    │
                          rewrite lock         compare uid+spec
                          file in WS           vs committed lock
                                                      │
                                            lobster_generator
                                            (ok / fail per test)

Lock file format

Auto-generated by the .update target. The engineer commits it after reviewing git diff.

schema_version: 3
requirements:
  - id: MessagePassing.OsIpcFaultHandling
    test_cases:
      - uid:   "//score/message_passing/ConnectionSuite:OsIpcFaultHandlingTest"
        given: a connected client
        when:  the OS IPC call fails
        then:  the client receives an error

Field	Notes
`requirements[].id`	Matches `lobster-tracing` values; extracted from requirements targets
`test_cases[].uid`	`//bazel_package/SuiteName:TestName` — package-scoped gtest tag
`test_cases[].given` / `when` / `then`	Individual GWT fields from `RecordProperty` annotations; any change makes the lock stale