File based rule checks

Test Function

The functionality of the Sphinx build rules can be verified with test rst files.

The function test_check_rules in test_rules_file_based.py is executed for each rst file in the directory rst. It creates a SphinxTestApp and a document source folder with an index.rst file that contains a toctree with the given rst file.

It uses the SphinxTestApp to build the documentation and checks for the expected/not expected warnings.

Create a test rst file

To add a new test case create a new rst file in the rst directory. The test files can also be organized in a subfolder structure below directory rst. The test files are expected to contain the following format:

#CHECK: <check functions>

#EXPECT[+x]: <warning message>
#EXPECT-NOT[+x]: <warning message>

<need information>

<check functions>
Check functions (comma separated) to be used for the Sphinx build. Following warnings will only be generated by these check functions. Only one CHECK statement per file. Usually at the very top. If CHECK is not provided, all checks are enabled.

<warning message>
Message text which is expected/not expected during the Sphinx build to be shown. This message is checked for the Sphinx-Needs directive specified after the EXPECT/EXPECT-NOT statement.

This message needs a ‘[+x]’offset after the ‘EXPECT/-NOT’ that should point to the need that should (not) emit the warning.

<need information>
One or more Sphinx-Needs directives needed for the Sphinx document build

Example:

#CHECK: check_options
#EXPECT[+2]: std_wp__test__abcd: is missing required attribute: `status`.

.. std_wp:: Test requirement
    :id: std_wp__test__abcd

This example verifies that the warning message std_wp__test__abcd: is missing required attribute: `status` is shown during the Sphinx build. Only the check_options check is enabled.

With the ‘[+2]’ after the ‘EXPECT’ we tell the parser that we want this warning to be emitted and checked for 2 lines underneath

There is multiple things that are not allowed for example, you need to have a new line between the EXPECT/-NOT and the need that it refers to

Negative Example

#EXPECT-NOT[+1]: std_wp__test__abcd: is missing required attribute: `status`.
.. std_wp:: Test requirement
    :id: std_wp__test__abcd

This will error and let you know that an offset of ‘1’ is not allowed and you need to add a new line beneath the Warning Statement

Graph check tests

Graph checks (defined in metamodel.yaml under graph_checks:) are all executed by a single Python function check_metamodel_graph. Use that function name in #CHECK::

#CHECK: check_metamodel_graph

Graph check test files live in tests/rst/graph/.

Warning message format

check_metamodel_graph emits warnings in this shape:

{need_id}: Parent need `{parent_id}` does not fulfill condition `{condition}`. Explanation: {explanation}

You only need to match a unique substring, not the full message. For example:

#EXPECT[+2]: wp__bad: Parent need `std_req__aspice_40__bp`

Setup needs

Prerequisite needs (link targets, shared fixtures) belong at the top of the file with no EXPECT/EXPECT-NOT annotation. The framework only checks annotated lines, so unannotated needs are invisible to the assertion logic.

The “exempt” case

A need that does not meet the condition: in the YAML check definition is not selected by the check at all — no warning is emitted. Test this with EXPECT-NOT:

#EXPECT-NOT[+2]: <something from the explanation>

.. workproduct:: No link at all
   :id: wp__no_link

Full example (graph check)

#CHECK: check_metamodel_graph

.. std_req:: ASPICE 40 IIC requirement
   :id: std_req__aspice_40__iic_1
   :status: valid

.. std_req:: ASPICE 40 non-IIC requirement
   :id: std_req__aspice_40__bp_1
   :status: valid

.. Positive: links to allowed target — no warning.

#EXPECT-NOT[+2]: ASPICE 40 IIC

.. workproduct:: Valid workproduct
   :id: wp__valid
   :complies: std_req__aspice_40__iic_1

.. Exempt: no complies link — condition not met, check skipped.

#EXPECT-NOT[+2]: ASPICE 40 IIC

.. workproduct:: Workproduct without link
   :id: wp__no_link

.. Negative: links to disallowed target — warning expected.

#EXPECT[+2]: wp__bad: Parent need `std_req__aspice_40__bp_1`

.. workproduct:: Invalid workproduct
   :id: wp__bad
   :complies: std_req__aspice_40__bp_1