ASPICE 4.0#
This incorporates the detailed description of the Automotive SPICE PAM v4.0 to support the performance of process assessments, so that the process assessment model can be used for its intended purpose.
Automotive SPICE PAM v4.0 provides following copyright release statement:
You may not alter, transform, or build upon this work without the prior consent of the VDA Quality Management Center. Such consent may be given provided ISO copyright is not infringed.
The detailed descriptions contained in this document may be incorporated as part of any tool or other material to support the performance of process assessments, so that this process assessment model can be used for its intended purpose, provided that any such material is not offered for sale.
All distribution of derivative works shall be made at no cost to the recipient.
Copyright notice#
There is no alteration, transformation or building upon in this project. Detailed descriptions are included to facilitate process assessment in projects and are provided in an open source framework free of charge.
Tailoring#
Contents:
- MAN.3 Project Management
- MAN.5 Risk Management
- SWE.1 Software Requirements Analysis
- SWE.2 Software Architectural Design
- SWE.3 Software Detailed Design and Unit Construction
- SWE.4 Software Unit Verification
- SWE.5 Software Component Verification and Integration Verification
- SWE.6 Software Verification
- MLE.1 Machine Learning Requirements Analysis
- MLE.2 Machine Learning Architecture
- MLE.3 Machine Learning Training
- MLE.4 Machine Learning Model Testing
- SUP.1 Quality Assurance
- SUP.8 Configuration Management
- SUP.9 Problem Resolution Management
- SUP.10 Change Request Management
- SUP.11 Machine Learning Data Management
- SPL.2 Product Release
- REU.2 Management of Products for Reuse
- PIM.3 Process Improvement
- Information Item Characteristics
PA 2.1 Process performance management process attribute#
The performance management process attribute is a measure of the extent to which the performance of the process is managed.
Process attribute achievements#
Strategy for the performance of the process is defined based on identified objectives.
Performance of the process is planned.
Performance of the process is monitored and adjusted to meet the planning.
Needs for human resources including responsibilities and authorities for performing the process are determined.
Needs for physical and material resources are determined.
Persons performing the process are prepared for executing their responsibilities.
Physical and material resources for performing the process are identified, made available, allocated and used.
Interfaces between the involved parties are managed to ensure both effective communication and the assignment of responsibilities.
Generic practices#
GP2.1.1: Identify the objectives and define a strategy for the performance of the process.
|
status: valid
|
||||
The scope of the process activities including the management of process performance and the management of work products are determined. Corresponding results to be achieved are determined. Process performance objectives and associated criteria are identified. Note Budget targets and delivery dates to the customer, targets for test coverage and process lead time are examples for process performance objectives. Note Performance objectives are the basis for planning and monitoring. Assumptions and constraints are considered when identifying the performance objectives. Approach and methodology for the process performance is determined. Note A process performance strategy may not necessarily be document-ed specifically for each process. Elements applicable for multiple processes may be documented jointly, e.g, as part of a common project handbook or in a joint test strategy. |
|||||
GP2.1.2: Plan the performance of the process.
|
status: valid
|
||||
The planning for the performance of the process is established according to the defined objectives, criteria, and strategy. Process activities and work packages are defined. Estimates for work packages are identified using appropriate methods. Note Schedule and milestones are defined. |
|||||
GP2.1.3: Determine resource needs.
|
status: valid
|
||||
The required amount of human resources, and experience, knowledge and skill needs for the for process performance are determined based on the planning. The needs for physical and material resources are determined based on the planning. Note Physical and material resources may include equipment, laboratories, materials, tools, licenses etc. Required responsibilities and authorities to perform the process, and to manage the corresponding work products are determined. Note The definition of responsibilities and authorities does not necessarily require formal role descriptions. |
|||||
GP2.1.4: Identify and make available resources.
|
status: valid
|
||||
The individuals performing and managing the process are identified and allocated according to the determined needs. The individuals performing and managing the process are being qualified to execute their responsibilities. Note Qualification of individuals may include training, mentoring, or coaching. The other resources, necessary for performing the process are identified, made available, allocated and used according to the determined needs. |
|||||
GP2.1.5: Monitor and adjust the performance of the process.
|
status: valid
|
||||
Process performance is monitored to identify deviations from the planning. Appropriate actions in case of deviations from the planning are taken. The planning is adjusted as necessary. |
|||||
GP2.1.6: Manage the interfaces between involved parties.
|
status: valid
|
||||
The individuals and groups including required external parties involved in the process performance are determined. Responsibilities are assigned to the relevant individuals or parties. Communication mechanisms between the involved parties are determined. Effective communication between the involved parties is established and maintained. |
|||||
PA 2.2 Work product management process attribute#
The work product management process attribute is a measure of the extent to which the work products produced by the process are appropriately managed.
Process attribute achievements#
Requirements for the work products of the process are defined.
Requirements for storage and control of the work products are defined.
The work products are appropriately identified, stored, and controlled.
The work products are reviewed and adjusted as necessary to meet requirements.
Generic practices#
GP2.2.1: Define the requirements for the work products.
|
status: valid
|
||||
The requirements for the content and structure of the work products to be produced are defined. Quality criteria for the work products are identified. Appropriate review and approval criteria for the work products are defined. Note Possible sources of documentation requirements may be e.g., best practices or lessons learned from other projects, standards, organization requirements, customer requirements, etc. Note There may be types of work products for which no review or approval is required, thus then there would be no need to define the corresponding criteria. |
|||||
GP2.2.2: Define the requirements for storage and control of the work products.
|
status: valid
|
||||
Requirements for the storage and control of the work products are defined, including their identification and distribution. Note Possible sources for the identification of requirements for storage and control may be e.g., legal requirements, data policies, best practices from other projects, tool related requirements, etc. Note Examples for work product storage are files in a file system, ticket in a tool, Wiki entry, paper documents etc. Note Where status of a work product is required in base practices, this should be managed via a defined status model. |
|||||
GP2.2.3: Identify, store and control the work products.
|
status: valid
|
||||
The work products to be controlled are identified. The work products are stored and controlled in accordance with the requirements. Change control is established for work products. Versioning and baselining of the work products is performed in accordance with the requirements for storage and control of the work products. The work products including the revision status are made available through appropriate mechanisms. |
|||||
GP2.2.4: Review and adjust work products.
|
status: valid
|
||||
The work products are reviewed against the defined requirements and criteria. Resolution of issues arising from work products reviews is ensured. |
|||||
PA 3.1 Process definition process attribute#
The process definition process attribute is a measure of the extent to which a standard process is maintained to support the deployment of the defined process.
Process attribute achievements#
A standard process is developed, established, and maintained that describes the fundamental elements that must be incorporated into a defined process.
The required inputs and the expected outputs for the standard process are defined.
Roles, responsibilities, authorities, and required competencies for performing the standard process are defined.
Tailoring guidelines for deriving the defined process from the standard process are defined.
Required physical and material resources and process infrastructure needs are determined as part of the standard process.
Suitable methods and required activities for monitoring the effectiveness, suitability and adequacy of the process are determined.
Generic practices#
GP3.1.1: Establish and maintain the standard process.
|
status: valid
|
||||
A suitable standard process is developed including required activities and their interactions. Inputs and outputs of the standard process are defined including the corresponding entry and exit criteria to determine the interactions and sequence with other processes. Process performance roles are identified and assigned to the standard process activities including their type of involvement, responsibilities, and authorities. Note An example for describing the involvement of the process roles in the activities is a RASI/RASIC representation. Suitable guidance, procedures, and templates are provided to support the execution of the process as needed. Note Procedures may also include description of specific methods to be used. Appropriate tailoring guidelines including predefined unambiguous criteria as well as predefined and unambiguous proceedings are defined based on identified deployment needs and context of the standard process. The standard process is maintained according to corresponding feedback from the monitoring of the deployed processes. Note For guidance on how to perform process improvements see the Process Improvement process (PIM.3). |
|||||
GP3.1.2: Determine the required competencies.
|
status: valid
|
||||
Required competencies, skills, and experience for performing the standard process are determined for the identified roles. Appropriate qualification methods to acquire the necessary competencies and skills are determined, maintained, and made available for the identified roles. Note Qualification methods are e.g., trainings, mentoring, self-study. Note Preparation includes e.g., identification or definition of trainings, mentoring concepts, self-learning material. |
|||||
GP3.1.3: Determine the required resources.
|
status: valid
|
||||
Required physical and material resources and process infrastructure needs for performing the standard process are determined. Note This may include e.g., facilities, tools, licenses, networks, services, and samples supporting the establishment of the required work environment. |
|||||
GP3.1.4: Determine suitable methods to monitor the standard process.
|
status: valid
|
||||
Methods and required activities for monitoring the effectiveness and adequacy of the standard process are determined. Note Methods and activities to gather feedback regarding the standard process may be lessons learned, process compliance checks, internal audits, management reviews, change requests, reflection of state-of-the-art such as applicable international standards, etc. Appropriate criteria and information needed to monitor the standard process are defined. Note Information about process performance may be of qualitative or quantitative nature. |
|||||
PA 3.2 Process deployment process attribute#
The process deployment process attribute is a measure of the extent to which the standard process is deployed as a defined process to achieve its process outcomes.
Process attribute achievements#
A defined process is deployed based upon an appropriately selected and/or tailored standard process.
Assignment of persons necessary for performing the defined process to roles is performed and communicated.
Required education, training and experience is ensured and monitored for the person(s) assigned to the roles.
Required resources for performing the defined process are made available, allocated, and maintained.
Appropriate information is collected and analyzed as a basis for understanding the behavior of the process.
Generic practices#
GP3.2.1: Deploy a defined process that satisfies the context specific requirements of the use of the standard process.
|
status: valid
|
||||
The defined process is appropriately selected and/or tailored from the standard process. Conformance of defined process with standard process requirements and tailoring criteria is verified. The defined process is used as managed process to achieve the process outcomes. Note Changes in the standard process may require updates of the defined process. |
|||||
GP3.2.2: Ensure required competencies for the defined roles.
|
status: valid
|
||||
Human resources are allocated to the defined roles according to the required competencies and skills. Assignment of persons to roles and corresponding responsibilities and authorities for performing the defined process are communicated. Gaps in competencies and skills are identified, and corresponding qualification measures are initiated and monitored. Availability and usage of the project staff are measured and monitored. |
|||||
GP3.2.3: Ensure required resources to support the performance of the defined process.
|
status: valid
|
||||
Required information to perform the defined process is made available, allocated and used. Required physical and material resources, process infrastructure and work environment are made available, allocated and used. Availability and usage of resources are measured and monitored. |
|||||
GP3.2.4: Monitor the performance of the defined process.
|
status: valid
|
||||
Information is collected and analyzed according to the determined process monitoring methods to understand the effectiveness and adequacy of the defined process. Results of the analysis are made available to all effected parties and used to identify where continual improvement of the standard and/or defined process can be made. Note For guidance on how to perform process improvements see the Process Improvement process (PIM.3). |
|||||
Appendix#
ID |
Title |
Status |
Content |
|---|---|---|---|
TA-ANALYSIS |
valid |
Collected data from tests and monitoring of deployed software is analysed according to specified objectives. **Guidance** This assertion is satisfied to the extent that test data, and data collected from monitoring of deployed versions of XYZ, has been analysed, and the results used to inform the refinement of expectations and risk analysis. The extent of the analysis is with sufficient precision to confirm that: - all expectations (TA-BEHAVIOURS) are met - all misbehaviours (TA-MISBEHAVIOURS) are detected or mitigated - all advance warning indicators (TA-INDICATORS) are monitored - misbehaviour/failure rates (calculated directly or inferred by statistics) are within acceptable tolerance Where test results expose misbehaviours not identified in our analysis (TA-ANALYSIS), we add the new misbehaviours to our Expectations (TA-BEHAVIOURS and TA-MISBEHAVIOURS). Where necessary, as informed by our ongoing confidence evaluation (TA-CONFIDENCE), we improve and repeat the analysis (TA-ANALYSIS). **Evidence** - Analysis of test data vs thresholds - Analysis of failures - Analysis of spikes and trends **Confidence scoring** Confidence scoring for TA-ANALYSIS is based on Key Performance Indicators (KPIs) that may indicate problems in development, test, or production **Checklist** - What fraction of expectations are covered by the test data? - What fraction of misbehaviours are covered by the monitored indicator data? - How confident are we that the indicator data are accurate and timely? - How reliable is the monitoring process? - How well does the production data correlate with our test data? - Are we publishing our data analysis? - Are we comparing and analysing production data vs test? - Are our results getting better, or worse? - Are we addressing spikes/regressions? - Do we have sensible/appropriate target failure rates? - Do we need to check the targets? - Are we achieving the targets? |
|
TA-BEHAVIOURS |
valid |
Expected or required behaviours for XYZ are identified, specified, verified and validated based on analysis. Although it is practically impossible to specify all of the necessary behaviours and required properties for complex software, we must clearly specify the most important of these (e.g. where harm could result if given criteria are not met), and verify that these are correctly provided by XYZ. **Guidance** This assertion is satisfied to the extent that we have: - Determined which Behaviours are critical for consumers of XYZ and recorded them as Expectations. - Verified these Behaviours are achieved. Expectations could be verified by: - Functional testing for the system. - Functional soak testing for the system. - Specifying architecture and verifying its implementation with pre-merge integration testing for components. - Specifying components and verifying their implementation using pre-merge unit testing. The number and combination of the above verification strategies will depend on the scale of the project. For example, unit testing is more suitable for the development of a small library than of an OS. **Evidence** - List of expectations **Confidence scoring** Confidence scoring for TA-BEHAVIOURS is based on our confidence that the list of Expectations is accurate and complete, that Expectations are verified by tests, and that the effectiveness of these tests is validated by appropriate strategies. **Checklist** - How has the list of expectations varied over time? - How confident can we be that this list is comprehensive? - Could some participants have incentives to manipulate information? - Could there be whole categories of expectations still undiscovered? - Can we identify expectations that have been understood but not specified? - Can we identify some new expectations, right now? - How confident can we be that this list covers all critical requirements? - How comprehensive is the list of tests? - Is every Expectation covered by at least one implemented test? - Are there any Expectations where we believe more coverage would help? |
|
TA-CONFIDENCE |
valid |
Confidence in XYZ is measured based on results of analysis **Guidance** To quantify confidence, either a subjective assessment or a statistical argument must be presented for each statement and then systematically and repeatably aggregated to assess whether the final deliverable is fit for purpose. To improve the accuracy of confidence evaluations in reflecting reality, the following steps are necessary: - Breaking down high-level claims into smaller, recursive requests. - Providing automated evaluations whenever possible, and using subjective assessments from appropriate parties when automation is not feasible. - Aggregating confidence scores from evidence nodes. - Continuously adjusting previous confidence measures based on new evidence, using previously established confidence values. As subjective assessments are progressively replaced with statistical arguments and past confidence evaluations are refined against new evidence, the accuracy of evaluations improves over time. When project circumstances inevitably change, existing statements are repurposed, with their associated confidence scores eventually offering insights into the systematic capability of the project to deliver according to set objectives. This process should itself be analysed to determine the maturity of any given confidence score. A suitable meta-analysis can assess long-term trends in score sourcing, score accumulation, and weighting mechanisms. **Evidence** - Confidence scores from other TA items **Confidence scoring** Confidence scoring for TA-CONFIDENCE is based on quality of the confidence scores given to Statements **Checklist** - What is the algorithm for combining/comparing the scores? - How confident are we that this algorithm is fit for purpose? - What are the trends for each score? - How well do our scores correlate with external feedback signals? |
|
TA-CONSTRAINTS |
valid |
Constraints on adaptation and deployment of XYZ are specified. **Guidance** Constraints on reuse, reconfiguration, modification, and deployment are specified to enhance the trustability of outputs. To ensure clarity, scoping boundaries regarding what the output cannot do - especially where common assumptions from applied domains may not hold - must be explicitly documented. These constraints are distinct from measures that mitigate misbehaviours; rather, they define the boundaries within which the system is designed to operate. This upfront documentation clarifies the intended use of specified Statements, highlights known limitations, and prevents misinterpretation. These constraints - categorized into explicit limitations and assumptions of use - serve as a guide for both stakeholders and users. They define the intended scope and provide a clear interface for how upstream and downstream systems can integrate, modify, install, reuse, or reconfigure to achieve the desired output. Additionally, the documentation explicitly defines the contexts in which the integrity of existing Statements is preserved and whether any reimplementation is necessary. Crucially, these limitations are not unresolved defects resulting from triage decisions but deliberate exclusions based on design choices. Each omission should be accompanied by a clear rationale, ensuring transparency for future scope expansion and guiding both upstream and downstream modifications. **Suggested evidence** - Installation manuals with worked examples - Configuration manuals with worked examples - Specification documentation with a clearly defined scope - User guides detailing limitations in interfaces designed for expandability or modularity - Documented strategies used by external users to address constraints and work with existing Statements **Confidence scoring** The reliability of these constraints should be assessed based on the absence of contradictions and obvious pitfalls within the defined Statements. **Checklist** - Are the constraints grounded in realistic expectations, backed by real-world examples? - Do they effectively guide downstreams in expanding upon existing Statements? - Do they provide clear guidance for upstreams on reusing components with well-defined claims? - Are any Statements explicitly designated as not reusable or adaptable? - Are there worked examples from downstream or upstream users demonstrating these constraints in practice? - Have there been any documented misunderstandings from users, and are these visibly resolved? - Do external users actively keep up with updates, and are they properly notified of any changes? |
|
TA-DATA |
valid |
Data is collected from tests, and from monitoring of deployed software, according to specified objectives. **Guidance** This assertion is satisfied if results from all tests and monitored deployments are captured accurately, ensuring: - Sufficient precision for meaningful analysis - Enough contextual information to reproduce the test setup, though not necessarily the exact results (e.g., runner ID, software version SHA) To prevent misinterpretation, all data storage mechanisms and locations must be documented, along with long-term storage strategies, ensuring historical analyses can be reliably replicated. **Evidence** - Time-series results for each test. - List of monitored indicators. - Time-series test data for each indicator. - Time-series production data for each indicator. **Confidence scoring** Confidence scoring for TA-DATA is based on comparison of actual failure rates with targets, and analysis of spikes and trends **Checklist** - Is all test data stored with long-term accessibility? - Is all monitoring data stored with long-term accessibility? - Are extensible data models implemented? - Is sensitive data handled correctly (broadcasted, stored, discarded, or anonymised) with appropriate encryption and redundancy? - Are proper backup mechanisms in place? - Are storage and backup limits tested? - Are all data changes traceable? - Are concurrent changes correctly managed and resolved? - Is data accessible only to intended parties? - Are any subsets of our data being published? |
|
TA-FIXES |
valid |
Known bugs or misbehaviours are analysed and triaged, and critical fixes or mitigations are implemented or applied. **Guidance** This assertion is satisfied to the extent that we have identified, triaged and applied fixes and/or mitigations to the faults identified in XYZ and the bugs and CVEs identified by upstream component projects. We can increase confidence by assessing known faults, bugs and vulnerabilities, to establish their relevance and impact for XYZ. In principle this should involve not just the code in XYZ, but also its dependencies (all the way down), and the tools used to construct the release. However we need to weigh the cost/benefit of this work, taking into account - the volume and quality of available bug and CVE reports - likelihood that our build/configuration/usecase is actually affected **Evidence** - List of known bugs fixed since last release - List of outstanding bugs still not fixed, with triage/prioritisation based on severity/relevance/impact - List of known CVEs fixed since last release - List of outstanding CVEs still not fixed, with triage/prioritisation based on severity/relevance/impact - List of XYZ component versions, showing where a newer version exists upstream - List of component version updates since last release - List of fixes applied to developed code since last release - List of fixes for developed code that are outstanding, not applied yet - List of XYZ faults outstanding (O) - List of XYZ faults fixed since last release (F) - List of XYZ faults mitigated since last release (M) **Confidence scoring** Confidence scoring for TA-FIXES can be based on - some function of [O, F, M] for XYZ - number of outstanding relevant bugs from components - bug triage results, accounting for undiscovered bugs - number of outstanding CVEs - CVE triage results, accounting for undiscovered bugs CVEs - confidence that known fixes have been applied - confidence that known mitigations have been applied - previous confidence score for TA-FIXES Each iteration, we should improve the algorithm based on measurements **Checklist** - How many faults have we identified in XYZ? - How many unknown faults remain to be found, based on the number that have been processed so far? - Is there any possibility that people could be motivated to manipulate the lists (e.g. bug bonus or pressure to close). - How many faults may be unrecorded (or incorrectly closed, or downplayed)? - How do we collect lists of bugs and CVEs from components? - How (and how often) do we check these lists for relevant bugs and CVEs? - How confident can we be that the lists are honestly maintained? - Could some participants have incentives to manipulate information? - How confident are we that the lists are comprehensive? - Could there be whole categories of bugs/CVEs still undiscovered? - How effective is our triage/prioritisation? - How many components have never been updated? - How confident are we that we could update them? - How confident are we that outstanding fixes do not impact our expectations? - How confident are we that outstanding fixes do not address misbehaviours? |
|
TA-INDICATORS |
valid |
Advance warning indicators for misbehaviours are identified, and monitoring mechanisms are specified, verified and validated based on analysis. Not all deviations from Expected Behaviour can be associated with a specific condition. Therefore, we must have a strategy for managing deviations that arise from unknown system states, process vulnerabilities or configurations. This is the role of *Advanced Warning Indicators (AWI)*. These are specific metrics which correlate with deviations from Expected Behaviour and can be monitored in real time. The system should return to a defined known-good state when AWIs exceed defined tolerances. **Guidance** This assertion is met to the extent that: - We have identified indicators that are strongly correlated with observed deviations from Expected Behaviour in testing and/or production. - The system returns to a defined known-good state when AWIs exceed defined tolerances. - The mechanism for returning to the known-good state is verified. - The selection of Advance Warning Indicators is validated against the set of possible deviations from Expected behaviour. Note, the set of possible deviations from Expected behaviour *is not* the same as the set of Misbehaviours identified in TA-MISBEHAVIOURS, as it includes deviations due to unknown causes. Deviations are easily determined by negating recorded Expectations. Potential AWIs could be identified using source code analysis, risk analysis or incident reports. A set of AWIs to be used in production should be identified by monitoring candidate signals in all tests (functional, soak, stress) and measuring correlation with deviations. The known-good state should be chosen with regard to the system's intended consumers and/or context. Canonical examples are mechanisms like reboots, resets, relaunches and restarts. The mechanism for returning to a known-good state can be verified using fault induction tests. Incidences of AWIs triggering a return to the known-good state in either testing or production should be considered as a Misbehaviour in TA-MISBEHAVIOURS. Relying on AWIs alone is not an acceptable mitigation strategy. TA-MISBEHAVIOURS and TA-INDICATORS are treated separately for this reason. The selection of AWIs can be validated by analysing failure data. For instance, a high number of instances of deviations with all AWIs in tolerance implies the set of AWIs is incorrect, or the tolerance is too lax. **Evidence** - Risk analyses - List of advance warning indicators - List of Expectations for monitoring mechanisms - List of implemented monitoring mechanisms - List of identified misbehaviours without advance warning indicators - List of advance warning indicators without implemented monitoring mechanisms - Advance warning signal data as time series (see TA-DATA) **Confidence scoring** Confidence scoring for TA-INDICATORS is based on confidence that the list of indicators is comprehensive / complete, that the indicators are useful, and that monitoring mechanisms have been implemented to collect the required data. **Checklist** - How appropriate/thorough are the analyses that led to the indicators? - How confident can we be that the list of indicators is comprehensive? - Could there be whole categories of warning indicators still missing? - How has the list of advance warning indicators varied over time? - How confident are we that the indicators are leading/predictive? - Are there misbehaviours that have no advance warning indicators? - Can we collect data for all indicators? - Are the monitoring mechanisms used included in our Trustable scope? - Are there gaps or trends in the data? - If there are gaps or trends, are they analysed and addressed? - Is the data actually predictive/useful? |
|
TA-INPUTS |
valid |
Components and tools used to construct and verify XYZ are assessed, to identify potential risks and issues **Guidance** To satisfy this assertion, the components and tools used to construct and verify XYZ releases need to be identified and assessed, to identify available sources of evidence about these dependencies. For components, we need to consider how their potential misbehaviours might impact our expectations for XYZ, identify sources of information (e.g. bug databases, published CVEs) that can be used to identify known risks or issues, and tests that can be used to identify these. These provide the inputs to TA-FIXES. For the tools we use to construct and verify XYZ, we need to consider how their misbehaviour might lead to an unintended change in XYZ, or fail to detect misbehaviours of XYZ during testing, or produce incorrect or incomplete data that we use when verifying an XYZ release. Where impacts are identified, we need to consider both how serious they might be (severity) and whether they would be detected by another tool, test or manual check (detectability). For impacts with a high severity and/or low detectability, additional analysis should be done to check whether existing tests are effective at detecting the misbehaviours or resulting impacts, and new tests or Expectations should be added to prevent or detect misbehaviours or impacts that are not currently addressed. **Evidence** - List of components used in construction of XYZ including - Indication of whether content is provided as source or binary - Record of component assessment: - Originating project and version - Date of assessments and identity of assessors - Role of component in XYZ - Sources of bug and risk data for the component - Potential misbehaviours and risks identified and assessed - List of tools used in construction and verification - Record of tool impact assessments: - Originating project and version of tool - Date of assessments and identity of assessors - Roles of tool in production of XYZ releases - Potential tool misbehaviours and impacts - Detectability and severity of impacts - Record of tool qualification reviews - For high impact tools only (low detectability and/or high severity) - Link to impact assessment - Date of reviews and identity of reviewers - Analysis of impacts and misbehaviours - Existing tests or measures (e.g. manual reviews) to address these - Additional tests and/or Expectations required **Confidence scoring** Confidence scoring for TA-INPUTS is based on the set of components and tools identified, how many of (and how often) these have been assessed for their risk and impact for XYZ, and the sources of risk and issue data identified. **Checklist** - Are there components that are not on the list? - Are there assessments for all components? - Has an assessment been done for the current version of the component? - Have sources of bug and/or vulnerability data been identified? - Have additional tests and/or Expectations been documented and linked to component assessment? - Are component tests run when integrating new versions of components? - Are there tools that are not on the list? - Are there impact assessments for all tools? - Have tools with high impact been qualified? - Were assessments or reviews done for the current tool versions? - Have additional tests and/or Expectations been documented and linked to tool assessments? - Are tool tests run when integrating new versions of tools? - Are tool and component tests included in release preparation? - Can patches be applied, and then upstreamed for long-term maintenance? |
|
TA-ITERATIONS |
valid |
All constructed iterations of XYZ include source code, build instructions, tests, results and attestations. **Guidance** This assertion is best satisfied by checking generated documentation to confirm that: - all components, dependencies and tools are identified in a manifest - the manifest includes links to source code - where source is unavailable, the supplier is identified **Evidence** - list of components with source - source code - build instructions - test code - test results summary - attestations - list of components where source code is not available - risk analysis - attestations **Confidence scoring** Confidence scoring for TA-ITERATIONS based on - number and importance of source components - number and importance of non-source components - assessment of attestations **Checklist** - How much of the software is provided as binary only, expressed as a fraction of the BoM list? - How much is binary, expressed as a fraction of the total storage footprint? - For binaries, what claims are being made and how confident are we in the people/organisations making the claims? - For third-party source code, what claims are we making, and how confident are we about these claims? - For software developed by us, what claims are we making, and how confident are we about these claims? |
|
TA-METHODOLOGIES |
valid |
Manual methodologies applied for XYZ by contributors, and their results, are managed according to specified objectives. **Guidance** To satisfy this assertion, the manual processes applied in the verification of XYZ must be documented, together with the methodologies used, the results of applying these processes to specific aspects and iterations of XYZ or its components, and evidence that they have been reviewed using documented criteria. Any data analysis for TA-ANALYSIS should ideally be mostly automated to establish continuous feedback mechanisms. However, specifically, manual processes - such as those used for quality control assurances and the appropriate assignment of responsibilities - must be documented as evidence for TA-METHODOLOGIES. **Evidence** - Manual process documentation - References to methodologies applied as part of these processes - Results of applying the processes - Criteria used to confirm that the processes were applied correctly - Review records for results **Confidence scoring** Confidence scoring for TA-METHODOLOGIES is based on identifying areas of need for manual processes, assessing the clarity of proposed processes, analysing the results of their implementation, and evaluating the evidence of effectiveness in comparison to the analysed results **Checklist** - Are the identified gaps documented clearly to justify using a manual process? - Are the goals for each process clearly defined? - Is the sequence of procedures documented in an unambiguous manner? - Can improvements to the processes be suggested and implemented? - How frequently are processes changed? - How are changes to manual processes communicated? - Are there any exceptions to the processes? - How is evidence of process adherence recorded? - How is the effectiveness of the process evaluated? - Is ongoing training required to follow these processes? |
|
TA-MISBEHAVIOURS |
valid |
Prohibited misbehaviours for XYZ are identified, and mitigations are specified, verified and validated based on analysis. The goal of TA-MISBEHAVIOURS is to force engineers to think critically about their work. This means understanding and mitigating as many of the situations that cause the software to deviate from Expected Behaviours as possible. This is not limited to the contents of the final binary. **Guidance** This assertion is satisfied to the extent that we can: - Show we have identified all of the ways in which XYZ could deviate from its Expected Behaviours. - Demonstrate that mitigations have been specified, verified and validated for all Misbehaviours. Once Expected Behaviours have been identified in TA-BEHAVIOURS, there are at least four classes of Misbehaviour that can be identified: - Reachable vulnerable system states that cause deviations from Expected Behaviour. These can be identified by stress testing, failures in functional and soak testing in TA-BEHAVIOURS and reporting in TA-FIXES. Long run trends in both test and production data should also be used to identify these states. - Potentially unreachable vulnerable system states that could lead to deviations from Expected Behaviour. These can be identified using risk/hazard analysis techniques including HAZOP, FMEDA and STPA. - Vulnerabilities in the development process that could lead to deviations from Expected Behaviour. This includes those that occur as a result of misuse, negligence or malintent. These can be identified by incident investigation, random sampling of process artifacts and STPA of processes. - Configurations in integrating projects (including the computer or embedded system that is the final product) that could lead to deviations from Expected Behaviour. Identified Misbehaviours must be mitigated. Mitigations include patching, re-designing components, re-designing architectures, removing components, testing, static analysis etc. They explicitly *do not* include the use of AWIs to return to a known-good state. These are treated specifically and in detail in TA-INDICATORS. Mitigations could be verified by: - Specifying and repeatedly executing false negative tests to confirm that functional tests detect known classes of misbehaviour. - Specifying fault induction tests or stress tests to demonstrate that the system continues providing the Expected Behaviour after entering a vulnerable system state. - Performing statistical analysis of test data, including using statistical path coverage to demonstrate that the vulnerable system state is never reached. - Conducting fault injections in development processes to demonstrate that vulnerabilities cannot be exploited (knowingly or otherwise) to affect either output binaries or our analysis of it, whether this is by manipulating the source code, build environment, test cases or any other means. - Stress testing of assumptions of use. That is, confirming assumptions of use are actually consistent with the system and its Expected Behaviours by intentionally misinterpreting or liberally interpreting them in a test environment. For example, we could consider testing XYZ on different pieces of hardware that satisfy its assumptions of use. Remember that a Misbehaviour is *anything* that could lead to a deviation from Expected Behaviour. The specific technologies in and applications of XYZ should always be considered in addition to the guidance above. **Suggested evidence** - List of identified Misbehaviours - List of Expectations for mitigations addressing identified Misbehaviours - Risk analysis - Test analysis, including: - False negative tests - Exception handling tests - Stress tests - Soak tests **Confidence scoring** Confidence scoring for TA-MISBEHAVIOURS is based on confidence that identification and coverage of misbehaviours by tests is complete when considered against the list of Expectations. **Checklist** - How has the list of misbehaviours varied over time? - How confident can we be that this list is comprehensive? - How well do the misbehaviours map to the expectations? - Could some participants have incentives to manipulate information? - Could there be whole categories of misbehaviours still undiscovered? - Can we identify misbehaviours that have been understood but not specified? - Can we identify some new misbehaviours, right now? - Is every misbehaviour represented by at least one fault induction test? - Are fault inductions used to demonstrate that tests which usually pass can and do fail appropriately? - Are all the fault induction results actually collected? - Are the results evaluated? |
|
TA-RELEASES |
valid |
Construction of XYZ releases is fully repeatable and the results are fully reproducible, with any exceptions documented and justified. **Guidance** This assertion is satisfied if the construction of a given iteration of XYZ is both *repeatable*, demonstrating that all of the required inputs are controlled, and *reproducible*, demonstrating that the construction toolchain and build environment(s) are controlled (as described by TA-TESTS). This assertion can be most effectively satisfied in a Continuous Integration environment with mirrored projects (see TA-SUPPLY_CHAIN), using build servers that have no internet access. The aim is to show that all build tools, XYZ components and dependencies are built from inputs that we control, that rebuilding leads to precisely the same binary fileset, and that builds can be repeated on any suitably configured server. Again this will not be achievable for components/tools provided in binary form, or accessed via an external service - we must consider our confidence in attestations made by/for the supply chain. All non-reproducible elements, such as timestamps or embedded random values from build metadata, are clearly identified and considered when evaluating reproducibility. **Evidence** - list of reproducible SHAs - list of non-reproducible elements with: - explanation and justification - details of what is not reproducible - evidence of configuration management for build instructions and infrastructure - evidence of repeatable builds **Confidence scoring** Calculate: R = number of reproducible components (including sources which have no build stage) N = number of non-reproducible B = number of binaries M = number of mirrored X = number of things not mirrored Confidence scoring for TA-RELEASES could possibly be calculated as R / (R + N + B + M / (M + X)) **Checklist** - How confident are we that all components are taken from within our controlled environment? - How confident are we that all of the tools we are using are also under our control? - Are our builds repeatable on a different server, or in a different context? - How sure are we that our builds don't access the internet? - How many of our components are non-reproducible? - How confident are we that our reproducibility check is correct? |
|
TA-SUPPLY-CHAIN |
valid |
All sources for XYZ and tools are mirrored in our controlled environment **Guidance** This assertion is satisfied to the extent that we have traced and captured source code for XYZ and all of its dependencies (including transitive dependencies, all the way down), and for all of the tools used to construct XYZ from source, and have mirrored versions of these inputs under our control. 'Mirrored' in this context means that we have a version of the upstream project that we keep up-to-date with additions and changes to the upstream project, but which is protected from changes that would delete the project, or remove parts of its history. Clearly this is not possible for components or tools are provided only in binary form, or accessed via online services - in these circumstances we can only assess confidence based on attestations made by the suppliers, and on our experience with the suppliers' people and processes. Keep in mind that even if repositories with source code for a particular component or tool are available, not all of it may be stored in version control system as plaintext. A deeper analysis is required in TA-INPUTS to assess the impact of any binaries present within the repositories of the components and tools used. **Evidence** - list of all XYZ components including - URL of mirrored projects in controlled environment - URL of upstream projects - successful build of XYZ from source - without access to external source projects - without access to cached data - update logs for mirrored projects - mirrors reject history rewrites - mirroring is configured via infrastructure under direct control **Confidence scoring** Confidence scoring for TA-SUPPLY_CHAIN is based on confidence that all inputs and dependencies are identified and mirrored, and that mirrored projects cannot be compromised. **Checklist** - Could there be other components, missed from the list? - Does the list include all toolchain components? - Does the toolchain include a bootstrap? - Could the content of a mirrored project be compromised by an upstream change? - Are mirrored projects up to date with the upstream project? |
|
TA-TESTS |
valid |
All tests for XYZ, and its build and test environments, are constructed from controlled/mirrored sources and are reproducible, with any exceptions documented **Guidance** This assertion is satisfied to the extent that we - have implemented all of the tests specified in TA-BEHAVIOURS - have implemented fault inductions specified in TA-MISBEHAVIOURS - have implemented or integrated test tooling and environments for these - can demonstrate that all of these are constructed from: - change-managed sources (see TA-UPDATES) - mirrored sources (see TA-SUPPLY_CHAIN) All of the above must ensure that test results are retroactively reproducible, which is easily achieved through automated end-to-end test execution alongside necessary environment setups. Note that with non-deterministic software, exact results may not be reproducible, but high-level takeaways and exact setup should still be possible. **Evidence** - list of tests - list of fault inductions - list of test environments - construction configuration and results **Confidence scoring** Confidence scoring for TA-TESTS is based on the presence of tests and our confidence in their implementation and construction. **Checklist** - How confident are we that we've implemented all of the specified tests? - How confident are we that we've implemented all of the specified fault inductions? - How confident are we that we have test tooling and environments enabling us to execute these tests and fault inductions? - How confident are we that all test components are taken from within our controlled environment? - How confident are we that all of the test environments we are using are also under our control? |
|
TA-UPDATES |
valid |
XYZ components, configurations and tools are updated under specified change and configuration management controls. **Guidance** This assertion requires that that we have control over all changes to XYZ, including changes to the configurations, components and tools we use to build it, and the versions of dependencies that we use. This means, the trustable controlled process is the only path to production for constructed target software. **Evidence** - change management process and configuration artifacts **Confidence scoring** Confidence scoring for TA-UPDATES is based on confidence that we have control over the changes that we make to XYZ, including its configuration and dependencies. **Checklist** - Where are the change and configuration management controls specified? - Are these controls enforced for all of components, tools and configurations? - Are there any ways in which these controls can be subverted, and have we mitigated them? |
|
TA-VALIDATION |
valid |
All specified tests are executed repeatedly, under defined conditions in controlled environments, according to specified objectives. **Guidance** This assertion is satisfied to the extent that all of the tests specified in TA-BEHAVIOURS and constructed in TA-TESTS are correctly executed in a controlled environment on a defined cadence (e.g. daily) or for each proposed change, and on all candidate release builds for XYZ. Note that correct behaviour of tests may be confirmed using fault induction (e.g. by introducing an error or misconfiguration into XYZ). **Evidence** - Test results from per-change tests - Test results from scheduled tests as time series **Confidence scoring** Confidence scoring for TA-VALIDATION is based on verification that we have results for all tests (both pass / fail and performance) **Checklist** - How confident are we that all test results are being captured? - Can we look at any individual test result, and establish what it relates to? - Can we trace from any test result to the expectation it relates to? - Can we identify precisely which environment (software and hardware) were used? - How many pass/fail results would be expected, based on the scheduled tests? - Do we have all of the expected results - Do we have time-series data for all of those results? - If there are any gaps, do we understand why? - Are the test validation strategies credible and appropriate? - What proportion of the implemented tests are validated? |
|
[Your Feature Name] |
draft |
||
[Your Feature Name] Architecture |
draft |
||
[Your Feature Name] Requirements Inspection Checklist |
draft |
||
[Your Feature Name] Requirements |
draft |
||
Platform DFA |
draft |
||
Platform Requirements |
draft |
||
[Your Platform Name] Security Analysis Checklist |
draft |
||
[Your Platform Name] Security Package Checklist |
draft |
||
[Your Platform Name] Security Analysis Checklist |
draft |
||
Platform Release Note |
draft |
.. attention:: The above directive must be updated. - Adjust ``status`` to be ``valid`` - Adjust ``safety`` and ``tags`` according to your needs |
|
Platform Safety Analysis Formal Review Report |
draft |
||
Platform Safety Manual |
draft |
||
Platform Safety Package Formal Review |
draft |
||
Platform Safety Plan |
draft |
||
Platform Safety Plan Formal Review |
draft |
||
Platform Security Analysis |
draft |
||
Platform Security Manual |
draft |
||
Platform Security Plan |
draft |
||
Platform Verification Report |
draft |
||
Process description Release Note v1.5.3 |
valid |
||
Process description Release Note v2.0.0 |
valid |
||
Process description Release Note v2.0.1 |
valid |
||
Stakeholder Requirements Inspection Checklist |
draft |
||
Architecture Process |
valid |
||
Concept Description |
valid |
||
Configuration Management Concept |
valid |
||
Documentation Management Concept |
valid |
||
Building Block Concept |
valid |
||
Lifecycle Concept |
valid |
||
Traceability Concept |
valid |
||
Concept Description |
valid |
||
Concept Description |
valid |
||
Concept Description |
valid |
||
Concept Description |
valid |
||
Process Meta Model |
valid |
||
Quality Management Concept |
valid |
||
Concept Description |
valid |
||
Requirements Concept |
valid |
||
Safety Analysis Concept |
valid |
||
Safety Management Concept |
valid |
||
Security Analysis Concept |
valid |
||
Concept Description |
valid |
||
Concept Description |
valid |
||
Verification Concept |
valid |
In this section a concept for the verification activities will be discussed. Inputs for this concepts are mainly the requirements and work products from * ISO26262 Part 6: "Product development at the software level" * ISO26262 Part 8: "Supporting processes" * ISO26262 Part 9: "Automotive Safety Integrity Level (ASIL)-oriented and safety-oriented analysis" |
|
Work product Inspections Concept |
valid |
||
Architecture Design Process |
valid |
||
Getting Started on Change Management |
valid |
||
Configuration Management Get Started |
valid |
||
Documentation Management Get Started |
valid |
||
Getting Started on Implementation |
valid |
||
Getting Started on Platform/Project Management |
valid |
||
Getting Started on Problem Resolution |
valid |
||
Getting Started on Process Management |
valid |
||
Getting Started on Quality Management |
valid |
||
Getting Started on Release Management |
valid |
||
Getting Started on Requirements |
valid |
||
Getting Started on Safety Analysis (FMEA and DFA) |
valid |
||
Getting started on Safety Management |
valid |
||
Getting Started on Security Analysis |
valid |
||
Getting Started on Change Management |
valid |
||
Getting Started on Tool Management |
valid |
||
Verification Get Started |
valid |
This document guides you through the initial steps of the software verification process, from creating test cases to executing and result reporting. It leverages the :need:`doc_concept__verification_process` and :need:`gd_guidl__verification_guide`. |
|
[Your Tool Name] |
draft |
||
Architecture Inspection Checklist Template |
valid |
For the content see here: - `Component Architecture Inspection Checklist <https://eclipse-score.github.io/module_template/main/components/component_example/architecture/chklst_arc_inspection.html>`__ - `Feature Architecture Inspection Checklist <https://eclipse-score.github.io/module_template/main/features/feature_example/architecture/chklst_arc_inspection.html>`__ These two documents have the same questions, but different scope and document naming. |
|
Change Request Review Checklist |
valid |
| **1. Purpose** | The purpose of this checklist is to collect the topics to be checked during a Change Request from any contributor. | It will not be filled out but considered during the review of the Change Request. | | **2. Checklist** | See also :ref:`review_concept` for further information about reviews in general and inspection in particular. .. list-table:: Feature/Component Request Review Checklist :header-rows: 1 :widths: 10,30,6 * - Id - Topic - Status [FAIL|PASS] * - 1 - Does the Feature/Component Request follow the template? - * - 2 - Are all required topics of the Feature/Component Request template filled? - * - 3 - Does the Change Request has a UID as specified? - * - 4 - Does the Change Request has a title as specified? - * - 5 - Does the Change Request has a description as specified? - * - 6 - Does the Change Request has a impact analysis as specified? - * - 7 - Does the Change Request has a correct safety, security level as specified? - * - 8 - Does the Change Request has a correct type as specified? - * - 9 - Does the Change Request has a links to affected work product as specified? - * - 10 - Does the Change Request has a milestone as specified? - * - 11 - Does the Change Request has verification measure as specified? - |
|
Documentation Review Checklist |
valid |
| **1. Purpose** | The purpose of this checklist is to collect the formal topics to be checked during a Documentation Review from any contributor. It will not be filled out but considered during the review. | | **2. Checklist** | See also :ref:`review_concept` for further information about reviews in general and inspection in particular. .. list-table:: Documentation Review Checklist :header-rows: 1 :widths: 10,30,6 * - Id - Topic - Status [FAIL|PASS] * - 1 - Are available templates used for the documents? - * - 2 - Does the Document have a unique id as specified in the naming conventions? - * - 3 - Are the attributes (safety, security, status) set correctly? - * - 4 - Is the Document stored in the correct location? - * - 5 - Is the document listed correctly in the documentation management plan? - * - 6 - Is the Document linked via realizes to the correct workproduct of the process description? - |
|
Implementation Inspection Checklist Template |
valid |
For the content see here: - `Component Implementation Inspection Checklist <https://eclipse-score.github.io/module_template/main/components/component_example/detailed_design/chklst_impl_inspection.html>`__ |
|
Platform Security Plan Formal Review Checklist |
valid |
For the content see here: :need:`doc__platform_name_security_plan_fdr` |
|
Problem Review Checklist |
valid |
| **1. Purpose** | The purpose of this checklist is to collect the topics to be checked during a Problem report from any contributor. | It will not be filled out but considered during the review and monitoring to closure of the Problem Report. | | **2. Checklist** | See also :ref:`review_concept` for further information about reviews in general and inspection in particular. .. list-table:: Problem Report Review Checklist :header-rows: 1 :widths: 10,30,6 * - Id - Topic - Status [FAIL|PASS] * - 1 - Does the Problem Report follow the template? - * - 2 - Are all required topics of the Problem template filled? - * - 3 - Does the Problem report have a UID as specified? - * - 4 - Does the Problem report have a title as specified? - * - 5 - Does the Problem report have a status as specified? - * - 6 - Does the Problem report have a submitter as specified? - * - 7 - Does the Problem report have a description as specified? - * - 8 - Does the Problem report have a stakeholder as specified? - * - 9 - Does the Problem report have a category as specified? - * - 10 - Does the Problem report have a classification as specified? - * - 11 - Does the Problem report have the affected release versions as specified? - * - 12 - Does the Problem report have an expected closure date as specified? - * - 13 - Does the Problem report have the solution measures specified, verified and reported? - * - 14 - If the Problem report is not closed and pending solution measures are open, escalated to the :need:`rl__safety_manager` or :need:`rl__security_manager`? - |
|
Requirements Inspection Checklist Template |
valid |
For the content see here: - `Component Requirements Inspection Checklist <https://eclipse-score.github.io/module_template/main/components/component_example/requirements/chklst_req_inspection.html>`__ - :need:`doc__feature_name_req_inspection` - :need:`doc__stakeholder_req_inspection` These have the same questions, but different scope and document naming. |
|
Quality Work Product Review Checklist |
valid |
||
Safety Analysis Checklist Template |
valid |
For the content see here: - :need:`doc__platform_safety_analysis_fdr` (platform) - `Safety Analysis Checklist <https://eclipse-score.github.io/module_template/main/module/safety_mgt/module_safety_analysis_fdr.html>`__ (module) |
|
Safety Package Formal Review Checklist |
valid |
For the content see here: `Safety Package Formal Review Checklist <https://eclipse-score.github.io/module_template/main/module/safety_mgt/module_safety_package_fdr.html>`__ |
|
Safety Plan Formal Review Checklist |
valid |
For the content see here: `Safety Plan Formal Review Checklist <https://eclipse-score.github.io/module_template/main/module/safety_mgt/module_safety_plan_fdr.html>`__ |
|
Security Analysis Checklist Template |
valid |
For the content see here: - :need:`doc__platform_name_security_analysis_fdr` (platform) - (tbd) `Security Analysis Checklist <https://eclipse-score.github.io/module_template/main/module/security_mgt/index.html>`__ (module) |
|
Security Package Formal Review Checklist |
valid |
For the content see here: `Security Package Formal Review Checklist <https://eclipse-score.github.io/module_template/main/module/security_mgt/module_security_package_fdr.html>`__ |
|
Module Security Plan Formal Review Checklist |
valid |
For the content see here: `Security Plan Formal Review Checklist <https://eclipse-score.github.io/module_template/main/module/security_mgt/module_security_plan_fdr.html>`__ |
|
Tool Verification Report Review Checklist |
valid |
| **1. Purpose** | The purpose of this checklist is to collect the topics to be checked during a tool verification. | It will not be filled out, but considered during the review and monitoring to complete the Tool Verification Report. | | **2. Checklist** See also :ref:`review_concept` for further information about reviews in general and inspection in particular. .. list-table:: Tool Verification Report Review Checklist :header-rows: 1 :widths: 10,30,6 * - Id - Topic - Status [FAIL|PASS] * - 1 - Is the tool uniquely by name and/or UID defined? - * - 2 - Is the tool version defined? - * - 3 - Is the tool verification status (draft, evaluated, qualified, released, rejected) defined? - * - 4 - Are the purposes of the tool (e.g. use cases) defined? - * - 5 - Are the inputs of the tool defined? - * - 6 - Are the outputs of the tool defined? - * - 7 - Are the configurations of the tool defined? - * - 8 - Are the environmental constraints/limitations defined? - * - 9 - Are the links to the tool documentations available? - * - 10 - Are the tool usage constraints/limitations available? - * - 11 - Are the possible malfunctions of the tool described? - * - 12 - Are the threats of the tool described? - * - 13 - Is the tool impact based on malfunctions/threats defined? - * - 14 - Are safety measures/security controls against the tool malfunctions/threats defined? - * - 15 - Is the tool error detection based on confidence on the defined safety measures/security controls defined? - * - 16 - Is the tool confidence level based on tool impact and tool error detection defined? - |
|
Analysis Tailored |
valid |
This part of the guideline links to all the requirements which are not fulfilled by the safety analysis process. Make sure these are tailored out in the safety/security/quality plans for your project (documented in the PMP). Reasoning given below must be confirmed there. The reasoning is: - see platform safety plan in PMP |
|
Architectural Design Guideline |
valid |
||
Change Request Guideline |
valid |
||
Change Management Requirements Tailored |
valid |
This part of the guideline links to all the requirements which are not fulfilled by the change management process. Make sure these are tailored out in the change management plan for your project (documented in the PMP). Reasoning given below must be confirmed there. The reasoning is: - Trustable Software Framework is planned for evaluation of existing OSS, thus the requirements for evaluation are not applicable. |
|
Classification of a component |
valid |
For the content see here: `Component Classification Template <https://eclipse-score.github.io/module_template/main/components/component_example/component_classification.html>`__ |
|
DFA failure initiators |
valid |
||
Documentation |
valid |
||
FMEA Fault Models |
valid |
| Fault Model for sequence diagrams |
|
IIC Requirements Tailored |
valid |
This part of the guideline links to all the information item characteristics (IIC) which are not fulfilled by the current process description. Make sure these are tailored out in the safety/security/quality plans for your project (documented in the PMP). Reasoning given below must be confirmed there. The reasoning is: - For Machine Learning related IICs, as the project does not develop ML components so far. - For IICs related to hardware development, as the project does not develop hardware components so far. - For IICs concerning commitment, agreements, organizational processes, as these are not in the scope of the current project. - For IICs concerning GPs (2 and 3), at the are not addressed yet in the current project phase or higher, which are out of scope for the current project phase. - For IICs concerning specific techniques or methods not used in the current project. |
|
Implementation Guideline |
valid |
||
Working model |
valid |
||
Platform Management Plan Requirements Tailored |
valid |
This part of the guideline links to all the requirements which are not fulfilled by the platform management plan process. Make sure these are tailored out in the corresponding sub-plans e.g. project management plan, etc. for your project (documented in the PMP). Reasoning given below must be confirmed there. The reasoning is: - Main deliveries are source code, which is not a product, thus product risk management and process risk is not applicable (for safety and security related risk separate processes as safety/security analysis exists). |
|
Problem Resolution Guideline |
valid |
||
Process Management Guideline |
valid |
||
Quality Management Plan Definitions Guideline |
valid |
| **Overall quality management:** | | **Quality culture:** | Quality as well as Safety and Security Culture is planned to grow in the SW platform. This shall be fostered by doing process conformance checks and work product reviews, as well as lessons learned | after each feature development completion and a process audit after each platform/project release. Delta audits allowed based on variation statement. | The main outcome is the :need:`wp__process_impr_report`, which is used to improve the processes for the platform/project. | | **Quality Management:** | ASPICE 4.0 standard is selected for quality management. Processes will always link to the :ref:`standard_iso26262` standard, :ref:`standard_isopas8926` standard, :ref:`standard_isosae21434` and to the `ASPICE 4.0 <https://eclipse-score.github.io/process_description/main/standards/aspice_40/aspice.html>`_ standard. | | **Communication:** | Cross functional teams are interdisciplinary, so the regular (sprint) planning and review meetings enable communication. The organization of the project is described in the Project Management Plan. Another main communication means are the Pull Request (PR) reviews. | Also the standard Eclipse Foundation communication strategies are used (e.g. mailing lists, messenger). | | **Quality issues, non-conformances and improvements:** | Feedback from the field, but also during development of change requests to existing features, bug reporting by the Open Source community or integration of existing SW components into new features may lead to the discovery of issues, non-conformances or improvements. | Non-conformance can also be deviations from the development process with impact on safety or security. | If these are known at the time of creation of a release they will be part of the :need:`wp__platform_sw_release_note` for the feature. | Other issues and non-conformances relevant for already delivered releases will be identified as such and communicated (as defined in Problem Resolution part of the Project Management Plan) via the :need:`wp__issue_track_system`. | | **Tailoring quality activities:** | Main tailoring driver is that the SW platform is pure SW development and is provided as "SW element" - this explains mainly the generic, platform wide tailoring. | Tailoring is done for the whole SW platform by defining only the relevant processes and their resulting outcomes and an argumentation why the others are not needed in `ASPICE 4.0 <https://eclipse-score.github.io/process_description/main/standards/aspice_40/aspice.html>`_. | | **Planning quality activities:** | In the Quality Management Plan the nomination of the :need:`rl__quality_manager` and the :need:`rl__project_lead` is documented. | The planning of quality activities is done using issues in the :need:`wp__issue_track_system` as specified in the Project Management part of the Project Management Plan. | It contains for each issue | * objective - as part of the issue description | * dependencies on other activities or information - by links to the respective issues | * responsible person for the activity - as issue assignee | * required resources for the activity - by selecting a team label (or "project") pointing to a team of committers dedicated to the issue resolution | * duration in time, including start and end point - by selecting a milestone | * UID of the resulting work products - stated in the issue title | | The planning of quality activities is part of | * generic planning, dealing with all work products needed only once for the platform. This is included in the Quality Management Platform Plan. | | **Planning supporting processes:** | Supporting processes (Requirements Management, Configuration Management, Change Management, Documentation Management, Tool Management) are planned within the Project Management Plan. | | **Planning integration and verification:** | Integration on the target hardware is not done in the scope of the SW platform project, but SW/SW integration up to the feature level is performed and its test results are part of the :need:`wp__verification_platform_ver_report`. | The integration on the target hardware, done by the distributor or OEM, is supported by delivering a set of HW/SW feature integration and Platform Integration Tests which were already run successfully on a reference HW platform. | This is planned by the respective workproducts: | * :need:`wp__verification_feat_int_test` | * :need:`wp__verification_platform_int_test` | Verification planning is documented in :need:`wp__verification_plan` | | **Scheduling of audits, conformance checks, work product reviews, release verification and approval:** | Scheduling is done in the same way as for all work products definition by issues. The respective work products are listed in :need:`doc_concept__wp_inspections`. | | **Planning quality trainings:** | Quality trainings are planned also by issues. They should cover the following aspects: | * Target audience | * Training objectives | * Training content | * Training schedule | * Responsible persons |
|
Release Handbook |
valid |
The release handbook incorporates an overview including a tutorial to explain the usage of the released software. It extends as a decent documentation for users and contributors beyond the pure release notes created according to the template :need:`gd_temp__rel_plat_rel_note`. It is intended to contain: * Overview of the technologies used within the project * Software architecture overview * Module structure overview * Integration process * Getting started guide * Contribution guide By this the software integration approach becomes clear and reflects the dependencies and functional unit integration of the various modules. It also describes how to build and run the software for the various target systems such as simulation, hardware architectures, and operating system(s). |
|
Release Management Guideline |
valid |
||
Requirements Guideline |
valid |
||
Requirements Tailored |
valid |
This part of the guideline links to all the requirements which are not fulfilled by the requirements engineering process. Make sure these are tailored out in the safety/security/quality plans for your project (documented in the PMP). Reasoning given below must be confirmed there. The reasoning is: - for "system" standard requirements: see platform safety plan in PMP - for "software" standard requirements: 644, 646: because they refer to (PMP) tailored work product, 643: because this refers to (PMP) tailored activity |
|
Safety manual generation |
valid |
| The safety manual collects several workproducts and adds some additional content mainly to instruct the user of | a SEooC (in this project on platform and module level) to safely use it in the context of the user's own safety | element. | Its main content is described in :need:`wp__platform_safety_manual` and :need:`wp__module_safety_manual` | A template exists to guide the definition of the safety manual on platform and module level (:need:`gd_temp__safety_manual`). |
|
Safety package automated generation |
valid |
| The safety package shall be generated progressively and automatically compiling the work products. | One of the checks to perform on the platform safety package is to check completeness of the | process compliance to standards, which can be seen from standard linkage charts in :ref:`external_standards`. |
|
Safety plan definitions |
valid |
**Safety culture:** Safety culture is planned to grow in the SW platform. This shall be fostered by doing a lessons learned after each feature development completion, using the ISO 26262-2 Table B.1 as a questionnaire. This lessons learned is the main input for process improvement managed by :need:`wp__process_impr_report` As starting point for safety culture we define a Committer selection process to already have professionals with safety experience in the teams. Additionally the SW platform's processes are defined with experience of several companies already performing successful safe SW development. This also improves independence of reviews for the process definitions. **Quality Management:** ASPICE standard is selected for quality management. Processes will always link to the :ref:`standard_iso26262` standard and to the :ref:`ASPICE PAM4 <standard_aspice_pam4>` standard. **Competence management:** The :need:`rl__project_lead` on SW platform level is responsible to define a competence management for the whole platform. Expectation is that the safety competence of the persons nominated for the roles is already given and only has to be checked. The exception from this are the committers, for these no safety competence needs to be enforced. So the module safety managers shall consult the :need:`module safety plan <wp__module_safety_plan>` and perform accordingly in their module project. **Communication:** Development teams are interdisciplinary, so the regular planning and review meetings enable communication (as defined in in the project specific :need:`project management plan <wp__project_mgt>`). Another main communication means are the Pull Request reviews. Also the standard Eclipse Foundation communication strategies are used (e.g. mailing lists) **Safety anomalies:** As the SW platform organization does not have own vehicles in the field, it relies on feedback from OEMs and Distributors on bugs discovered in the field. The need for this feedback is part of each safety manual. But also during development of change requests to existing features, bug reporting by the Open Source community or integration of existing SW components into new features may lead to the discovery of new safety anomalies. Safety anomalies can also be deviations from the development process with impact on safety. If these are known at the time of creation of a release they will be part of the :need:`wp__module_safety_package` or :need:`wp__platform_safety_package` for the SEooC. Safety anomalies relevant for already delivered releases will be identified as such and communicated (as defined in Problem Resolution part of :need:`wp__platform_mgmt`) via the :need:`wp__issue_track_system` (which is also Open Source). **Tailoring Safety Activities:** The software platform is developed purely as software and provided as a Safety Element out of Context (SEooC). That is why only software relevant parts of :ref:`standard_iso26262` are used. This requires a generic, platform-wide approach to tailoring so that safety processes remain efficient, relevant, and compliant with the software relevant ISO 26262. Before any tailoring is performed, an **impact analysis** must be conducted in accordance with ISO 26262. This analysis is performed on element level, not on the item level as previously stated. This analysis determines whether the element is a new development, a modification, or an existing element with a modified environment. The results guide the extent and nature of any tailoring. If tailoring is necessary, it must be justified and documented. The rationale should demonstrate that the tailored approach is sufficient to achieve the required level of functional safety. Tailoring may be driven by factors such as: - Qualification of software components, - Confidence in the use of software tools. Note: Proven-in-use arguments are generally not applicable for a reusable SEooC platform intended for integration into various target applications and environments. For the software platform as a whole, tailoring is achieved by clearly defining which work products are relevant and providing a reasoned argument for omitting others. This is documented in the platform safety plan and the ISO 26262 reference documentation. Additional tailoring may apply at the module or feature level, particularly for SEooC developments where reuse of existing qualified components is the main driver. Such tailoring is documented in the respective module safety plans. When safety activities are tailored because an element is developed as SEooC: a) The SEooC development must be based on a requirements specification derived from well-defined assumptions about its intended use and context, including all relevant external interfaces. b) These assumptions must be validated when the SEooC is integrated into its target application. This approach ensures that safety activities are focused, efficient, and appropriate to the context of a reusable software platform, while maintaining compliance with ISO 26262 requirements and intent. **Planning safety activities:** In the safety plan the nomination of the safety manager and the project manager is documented. The planning of safety activities is done as for the project defined in the :need:`wp__project_mgt` by using issues in the :need:`wp__issue_track_system`. It should contain for each issue: * objective - as part of the issue description * dependencies on other activities or information - by links to the respective issues * responsible person for the activity - as issue assignee * required resources for the activity - by selecting a team label (or "project") pointing to a team of committers dedicated to the issue resolution * duration in time, including start and end point - by selecting a milestone * UID of the resulting work products - stated in the issue title The planning of safety activities is divided into the * platform SEooC planning, dealing with all work products needed only once for the platform. This is included in :need:`wp__platform_safety_plan` * module SEooC planning, dealing with all work products needed for each module development (initiated by a change request), included in :need:`wp__module_safety_plan`. This module safety planning also includes the planning of OSS component qualification based on :need:`gd_guidl__component_classification`. Templates exists to guide this: :need:`gd_temp__platform_safety_plan`, :need:`gd_temp__module_safety_plan`. These include linkage to the project planning according to the defined issue structuring schema. **Reporting safety activities:** Reporting is based on work products and documents status and supported by the safety plan templates and document management. A safety plan is completed and the safety package can be released when all planned work products and documents are in status "valid". **Planning safety activities for subsequent releases** After the first release of the platform or a module with full safety coverage (i.e. all work products are in "valid" state) there will be further development. This further development is initiated by creating a feature/component change request including a change impact analysis (see :need:`doc_concept__change_process`). As part of the "implementation and monitoring of the change request", the documents affected by the change will be set back to "draft" and the implementation (change of work products) planned by issues, as appropriate to the size of the change. At least the change request issue will be linked to the updated safety plan. Note that there may be more than one change request for a module per release cycle. **Planning supporting processes:** Supporting processes (Requirements Management, Configuration Management, Change Management, Documentation Management, Tool Management) are planned within the :need:`wp__platform_mgmt` **Planning integration and verification:** Integration on the target hardware is not done in the scope of the SW platform project, but SW/SW integration up to the feature level is performed and its test results are part of the :need:`wp__verification_platform_ver_report`. The integration on the target hardware done by the user is supported by delivering a set of SW integration tests which were already run successfully on a reference HW platform. This is planned by the respective work products: * :need:`wp__verification_feat_int_test` * :need:`wp__verification_platform_int_test` Verification planning is documented in :need:`wp__verification_plan` Any unspecified functions, such as code for debugging or instrumentation, must either be deactivated or removed prior to release, unless their presence does not affect safety compliance. **Scheduling of formal document reviews and safety audit:** Scheduling is done in the same way as for all work products definition by issues. The respective work products are :need:`wp__fdr_reports` and :need:`wp__audit_report` A person responsible for carrying out the functional safety audit shall be appointed as part of the scheduling process. This person has to have the required skillset and knowledge. The functional safety auditor may appoint one or more assistants to support the audit. These assistants may not be fully independent from the developers of the relevant item, elements, or work products, but must possess at least a basic level of independence. The auditor is responsible for appraising the input from any assistants to ensure that the audit remains objective and that an unbiased opinion is provided. The planning and follow-up of the audit shall also take into account the type of report to be issued—whether it is an acceptance, conditional acceptance (with required corrective actions and conditions for acceptance), or a rejection. Any conditions or corrective actions identified in the report must be addressed and tracked to completion as part of the Safety Management process. **Planning of dependent failures and safety analyses:** In cases where the components consist of sub-components there will be more than one architecture level. DFA and Safety analysis will then be done on these multiple levels. See the respective work products: * feature level: :need:`wp__feature_fmea` and :need:`wp__feature_dfa` * component level: :need:`wp__sw_component_fmea` and :need:`wp__sw_component_dfa` **Provision of the confidence in the use of software tools:** Tool Management planning is part of the :need:`wp__platform_mgmt`. The respective work product to be planned as an issue of the generic safety plan is the :need:`wp__tool_verification_report`, which contains tool evaluation and if applicable qualification of the SW platform toolchain. Components developed in different programming languages will have different toolchains. They will be qualified once for the SW platform. **(OSS) Component qualification planning:** Based on the component classification as described in :need:`gd_guidl__component_classification`, the qualification of the component is planned as part of the :need:`gd_temp__module_safety_plan`. The template contains guidance how to do this and to document in the "OSS (sub-)component <name> Workproducts" list. As an alternative the module safety manager can also decide to use the :ref:`external_tsf` to reach enough trust in an externally provided (OSS) to use it for safety related functionality in the scope of the SW platform. This approach is also documented in the module safety plan. |
|
Safety Mgt Tailored |
valid |
This part of the guideline links to all the requirements which are not fulfilled by the safety management process. Make sure these are tailored out in the safety/security/quality plans for your project (documented in the PMP). Reasoning given below must be confirmed there. The reasoning is: - for "support" standard requirements: The requirement is not applicable for an ASIL_B process - for "management" standard requirements: 6453 - not proven in use argument, 6454 - no HW part of SW platform, 6456 - no confidence in use, 64610 - no distributed development - for "management" standard requirements 6412*: No assessment planned, as also no finalized safety case is planned |
|
Safety Analysis (DFA and FMEA) Guideline |
valid |
||
Security Analysis threat scenarios |
valid |
||
Security Analysis Guideline |
valid |
||
Security Manual Generation |
valid |
The Security Manual collects several work products and adds some additional content mainly to instruct the user of a OoC (in this project on platform and module level) to securely use it in the context of the user's OoC and requirements for post-development. Its main content is described in :need:`wp__platform_security_manual` and :need:`wp__module_security_manual`. A template exists to guide the definition of the security manual on platform and module level (`Module Security Manual Template <https://eclipse-score.github.io/module_template/main/module/manuals/security_manual.html>`__). |
|
Security Package Automated Generation |
valid |
The Security Package shall be generated progressively and automatically compiling the work products. One of the checks to perform on the platform safety package is to check completeness of the process compliance to standards, which can be seen from standard linkage charts in :ref:`external_standards`. |
|
Security plan definitions |
valid |
**Security Culture:** Security culture is planned to grow in the SW platform. This shall be fostered by doing a lessons learned after each feature development completion, using the ISO SAE 21434 Annex B, Table B.1 as a questionnaire. This lessons learned is the main input for process improvement managed by :need:`wp__process_impr_report`. As starting point for security culture we define a Committer selection process to already have professionals with security experience in the teams. Additionally the SW platform's processes are defined with experience of several companies already performing successful safe and secure SW development. This also improves independence of reviews for the process definitions. **Quality Management:** ASPICE standard is selected for quality management. Processes will always link to the :ref:`standard_isosae21434` standard and to the :ref:`standard_aspice_pam4` standard. **Competence Management:** The :need:`rl__security_manager` on SW platform level is responsible to define a competence management for the whole platform. Expectation is that the security competence of the persons nominated for the roles is already given and only has to be checked. The exception from this are the committers, for these no security competence needs to be enforced. So the Module Security Managers shall consult the :need:`wp__platform_security_plan` and perform accordingly in their module project. **Communication:** Development teams are interdisciplinary, so the regular (sprint) planning and review meetings enable communication (as defined in :need:`wp__platform_mgmt`). Another main communication means are the Pull Request reviews. Also the standard Eclipse Foundation communication strategies are used (e.g. mailing lists) **Security Weaknesses, Vulnerabilities:** As the SW platform organization does not have own vehicles in the field, it relies on feedback from OEMs and Distributors on bugs discovered in the field. The need for this feedback is part of each Security Manual. But also during development of change requests to existing features, bug reporting by the Open Source community or integration of existing SW components into new features may lead to the discovery of new security weaknesses and vulnerabilities. Security weaknesses and vulnerabilities can also be deviations from the development process with impact on security. If these are known at the time of creation of a release they will be part of the :need:`wp__module_security_package` or :need:`wp__platform_security_package` for the OoC. Security weaknesses and vulnerabilities relevant for already delivered releases will be identified as such and communicated (as defined in Problem Resolution part of :need:`wp__platform_mgmt`) via the :need:`wp__issue_track_system` (which is also Open Source). **Tailoring Security Activities:** Main tailoring driver is that the SW platform is pure SW development and is provided as "(component) OoC" - this explains mainly the generic, platform wide tailoring. Tailoring is done for the whole SW platform by defining only the relevant work products and an argumentation why the others are not needed in :ref:`standard_isosae21434` and :need:`wp__platform_security_plan`. But there may be also additional tailoring for each module/component OoC development to restrict further the work products. This is documented in every Module Security plan. Here the usage of already existing components is the main tailoring driver. **Planning Security Activities:** In the Security Plan the nomination of the Security Manager and the Project Lead is documented. The planning of security activities is done using issues in the :need:`wp__issue_track_system` as specified in the :need:`wp__platform_mgmt`. It contains for each issue * objective - as part of the issue description * dependencies on other activities or information - by links to the respective issues * responsible person for the activity - as issue assignee * required resources for the activity - by selecting a team label (or "project") pointing to a team of committers dedicated to the issue resolution * duration in time, including start and end point - by selecting a milestone * UID of the resulting work products - stated in the issue title The planning of security activities is divided into the * Platform OoC planning, dealing with all work products needed only once for the platform. This is included in :need:`wp__platform_security_plan` * Module/Component OoC planning, dealing with all work products needed for each module development (initiated by a change request), included in :need:`wp__module_security_plan`. A template exists to guide this: :need:`gd_temp__module_security_plan`. **Planning Supporting Processes:** Supporting processes (Requirements Management, Configuration Management, Change Management, Documentation Management, Tool Management) are planned within the :need:`wp__platform_mgmt` **Planning Integration and Verification:** Integration on the target hardware is not done in the scope of the SW platform project, but SW/SW integration up to the feature level is performed and its test results are part of the :need:`wp__verification_platform_ver_report`. The integration on the target hardware done by the distributor or OEM is supported by delivering a set of HW/SW integration tests which were already run successfully on a reference HW platform. This is planned by the respective work products: * :need:`wp__verification_feat_int_test` * :need:`wp__verification_platform_int_test` Verification planning is documented in :need:`wp__verification_plan`. Any unspecified functions, such as code for debugging or instrumentation, must either be deactivated or removed prior to release, unless their presence does not affect security compliance. **Scheduling of Reviews, Audit and Assessment:** Scheduling is done in the same way as for all work products definition by issues. The respective work products are :need:`wp__fdr_reports_security`, and :need:`wp__audit_report_security` **Planning of Security Analyses:** In cases where the components consist of sub-components there will be more than one architecture level. Security analysis will then be done on these multiple levels. See the respective work products: * platform level: :need:`wp__platform_security_analysis` * feature level: :need:`wp__feature_security_analysis` * component level: :need:`wp__sw_component_security_analysis` Analyses shall be based on `STRIDE <https://en.wikipedia.org/wiki/STRIDE_model>`_ model. **Provision of the confidence in the use of software tools:** Tool Management planning is part of the :need:`wp__platform_mgmt`. The respective work product to be planned as an issue of the generic security plan is the :need:`wp__tool_verification_report`, which contains tool evaluation and if applicable qualification of the SW platform toolchain. Components developed in C++ and Rust will have different toolchains. Both will be qualified once for the SW platform. **Provision of a Software Bill of Materials (SBOM) and Vulnerability Management** SBOMs provide a comprehensive inventory of all components and dependencies within a software project, thus they can be interpreted as configuration information. `Eclipse Project Handbook: Software Bill of Material <https://www.eclipse.org/projects/handbook/#sbom>`_ recommends to generate SBOMs and contains also information how to generate SBOMs. SBOMs are used as sources for collection of information and as trigger for further investigations as identifying weaknesses and vulnerabilities. `Eclipse Foundation Security Team <https://www.eclipse.org/projects/handbook/#vulnerability-team>`_ provides help and advice to Eclipse projects on security issues and is the first point of contact for handling security vulnerabilities. Nevertheless :need:`rl__contributor` and :need:`rl__committer` are responsible for following the `Eclipse Foundation Security Policy <https://www.eclipse.org/security/policy/>`_. The :need:`Security Team <rl__security_team>` is responsible for coordinating the resolution of vulnerabilities within the Project. |
|
STRIDE Threat Model |
valid |
| Threat Model for sequence diagrams using STRIDE methodology |
|
Tool Qualification |
valid |
| The tool qualification shall be based on the method validation of the software tool. |
|
Tool Requirements Tailored |
valid |
This part of the guideline links to all the requirements which are not fulfilled by the tool management process. Make sure these are tailored out in the safety plans for your project (documented in the PMP). Reasoning given below must be confirmed there. The reasoning is: - Some methods for tool qualification are not applied |
|
Verification Guideline |
valid |
This guideline outlines the responsibilities and procedures for developers performing verification activities (testcase creation, inspection, and review) for documentation, code (like Rust and C/C++) elements of the project/platform and its tooling. Note that rust, python and gTest are used for test case creation. |
|
Verification Requirements Tailored |
valid |
This part of the guideline links to all the requirements which are not fulfilled by the verification process. Make sure these are tailored out in the safety/security/quality plans for your project (documented in the PMP). Reasoning given below must be confirmed there. The reasoning is: - For SW requirement 945 & 1047 tests are executed in target environment regarding code and hardware architecture. An integrator of the software can re-execute them on source or unmodified binary in the product environment. - For SW requirement 1045 "Function and call coverage" are not considered explicitly due to lower target safety integrity level (ASIL_B). - For SW requirement 1046 the software is not a production release, but work for the system integrator or distributor with a production release. - For SW requirement 1141, 1143, 1144 the SW only points to AoUs, but will not be executed on the final product environment as part of the project scope. - For SW requirement 1142 there will be :need:`wp__verification_platform_int_test` available for re-use, but they are not supposed to deliver full coverage. Fault Injection Testing is not considered for higher level integration testing. |
|
Test Specification Guideline |
valid |
||
Quality work product review |
valid |
||
Verification Derivation Technique |
valid |
Following derivation techniques are explained * :ref:`Requirements analysis <ver_req_anal>` * :ref:`Boundary Values <ver_boundary>` * :ref:`Equivalence Classes <ver_equivalence>` * :ref:`Fuzzy Testing <ver_fuzzy>` |
|
Verification Methods |
valid |
Following methods are explained * :ref:`Fault Injection <ver_fault>` * :ref:`Interface Test <ver_interface>` * :ref:`Structural Coverage <ver_structural>` * :ref:`Static Code Analysis <ver_sta>` |
|
Architecture attribute: fulfils |
valid |
Architectural views (feature/comp_arc_sta, feature/comp_arc_dyn) and interfaces (logic/real_arc_int) should be linked to a requirement on the corresponding level. **Examples:** * feat_req <-> feat_arc_(sta|dyn), logic_arc_int * comp_req <-> comp_arc_(sta|dyn), real_arc_int |
|
Architecture attribute: fulfils (AoU) |
valid |
Architectural elements (feat_arc_sta, comp) shall be linked to AoUs if the element fulfills these. |
|
Check of Architecture mandatory attributes |
valid |
It shall be checked if all mandatory attributes for each architectural element are provided by the user. For all elements following attributes shall be mandatory: .. needtable:: Overview mandatory requirement attributes :filter: "mandatory" in tags and "attribute" in tags and "architecture_design" in tags and type == "gd_req" and is_external == False :style: table :columns: title :colwidths: 30 |
|
Architecture attribute: safety |
valid |
Each architectural element shall have a automotive safety integrity level (ASIL) identifier: * QM * ASIL_B |
|
Architecture attribute: security |
valid |
Each architectural element shall have a security relevance identifier: * Yes * No |
|
Architecture attribute: status |
valid |
Each architectural element shall have a status: * valid * invalid |
|
Architecture attribute: UID |
valid |
Each architectural element shall have a unique ID. It shall be in a format which is also human readable and consists of * type of architectural element * structural element (e.g. some part of the feature tree, component acronym) * keyword describing the content of the architectural element Check your project's naming conventions (should be called "doc__naming_conventions") |
|
Structuring of the architectural elements |
valid |
Following architectural elements shall be defined on the respective hierarchical level: * Logical Level * Feature (feat) * Feature (feature_arc_sta) * Feature (feature_arc_dyn) * Logical Interface (logic_arc_int) * Logical Interface Operation (logic_arc_int_op) * Component Level * Component (comp) * Component (comp_arc_sta) * Component (comp_arc_dyn) * Interface (real_arc_int) * Interface Operation (real_arc_int_op) * Module * SW Module (mod) |
|
Correlations of the architectural building blocks |
valid |
For modeling the viewpoints following relations shall be used: .. figure:: ../_assets/metamodel_architectural_design.drawio.svg :width: 100% :align: center :alt: Definition of the Metamodel for Architectural Design |
|
Check of Architecture consistency in dynamic architecture |
valid |
It shall be checked if all SW components which are mentioned in the dynamic architecture views are defined in the static architecture. |
|
Check of Architecture consistency interfaces in modules |
valid |
It shall be checked if any interface linked to the features (link from Logical Arc. Interface to Feature via the "included_by" link must be defined and exists) is matched by an "implements" link in the Module (from component to Logical Arc. Interface). Additionally it shall be checked if the feature architecture are linked against at least one logical architectural interface. |
|
Hierarchical structure of architectural elements |
valid |
The architectural elements shall be hierarchically structured on two levels: * Feature Level (=Logical Level) * Component Level (allows also recursive decomposition into internal components) |
|
Check of Architecture linkage to AoU |
valid |
It shall be checked that architectural static view (feature/comp_arc_sta) are not linked to its own AoU ("own" means the AoU linked as "mitigated_by" to the Safety/Security Analysis linked via "violates" to the element, another equivalent distinguishing is that the "own" AoU are in the same repository whereas the "other" are in another repository). |
|
Check of Architecture linkage requirement |
valid |
It shall be checked that each architectural element (safety!=QM) is linked against at least one safety requirement (safety!=QM). It shall be checked that architectural elements with safety=QM are not linked against safety requirements (safety!=QM). |
|
Check of Architecture linkage requirement type |
valid |
It shall be checked that requirements of a respective type can only be linked to architectural elements according to following traceability: * Functional requirements <-> static / dynamic architectural elements (feat_arc_sta, feat_arc_dyn) * Interface requirements <-> interface architectural elements (logic_arc_int, logic_arc_int_op) |
|
Check of Architecture linkage metamodel |
valid |
It shall be checked that every valid safety architectural element is linked according to the defined model :need:`gd_req__arch_build_blocks_corr`. |
|
Check of Architecture linkage safety |
valid |
It shall be checked that valid safety architectural elements (Safety!=QM) can only be linked against valid safety architectural elements. |
|
Check of Architecture linkage security |
valid |
It shall be checked that security relevant architectural elements (Security==YES) can only be linked against security relevant architectural elements. |
|
Architecture Modeling |
valid |
For architecture design a model based approach should be used. The model shall consist of different architectural elements. |
|
Monitor architecture process performance |
valid |
Information about the execution and outcomes of the architecture process shall be collected and analyzed to evaluate the effectiveness and adequacy of the defined process. Analysis results shall be made available to affected parties and used to identify and trigger continual improvement actions for the architecture process. Monitoring is expected as a periodic, predominantly manual activity (for example as part of quality checks and retrospectives), not as a fully automated check. Improvement actions should be handled according to :ref:`pm_monitor_improve_process`. |
|
Architecture traceability |
valid |
Requirements shall be satisfied by an architectural element on the corresponding level. **Examples:** * feat_req <-> feat * comp_req <-> comp .. note:: In general the traceability is visualized in :ref:`general_concepts_traceability` |
|
Architecture Viewpoints |
valid |
The architecture shall be shown on following views on each architectural level: * Package Diagram (feat_arc_sta, comp_arc_sta) * Sequence Diagram (feat_arc_dyn, comp_arc_dyn) * Interface View (logic_arc_int, real_arc_int) At the module level, only one additional static view (mod_view_sta) shall be created. |
|
Change Request attribute: Affected Work Products |
valid |
Links to the work products affected by the Change Request |
|
Change Request attribute: description |
valid |
Exact description of the Change Request, including impact analysis on functional safety, security, implementation (schedule, risks, resources) verification (measures defined). |
|
Change Request attribute: safety |
valid |
Each Change Request shall have a automotive safety integrity level (ASIL) identifier: * QM * ASIL_B |
|
Change Request attribute: security |
valid |
Each Change Request shall have a security relevance identifier: * Yes * No |
|
Change Requests mandatory attributes provided |
valid |
It shall be checked if all mandatory attributes for each Change Request is provided by the user. For all requirements following attributes shall be mandatory: .. needtable:: Overview mandatory change request attributes :filter: "mandatory" in tags and "attribute" in tags and "change_management" in tags and is_external == False :style: table :columns: title :colwidths: 30 |
|
Change Request attribute: Milestone |
valid |
Milestone until the Change Request must be implemented (used for prioritization) |
|
Change Request attribute: status |
valid |
Each Change Request shall have a status: * open * in review * in implementation * closed * rejected |
|
Change Request attribute: title |
valid |
Reason for the Change Request |
|
Change Request attribute: Tracking |
valid |
Links to the issues implementing Change Request |
|
Change Request attribute: Types |
valid |
* Feature * Feature Modification * Component * Component Modification Feature This Change Request describes a potential new feature for the platform. The Change Request uses the Feature request template: :ref:`chm_feature_templates`. Feature Modification This Change Request describes a scope modification of an existing feature (requirement or work product). The Change Request modifies the already existing Feature Request template: :ref:`chm_feature_templates`. Component This Change Request describes a potential new component for the platform. The Change Request uses the Component request template: :ref:`chm_component_templates`. Component Modification This Change Request describes a scope modification of an existing component (requirement or work product). The Change Request modifies the already existing Component Request template: :ref:`chm_component_templates`. |
|
Change Request attribute: UID |
valid |
Each Change Request shall have a unique ID. It shall be in an integer number. |
|
Change Requests Impact Analysis Tool |
valid |
It shall be reported, which work products and elements are affected by adding a new feature or component or by a modification of an existing feature or component. Below picture illustrates the relations: .. figure:: ../_assets/impact_analysis.drawio.svg :width: 100% :align: center :alt: Which changes to follow for impact analysis. The picture is derived from the building blocks metamodel containing all the work products (:ref:`general_concepts_building_blocks`). Its arrows show how every work product is linked (manually) to other work products (e.g. feature requirements are linked to stakeholder requirements via "derived_from"). The color code describes which of these Links need to be followed for the impact analysis (so for the example this means: if a stakeholder requirement is changed, the feature requirements linked are affected, but not the other way round). "Black" links do not need to be followed, these are the "verifies" links. And these are expected to fail after implementation change, so the impact on testing will be detected by the test automation without the need for an additional tooling. Note that for some links it is expected to have both directions followed (red arrows) - these are the artefacts which reside in different repositories. Note also: The impact analysis needs to follow the links iteratively, so first to the directly connected work products, then to the next, ... (see the next illustration): .. figure:: ../_assets/impact_iteration.drawio.svg :width: 100% :align: center :alt: How to follow changes for impact analysis iteratively. |
|
Baseline Differences |
valid |
It shall be possible to show the differences between two baselines. Note: This could be done by showing all the commits which happened between these baselines in one release branch. |
|
Document attributes |
valid |
It shall be prohibited to override any mandatory attribute value of an docs-as-code element. Note: This requirement exists because docs-as-code attributes may be globally overridden, which leads to a mismatch of code and the representing generated documentation. Exception only exists for optional tags, which can be used for filtering and reporting purposes. |
|
Global tags extension |
valid |
It shall be possible to define global tags with the docs-as-code tool, which can be used for filtering and reporting. Note: This requirement exists to enable the use of global tags for filtering and reporting purposes, while still prohibiting the overriding of mandatory attributes or elements globally. |
|
Storage of pull requests documentation |
valid |
The content of pull requests (conversation, commits, files changed) shall be stored permanently for every release. Note: This requirement exists because PRs may be modified after a release, which could corrupt the documented inspection results recorded during the review. |
|
Permanent Storage |
valid |
At least every platform release shall be stored permanently as a collection of text documents (docs and code) including the used OSS tooling on project owned servers. Note: This is to ensure to have the development artefacts available during the complete lifetime of the products (cars) the SW platform is used in. |
|
Unique Id |
valid |
The Docs-as-Code tool shall check that the Id's of the configuration items (documented in doc-as-code) are unique. Note: For definition of configuration items see :need:`doc_concept__configuration_process` |
|
Document Approver |
valid |
The version management tool shall document and report (be able to show) the approver of a document. I.e. for each change of a document the approver of the change is stored, which is usually the person with "write-rights" on the document approving the merge of a Pull Request (this may also be more than one person). Note that every approver is also reviewer. |
|
Document attribute: status |
valid |
Each document, shall have a status depending on the document lifecycle models below: * Model 1: VALID (e.g. for simple reports) * Model 2: DRAFT -> VALID (e.g. for plans) * Model 3: DRAFT-> VALID -> VALID(inspected) (e.g. for stakeholder requirements) Compare :need:`gd_guidl__documentation` |
|
Document attributes |
valid |
Generic documents shall have the following mandatory manual attributes: * id * status * security * safety * realizes The attribute "tags" is optional. Compare also :need:`gd_temp__documentation` |
|
Document Author |
valid |
The version management tool shall document and report (be able to show) the authorship of a document. I.e. for each change of a document the author of the changes is stored. |
|
Document Reviewer |
valid |
The version management tool shall document and report (be able to show) the reviewers of a document. I.e. for each change of a document the reviewers of the change are stored. |
|
Document Types |
valid |
There are the following document types: * document * doc_tool .. note:: The type "document" is the GENERIC type, which can be used for realizing concrete work products, with the exception of the tool verification report (which uses the "doc_tool" type). The "doc_tool" type is described in :ref:`tlm_process_requirements`. .. note:: Process documents are not documents realizing work products, types for that shall only used for process definition, as defined * gd_chklst * gd_guidl * gd_req * gd_temp * doc_concept * doc_getstrt * workproduct * workflow * role |
|
Version for inspected architecture |
valid |
The version of architecture element shall not change by an inspection. This means: In case the status of the element (see :need:`gd_req__arch_attr_status`) is used to notify if it is inspected (or another attribute is introduced), this shall be ignored for versioning. Note: this applies only if architecture also has a version. |
|
Checklist templates in pull requests |
valid |
For every pull request that modifies a work product subject to inspection, a pull‑request template containing the applicable inspection checklist items shall be provided. When possible, the template should be applied automatically based on the files modified in the pull request. As of now, requirements and architecture inspection checklists shall be added manually because they are not automatically applied. |
|
Version for inspected requirements |
valid |
The version of a requirement shall not change by an inspection. This means: In case the status of the requirement (see :need:`gd_req__req_attr_status`) is used to notify if a requirement is inspected (or another attribute is introduced), this shall be ignored for versioning. |
|
Status Reset Check |
valid |
It shall be checked that the status is reset to valid whenever a requirement is modified (changes version). |
|
Status Set Check |
valid |
It shall be checked that only a PR with the inspection checklist filled out can set a status to valid(inspected). |
|
Design Complexity Analysis |
valid |
A complexity analysis for the components shall be performed by automated tool support. It shall consider appropriate code metrics like lines of code, cyclomatic complexity, number of public interfaces, number of parameters and so on. The results of the analysis shall be documented in the SW Verification Report. As default an exceeds of the following limits shall be reported for the complexity measures (ASIL B / QM): * Lines of Code per function: 100 / 200 * Lines of Code per file : 2000 / 4000 * Cyclomatic complexity per function: 10 / 20 * Number of parameters per function: 5 / 7 * Number of public interfaces per component: 3 / 10 The project may specify own limits for the complexity measures in the project guidelines, if there is a rationale for deviating from the default limits. Therefore the tooling shall support configuration of the limits globally for the project and per component, if the component wants to use own lower (more strict) limits. |
|
Dependency Analysis |
valid |
For each component a dependency tree view shall be created to support design inspection and Safety Analysis. It shall show the libraries used by the component (i.e. which libraries are linked to the component, defined as CI build tool target) up to the leaves of the tree. |
|
Design Decision Rationale |
valid |
The detailed design decisions shall be documented and justified, in particular those that shape the decomposition of a component into its units. The rationale shall be captured close to the design, i.e. in the source and header files (e.g. as doxygen-style comments) and, where applicable, in the detailed design documentation. |
|
Diagram Linkage check Component ID |
valid |
Each diagram shall be linked to the corresponding component id via the attribute belongs_to. |
|
Diagram mandatory consistency |
valid |
It shall be checked if all diagrams are consistent with the source code and the design principles outlined in the development plan. This includes checking that the naming of the units, their interfaces and functions in any diagrams or descriptions matches the naming in the source code to ensure traceability. That means the diagrams and descriptions should not be outdated and be consistent with the source code and not introduce new terminology or concepts that are not present in the code. |
|
Diagram attribute: description |
valid |
Each diagram shall have a description. |
|
Diagram Linkage Component ID |
valid |
Each diagram shall be automatically linked (inverse direction) to the corresponding component id via the "belongs by" linkage. |
|
Diagram attribute: title |
valid |
The title of the diagram shall provide a short summary of the description, but is not an "additional" requirement. This means for example that the word "shall" is not allowed in the title for all diagram. |
|
Diagram attribute: UID |
valid |
Each diagram shall have a unique identifier (UID) (realized by its file name). It shall consist of three parts: * type of diagram * structural element * keyword describing the content of the diagram Consider the project's naming convention. |
|
Interface attribute: description |
valid |
Each interface shall have a description in the source code if the source code does not already provide sufficient information. It should provide a clear and comprehensive explanation of the interface's purpose, its inputs and outputs, and how it interacts with other units or components. The description should be sufficient to allow users of the interface to understand how to interact with it without needing to read the implementation details. It should also include any relevant information about the expected behavior, constraints, and any assumptions made in the design of the interface. The documentation should be maintained and updated as the implementation evolves to ensure it remains accurate and useful. |
|
Interface attribute: UID |
valid |
Each interface shall have a unique ID, which is defined by the path and file name(s) of the interface within the component and the namespace and name of the interface inside the file. The name should be descriptive and reflect the functionality of the interface to ensure traceability and understandability. Consider the project's naming convention. |
|
Static Diagram for Unit Interactions |
valid |
A static diagram is optional. The decomposition of a component into its units is primarily represented by the directory and file structure (see :need:`doc_concept__imp_concept`), so a static diagram is only added when it helps to explain complex components or a large number of unit interactions. If a static diagram is provided, it shall represent the units and their relationships using UML notations, and it shall provide the attributes and linkages defined below. |
|
Unit attribute: description |
valid |
Each unit shall have a description in the source code. |
|
Unit attribute: UID |
valid |
Each unit shall have a unique ID. It is build by the path and file name(s) of the unit within the component and following a consistent naming convention. The name should be descriptive and reflect the functionality of the unit to ensure traceability and understandability. The naming convention should be defined in the project guidelines and consistently applied across all units. |
|
Problem attribute: analysis results |
valid |
Record analysis results (e.g. reason for rejection, safety, security, quality impact) as comments. |
|
Problem attribute: classification |
valid |
Each Problem shall have a classification identifier: * minor * major * critical * blocker |
|
Problem attribute: description |
valid |
Exact description of the Problem, including potential cause and impact of the problem. Record especially, if functional safety or cybersecurity may affected here. Record potential affected parties, and if it may required to notify them about the potential problem. |
|
Problem attribute: milestone |
valid |
Milestone until the Problem must be implemented (used for prioritization) |
|
Problem attribute:: safety affected |
valid |
Each Problem shall have a safety relevance identifier: * Yes * No Note: If neither security or safety relevance is set the bug is always quality relevant. |
|
Problem attribute:: security affected |
valid |
Each Problem shall have a security relevance identifier: * Yes * No Note: If neither security or safety relevance is set the bug is always quality relevant. |
|
Problem attribute: stakeholder |
valid |
Assign responsible stakeholder for analyzing the problem Assign responsible stakeholder to resolve the problem |
|
Problem attribute: status |
valid |
Each Problem shall have a status: * open * in review * in implementation * closed * rejected |
|
Problem attribute: title |
valid |
Reason for Problem Report |
|
Problem attribute: UID |
valid |
Each Problem shall have a unique ID. It shall be in an integer number. |
|
Problem Report issues closing constraints |
valid |
ISSUEs related to Problem Reports shall not automatically closed, if linked ISSUEs or PRs are closed or merged and these ISSUEs shall be closed only manually from the :need:`Committer <rl__committer>`. |
|
Problem Resolution mandatory attributes provided |
valid |
It shall be checked if all mandatory attributes for each Problem is provided by the user. Following attributes shall be mandatory: .. needtable:: Overview of mandatory problem attributes :filter: "mandatory" in tags and "attribute" and "problem_resolution" in tags and is_external == False :style: table :columns: title :colwidths: 30 Note: See also template for problem report: :need:`gd_temp__problem_template` |
|
Process Model Building Blocks |
valid |
The process model building blocks are defined. Compare also the process model overview here: :ref:`processes_introduction`. Following building blocks are defined: * Workflow * Work Product * Role (includes Team) Additionally there shall be the following additions for workflows: * Getting Started * Concept * Guidance Additionally there shall be the following details for guidance: * Guideline * Template * Checklist * Method * Process Requirements Additionally there shall be the following additions for standard compliance: * Standard Work Products * Standard Requirements Additionally there shall be the following additions to allow deployment examples or templates for later deployment for work products: * Document |
|
Building blocks attributes |
valid |
Each building block shall have defined attributes as defined here: :ref:`process_management_templates`. Some have further constraints as defined here: * The UID attribute must be unique. * The STATUS attribute shall have a status: <draft|valid> * The SAFETY attribute shall have level: <QM | ASIL_B> * The SECURITY attribute shall have level: <YES|NO> * Each building block shall have a description |
|
Building blocks check |
valid |
It shall be checked, that all attributes defined here :ref:`process_management_templates` are provided and correctly linked by the user. |
|
Building blocks linkage |
valid |
Each building block shall have defined links as defined here: :ref:`process_management_templates`. |
|
Quality report automated generation |
valid |
| The quality report shall be generated progressively and automatically compiling the work products. | A template exists to guide the reporting and the right collection of the required work products. |
|
Release note automated generation |
valid |
| The release note shall be generated progressively and automatically compiling the content as far as possible. | This shall be done according to templates :need:`gd_temp__rel_plat_rel_note` and :need:`gd_temp__rel_mod_rel_note`. |
|
Requirement attribute: description |
valid |
Each requirement shall have a description. .. note:: *ISO/IEC/IEEE/29148 - Systems and software engineering — Life cycle processes — Requirements engineering* defines general concepts including terms and examples for functional requirements syntax. The concepts shall apply. |
|
Requirement attribute: link to implementation |
valid |
It shall be possible to link requirements to code (to the respective line of code in an attribute of the requirement). |
|
Requirement attribute: rationale |
valid |
Each stakeholder requirement shall provide an attribute called rationale. The rationale shall contain the reason why the requirement is needed. |
|
Requirement attribute: safety |
valid |
Each requirement, apart from process and tool requirements, shall have a automotive safety integrity level (ASIL) identifier: * QM * ASIL_B |
|
Requirements attribute: security |
valid |
Each requirement, apart from process and tool requirements, shall have a security relevance identifier: * Yes * No |
|
Requirement attribute: status |
valid |
Each requirement, apart from process and tool requirements, shall have a status: * valid * invalid |
|
Requirement attribute: complete test coverage |
valid |
It shall be possible to specify if a requirement is completely satisfied by the linked test case(s). * Yes * No |
|
Requirement attribute: link to test |
valid |
It shall be possible to link requirements to tests and automatically include a link to the test case in the attribute testlink. |
|
Requirement attribute: title |
valid |
The title of the requirement shall provide a short summary of the description, but is not an "additional" requirement. This means for example that the word "shall" is not allowed in the title for all requirements. |
|
Requirement attribute: type |
valid |
Each requirement, apart from process and tool requirements, shall have a type of one of following options: * Functional * Interface * Process * Non-Functional |
|
Requirement attribute: UID |
valid |
Each requirement shall have a unique ID. It shall consist of three parts: * type of requirement * structural element (e.g. some part of the feature tree, component acronym) * keyword describing the content of the requirement. Consider the project's naming convention. |
|
Requirement attribute: valid_from |
valid |
Stakeholder and feature requirements can have a validity attribute that tells from which milestone onwards the requirement is part of a feature. This validity attribute is defined as including the defined milestone. Milestone shall be valid release version tag, e.g. v1.0.2 as defined in Platform Release Note Template: :need:`gd_temp__rel_plat_rel_note`. Thus the corresponding requirement is valid from the defined milestone, including it. |
|
Requirement attribute: valid_until |
valid |
Stakeholder and feature requirements can have a validity attribute that tells until which milestone the requirement is part of a feature. This validity attribute is defined as excluding the defined milestone. Milestone shall be valid release version tag, e.g. v1.0.2 as defined in Platform Release Note Template: :need:`gd_temp__rel_plat_rel_note`. Thus the corresponding requirement is only valid until the defined milestone, excluding it. |
|
Requirement attribute: versioning |
valid |
A versioning for requirements shall be provided. For this all significant attributes shall be taken into account: see :ref:`requirement_versioning` |
|
Requirements mandatory attributes provided |
valid |
It shall be checked if all mandatory attributes for each requirement is provided by the user. For all requirements following attributes shall be mandatory: .. needtable:: Overview mandatory requirement attributes :filter: "mandatory" in tags and "attribute" in tags and "requirements_engineering" in tags and type == "gd_req" and is_external == False :style: table :columns: title :colwidths: 30 There is one exception from the above: "rationale" attribute is mandatory only for stakeholder requirements. |
|
Requirements no weak words |
valid |
It shall be ensured that no *weak words* are contained in the requirement description for: * Stakeholder Requirements * Feature Requirements * Component Requirements * Tool Requirements List of these weak words: just, about, really, some, thing, absolutely (to be extended) |
|
Requirement Linkage |
valid |
Requirements shall be linked to its adjacent level via the attribute derived_from. * stakeholder requirements <- feature requirements * feature requirements <- component requirements * process requirements or stakeholder requirements <- tool requirements |
|
Requirement Linkage to AoU |
valid |
Requirements shall be linked to AoU via the attribute covers, if they already cover these. * AoU <- feature requirements * AoU <- component requirements Note: "Already" illustrates the understanding that AoU come from used (external) components which appear later in the development (after deciding an architecture). |
|
Requirements linkage architecture |
valid |
It shall be checked if every feature requirement is linked via fulfilled_by at least to one valid architectural diagram/interface on the same level. This should also include requirement type checking: * If the requirement is of type "Functional" it shall be linked to architecture diagram elements. * If the requirement is of type "Interface" it shall be linked to architecture interface elements. Note that the linking done via "fulfils" from the architecture element to the requirement as described for example in :ref:`allocate_feature_requirements`. |
|
Requirements mandatory architecture linkage |
valid |
Every feature- and component requirement shall be linked via satisfied_by at least to one valid architectural element on the same level. * feat_req -> feature (feat) * comp_req -> component (comp) |
|
Requirements linkage architecture switch |
valid |
The check :need:`gd_req__req_linkage_architecture` shall only be enabled for a release build, otherwise it would block creating requirements first without architecture. |
|
Requirements linkage level |
valid |
Every feature- and component requirement shall be linked to at least one parent requirement according to the defined traceability scheme: :ref:`traceability concept for requirements` |
|
Requirements linkage safety |
valid |
It shall be checked that (child) QM requirements (Safety == QM) can not be linked against a (parent) safety requirement (Safety != QM). Note: This ensures that safety requirements are properly derived into their children. Also a mix of safe and QM aspects in a parent is avoided by this. |
|
Requirements Structure |
valid |
Requirements shall be hierarchically grouped into three levels. Following levels are defined: * Stakeholder requirement * Feature requirement * Component requirement Additionally there shall be the following logical groups of requirements: * Assumption of use requirement * Process requirement * Tool requirement |
|
Requirement check: suspicious |
valid |
Based on the requirement versioning it shall be checked if a requirement was updated but not the linked tests. In case an update was detected the attribute `complete test coverage` shall be set to "No" Note: This refers to :need:`gd_req__req_attr_test_covered` |
|
Requirement Traceability |
valid |
Bi-directional traceability shall be provided by adding a "back-link" via attribute derives (i.e. make a <-> out of the <- in :need:`gd_req__req_linkage`). |
|
Requirements validity |
valid |
Validity attributes (:need:`gd_req__req_attr_valid_from` and :need:`gd_req__req_attr_valid_until`) shall be checked for correctness (i.e. they denote an existing milestone) and consistent (e.g. the until is not before from) Several of the above checks are not to be executed on requirements not valid in the next milestone, these are TBD |
|
Safety Analysis content: argument |
valid |
The argument shall describe why the mitigation (e.g. prevention, detection or mitigation) is sufficient or not. If it is not sufficient, the argument shall describe how the mitigation can be improved to achieve sufficiency. The argument shall be written in the content. |
|
Safety Analysis attribute: link to Aou |
valid |
It shall be possible to link Aou. |
|
DFA attribute: failure ID |
valid |
Each DFA shall have a failure ID. The failure ID is used to identify the related fault <:need:`gd_guidl__dfa_failure_initiators`>. The failure ID links to the corresponding failure initiator which describes how a potential violation can occur. |
|
FMEA attribute: failure root cause |
valid |
Each FMEA may provide a short description of the root cause of the failure. |
|
FMEA attribute: fault ID |
valid |
Each FMEA shall have a fault ID. The fault ID is used to identify the related fault <:need:`gd_guidl__fault_models`>. The fault ID links to the corresponding fault which describes how a potential violation can occur. |
|
Safety Analysis attribute: failure effect |
valid |
Every Safety Analysis shall have a short description of the failure effect (e.g. failure lead to an unintended actuation of the analysed element) |
|
Safety Analysis mandatory attributes provided |
valid |
It shall be checked if all mandatory attributes for each Safety Analysis are provided by the user. For all Safety Analysis following attributes shall be mandatory: .. needtable:: Overview mandatory Safety Analysis attributes :filter: "mandatory" in tags and "attribute" in tags and "safety_analysis" in tags and type == "gd_req" :style: table :columns: title :colwidths: 30 |
|
Safety Analysis attribute: mitigated by |
valid |
Each violation shall have an associated mitigation (e.g. prevention, detection or mitigation) or AoU. If mitigation has not yet been implemented, do not use this option. If status == valid then mitigated_by is mandatory. |
|
Safety Analysis attribute: mitigation issue |
valid |
If a new mitigation (e.g. prevention, detection or mitigation) is needed, link to the issue and keep status sufficient == no until mitigation is sufficient. |
|
Safety Analysis attribute: Requirements linkage |
valid |
Each Safety Analysis shall be automatically linked to the corresponding Safety Requirement via the mitigates linkage. |
|
Safety Analysis attribute: check Requirements linkage |
valid |
Safety Analysis shall be linked to a requirement on the corresponding level via the attribute "mitigated by". |
|
Safety Analysis attribute: safety relevant |
valid |
Each Safety Analysis may indicate whether the analysed failure is safety relevant. The value shall be either <yes> or <no>. |
|
Safety Analysis attribute: status |
valid |
Each Safety Analysis shall have a status which can be either "valid" or "invalid". |
|
Safety Analysis attribute: sufficient |
valid |
The mitigation(s) (e.g. prevention, detection or mitigation) shall be rated as sufficient with <yes> or <no>. A mitigation can only be sufficient if a mitigation is linked via the attribute mitigation. |
|
Safety Analysis attribute: title |
valid |
The title of the Safety Analysis shall provide a short summary of the description |
|
Safety Analysis attribute: UID |
valid |
Each Safety Analysis shall have a unique ID. It shall be in a format which is also human readable and consists of * Type of Safety Analysis (DFA or FMEA) * Name of analysed structural element (e.g. Persistency, FEO, etc.) * Element descriptor (e.g. KVS__Open KVS or KVS__GetKeyValue) The naming convention shall be defined in the project and shall be used consistently. |
|
Safety Analysis attribute: versioning |
valid |
It shall be possible to detect any differences in mandatory attributes compared to the versioning: :need:`gd_req__saf_attr_mandatory` |
|
Safety Analysis Linkage |
valid |
Each Safety Analysis shall be automatically linked (inverse direction) to the corresponding architecture view via the "violates by" linkage. The linkages are defined in detail at :need:`gd_req__saf_linkage_check`. |
|
Safety Analysis Linkage check |
valid |
Safety Analysis shall be linked to the architecture view on the corresponding level via the attribute violates. The following linkages are defined in detail: * plat_saf_dfa <-> feat_arc_sta * feat_saf_dfa <-> feat_arc_sta * feat_saf_dfa <-> comp_arc_sta * feat_saf_fmea <-> feat_arc_dyn, feat_arc_sta * comp_saf_fmea <-> comp_arc_dyn, comp_arc_sta |
|
Safety Analysis linkage safety |
valid |
It shall be checked that Safety Analysis (DFA and FMEA) can only be linked via mitigate_by against <Feature | Component | AoU> Requirements with at least one Requirement with the same ASIL or with a higher ASIL as the corresponding ASIL of the Feature or Component that is analysed and linked via violates. |
|
Safety Analysis Linkage status check |
valid |
It shall be checked that Safety Analysis can only be linked against valid safety elements (architecture view, requirement, AoU). A valid safety element has the attribute 'status == valid' and safety != QM. |
|
Safety Analysis Structure |
valid |
Safety Analysis (FMEA and DFA) shall be hierarchically grouped into different levels. Following levels are defined: * Platform DFA * Feature DFA/FMEA * Component DFA/FMEA |
|
Safety Management Document Status |
valid |
Safety plans shall contain documents references where the status is derived automatically. Note: This can be done by defining the document as a sphinx-need and using sphinx mechanisms. |
|
Safety Management Work Product Status |
valid |
Safety plans shall contain work product references where the accumulated status is derived automatically. Note: This can be done as for documents if the work product is a single sphinx-need. For work products collections (e.g. all requirements of a component) an accumulated status is needed (e.g. like "% valid state") |
|
Security Analysis content: argument |
valid |
The argument shall describe why the mitigation is sufficient or not. If it is not sufficient, the argument shall describe how the mitigation can be improved to achieve sufficiency. The argument shall be written in the content. |
|
Security Analysis attribute: link to Aou |
valid |
It shall be possible to link AoU. |
|
Security Analysis mandatory attributes provided |
valid |
It shall be checked if all mandatory attributes for each Security Analysis are provided by the user. For all Security Analysis following attributes shall be mandatory: .. needtable:: Overview mandatory Security Analysis attributes :filter: "mandatory" in tags and "attribute" in tags and "security_analysis" in tags and type == "gd_req" :style: table :columns: title :colwidths: 30 |
|
Security Analysis attribute: mitigated by |
valid |
Each threat shall have an associated treatment (accept, avoid, reduce, share) or AoU. If mitigation has not yet been implemented, do not use this option. If status == valid then mitigated_by is mandatory. |
|
Security Analysis attribute: mitigation issue |
valid |
If a new security mitigation (avoid, reduce, or share) is needed, link to the issue and keep status invalid until the mitigation is sufficient. |
|
Security Analysis attribute: Requirements linkage |
valid |
Each Security Analysis shall be automatically linked to the corresponding Security Requirement via the mitigates linkage. |
|
Security Analysis attribute: check Requirements linkage |
valid |
Security Analysis shall be linked to a requirement on the corresponding level via the attribute "mitigated by". |
|
Security Analysis attribute: status |
valid |
Each Security Analysis shall have the status invalid until the analysis is finished. The status shall be set to valid if the analysis is finished and all issues are closed. |
|
Threat Model attribute: threat ID |
valid |
Each threat used for Security Analysis shall have a threat ID. The threat ID is used to identify the related threat <:need:`gd_guidl__threat_models_stride`>. The threat ID links to the corresponding threat which describes how a potential attack can occur. |
|
Security Analysis attribute: sufficient |
valid |
The mitigation(s) shall be rated as sufficient with <yes> or <no>. A mitigation can only be sufficient if a mitigation is linked via the attribute mitigation. |
|
Security Analysis attribute: threat impact |
valid |
Every Security Analysis shall have a short description of the threat impact (e.g. threat leads to unauthorized access of the analyzed element) |
|
Security Analysis attribute: threat scenario ID |
valid |
Each threat scenario used for the Security Analysis shall have a threat scenario ID. The threat scenario ID is used to identify the related threat <:need:`gd_guidl__sec_ana_threat_scenarios`>. The threat scenario ID links to the corresponding threat scenario which describes how a potential attack can occur. |
|
Security Analysis attribute: title |
valid |
The title of the Security Analysis shall provide a short summary of the description |
|
Security Analysis attribute: UID |
valid |
Each Security Analysis shall have a unique ID. It shall be in a format which is also human readable and consists of * Type of Security Analysis * Name of analyzed structural element (e.g. Persistency, FEO, etc.) * Element descriptor (e.g. KVS__Open KVS or KVS__GetKeyValue) The naming convention shall be defined in the project and shall be used consistently. |
|
Security Analysis attribute: versioning |
valid |
It shall be possible to detect any differences in mandatory attributes compared to the versioning: :need:`gd_req__sec_attr_mandatory`. |
|
Security Analysis finalization check |
valid |
It shall be checked if all artifacts of the analysis are "valid" and "sufficient". |
|
Security Analysis Linkage |
valid |
Each Security Analysis shall be automatically linked (inverse direction) to the corresponding architecture view via the "violates by" linkage. |
|
Security Analysis Linkage check |
valid |
Security Analysis shall be linked to the architecture view on the corresponding level via the attribute violates. |
|
Security Analysis linkage security |
valid |
It shall be checked that Security Analysis can only be linked via mitigate_by against <Feature | Component | AoU> Requirements with at least one Requirement with the security identifier set that is analyzed and linked via violates. |
|
Security Analysis Linkage status check |
valid |
It shall be checked that the Security Analysis can only be linked against valid security elements (architecture view, requirement, AoU). A valid security element has the attribute 'status == valid' and is security-relevant. |
|
Security Analysis Structure |
valid |
Security Analysis shall be hierarchically grouped into different levels. Following levels are defined: * Platform * Feature * Component |
|
Security Management attribute: status derivation |
valid |
Security Plans shall contain documents references where the status is derived automatically. Note: This can be done by defining the document as a sphinx-need and using sphinx mechanisms. |
|
Security Management attribute: status accumulation |
valid |
Security Plans shall contain work product references where the accumulated status is derived automatically. Note: This can be done as for documents if the work product is a single sphinx-need. For work products collections (e.g. all requirements of a component) an accumulated status is needed (e.g. like "% valid state") |
|
Tool attribute:: safety affected |
valid |
Each Tool Verification Report shall have a safety relevance identifier: * YES * NO |
|
Tool attribute:: security affected |
valid |
Each Tool Verification Report shall have a security relevance identifier: * YES * NO |
|
Tool attribute: status |
valid |
Each Tool Verification Report shall have a status. * draft * evaluated * qualified * released * rejected |
|
Tool attribute:: tcl |
valid |
Each Tool Verification Report shall have a tool confidence level: * LOW * HIGH |
|
Tool attribute: UID |
valid |
Each Tool Verification Report shall have a unique ID. |
|
Tool attribute:: version |
valid |
Each Tool Verification Report shall document the tool version it applies to: * v.X.Y.Z (major, minor, patch) |
|
Tool Management mandatory attributes provided |
valid |
It shall be checked if all mandatory attributes for each Tool Verification Report is provided by the user. For all requirements following attributes shall be mandatory: .. needtable:: Overview mandatory problem attributes :filter: "mandatory" in tags and "attribute" and "tool_management" in tags and is_external == False :style: table :columns: title :colwidths: 30 |
|
Verification Documentation Checks |
valid |
The following checks shall be implemented on test metadata: - TestType and DerivationTechnique shall be set - Description shall not be empty - In a Platform Integration Test Partially/FullyVerifies shall be set to at least one Platform Requirement - If Partially/FullyVerifies are set in Feature Integration Test these shall link to at least one Feature Requirement - If Partially/FullyVerifies are set in Component Integration Test these shall link to at least one Component Requirement - If Partially/FullyVerifies are set in Unit Test these shall link to at least one Component Requirement |
|
Verification Documentation Checks Extended |
valid |
The following checks shall be implemented on test metadata: - If TestType is set to requirements-based then PartiallyVerifies or FullyVerifies shall contain a link to at least one requirement - If TestType is set to interface-test then PartiallyVerifies or FullyVerifies shall contain a link to at least one interface |
|
CI reference integration execution |
valid |
The CI reference integration execution shall be triggered on regular basis to guarantee the inter-operation of all integrated components when a new component is added to the system or an existing component is updated via a PR. **TODO: Align this to the ongoing work in https://github.com/eclipse-score/reference_integration/pull/190 which reworks the CI reference integration execution.** |
|
Verification of External Components AoUs |
valid |
External components AoUs shall be verified by integration tests or static analysis tooling. Note1: These external components AoU need to be checked generally for the complete platform, so we create a process automation requirement for this. Note2: One example would be the checking if safety components are only using "safe" functions of the operating system. |
|
Independence |
valid |
The approver of a pull request shall differ from the author(s) of the pull request in all pull requests. |
|
Linking Requirements to Tests |
valid |
For linking test suites to requirements following metadata shall be used: * Verifies * PartiallyVerifies * FullyVerifies * Description * TestType * Fault Injection (fault-injection) * Interface Test (interface-test) * Requirements-based Test (requirements-based) * Resource Usage Evaluation (resource-usage) * DerivationTechnique * Analysis of requirements (requirements-analysis) * Analysis of design (design-analysis) * Analysis of boundary values (boundary-values) * Analysis of equivalence classes (equivalence-classes) * Fuzzy testing (fuzz-testing) * Error guessing based on knowledge or experience (error-guessing) * Explorative testing (explorative-testing) More information can be found in the :need:`gd_guidl__verification_guide`, :need:`doc_concept__verification_process`, and :need:`gd_guidl__verification_specification` . |
|
Linking Requirements to Tests (C++) |
valid |
For linking C++ tests to requirements **record properties** shall be used. Attributes which are common for all test cases can be specified in the Setup Function (SetUp()), the other attributes which are specific for each test case need to be specified within the test case: A more detailed description of how to link code to requirements looks like is available in the :ref:`verification_template_cpp` |
|
Linking Requirements to Tests (Python) |
valid |
For linking python tests to requirements **metadata** shall be used. For this the 'add_test_properties' decorator has been provided. You need to add it to the test and fill out: * partially_verifies OR fully_verifies * test_type * derivation_technique For allowed values for test_type & derivation_technique please check :need:`gd_req__verification_link_tests` Further more, this decorator will also check if your test has a `docstring` which should act as the description of the test. A more detailed description of how to link code to requirements looks like is available in the :ref:`verification_template_python` |
|
Linking Requirements to Tests (Rust) |
valid |
For linking Rust tests to requirements **#[record_property]** shall be used: A more detailed description of how to link code to requirements looks like is available in the :ref:`verification_template_rust` |
|
Verification Report Archiving |
valid |
The tool automation shall automatically archive the Verification reports for releases. The reports are generated according to :need:`gd_req__verification_reporting`. |
|
Verification Reporting |
valid |
The tool automation shall automatically generate the Verification reports. These may be independent documents (i.e. not integrated into docs-as-code based repositories). The content of the reports is specified in :need:`gd_temp__platform_ver_report` and :need:`gd_temp__mod_ver_report`. The execution results of test cases are marked with a clear pass/fail result. |
|
Static Code Analysis Classification |
valid |
Static code analysis findings shall be classified according to the following categories: - Critical - High - Medium - Low The actual classification shall be either provided by the tool as a suggestion or defined manually by :need:`rl__quality_manager` together with :need:`rl__committer`. Alternative rating like class A, B, C, D or 1, 2, 3, 4 can be used as well, but the principle mapping to the above categories shall be preserved and documented. |
|
Component Architecture Templates |
valid |
For the content see the `module template documentation <https://eclipse-score.github.io/module_template/main/components/component_example/architecture/component_architecture.html>`__. |
|
Feature Architecture Templates |
valid |
For the content see here: :ref:`feature_architecture_template` |
|
Component Request Template |
valid |
for the content see `Component Request Template <https://eclipse-score.github.io/module_template/main/components/component_example/index.html>`__ |
|
Decision Record Template |
valid |
For the content see here: :ref:`decision_record_template`. |
|
Feature Request Template |
valid |
for the content see :need:`doc__feature_name` |
|
Impact Analysis Template |
valid |
||
Component DFA Templates |
valid |
For the content see here: `Component DFA Template <https://eclipse-score.github.io/module_template/main/components/component_example/safety_analysis/dfa.html>`__ |
|
Component FMEA Template |
valid |
For the content see here: `Component FMEA Template <https://eclipse-score.github.io/module_template/main/components/component_example/safety_analysis/fmea.html>`__ |
|
Component Threat Template |
valid |
For the content see here: (tbd) `Component Threat Template <https://eclipse-score.github.io/module_template/main/components/component_example/index.html>`__ Future PR (https://github.com/eclipse-score/process_description/issues/409). |
|
Component Security Analysis Templates |
valid |
For the content see here: (tbd) `Component Security Analysis Template <https://eclipse-score.github.io/module_template/main/components/component_example/index.html>`__ Future PR (https://github.com/eclipse-score/process_description/issues/409). |
|
Component Classification Template |
valid |
For the content see here: `Component Classification Template <https://eclipse-score.github.io/module_template/main/components/component_example/component_classification.html>`__ |
|
Configuration Management Plan Template |
valid |
||
Detailed Design Templates |
valid |
For the content see here: `Detailed Design Template <https://eclipse-score.github.io/module_template/main/components/component_example/detailed_design/index.html>`__ |
|
Documentation Template |
valid |
| .. document:: <Document Name> | :id: doc__<Document Name> | :status: <valid|invalid> | :security: <YES|NO> | :safety: <QM|ASIL_B> | :realizes: wp__<name of wp in process description> |
|
Feature DFA Templates |
valid |
For the content see here: `Feature DFA Template <https://eclipse-score.github.io/module_template/main/features/feature_example/safety_analysis/dfa.html>`__ |
|
Feature FMEA Template |
valid |
For the content see here: `Feature FMEA Template <https://eclipse-score.github.io/module_template/main/features/feature_example/safety_analysis/fmea.html>`__ |
|
Feature Threat Template |
valid |
For the content see here: (tbd)need:`doc__feature_name_threat` Future PR (https://github.com/eclipse-score/process_description/issues/409). |
|
Feature Security Analysis Templates |
valid |
For the content see here: (tbd)need:`doc__feature_name_security_analysis` Future PR (https://github.com/eclipse-score/process_description/issues/409). |
|
Feature Safety Work Products Template |
valid |
For the content see here: `Feature Safety Work Products Template <https://eclipse-score.github.io/module_template/main/features/feature_example/safety_planning/index.html>`__ |
|
Feature Security Work Products Template |
valid |
For the content see here: `Feature Security Work Products Template <https://eclipse-score.github.io/module_template/main/features/feature_example/security_planning/index.html>`__ |
|
Module Verification Report Template |
valid |
This document implements :need:`wp__verification_module_ver_report`. | For the content, see `Module Verification Report Template <https://eclipse-score.github.io/module_template/main/verification_report/module_verification_report.html>`__. |
|
Module Safety Plan Template |
valid |
For the content see here: `Module Safety Plan Template <https://eclipse-score.github.io/module_template/main/module/safety_mgt/module_safety_plan.html>`__ |
|
Module Security Manual Template |
valid |
For the content see here: `Module Security Manual Template <https://eclipse-score.github.io/module_template/main/module/manuals/security_manual.html>`__ |
|
Module Security Plan Template |
valid |
For the content see here: `Module Security Plan Template <https://eclipse-score.github.io/module_template/main/module/security_mgt/module_security_plan.html>`__ |
|
Platform DFA Templates |
valid |
For the content see here: :need:`doc__platform_dfa` |
|
Platform Security Analysis Templates |
valid |
For the content see here: (tbd)need:`doc__platform_security_analysis` Future PR (https://github.com/eclipse-score/process_description/issues/409). |
|
Platform Management Plan Template |
valid |
||
Platform Safety Plan Template |
valid |
For the content see here: :need:`doc__platform_safety_plan` |
|
Platform Security Manual Template |
valid |
For the content see here: :need:`doc__platform_security_manual` |
|
Platform Security Plan Template |
valid |
For the content see here: :need:`doc__platform_security_plan` |
|
Platform Verification Report Template |
valid |
This document implements :need:`wp__verification_platform_ver_report`. | For the content, see :need:`doc__platform_verification_report`. |
|
Problem Template |
valid |
||
Checklist Template |
valid |
.. code-block:: rst .. gd_chklst:: <Title reflecting process area checklist> :id: gd_chklst__<process area or abbreviation>_<...> :status: <draft|valid> :version: 1 :tags: <process area or abbreviation> :complies: <standard requirement:std_req__<...>>, ..., <standard requirement:std_req__<...>> |
|
Concept Description Template |
valid |
.. code-block:: rst .. doc_concept:: <Title reflecting process area concept description> :id: doc_concept__<process area or abbreviation> :status: <draft|valid> :version: 1 :tags: <process area or abbreviation> |
|
Document Template |
valid |
.. code-block:: rst .. document:: <Title reflecting the deployed work product> :id: doc__<work product> :status: <draft|valid> :version: 1 :safety: <QM | ASIL_B> :security: <YES|NO> :realizes: wp__<work product reference>, ..., wp__<work product reference> :tags: <depends on where it is deployed> |
|
Getting Started Template |
valid |
.. code-block:: rst .. doc_getstrt:: <Title reflecting process area getting started> :id: doc_getstrt__<process area or abbreviation> :status: <draft|valid> :version: 1 :tags: <process area or abbreviation> |
|
Guideline Template |
valid |
.. code-block:: rst .. gd_guidl:: <Title reflecting process area guideline> :id: gd_guidl__<process area or abbreviation>_<...> :status: <draft|valid> :version: 1 :tags: <process area or abbreviation> :complies: <standard requirement:std_req__<...>>, ..., <standard requirement:std_req__<...>> |
|
Method Template |
valid |
.. code-block:: rst .. gd_method:: <Title reflecting process area method> :id: gd_meth__<process area or abbreviation>_<...> :status: <draft|valid> :version: 1 :tags: <process area or abbreviation> :complies: <standard requirement:std_req__<...>>, ..., <standard requirement:std_req__<...>> |
|
Process Requirement Template |
valid |
.. code-block:: rst .. gd_req:: <Title reflecting process area requirement> :id: gd_req__<process area or abbreviation>_<...> :status: <draft|valid> :version: 1 :tags: <process area or abbreviation> :satisfies: <defined workflow:wf__<...>>, ..., <defined workflow:wf__<...>> :complies: <standard requirement:std_req__<...>>, ..., <standard requirement:std_req__<...>> |
|
Role Template |
valid |
.. code-block:: rst Single person role .. role:: <Title reflecting role> :id: rl__<role> :status: <draft|valid> :version: 1 :tags: <process area or abbreviation> Team role .. role:: <Title reflecting team role> :id: rl__<process area or abbreviation>_<team role<>> :status: <draft|valid> :version: 1 :tags: <process area or abbreviation> :contains: <role:rl__<...>>, ..., <role:rl__<...>> |
|
Standard Requirement Template |
valid |
.. code-block:: rst .. std_req:: <Title reflecting standard requirement> :id: std_req__<standard>__<...> :status: <draft|valid> :version: 1 :tags: <standard> |
|
Standard Work Product Template |
valid |
.. code-block:: rst .. std_wp:: <Title reflecting standard work product> :id: std_wp__<standard>__<...> :status: <draft|valid> :version: 1 :tags: <standard> |
|
Template Template |
valid |
.. code-block:: rst .. gd_temp:: <Title reflecting process area template> :id: gd_temp__<process area or abbreviation>_<...> :status: <draft|valid> :version: 1 :tags: <process area or abbreviation> :complies: <standard requirement:std_req__<...>>, ..., <standard requirement:std_req__<...>> |
|
Workflow Template |
valid |
.. code-block:: rst .. workflow:: <Title reflecting activity> :id: wf__<process area or abbreviation>_<activity> :status: <draft|valid> :version: 1 :tags: <process area or abbreviation> :responsible: <defined role:rl__<...>> :approved_by: <defined role:rl__<...>>, ..., <defined role:rl__<...>> :supported_by: <defined role:rl__<...>>, ..., <defined role:rl__<...>> :input: <defined workproduct:wp__<...>> :output: <defined workproduct:wp__<...>> :contains: <defined guidances: guideline:gd_guidl__<...>, template:gd_temp__<...>, checklist:gd_chklst__<...>, method:gd_meth__<...> :has: <concept:doc_concept__<...>, getting started:doc_getstrt__<...>> |
|
Work Product Template |
valid |
.. code-block:: rst .. workproduct:: <Title reflecting work product> :id: wp__<process area or abbreviation>_<work product> :status: <draft|valid> :version: 1 :tags: <process area or abbreviation> :complies: <standard work product:std_wp__<...>>, ..., <standard work product:std_wp__<...>>, <standard requirement:std_req__aspice_40__iic-<...>>, ..., <std_req__aspice_40__iic-<...>> |
|
Quality Management Plan Template |
valid |
||
Quality Report Template |
valid |
This document implements :need:`wp__qms_report` and based on the :need:`wp__qms_plan`. It summarizes the results of the quality related activities. It shall be referred in the :need:`wp__platform_sw_release_note` of a platform release. The Quality Report contains: **1. Quality Overview** **1.1. Quality Objectives and Goals** - Quality objectives and goals as defined in the :need:`wp__qms_plan` - Status of achievement of the quality objectives and goals **1.2. Lists of all work products** - List of all work products, compared to the :need:`wp__qms_plan` - Status (valid / invalid) **1.3. Lists of all reports** - List of all reports, compared to the :need:`wp__qms_plan` - Results of the reports **1.4. Test coverage** - Overview of test coverage (overall, requirements, architecture) - Ratio of test coverage to the :need:`wp__verification_plan` **1.5. Issues** - List of all issues, compared to the :need:`wp__qms_plan` - Status of the issues (open / closed) **1.6. Process Improvement** - List of all process improvements, compared to the :need:`wp__process_impr_report` - Status of the process improvements (open / closed) **Note1:** All the above lists are generated automatically |
|
Release Issue Template |
valid |
| Copy the below steps into the release ticket: | | Release <add version number> for <platform/module_name> | ------------------------------------------------------- | | 1. Link this issue to the correct milestone and assign to a project/module lead | 2. Check respective Verification report on the release candidate's baseline | 3. Check bugfixes or justify failed tests | 4. Check the safety package completeness (includes "valid" documents and work products status, supported by the safety manager) | 5. Create/update the release note (pull request to close this issue) | 6. Document project manager's consent by asking review approval of the release note | 7. Create the "release" in version management tool according to :need:`gd_guidl__rel_management` | 8. Merge PR and close this issue to complete the release |
|
Module Release Note Template |
valid |
For the content see here: `Module Release Note Template <https://eclipse-score.github.io/module_template/main/module/release/release_note.html>`__ |
|
Platform Release Note Template |
valid |
For the content see here: :need:`doc__platform_release_note` |
|
AoU Requirement Template |
valid |
See the Assumption of Use requirement snippets in the `module template documentation <https://eclipse-score.github.io/module_template/main/components/component_example/requirements/index.html>`__. |
|
Component Requirements Template |
valid |
See the component requirements template in `module template documentation <https://eclipse-score.github.io/module_template/main/components/component_example/requirements/index.html>`__. |
|
Feature Requirements Template |
valid |
See the feature requirements template in :need:`doc__feature_name_requirements` |
|
Requirement Formulation Template |
valid |
Requirements shall be specified according to the following schema: <The SW Platform|Feature|Component> shall <main verb> <object> <parameter> <temporal/logical conjunction> <Note: (optional, not to be verified)> .. list-table:: Sentence Table :header-rows: 1 * - Addressee of the requirement (subject) - shall - main verb - object of the requirement - parameter of the requirement - temporal/logical conjunction * - The development object (who/what) - shall - do something - for whom or what - which target value/condition - when, under which conditions * - Example 1: The component - shall - detect - if a key-value pair got corrupted - and set its status to INVALID - during every restart of the SW platform. * - Example 2: The software platform - shall - enable - users - to ensure the compatibility of application software - across vehicle variants and vehicle software releases. * - Example 3: The linter-tool - shall - check - correctness of .rst files format - - upon each commit. .. note:: Of the last three columns of the above sentence template table, filling one is mandatory the others are optional. |
|
Stakeholder Requirements Template |
valid |
See the stakeholder requirements template in :need:`doc__platform_name_requirements` |
|
Tool Requirements Template |
valid |
.. code-block:: rst .. tool_req:: <Title> :id: tool_req__<tool>__<Title> :security: <YES|NO> :safety: <QM|ASIL_B> :derived_from: <link to process req id> :status: <valid|invalid> :version: 1 :implemented: <YES|PARTIAL|NO> |
|
Safety Manual Template |
valid |
For the content see here: `Safety Manual Template <https://eclipse-score.github.io/module_template/main/module/manuals/safety_manual.html>`__ |
|
Software Development Plan Template |
valid |
||
Tool Verification Report Template |
valid |
For the content see here: :need:`doc_tool__tool_name_version` |
|
Platform Verification Plan Template |
valid |
This document implements :need:`wp__verification_plan`. It provides a template for a software verification plan and should be adapted to the specific project needs. The document should start with a header following the :need:`gd_temp__documentation`. .. code-block:: rst .. document:: Software Verification Plan :id: doc__verification_plan :status: draft :version: 1 :safety: ASIL_B :tags: platform_management Software Verification Plan ************************** This document provides a template for a software verification plan. It should be adapted to the specific needs of project. Purpose ======= This section should briefly describe the overall goal of the verification plan. It should state the plan's intended audience and the information it aims to provide. This might include clarifying the scope of the verification activities and linking to other relevant documents. Objectives and Scope ==================== Objectives ---------- This section outlines the key objectives of the software verification effort. Examples include correctness, completeness, reliability, performance, maintainability, compliance, and traceability. Each objective should be clearly defined and measurable. Verification Scope and Constraints ---------------------------------- This section details what software components and functionalities are included in the verification process. It should also clearly specify any limitations or exclusions. This section should address external dependencies and integrations. Risks and Mitigation -------------------- This section identifies potential risks associated with the verification activities and outlines strategies to mitigate those risks. This may involve referencing the :need:`wp__platform_mgmt`. Schedules --------- This section defines the timeline for different verification activities. It might include milestones, deadlines, and dependencies between tasks. Approach ======== General Approach ---------------- This section provides a high-level overview of the verification strategy. It should describe the overall methodology (e.g., Continuous Integration), approaches used, and rationale behind the choices made. Software Integration -------------------- This section details how software components are integrated into the system. It should describe the integration process, including procedures for handling new features, bug fixes, and code changes. Levels of Integration and Verification -------------------------------------- This section defines the different levels of integration and verification that will be performed (e.g., unit, component, system). Each level should be clearly defined, with associated criteria for successful completion. Verification Methods -------------------- This section lists the specific verification methods used, such as static analysis, dynamic testing, reviews, and inspections. Each method should be briefly described, including its purpose and applicability at different levels of verification. Reference tables can list methods, identifiers, applicable levels and ASIL relevance. Test Derivation Methods ^^^^^^^^^^^^^^^^^^^^^^^ This section details the techniques used to derive test cases (e.g., boundary value analysis, equivalence partitioning, requirements tracing). It should clarify which techniques are used at each level of testing and for different ASIL levels. Again, a reference table is recommended. Quality Criteria ---------------- This section specifies the quality criteria that must be met for successful verification. These criteria might include code coverage metrics, defect density, or other relevant measures. The criteria should be defined with quantifiable goals for different ASIL levels. The strategy on how to achieve the defined coverage goals is described in the below sub-sections. Structural coverage ^^^^^^^^^^^^^^^^^^^ This section defines how structural coverage is measured and achieved. It should describe the specific coverage metrics used (e.g., statement, branch, path coverage) and the tools and processes used to achieve these metrics. The confirmation or any deviation of the coverage percentage value is documented in this section. Coverage of detailed design ^^^^^^^^^^^^^^^^^^^^^^^^^^^ This section defines how coverage of the detailed design is measured and achieved. Coverage metrics with defined thresholds should be e.g. based on Implementation inspections. Evidence of the inspection is given e.g. by the respective work product and its review. The inspection needs to cover the implementation of the detailed design to be complete. Additionally, other measures can be taken to support the coverage of the detailed design like: - Structural coverage as defined by their specific thresholds - Static analysis and Linting - Structural code coverage (e.g. by statement, branch, path coverage) - Code quality metrics (e.g. by linting and static analysis) - Traceability coverage (e.g. by a 100% requirements coverage by test cases) The confirmation or any deviation of the coverage percentage value is documented in this section. Coverage of architectural design ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This section defines how coverage of the architectural design is measured and achieved. It describes the metrics used to ensure completeness and quality of the architecture and the verification methods applied to achieve the defined coverage goals Examples are: - :need:`wp__sw_arch_verification` - done by walkthrough (QM) or inspection (safety-critical parts) - :need:`wp__sw_component_fmea` and :need:`wp__sw_component_dfa` for safety-critical parts - :need:`wp__feature_fmea` and :need:`wp__feature_dfa` for safety-critical parts Each architectural element has to be reviewed against the availability of the above artifacts. The confirmation or any deviation of the coverage percentage value is documented in this section. Coverage of software requirements specifications ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ For a release the ``valid`` requirements need to have a complete test coverage of linked test cases. Tests which are suitable for the coverage are: - :need:`wp__verification_comp_int_test` - :need:`wp__verification_feat_int_test` - :need:`wp__verification_platform_int_test` - :need:`wp__verification_sw_unit_test` The confirmation or any deviation of the coverage percentage value is documented in this section. Static code analysis ^^^^^^^^^^^^^^^^^^^^ Static code analysis is performed to ensure compliance with coding standards and to identify potential issues early in the development process. Static analysis requires tool support. Rule sets need to be defined and configured in the respective tools. The rule sets should include relevant rulesets (like MISRA‑C++) enabling a programming language to be usable in safety critical development. Additionally, project‑specific rules that address architectural constraints and coding practices need to be defined and enforced. Static analysis tool perform e.g. a semantic analysis of the codebase to detect deeper correctness and potential security/safety issues. Test Development ---------------- This section describes the process for developing and maintaining test cases. It should cover aspects such as test automation, test data management, and version control. Pre-existing test cases ^^^^^^^^^^^^^^^^^^^^^^^ This section describes how pre-existing test cases are handled which are e.g. available from an OSS component. It should be stated how they are reviewed, integrated, extended (e.g. with respective documentation), and adopted to the needs described in the project (e.g. usage of documentation templates and traceability) Test Execution and Result Analysis ---------------------------------- This section describes how tests will be executed and the procedures for analyzing the results. It should outline the tools and processes used for test execution and reporting. Manual test execution ^^^^^^^^^^^^^^^^^^^^^ The automation rate for test case execution is expected to be above 99%. For manual test execution it should be described how to re-execute tests manually and how to report potential issues. Test Selection and Regression Testing ------------------------------------- This section describes the approach to selecting test cases for execution and the strategy for regression testing to ensure that new changes don't introduce regressions. Work Products and Traceability ------------------------------ This section lists all the key deliverables related to the verification process. It should also describe how traceability between requirements, design, code, and test cases is maintained. Environments and Resources ========================== Roles ----- This section defines the roles and responsibilities of individuals involved in the verification process. It can refer and should be based on the definition in the verification process :ref:`verification_roles`. Independence of verification ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ This section describes how independence is achieved in the project. As there are no separated roles for a software developer and test developer with :need:`rl__contributor` and :need:`rl__committer` it is important to achieve independence. This is done by having different people responsible for the test implementation and the actual code which gets tested. Tools ----- This section lists the tools used for verification, including build systems, test frameworks, static analysis tools, and other relevant software. Verification Setups and Variants -------------------------------- This section describes the different test environments and configurations used for verification. Test Execution Environment and Reference Hardware ------------------------------------------------- This section describes the hardware and software environments used for test execution. It should include information about any specific hardware platforms or simulators used. It should also define how the verification environment interacts with the CI system, including access control and maintenance. |
|
Verification Specification Template |
valid |
The sections below are seen as typical ways when writing tests and their specification. Their usage differs based on the selected testing framework and the implementation language of the component(s). |
|
Architecture Community Member |
valid |
The architecture community members are responsible for the features and components of the platform. Feature and Components requests, which add new ones or modifications, are in their responsibility. They are aligned with the Project Leads. |
|
Committer |
valid |
(Eclipse) Open Source Role, person(s) who accept(s) possible contribution(s) as pull request(s) to the main line and maintains the product. Required skills and knowledge of standards * Experience in reviewing architectural designs for correctness, consistency, and completeness * Knowledge of ASPICE SWE.2 base practices * Understanding of traceability requirements between architecture and requirements * Understanding of software design patterns and principles (e.g., SOLID, separation of concerns) * Knowledge of the platform domain (middleware, OS abstraction, communication) * Ability to work with Sphinx-Needs and PlantUML tooling * Understanding of safety/security attributes and their impact on architecture * Knowledge of existing architecture examples in module template * Knowledge of UML notation (component diagrams, sequence diagrams) .. note:: Defines and enforces processes. |
|
Contributor |
valid |
(Eclipse) Open Source Role, person(s) who provide(s) possible contribution(s) as pull request(s) to the main line. Any contributor which contributes code, tests or documentation to the project. .. note:: Follows the processes defined by the :need:`rl__process_community` |
|
Delivery Team |
valid |
The delivery team is responsible for all artifacts within the Delivery Container SEooCs containing the Dependable Elements. Each Delivery Container has only one responsible team. One of the committers in the team acts as the "Project Manager" and is responsible for planning and reporting. Depending on the delivery container artifacts, some of them are assigned as codeowner. |
|
Infrastructure Tooling Community Member |
valid |
The infrastructure and tooling community members are responsible for the infrastructure and tooling setup for development, but also the rest of the tool chain. |
|
Platform Team |
valid |
The platform team is responsible for all artifacts within the platform SEooC. Additionally it is also responsible for the overall process including its support by tooling. Depending on the platform artifacts, some of them are assigned as codeowner. |
|
Process Community Member |
valid |
The process community members are responsible for the definition of the process architecture of the project integrated management system and how they processes interact. The approval and release of the process is done by the safety, quality and security managers and the project leads (for the parts which affect them). |
|
Project Lead |
valid |
The Project Leads decide about strategy, approve feature requests and perform the project management of the <Project>. Required skills * Degree: Master's degree in electrical engineering/computer science/mathematics, or similar degree, or comparable work experience * Solid understanding of project management * Technical know-how of embedded systems * Preferred training: Basic and Management specific safety and security trainings Experience * 3 years of experience in project or line management Responsibility * Decisions about strategical topics * Filling the Project Lead role according to the `Eclipse Foundation Project Handbook <https://www.eclipse.org/projects/handbook>`_ * Review and approval of contributions, e.g. Feature Requests, which add or modify features * Project management of the <Project> development - i.e. filling the project management role as defined by ISO26262 * High-level project control and coordination between multiple software modules * Escalation instance * Planning and Approval the releases of the <Project> * Approves security related artifacts likes security plan, security audit, security reviews, SBOM, security monitoring/verification, security trainings and including status reporting of security activities Authority * Ultimate decisions on escalated topics * Decide on addition/removal of modules repositories or split-off of projects |
|
Quality Manager |
valid |
The quality managers shall be responsible for the planning and coordination of the quality activities, i.e. the quality management. They shall lead and monitor the quality relevant activities of the project. Required skills * Master’s degree in electrical engineering/computer science/mathematics, or similar degree, or comparable work experience * Solid understanding of software quality management * Knowledge in project management * Deep understanding of quality criteria and the correlating methods and procedures to achieve and verify them * Technical know-how of embedded systems * Preferred Training: ASPICE 4.0 (see training path, https://intacs.info/index.php/certification-center) at least Expert or Professional Assessor Knowledge of standards * Deep knowledge of ASPICE PAM 4.0 * Basic knowledge of ISO 26262 * Basic knowledge of ISOSAE 21434 Experience * 2 years of experience in software quality management * Experience in managing projects Responsibility * Creating/maintain Quality Management Plan * Verify/approve Platform release * Execute Platform Process Audit * Execute Feature Contribution Conformance Checks * Execute Work Product Reviews * Consult and execute Quality Trainings * Monitoring/improving of Quality activities Authority * Escalation of planning topics to the :need:`rl__project_lead` * Definition of quality assurance measures * Refusing the approval of work products as defined in the workflows |
|
Release Team |
valid |
The release team is responsible for the release. The release team consists of different stakeholders like module leads, project leads and quality managers. |
|
Safety Engineer |
valid |
The Safety Engineer is responsible for the Safety Analysis (FMEA and DFA) in the project. There might be several analysis on different levels (e.g., Platform DFA, Feature and Component FMEA/DFA). Required skills * Degree: Master's degree in Electrical Engineering/Computer Science/Mathematics, or similar degree, or comparable work experience * Deep understanding of functional Safety Engineering including Safety Analysis (e.g., FMEA, DFA) * Knowledge of Safety Management to ensure collaboration with the Safety Manager * Technical know-how of embedded systems * Preferred training: Automotive Functional Safety Expert (AFSE) or similar Knowledge of standards * ISO 26262 * ISO SAE 21434 Experience * More than five years of experience in Safety Engineering * Experience Safety Analysis methods (e.g., FMEA, DFA) * Experience in automotive software development projects * Experience in creation of workproducts according ISO 26262 Responsibility * Analyse Platform, Feature and Component Architecture by performing FMEA and DFA * Monitor Safety Analyses and DFA * Verify Safety Analyses and DFA * Create the Safety Manual Authority * Escalation of safety topics to the Safety Manager * Creation of Issues in the Issue Tracking System for needed mitigations (e.g. prevention, detection or mitigation) |
|
Safety External Auditor |
valid |
Required skills, Knowledge of standards, Experience * External Auditor comes from organization specialized in safety audits and assessment, thus sufficient skill should be guaranteed by the sending organization. * For performing the formal document reviews also a safety manager from another Eclipse Safety project can play the role of an external auditor, in this case the same skills apply as for the safety manager. Responsibility * Performing and reporting of safety audit Authority * Decision on the passing or failing of an audit |
|
Safety Manager |
valid |
The safety manager is responsible for making sure that ISO26262 is complied to in the project. He/She shall lead and monitor the safety relevant activities of the project. This role is assigned through a transparent, meritocratic election process similar to committer elections. Only existing committers are eligible. Nominations must include evidence of relevant contributions and safety expertise. The election must be public, archived, and follow Eclipse Foundation principles of openness and neutrality. The criteria for nomination and election must be documented and published on the project’s website. Required skills * Degree: Master's degree in electrical engineering/computer science/mathematics, or similar degree, or comparable work experience * Solid understanding of functional safety management * Knowledge in project management * Deep understanding of quality criteria and the correlating methods and procedures to achieve and verify them * Technical know-how of embedded systems * Preferred training: Automotive Functional Safety Expert (AFSE) or similar Knowledge of standards * ISO 26262 Experience * 3 years of experience in the management of safety topics * Experience in managing projects * Experience in managing safety anomalies Responsibility * Creating the Safety Plan * Functional Safety related project status reporting * Creation and Monitoring of completeness of the safety package * Reporting of safety anomalies * Verify, that the preconditions for the "release for production", which are part of the release notes, are fulfilled, and the correctness, completeness and consistency of the release notes * Coaching the project team w.r.t all questions related to functional safety * Planning of safety audit * Approval of OSS component classification and safety analyses (incl. DFA) * Approval of the Safety Package * Creating the safety manuals on platform and module level * Performing/approval of formal document reviews on safety plans, safety package and safety analysis (incl. DFA) * Checking that every person in his team has sufficient safety skills for his role Authority * Escalation of planning topics to the project manager defined in the safety plan * Initiate the publication of a safety anomaly * Recommend the Release of a SW platform or a module * Refusing the approval of work products as defined in the workflows * Refusing the approval of his team's role nomination (i.e. requesting that the role will be withdrawn) |
|
Security Engineer |
valid |
The Security Engineer is responsible for the Security Analysis in the project. There might be several analyses on different levels (Platform, Feature and Component). Required skills * Degree: Master's degree in electrical engineering/computer science/mathematics, or similar degree, or comparable work experience * Deep understanding of cybersecurity engineering including security analysis * Knowledge of Security Management to ensure collaboration with the Security Manager * Technical know-how of embedded systems * Preferred training: Automotive Cybersecurity Expert or similar Knowledge of standards * ISO/SAE 21434 * ISO 26262 Experience * More than five years of experience in security engineering * Experience with security analysis methods * Experience in automotive software development projects * Experience in creation of work products according to ISO/SAE 21434 Responsibility * Analyze Feature and Component Architecture by performing Security Analysis * Monitor Security Analysis * Verify Security Analysis * Create the Security Manual Authority * Escalation of security topics to the Security Manager * Creation of Issues in the Issue Tracking System for needed mitigations (accept, avoid, reduce, share) |
|
Security External Auditor |
valid |
Required skills, Knowledge of security standards (ISO 21434), Experience * External Auditor comes from organization specialized in secrity audits and assessment, thus sufficient skill should be guaranteed by the sending organization. * For performing the formal document reviews also a Security Manager from another Eclipse Safety project can play the role of an external auditor, in this case the same skills apply as for the Security Manager. Responsibility * Performing and reporting of secrity audit Authority * Decision on the passing or failing of an audit |
|
Security Manager |
valid |
The Security Manager is responsible for making sure that ISO SAE 21434 is complied to in the project. The Security Manager shall lead and monitor the security relevant activities of the project. Required skills * Degree: Master's degree in electrical engineering/computer science/mathematics, or similar degree, or comparable work experience * Solid understanding of security management * Knowledge in project management * Deep understanding of quality criteria and the correlating methods and procedures to achieve and verify them * Technical know-how of embedded systems * Preferred training: (Automotive) Cybersecurity Specialist (CySec) or similar Knowledge of standards * ISO SAE 21434 Experience * 3 years of experience in the management of security topics * Experience in managing projects * Experience in managing security weaknesses, vulnerabilities Responsibility * Creates and maintains following security artifacts at platform level: Platform Security Plan, Platform Security Package, Platform Security Manual, Platform SBOM * Approves following security artifacts at module: Module Security Plan, Module Security Package, Module Security Manual, Module SBOM * Verifies, that the preconditions for the "release for production", which are part of the release notes, are fulfilled, and the correctness, completeness and consistency of the release notes * Supports reporting of security related project status * Reports security weaknesses, vulnerabilities * Coaches the project team w.r.t all questions related to security * Plans and approves the security audit (to be discussed, currently not in scope) * Plans and approves the formal security reviews * Approval of security analyses * Checks that every person in his team has sufficient security skills for their role Authority * Escalation of planning topics to the project manager defined in the Security Plan * Initiate the publication of a security weakness, vulnerability * Recommend the Release of a SW platform or a module * Refusing the approval of work products as defined in the workflows * Refusing the approval of his team's role nomination (i.e. requesting that the role will be withdrawn) |
|
Project Security Team |
valid |
(Eclipse) Open Source Role, person(s) who is(are) responsible for coordinating the resolution of Vulnerabilities within the Project. By default, the project Security Team includes all Committers. However, the Project may choose a different arrangement and establish specific criteria for team nominations. |
|
Testing Community Member |
valid |
The testing community members are responsible for the test case development from component to platform level. They shall be included in any requirements reviews. They can also improve independence argumentation when involved in the development of unit testing on safety critical units. In this way the testing community takes a supportive role for unit testing. |
|
GP2.1.1: Identify the objectives and define a strategy for the performance of the process. |
valid |
The scope of the process activities including the management of process performance and the management of work products are determined. Corresponding results to be achieved are determined. Process performance objectives and associated criteria are identified. .. note:: Budget targets and delivery dates to the customer, targets for test coverage and process lead time are examples for process performance objectives. .. note:: Performance objectives are the basis for planning and monitoring. Assumptions and constraints are considered when identifying the performance objectives. Approach and methodology for the process performance is determined. .. note:: A process performance strategy may not necessarily be document-ed specifically for each process. Elements applicable for multiple processes may be documented jointly, e.g, as part of a common project handbook or in a joint test strategy. |
|
GP2.1.2: Plan the performance of the process. |
valid |
The planning for the performance of the process is established according to the defined objectives, criteria, and strategy. Process activities and work packages are defined. Estimates for work packages are identified using appropriate methods. .. note:: Schedule and milestones are defined. |
|
GP2.1.3: Determine resource needs. |
valid |
The required amount of human resources, and experience, knowledge and skill needs for the for process performance are determined based on the planning. The needs for physical and material resources are determined based on the planning. .. note:: Physical and material resources may include equipment, laboratories, materials, tools, licenses etc. Required responsibilities and authorities to perform the process, and to manage the corresponding work products are determined. .. note:: The definition of responsibilities and authorities does not necessarily require formal role descriptions. |
|
GP2.1.4: Identify and make available resources. |
valid |
The individuals performing and managing the process are identified and allocated according to the determined needs. The individuals performing and managing the process are being qualified to execute their responsibilities. .. note:: Qualification of individuals may include training, mentoring, or coaching. The other resources, necessary for performing the process are identified, made available, allocated and used according to the determined needs. |
|
GP2.1.5: Monitor and adjust the performance of the process. |
valid |
Process performance is monitored to identify deviations from the planning. Appropriate actions in case of deviations from the planning are taken. The planning is adjusted as necessary. |
|
GP2.1.6: Manage the interfaces between involved parties. |
valid |
The individuals and groups including required external parties involved in the process performance are determined. Responsibilities are assigned to the relevant individuals or parties. Communication mechanisms between the involved parties are determined. Effective communication between the involved parties is established and maintained. |
|
GP2.2.1: Define the requirements for the work products. |
valid |
The requirements for the content and structure of the work products to be produced are defined. Quality criteria for the work products are identified. Appropriate review and approval criteria for the work products are defined. .. note:: Possible sources of documentation requirements may be e.g., best practices or lessons learned from other projects, standards, organization requirements, customer requirements, etc. .. note:: There may be types of work products for which no review or approval is required, thus then there would be no need to define the corresponding criteria. |
|
GP2.2.2: Define the requirements for storage and control of the work products. |
valid |
Requirements for the storage and control of the work products are defined, including their identification and distribution. .. note:: Possible sources for the identification of requirements for storage and control may be e.g., legal requirements, data policies, best practices from other projects, tool related requirements, etc. .. note:: Examples for work product storage are files in a file system, ticket in a tool, Wiki entry, paper documents etc. .. note:: Where status of a work product is required in base practices, this should be managed via a defined status model. |
|
GP2.2.3: Identify, store and control the work products. |
valid |
The work products to be controlled are identified. The work products are stored and controlled in accordance with the requirements. Change control is established for work products. Versioning and baselining of the work products is performed in accordance with the requirements for storage and control of the work products. The work products including the revision status are made available through appropriate mechanisms. |
|
GP2.2.4: Review and adjust work products. |
valid |
The work products are reviewed against the defined requirements and criteria. Resolution of issues arising from work products reviews is ensured. |
|
GP3.1.1: Establish and maintain the standard process. |
valid |
A suitable standard process is developed including required activities and their interactions. Inputs and outputs of the standard process are defined including the corresponding entry and exit criteria to determine the interactions and sequence with other processes. Process performance roles are identified and assigned to the standard process activities including their type of involvement, responsibilities, and authorities. .. note:: An example for describing the involvement of the process roles in the activities is a RASI/RASIC representation. Suitable guidance, procedures, and templates are provided to support the execution of the process as needed. .. note:: Procedures may also include description of specific methods to be used. Appropriate tailoring guidelines including predefined unambiguous criteria as well as predefined and unambiguous proceedings are defined based on identified deployment needs and context of the standard process. The standard process is maintained according to corresponding feedback from the monitoring of the deployed processes. .. note:: For guidance on how to perform process improvements see the Process Improvement process (PIM.3). |
|
GP3.1.2: Determine the required competencies. |
valid |
Required competencies, skills, and experience for performing the standard process are determined for the identified roles. Appropriate qualification methods to acquire the necessary competencies and skills are determined, maintained, and made available for the identified roles. .. note:: Qualification methods are e.g., trainings, mentoring, self-study. .. note:: Preparation includes e.g., identification or definition of trainings, mentoring concepts, self-learning material. |
|
GP3.1.3: Determine the required resources. |
valid |
Required physical and material resources and process infrastructure needs for performing the standard process are determined. .. note:: This may include e.g., facilities, tools, licenses, networks, services, and samples supporting the establishment of the required work environment. |
|
GP3.1.4: Determine suitable methods to monitor the standard process. |
valid |
Methods and required activities for monitoring the effectiveness and adequacy of the standard process are determined. .. note:: Methods and activities to gather feedback regarding the standard process may be lessons learned, process compliance checks, internal audits, management reviews, change requests, reflection of state-of-the-art such as applicable international standards, etc. Appropriate criteria and information needed to monitor the standard process are defined. .. note:: Information about process performance may be of qualitative or quantitative nature. |
|
GP3.2.1: Deploy a defined process that satisfies the context specific requirements of the use of the standard process. |
valid |
The defined process is appropriately selected and/or tailored from the standard process. Conformance of defined process with standard process requirements and tailoring criteria is verified. The defined process is used as managed process to achieve the process outcomes. .. note:: Changes in the standard process may require updates of the defined process. |
|
GP3.2.2: Ensure required competencies for the defined roles. |
valid |
Human resources are allocated to the defined roles according to the required competencies and skills. Assignment of persons to roles and corresponding responsibilities and authorities for performing the defined process are communicated. Gaps in competencies and skills are identified, and corresponding qualification measures are initiated and monitored. Availability and usage of the project staff are measured and monitored. |
|
GP3.2.3: Ensure required resources to support the performance of the defined process. |
valid |
Required information to perform the defined process is made available, allocated and used. Required physical and material resources, process infrastructure and work environment are made available, allocated and used. Availability and usage of resources are measured and monitored. |
|
GP3.2.4: Monitor the performance of the defined process. |
valid |
Information is collected and analyzed according to the determined process monitoring methods to understand the effectiveness and adequacy of the defined process. Results of the analysis are made available to all effected parties and used to identify where continual improvement of the standard and/or defined process can be made. .. note:: For guidance on how to perform process improvements see the Process Improvement process (PIM.3). |
|
01-03 Software Component |
valid |
Software Component may have the following characteristics: - Software element in the software architecture above the software unit level. - Represented by a design model element or executable code such as libs or scripts and a configuration description, if applicable. |
|
01-50 Integrated Software |
valid |
Integrated Software may have the following characteristics: - Software executable (e.g, simulator with stubbing, debug-able, object code) including: - application parameter files (being a technical implementation solution for configurability-oriented requirements) - all configured software elements |
|
01-52 Configuration item list |
valid |
Configuration item list may have the following characteristics: - Items under configuration control - The name of work products and an associated reference (to file, to tool artifact) - Configuration item attributes and properties |
|
01-53 Trained ML model |
valid |
- The trained ML model is the output of the training process. It consists of the software representing the ML architecture, the set of weights which were optimized during the training, and the final set of hyperparameters. |
|
01-54 Hyperparameter |
valid |
- Hyperparameters are used to control the ML model which has to be trained, e.g.: - Learn rate of training - Scaling of network (number of layers or neurons per layer) - Loss function - Minimum characteristics: - Description - Initial value - Final value upon communicating the results of the ML training |
|
02-01 Commitment / agreement |
valid |
Commitment / agreement may have the following characteristics: - Signed off by all parties involved in the commitment/agreement - Establishes what the commitment is for - Establishes the resources required to fulfill the commitment, such as: - time - people - budget - equipment - facilities |
|
03-06 Process performance information |
valid |
Process performance information may have the following characteristics: - Measurements about defined quantitative or qualitative measurable indicators, that match defined information needs. - Measurement metrics for the calculation of the quantitatively or qualitatively measurable indicators - Data comparing process performance against expected levels - Examples for project performance information: - resource utilization against established target - time schedule against established target - activity or task completion criteria met - defined input and output work products available - process quality against quality expectations and/or criteria - product quality against quality expectations and/or criteria - highlight product performance issues, trends - Examples for service level performance information: - references any goals established - real time metrics related to aspects such as: - capacity - throughput - operational performance - operational service - service outage time - up time - job run time |
|
03-50 Verification Measure Data |
valid |
Verification Measure Data may have the following characteristics: - Verification measure data are data recorded during the execution of a verification measure, e.g.: - for test cases: raw data, logs, traces, tool generated outputs - measurements: values - calculations: values - simulations: protocol - reviews such as optical inspections à findings record - analyses: values |
|
03-51 ML data set |
valid |
- Selection of ML Data for e.g., ML model training (ML Training and Validation Data Set) or test of the trained and deployed ML model (ML Test Data Set). |
|
03-53 ML data |
valid |
- Datum to be used for Machine Learning. The datum has to be attributed by metadata, e.g., unique ID and data characteristics. Examples: - Visual data like a photo or videos (but a video could also be considered as sequence of photos depending on the intended use) - Audio recording - Sensor data - Data created by an algorithm - Data might be processed to create additional data. E.g., processing could add noise, change colors or merge pictures. |
|
04-02 Domain architecture |
valid |
Definition not available yet in PAM4.0 document. |
|
04-04 Software Architecture |
valid |
Software Architecture may have the following characteristics: - A justifying rationale for the chosen architecture. - Individual functional and non-functional behavior of the software component - Settings for application parameters (being a technical implementation solution for configurability-oriented requirements) - Technical characteristics of interfaces for relationships between software components such as: - Synchronization of Processes and tasks - Programming language call - APIs - Specifications of SW libraries - Method definitions in an object- oriented class definitions or UML/SysML interface classes - Callback functions, “hooks” - Dynamics of software components and software states such as: - Logical software operating modes (e.g, start-up, shutdown, normal mode, calibration, diagnosis, etc.) - intercommunication (processes, tasks, threads) and priority - time slices and cycle time - interrupts with their priorities - interactions between software components - Explanatory annotations, e.g, with natural language, for single elements or entire diagrams/models. |
|
04-05 Software detailed design |
valid |
Software detailed design may have the following characteristics: - Elements of a software detailed design: - Control flow definition - Format of input/output data - Algorithms - Defined data structures - Justified global variables - Explanatory annotations, e.g, with natural language, for single elements or entire diagrams/models - Examples for expression languages, depending on the complexity or criticality of a software unit: - natural language or informal languages - semi-formal languages (e.g, UML, SysML) - formal languages (e.g, model-based approach) |
|
04-51 ML architecture |
valid |
- An ML architecture is basically a special part of a software architecture (see 04-04). Additionally - ML architecture describes the overall structure of the ML-based software element - ML architecture specifies ML architectural elements including an ML model and other ML architectural elements, provided to train, deploy, and test the ML model. - describes interfaces within the ML-based software element and to other software elements - ML architecture describes details of the ML model like used layers, activation functions, loss function, and backpropagation - ML architecture contains defined hyperparameter ranges and initial values for training start - resource consumption objectives are defined - ML architecture contains allocated ML requirements |
|
06-04 Training material |
valid |
Training material may have the following characteristics: - Updated and available for new releases - Coverage of system, application, operations, maintenance as appropriate to the application - Course listings and availability |
|
06-50 Integration Sequence Instruction |
valid |
Integration Sequence Instruction may have the following characteristics: - Identification of required physical elements (e.g., hardware, mechanical, wiring elements), and software executables and application parameters (being a technical implementation solution for configurability-oriented requirements) - necessary sequence or ordering of integration - preconditions for starting system integration |
|
06-51 Tailoring guideline |
valid |
Tailoring guideline may have the following characteristics: - Criteria for tailoring, - Proceeding of tailoring describing how to derive and document the defined process from the standard process including responsibility for tailoring and corresponding approval - Requirements for the defined process to ensure integrity and consistency of the defined process - Subset of process assets that is essential for the defined process |
|
06-52 Backup and recovery mechanism information |
valid |
Backup and recovery mechanism information may have the following characteristics: - Description / confirmation of existing backup and recovery mechanisms - References to corresponding procedures or regulations |
|
07-04 Process metric |
valid |
Process metric may have the following characteristics: - Measurements about the process' performance: - ability to produce sufficient work products - adherence to the process - time it takes to perform process - defects related to the process - Measures the impact of process change - Measures the efficiency of the process |
|
07-05 Project metric |
valid |
Project metric may have the following characteristics: - Monitors key processes and critical tasks, provides status information to the project on: - project performance against established plan - resource utilization against established plan - time schedule against established plan - process quality against quality expectations and/or criteria - product quality against quality expectations and/or criteria - highlight product performance problems, trends - Measures the results of project activities: - tasks are performed on schedule - product's development is within the resource commitments allocated - References any goals established |
|
07-06 Quality metric |
valid |
Quality metric may have the following characteristics: - Measures quality attributes of the work products defined: - functionality - reliability - usability - efficiency - maintainability - portability - Measures quality attributes of the "end customer" quality perception .. note:: Refer ISO/IEC 25010 for detailed information on measurement of product quality. |
|
07-08 Service level metric |
valid |
Service level metric may have the following characteristics: - Real time metrics taken while a system is operational, it measures the system's performance or expected service level - Identifies aspects such as: - capacity - throughput - operational performance - operational service - service outage time - up time - job run time |
|
07-51 Measurement result |
valid |
Measurement result may have the following characteristics: Result of gathering qualitative or quantitative data, e.g., - Process metric - Measurements about the process' performance: -- ability to produce sufficient work products -- adherence to the process -- time it takes to perform process -- defects related to the process - Measures the impact of process change - Measures the efficiency of the process - Project metric - Monitors key processes and critical tasks, provides status information to the project on: -- project performance against established plan -- resource utilization against established plan -- time schedule against established plan -- process quality against quality expectations and/or criteria -- product quality against quality expectations and/or criteria -- highlight product performance problems, trends - Measures the results of project activities: - tasks are performed on schedule - product's development is within the resource commitments allocated - References any goals established - Quality metric - Measures quality attributes of the work products defined: -- functionality -- reliability -- usability -- efficiency -- maintainability -- portability - Measures quality attributes of the "end customer" quality perceptionService level metric - Benchmarking data - Customer satisfaction survey |
|
07-61 Quantitative process metric |
valid |
Quantitative process metric may have the following characteristics: - Quantitatively measurable indicators that match information needs derived from business goals - Relation of the quantitatively measurable indicators to process elements in process descriptions or repositories and tools - Process measurement metrics for the calculation of the quantitatively measurable indicators, sbased on data from related process elements, repositories, or tools |
|
07-62 Process analysis technique |
valid |
Process analysis technique may have the following characteristics: - Methods for statistical analysis of process data - Frequency of data collection. |
|
07-63 Process control limits |
valid |
Process control limits may have the following characteristics: - Quantitative control limits for the quantitative process metrics |
|
07-64 Process measurement data |
valid |
Process measurement data may have the following characteristics: - Data collected across process instances - Attributes of data, e.g., timestamps - Relation to process measurement metrics - Storage and retrieval - Effective controls over access |
|
08-53 Scope of work |
valid |
Scope of work may have the following characteristics: - Summary of deliverables for a project - Intended use for the deliverables - Main functions to be realized - Target delivery date and major milestones - Work products and activities that are not in scope of the project as needed - Target markets - Applicable standards and legal requirements - Reuse options - Integration of third party deliveries |
|
08-54 Feasibility analysis |
valid |
Feasibility analysis may have the following characteristics: - Statement about the ability of the project to achieve the project objectives with available resources |
|
08-55 Risk measure |
valid |
Risk measure may have the following characteristics: - Identifies - the risk to be mitigated, avoided, or shared (transferred) - the activities to mitigate, avoid, or share (transfer) the risk - the originator of the measure - criteria for successful implementation - criteria for cancellation of activities - frequency of monitoring - Risk treatment alternatives: - treatment option selected- avoid/reduce/transfer - alternative descriptions - recommended alternative(s) - justifications |
|
08-56 Schedule |
valid |
Schedule may have the following characteristics: - Identifies the activities to be performed - Identifies the expected, and actual, start and completion date for required activities against progress/completion of activities - Identifies dependencies between activities and critical path - Has a mapping to scheduled resources and input data - Identifies resource allocation, resource workload, and critical resources .. note:: A schedule is consistent with the defined work packages, see 14-10 |
|
08-58 Verification Measure Selection Set |
valid |
Verification Measure Selection Set may have the following characteristics: - Include criteria for re-verification in the case of changes (regression). - Identification of verification measures, also for regression testing. |
|
08-60 Verification Measure |
valid |
Verification Measure may have the following characteristics: - A verification measure can be a test case, a measurement, a calculation, a simulation, a review, an optical inspection, or an analysis - The specification of a verification measure includes - pass/fail criteria for verification measures (test completion and ending criteria) - a definition of entry and exit criteria for the verification measures, and abort and re-start criteria - Techniques (e.g., black-box and/or white-box-testing, equivalence classes and boundary values, fault injection for Functional Safety, penetration testing for Cybersecurity, back-to-back testing for model-based development, ICT) - Necessary verification environment & infrastructure - Necessary sequence or ordering |
|
08-61 Resource allocation |
valid |
Resource allocation may have the following characteristics: - Detailed / named resources are allocated to process tasks - Overall resource workload is considered (e.g., allocation of resources to multiple projects) .. note:: Work breakdown structure may be used to refine the detailed resource allocation .. note:: A resource allocation may be integrated in a/ be a part of the schedule, see 08-56 .. note:: Resources to be allocated are e.g., personnel/human resources for project roles and physical and material resources such as (special/limited) equipment, tool, licenses, test hardware, test vehicle, climate chambers etc. |
|
08-62 Communication matrix |
valid |
Communication matrix may have the following characteristics: - List of relevant process internal / external stakeholders - Roles and contact information of the parties involved - Definition of required interfaces between stakeholders - Communication subject - Communication means and frequency - Documentation needs of the communication (e.g., type of communication record) |
|
08-63 Process Monitoring Method |
valid |
Process Monitoring Method may have the following characteristics: - Measures including criteria for monitoring effectiveness, suitability, and adequacy of the standard process - Method for collecting and analyzing the monitoring measures |
|
08-64 ML test approach |
valid |
- The ML test approach describes - ML test scenarios with distribution of data characteristics (e.g., gender, weather conditions, street conditions within the ODD) defined by ML requirements - quantity of each ML test scenario inside the test data set - expected test result per test datum - pass/fail criteria for the ML testing - entry and exit criteria for the ML testing - the required ML testing infrastructure and environment configuration |
|
08-65 ML training and validation approach |
valid |
Process Monitoring Method may have the following characteristics: - ML Training and Validation approach describes at least: - entry and exit criteria of the ML training - approaches for hyperparameter tuning / optimization to be used in the training - approach for data set creation and modification - training environment, including required training hardware (e.g., GPU, or supercomputer to be used) - interface adapter for provision of input data and storage of output data - if required, actions to organize the data set and training environment - The ML training and validation approach may additionally include robustification methods like random dropout |
|
08-66 Measures against deviations in quantitative process analysis |
valid |
Measures against deviations in quantitative process analysis may have the following characteristics: - Definition of counter measures actions to address each assignable cause of special causes of variation, or common causes of variation - Effective implementation of these counter measures |
|
10-00 Process description |
valid |
Process description may have the following characteristics: - Process description of a standard or defined process (e.g., after tailoring), including: - scope and the intended use of the process - process activities including description and dependencies - entry and exit criteria such as input information needed and expected outputs for activities - Roles assigned to process activities (e.g., as RASIC ) or work products - guidelines - templates - specific methods/work instructions |
|
10-50 Role description |
valid |
Role description may have the following characteristics: - Name/identifier (unique within the organization) - Assigned activities (e.g., as RASIC) - Responsibilities and authorities - Required competencies, skills, and experience |
|
10-51 Qualification method description |
valid |
Qualification method description may have the following characteristics: - Training courses - Training materials - Mentoring/coaching concepts - Self-learning material |
|
10-52 Process resource and infrastructure description |
valid |
Process resource and infrastructure description may have the following characteristics: - Required facilities - Required tools and corresponding licenses - Required networks - Required services - Required samples |
|
11-03 Release note |
valid |
Release note may have the following characteristics: - Coverage for key elements (as appropriate to the application): - Description of what is new or changed (including features removed) - System information and requirements - Identification of conversion programs and instructions - Release numbering implementation may include: - the major release number - the feature release number - the defect repair number - the alpha or beta release; and the iteration within the alpha or beta release - Identification of the component list (version identification included): - hardware / software / product elements, libraries, etc. - associated documentation list - New/changed parameter information (e.g., for application parameters or global variables) and/or commands. Note that application parameters are a technical implementation solution for configurability-oriented requirements) - Backup and recovery information - List of open known problems, faults, warning information, etc. - Identification of verification and diagnostic procedures - Technical support information - Copyright and license information - The release note may include an introduction, the environmental requirements, installation procedures, product invocation, new feature identification and a list of defect resolutions, known defects and workarounds |
|
11-04 Product release package |
valid |
Product release package may have the following characteristics: - Includes the hardware/software/product - Includes and associated release elements such as: - system hardware/software/product elements - associated customer documentation - application parameter definitions defined - command language defined - installation instructions - release letter |
|
11-05 Software Unit |
valid |
Software Unit may have the following characteristics: - a representation of a software element at the lowest level in a conceptual model, which is decided not to be further subdivided and that is a part of a software component, or - a representation of a software unit under verification such as commented source code, auto-code, an object file, a library, an executable, or an executable model as input to verification |
|
11-50 Deployed ML model |
valid |
- It is derived from the trained ML model (see 01-53) and is to be integrated into the target system. - It may differ from the trained ML model which often requires powerful hardware and uses interpretative languages. |
|
12-03 Reuse candidate |
valid |
Reuse candidate may have the following characteristics: - Identifies the product to be reused - Identifies the responsible person for the products to be reused - Identifies the reuse goals and objectives - Identifies the list of reuse assets - Identifies the issues/risks of reusing the component including specific requirements (hardware, software, resource and other reuse components) - Identifies the person who will be qualifying the reuse candidate |
|
13-06 Delivery evidence |
valid |
Delivery evidence may have the following characteristics: - Evidence of items shipped/delivered electronically to customer - Identification of: - to whom it was sent - address, where delivered - delivery date - receipt of delivered product |
|
13-07 Problem |
valid |
Problem may have the following characteristics: - Identifies the submitter of the problem - Identifies the group/person(s) responsible for providing problem resolution - Includes a description of the problem - Identifies classification of the problem (criticality, urgency, relevance etc.) - Identifies the status of the problem - States such as “open”, “in review”, “in implementation”, “closed”, “rejected”, “cancelled”, … - Transitions between states with conditions and authorities - Identifies the expected closure date |
|
13-08 Baseline |
valid |
Baseline may have the following characteristics: - Identifies a state of one or a set of work products and artifacts which are consistent and complete - Basis for next process steps and/or delivery - Is unique and may not be changed .. note:: This should be established before a release to identify consistent and complete delivery |
|
13-09 Meeting support evidence |
valid |
Meeting support evidence may have the following characteristics: - Agenda and minutes that are records that define: - purpose of meeting - attendees - date, place held - reference to previous minutes - what was accomplished - identifies issues raised - any open issues - next meeting if any |
|
13-13 Product release approval |
valid |
Product release approval support evidence may have the following characteristics: - Content information of what is to be shipped or delivered - Identification of: - for whom it is intended - the address where to deliver - the date released - Evidence of supplier approval |
|
13-14 Progress status |
valid |
Progress status may have the following characteristics: Status of a plan(s) (actual against planned) such as: - status of actual activities/work packages against planned activities/work package - status of actual results against established objectives/goals - status of actual resources allocation against planned resources - status of actual cost against budget estimates - status of actual time against planned schedule - status of actual quality against planned quality Record of any deviations from planned activities and reason why |
|
13-16 Change request |
valid |
Change request may have the following characteristics: - Identifies purpose of change - Identifies requester contact information - Impacted system(s) - Impact to operations of existing system(s) defined - Impact to associated documentation defined - Criticality of the request, due date - Information supporting the tracking of change requests to closure - progress status attribute (e.g., open, allocated, implemented, closed) - time stamp of status change - person who changed a status - rationale for changing a status |
|
13-18 Quality conformance evidence |
valid |
Quality conformance evidence may have the following characteristics: - Identifies what tasks/activities/process produce the information - Identifies when the data was collected - Identifies source of any associated data - Identifies the associated quality criteria - Identifies any associated measurements using the information |
|
13-19 Review evidence |
valid |
Review evidence may have the following characteristics: - Provides the context information about the review: - what was reviewed - lists reviewers who attended and their area of responsibility - status of the review - Provides information about the scope of the review: - checklists - review criteria - requirements - compliance to standards - Effort information about: - preparation time spent for the review - time spent in the review - Review findings: - non-conformances - improvement suggestions |
|
13-25 Verification results |
valid |
Verification results may have the following characteristics: - Verification data and logs - Verification measure passed - Verification measure not passed - Verification measure not executed, and a rationale - Information about the verification execution (date, “object-under-verification”, etc.) - Abstraction or summary of verification results |
|
13-50 ML test results |
valid |
- Test data and logs - Test data with correct results - Test data with incorrect results - Test data not executed, and a rationale - Information about the test execution (date, participants, model version etc.) - Abstraction or summary of ML test results |
|
13-51 Consistency Evidence |
valid |
Consistency Evidence may have the following characteristics: - Demonstrates bidirectional traceability between artifacts or information in artifacts, throughout all phases of the life cycle, by e.g., - tool links - hyperlinks - editorial references - naming conventions - Evidence that the content of the referenced or mapped information coheres semantically along the traceability chain, e.g., by - performing pair working or group work - performing by peers, e.g., spot checks - maintaining revision histories in documents - providing change commenting (via e.g., meta-information) of database or repository entries .. note:: This evidence can be accompanied by e.g., Definition of Done (DoD) approaches. |
|
13-52 Communication Evidence |
valid |
Communication Evidence may have the following characteristics: - All forms of interpersonal communication such as - e-mails, also automatically generated ones - tool-supported workflows - meeting, verbally or via meeting minutes (e.g., daily standups) - podcast - blog - videos - forum - live chat - wikis - photo protocol |
|
13-53 Qualification evidence |
valid |
Definition not available yet in PAM4.0 document. |
|
13-55 Process resource and infrastructure documentation |
valid |
Process resource and infrastructure documentation may have the following characteristics: - Information on availability, allocation, and usage of - Facilities - Tools and corresponding licenses - Networks - Services - Samples - for non-standard and critical resources and infrastructure. |
|
14-01 Change history |
valid |
Change history may have the following characteristics: - Historical records of all changes made to an object (document, file, software component, etc.): - description of change - version information about changed object - date of change - change requester information - change control record information |
|
14-02 Corrective action |
valid |
Corrective action may have the following characteristics: - Identifies the initial problem - Identifies the ownership for completion of defined action - Defines a solution (series of actions to fix problem) - Identifies the open date and target closure date - Contains a status indicator - Indicates follow up audit actions |
|
14-10 Work package |
valid |
Work package may have the following characteristics: - Defines activities to be performed - Documents ownership for activities e.g., by domains - Documents critical dependencies to other work packages - Documents input and output work products - Documents the critical dependencies between defined work products - Information needed to perform these activities - Estimates of effort, duration .. note:: The work package descriptions may be integrated into the/be a part of a schedule, see 08-56 |
|
14-50 Stakeholder groups list |
valid |
Stakeholder groups list may have the following characteristics: Identifies: - involved parties - weight/importance of each stakeholder group - representative(s) for each stakeholder group - information needs of each stakeholder group |
|
14-53 Role Assignment |
valid |
Role Assignment may have the following characteristics: - Assignment of person(s) to roles - required competencies vs existing competencies - required skills vs existing skills - required experience and trainings based on identified competencies / skills gap |
|
14-55 Software Bill of materials |
valid |
Software Bill of materials may have the following characteristics: - Uniquely identifies type, supplier, and amount of the complete set of all software parts of the software |
|
15-06 Project status |
valid |
Project status may have the following characteristics: - Status of in regards to progress and consistency of schedule, work item content, tasks, resources (human resources, infrastructure, hardware/materials, budget), skills and competence of human resources - planned progress and expenditure against dates/deadlines and actual expenditure - reasons for variance from planned progress - threats to continued progress - issues which may affect the ability of the project to achieve its goals - contingency actions |
|
15-07 Reuse analysis evidence |
valid |
Reuse analysis evidence may have the following characteristics: - Identification of reuse opportunities - Identification of constraints for reuse - Identification of regression test cases - Identification of reuse infrastructure - Identification of known defects |
|
15-09 Risk status |
valid |
Risk status may have the following characteristics: - Identifies the status, or the change, of an identified risk: - risk statement - risk source - risk impact and risk probability - categories and risk thresholds, e.g., for prioritization or setting a status - risk treatment activities in progress |
|
15-12 Problem status |
valid |
Problem status may have the following characteristics: - Indicates progress of problem resolution - Status of problem e.g., - by problem categories/classification - by problem resolution stage |
|
15-13 Assessment/audit report |
valid |
Assessment/audit report may have the following characteristics: - States the purpose of assessment - Method used for assessment - Requirements used for the assessment - Assumptions and limitations - Identifies the context and scope information required: -- date of assessment -- organizational unit assessed -- sponsor information -- assessment team -- attendees -- scope/coverage -- assesses and information -- assessment tool used - Records the result: -- Data -- identifies the gaps, potentials, weaknesses or non-conformances that require corrective actions |
|
15-16 Improvement opportunity |
valid |
Improvement opportunity may have the following characteristics: - Identifies what the problem is - Identifies what the cause of a problem is - Suggest what could be done to fix the problem - Identifies the value (expected benefit) in performing the improvement - Identifies the penalty for not making the improvement |
|
15-51 Analysis Results |
valid |
Analysis Results may have the following characteristics: - Identification of the object under analysis - The analysis criteria used, e.g.: - selection criteria or prioritization scheme used - decision criteria - quality criteria - The analysis results, e.g.: - what was decided/selected - reason for the selection - assumptions made - potential negative impact - Aspects of the analysis may include - correctness - understandability - verifiability - feasibility - validity |
|
15-52 Verification Results |
valid |
Verification Results may have the following characteristics: - Verification data and logs - Verification measure passed - Verification measure not passed - Verification measure not executed - Information about the test execution (date, tester name etc.) - Abstraction or summary of verification results |
|
15-54 Tailoring documentation |
valid |
Tailoring documentation results may have the following characteristics: - Applied criteria for tailoring, - Evidence that the defined process is tailored from the standard process according to the defined criteria |
|
15-55 Problem analysis evidence |
valid |
Problem analysis evidence may have the following characteristics: - Author and involved parties - Date of the analysis - Context and root cause of the problem - Analysis result may include - Impact - Potential negative impact - Affected parties - Potential solution (if known) |
|
15-56 Configuration status |
valid |
Configuration status may have the following characteristics: - Summary of configuration management records including relevant status - Analysis of the configuration management overall state - Identification of baselines made |
|
15-57 Quantitative process analysis results |
valid |
Quantitative process analysis results may have the following characteristics: - Deviations, and distributions, of the quantitative performance of individual process instances performance from the established quantitative control limits (special causes of variations) |
|
15-58 Common cause of variation analysis results |
valid |
Common cause of variation analysis results may have the following characteristics: - Identification of common causes - deviations of the quantitative performance of all process instances from the established quantitative control limits - distributions of the quantitative performance of all process instances within established quantitative control limits |
|
16-00 Repository |
valid |
Repository may have the following characteristics: Charcteristics according to Wikipedia, as the PAM 4.0 is lacking it in the IIC. A software repository, or repo for short, is a storage location for software packages. Often a table of contents is also stored, along with metadata. A software repository is typically managed by source or version control, or repository managers. Package managers allow automatically installing and updating repositories, sometimes called "packages". |
|
16-03 Configuration management system |
valid |
Configuration management system may have the following characteristics: - Supports the configuration management for the scope of the configuration item list contents - Correct configuration of products - Can recreate any release or test configuration - Ability to report configuration status - Has to cover all relevant tools |
|
16-06 Process repository |
valid |
Process repository may have the following characteristics: - Contains process descriptions - Supports multiple presentations of process assets |
|
16-50 Organizational structure |
valid |
Organizational structure may have the following characteristics: - Disciplinary reporting line - Organizational units and sub-units, if applicable |
|
16-52 ML data management system |
valid |
- The ML data management system is part of the configuration management system (see 16-03) and - Supports data management activities like data collection, description, ingestion, exploration, profiling, labeling/annotation, selection, structuring and cleansing - Provides the data for different purposes, e.g., training, testing - Supports the relevant sources of ML data |
|
17-00 Requirement |
valid |
Requirements may have the following characteristics: - An expectation of functions and capabilities (e.g., non-functional requirements), or one of its interfaces - from a black-box perspective - that is verifiable, does not imply a design or implementation decision, is unambiguous, and does not introduce contradictions to other requirements. - A requirements statement that implies, or represents, a design or implementation decision is called “Design Constraint”. - Examples for requirements aspects at the system level are thermal characteristics such as - heat dissipation - dimensions - weight - materials - Examples of aspects related to requirements about system interfaces are - connectors - cables - housing - Examples for requirements at the hardware level are - lifetime and mission profile, lifetime robustness - maximum price - storage and transportation requirements - functional behavior of analog or digital circuits and logic - quiescent current, voltage impulse responsiveness to crank, startstop, drop-out, load dump - temperature, maximum hardware heat dissipation - power consumption depending on the operating state such as sleep-mode, start-up, reset conditions - frequencies, modulation, signal delays, filters, control loops - power-up and power-down sequences, accuracy and precision of signal acquisition or signal processing time - computing resources such as memory space and CPU clock tolerances - maximum abrasive wear and shearing forces for e.g., pins or soldering joints - requirements resulting from lessons learned - safety related requirements derived from the technical safety concept |
|
17-05 Requirements for work products |
valid |
Requirements for work products may have the following characteristics: - Requirements for content and structure, storage and control - Identifies documentation specific meta data, such as id, date, author information, ownership, access rights, review and approval status with, where applicable, status model and workflow, or others - Identifies requirements on documentation structure, e.g., table of content or figures or other formal aspects - May be provided by documentation templates - May be based on tool specific templates - Defines the storage location such as data repository, tool, versioning system - Requirements for versioning - Requirements for baselining - Distribution of the documents - Maintenance and disposal of the documents - May be specific for certain types of documents |
|
17-54 Requirement Attribute |
valid |
Requirement Attributes may have the following characteristics: - Meta-attributes that support structuring and definition of release scopes of requirements. - Can be realized by means of tools. .. note:: usage of requirements attributes may further support analysis of requirements. |
|
17-55 Resource needs |
valid |
Resource needs may have the following characteristics: - Identification of required resources for process performance - Staff including competencies, skills and authorities needs - Material, equipment, and infrastructure - Time and budget .. note:: Needs are derived from Work Breakdown structure and schedule |
|
18-00 Standard |
valid |
Standard may have the following characteristics: - Identification of to whom/what they apply - Expectations for conformance are identified - Conformance to requirements can be demonstrated - Provisions for tailoring or exception to the requirements are included |
|
18-06 Product release criteria |
valid |
Product release criteria may have the following characteristics: - Defines expectations for product release: - release type and status - required elements of the release - product completeness including documentation - adequacy and coverage of testing - limit for open defects - change control status |
|
18-07 Quality criteria |
valid |
Quality criteria may have the following characteristics: - Defines the expectations for work products and process performance - Including thresholds/tolerance levels, required measurements, required checkpoints - Defines what is an adequate work product (required elements, completeness expected, accuracy, etc.) - Defines what constitutes the completeness of the defined tasks - Defines what constitutes the performance of the defined tasks - Establishes expected performance attributes |
|
18-52 Escalation path |
valid |
Escalation path may have the following characteristics: - Defined mechanisms to report and confirm escalation relevant issues - Identifies stakeholders to be included in the escalation path - Identifies levels of escalation |
|
18-53 Configuration item selection criteria |
valid |
Configuration item selection criteria may have the following characteristics: - Identify types of work products to be subject to configuration control |
|
18-57 Change analysis criteria |
valid |
Change analysis criteria may have the following characteristics: - Defines analysis criteria, such as - resource requirements - scheduling issues - risks - benefits |
|
18-58 Process performance objectives |
valid |
Process performance objectives may have the following characteristics: - Objectives for the process of creating the process outcomes and capability level 2 achievements, and corresponding evaluation criteria - Assumptions and constraints, if applicable - Used as the basis for deriving a detailed planning - Examples: - Effort, costs, or budget targets (e.g., min/max limits) - Process-specific deadlines in line with milestones, or frequency of activities (o e.g., dates for deliveries to the customer, quality gates) - Metrics (e.g., max. number of open change requests per release, max. ratio of configuration items in status “in work” at certain milestones before next delivery / release date) |
|
18-59 Review and approval criteria for work products |
valid |
Process performance objectives may have the following characteristics: - Specifies for each type of work products review and approval needs - If and when a review is required - Who shall review it - Who shall approve it - Review method(s) to be used - Criteria for approval |
|
18-70 Business goals |
valid |
Business goals may have the following characteristics: - Explanation of the business goals - Requirements for the business needs - Associations to other goals - Reasons for the existence of the goals and needs, level of degree of the need and effect on the business not having that need - Conditions, constraints, assumptions - Timeframe for achievement - Authorization at the highest level |
|
18-80 Improvement opportunity |
valid |
Improvement opportunity may have the following characteristics: - Cause of the improvement need, e.g., - from qualitative or quantitative process performance analysis, evaluations, and monitoring - industry best practice review, state-of-the-art observations, market studies etc. - Improvement objectives derived from organizational business goals and improvement needs - Organizational scope - Process scope - Activities to be performed to keep all those affected by the improvement informed - Priorities |
|
18-81 Improvement evaluation results |
valid |
Improvement evaluation results may have the following characteristics: - Operational impacts of identified changes on the product(s) and processes - Expected benefit - Conditions, constraints, assumptions |
|
19-01 Process performance strategy |
valid |
Process performance strategy may have the following characteristics: - The operational approach to achieve the process outcomes, consistent with the Process Performance Objectives (18-58), e.g.: - proceedings, including the monitoring of the performance of the process - methodology - scope(s) of the strategy within the process, e.g.: - development sites - application domain-specific differences (e.g., software drivers versus. powertrain software) - disciplines (e.g., different configuration management approaches for software and hardware, or combined approaches) - options due to socio-cultural differences |
|
19-50 ML data quality approach |
valid |
- The ML data quality approach - Defines Quality criteria (see 18-07) e.g., the relevant data sources, reliability and consistency of labelling, completeness against ML data requirements - Describes analysis activities of the data - Describes activities to ensure the quality of the data to avoid issues e.g., data bias, bad labeling |
|
MAN.3.BP1: Define the scope of work |
valid |
Identify the project's goals, motivation and boundaries. |
|
MAN.3.BP10: Review and report progress of the project |
valid |
Regularly review and report the status of the project and the fulfillment of work packages against estimated effort and duration to all affected parties. Prevent recurrence of identified problems. .. note:: Project reviews may be executed at regular intervals by the management. Project reviews may contribute to identify best practices and lessons learned. .. note:: Refer to SUP.9 for resolution of problems |
|
MAN.3.BP2: Define project life cycle |
valid |
Define the life cycle for the project, which is appropriate to the scope, context, and complexity of the project. Define a release scope for relevant milestones. .. note:: This may include the alignment of the project life cycle with the customer's development process. |
|
MAN.3.BP3: Evaluate feasibility of the project |
valid |
Evaluate the feasibility of achieving the goals of the project with respect to time, project estimates, and available resources. .. note:: The evaluation of feasibility may consider technical constraints of the project. |
|
MAN.3.BP4: Define and monitor work packages |
valid |
Define and monitor work packages and their dependencies according to defined project life cycle and estimations. .. note:: The structure and the size of the work packages support an adequate progress monitoring. .. note:: Work packages may be organized in a work breakdown structure. |
|
MAN.3.BP5: Define and monitor project estimates and resources |
valid |
Define and monitor project estimates of effort and resources based on project's goals, project risks, motivation and boundaries. .. note:: Examples of necessary resources are budget, people, product samples, or infrastructure .. note:: Project risks (using MAN.5) may be considered. .. note:: Estimations and resources may include engineering, management and supporting processes. |
|
MAN.3.BP6: Define and monitor required skills, knowledge, and experience |
valid |
Identify and monitor the required skills, knowledge, and experience for the project in line with the estimates and work packages. .. note:: Training, mentoring or coaching of individuals may be applied to resolve deviations from required skills and knowledge. |
|
MAN.3.BP7: Define and monitor project interfaces and agreed commitments |
valid |
Identify and agree interfaces of the project with affected stakeholders and monitor agreed commitments. Define an escalation mechanism for commitments that are not fulfilled. .. note:: Affected stakeholders may include other projects, organizational units, sub-contractors, and service providers. |
|
MAN.3.BP8: Define and monitor project schedule |
valid |
Allocate resources to work packages and schedule each activity of the project. Monitor the performance of activities against schedule. |
|
MAN.3.BP9: Ensure consistency |
valid |
Regularly adjust estimates, resources, skills, work packages and their dependencies, schedules, plans, interfaces, and commitments for the project to ensure consistency with the scope of work. .. note:: This may include the consideration of critical dependencies, that are an input for risk management. |
|
MAN.5.BP1: Identify sources of risks |
valid |
Identify and regularly update the sources of risks with affected parties. .. note:: Risks may include technical, economical, and schedule risks. .. note:: Risks may include the suppliers’ deliverables and services. .. note:: The risk sources may vary across the entire project life cycle. |
|
MAN.5.BP2: Identify potential undesirable events |
valid |
Identify potential undesirable events within the scope of the risk management for the project. |
|
MAN.5.BP3: Determine risks |
valid |
Determine the probability and severity of the undesirable events to support priorities for the mitigation of the risks. .. note:: Different methods may be used to analyze technical risks of a system, for example, functional analysis, simulation, FMEA, FTA etc. |
|
MAN.5.BP4: Define risk treatment options |
valid |
For each risk select a treatment option to accept, mitigate, avoid, or share (transfer) the risk. |
|
MAN.5.BP5: Define and perform risk treatment activities |
valid |
Define and perform risk activities for risk treatment options. |
|
MAN.5.BP6: Monitor risks |
valid |
Regularly re-evaluate the risk related to the identified potential undesirable events to determine changes in the status of a risk and to evaluate the progress of the risk treatment activities. .. note:: Risks of high priority may need to be communicated to and monitored by higher levels of management. |
|
MAN.5.BP7: Take corrective action |
valid |
When risk treatment activities are not effective, take appropriate corrective action. .. note:: Corrective actions may involve reevaluation of risks, developing and implementing new mitigation concepts or adjusting the existing concepts. |
|
MLE.1.BP1: Specify ML requirements |
valid |
Use the software requirements and the software architecture to identify and specify functional and non-functional ML requirements, as well as ML data requirements specifying data characteristics (e.g., gender, weather conditions, street conditions within the ODD) and their expected distributions. .. note:: Non-functional requirements may include relevant characteristics of the ODD and KPIs as robustness, performance, and level of trustworthiness. .. note:: The ML data requirements are input for SUP.11 Machine Learning Data Management but also for other MLE processes. .. note:: In case of ML development only, stakeholder requirements represent the software requirements. |
|
MLE.1.BP2: Structure ML requirements |
valid |
Structure and prioritize the ML requirements. .. note:: Examples for structuring criteria can be grouping (e.g., by functionality) or variants identification. .. note:: Prioritization can be done according to project or stakeholder needs via e.g., definition of release scopes. Refer to SPL.2.BP1. |
|
MLE.1.BP3: Analyze ML requirements |
valid |
Analyze the specified ML requirements including their interdependencies to ensure correctness, technical feasibility, and ability for machine learning model testing, and to support project management regarding project estimates. .. note:: See MAN.3.BP3 for project feasibility and MAN.3.BP5 for project estimates. |
|
MLE.1.BP4: Analyze the impact on the ML operating environment |
valid |
Analyze the impact that the ML requirements will have on interfaces of software components and the ML operating environment. .. note:: The ML operating environment is defined as the infrastructure and information which both the trained ML model and the deployed ML model need for execution. |
|
MLE.1.BP5: Ensure consistency and establish bidirectional traceability |
valid |
Ensure consistency and establish bidirectional traceability between ML requirements and software requirements and between ML requirements and the software architecture. .. note:: Bidirectional traceability supports consistency, facilitates impact analyses of change requests, and verification coverage demonstration. Traceability alone, e.g., the existence of links, does not necessarily mean that the information is consistent with each other. .. note:: Redundant traceability is not intended, but at least one out of the given traceability paths. |
|
MLE.1.BP6: Communicate agreed ML requirements and impact on the operating environment |
valid |
Communicate the agreed ML requirements, and the results of the impact analysis on the ML operating environment to all affected parties. |
|
MLE.2.BP1: Develop ML architecture |
valid |
Develop and document the ML architecture that specifies ML architectural elements including details of the ML model, pre- and postprocessing, and hyperparameters which are required to create, train, test, and deploy the ML model. .. note:: Necessary details of the ML model may include layers, activation functions, and backpropagation. The level of detail of the ML model may not need to cover aspects like single neurons. .. note:: The details of the ML model may differ between the ML model used during training and the deployed ML model. |
|
MLE.2.BP2: Determine hyperparameter ranges and initial values |
valid |
Determine and document the hyperparameter ranges and the initial values as a basis for the training. |
|
MLE.2.BP3: Analyze ML architectural elements |
valid |
Define criteria for analysis of the ML architectural elements. Analyze ML architectural elements according to the defined criteria. .. note:: Trustworthiness and explainability might be criteria for the analysis of the ML architectural elements. |
|
MLE.2.BP4: Define interfaces of the ML architectural elements |
valid |
Determine and document the internal and external interfaces of each ML architectural element including its interfaces to related software components. |
|
MLE.2.BP5: Define resource consumption objectives for the ML architectural elements |
valid |
Determine and document the resource consumption objectives for all relevant ML architectural elements during training and deployment. |
|
MLE.2.BP6: Ensure consistency and establish bidirectional traceability |
valid |
Ensure consistency and establish bidirectional traceability between the ML architectural elements and the ML requirements. .. note:: Bidirectional traceability supports consistency, and facilitates impact analyses of change requests, and verification coverage demonstration. Traceability alone, e.g., the existence of links, does not necessarily mean that the information is consistent with each other. .. note:: The bidirectional traceability should be established on a reasonable level of abstraction to the ML architectural elements. |
|
MLE.2.BP7: Communicate agreed ML architecture |
valid |
Inform all affected parties about the agreed ML architecture including the details of the ML model and the initial hyperparameter values. |
|
MLE.3.BP1: Specify ML training and validation approach |
valid |
Specify an approach which supports the training and validation of the ML model to meet the defined ML requirements. The ML training and validation approach includes * entry and exit criteria of the training and validation, * approaches for hyperparameter tuning / optimization, * approach for data set creation and modification, and * training and validation environment .. note:: The ML training and validation approach may include random dropout and other robustification methods. .. note:: ML validation is the optimization of the hyperparameters during Machine Learning Training (MLE.3). The term "validation" has a different meaning than VAL.1. .. note:: The training environment should reflect the environment of the deployed model. |
|
MLE.3.BP2: Create ML training and validation data set |
valid |
Select data from the ML data collection provided by SUP.11 and assign them to the data set for training and validation of the ML model according to the specified ML training and validation approach. .. note:: The ML training and validation data set may include corner cases, unexpected cases, and normal cases depending on the ML requirements. .. note:: A separated data set for training and validation might not be required in some cases (e.g., k-fold cross validation, no optimization of hyperparameters). |
|
MLE.3.BP3: Create and optimize ML model |
valid |
Create the ML model according to the ML architecture and train it, using the identified ML training and validation data set according to the ML training and validation approach to meet the defined ML requirements, and training and validation exit criteria. |
|
MLE.3.BP4: Ensure consistency and establish bidirectional traceability |
valid |
Ensure consistency and establish bidirectional traceability between the ML training and validation data set and the ML data requirements. .. note:: Bidirectional traceability supports consistency and facilitates impact analyses of change requests. Traceability alone, e.g., the existence of links, does not necessarily mean that the information is consistent with each other. |
|
MLE.3.BP5: Summarize and communicate agreed trained ML model |
valid |
Summarize the results of the optimization and inform all affected parties about the agreed trained ML model. |
|
MLE.4.BP1: Specify an ML test approach |
valid |
Specify an ML test approach suitable to provide evidence for compliance of the trained ML model and the deployed ML model with the ML requirements. The ML test approach includes * ML test scenarios with distribution of data characteristics (e.g., gender, weather conditions, street conditions within the ODD) defined by ML requirements, * distribution and frequency of each ML test scenario inside the ML test data set, * expected test result per test datum, * entry and exit criteria of the testing, * approach for data set creation and modification, and * the required testing infrastructure and environment setup. .. note:: Expected test result per test datum might require labeling of test data to support comparison of output of the ML model with the expected output. .. note:: Test datum is the smallest amount of data which is processed by the ML model into only one output. E.g., one image in photo processing or an audio sequence in voice recognition. .. note:: Data characteristic is one property of the data that may have different expressions in the ODD. E.g., weather condition may contain expressions like sunny, foggy or rainy. .. note:: An ML test scenario is a combination of expressions of all defined data characteristics e.g., weather conditions = sunny, street conditions = gravel road. |
|
MLE.4.BP2: Create ML test data set |
valid |
Create the ML test data set needed for testing of the trained ML model and testing of the deployed ML model from the ML data collection provided by SUP.11 considering the ML test approach. The ML test data set shall not be used for training. .. note:: The ML test data set for the trained ML model might differ from the test data set of the deployed ML model. .. note:: Additional data sets might be used for special purposes like assurance of safety, fairness, robustness. |
|
MLE.4.BP3: Test trained ML model |
valid |
Test the trained ML model according to the ML test approach using the created ML test data set. Record and evaluate the ML test results. .. note:: Evaluation of test logs might include pattern analysis of failed test data to support e.g., trustworthiness. |
|
MLE.4.BP4: Derive deployed ML model |
valid |
Derive the deployed ML model from the trained ML model according to the ML architecture. The deployed ML model shall be used for testing and delivery to software integration. .. note:: The deployed ML model will be integrated into the target system and may differ from the trained ML model which often requires powerful hardware and uses interpretative languages. |
|
MLE.4.BP5: Test deployed ML model |
valid |
Test the deployed ML model according to the ML test approach using the created ML test data set. Record and evaluate the ML test results. |
|
MLE.4.BP6: Ensure consistency and establish bidirectional traceability |
valid |
Ensure consistency and establish bidirectional traceability between the ML test approach and the ML requirements, and the ML test data set and the ML data requirements; and bidirectional traceability is established between the ML test approach and ML test results. .. note:: Bidirectional traceability supports consistency, and facilitates impact analyses of change requests, and verification coverage demonstration. Traceability alone, e.g., the existence of links, does not necessarily mean that the information is consistent with each other. |
|
MLE.4.BP7: Summarize and communicate results |
valid |
Summarize the ML test results of the ML model. Inform all affected parties about the agreed results and the deployed ML model. |
|
PIM.3.BP1: Establish commitment |
valid |
Establish commitment to support the process improvement staff, to provide resources and further enablers to sustain improvement actions. .. note:: The process improvement process is a generic process, which can be used at all levels (e.g, organizational level, process level, project level, etc.) and which can be used to improve all processes. .. note:: Commitment at all levels of management may support process improvement. .. note:: Enablers for improvement measures may include trainings, methods, infrastructure, etc. |
|
PIM.3.BP2: Identify improvement measures |
valid |
||
PIM.3.BP3: Establish process improvement goals |
valid |
Analyze the current status of the existing processes and establish improvement goals. .. note:: The current status of processes may be determined by process assessment. |
|
PIM.3.BP4: Prioritize improvements |
valid |
Prioritize the improvement goals and improvement measures. |
|
PIM.3.BP5: Define process improvement measures |
valid |
Process improvement measures are defined. .. note:: Improvements may be documented in incremental steps. |
|
PIM.3.BP6: Implement process improvement measures |
valid |
Implement and apply the improvements to the processes. Update the Process documentation and train people as needed. .. note:: Process application can be supported by establishing policies, adequate process infrastructure, process training, process coaching and tailoring processes to local needs. .. note:: Improvements may be piloted before roll out within the organization. |
|
PIM.3.BP7: Confirm process improvement |
valid |
The effects of process implementation are monitored and measured, and the achievement of defined improvement goals is confirmed. |
|
PIM.3.BP8: Communicate results of improvement |
valid |
Knowledge gained from the improvements and progress of the improvement implementation is communicated to affected parties. |
|
REU.2.BP1: Select products for reuse |
valid |
Select the products to be reused using defined criteria. .. note:: Products for reuse may be systems, hardware or software components, third party components or legacy components. |
|
REU.2.BP2: Analyze the reuse capability of the product |
valid |
Analyze the designated target architecture and the product to be reused to determine its applicability in the target architecture according to relevant criteria. .. note:: Examples for criteria can be requirements compliance, verifiability of the product to be reused in the target architecture, or portability/interoperability. |
|
REU.2.BP3: Define limitations for reuse |
valid |
Define and communicate limitations for the products to be reused. .. note:: Limitations may address parameters of operational environment. |
|
REU.2.BP4: Ensure qualification of products for reuse |
valid |
Provide evidence that the product for reuse is qualified for the intended use of the deliverable. .. note:: Qualification may be demonstrated by verification evidence. .. note:: Verification may include the appropriateness of documentation. |
|
REU.2.BP5: Provide products for reuse |
valid |
Make available the product to be reused to affected parties. .. note:: Refer to HWE.3, SWE.5 or SYS.4 for more information on integration of hardware, software, or system components. |
|
REU.2.BP6: Communicate information about effectiveness of reuse activities |
valid |
Establish communication and notification mechanism about experiences and technical outcomes to the provider of reused products. .. note:: The communication with the provider of a reused product may depend on whether the product is under development or not. |
|
SPL.2.BP1: Define the functional content of releases |
valid |
Define the functionality to be included and the release criteria for each release. .. note:: This may include the hardware elements, software elements, and extra application parameter files (influencing the identified system functionality) that are needed for the release. |
|
SPL.2.BP2: Define release package |
valid |
Define the release as well as supporting tools and information. .. note:: The release package may include also programming tools. |
|
SPL.2.BP3: Ensure unique identification of releases |
valid |
Ensure a unique identification of the release based upon the intended purpose and expectations of the release. .. note:: Unique identification may be realized by a classification and numbering scheme for product releases. |
|
SPL.2.BP4: Build the release from items under configuration control |
valid |
Build the release from items under configuration control to ensure integrity. .. note:: This practice may be supported by the SUP.8 Configuration Management Process. |
|
SPL.2.BP5: Ensure release approval before delivery |
valid |
Criteria for the release are satisfied before delivery takes place. |
|
SPL.2.BP6: Provide a release note |
valid |
A release is accompanied by information detailing key characteristics of the release. .. note:: The release note may include information about legal aspects like relevant target markets, legislation that is considered etc. See also VAL.1 Validation. |
|
SPL.2.BP7: Communicate the type, service level and duration of support for a release |
valid |
Identify and communicate the type, service level and duration of support for a release. |
|
SPL.2.BP8: Deliver the release package to the intended customer |
valid |
Deliver the release package to the intended customer. .. note:: The intended customer may be an internal organizational unit or an external organization. |
|
SUP.1.BP1: Ensure independence of quality assurance |
valid |
Ensure that quality assurance is performed independently and objectively without conflicts of interest. .. note:: Possible inputs for evaluating the independence may be assignment to financial and/or organizational structure as well as responsibility for processes that are subject to quality assurance (no self-monitoring). |
|
SUP.1.BP2: Define criteria for quality assurance |
valid |
Define quality criteria for work products as well as for process tasks and their performance. .. note:: Quality criteria may consider internal and external inputs such as customer requirements, standards, milestones, etc. |
|
SUP.1.BP3: Assure quality of work products |
valid |
Identify work products subject to quality assurance according to the quality criteria. Perform appropriate activities to evaluate the work products against the defined quality criteria and document the results. .. note:: Quality assurance activities may include reviews, problem analysis and lessons learned that improve the work products for further use. |
|
SUP.1.BP4: Assure quality of process activities |
valid |
Identify processes subject to quality assurance according to the quality criteria. Perform appropriate activities to evaluate the processes against their defined quality criteria and associated target values and document the results. .. note:: Quality assurance activities may include process assessments, problem analysis, regular check of methods, tools, and the adherence to defined processes, and consideration of lessons learned. |
|
SUP.1.BP5: Summarize and communicate quality assurance activities and results |
valid |
Regularly report performance, non-conformances, and trends of quality assurance activities to all affected parties. |
|
SUP.1.BP6: Ensure resolution of non-conformances |
valid |
Analyze, track, correct, resolve, and further prevent non-conformances found in quality assurance activities. .. note:: Non-conformances detected in work products may be entered into the problem resolution management process (SUP.9). .. note:: Non-conformances detected in the process definition or implementation may be entered into a process improvement process (PIM.3). |
|
SUP.1.BP7: Escalate non-conformances |
valid |
Escalate relevant non-conformances to appropriate levels of management and other relevant stakeholders to facilitate their resolution. .. note:: The decision whether to escalate non-conformances may be based on criteria such as delay of resolution, urgency, and risk. |
|
SUP.10.BP1: Identify and record the change requests |
valid |
The scope for application of change requests is identified. Each change request is uniquely identified, described, and recorded, including the initiator and reason of the change request. A status is assigned to each change request to facilitate tracking. .. note:: Change requests may be used for changes related to e.g., product, process, methods. .. note:: Example values for the change request status are “open”, “under investigation”, “implemented”, etc. .. note:: The change request handling may differ across the product life cycle e.g., during prototype |
|
SUP.10.BP2: Analyze and assess change requests |
valid |
Change requests are analyzed by relevant parties according to analysis criteria. Work products affected by the change request and dependencies to other change requests are determined. The impact of the change requests is assessed. .. note:: Examples for analysis criteria are: resource requirements, scheduling issues, risks, benefits, etc. |
|
SUP.10.BP3: Approve change requests before implementation |
valid |
Change requests are prioritized and approved for implementation based on analysis results and availability of resources. .. note:: A Change Control Board (CCB) is an example mechanism used to approve change requests. .. note:: Prioritization of change requests may be done by allocation to releases. |
|
SUP.10.BP4: Establish bidirectional traceability |
valid |
Establish bidirectional traceability between change requests and work products affected by the change requests. In case that the change request is initiated by a problem, establish bidirectional traceability between change requests and the corresponding problem reports. |
|
SUP.10.BP5: Confirm the implementation of change requests |
valid |
The implementation of change requests is confirmed before closure by relevant stakeholders. |
|
SUP.10.BP6: Track change requests to closure |
valid |
Change requests are tracked to closure. The status of change requests is communicated to all affected parties. .. note:: Examples for informing affected parties can be daily standup meetings or tool-supported workflows. |
|
SUP.11.BP1: Establish an ML data management system |
valid |
Establish an ML data management system which supports * ML data management activities, * relevant sources of ML data, * ML data life cycle including a status model, and * interfaces to affected parties. .. note:: Supported ML data management activities may include data collection, labeling/annotation, and structuring. |
|
SUP.11.BP2: Develop an ML data quality approach |
valid |
Develop an approach to ensure that the quality of ML data is analyzed based on defined ML data quality criteria and activities are performed to support avoidance of biases of data. .. note:: Examples of ML data quality criteria are relevant data sources, reliability and consistency of labelling, completeness against ML data requirements. .. note:: The ML data management system should support the quality criteria and activities of the ML data quality approach. .. note:: Biases to avoid may include sampling bias (e.g., gender, age) and feedback loop bias. .. note:: For creation of ML data sets see MLE.3.BP2 and MLE.4.BP2. |
|
SUP.11.BP3: Collect ML data |
valid |
Relevant sources for raw data are identified and continuously monitored for changes. The raw data is collected according to the ML data requirements. .. note:: The identification and collection of ML data might be an organizational responsibility. .. note:: Continuous monitoring should include the ODD and may lead to changes of the ML requirements. |
|
SUP.11.BP4: Process ML data |
valid |
The raw data are processed (annotated, analyzed, and structured) according to the ML data requirements. |
|
SUP.11.BP5: Assure quality of ML data |
valid |
Perform the activities according to the ML data quality approach to ensure that the ML data meets the defined ML data quality criteria. .. note:: These activities may include sample-based reviews or statistical methods. |
|
SUP.11.BP6: Communicate agreed processed ML data |
valid |
Inform all affected parties about the agreed processed ML data and provide them to the affected parties. |
|
SUP.8.BP1: Identify configuration items |
valid |
Define selection criteria for identifying relevant work products to be subject to configuration management. Identify and document configuration items according to the defined selection criteria. .. note:: Configuration items are representing work products or group of work products which are subject to configuration management as a single entity. .. note:: Configuration items may vary in complexity, size, and type, ranging from an entire system including all system, hardware, and software documentation down to a single element or document. .. note:: The selection criteria may be applied to single work products or a group of work products. |
|
SUP.8.BP2: Define configuration item properties |
valid |
Define the necessary properties needed for the modification and control of configuration items. .. note:: The configuration item properties may be defined for single configuration items or a group of items. .. note:: Configuration item properties may include a status model (e.g., Under Work, Tested, Released, etc.), storage location, access rights, etc. .. note:: The application of properties may be implemented by attributes of configuration items. |
|
SUP.8.BP3: Establish configuration management |
valid |
Establish configuration management mechanisms for control of identified configuration items including the configuration item properties, including mechanisms for controlling parallel modifications of configuration items. .. note:: This may include specific mechanisms for different configuration item types, such as branch and merge management, or checkout control. |
|
SUP.8.BP4: Control modifications |
valid |
Control modifications using the configuration management mechanisms. .. note:: This may include the application of a defined status model for configuration items. |
|
SUP.8.BP5: Establish baselines |
valid |
Define and establish baselines for internal purposes, and for external product delivery, for all relevant configuration items. |
|
SUP.8.BP6: Summarize and communicate configuration status |
valid |
Record, summarize, and communicate the status of configuration items and established baselines to affected parties in order to support the monitoring of progress and status. .. note:: Regular communication of the configuration status, e.g., based on a defined status model supports project management, quality activities, and dedicated project phases such as software integration. |
|
SUP.8.BP7: Ensure completeness and consistency |
valid |
Ensure that the information about configuration items is correct and complete including configuration item properties. Ensure the completeness and consistency of baselines. .. note:: Completeness and consistency of a baseline means that all required configuration items are included and consistent, and have the required status. This can be used to support e.g., project gate approval. |
|
SUP.8.BP8: Verify backup and recovery mechanisms availability. |
valid |
Verify the availability of appropriate backup and recovery mechanisms for the configuration management including the controlled configuration items. Initiate measures in case of insufficient backup and recovery mechanisms. .. note:: Backup and recovery mechanisms may be defined and implemented by organizational units outside the project team. This may include references to corresponding procedures or regulations. |
|
SUP.9.BP1: Identify and record the problem |
valid |
Each problem is uniquely identified, described and recorded. A status is assigned to each problem to facilitate tracking. Supporting information is provided to reproduce and diagnose the problem. .. note:: Problems may relate to e.g., product, resources, or methods. .. note:: Example values for the problem status are “new”, “solved”, “closed”, etc. .. note:: Supporting information may include e.g, the origin of the problem, how it can be reproduced, environmental information, by whom it has been detected. .. note:: Unique identification supports traceability to changes made as needed by the change request management process (SUP.10). |
|
SUP.9.BP2: Determine the cause and the impact of the problem |
valid |
Analyze the problem, determine its cause, including common causes if existing, and impact. Involve relevant parties. Categorize the problem. .. note:: Problem categorization (e.g., light, medium, severe) may be based on severity, criticality, urgency, etc. |
|
SUP.9.BP3: Authorize urgent resolution action |
valid |
Obtain authorization for immediate action if a problem requires an urgent resolution according to the categorization. |
|
SUP.9.BP4: Raise alert notifications |
valid |
If according to the categorization the problem has a high impact on other systems or other affected parties, an alert notification needs to be raised accordingly. |
|
SUP.9.BP5: Initiate problem resolution |
valid |
Initiate appropriate actions according to the categorization to resolve the problem long-term, including review of those actions or initiate a change request. This includes synchronization and consistency with short-term urgent resolution actions, if applicable. |
|
SUP.9.BP6: Track problems to closure |
valid |
Track the status of problems to closure including all related change requests. The closure of problems is accepted by relevant stakeholders. |
|
SUP.9.BP7: Report the status of problem resolution activities |
valid |
Collect and analyze problem resolution management data, identify trends, and initiate related actions. Regularly report the results of data analysis, the identified trends and the status of problem resolution activities to relevant stakeholders. .. note:: Collected data may contain information about where the problems occurred, how and when they were found, what their impacts were, etc. |
|
SWE.1.BP1: Specify software requirements |
valid |
Use the system requirements and the system architecture to identify and document the functional and non-functional requirements for the software according to defined characteristics for requirements. .. note:: Characteristics of requirements are defined in standards such as ISO IEEE 29148, ISO 26262-8:2018, or the INCOSE Guide for Writing Requirements. .. note:: Examples for defined characteristics of requirements shared by technical standards are verifiability (i.e., verification criteria being inherent in the requirements text), unambiguity/comprehensibility, freedom from design and implementation, and not contradicting any other requirement. .. note:: In case of software-only development, the system requirements and the system architecture refer to a given operating environment. In that case, stakeholder requirements can be used as the basis for identifying the required functions and capabilities of the software. .. note:: The hardware-software-interface (HSI) definition puts in context hardware and therefore it is an interface decision at the system design level. If such a HSI exists, then it may provide input to software requirements. |
|
SWE.1.BP2: Structure software requirements |
valid |
Structure and prioritize the software requirements. .. note:: Examples for structuring criteria can be grouping (e.g., by functionality) or expressing product variants. .. note:: Prioritization can be done according to project or stakeholder needs via e.g., definition of release scopes. Refer to SPL.2.BP1. |
|
SWE.1.BP3: Analyze software requirements |
valid |
Analyze the specified software requirements including their interdependencies to ensure correctness, technical feasibility, and to support project management regarding project estimates. .. note:: See MAN.3.BP3 for project feasibility and MAN.3.BP5 for project estimates. .. note:: Technical feasibility can be evaluated based on e.g., platform or product line, or by prototyping. |
|
SWE.1.BP4: Analyze the impact on the operating environment |
valid |
Analyze the impact that the software requirements will have on elements in the operating environment. |
|
SWE.1.BP5: Ensure consistency and establish bidirectional traceability |
valid |
Ensure consistency and establish bidirectional traceability between software requirements and system architecture. Ensure consistency and establish bidirectional traceability between software requirements and system requirements. .. note:: Redundant traceability is not intended. .. note:: There may be non-functional system requirements that the software requirements do not trace to. Examples are process requirements or requirements related to later software product lifecycle phases such as incident handling. Such requirements are still subject to verification. .. note:: Bidirectional traceability supports consistency, and facilitates impact analysis of change requests, and demonstration of verification coverage. Traceability alone, e.g., the existence of links, does not necessarily mean that the information is consistent with each other. .. note:: In case of software development only, the system requirements and system architecture refer to a given operating environment. In that case, consistency and bidirectional traceability can be ensured between stakeholder requirements and software requirements. |
|
SWE.1.BP6: Communicate agreed software requirements and impact on the operating environment |
valid |
Communicate the agreed software requirements, and the results of the analysis of impact on the operating environment, to all affected parties. |
|
SWE.2.BP1: Specify static aspects of the software architecture |
valid |
Specify and document the static aspects of the software architecture with respect to the functional and non-functional software requirements, including external interfaces and a defined set of software components with their interfaces and relationships. .. note:: The hardware-software-interface (HSI) definition puts in context the hardware design and therefore is an aspect of system design (SYS.3). |
|
SWE.2.BP2: Specify dynamic aspects of the software architecture |
valid |
Specify and document the dynamic aspects of the software architecture with respect to the functional and non- functional software requirements, including the behavior of the software components and their interaction in different software modes, and concurrency aspects. .. note:: Examples for concurrency aspects are application-relevant interrupt handling, preemptive processing, multi-threading. .. note:: Examples for behavioral descriptions are natural language or semi-formal notation (e.g, SysML, UML). |
|
SWE.2.BP3: Analyze software architecture |
valid |
Analyze the software architecture regarding relevant technical design aspects and to support project management regarding project estimates. Document a rationale for the software architectural design decision. .. note:: See MAN.3.BP3 for project feasibility and MAN.3.BP5 for project estimates. .. note:: The analysis may include the suitability of pre-existing software components for the current application. .. note:: Examples of methods suitable for analyzing technical aspects are prototypes, simulations, qualitative analyses. .. note:: Examples of technical aspects are functionality, timings, and resource consumption (e.g, ROM, RAM, external / internal EEPROM or Data Flash or CPU load). .. note:: Design rationales can include arguments such as proven-in-use, reuse of a software framework or software product line, a make-or-buy decision, or found in an evolutionary way (e.g, set-based design). |
|
SWE.2.BP4: Ensure consistency and establish bidirectional traceability |
valid |
Ensure consistency and establish bidirectional traceability between the software architecture and the software requirements. .. note:: There may be non-functional software requirements that the software architectural design does not trace to. Examples are development process requirements. Such requirements are still subject to verification. .. note:: Bidirectional traceability supports consistency, and facilitates impact analysis of change requests, and demonstration of verification coverage. Traceability alone, e.g, the existence of links, does not necessarily mean that the information is consistent with each other. |
|
SWE.2.BP5: Communicate agreed software architecture |
valid |
Communicate the agreed software architecture to all affected parties. |
|
SWE.3.BP1: Specify the static aspects of the detailed design |
valid |
For each software component specify the behavior of its software units, their static structure and relationships, their interfaces including - valid data value ranges for inputs and outputs (from the application domain perspective), and - physical or measurement units applicable to inputs and outputs (from the application domain perspective). .. note:: The boundary of a software unit is independent from the software unit’s representation in the source code, code file structure, or model-based implementation, respectively. It is rather driven by the semantics of the application domain perspective. Therefore, a software unit may be, at the code level, represented by a single subroutine or a set of subroutines. .. note:: Examples of valid data value ranges with applicable physical units from the application domain perspective are ‘0..200 [m/s]’, ‘0..3.8 [A]’ or ‘1..100 [N]’. For mapping such application domain value ranges to programming language-level data types (such as unsigned Integer with a value range of 0..65535) refer to {need}`std_bp_aspice-40__SWE-3-BP2`. .. note:: Examples of a measurement unit are ‘%’ or ‘‰’ .. note:: A counter is an example of a parameter, or a return value, to which neither a physical nor a surement unit is applicable. .. note:: The hardware-software-interface (HSI) definition puts in context the hardware design and refore is an aspect of system design (SYS.3). |
|
SWE.3.BP2: Specify dynamic aspects of the detailed design |
valid |
Specify and document the dynamic aspects of the detailed design with respect to the software architecture, including the interactions between relevant software units to fulfill the component’s dynamic behavior. .. note:: Examples for behavioral descriptions are natural language or semi-formal notation (e.g, SysML, UML). |
|
SWE.3.BP3: Develop software units |
valid |
Develop and document the software units consistent with the detailed design, and according to coding principles. .. note:: Examples for coding principles at capability level 1 are not to use implicit type conversions, only one entry and one exit point in subroutines, and range checks (design-by-contract, defensive programming). Further examples see e.g, :need:`std_req__iso26262__software_845` together with table 6. |
|
SWE.3.BP4: Ensure consistency and establish bidirectional traceability |
valid |
Ensure consistency and establish bidirectional traceability between the software detailed design and the software architecture. Ensure consistency and establish bidirectional traceability between the developed software units and the software detailed design. Ensure consistency and establish traceability between the software detailed design and the software requirements. .. note:: Redundancy should be avoided by establishing a combination of these approaches. .. note:: Examples for tracing a software unit in the detailed design to a software requirement directly are communication matrices or basis software aspects such as a list of diagnosis identifiers inherent in an Autosar configuration. .. note:: Bidirectional traceability supports consistency, and facilitates impact analysis of change requests, and demonstration of verification coverage. Traceability alone, e.g., the existence of links, does not necessarily mean that the information is consistent with each other. |
|
SWE.3.BP5: Communicate agreed software detailed design and developed software units |
valid |
Communicate the agreed software detailed design and developed software units to all affected parties. |
|
SWE.4.BP1: Specify software unit verification measures |
valid |
Specify verification measures for each software unit defined in the software detailed design, including - pass/fail criteria for verification measures, - entry and exit criteria for verification measures, and - the required verification infrastructure. .. note:: Examples for unit verification measures are static analysis, code reviews, and unit testing. .. note:: Static analysis can be done based on MISRA rulesets and other coding standards. |
|
SWE.4.BP2: Select software unit verification measures |
valid |
Document the selection of verification measures considering selection criteria including criteria for regression verification. The documented selection of verification measures shall have sufficient coverage according to the release scope. |
|
SWE.4.BP3: Verify software units |
valid |
Perform software unit verification using the selected verification measures. Record the verification results including pass/fail status and corresponding verification measure data. .. note:: See SUP.9 for handling of verification results that deviate from expected results |
|
SWE.4.BP4: Ensure consistency and establish bidirectional traceability |
valid |
Ensure consistency and establish bidirectional traceability between verification measures and the software units defined in the detailed design. Establish bidirectional traceability between the verification results and the verification measures. .. note:: Bidirectional traceability supports consistency, and facilitates impact analysis of change requests, and demonstration of verification coverage. Traceability alone, e.g., the existence of links, does not necessarily mean that the information is consistent with each other. |
|
SWE.4.BP5: Summarize and communicate results |
valid |
Summarize the results of software unit verification and communicate them to all affected parties. .. note:: Providing all necessary information from the test case execution in a summary enables other parties to judge the consequences. |
|
SWE.5.BP1: Specify software integration verification measures |
valid |
Specify verification measures, based on a defined sequence and preconditions for the integration of software elements, against the defined static and dynamic aspects of the software architecture, including - techniques for the verification measures; - pass/fail criteria for verification measures; - entry and exit criteria for verification measures, and - the required verification infrastructure and environment setup. .. note:: Examples on which the software integration verification measures may focus on are the correct dataflow and dynamic interaction between software components together with their timing dependencies, the correct interpretation of data by all software components using an interface, and the compliance to resource consumption objectives. .. note:: The software integration verification measure may be supported by using hardware debug interfaces or simulation environments (e.g, Software-in-the-Loop-Simulation). |
|
SWE.5.BP2: Specify verification measures for verifying software component behavior |
valid |
Specify verification measures for software component verification against the defined software components’ behavior and their interfaces in the software architecture, including - techniques for the verification measures, - entry and exit criteria for verification measures, - pass/fail criteria for verification measures, and - the required verification infrastructure and environment setup. .. note:: Verification measures are related to software components but not to the software units since software unit verification is addressed in the process :doc:`SWE.4 Software Unit Verification <swe.4>`. |
|
SWE.5.BP3: Select verification measures |
valid |
Document the selection of integration verification measures for each integration step considering selection criteria including criteria for regression verification. The documented selection of verification measures shall have sufficient coverage according to the release scope. .. note:: Examples for selection criteria can be the need for continuous integration /continuous development regression verification (due to e.g, changes to the software architectural or detailed design), or the intended use of the delivered product release (e.g, test bench, test track, public road etc.). |
|
SWE.5.BP4: Integrate software elements and perform integration verification |
valid |
Integrate the software elements until the software is fully integrated according to the specified interfaces and interactions between the Software elements, and according to the defined sequence and defined preconditions. Perform the selected integration verification measures. Record the verification measure data including pass/fail status and corresponding verification measure data. .. note:: Examples for preconditions for starting software integration are qualification of pre-existing software components, off-the-shelf software components, open-source-software, or auto-code generated software. .. note:: Defined preconditions may allow e.g, big-bang-integration of all software components, continuous integration, as well as stepwise integration (e.g, across software units and/or software components up to the fully integrated software) with accompanying verification measures. .. note:: See SUP.9 for handling deviations of verification results deviate expected results. |
|
SWE.5.BP5: Perform software component verification |
valid |
Perform the selected verification measures for verifying software component behavior. Record the verification results including pass/fail status and corresponding verification measure data. .. note:: See SUP.9 for handling deviations of verification results deviate expected results. |
|
SWE.5.BP6: Ensure consistency and establish bidirectional traceability |
valid |
Ensure consistency and establish bidirectional traceability between verification measures and the static and dynamic aspects of the software architecture and detailed design. Establish bidirectional traceability between verification results and verification measures. .. note:: Bidirectional traceability supports consistency, and facilitates impact analysis of change requests, and demonstration of verification coverage. Traceability alone, e.g., the existence of links, does not necessarily mean that the information is consistent with each other. |
|
SWE.5.BP7: Summarize and communicate results |
valid |
Summarize the software component verification and the software integration verification results and communicate them to all affected parties. .. note:: Providing all necessary information from the test case execution in a summary enables other parties to judge the consequences. |
|
SWE.6.BP1: Specify verification measures for software verification |
valid |
Specify the verification measures for software verification suitable to provide evidence for compliance of the integrated software with the functional and non-functional information in the software requirements, including - techniques for the verification measures, - pass/fail criteria for verification measures, - a definition of entry and exit criteria for the verification measures, - necessary sequence of verification measures, and - the required verification infrastructure and environment setup. .. note:: The selection of appropriate techniques for verification measures may depend on the content of the respective software requirement (e.g, boundary values and equivalence classes for data range-oriented requirements, positive/sunny-day-test vs. negative testing such as fault injection), or on requirements-based testing vs. “error guessing based on knowledge or experience”. |
|
SWE.6.BP2: Select verification measures |
valid |
Document the selection of verification measures considering selection criteria including criteria for regression verification. The documented selection of verification measures shall have sufficient coverage according to the release scope. .. note:: Examples for selection criteria can be prioritization of requirements, continuous development, the need for regression verification (due to e.g., changes to the software requirements), or the intended use of the delivered product release (test bench, test track, public road etc.) |
|
SWE.6.BP3: Verify the integrated software |
valid |
Perform the verification of the integrated software using the selected verification measures. Record the verification results including pass/fail status and corresponding verification measure data. .. note:: See SUP.9 for handling verification results that deviate from expected results. |
|
SWE.6.BP4: Ensure consistency and establish bidirectional traceability |
valid |
Ensure consistency and establish bidirectional traceability between verification measures and software requirements. Establish bidirectional traceability between verification results and verification measures. .. note:: Bidirectional traceability supports consistency, and facilitates impact analysis of change requests, and demonstration of verification coverage. Traceability alone, e.g., the existence of links, does not necessarily mean that the information is consistent with each other. |
|
SWE.6.BP5: Summarize and communicate results |
valid |
Summarize the software verification results and communicate them to all affected parties. .. note:: Providing all necessary information from the test case execution in a summary enables other parties to judge the consequences. |
|
analysis_641 |
valid |
||
analysis_642 |
valid |
||
analysis_643 |
valid |
||
analysis_644 |
valid |
||
analysis_741 |
valid |
||
analysis_742 |
valid |
||
analysis_743 |
valid |
||
analysis_744 |
valid |
||
analysis_745 |
valid |
||
analysis_746 |
valid |
||
analysis_747 |
valid |
||
analysis_748 |
valid |
||
analysis_749 |
valid |
||
analysis_841 |
valid |
||
analysis_8410 |
valid |
||
analysis_842 |
valid |
||
analysis_843 |
valid |
||
analysis_844 |
valid |
||
analysis_845 |
valid |
||
analysis_846 |
valid |
||
analysis_847 |
valid |
||
analysis_848 |
valid |
||
analysis_849 |
valid |
||
management_5421 |
valid |
||
management_5422 |
valid |
||
management_5423 |
valid |
||
management_5424 |
valid |
||
management_5425 |
valid |
||
management_5426 |
valid |
||
management_5427 |
valid |
||
management_5431 |
valid |
||
management_5432 |
valid |
||
management_5433 |
valid |
||
management_5434 |
valid |
||
management_5435 |
valid |
||
management_5441 |
valid |
||
management_5451 |
valid |
||
management_5461 |
valid |
||
management_64101 |
valid |
||
management_64102 |
valid |
||
management_64103 |
valid |
||
management_64104 |
valid |
||
management_64105 |
valid |
||
management_64111 |
valid |
||
management_64112 |
valid |
||
management_64113 |
valid |
||
management_64114 |
valid |
||
management_64121 |
valid |
||
management_641210 |
valid |
||
management_641211 |
valid |
||
management_641212 |
valid |
||
management_641213 |
valid |
||
management_64122 |
valid |
||
management_64123 |
valid |
||
management_64124 |
valid |
||
management_64125 |
valid |
||
management_64126 |
valid |
||
management_64127 |
valid |
||
management_64128 |
valid |
||
management_64129 |
valid |
||
management_64131 |
valid |
||
management_64132 |
valid |
||
management_64133 |
valid |
||
management_64134 |
valid |
||
management_64135 |
valid |
||
management_6421 |
valid |
||
management_6422 |
valid |
||
management_6423 |
valid |
||
management_6424 |
valid |
||
management_6431 |
valid |
||
management_6432 |
valid |
||
management_6433 |
valid |
||
management_644 |
valid |
||
management_6451 |
valid |
||
management_6452 |
valid |
||
management_6453 |
valid |
||
management_6454 |
valid |
||
management_6455 |
valid |
||
management_6456 |
valid |
||
management_6457 |
valid |
||
management_6461 |
valid |
||
management_64610 |
valid |
||
management_6462 |
valid |
||
management_6463 |
valid |
||
management_6464 |
valid |
||
management_6465 |
valid |
||
management_6466 |
valid |
||
management_6467 |
valid |
||
management_6468 |
valid |
||
management_6469 |
valid |
||
management_6471 |
valid |
||
management_6472 |
valid |
||
management_6481 |
valid |
||
management_6482 |
valid |
||
management_6491 |
valid |
||
management_6492 |
valid |
||
management_6493 |
valid |
||
software_1041 |
valid |
||
software_1042 |
valid |
||
software_1043 |
valid |
||
software_1044 |
valid |
||
software_1045 |
valid |
||
software_1046 |
valid |
||
software_1047 |
valid |
||
software_1141 |
valid |
||
software_1142 |
valid |
||
software_1143 |
valid |
||
software_1144 |
valid |
||
software_541 |
valid |
||
software_542 |
valid |
||
software_543 |
valid |
||
software_641 |
valid |
||
software_642 |
valid |
||
software_643 |
valid |
||
software_644 |
valid |
||
software_645 |
valid |
||
software_646 |
valid |
||
software_647 |
valid |
||
software_741 |
valid |
||
software_7410 |
valid |
||
software_7411 |
valid |
||
software_7412 |
valid |
||
software_7413 |
valid |
||
software_7414 |
valid |
||
software_742 |
valid |
||
software_743 |
valid |
||
software_744 |
valid |
||
software_745 |
valid |
||
software_746 |
valid |
||
software_747 |
valid |
||
software_748 |
valid |
||
software_749 |
valid |
||
software_841 |
valid |
||
software_842 |
valid |
||
software_843 |
valid |
||
software_844 |
valid |
||
software_845 |
valid |
||
software_941 |
valid |
||
software_942 |
valid |
||
software_943 |
valid |
||
software_944 |
valid |
||
software_945 |
valid |
||
software_app_c_41 |
valid |
||
software_app_c_42 |
valid |
||
software_app_c_43 |
valid |
||
software_app_c_44 |
valid |
||
software_app_c_45 |
valid |
||
support_1041 |
valid |
||
support_1042 |
valid |
||
support_1043 |
valid |
||
support_1044 |
valid |
||
support_1045 |
valid |
||
support_1046 |
valid |
||
support_1141 |
valid |
||
support_1142 |
valid |
||
support_1143 |
valid |
||
support_11441 |
valid |
||
support_11442 |
valid |
||
support_11451 |
valid |
||
support_11452 |
valid |
||
support_11453 |
valid |
||
support_11454 |
valid |
||
support_11461 |
valid |
||
support_11462 |
valid |
||
support_11471 |
valid |
||
support_11472 |
valid |
||
support_11473 |
valid |
||
support_11474 |
valid |
||
support_11481 |
valid |
||
support_11482 |
valid |
||
support_11483 |
valid |
||
support_11491 |
valid |
||
support_11492 |
valid |
||
support_12421 |
valid |
||
support_12422 |
valid |
||
support_12423 |
valid |
||
support_12424 |
valid |
||
support_12425 |
valid |
||
support_1243 |
valid |
||
support_641 |
valid |
||
support_6421 |
valid |
||
support_6422 |
valid |
||
support_6423 |
valid |
||
support_6424 |
valid |
||
support_6425 |
valid |
||
support_6431 |
valid |
||
support_6432 |
valid |
||
support_6433 |
valid |
||
support_6434 |
valid |
||
support_741 |
valid |
||
support_742 |
valid |
||
support_743 |
valid |
||
support_744 |
valid |
||
support_745 |
valid |
||
support_8411 |
valid |
||
support_8412 |
valid |
||
support_8413 |
valid |
||
support_8414 |
valid |
||
support_8421 |
valid |
||
support_8422 |
valid |
||
support_8431 |
valid |
||
support_8432 |
valid |
||
support_8441 |
valid |
||
support_8442 |
valid |
||
support_8451 |
valid |
||
support_8452 |
valid |
||
support_8453 |
valid |
||
support_9411 |
valid |
||
support_9412 |
valid |
||
support_9421 |
valid |
||
support_9422 |
valid |
||
support_9423 |
valid |
||
support_9424 |
valid |
||
support_9431 |
valid |
||
support_9432 |
valid |
||
support_9433 |
valid |
||
support_9434 |
valid |
||
system_6411 |
valid |
||
system_6412 |
valid |
||
system_6413 |
valid |
||
system_6414 |
valid |
||
system_6421 |
valid |
||
system_6422 |
valid |
||
system_6423 |
valid |
||
system_6424 |
valid |
||
system_6425 |
valid |
||
Pas1 |
valid |
||
Pas2 |
valid |
||
Pas11 |
valid |
||
Pas3 |
valid |
||
Pas4 |
valid |
||
Pas5 |
valid |
||
Pas6 |
valid |
||
Pas7 |
valid |
||
Pas8 |
valid |
||
Pas9 |
valid |
||
Pas10 |
valid |
||
Pas12 |
valid |
||
Pas13 |
valid |
||
Pas14 |
valid |
||
Pas19 |
valid |
||
Pas18 |
valid |
||
Pas17 |
valid |
||
Pas18 |
valid |
||
Pas19 |
valid |
||
Pas20 |
valid |
||
Pas21 |
valid |
||
Pas22 |
valid |
||
Pas23 |
valid |
||
Pas24 |
valid |
||
Pas25 |
valid |
||
Pas26 |
valid |
||
Pas27 |
valid |
||
Pas28 |
valid |
||
Pas29 |
valid |
||
Pas30 |
valid |
||
Pas31 |
valid |
||
Pas32 |
valid |
||
Pas33 |
valid |
||
assessment_15621 |
valid |
||
assessment_15622 |
valid |
||
assessment_15721 |
valid |
||
assessment_15722 |
valid |
||
assessment_15723 |
valid |
||
assessment_15724 |
valid |
||
assessment_15725 |
valid |
||
assessment_15821 |
valid |
||
assessment_15822 |
valid |
||
assessment_15921 |
valid |
||
continual_8321 |
valid |
||
continual_8322 |
valid |
||
continual_8323 |
valid |
||
continual_8421 |
valid |
||
continual_8521 |
valid |
||
continual_8522 |
valid |
||
continual_8621 |
valid |
||
continual_8622 |
valid |
||
development_10411 |
valid |
||
development_10412 |
valid |
||
development_10413 |
valid |
||
development_10414 |
valid |
||
development_10415 |
valid |
||
development_10416 |
valid |
||
development_10417 |
valid |
||
development_10418 |
valid |
||
development_10421 |
valid |
||
development_10422 |
valid |
||
development_10423 |
valid |
||
development_10424 |
valid |
||
development_10425 |
valid |
||
maintenance_13321 |
valid |
||
maintenance_13322 |
valid |
||
maintenance_13421 |
valid |
||
org_management_5421 |
valid |
||
org_management_5422 |
valid |
||
org_management_5423 |
valid |
||
org_management_5441 |
valid |
||
org_management_5442 |
valid |
||
org_management_5443 |
valid |
||
org_management_5451 |
valid |
||
org_management_5452 |
valid |
||
org_management_5461 |
valid |
||
prj_management_6411 |
valid |
||
prj_management_6421 |
valid |
||
prj_management_64210 |
valid |
||
prj_management_64211 |
valid |
||
prj_management_6422 |
valid |
||
prj_management_6423 |
valid |
||
prj_management_6424 |
valid |
||
prj_management_6425 |
valid |
||
prj_management_6426 |
valid |
||
prj_management_6427 |
valid |
||
prj_management_6428 |
valid |
||
prj_management_6429 |
valid |
||
prj_management_6431 |
valid |
||
prj_management_6432 |
valid |
||
prj_management_6441 |
valid |
||
prj_management_6442 |
valid |
||
prj_management_6443 |
valid |
||
prj_management_6451 |
valid |
||
prj_management_6452 |
valid |
||
prj_management_6453 |
valid |
||
prj_management_6461 |
valid |
||
prj_management_6462 |
valid |
||
prj_management_6471 |
valid |
||
prj_management_6491 |
valid |
||
prj_management_6492 |
valid |
||
analysis_551 |
valid |
||
analysis_552 |
valid |
||
analysis_651 |
valid |
||
analysis_751 |
valid |
||
analysis_752 |
valid |
||
analysis_851 |
valid |
||
analysis_852 |
valid |
||
management_551 |
valid |
||
management_552 |
valid |
||
management_553 |
valid |
||
management_554 |
valid |
||
management_651 |
valid |
||
management_652 |
valid |
||
management_653 |
valid |
||
management_654 |
valid |
||
management_655 |
valid |
||
management_656 |
valid |
||
management_751 |
valid |
||
software_1051 |
valid |
||
software_1052 |
valid |
||
software_1053 |
valid |
||
software_1151 |
valid |
||
software_1152 |
valid |
||
software_551 |
valid |
||
software_651 |
valid |
||
software_652 |
valid |
||
software_653 |
valid |
||
software_751 |
valid |
||
software_752 |
valid |
||
software_753 |
valid |
||
software_754 |
valid |
||
software_851 |
valid |
||
software_852 |
valid |
||
software_951 |
valid |
||
software_952 |
valid |
||
software_C51 |
valid |
||
software_C52 |
valid |
||
software_C53 |
valid |
||
software_C54 |
valid |
||
software_C55 |
valid |
||
software_C56 |
valid |
||
software_C57 |
valid |
||
software_C58 |
valid |
||
support_1051 |
valid |
||
support_1052 |
valid |
||
support_1151 |
valid |
||
support_1152 |
valid |
||
support_1251 |
valid |
||
support_1252 |
valid |
||
support_1253 |
valid |
||
support_1351 |
valid |
||
support_1352 |
valid |
||
support_1353 |
valid |
||
support_1451 |
valid |
||
support_1452 |
valid |
||
support_1551 |
valid |
||
support_1651 |
valid |
||
support_551 |
valid |
||
support_552 |
valid |
||
support_553 |
valid |
||
support_554 |
valid |
||
support_555 |
valid |
||
support_751 |
valid |
||
support_851 |
valid |
||
support_852 |
valid |
||
support_853 |
valid |
||
support_854 |
valid |
||
support_951 |
valid |
||
support_952 |
valid |
||
support_953 |
valid |
||
system_651 |
valid |
||
system_652 |
valid |
||
system_653 |
valid |
||
system_654 |
valid |
||
system_655 |
valid |
||
system_656 |
valid |
||
system_657 |
valid |
||
system_751 |
valid |
||
system_752 |
valid |
||
system_851 |
valid |
||
system_852 |
valid |
||
isopas8926__4511 |
valid |
||
isopas8926__4512 |
valid |
||
isopas8926__4521 |
valid |
||
isopas8926__4522 |
valid |
||
isopas8926__4523 |
valid |
||
isopas8926__4524 |
valid |
||
isopas8926__4525 |
valid |
||
isopas8926__4526 |
valid |
||
isopas8926__4527 |
valid |
||
assessment_15331 |
valid |
||
assessment_15332 |
valid |
||
assessment_15431 |
valid |
||
assessment_15531 |
valid |
||
assessment_15631 |
valid |
||
assessment_15731 |
valid |
||
assessment_15831 |
valid |
||
assessment_15931 |
valid |
||
continual_8331 |
valid |
||
continual_8332 |
valid |
||
continual_8333 |
valid |
||
continual_8431 |
valid |
||
continual_8531 |
valid |
||
continual_8631 |
valid |
||
development_1051 |
valid |
||
development_1052 |
valid |
||
development_1053 |
valid |
||
development_1054 |
valid |
||
development_1055 |
valid |
||
development_1056 |
valid |
||
development_1057 |
valid |
||
maintenance_13331 |
valid |
||
org_management_551 |
valid |
||
org_management_552 |
valid |
||
org_management_553 |
valid |
||
org_management_554 |
valid |
||
org_management_555 |
valid |
||
prj_management_651 |
valid |
||
prj_management_652 |
valid |
||
prj_management_653 |
valid |
||
prj_management_654 |
valid |
||
TT-CHANGES |
valid |
XYZ is actively maintained, with regular updates to dependencies, and changes are verified to prevent regressions. **Guidance** We expect that XYZ will need to be modified many times during its useful/production lifetime, and therefore we need to be sure that we can make changes without breaking it. In practice this means being able to deal with updates to dependencies and tools, as well as updates to XYZ itself. Note that this implies that we need to be able to: - verify that updated XYZ still satisfies its expectations (see below), and - understand the behaviour of upstream/suppliers in delivering updates (e.g. frequency of planned updates, responsiveness for unplanned updates such as security fixes). We need to consider the maturity of XYZ, since new software is likely to contain more undiscovered faults/bugs and thus require more changes. To support this we need to be able to understand, quantify and analyse changes made to XYZ (and its dependencies) on an ongoing basis, and to assess the XYZ approach to bugs and breaking changes. We also need to be able to make modifications to any/all third-party components of XYZ and dependencies of XYZ, unless we are completely confident that suppliers/upstream will satisfy our needs throughout XYZ's production lifecycle. |
|
TT-CONFIDENCE |
valid |
Confidence in XYZ is measured by analysing actual performance in tests and in production. **Guidance** Our overall objective is to deliver releases of XYZ that meet our expectations and do not cause harm. By collecting and assessing evidence for all of the factors above, we aim to assess (ideally measure) confidence in each release candidate, to support go/nogo decision-making, In assessing confidence we need to consider various categories of evidence including: - subjective (e.g. provenance, reviews and approvals) - binary (e.g. test pass/fail) - stochastic (e.g. scheduling test results over time) - empirical (e.g. advance warning signal monitoring data from production deployments) |
|
TT-CONSTRUCTION |
valid |
Tools are provided to build XYZ from trusted sources (also provided) with full reproducibility. **Guidance** Where possible we prefer to build, configure and install XYZ from source code, because this reduces (but does not eliminate) the possibility of supply chain tampering. When constructing XYZ we aspire to a set of best practices including: - reproducible builds - construction from a given set of input source files and build instructions leads to a specific fileset - re-running the build leads to exactly the same fileset, bit-for-bit - ensuring that all XYZ dependencies are known and controlled (no reliance on external/internet resources, or unique/golden/blessed build server); and - automated build, configuration and deployment of XYZ based on declarative instructions, kept in version control (e.g. no engineer laptop in the loop for production releases) Some of these constraints may be relaxed during XYZ development/engineering phases, but they must all be fully applied for production releases. Note that when we receive only binaries, without source code, we must rely much more heavily on Provenance; who supplied the binaries, and how can we trust their agenda, processes, timescales and deliveries? |
|
TT-EXPECTATIONS |
valid |
Documentation is provided, specifying what XYZ is expected to do, and what it must not do, and how this is verified. **Guidance** While most modern software is developed without a formal requirement/specification process, we need to be clear about what we expect from XYZ, communicate these expectations, and verify that they are met. In most (almost all?) cases, we need to verify our expectations by tests. These tests must be automated and applied for every candidate release of XYZ. It is not sufficient to demonstrate that the software does what we expect. We also need to analyse the potential risks in our scenario, identify unacceptable and/or dangerous misbehaviours and verify that they are absent, prevented or mitigated. In most cases it is not sufficient to demonstrate behaviours and mitigations only in a factory/laboratory environment. We also need to establish methods for monitoring critical behaviours and misbehaviours in production, and methods for taking appropriate action based on advance warning signals of potential misbehaviour. |
|
TT-PROVENANCE |
valid |
All source code (and attestations for claims) for XYZ are provided with known provenance. **Guidance** Our trust in XYZ is informed by both the individuals and organisations contributing to it, and by the software tools and components used to develop and maintain it. Ideally we want all XYZ contributors to be expert, motivated, reliable, transparent and ethical. Unfortunately this is not always achievable in practice: - Given the scale, complexity and evolution of modern software systems it is impossible for engineers to be expert in all topics. - Even the most competent engineers have bad days. - Many engineers are unable to share information due to commercial secrecy agreements. - Individuals and teams may be motivated or manipulated by external pressures, e.g. money and politics. We can and should, however, consider who produced XYZ and its components, their motivations and practices, the assertions they make, supporting evidence for these assertions, and feedback from users of XYZ if available. Similarly, we want existing software used to create XYZ to be well documented, actively maintained, thoroughly tested, bug-free and well suited to its use in XYZ. In practice, we will rarely have all of this, but we can at least evaluate the software components of XYZ, and the tools used to construct it. |
|
TT-RESULTS |
valid |
Evidence is provided to demonstrate that XYZ does what it is supposed to do, and does not do what it must not do. **Guidance** We need to perform tests to verify expected behaviours and properties of XYZ in advance of every candidate release. We also need to validate these tests, to confirm that they do actually test for the expected behaviours or properties, and that test failures are properly detected. Usually this can be done by inducting software faults to exercise the tests and checking that the results record the expected failure. Similarly we need to verify that prevention measures and mitigations continue to work for each XYZ candidate release, and in production. All these validated test results and monitored advance warning signal data from production must be collected and analysed on an ongoing basis. We need to notice when results or trends in advance warning signals change, and react appropriately. |
|
TRUSTABLE SOFTWARE |
valid |
This release of XYZ is Trustable. Trustability of the release is based on aggregation of the evidence from all of the Trustable Tenets and Trustable Assertions. The algorithm for aggregation may involve weighting of specific Tenets or Assertions based on project priorities or experience. |
|
Analyse Component Architecture |
valid |
| The FMEA and DFA for the component is executed. |
|
Analyse Feature Architecture |
valid |
| The FMEA and DFA for the feature is executed. |
|
Analyze Platform Feature Architecture |
valid |
| With a platform DFA the potential common usage of features shall be analysed. It shall be used as an input for all other DFA's. | There will be only one platform DFA. |
|
Analyse Component Architecture |
valid |
| The Security Analysis for the component is executed. |
|
Analyse Feature Architecture |
valid |
| The Security Analysis for the feature is executed. |
|
Analyze Platform |
valid |
| With a platform Security Analysis the potential attack surfaces of features shall | be analyzed. It shall be used as an input for all other analysis. | There will be only one platform Security Analysis. |
|
Analyze Change Request |
valid |
The Change Request is analyzed. Until the template is not filled out properly, the Change Request may be set back to “open” from the :need:`Architecture Team <rl__architecture_community>`. If the Change Request shall be implemented, the Change Request status is set to "in implementation", otherwise to "rejected". If the author, :need:`Contributor <rl__contributor>`. of the Change Request decides to cancel it, thus the status is set to "rejected" too. |
|
Close Change Request |
valid |
The Change Request is closed. Depending on the type of the Change Request (Feature or Component), different Teams are responsibly for this workflow. For feature: The Change Request is closed only, if the implementation is sufficient. That is verified by the :need:`Platform Team <rl__platform_team>` finally, especially the selected codeowners of the team must approve. For component: The Change Request is closed only, if the implementation is sufficient. That is verified by the :need:`Delivery Team <rl__delivery_team>` finally, especially the selected codeowners of the team must approve. Otherwise the responsible teams keeps the status "in implementation". |
|
Create Change Request |
valid |
The Change Request is created. For creating the Change Request template must be used. Check here for the available templates: :need:`doc_getstrt__change_process` or :need:`gd_guidl__change_change_request`. To start the analysis the Change Request status is changed to "in review" |
|
Implement and Monitor Change Request |
valid |
The Change Request is implemented and monitored. This may require additional activities, including creating ISSUEs and PRs. These are linked to the Change Request and monitored until closure. The Change Request is done, if all linked activities has been closed and confirmed, which is done by the approval of the selected codeowners. Depending on the type of the Change Request (Feature or Component), different Teams are responsibly for this workflow. For feature: Before closing the Change Request, :need:`Platform Team <rl__platform_team>` must check the correctness. For component: Before closing the Change Request, :need:`Delivery Team <rl__delivery_team>` must check the correctness. The responsible team may still reject it, thus the status is set to "rejected". |
|
Consult and Execute Quality Trainings |
valid |
| The :need:`rl__quality_manager` consults all project/platform stakeholder as defined in :need:`doc_concept__quality_process` for quality topics and executes regularly quality trainings. |
|
Consult and Execute Security Trainings |
valid |
| The Security Manager :need:`rl__security_manager` consults all project/platform stakeholder as defined in :need:`doc_concept__security_management_process` for security topics and executes regularly security trainings. |
|
Create Component Classification |
valid |
| The Safety Manager shall approve the OSS component classification performed by an expert on this component. |
|
Create/Maintain Components architecture |
valid |
The component architectures are created and maintained. |
|
Create/Maintain Feature architecture |
valid |
The feature architectures are created and maintained. This is supported by the architecture community (which for example has to approve logical interfaces on feature level). |
|
Create/Maintain Platform architecture |
valid |
The platform architecture is created and maintained. |
|
Create/Maintain Process Management Strategy |
valid |
The process management strategy is created and maintained. Policies are reviewed and updated, if required. |
|
Create/Maintain Quality Management Plan |
valid |
| The Quality Management Plan is created and maintained by the :need:`rl__quality_manager`. |
|
Create/Maintain Safety Manual |
valid |
| The Safety Engineer collects the necessary input for the safety manuals on platform and module level and documents it. | The safety manager makes sure all items are in valid state for a release of the safety manual. | Also for the safety manual a template exists as a guidance. |
|
Create/Maintain Safety Package |
valid |
| The Safety Manager in the project is NOT responsible to provide the argument for the achievement of functional safety. | But the Safety Manager creates and maintains the safety package in the sense of a collection of safety related work products. | The generation and the maintenance of this draft safety package shall be automated as much as possible. | It does not contain the final argumentation of the safety of the product. | As the safety package is only a collection of work products, the safety plan (template) can be used for documentation. |
|
Create/Maintain Safety Plan |
valid |
| The Safety Manager is responsible for the planning and coordination of the safety activities for the platform. | The Safety Manager creates and maintains the safety plan. | For this a template exists to guide the creator of the safety plan. |
|
Create/Maintain Security Manual |
valid |
| The Security Engineer collects the necessary input for the Security Manuals on | platform and module level and documents it. | He makes sure all items are in valid state for a release of the Security Manual. | Also for the Security Manual a template exists as a guidance. |
|
Create/Maintain Security Package |
valid |
| The Security Manager is NOT responsible to provide the argument for the achievement of security. | But the Security Manager creates and maintains the Security Package in the sense of a collection of security related work products. | The generation and the maintenance of this draft Security Package shall be automated as much as possible. | It does not contain the final argumentation of the security of the product. | As the Security Package is only a collection of work products, the Security Plan (template) can be used for documentation. |
|
Create/Maintain Security Plan |
valid |
| The Security Manager is responsible for the planning and coordination of the security activities for the platform/module. | The Security Manager creates and maintains the Security Plan. | For this a template exists to guide the creator of the Security Plan. |
|
Create/Maintain SBOM |
valid |
| The Committer is responsible to create and the maintain the SBOM for the platform/module. | The Committer makes sure all components and dependencies are identified and made available. |
|
Define/Approve Process Description |
valid |
The process description is defined and approved. |
|
Execute Feature Contribution Conformance Checks |
valid |
| The conformance of the feature contribution is checked. The conformance check consists of | * No open issues that are related to quality that are relevant for the feature contribution | * All required work products are provided and fulfill the quality criteria as defined in :need:`wp__qms_plan` | * Work product reviews are performed and passed for all required work products as defined in :need:`wp__qms_plan` | * Quality management plan is up to date and reflects the current state of the :need:`wp__platform_mgmt` |
|
Execute Platform Process Audit |
valid |
| The project/platform processes are audited. |
|
Execute Work Product Reviews |
valid |
| The quality of the work products is assured. |
|
Impact Analysis of Change Request |
valid |
| In accordance with ISO 26262-2:2018 section 5.2.2.3 d/e (Impact Analysis), the project implements a dedicated workflow for analyzing change requests. | The Safety Manager is responsible for ensuring that each change request is analyzed for its impact on safety, as required by ISO 26262-2:2018. | Impact analysis is performed at the element level (e.g., module or component) rather than the item (system) level, reflecting the modular architecture of the platform. This tailoring is documented in the safety plan and justified by the project structure and scope. | The analysis includes: | - Reviewing the change request and its context | - Assessing the impact on affected elements, safety requirements, and work products | - Documenting the rationale for decisions regarding acceptance, implementation, or rejection of the change | The outcome is a change impact analysis report and a documented decision, which are reviewed and approved as part of the Safety Management process. |
|
Monitor/Improve Process Implementation |
valid |
The process strategy and description implementation is monitored and improvements are triggered, if required. |
|
Monitor/Verify Requirements |
valid |
The requirements are monitored and verified. The inspection shall be implemented as integral part of the review in version management tool. |
|
Monitor/Improve Quality Activities |
valid |
| The :need:`rl__quality_manager` is responsible for the monitoring of the activities against the quality management plan. | The :need:`rl__quality_manager` is responsible to adjust the quality management plan, if deviations are detected. |
|
Monitor FMEA and DFA |
valid |
| The FMEA and DFA are monitored. |
|
Monitor Security Analysis |
valid |
| The Security Analyses are monitored. |
|
Monitor/Verify Architecture |
valid |
The architecture designs are monitored and verified. The inspection shall be implemented as integral part of the review in GitHub. |
|
Monitor/Verify Safety |
valid |
| The Safety Manager is responsible for the monitoring of the safety activities against the safety plan. | The Safety Manager is responsible to verify, that the preconditions for the release, which are part of the release notes, are fulfilled. | The Safety Manager is responsible to verify the correctness, completeness and consistency of the release notes. |
|
Monitor/Verify Security |
valid |
| The Security Manager is responsible for the monitoring of the security activities against the Security Plan. | The Security Manager is responsible to verify, that the preconditions for the "release for production", which are part of the release notes, are fulfilled. | The Security Manager is responsible to verify the correctness, completeness and consistency of the release notes. | The Security Manager is responsible for the monitoring of security information as defined in the Security Plan. | The Security Manager is responsible to identify weaknesses and vulnerabilities based on received information, and to analyse and manage the vulnerabilities until closure. | Beside reporting vulnerabilities in the :need:`wp__issue_track_system`, also `Eclipse general vulnerability tracker <https://gitlab.eclipse.org/security>`_ may be used. |
|
Perform Formal Reviews |
valid |
| An "external" safety manager is responsible the formal reviews on safety plan, safety package and safety analysis. | In this context "external" means that the person is not the Safety Manager of the platform/module (i.e. created or approved the respective work product). | The Safety Manager (of the platform/module) shall support the "external" safety manager during the reviews. | The Safety Manager (of the platform/module) shall approve the formal reviews. | Checklists exist to guide the creator of the relevant safety documents. |
|
Perform Formal Security Reviews |
valid |
| The external auditor is responsible to perform the formal reviews on Security plan and Security Analysis. | The Security Manager shall support the external auditor during the reviews. | The Project Lead and and the Security Manager shall approve the formal reviews. | Therefore a checklists exist to guide the creator of the relevant security documents. | | This is currently tailored out (needs discussion). |
|
Perform Safety Audit |
valid |
| The external auditor is responsible to perform a safety audit. | The Safety Manager and the process community shall support the external auditor during this. | The Project Manager and and the Safety Manager shall approve the audit report. |
|
Perform Security Audit |
valid |
| The external auditor is responsible to perform a security audit. | The Security Manager and the process community shall support the external auditor during this. | The Project Manager and and the Security Manager shall approve the audit report. | | This is currently tailored out (needs discussion). |
|
Create/Maintain Platform Management Plan |
valid |
The Platform Management Plan shall include the plans as defined by the :ref:`Platform Management Plan Template <platform_management_templates>`. The project management plan should contain the scope of work, project life cycle, work packages, planning and monitoring approaches, project schedule, escalation and communication path. |
|
Monitor/Improve Platform Management Plan |
valid |
The :need:`Project Lead <rl__project_lead>` is responsible for the monitoring and reporting of the work products and activities against the platform management plan. The :need:`Project Lead <rl__project_lead>` is responsible to adjust the plan, if deviations are detected. |
|
Analyze Problem Report |
valid |
The Problem Report is analyzed. Until the template is not filled out properly, the Problem may be set back to “open” from the :need:`Committer <rl__committer>`. If the Problem shall be resolved, the Problem status is set to "in implementation", otherwise to "rejected". The author of the Problem Report may cancel it, thus the status is set to "rejected". |
|
Close Problem Resolution |
valid |
The Problem Resolution is closed. The Problem Resolution is closed only, if the implementation is sufficient. That is verified by the :need:`Committer <rl__committer>` finally. Otherwise the :need:`Committer <rl__committer>` keeps the status "in implementation". |
|
Create Problem Report |
valid |
The Problem Report is created. For creating the Problem Report template must be used. To start the review and the analysis the Problem status is changed to "in review" |
|
Initiate and Monitor Problem Resolution |
valid |
The Problem Resolution is implemented and monitored. This may require Change Requests or other activities, including creating ISSUEs and PRs. These are linked to the Problem and monitored until closure. The Problem Resolution is done, if all linked activities has been closed and confirmed. Before closing the Problem Resolution, :need:`Committer <rl__committer>` must check the correctness. The :need:`Committer <rl__committer>` may still reject it, thus the status is set to "rejected". The author of the Problem Report may cancel it, thus the status is set to "rejected". |
|
Create/Maintain Module Release Note |
valid |
The module release note is created for each release by the committer acting as the module lead. It may be updated later in case of bugs are found after the release is published. |
|
Plan Module Release |
valid |
The module release plan is created as part of the modules planning and documented as part of the module's project planning. |
|
Plan Platform Release |
valid |
The platform release plan is created as part of the project planning and documented in the platform management plan. |
|
Create/Maintain Platform Handbook |
valid |
The platform handbook is prepared and approved by the project lead circle. It may be updated later in case of bugs found after the release is published. |
|
Create/Maintain Platform Release Note |
valid |
The platform release note is prepared and approved by the project lead circle. It may be updated later in case of bugs found after the release is published. |
|
Create/Maintain Component AoUs |
valid |
Based on the safety concept on component level, component AoUs can be derived. See also :ref:`aou_workflow` |
|
Create/Maintain Component requirements |
valid |
||
Create/Maintain Feature AoUs |
valid |
Based on the safety concept on feature level, feature AoUs can be derived. See also :ref:`aou_workflow` |
|
Create/Maintain Feature requirements |
valid |
Depending on the stakeholder requirements feature requirements can be derived. This can be done by any contributor and will be approved by the architecture community. If needed safety and security managers can provide support. In case of modification of already implemented feature requirements the feature delivery team will be consulted. |
|
Create/Maintain Stakeholder requirements and SW-Platform AoU |
valid |
Stakeholder requirements and SW-Platform Assumptions of Use (AoU) can be created during a change request. Any contributor can create a stakeholder requirement (or AoU) and propose it for approval. |
|
Create/Maintain Tool Requirements |
valid |
Based on the process descriptions (which comply to standards) and/or stakeholder/feature/component requirements tool requirements are derived. |
|
Create/Maintain Implementation |
valid |
The implementation is created, consisting of - Detailed Design - Unit - Interface |
|
Create/Maintain Software Development Plan |
valid |
The Software Development Plan shall describe - Design and programming language selection - Guidelines for design and coding - Development tools |
|
Verify Implementation |
valid |
The Implementation Verification of the Detailed Design and Code consists of the following topics - Detailed Design and Code Inspection - Static and Dynamic Code Analysis performed by a tool. Acceptance criteria are defined in the Verification Plan :need:`gd_temp__verification_plan`. |
|
Approve Tool Verification Report |
valid |
Finally the Tool Verification Report is verified and approved, and thus the status is set to released. If the verification is not successful or due to any other reason, e.g. the tool is not needed any more as planned, the tool verification may also rejected at this point. |
|
Create Tool Verification Report |
valid |
The Tool Verification Report is created during identification of a tool in status draft. For creating the Tool Verification Report the content of the linked template shall be used. |
|
Evaluate Tool and Update Tool Verification Report |
valid |
Each identified tool is evaluated. During evaluation the Tool Verification Report is updated accordingly. Stakeholder, feature, component, and process/tool requirements may be used as input for the tool evaluation and verification. After successful evaluation the status of the Tool Verification Report is set to evaluated. The successful evaluation shall contain a statement, if the tool shall be qualified or not. If tool qualification is not needed, the next step is :need:`wf__tool_approve_tool_verification_report` otherwise continue with :need:`wf__tool_qualify_tool`. |
|
Qualify Tool and Update Tool Verification Report |
valid |
The identified tool is qualified, if applicable. During qualification the Tool Verification Report is updated accordingly. After successful qualification the status of the Tool Verification Report is set to qualified. |
|
Create/Maintain Component Integration Test |
valid |
Component Integration test cases are based on component architecture and component requirements. They also cover the detailed design and integration of units forming a component. The integration testing of component architecture is optional in case a component is standalone and has no (sub-)components. Any contributor can create a component integration test and create a PR for it. During the review process the test cases will be approved by a committer. Committer and contributor need to differ. The tests are automatically executed as part of the CI after PR merge. In case of changes at inputs, the workflow need to be executed again as part of maintenance. |
|
Create/Maintain Feature Integration Test |
valid |
Feature Integration test cases are based on feature requirements and architecture of a specific feature. Any contributor can create a feature integration test and create a PR for it. During the review process the test cases will be approved by a committer. Committer and contributor need to differ. The tests are automatically executed as part of the CI after PR merge. In case of changes at inputs, the workflow need to be executed again as part of maintenance. |
|
Create Module Verification Report |
valid |
The verification report is created and maintained by a :need:`rl__committer`. It is based on the :need:`wp__verification_plan` and covers all the components of a developed module. This includes their requirements, AoUs, Architecture, Detailed Design, Units, DFA, Safety Analyses, Unit Code coverage. The respective necessary test methods and rigor of their application is defined in the :need:`wp__verification_plan`. In case of externally provided pre-existing software maintained outside of the project, the Module Verification Report also applies as documentation for the Qualification Verification Report. The respective component(s) are verified with the same methods and deviation techniques as mentioned in the :need:`wp__verification_plan`. The report will be filled by the :need:`rl__committer` responsible for integration of the external component and will get support by the :need:`rl__contributor` who proposed the component to the added to the project scope. The report is valid for ONE version of a module. |
|
Create Verification Plan |
valid |
The verification plan is created by :need:`rl__committer`. It clearly outlines all aspects of the verification activities, provide a roadmap for the verification efforts throughout the software development lifecycle. The plan should be dynamic and updated as needed throughout the project lifecycle by :need:`wf__verification_plan_maintain`. |
|
Maintain Verification Plan |
valid |
The verification plan is maintained by :need:`rl__committer`. The plan should be dynamic and updated as needed throughout the project lifecycle, as verification activities may be impacted, by new requirements, architectural decisions, introduction of tools. Note that during the initial creation of the verification plan in :need:`wf__verification_plan` not every input down to component level may be available. |
|
Create/Maintain Platform Integration Test |
valid |
Platform Integration Test cases are based on Stakeholder requirements. This is the highest test level. Any contributor can create a platform integration test and create a PR for it. During the review process the test cases will be approved by a committer. Committer and contributor need to differ. The tests are automatically executed as part of the CI after PR merge. In case of changes at inputs, the workflow need to be executed again as part of maintenance. |
|
Create Platform Verification Report |
valid |
The verification report is created and maintained by a :need:`rl__committer`. It is based on the :need:`wp__verification_plan` and covers all the selected features of a SW platform. This includes their requirements, AoUs, Architecture, DFA, Safety Analyses, The respective necessary test methods and rigor of their application is defined in the :need:`wp__verification_plan` and :need:`wp__platform_mgmt`. The report is valid for ONE specific platform version baseline. |
|
Set Requirement Test Coverage |
valid |
The requirement attribute `complete test coverage` is set to `yes` by a :need:`rl__committer` when it is verified that the requirement is fully covered by test cases. This means the linked test cases in sum fully satisfy the requirement. A fully covered requirement will be set to `no` automatically via the implementation of the check described in :need:`gd_req__req_suspicious`. A recheck is needed in this case to get the status back to `yes`. |
|
Create/Perform Unit Test |
valid |
Every Unit shall have at least one Unit Test. They verify the detailed design of the implementation. Unit tests are automatically executed as part of the CI after PR merge. In case of changes at inputs, the workflow need to be executed again as part of maintenance. Any contributor can create a component test and create a PR for it. During the review process the test cases will be approved by a committer. Committer and contributor need to differ. The actual :need:`rl__committer` of the implementation can also be the creator of the unit tests. Independence is achieved by different approver at PRs and by the :need:`wp__verification_module_ver_report`. The typical steps when creating a unit tests are: #. Check the detailed design of the component. Create a test for every interface of the unit showing at least every flow in dynamic diagrams. #. Only in the exceptional case where a single unit fully realises a component requirement, follow the detailed design to that component requirement and cover it with the unit test. #. Fill in the test attributes based on the previous steps and provide a description. #. Link the test against the detailed design and, in the exceptional case above, the component requirement. |
|
Verify/Approve Module Release |
valid |
| The module release is verified and approved. |
|
Verify/Approve Platform Release |
valid |
| The project/platform release is verified and approved. |
|
Verify FMEA and DFA |
valid |
| The FMEA and DFA are verified. The verification criteria is that it can be proven that the safety requirements for functions and the corresponding safety monitoring are not violated. |
|
Verify Security Analysis |
valid |
| The Security Analyses are verified. The verification criteria is that it can be | proven that the security requirements for functions and the corresponding security | monitoring are not violated. |
|
Process Safety Audit Report |
valid |
Examination of an implemented process with regard to the process objectives and that those match the ISO 26262. |
|
Process Security Audit Report |
valid |
Examination of an implemented process with regard to the process objectives and that those match the ISO SAE 21434. (Currently tailored out, needs discussion) |
|
Platform Change Management Plan |
valid |
Change Management Plan (Part of the Platform Management Plan) |
|
Component Request |
valid |
| - Component request for a new component or a component modification | | Change Request for a new component or a modification of an existing component, | which changes the scope of the component. | This includes the allocation of components into SW Modules. |
|
Component Architecture |
valid |
Component Architecture linked to Component Requirements * Static view (Sphinx Needs) - Component interfaces (to outside of component) and interfaces between own (internal) components * Dynamic view (UML) - Sequences of components interactions and components states * Interface view (Sphinx Needs) - Overview of used and provided interfaces Technical concept on component level. |
|
Platform Configuration Management Plan |
valid |
Config Management Plan (Part of the Platform Management Plan, :need:`wp__platform_mgmt`) |
|
Documentation Management Plan |
valid |
Document Management Plan (Part of the Platform Management Plan) Defines the documentation strategy which covers the following aspects: * How to review, update, (re-)approve documentations? * What is the status of a document and how are changes identified? * How to ensure the availability of the documents over time? * How to ensure the controlled distribution of documents? * How to avoid distribution of obsolete documents? * Which formal elements are used? * List of all documents including their status |
|
Formal Document Review Reports |
valid |
Review that a work product provides sufficient and convincing evidence of their contribution to the achievement of functional safety considering the corresponding objectives and requirements of ISO 26262. Will contain formal review report for Safety Plan, Safety Package, Safety Analyses and DFA |
|
Formal Document Review Reports |
valid |
Review that a work product provides sufficient and convincing evidence of their contribution to the achievement of security considering the corresponding objectives and requirements of ISO SAE 21434. Will contain formal review report for Security Plan, Security Package and Security Analyses. For the different review checklist see here: - Review checklist for Security Plans: :need:`doc__platform_name_security_plan_fdr` and `Module Security Plan Formal Review Checklist <https://eclipse-score.github.io/module_template/main/module/security_mgt/module_security_plan_fdr.html>`__ - Review checklist for Security Packages: :need:`doc__platform_name_security_package_fdr` and `Module Security Package Formal Review Checklist <https://eclipse-score.github.io/module_template/main/module/security_mgt/module_security_package_fdr.html>`__ |
|
Feature Request |
valid |
| - Feature request for a new feature or a feature modification | | Change Request for a new feature or a modification of an existing feature, | which changes the scope of the feature. |
|
Feature Architecture |
valid |
Feature Architecture linked to Feature Requirements, i.e. interaction of components * Static view (Sphinx Needs) - Feature interfaces (to outside of Feature) and interfaces between own components * Dynamic view (UML) - Sequences of component interactions and state diagrams * Interface view (Sphinx Needs) - Overview of used and provided interfaces Technical concept on platform or feature level. |
|
Feature DFA |
valid |
Dependent Failure Analysis on feature level. Detections, preventions, mitigations linked to Software Feature Requirements or Feature Assumptions of Use. Perform analysis on interactions between safety related and non-safety related components or components with different ASIL of one feature. |
|
Feature FMEA |
valid |
FMEA verifies the feature architecture (as part of SW Safety Concept) Detections, preventions, mitigations linked to Software Feature Requirements or Feature Assumptions of Use. |
|
Feature Security Analysis |
valid |
Bottom-Up Security Analysis with defined methods, verifies the feature architecture (as part of SW Security Concept) - Mitigations linked to Software Feature Requirements or Assumptions of Use Analyze the attack surfaces between modules/components that references all feature static architecture diagrams, highlighting potential shared attack vectors. Mitigations linked to Software Feature Security Requirements or Feature Assumptions of Use Perform analysis on interactions between security-relevant and non-security-relevant modules/components or modules/components with different security levels of one feature. |
|
Issue tracking system |
valid |
| - Change request | - Change request plan | - Change report | - Identified safety anomaly report | - Impact analysis | | Safety anomaly: Conditions that deviate from expectations and that can lead to harm. | The documentation of a change request shall contain the list of changed work products, | the details of the change and the planned date of deployment of the change. | In case a anomaly cannot be closed it shall be escalated to the :need:`Project Lead <rl__project_lead>`. |
|
Module Safety Manual |
valid |
The safety manual describes: * The Assumed Platform Requirements (Safety related); * the safety concept of the SEooC (i.e. which faults are taken care of); * the Assumptions of Use (of the modules's components and of the associated feature); * a link to the platform safety manual (containing the general AoUs every user has to obey additionally); * a link to the (module) user manual; * the reactions of the implemented functions under anomalous operating conditions; and * a description of known anomalies with corresponding workaround measures. This is on module level. One manual per each module. |
|
Module Safety Package |
valid |
Compiled Safety Relevant Work Products. For Module SEooC. Note that the module safety package does not contain an argument that the module is safe. |
|
Module Safety Plan |
valid |
Plan to manage and guide the execution of the safety activities of a project including dates, milestones, tasks, deliverables, responsibilities (including the Safety Manager appointment) and resources. Guidelines on how an impact analysis shall be concluded on each item or element involved together with it's connected items or elements. This is on following level: * Module (contains activities planning based on a Change Request) |
|
Module Security Manual |
valid |
The Security Manual describes: * the assumed platform requirements (security related, including for post-development); * the security concept of the OoC (i.e. which attack paths are taken care of); * the assumptions of use (of the modules's components); * a link to the user manual; * the reactions of the implemented functions under threatened operating conditions; and * a description of known vulnerabilities with corresponding workaround measures. This is on module level. One manual per each module. For template see here: `Module Security Manual Template <https://eclipse-score.github.io/module_template/main/module/manuals/security_manual.html>`__ |
|
Module Security Package |
valid |
Compiled security relevant work products. For Module OoC. Note that the Module Security Package does not contain an argument that the module is safe and secure. |
|
Module Security Plan |
valid |
Plan to manage and guide the execution of the security activities of a project including dates, milestones, tasks, deliverables, responsibilities (including the Security Manager appointment) and resources. Guidelines on how an impact analysis shall be concluded on each item or element involved together with it's connected items or elements. For the template see here: `Module Security Manual Template <https://eclipse-score.github.io/module_template/main/module/manuals/security_manual.html>`__ This is on following level: * Module (contains activities planning based on a Change Request) |
|
Module Release Notes |
valid |
The module release note provides clarity what is included in the current version of the software module release. It shall indicate also the distinct changes to previous versions and provide information regarding the time of the release. It includes known bugs from own testing and field reporting, with clear statement, that these bugs do not lead to violation of any safety requirements or which workaround measures need to be applied. It contains also the build configuration capable to create the SEooC Library for the reference HW, module level. Note: Embedded software in the sense of the Iso (i.e. deployed on the production HW) is not part of our delivery. |
|
Module Release Plan |
valid |
The module release plan is a strategic document that outlines the features planned for upcoming module releases along with their estimated release dates. It provides a roadmap for the development and release of new features, ensuring that all stakeholders are aligned on the module's future direction. |
|
Platform Architecture |
valid |
Platform Architecture describes the overall software structure with the belonging features, modules and their logical interfaces, i.e. top-level decomposition of the platform into features and their interactions * Static view - Overview of features, SW modules and their relationships within the platform |
|
Platform DFA |
valid |
Analyse the dependencies between features that references all platform feature static architecture diagrams, highlighting potential shared use of features. |
|
Platform Handbook |
valid |
The platform handbook is a tutorial to explain how the project works from a technical perspective. It explains the background of the project, but also what the project is not about. Further it contains: - Overview of the technologies used within the project - Software architecture overview - Module structure overview - Integration process - Getting started guide - Contribution guide |
|
Platform Management Plan |
valid |
The Platform Management Plan shall include the plans as defined by the :ref:`Platform Management Plan Template <platform_management_templates>`. The main purpose of the plan documents is to define strategies, so that * all work products can be uniquely identified and reproduced in a controlled manner at any time * relations and differences between versions can be traced * contributions are managed, analysed and controlled including changes of the work products during the project life cycle * documents are concise, clearly structured, understandable for intended users, verifiable, maintainable, and organized according to <Project> procedures to facilitate information retrieval |
|
Platform Safety Manual |
valid |
The safety manual describes: * The Assumed Platform Requirements (Safety related); * the safety concept of the SEooC (i.e. which faults are taken care of); * the Assumptions of Use (of the platform level), including AoU of external components to be fulfilled also by the user; * links to all the module safety manuals of the platform integration; * a link to the (platform) user manual; * the reactions of the implemented functions under anomalous operating conditions; and * a description of known anomalies with corresponding workaround measures. This is on platform level. Only one manual for the entire platform. |
|
Platform Safety Package |
valid |
Compiled Safety Relevant Work Products. For Platform SEooC. Note that the platform safety package does not contain an argument that the platform is safe. |
|
Platform Safety Plan |
valid |
Plan to manage and guide the execution of the safety activities of a project including dates, milestones, tasks, deliverables, responsibilities (including the Safety Manager appointment) and resources. This platform safety plan also takes into account the eclipse organization's rules relevant for safety development. Guidelines on how an change impact analysis shall be concluded on each item or element involved together with it's connected items or elements. This is on following level: * Project/Platform (contains definitions how safety planning is performed generally in the project) |
|
Platform Security Analysis |
valid |
Analyze the attack surfaces between features that references all platform feature static architecture diagrams, highlighting potential shared attack vectors. |
|
Platform Security Manual |
valid |
The Security Manual describes: * the assumed platform requirements (security related, including for post-development); * the security concept of the OoC (i.e. which attack paths are taken care of); * the assumptions of use (of the features); * a link to the user manual; * the reactions of the implemented functions under threatened operating conditions; and * a description of known vulnerabilities with corresponding workaround measures. This is on platform level. Only one manual for the entire platform. For template see here: :need:`doc__platform_security_manual` |
|
Platform Security Package |
valid |
Compiled security relevant work products. For platform OoC. Note that the Platform Security Package does not contain an argument that the platform is safe and secure. |
|
Platform Security Plan |
valid |
Plan to manage and guide the execution of the security activities of a project including dates, milestones, tasks, deliverables, responsibilities (including the Security Manager appointment) and resources. This Platform Security Plan also takes into account the eclipse organization's rules relevant for security development. Guidelines on how an change impact analysis shall be concluded on each item or element involved together with it's connected items or elements. For the template see here: :need:`doc__platform_security_manual` This is on following level: * Project/Platform (contains definitions how security planning is performed generally in the project) |
|
Platform Release Notes |
valid |
The platform release note provides clarity what is included in the current version of the platform release. The platform release note mentions all individual software modules used in the platform release and their used released versions. It shall indicate also the distinct changes to previous platform versions and provide information regarding the time of the release. It includes known bugs from own testing and field reporting, with clear statement, that these bugs do not lead to violation of any safety requirements or which workaround measures need to be applied. It contains also the build configuration capable to create the SEooC Library for the reference HW, platform level. Note: Embedded software in the sense of the Iso (i.e. deployed on the production HW) is not part of our delivery. |
|
Platform Release Plan |
valid |
The platform release plan is a high-level document that outlines which software modules will be included in the overall platform and what features can be expected within the platform. It provides a strategic overview of the platform's development, ensuring that all stakeholders are aligned on the platform's future direction and the integration of various software modules. |
|
Policies |
valid |
In general the project follows the Eclipse Foundation Development Process (EDP, `Eclipse Foundation Development Process <https://www.eclipse.org/projects/dev_process/>`_). The EDP defines important concepts, including the Open Source Rules of Engagement, the organizational framework for open source projects and teams, releases, reviews, and more. Further the Eclipse Foundation Security Policy (`Eclipse Foundation Security Policy <https://www.eclipse.org/security/policy/>`_) applies. The Eclipse Foundation Functional Safety Process (EFFSP, currently in DRAFT `Eclipse Foundation Functional Safety Process <https://gitlab.eclipse.org/eclipsefdn/emo-team/policies/functional-safety-process/-/blob/main/source/fsp.adoc?ref_type=heads>`_) applies. Concerning the use of Generative Artificial Intelligence `Usage Guidelines <https://www.eclipse.org/projects/guidelines/genai/>`_ applies. Project specific Policies for functional safety and cybersecurity may extend the ones from ECLIPSE. |
|
Platform Problem Resolution Plan |
valid |
Problem Resolution Plan (Part of the Platform Management Plan) |
|
Process Description |
valid |
The process description is defined here: :ref:`process_description`. :ref:`process_areas` as part of that representing the process definitions. |
|
Process Improvement Report |
valid |
| Process Improvement Report that describes the improvement with description of: | * Description of used methods, assumptions, involved persons, etc. | * Scope of improvement | * Estimations regarding impact and time | * Commitment on the improvement | * Description of the improvement incl. goals and prioritisation | * Measures to validate the effectiveness of the improvement |
|
Process Management Strategy |
valid |
Strategy to manage and guide execution of the process management activities. The process management strategy in general is described here: :ref:`process_description`. The process (meta) model (see :ref:`processes_introduction`) summarizes the strategy. Further strategical concepts are defined here: :ref:`process_general_concepts`. |
|
Project Management Plan |
valid |
Project Management Plan (Part of the Platform Management Plan) Plan to manage, analyse and control changes of the work products during the project life cycle. Defines the project stakeholder. Plan to communicate to the project stakeholder. Defines the schedule of the project. Defines escalation path. |
|
Quality Management Plan |
valid |
| Quality Management Plan to define the quality aspects like: | * Quality strategy | * Description of independencies and objective assurance | * Quality assurance activities | * Role definitions | * Quality Goals | * Schedules | * Quality trainings |
|
Quality report |
valid |
| The quality report summarizes the results of the quality related activities |
|
Component Requirements |
valid |
SW Requirements for components, broken down from feature requirements to the realizing component. These include configuration specification. |
|
Component Assumptions of Use |
valid |
SW Safety Requirements for the user of the component, exportable requirements for the user to integrate in their req mgt system. |
|
Feature Requirements |
valid |
Feature requirements describe in a more detailed way the functionality which will fulfill a set of stakeholder requirements. A "feature" itself represents a set of requirements. It describes the interaction of the components to form a feature. It shall also be the basis for integration testing on platform level. |
|
Feature Assumptions of Use |
valid |
SW Safety Requirements for the user of the feature, exportable requirements for the user to integrate in their req mgt system. |
|
Requirements Inspection |
valid |
Depends on requirements management tooling, expect text based requirements. Review done with inspection checklist. This checklist may be integrated in requirements/version management tooling. |
|
Process/Tool Requirements |
valid |
Process and Tool requirements describe activities needed to be executed in the development process either manually or automated (i.e. tool supported). Tool requirements (if derived from process requirements) are more detailed as those are also the basis for tool (qualification) testing. |
|
Stakeholder Requirements |
valid |
Technical requirements from a stakeholder viewpoint on SW-platform level, contain "assumed Technical Safety Requirements" in SW-Platform SEooC development. |
|
SW-Platform Assumptions of Use |
valid |
SW Safety Requirements for the user of the platform, exportable requirements for the user to integrate in their requirements management system. |
|
Tailoring Documents |
valid |
This work product argues why some safety work products are not needed in the project. It may have several levels: * Project/Platform * Feature/Component It belongs to the Safety Plan. |
|
Architecture Verification |
valid |
Depends on architecture guideline and tooling. May include several methods like inspection, modelling, ... which are selected in projects SW Verification Plan. |
|
Software component classification |
valid |
The classification shall include: * the unique identification of the pre-developed software component; * the maximum ASIL of the safety requirements allocated to it; * a development processes analysis; and * a complexity analysis of the pre-developed SW component; and * finally a SW component classification as input for the safety planning (which is to cover the determined gaps, if any, by additional verification measures). |
|
Component DFA |
valid |
Dependent Failure Analysis on component level. Detections, preventions, mitigations linked to Software Component Requirements or Assumptions of Use. Perform analysis of safety related and non-safety related sub-elements or sub-elements with different ASIL. Perform analysis on interactions between safety related and non-safety related lower level components or lower level components with different ASIL of one (higher level) component. |
|
Component FMEA |
valid |
FMEA, verifies the component architecture (as part of SW Safety Concept) Detections, preventions, mitigations linked to Software Component Requirements or Assumptions of Use. |
|
Component Security Analysis |
valid |
Bottom-Up Security Analysis with defined methods, verifies the component architecture (as part of SW Security Concept) - Mitigations linked to Software Component Requirements or Assumptions of Use Analyze the attack surfaces between sub-components, units that references all component static architecture diagrams, highlighting potential shared attack vectors. Mitigations linked to Software Component Security Requirements or Assumptions of Use Perform analysis of security-relevant and non-security-relevant sub-elements or sub-elements with different security levels. Perform analysis on interactions between security-relevant and non-security-relevant sub-components or sub-components with different security levels of one component. Including potential influences from the other components in the component's module. |
|
Software Development Plan |
valid |
Process description of SW development including: - selection of design and programming language - design guideline - coding guideline (e.g. MISRA, can also include style guide or naming convention) - SW configuration guideline - development tools |
|
Implementation |
valid |
Implementation includes source code and detailed design (e.g. in form of comments or linked graphical representations) and SW configuration (e.g. #ifdef). The "how to" is described in the SW Development Plan guidelines. |
|
Implementation Inspection |
valid |
Github review with integrated inspection checklist, only valid Detailed Design and Code get merged. |
|
Module Software Bill of Material (SBOM) |
valid |
Module Software Bill of Material - comprehensive inventory of software components to ensure security, integrity, and compliance. |
|
Platform Software Bill of Material (SBOM) |
valid |
Platform Software Bill of Material - comprehensive inventory of software components to ensure security, integrity, and compliance. |
|
Tailoring Document Work Products |
valid |
This work product "definition" links to all the work products which are not covered by the processes work products documented. Make sure these are tailored out in the safety, security and quality plans for your project (documented in the PMP), to be able to demonstrate completeness. It is not really a work product definition, but this is the best way to link to the tailored out standard work products. |
|
Platform Tool Management Plan |
valid |
Tool Management Plan (Part of the Platform Management Plan) |
|
Tool Verification Report |
valid |
According to the safety tool process, each tool's confidence level (TCL) must be determined. In addition security topics must be considered, as tool user manuals, access control for tools, proper tool documentation, guidelines, protection against malware introduction, etc. Based on TCL the appropriate qualification methods shall be applied. For the project the method **validation of software tool** must be used. |
|
Training path |
valid |
| Trainings shall give dedicated information how to apply the processes and work products in the project. |
|
Component Integration test |
valid |
Component Integration Testing verifies the component architecture and component requirements: - all interfaces from Static view and - all flows from Dynamic View - integration of units into components based on detailed design - detailed design of complex units Performance (i.e. RAM and processor usage) is only tested on reference HW. As an optional part component integration tests can cover additional testing for complex units touching specifically the detailed design. This is needed where :need:`wp__verification_sw_unit_test` is not sufficient to cover the measures defined in the implementation of the :need:`wp__verification_plan`. |
|
Feature Integration test |
valid |
Integration Testing verifies feature requirements and architecture: - all interfaces from Static view and - all flows from Dynamic View and - performance and resource consumption: i.e. RAM and processor usage on reference HW - If software partitioning (operating system processes) is used to implement freedom from interference effectiveness evidence shall be generated during integration tests. |
|
Module Verification Report |
valid |
Verification Report contains: - List of requirements (and architecture/detailed design tags) tested by which test (can be several levels), passed/failed and completeness verdict, including normal operation and failure reactions - The list of requirements may also contain other verification methods like "Analysis" - Structural Coverage (C0 and C1, from unit testing on host) per unit - Static Code Analysis (including compiler warnings, automated checking of coding guidelines and additional checks) - Formal evidence about the performed DFA - Formal evidence about the performed Safety Analyses - Software component qualification verification report - Test result per test case from :need:`wp__verification_sw_unit_test` and :need:`wp__verification_comp_int_test` with status passed/failed/not_run - Test log per test case from :need:`wp__verification_sw_unit_test` and :need:`wp__verification_comp_int_test` with status passed/failed/not_run including a link to the matching retained execution log artifacts for the specific release version It also serves as SW Component Qualification Verification Report for pre-existing Open Source Projects developed and maintained outside of the project to which this process is applied to. It is also described in the :need:`gd_temp__mod_ver_report` as a dedicated section. |
|
Verification Plan |
valid |
Verification planning for each phase of the safety lifecycle must detail the work products, objectives, methods, criteria, environments, equipment, resources, actions for anomalies, and regression strategies, considering method adequacy, complexity, prior experiences, and technology maturity or risks. This also covers the work product Verification Specification. |
|
Platform Integration Test |
valid |
Platform Integration Testing verifies Stakeholder Requirements performed on reference HW. Depending on the nature of the project, respective tailoring (e.g. for reduced requirements coverage) has to be reflected in the :need:`wp__verification_plan` and :need:`wp__platform_safety_plan`. If software partitioning (operating system processes) is used to implement freedom from interference effectiveness evidence shall be generated during integration tests. |
|
Platform Verification Report |
valid |
Verification Report contains: - List of requirements (stakeholder and feature) and architecture tested by which test (can be several levels), passed/failed and completeness verdict, including normal operation and failure reactions - The list of requirements may also contain other verification methods like "Analysis" - Formal evidence about the performed DFA - Formal evidence about the performed Safety Analyses - Test result per test case from :need:`wp__verification_platform_int_test` and :need:`wp__verification_feat_int_test` with status passed/failed/not_run - Test log per executed test case from :need:`wp__verification_platform_int_test` and :need:`wp__verification_feat_int_test` with status passed/failed including a link to the matching retained execution log artifacts for the specific release version |
|
Unit test |
valid |
Unit testing verifies detailed design (traced to). Respective tooling is defined in :need:`wp__platform_mgmt`, :need:`wp__verification_plan` and integrated in CI/Build. Unit testing is in responsible of the :need:`rl__contributor` providing the :need:`wp__sw_implementation`. The compiled tests run against the actual source code compiled for the component or reference integration and their targeted HW architecture (e.g. arm64 or x86). |