JLS-01 | Reviewed: ⨯ | Score: 0.0#
The CI pipeline in the upstream nlohmann/json repository executes the unit and integration test suites on each pull request (opened, reopened, synchronized).
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All specified tests are executed repeatedly, under defined conditions in controlled environments, according to specified objectives. |
0.00 |
✔ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/blob/develop/.github/workflows/ubuntu.yml#L9
https://github.com/nlohmann/json/blob/develop/.github/workflows/ubuntu.yml#L9
- The trigger condition for the CI workflow that executes the test suites.
Fallacies:
None
Graph:
date-time |
JLS-01 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-02 | Reviewed: ⨯ | Score: 0.0#
Fuzz testing is used in the nlohmann/json repository to uncover edge cases and failure modes throughout development.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Prohibited misbehaviours for the nlohmann/json library are identified, and mitigations are specified, verified and validated based on analysis. |
0.00 |
⨯ Item Reviewed |
|
All tests for the nlohmann/json library, and its build and test environments, are constructed from controlled/mirrored sources and are reproducible, with any exceptions documented. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://introspector.oss-fuzz.com/project-profile?project=json
https://introspector.oss-fuzz.com/project-profile?project=json
- most recent report for fuzzing introspection of nlohmann/json with historical plots
website: https://storage.googleapis.com/oss-fuzz-introspector/json/inspector-report/20250824/fuzz_report.html
https://storage.googleapis.com/oss-fuzz-introspector/json/inspector-report/20250824/fuzz_report.html
- persistent storage of fuzz-testing-report for nlohmann/json version 3.12.0 on 24.08.2025
website: https://raw.githubusercontent.com/nlohmann/json/refs/heads/develop/.github/workflows/cifuzz.yml
https://raw.githubusercontent.com/nlohmann/json/refs/heads/develop/.github/workflows/cifuzz.yml
- Configuration file for Fuzz-Testing pipeline in the original nlohmann/json repository
website: https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
- OpenSSF Scorecard Report for nlohmann/json, where score for ‘Fuzzing’ supports this statement.
Fallacies:
None
Graph:
date-time |
JLS-02 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-03 | Reviewed: ⨯ | Score: 0.0#
Automated tests within the TSF documentation are reviewed by a Subject Matter Expert to verify they test the properties they claim to.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Expected or required behaviours for the nlohmann/json library are identified, specified, verified and validated based on analysis. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
None
Fallacies:
None
Graph:
date-time |
JLS-03 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-04 | Reviewed: ✔ | Score: 1.0#
External dependencies within nlohmann/json are checked for potential security vulnerabilities with each pull request to main. Merging is blocked until all warnings are resolved.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
In the nlohmann/json repository, known bugs or misbehaviours are analysed and triaged, and critical fixes or mitigations are implemented or applied. |
0.38 |
✔ Item Reviewed |
|
All inputs to the nlohmann/json library are assessed, to identify potential risks and issues. |
0.32 |
⨯ Item Reviewed |
Supporting Items:
None
References:
.github/workflows/dependency-review.yml
# Dependency Review Action
#
# This Action will scan dependency manifest files that change as part of a Pull Request,
# surfacing known-vulnerable versions of the packages declared or updated in the PR.
# Once installed, if the workflow run is marked as required,
# PRs introducing known-vulnerable packages will be blocked from merging.
#
# Source repository: https://github.com/actions/dependency-review-action
name: 'Dependency Review'
on:
workflow_call:
inputs:
artifact_id:
description: 'Unique identifier for artifacts'
required: true
type: string
permissions:
contents: read
jobs:
dependency-review:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@c6295a65d1254861815972266d5933fd6e532bdf # v2.11.1
with:
egress-policy: audit
- name: 'Checkout Repository'
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: 'Dependency Review'
uses: actions/dependency-review-action@ce3cf9537a52e8119d91fd484ab5b8a807627bf8 # v4.6.0
- name: Generate dependency_review artifact
run: |
echo "Generating Dependency Review artifact..."
mkdir -p dependency_review
echo "dependency review processed for ${{ inputs.artifact_id }}" > dependency_review/dependency_review.txt
- name: Upload dependency_review artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ${{ inputs.artifact_id }}
path: dependency_review/
Description: The workflow scans PRs for dependency changes and vulnerabilities.
Fallacies:
None
Graph:
date-time |
JLS-04 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
1.00 |
2025-12-16 14:03:34.403563 |
1.00 |
JLS-05 | Reviewed: ✔ | Score: 0.875#
The nlohmann/json library is widely used and actively maintained; bugs and misbehaviours are tracked publicly and transparently.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
In the nlohmann/json repository, known bugs or misbehaviours are analysed and triaged, and critical fixes or mitigations are implemented or applied. |
0.38 |
✔ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/issues
https://github.com/nlohmann/json/issues
- contains the collected github-issues for nlohmann/json
website: https://github.com/nlohmann/json/graphs/commit-activity
https://github.com/nlohmann/json/graphs/commit-activity
- presents the commit activity of the past year
website: https://github.com/nlohmann/json/graphs/contributors
https://github.com/nlohmann/json/graphs/contributors
- presents commits over time and per contributor
website: https://github.com/nlohmann/json/forks?include=active&page=1&period=&sort_by=last_updated
https://github.com/nlohmann/json/forks?include=active&page=1&period=&sort_by=last_updated
- lists all forks of nlohmann/json by last updated
website: https://github.com/nlohmann/json/pulse
https://github.com/nlohmann/json/pulse
- presents activity over the past week
website: https://github.com/orgs/score-json/discussions/27#discussion-8594385
https://github.com/orgs/score-json/discussions/27#discussion-8594385
- comparison between JSON libraries demonstrating the popularity of nlohmann/json
website: https://json.nlohmann.me/home/customers/
https://json.nlohmann.me/home/customers/
- presents a list of a subset of all customers who are using the nlohmann/json library
website: https://github.com/nlohmann/json/releases/tag/v3.12.0
https://github.com/nlohmann/json/releases/tag/v3.12.0
- release notes for v3.12.0, listing bugs, CVEs and warnings which were either fixed or mitigated since last release
Fallacies:
None
Graph:
date-time |
JLS-05 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.88 |
2025-12-16 14:03:34.403563 |
0.88 |
JLS-06 | Reviewed: ⨯ | Score: 0.0#
Pull requests in the nlohmann/json repository are merged only after code review.
aschemmel-tech: taken over score from “OpenSSF Scorecard Report”
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
nlohmann/json library components, configurations and tools are updated under specified change and configuration management controls. |
0.65 |
✔ Item Reviewed |
Supporting Items:
None
References:
website: https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
- OpenSSF Scorecard Report for nlohmann/json, where score for ‘Code-Review’ reflects this statement.
Fallacies:
None
Graph:
date-time |
JLS-06 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.35 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-07 | Reviewed: ✔ | Score: 1.0#
The develop branch of nlohmann/json is protected, i.e. no direct commits are possible.
aschemmel-tech: no other pushes as from nlohman are observable
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
nlohmann/json library components, configurations and tools are updated under specified change and configuration management controls. |
0.65 |
✔ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/actions?query=event%3Apush+branch%3Adevelop
https://github.com/nlohmann/json/actions?query=event%3Apush+branch%3Adevelop
- List of all pushes to the develop branch in nlohmann/json, showing that all commits are done by nlohmann and indicating that direct commits are not possible.
Fallacies:
None
Graph:
date-time |
JLS-07 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
1.00 |
2025-12-16 14:03:34.403563 |
1.00 |
JLS-08 | Reviewed: ⨯ | Score: 0.0#
Each leaf node in the Trustable Graph that is not an Assumption-of-Use (AoU) is scored either based on SME review(s) alone or on a combination of SME review(s) and an automatic validator.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Confidence in the nlohmann/json library is measured based on results of analysis. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://eclipse-score.github.io/inc_nlohmann_json/main/generated/dashboard.html#summary
https://eclipse-score.github.io/inc_nlohmann_json/main/generated/dashboard.html#summary
- Dashboard showing distributions of evidence scores and SME (subject-matter expert) scores.
website: https://eclipse-score.github.io/inc_nlohmann_json/main/generated/trustable_report_for_Software.html
https://eclipse-score.github.io/inc_nlohmann_json/main/generated/trustable_report_for_Software.html
- Trustable Compliance Report showing scores for statements.
website: https://codethinklabs.gitlab.io/trustable/trustable/methodology.html#documenting-assumptions
https://codethinklabs.gitlab.io/trustable/trustable/methodology.html#documenting-assumptions
- Definition of Assumptions as part of the methodology
Fallacies:
None
Graph:
date-time |
JLS-08 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-09 | Reviewed: ⨯ | Score: 0.0#
Scores within the TSF documentation are reasonably, systematically and repeatably accumulated.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Confidence in the nlohmann/json library is measured based on results of analysis. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://eclipse-score.github.io/inc_nlohmann_json/main/concept.html
https://eclipse-score.github.io/inc_nlohmann_json/main/concept.html
- Description of the algorithm on how scores are accumulated.
Fallacies:
None
Graph:
date-time |
JLS-09 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-10 | Reviewed: ⨯ | Score: 0.0#
Every release of the nlohmann/json library includes source code, build instructions, test code and test results summaries.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All constructed iterations of the nlohmann/json library include source code, build instructions, tests, results and attestations. |
0.00 |
⨯ Item Reviewed |
|
Construction of releases for the nlohmann/json library is fully repeatable and the results are fully reproducible, with any exceptions documented and justified. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/releases
https://github.com/nlohmann/json/releases
- List of nlohmann/json releases consisting of source code, build instructions, test code and test result summaries.
Fallacies:
None
Graph:
date-time |
JLS-10 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-11 | Reviewed: ✔ | Score: 0.0#
Outstanding bugs or misbehaviours are analyzed within eclipse-score/inc_nlohmann_json to determine whether they are relevant for S-CORE’s use cases of the nlohmann/json library.
aschemmel-tech: most but not all issues are analyzed for impact on S-CORE.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
In the nlohmann/json repository, known bugs or misbehaviours are analysed and triaged, and critical fixes or mitigations are implemented or applied. |
0.38 |
✔ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/eclipse-score/inc_nlohmann_json/blob/save_historical_data/TSF/misbehaviours.md
https://github.com/eclipse-score/inc_nlohmann_json/blob/save_historical_data/TSF/misbehaviours.md
- List of outstanding bugs as well as fixes for developed code that are outstanding, not yet applied.
Fallacies:
None
Graph:
date-time |
JLS-11 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-12 | Reviewed: ✔ | Score: 1.0#
The nlohmann/json repository has well-defined community standards, including a contribution guideline and a security policy.
aschemmel-tech: all community standard are covered.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
nlohmann/json library components, configurations and tools are updated under specified change and configuration management controls. |
0.65 |
✔ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/community
https://github.com/nlohmann/json/community
- List of defined community standards in nlohmann/json
Fallacies:
None
Graph:
date-time |
JLS-12 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
1.00 |
2025-12-16 14:03:34.403563 |
1.00 |
JLS-13 | Reviewed: ⨯ | Score: 0.0#
For changes that affect the behaviour or public API of the nlohmann/json library, contributors manually update the library documentation and locally rebuild it for verification.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Manual methodologies applied for the nlohmann/json library by contributors, and their results, are managed according to specified objectives. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://json.nlohmann.me/community/contribution_guidelines/#update-the-documentation
https://json.nlohmann.me/community/contribution_guidelines/#update-the-documentation
- Contribution guidelines describing how to update and locally build the mkdocs-based documentation
website: https://github.com/nlohmann/json/releases
https://github.com/nlohmann/json/releases
- Release notes summarising behavioural changes and documentation updates for each version
website: https://json.nlohmann.me
https://json.nlohmann.me
- Published documentation site for the nlohmann/json library
Fallacies:
None
Graph:
date-time |
JLS-13 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-14 | Reviewed: ⨯ | Score: 0.0#
The SHA value of the nlohmann/json library in use within eclipse-score/inc_nlohmann_json coincides with the SHA value provided by Niels Lohmann for that version.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Given the same version of the repository, configuration, and documented environment, the build process shall produce identical single-include header files. |
0.00 |
⨯ Item Reviewed |
|
Construction of releases for the nlohmann/json library is fully repeatable and the results are fully reproducible, with any exceptions documented and justified. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/tree/v3.12.0
https://github.com/nlohmann/json/tree/v3.12.0
- release site of nlohmann/json containing the sha values
Fallacies:
None
Graph:
date-time |
JLS-14 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-16 | Reviewed: ⨯ | Score: 0.0#
A list of tests, which is extracted from the test execution, is provided, along with a list of test environments.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All tests for the nlohmann/json library, and its build and test environments, are constructed from controlled/mirrored sources and are reproducible, with any exceptions documented. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
TSF/docs/list_of_test_environments.md
## List of all unit-tests with test environments
This list contains all unit-tests possibly running in this project.
These tests are compiled from the source-code, where the individual unit-tests are arranged in TEST_CASEs containing possibly nested SECTIONs.
To reflect the structure of the nested sections, nested lists are utilised, where the top-level list represents the list of TEST_CASEs.
It should be noted that not all unit-tests in a test-file are executed with every compiler-configuration.
### List of tests in file unit-32bit.cpp
* value_in_range_of trait
* 32bit
* BJData
* parse errors
* array
* optimized array: negative size
* optimized array: integer value overflow
All tests in this file were run in the following configurations:
* Linux-g++ with standard gnu++11
### List of tests in file unit-algorithms.cpp
* algorithms
* non-modifying sequence operations
* std::all_of
* std::any_of
* std::none_of
* std::for_each
* reading
* writing
* std::count
* std::count_if
* std::mismatch
* std::equal
* using operator==
* using user-defined comparison
* std::find
* std::find_if
* std::find_if_not
* std::adjacent_find
* modifying sequence operations
* std::reverse
* std::rotate
* std::partition
* sorting operations
* std::sort
* with standard comparison
* with user-defined comparison
* sorting an object
* std::partial_sort
* set operations
* std::merge
* std::set_difference
* std::set_intersection
* std::set_union
* std::set_symmetric_difference
* heap operations
* iota
* int
* double
* char
* copy
* copy without if
* copy if
* copy n
* copy n chars
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-allocator.cpp
* bad_alloc
* bad_alloc
* controlled bad_alloc
* class json_value
* json_value(value_t)
* object
* array
* string
* json_value(const string_t&)
* class basic_json
* basic_json(const CompatibleObjectType&)
* basic_json(const CompatibleArrayType&)
* basic_json(const typename string_t::value_type*)
* basic_json(const typename string_t::value_type*)
* bad my_allocator::construct
* my_allocator::construct doesn't forward
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-alt-string.cpp
* alternative string type
* dump
* parse
* items
* equality
* JSON pointer
* patch
* diff
* flatten
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-arrays.cpp
* accept
* boundaries
* parse
* whitespace
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-assert_macro.cpp
* JSON_ASSERT(x)
* basic_json(first, second)
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-binary_formats.cpp
* Binary Formats
* canada.json
* twitter.json
* citm_catalog.json
* jeopardy.json
* sample.json
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using GNU 11.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 8.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.9.3 with standard gnu++11
* 1 test case was skipped when using Clang 18.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 12.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 10.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 9.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 6.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 19.1.7 with standard gnu++11
* 1 test case was skipped when using Clang 17.0.6 with standard gnu++11
* 1 test case was skipped when using Intel 2021.5.0.20211109 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 7.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 5.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.8.5 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-bjdata.cpp
* value_in_range_of trait
* BJData
* binary_reader BJData LUT arrays are sorted
* individual values
* discarded
* null
* boolean
* true
* false
* byte
* 0..255 (uint8)
* number
* signed
* -9223372036854775808..-2147483649 (int64)
* -2147483648..-32769 (int32)
* -32768..-129 (int16)
* -9263 (int16)
* -128..-1 (int8)
* 0..127 (int8)
* 128..255 (uint8)
* 256..32767 (int16)
* 32768..65535 (uint16)
* 65536..2147483647 (int32)
* 2147483648..4294967295 (uint32)
* 4294967296..9223372036854775807 (int64)
* 9223372036854775808..18446744073709551615 (uint64)
* unsigned
* 0..127 (int8)
* 128..255 (uint8)
* 256..32767 (int16)
* 32768..65535 (uint16)
* 65536..2147483647 (int32)
* 2147483648..4294967295 (uint32)
* 4294967296..9223372036854775807 (int64)
* 9223372036854775808..18446744073709551615 (uint64)
* float64
* 3.1415925
* half-precision float
* simple half floats
* errors
* no byte follows
* only one byte follows
* half-precision float (edge cases)
* exp = 0b00000
* 0 (0 00000 0000000000)
* -0 (1 00000 0000000000)
* 2**-24 (0 00000 0000000001)
* exp = 0b11111
* infinity (0 11111 0000000000)
* -infinity (1 11111 0000000000)
* other values from https://en.wikipedia.org/wiki/Half-precision_floating-point_format
* 1 (0 01111 0000000000)
* -2 (1 10000 0000000000)
* 65504 (0 11110 1111111111)
* infinity
* NaN
* high-precision number
* unsigned integer number
* signed integer number
* floating-point number
* errors
* string
* N = 0..127
* N = 128..255
* N = 256..32767
* N = 32768..65535
* N = 65536..2147483647
* binary
* N = 0..127
* N = 128..255
* N = 256..32767
* N = 32768..65535
* N = 65536..2147483647
* Other Serializations
* No Count No Type
* Yes Count No Type
* array
* empty
* size=false type=false
* size=true type=false
* size=true type=true
* [null]
* size=false type=false
* size=true type=false
* size=true type=true
* [1,2,3,4,5]
* size=false type=false
* size=true type=false
* size=true type=true
* [[[[]]]]
* size=false type=false
* size=true type=false
* size=true type=true
* array with int16_t elements
* size=false type=false
* size=true type=false
* array with uint16_t elements
* size=false type=false
* size=true type=false
* array with int32_t elements
* size=false type=false
* size=true type=false
* object
* empty
* size=false type=false
* size=true type=false
* size=true type=true
* {\
* size=false type=false
* size=true type=false
* {\
* size=false type=false
* size=true type=false
* size=true type=true ignore object type marker
* errors
* strict mode
* non-strict mode
* strict mode
* SAX aborts
* start_array()
* start_object()
* key() in object
* start_array(len)
* start_object(len)
* key() in object with length
* start_array() in ndarray _ArraySize_
* number_integer() in ndarray _ArraySize_
* key() in ndarray _ArrayType_
* string() in ndarray _ArrayType_
* key() in ndarray _ArrayData_
* string() in ndarray _ArrayData_
* string() in ndarray _ArrayType_
* start_array() in ndarray _ArrayData_
* parsing values
* strings
* number
* float
* array
* optimized version (length only)
* optimized version (type and length)
* optimized ndarray (type and vector-size as optimized 1D array)
* optimized ndarray (type and vector-size ndarray with JData annotations)
* optimized ndarray (type and vector-size as 1D array)
* optimized ndarray (type and vector-size as size-optimized array)
* invalid ndarray annotations remains as object
* parse errors
* empty byte vector
* char
* eof after C byte
* byte out of range
* byte
* parse bjdata markers in ubjson
* strings
* eof after S byte
* invalid byte
* parse bjdata markers in ubjson
* array
* optimized array: no size following type
* optimized array: negative size
* optimized array: integer value overflow
* do not accept NTFZ markers in ndarray optimized type (with count)
* do not accept NTFZ markers in ndarray optimized type (without count)
* strings
* sizes
* parse bjdata markers as array size in ubjson
* types
* arrays
* ndarrays
* objects
* writing optimized values
* integer
* array of i
* array of U
* array of I
* array of u
* array of l
* array of m
* array of L
* unsigned integer
* array of i
* array of U
* array of I
* array of u
* array of l
* array of m
* array of L
* array of M
* Universal Binary JSON Specification Examples 1
* Null Value
* No-Op Value
* Boolean Types
* Numeric Types
* Char Type
* Byte Type
* String Type
* English
* Russian
* Russian
* Array Type
* size=false type=false
* size=true type=false
* size=true type=true
* Object Type
* size=false type=false
* size=true type=false
* size=true type=true
* Optimized Format
* Array Example
* No Optimization
* Optimized with count
* Optimized with type & count
* Object Example
* No Optimization
* Optimized with count
* Optimized with type & count
* Special Cases (Null, No-Op and Boolean)
* Array
* Object
* all BJData first bytes
* BJData roundtrips
* input from self-generated BJData files
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using GNU 11.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 8.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.9.3 with standard gnu++11
* 1 test case was skipped when using Clang 18.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 12.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 10.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 9.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 6.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 19.1.7 with standard gnu++11
* 1 test case was skipped when using Clang 17.0.6 with standard gnu++11
* 1 test case was skipped when using Intel 2021.5.0.20211109 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 7.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 5.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.8.5 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-bson.cpp
* BSON
* individual values not supported
* null
* boolean
* true
* false
* number
* float
* string
* array
* keys containing code-point U+0000 cannot be serialized to BSON
* string length must be at least 1
* objects
* empty object
* non-empty object with bool
* non-empty object with bool
* non-empty object with double
* non-empty object with string
* non-empty object with null member
* non-empty object with integer (32-bit) member
* non-empty object with integer (64-bit) member
* non-empty object with negative integer (32-bit) member
* non-empty object with negative integer (64-bit) member
* non-empty object with unsigned integer (64-bit) member
* non-empty object with small unsigned integer member
* non-empty object with object member
* non-empty object with array member
* non-empty object with non-empty array member
* non-empty object with binary member
* non-empty object with binary member with subtype
* Some more complex document
* Examples from https://bsonspec.org/faq.html
* Example 1
* Example 2
* BSON input/output_adapters
* roundtrips
* std::ostringstream
* std::string
* std::vector
* Incomplete BSON Input
* Incomplete BSON Input 1
* Incomplete BSON Input 2
* Incomplete BSON Input 3
* Incomplete BSON Input 4
* Improve coverage
* key
* array
* Negative size of binary value
* Unsupported BSON input
* BSON numerical data
* number
* signed
* std::int64_t: INT64_MIN .. INT32_MIN-1
* signed std::int32_t: INT32_MIN .. INT32_MAX
* signed std::int64_t: INT32_MAX+1 .. INT64_MAX
* unsigned
* unsigned std::uint64_t: 0 .. INT32_MAX
* unsigned std::uint64_t: INT32_MAX+1 .. INT64_MAX
* unsigned std::uint64_t: INT64_MAX+1 .. UINT64_MAX
* BSON roundtrips
* reference files
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using GNU 11.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 8.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.9.3 with standard gnu++11
* 1 test case was skipped when using Clang 18.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 12.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 10.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 9.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 6.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 19.1.7 with standard gnu++11
* 1 test case was skipped when using Clang 17.0.6 with standard gnu++11
* 1 test case was skipped when using Intel 2021.5.0.20211109 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 7.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 5.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.8.5 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-byte_container_with_subtype.cpp
* byte_container_with_subtype
* empty container
* subtyped container
* comparisons
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-byte_order_mark.cpp
* accept
* UTF-8
* single BOM
* multiple BOM
* unexpected BOM
* Other byte-order marks
* UTF-16
* UTF-32
* parse
* UTF-8
* multiple BOM
* unexpected BOM
* other BOM
* UTF-16
* UTF-32
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-capacity.cpp
* capacity
* empty()
* boolean
* result of empty
* definition of empty
* string
* result of empty
* definition of empty
* array
* empty array
* result of empty
* definition of empty
* filled array
* result of empty
* definition of empty
* object
* empty object
* result of empty
* definition of empty
* filled object
* result of empty
* definition of empty
* number (integer)
* result of empty
* definition of empty
* number (unsigned)
* result of empty
* definition of empty
* number (float)
* result of empty
* definition of empty
* null
* result of empty
* definition of empty
* size()
* boolean
* result of size
* definition of size
* string
* result of size
* definition of size
* array
* empty array
* result of size
* definition of size
* filled array
* result of size
* definition of size
* object
* empty object
* result of size
* definition of size
* filled object
* result of size
* definition of size
* number (integer)
* result of size
* definition of size
* number (unsigned)
* result of size
* definition of size
* number (float)
* result of size
* definition of size
* null
* result of size
* definition of size
* max_size()
* boolean
* result of max_size
* string
* result of max_size
* array
* empty array
* result of max_size
* filled array
* result of max_size
* object
* empty object
* result of max_size
* filled object
* result of max_size
* number (integer)
* result of max_size
* number (unsigned)
* result of max_size
* number (float)
* result of max_size
* null
* result of max_size
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-cbor.cpp
* CBOR
* individual values
* discarded
* NaN
* Infinity
* null
* boolean
* true
* false
* number
* signed
* -9223372036854775808..-4294967297
* -4294967296..-65537
* -65536..-257
* -9263 (int 16)
* -256..-24
* -24..-1
* 0..23
* 24..255
* 256..65535
* 65536..4294967295
* 4294967296..4611686018427387903
* -32768..-129 (int 16)
* unsigned
* 0..23 (Integer)
* 24..255 (one-byte uint8_t)
* 256..65535 (two-byte uint16_t)
* 65536..4294967295 (four-byte uint32_t)
* 4294967296..4611686018427387903 (eight-byte uint64_t)
* double-precision float
* 3.1415925
* single-precision float
* 0.5
* 0.0
* -0.0
* 100.0
* 200.0
* 3.40282e+38(max float)
* -3.40282e+38(lowest float)
* 1 + 3.40282e+38(more than max float)
* -1 - 3.40282e+38(less than lowest float)
* half-precision float (edge cases)
* errors
* no byte follows
* only one byte follows
* exp = 0b00000
* 0 (0 00000 0000000000)
* -0 (1 00000 0000000000)
* 2**-24 (0 00000 0000000001)
* exp = 0b11111
* infinity (0 11111 0000000000)
* -infinity (1 11111 0000000000)
* other values from https://en.wikipedia.org/wiki/Half-precision_floating-point_format
* 1 (0 01111 0000000000)
* -2 (1 10000 0000000000)
* 65504 (0 11110 1111111111)
* infinity
* NaN
* string
* N = 0..23
* N = 24..255
* N = 256..65535
* N = 65536..4294967295
* array
* empty
* [null]
* [1,2,3,4,5]
* [[[[]]]]
* array with uint16_t elements
* array with uint32_t elements
* object
* empty
* {\
* {\
* object with uint8_t elements
* object with uint16_t elements
* object with uint32_t elements
* binary
* N = 0..23
* N = 24..255
* N = 256..65535
* N = 65536..4294967295
* indefinite size
* binary in array
* binary in object
* SAX callback with binary
* additional deserialization
* 0x5b (byte array)
* 0x7b (string)
* 0x9b (array)
* 0xbb (map)
* errors
* empty byte vector
* too short byte vector
* unsupported bytes
* concrete examples
* all unsupported bytes
* invalid string in map
* strict mode
* non-strict mode
* strict mode
* SAX aborts
* start_array(len)
* start_object(len)
* key()
* single CBOR roundtrip
* sample.json
* roundtrips
* std::ostringstream
* std::string
* CBOR regressions
* fuzz test results
* CBOR roundtrips
* input from flynn
* all CBOR first bytes
* examples from RFC 7049 Appendix A
* numbers
* simple values
* strings
* byte arrays
* arrays
* objects
* Tagged values
* 0xC6..0xD4
* 0xD8 - 1 byte follows
* success
* missing byte after tag
* 0xD9 - 2 byte follow
* success
* missing byte after tag
* 0xDA - 4 bytes follow
* success
* missing bytes after tag
* 0xDB - 8 bytes follow
* success
* missing byte after tag
* tagged binary
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using GNU 11.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 8.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.9.3 with standard gnu++11
* 1 test case was skipped when using Clang 18.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 12.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 10.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 9.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 6.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 19.1.7 with standard gnu++11
* 1 test case was skipped when using Clang 17.0.6 with standard gnu++11
* 1 test case was skipped when using Intel 2021.5.0.20211109 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 7.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 5.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.8.5 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-class_const_iterator.cpp
* const_iterator class
* construction
* constructor
* null
* object
* array
* copy assignment
* copy constructor from non-const iterator
* create from uninitialized iterator
* create from initialized iterator
* initialization
* set_begin
* null
* object
* array
* set_end
* null
* object
* array
* element access
* operator*
* null
* number
* object
* array
* operator->
* null
* number
* object
* array
* increment/decrement
* post-increment
* null
* number
* object
* array
* pre-increment
* null
* number
* object
* array
* post-decrement
* null
* number
* object
* array
* pre-decrement
* null
* number
* object
* array
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-class_iterator.cpp
* iterator class
* construction
* constructor
* null
* object
* array
* copy assignment
* initialization
* set_begin
* null
* object
* array
* set_end
* null
* object
* array
* element access
* operator*
* null
* number
* object
* array
* operator->
* null
* number
* object
* array
* increment/decrement
* post-increment
* null
* number
* object
* array
* pre-increment
* null
* number
* object
* array
* post-decrement
* null
* number
* object
* array
* pre-decrement
* null
* number
* object
* array
* equality-preserving
* post-increment
* primitive_iterator_t
* iter_impl
* json_reverse_iterator
* post-decrement
* primitive_iterator_t
* iter_impl
* json_reverse_iterator
* cert-dcl21-cpp
* post-increment
* primitive_iterator_t
* iter_impl
* json_reverse_iterator
* post-decrement
* primitive_iterator_t
* iter_impl
* json_reverse_iterator
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-class_lexer.cpp
* lexer class
* scan
* structural characters
* literal names
* numbers
* whitespace
* token_type_name
* parse errors on first character
* very large string
* fail on comments
* ignore comments
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-class_parser.cpp
* parser class
* parse
* null
* true
* false
* array
* empty array
* nonempty array
* object
* empty object
* nonempty object
* string
* errors
* additional test for null byte
* escaped
* number
* integers
* without exponent
* with exponent
* edge cases
* over the edge cases
* floating-point
* without exponent
* with exponent
* overflow
* invalid numbers
* accept
* null
* true
* false
* array
* empty array
* nonempty array
* object
* empty object
* nonempty object
* string
* errors
* escaped
* number
* integers
* without exponent
* with exponent
* edge cases
* over the edge cases
* floating-point
* without exponent
* with exponent
* overflow
* invalid numbers
* parse errors
* parse errors (accept)
* tests found by mutate++
* callback function
* filter nothing
* filter everything
* filter specific element
* filter object in array
* filter specific events
* first closing event
* special cases
* constructing from contiguous containers
* from std::vector
* from std::array
* from array
* from char literal
* from std::string
* from std::initializer_list
* from std::valarray
* improve test coverage
* parser with callback
* SAX parser
* } without value
* } with value
* second key
* ] without value
* ] with value
* float
* false
* null
* true
* unsigned
* integer
* string
* error messages for comments
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-class_parser_core.cpp
* parser class - core
* parse
* null
* true
* false
* array
* empty array
* nonempty array
* object
* empty object
* nonempty object
* string
* errors
* additional test for null byte
* escaped
* number
* integers
* without exponent
* with exponent
* edge cases
* over the edge cases
* floating-point
* without exponent
* with exponent
* overflow
* invalid numbers
* accept
* null
* true
* false
* array
* empty array
* nonempty array
* object
* empty object
* nonempty object
* string
* errors
* escaped
* number
* integers
* without exponent
* with exponent
* edge cases
* over the edge cases
* floating-point
* without exponent
* with exponent
* overflow
* invalid numbers
* parse errors
* parse errors (accept)
* tests found by mutate++
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-class_parser_diagnostic_positions.cpp
* parser class
* parse
* null
* true
* false
* array
* empty array
* nonempty array
* object
* empty object
* nonempty object
* string
* errors
* additional test for null byte
* escaped
* number
* integers
* without exponent
* with exponent
* edge cases
* over the edge cases
* floating-point
* without exponent
* with exponent
* overflow
* invalid numbers
* accept
* null
* true
* false
* array
* empty array
* nonempty array
* object
* empty object
* nonempty object
* string
* errors
* escaped
* number
* integers
* without exponent
* with exponent
* edge cases
* over the edge cases
* floating-point
* without exponent
* with exponent
* overflow
* invalid numbers
* parse errors
* parse errors (accept)
* tests found by mutate++
* callback function
* filter nothing
* filter everything
* filter specific element
* filter object in array
* filter specific events
* first closing event
* special cases
* constructing from contiguous containers
* from std::vector
* from std::array
* from array
* from char literal
* from std::string
* from std::initializer_list
* from std::valarray
* improve test coverage
* parser with callback
* SAX parser
* } without value
* } with value
* second key
* ] without value
* ] with value
* float
* false
* null
* true
* unsigned
* integer
* string
* error messages for comments
* with callback
* filter nothing
* filter element
* without callback
* retrieve start position and end position
* for object
* for array
* for array with objects
* for two levels of nesting objects
* for simple types
* no nested
* with callback
* without callback
* string type
* number type
* boolean type
* null type
* with leading whitespace and newlines around root JSON
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-comparison.cpp
* lexicographical comparison operators
* types
* comparison: less
* comparison: 3-way
* values
* compares unordered
* compares unordered (inverse)
* comparison: equal
* comparison: not equal
* comparison: less
* comparison: less than or equal equal
* comparison: greater than
* comparison: greater than or equal
* comparison: 3-way
* parser callback regression
* filter specific element
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 11.5.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++20
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* Clang 18.1.8 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 12.5.0 with standard gnu++20
* GNU 10.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* GNU 9.5.0 with standard gnu++11
* GNU 9.5.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 19.1.7 with standard gnu++20
* Clang 17.0.6 with standard gnu++11
* Clang 17.0.6 with standard gnu++20
* Intel 2021.5.0.20211109 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++20
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++20
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* Linux-c++ with standard gnu++11
* Linux-c++ with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
### List of tests in file unit-concepts.cpp
* concepts
* container requirements for json
* class json
* DefaultConstructible
* MoveConstructible
* CopyConstructible
* MoveAssignable
* CopyAssignable
* Destructible
* StandardLayoutType
* class iterator
* CopyConstructible
* CopyAssignable
* Destructible
* Swappable
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-constructor1.cpp
* constructors
* create an empty value with a given type
* null
* discarded
* object
* array
* boolean
* string
* number_integer
* number_unsigned
* number_float
* binary
* create a null object (implicitly)
* no parameter
* create a null object (explicitly)
* parameter
* create an object (explicit)
* empty object
* filled object
* create an object (implicit)
* std::map<json::string_t, json>
* std::map<std::string, std::string> #600
* std::map<const char*, json>
* std::multimap<json::string_t, json>
* std::unordered_map<json::string_t, json>
* std::unordered_multimap<json::string_t, json>
* associative container literal
* create an array (explicit)
* empty array
* filled array
* create an array (implicit)
* std::list<json>
* std::pair
* std::pair with discarded values
* std::tuple
* std::tuple with discarded values
* std::pair/tuple/array failures
* std::forward_list<json>
* std::array<json, 6>
* std::valarray<int>
* std::valarray<double>
* std::vector<json>
* std::deque<json>
* std::set<json>
* std::unordered_set<json>
* sequence container literal
* create a string (explicit)
* empty string
* filled string
* create a string (implicit)
* std::string
* char[]
* const char*
* string literal
* create a boolean (explicit)
* empty boolean
* filled boolean (true)
* filled boolean (false)
* from std::vector<bool>::reference
* from std::vector<bool>::const_reference
* create a binary (explicit)
* empty binary
* filled binary
* create an integer number (explicit)
* uninitialized value
* initialized value
* create an integer number (implicit)
* short
* unsigned short
* int
* unsigned int
* long
* unsigned long
* long long
* unsigned long long
* int8_t
* int16_t
* int32_t
* int64_t
* int_fast8_t
* int_fast16_t
* int_fast32_t
* int_fast64_t
* int_least8_t
* int_least16_t
* int_least32_t
* int_least64_t
* uint8_t
* uint16_t
* uint32_t
* uint64_t
* uint_fast8_t
* uint_fast16_t
* uint_fast32_t
* uint_fast64_t
* uint_least8_t
* uint_least16_t
* uint_least32_t
* uint_least64_t
* integer literal without suffix
* integer literal with u suffix
* integer literal with l suffix
* integer literal with ul suffix
* integer literal with ll suffix
* integer literal with ull suffix
* create a floating-point number (explicit)
* uninitialized value
* initialized value
* NaN
* infinity
* create a floating-point number (implicit)
* float
* double
* long double
* floating-point literal without suffix
* integer literal with f suffix
* integer literal with l suffix
* create a container (array or object) from an initializer list
* empty initializer list
* explicit
* implicit
* one element
* array
* explicit
* implicit
* object
* explicit
* implicit
* string
* explicit
* implicit
* boolean
* explicit
* implicit
* number (integer)
* explicit
* implicit
* number (unsigned)
* explicit
* implicit
* number (floating-point)
* explicit
* implicit
* more elements
* explicit
* implicit
* implicit type deduction
* object
* array
* explicit type deduction
* empty object
* object
* object with error
* empty array
* array
* move from initializer_list
* string
* constructor with implicit types (array)
* constructor with implicit types (object)
* constructor with implicit types (object key)
* array
* constructor with implicit types (array)
* constructor with implicit types (object)
* assignment with implicit types (array)
* assignment with implicit types (object)
* object
* constructor with implicit types (array)
* constructor with implicit types (object)
* assignment with implicit types (array)
* assignment with implicit types (object)
* json
* constructor with implicit types (array)
* constructor with implicit types (object)
* assignment with implicit types (array)
* assignment with implicit types (object)
* create an array of n copies of a given value
* cnt = 0
* cnt = 1
* cnt = 3
* create a JSON container from an iterator range
* object
* json(begin(), end())
* json(begin(), begin())
* construct from subrange
* incompatible iterators
* array
* json(begin(), end())
* json(begin(), begin())
* construct from subrange
* incompatible iterators
* other values
* construct with two valid iterators
* null
* string
* number (boolean)
* number (integer)
* number (unsigned)
* number (floating point)
* binary
* construct with two invalid iterators
* string
* number (boolean)
* number (integer)
* number (integer)
* number (floating point)
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-constructor2.cpp
* other constructors and destructor
* copy constructor
* object
* array
* null
* boolean
* string
* number (integer)
* number (unsigned)
* number (floating-point)
* binary
* move constructor
* copy assignment
* object
* array
* null
* boolean
* string
* number (integer)
* number (unsigned)
* number (floating-point)
* binary
* destructor
* object
* array
* string
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-convenience.cpp
* convenience functions
* type name as string
* string escape
* string concat
* std::string
* alt_string_iter
* alt_string_data
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-conversions.cpp
* value conversion
* get an object (explicit)
* json::object_t
* std::map<json::string_t, json>
* std::multimap<json::string_t, json>
* std::unordered_map<json::string_t, json>
* std::unordered_multimap<json::string_t, json>
* exception in case of a non-object type
* get an object (explicit, get_to)
* json::object_t
* std::map<json::string_t, json>
* std::multimap<json::string_t, json>
* std::unordered_map<json::string_t, json>
* std::unordered_multimap<json::string_t, json>
* get an object (implicit)
* json::object_t
* std::map<json::string_t, json>
* std::multimap<json::string_t, json>
* std::unordered_map<json::string_t, json>
* std::unordered_multimap<json::string_t, json>
* get an array (explicit)
* json::array_t
* std::list<json>
* std::forward_list<json>
* std::vector<json>
* reserve is called on containers that supports it
* built-in arrays
* std::deque<json>
* exception in case of a non-array type
* get an array (explicit, get_to)
* json::array_t
* std::valarray<json>
* std::list<json>
* std::forward_list<json>
* std::vector<json>
* built-in arrays
* built-in arrays: 2D
* built-in arrays: 3D
* built-in arrays: 4D
* std::deque<json>
* get an array (implicit)
* json::array_t
* std::list<json>
* std::forward_list<json>
* std::vector<json>
* std::deque<json>
* get a string (explicit)
* string_t
* std::string
* std::string_view
* exception in case of a non-string type
* exception in case of a non-string type using string_view
* get a string (explicit, get_to)
* string_t
* std::string
* std::string_view
* get null (explicit)
* get a string (implicit)
* string_t
* std::string_view
* std::string
* get a boolean (explicit)
* boolean_t
* uint8_t
* bool
* exception in case of a non-number type
* get a boolean (implicit)
* boolean_t
* bool
* get an integer number (explicit)
* number_integer_t
* number_unsigned_t
* short
* unsigned short
* int
* unsigned int
* long
* unsigned long
* long long
* unsigned long long
* int8_t
* int16_t
* int32_t
* int64_t
* int8_fast_t
* int16_fast_t
* int32_fast_t
* int64_fast_t
* int8_least_t
* int16_least_t
* int32_least_t
* int64_least_t
* uint8_t
* uint16_t
* uint32_t
* uint64_t
* uint8_fast_t
* uint16_fast_t
* uint32_fast_t
* uint64_fast_t
* uint8_least_t
* uint16_least_t
* uint32_least_t
* uint64_least_t
* exception in case of a non-number type
* get an integer number (implicit)
* number_integer_t
* number_unsigned_t
* short
* unsigned short
* int
* unsigned int
* long
* unsigned long
* long long
* unsigned long long
* int8_t
* int16_t
* int32_t
* int64_t
* int8_fast_t
* int16_fast_t
* int32_fast_t
* int64_fast_t
* int8_least_t
* int16_least_t
* int32_least_t
* int64_least_t
* uint8_t
* uint16_t
* uint32_t
* uint64_t
* uint8_fast_t
* uint16_fast_t
* uint32_fast_t
* uint64_fast_t
* uint8_least_t
* uint16_least_t
* uint32_least_t
* uint64_least_t
* get a floating-point number (explicit)
* number_float_t
* float
* double
* exception in case of a non-string type
* get a floating-point number (implicit)
* number_float_t
* float
* double
* get a binary value (explicit)
* binary_t
* get_binary()
* non-const
* non-const
* exception in case of a non-string type
* get a binary value (implicit)
* binary_t
* get an enum
* more involved conversions
* object-like STL containers
* std::map
* std::unordered_map
* std::multimap
* std::unordered_multimap
* exception in case of a non-object type
* array-like STL containers
* std::list
* std::forward_list
* std::array
* std::array is larger than JSON
* std::array is smaller than JSON
* std::valarray
* std::vector
* std::deque
* std::set
* std::unordered_set
* std::map (array of pairs)
* superfluous entries
* std::unordered_map (array of pairs)
* superfluous entries
* exception in case of a non-object type
* JSON to enum mapping
* enum class
* traditional enum
* std::filesystem::path
* ascii
* utf-8
* std::optional
* null
* string
* bool
* number
* array
* object
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++17
* GNU 11.5.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++14
* GNU 11.5.0 with standard gnu++17
* GNU 8.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++14
* GNU 8.5.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* Clang 20.1.8 with standard gnu++17
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++14
* GNU 13.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++17
* GNU 4.9.3 with standard gnu++11
* GNU 4.9.3 with standard gnu++14
* Clang 18.1.8 with standard gnu++11
* Clang 18.1.8 with standard gnu++14
* Clang 18.1.8 with standard gnu++17
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* GNU 14.3.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++14
* GNU 13.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* Clang 20.1.8 with standard gnu++17
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 12.5.0 with standard gnu++14
* GNU 12.5.0 with standard gnu++17
* GNU 10.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++14
* GNU 10.5.0 with standard gnu++17
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++14
* GNU 13.3.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* Clang 20.1.8 with standard gnu++17
* GNU 9.5.0 with standard gnu++11
* GNU 9.5.0 with standard gnu++14
* GNU 9.5.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 6.4.0 with standard gnu++14
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 19.1.7 with standard gnu++14
* Clang 19.1.7 with standard gnu++17
* Clang 17.0.6 with standard gnu++11
* Clang 17.0.6 with standard gnu++14
* Clang 17.0.6 with standard gnu++17
* Intel 2021.5.0.20211109 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++14
* Intel 2021.5.0.20211109 with standard gnu++17
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 7.5.0 with standard gnu++14
* GNU 7.5.0 with standard gnu++17
* GNU 13.4.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++14
* GNU 13.4.0 with standard gnu++17
* GNU 5.5.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* GNU 4.8.5 with standard gnu++14
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++17
* Linux-c++ with standard gnu++11
* Linux-c++ with standard gnu++14
* Linux-c++ with standard gnu++17
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++14
* GNU 13.3.0 with standard gnu++17
### List of tests in file unit-custom-base-class.cpp
* JSON Node Metadata
* type int
* type vector<int>
* copy ctor
* move ctor
* move assign
* copy assign
* type unique_ptr<int>
* type vector<int> in json array
* JSON Visit Node
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-deserialization.cpp
* deserialization
* successful deserialization
* stream
* string literal
* string_t
* operator<<
* operator>>
* user-defined string literal
* unsuccessful deserialization
* stream
* string
* operator<<
* operator>>
* user-defined string literal
* contiguous containers
* directly
* from std::vector
* from std::array
* from array
* from chars
* from std::string
* from std::initializer_list
* empty container
* via iterator range
* from std::vector
* from std::array
* from array
* from std::string
* from std::initializer_list
* from std::valarray
* with empty range
* iterator_input_adapter advances iterators correctly
* error cases
* case 1
* case 2
* case 3
* case 4
* case 5
* case 6
* case 7
* case 8
* case 9
* case 10
* case 11
* case 12
* case 13
* case 14
* case 15
* case 16
* ignoring byte-order marks
* BOM only
* BOM and content
* 2 byte of BOM
* 1 byte of BOM
* variations
* preserve state after parsing
* SAX and early abort
* JSON Lines
* Example file
* Example file without trailing newline
* deserialization of different character types (ASCII)
* deserialization of different character types (UTF-8)
* deserialization of different character types (UTF-16)
* deserialization of different character types (UTF-32)
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 11.5.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++20
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* Clang 18.1.8 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 12.5.0 with standard gnu++20
* GNU 10.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* GNU 9.5.0 with standard gnu++11
* GNU 9.5.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 19.1.7 with standard gnu++20
* Clang 17.0.6 with standard gnu++11
* Clang 17.0.6 with standard gnu++20
* Intel 2021.5.0.20211109 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++20
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++20
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* Linux-c++ with standard gnu++11
* Linux-c++ with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
### List of tests in file unit-diagnostic-positions-only.cpp
* Better diagnostics with positions only
* invalid type
* invalid type without positions
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-diagnostic-positions.cpp
* Better diagnostics with positions
* invalid type
* invalid type without positions
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-diagnostics.cpp
* Better diagnostics
* empty JSON Pointer
* invalid type
* missing key
* array index out of range
* array index at wrong type
* wrong iterator
* JSON Pointer escaping
* Parse error
* Wrong type in update()
* Regression tests for extended diagnostics
* Regression test for https://github.com/nlohmann/json/pull/2562#pullrequestreview-574858448
* Regression test for https://github.com/nlohmann/json/pull/2562/files/380a613f2b5d32425021129cd1f371ddcfd54ddf#r563259793
* Regression test for issue #2838 - Assertion failure when inserting into arrays with JSON_DIAGNOSTICS set
* Regression test for issue #2962 - JSON_DIAGNOSTICS assertion for ordered_json
* Regression test for issue #3007 - Parent pointers properly set when using update()
* Regression test for issue #3032 - Yet another assertion failure when inserting into arrays with JSON_DIAGNOSTICS set
* Regression test for issue #3915 - JSON_DIAGNOSTICS trigger assertion
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-disabled_exceptions.cpp
* Tests with disabled exceptions
* issue #2824 - encoding of json::exception::what()
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-element_access1.cpp
* element access 1
* array
* access specified element with bounds checking
* access within bounds
* access outside bounds
* access on non-array type
* null
* boolean
* string
* object
* number (integer)
* number (unsigned)
* number (floating-point)
* front and back
* access specified element
* access within bounds
* access on non-array type
* null
* standard tests
* implicit transformation to properly filled array
* boolean
* string
* object
* number (integer)
* number (unsigned)
* number (floating-point)
* remove specified element
* remove element by index
* remove element by iterator
* erase(begin())
* erase(begin(), end())
* erase(begin(), begin())
* erase at offset
* erase subrange
* different arrays
* remove element by index in non-array type
* null
* boolean
* string
* object
* number (integer)
* number (unsigned)
* number (floating-point)
* other values
* front and back
* null
* string
* number (boolean)
* number (integer)
* number (unsigned)
* number (floating point)
* erase with one valid iterator
* null
* string
* number (boolean)
* number (integer)
* number (unsigned)
* number (floating point)
* binary
* erase with one invalid iterator
* string
* number (boolean)
* number (integer)
* number (unsigned)
* number (floating point)
* erase with two valid iterators
* null
* string
* number (boolean)
* number (integer)
* number (unsigned)
* number (floating point)
* binary
* erase with two invalid iterators
* string
* number (boolean)
* number (integer)
* number (unsigned)
* number (floating point)
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-element_access2.cpp
* element access 2
* object
* access specified element with bounds checking
* access within bounds
* access outside bounds
* access on non-object type
* null
* boolean
* string
* array
* number (integer)
* number (unsigned)
* number (floating-point)
* access specified element with default value
* given a key
* access existing value
* access non-existing value
* access on non-object type
* null
* boolean
* string
* array
* number (integer)
* number (unsigned)
* number (floating-point)
* given a JSON pointer
* access existing value
* access on non-object type
* null
* boolean
* string
* array
* number (integer)
* number (unsigned)
* number (floating-point)
* non-const operator[]
* front and back
* access specified element
* access within bounds
* access within bounds (string_view)
* access on non-object type
* null
* boolean
* string
* array
* number (integer)
* number (unsigned)
* number (floating-point)
* remove specified element
* remove element by key
* remove element by key (string_view)
* remove element by iterator
* erase(begin())
* erase(begin(), end())
* erase(begin(), begin())
* erase at offset
* erase subrange
* different objects
* remove element by key in non-object type
* null
* boolean
* string
* array
* number (integer)
* number (floating-point)
* find an element in an object
* existing element
* nonexisting element
* all types
* null
* string
* object
* array
* boolean
* number (integer)
* number (unsigned)
* number (floating-point)
* count keys in an object
* existing element
* nonexisting element
* all types
* null
* string
* object
* array
* boolean
* number (integer)
* number (unsigned)
* number (floating-point)
* check existence of key in an object
* existing element
* nonexisting element
* all types
* null
* string
* object
* array
* boolean
* number (integer)
* number (unsigned)
* number (floating-point)
* element access 2 (throwing tests)
* object
* access specified element with default value
* given a JSON pointer
* access non-existing value
* element access 2 (additional value() tests)
* deduced ValueType
* literal key
* const char * key
* const char(&)[] key
* string_t/object_t::key_type key
* std::string_view key
* explicit ValueType
* literal key
* const char * key
* const char(&)[] key
* string_t/object_t::key_type key
* std::string_view key
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++17
* GNU 11.5.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++14
* GNU 11.5.0 with standard gnu++17
* GNU 8.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++14
* GNU 8.5.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* Clang 20.1.8 with standard gnu++17
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++14
* GNU 13.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++17
* GNU 4.9.3 with standard gnu++11
* GNU 4.9.3 with standard gnu++14
* Clang 18.1.8 with standard gnu++11
* Clang 18.1.8 with standard gnu++14
* Clang 18.1.8 with standard gnu++17
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* GNU 14.3.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++14
* GNU 13.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* Clang 20.1.8 with standard gnu++17
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 12.5.0 with standard gnu++14
* GNU 12.5.0 with standard gnu++17
* GNU 10.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++14
* GNU 10.5.0 with standard gnu++17
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++14
* GNU 13.3.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* Clang 20.1.8 with standard gnu++17
* GNU 9.5.0 with standard gnu++11
* GNU 9.5.0 with standard gnu++14
* GNU 9.5.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 6.4.0 with standard gnu++14
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 19.1.7 with standard gnu++14
* Clang 19.1.7 with standard gnu++17
* Clang 17.0.6 with standard gnu++11
* Clang 17.0.6 with standard gnu++14
* Clang 17.0.6 with standard gnu++17
* Intel 2021.5.0.20211109 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++14
* Intel 2021.5.0.20211109 with standard gnu++17
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 7.5.0 with standard gnu++14
* GNU 7.5.0 with standard gnu++17
* GNU 13.4.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++14
* GNU 13.4.0 with standard gnu++17
* GNU 5.5.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* GNU 4.8.5 with standard gnu++14
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++17
* Linux-c++ with standard gnu++11
* Linux-c++ with standard gnu++14
* Linux-c++ with standard gnu++17
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++14
* GNU 13.3.0 with standard gnu++17
### List of tests in file unit-hash.cpp
* hash<nlohmann::json>
* hash<nlohmann::ordered_json>
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-inspection.cpp
* object inspection
* convenience type checker
* object
* array
* null
* boolean
* string
* number (integer)
* number (unsigned)
* number (floating-point)
* binary
* discarded
* serialization
* no indent / indent=-1
* indent=0
* indent=1, space='\t'
* indent=4
* indent=x
* dump and floating-point numbers
* dump and small floating-point numbers
* dump and non-ASCII characters
* dump with ensure_ascii and non-ASCII characters
* full Unicode escaping to ASCII
* parsing yields the same JSON value
* dumping yields the same JSON text
* serialization of discarded element
* check that precision is reset after serialization
* round trips
* return the type of the object (explicit)
* null
* object
* array
* boolean
* string
* number (integer)
* number (unsigned)
* number (floating-point)
* return the type of the object (implicit)
* null
* object
* array
* boolean
* string
* number (integer)
* number (unsigned)
* number (floating-point)
* binary
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-items.cpp
* iterator_wrapper
* object
* value
* reference
* const value
* const reference
* const object
* value
* reference
* const value
* const reference
* array
* value
* reference
* const value
* const reference
* const array
* value
* reference
* const value
* const reference
* primitive
* value
* reference
* const value
* const reference
* const primitive
* value
* reference
* const value
* const reference
* items()
* object
* value
* reference
* const value
* const reference
* structured bindings
* const object
* value
* reference
* const value
* const reference
* array
* value
* reference
* const value
* const reference
* const array
* value
* reference
* const value
* const reference
* primitive
* value
* reference
* const value
* const reference
* const primitive
* value
* reference
* const value
* const reference
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 11.5.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++17
* GNU 8.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++17
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* Clang 18.1.8 with standard gnu++17
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++17
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 12.5.0 with standard gnu++17
* GNU 10.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++17
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++17
* GNU 9.5.0 with standard gnu++11
* GNU 9.5.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 19.1.7 with standard gnu++17
* Clang 17.0.6 with standard gnu++11
* Clang 17.0.6 with standard gnu++17
* Intel 2021.5.0.20211109 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++17
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 7.5.0 with standard gnu++17
* GNU 13.4.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++17
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* Linux-c++ with standard gnu++11
* Linux-c++ with standard gnu++17
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++17
### List of tests in file unit-iterators1.cpp
* iterators 1
* basic behavior
* uninitialized
* boolean
* json + begin/end
* const json + begin/end
* json + cbegin/cend
* const json + cbegin/cend
* json + rbegin/rend
* json + crbegin/crend
* const json + crbegin/crend
* additional tests
* !(begin != begin)
* !(end != end)
* begin < end
* begin <= end
* end > begin
* end >= begin
* end == end
* end <= end
* begin == begin
* begin <= begin
* begin >= begin
* !(begin == end)
* begin != end
* begin+1 == end
* begin == end-1
* begin != end+1
* end != end+1
* begin+1 != begin+2
* begin+1 < begin+2
* begin+1 <= begin+2
* end+1 != end+2
* key/value
* string
* json + begin/end
* const json + begin/end
* json + cbegin/cend
* const json + cbegin/cend
* json + rbegin/rend
* json + crbegin/crend
* const json + crbegin/crend
* key/value
* array
* json + begin/end
* const json + begin/end
* json + cbegin/cend
* const json + cbegin/cend
* json + rbegin/rend
* json + crbegin/crend
* const json + crbegin/crend
* key/value
* object
* json + begin/end
* const json + begin/end
* json + cbegin/cend
* const json + cbegin/cend
* json + rbegin/rend
* json + crbegin/crend
* const json + crbegin/crend
* key/value
* number (integer)
* json + begin/end
* const json + begin/end
* json + cbegin/cend
* const json + cbegin/cend
* json + rbegin/rend
* json + crbegin/crend
* const json + crbegin/crend
* key/value
* number (unsigned)
* json + begin/end
* const json + begin/end
* json + cbegin/cend
* const json + cbegin/cend
* json + rbegin/rend
* json + crbegin/crend
* const json + crbegin/crend
* key/value
* number (float)
* json + begin/end
* const json + begin/end
* json + cbegin/cend
* const json + cbegin/cend
* json + rbegin/rend
* json + crbegin/crend
* const json + crbegin/crend
* key/value
* null
* json + begin/end
* const json + begin/end
* json + cbegin/cend
* const json + cbegin/cend
* json + rbegin/rend
* json + crbegin/crend
* const json + crbegin/crend
* key/value
* conversion from iterator to const iterator
* boolean
* string
* array
* object
* number (integer)
* number (unsigned)
* number (float)
* null
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-iterators2.cpp
* iterators 2
* iterator comparisons
* iterator arithmetic
* addition and subtraction
* object
* array
* null
* value
* subscript operator
* object
* array
* null
* value
* reverse iterator comparisons
* reverse iterator arithmetic
* addition and subtraction
* object
* array
* null
* value
* subscript operator
* object
* array
* null
* value
* ranges
* concepts
* algorithms
* copy
* find_if
* views
* reverse
* transform
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 11.5.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++20
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* Clang 18.1.8 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 12.5.0 with standard gnu++20
* GNU 10.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* GNU 9.5.0 with standard gnu++11
* GNU 9.5.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 19.1.7 with standard gnu++20
* Clang 17.0.6 with standard gnu++11
* Clang 17.0.6 with standard gnu++20
* Intel 2021.5.0.20211109 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++20
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++20
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* Linux-c++ with standard gnu++11
* Linux-c++ with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
### List of tests in file unit-iterators3.cpp
* checking forward-iterators
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 11.5.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++14
* GNU 8.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 4.9.3 with standard gnu++11
* GNU 4.9.3 with standard gnu++14
* Clang 18.1.8 with standard gnu++11
* Clang 18.1.8 with standard gnu++14
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 12.5.0 with standard gnu++14
* GNU 10.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++14
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.5.0 with standard gnu++11
* GNU 9.5.0 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 6.4.0 with standard gnu++14
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 19.1.7 with standard gnu++14
* Clang 17.0.6 with standard gnu++11
* Clang 17.0.6 with standard gnu++14
* Intel 2021.5.0.20211109 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++14
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 7.5.0 with standard gnu++14
* GNU 13.4.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++14
* GNU 5.5.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* GNU 4.8.5 with standard gnu++14
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++14
* Linux-c++ with standard gnu++11
* Linux-c++ with standard gnu++14
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++14
### List of tests in file unit-json_patch.cpp
* JSON patch
* examples from RFC 6902
* 4. Operations
* 4.1 add
* 4.2 remove
* A.1. Adding an Object Member
* A.2. Adding an Array Element
* A.3. Removing an Object Member
* A.4. Removing an Array Element
* A.5. Replacing a Value
* A.6. Moving a Value
* A.7. Moving a Value
* A.8. Testing a Value: Success
* A.9. Testing a Value: Error
* A.10. Adding a Nested Member Object
* A.11. Ignoring Unrecognized Elements
* A.12. Adding to a Nonexistent Target
* A.14. Escape Ordering
* A.15. Comparing Strings and Numbers
* A.16. Adding an Array Value
* own examples
* add
* add to the root element
* add to end of the array
* copy
* replace
* documentation GIF
* errors
* unknown operation
* not an array
* not an array of objects
* missing 'op'
* non-string 'op'
* invalid operation
* add
* missing 'path'
* non-string 'path'
* missing 'value'
* invalid array index
* remove
* missing 'path'
* non-string 'path'
* nonexisting target location (array)
* nonexisting target location (object)
* root element as target location
* replace
* missing 'path'
* non-string 'path'
* missing 'value'
* nonexisting target location (array)
* nonexisting target location (object)
* move
* missing 'path'
* non-string 'path'
* missing 'from'
* non-string 'from'
* nonexisting from location (array)
* nonexisting from location (object)
* copy
* missing 'path'
* non-string 'path'
* missing 'from'
* non-string 'from'
* nonexisting from location (array)
* nonexisting from location (object)
* test
* missing 'path'
* non-string 'path'
* missing 'value'
* Examples from jsonpatch.com
* Simple Example
* Operations
* add
* remove
* replace
* copy
* move
* test
* Examples from bruth.github.io/jsonpatch-js
* add
* remove
* replace
* move
* copy
* copy
* Tests from github.com/json-patch/json-patch-tests
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-json_pointer.cpp
* JSON pointers
* errors
* array index error
* examples from RFC 6901
* nonconst access
* const access
* user-defined string literal
* array access
* nonconst access
* const access
* flatten
* string representation
* conversion
* array
* object
* empty, push, pop and parent
* operators
* equality comparison
* exceptions
* less-than comparison
* usable as map key
* backwards compatibility and mixing
* equality comparison
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 11.5.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++20
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* Clang 18.1.8 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 12.5.0 with standard gnu++20
* GNU 10.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* GNU 9.5.0 with standard gnu++11
* GNU 9.5.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 19.1.7 with standard gnu++20
* Clang 17.0.6 with standard gnu++11
* Clang 17.0.6 with standard gnu++20
* Intel 2021.5.0.20211109 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++20
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++20
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++20
* Linux-c++ with standard gnu++11
* Linux-c++ with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++20
### List of tests in file unit-large_json.cpp
* tests on very large JSONs
* issue #1419 - Segmentation fault (stack overflow) due to unbounded recursion
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-literals.cpp
* accept
* unicode
* capitalisation
* true
* null
* false
* illegal literals
* nil
* truth
* const
* none
* self
* super
* this
* undefined
* illegal literal numbers
* inf
* infinity
* NaN
* parse
* values
* whitespace
* capitalisation
* true
* null
* false
* illegal literals
* nil
* truth
* const
* none
* self
* super
* this
* undefined
* illegal literal numbers
* inf
* infinity
* NaN
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-locale-cpp.cpp
* locale-dependent test (LC_NUMERIC=C)
* check if locale is properly set
* parsing
* SAX parsing
* locale-dependent test (LC_NUMERIC=de_DE)
* check if locale is properly set
* parsing
* SAX parsing
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-merge_patch.cpp
* JSON Merge Patch
* examples from RFC 7396
* Section 1
* Section 3
* Appendix A
* Example 1
* Example 2
* Example 3
* Example 4
* Example 5
* Example 6
* Example 7
* Example 8
* Example 9
* Example 10
* Example 11
* Example 12
* Example 13
* Example 14
* Example 15
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-meta.cpp
* version information
* meta()
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-modifiers.cpp
* modifiers
* clear()
* boolean
* string
* array
* empty array
* filled array
* object
* empty object
* filled object
* binary
* empty binary
* filled binary
* number (integer)
* number (unsigned)
* number (float)
* null
* push_back()
* to array
* json&&
* null
* array
* other type
* const json&
* null
* array
* other type
* to object
* null
* object
* other type
* with initializer_list
* null
* array
* object
* emplace_back()
* to array
* null
* array
* multiple values
* other type
* emplace()
* to object
* null
* object
* other type
* operator+=
* to array
* json&&
* null
* array
* other type
* const json&
* null
* array
* other type
* to object
* null
* object
* other type
* with initializer_list
* null
* array
* object
* insert()
* value at position
* insert before begin()
* insert in the middle
* insert before end()
* rvalue at position
* insert before begin()
* insert in the middle
* insert before end()
* copies at position
* insert before begin()
* insert in the middle
* insert before end()
* insert nothing (count = 0)
* range for array
* proper usage
* empty range
* invalid iterators
* range for object
* proper usage
* empty range
* invalid iterators
* initializer list at position
* insert before begin()
* insert in the middle
* insert before end()
* invalid iterator
* non-array type
* update()
* non-recursive (default)
* const reference
* proper usage
* wrong types
* iterator range
* proper usage
* empty range
* invalid iterators
* recursive
* const reference
* extend object
* replace object
* swap()
* json
* member swap
* nonmember swap
* array_t
* array_t type
* non-array_t type
* object_t
* object_t type
* non-object_t type
* string_t
* string_t type
* non-string_t type
* binary_t
* binary_t type
* binary_t::container_type type
* non-binary_t type
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-msgpack.cpp
* MessagePack
* individual values
* discarded
* null
* boolean
* true
* false
* number
* signed
* -32..-1 (negative fixnum)
* 0..127 (positive fixnum)
* 128..255 (int 8)
* 256..65535 (int 16)
* 65536..4294967295 (int 32)
* 4294967296..9223372036854775807 (int 64)
* -128..-33 (int 8)
* -9263 (int 16)
* -32768..-129 (int 16)
* -32769..-2147483648
* -9223372036854775808..-2147483649 (int 64)
* unsigned
* 0..127 (positive fixnum)
* 128..255 (uint 8)
* 256..65535 (uint 16)
* 65536..4294967295 (uint 32)
* 4294967296..18446744073709551615 (uint 64)
* float
* 3.1415925
* 1.0
* 128.128
* string
* N = 0..31
* N = 32..255
* N = 256..65535
* N = 65536..4294967295
* array
* empty
* [null]
* [1,2,3,4,5]
* [[[[]]]]
* array 16
* array 32
* object
* empty
* {\
* {\
* map 16
* map 32
* extension
* N = 0..255
* N = 256..65535
* N = 65536..4294967295
* binary
* N = 0..255
* N = 256..65535
* N = 65536..4294967295
* from float32
* errors
* empty byte vector
* too short byte vector
* unexpected end inside int with stream
* misuse wchar for binary
* unsupported bytes
* concrete examples
* all unsupported bytes
* invalid string in map
* strict mode
* non-strict mode
* strict mode
* SAX aborts
* start_array(len)
* start_object(len)
* key()
* single MessagePack roundtrip
* sample.json
* roundtrips
* std::ostringstream
* std::string
* MessagePack roundtrips
* input from msgpack-python
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using GNU 11.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 8.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.9.3 with standard gnu++11
* 1 test case was skipped when using Clang 18.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 12.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 10.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 9.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 6.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 19.1.7 with standard gnu++11
* 1 test case was skipped when using Clang 17.0.6 with standard gnu++11
* 1 test case was skipped when using Intel 2021.5.0.20211109 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 7.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 5.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.8.5 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-no-mem-leak-on-adl-serialize.cpp
* check_for_mem_leak_on_adl_to_json-1
* check_for_mem_leak_on_adl_to_json-2
* check_for_mem_leak_on_adl_to_json-2
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-noexcept.cpp
* noexcept
* nothrow-copy-constructible exceptions
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-numbers.cpp
* accept
* exponents
* U+0425
* U+0436
* leading zeroes
* operators
* plus
* minus
* brackets
* factorial
* multiplication
* division
* comma
* whitespace
* space
* tab
* new line
* Carriage return
* Leading and tailing
* space
* tab
* newline
* Carriage return
* Mixed
* Leading zeroes
* bases
* Octal
* Hexadecimal
* parse
* exponents
* U+0425
* U+0436
* leading zeroes
* leading plus
* Capitalisation
* operators
* plus
* minus
* brackets
* factorial
* multiplication
* division
* comma
* trailing zeroes
* whitespace
* invalid whitespace
* space
* tab
* new line
* Carriage return
* Leading zeroes
* Precision
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-objects.cpp
* accept
* names
* numbers
* arrays
* objects
* literals
* strings
* control characters
* unicode
* escaped UTF-16 surrogates
* whitespace
* empty object
* non-empty object
* member separator
* parse
* whitespace
* empty object
* non-empty object
* member separator
* names
* numbers
* arrays
* objects
* literals
* duplicate names
* 100,000 identical keys
* first and last key duplicate
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-ordered_json.cpp
* ordered_json
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-ordered_map.cpp
* ordered_map
* constructor
* constructor from iterator range
* copy assignment
* at
* with Key&&
* with const Key&&
* with string literal
* operator[]
* with Key&&
* with const Key&&
* with string literal
* erase
* with Key&&
* with const Key&&
* with string literal
* with iterator
* with iterator pair
* range in the middle
* range at the beginning
* range at the end
* count
* find
* insert
* const value_type&
* value_type&&
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-pointer_access.cpp
* pointer access
* pointer access to object_t
* pointer access to const object_t
* pointer access to array_t
* pointer access to const array_t
* pointer access to string_t
* pointer access to const string_t
* pointer access to boolean_t
* pointer access to const boolean_t
* pointer access to number_integer_t
* pointer access to const number_integer_t
* pointer access to number_unsigned_t
* pointer access to const number_unsigned_t
* pointer access to number_float_t
* pointer access to const number_float_t
* pointer access to const binary_t
* pointer access to const binary_t
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-readme.cpp
* README
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using GNU 11.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 8.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.9.3 with standard gnu++11
* 1 test case was skipped when using Clang 18.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 12.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 10.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 9.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 6.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 19.1.7 with standard gnu++11
* 1 test case was skipped when using Clang 17.0.6 with standard gnu++11
* 1 test case was skipped when using Intel 2021.5.0.20211109 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 7.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 5.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.8.5 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-reference_access.cpp
* reference access
* reference access to object_t
* const reference access to const object_t
* reference access to array_t
* reference access to string_t
* reference access to boolean_t
* reference access to number_integer_t
* reference access to number_unsigned_t
* reference access to number_float_t
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-regression1.cpp
* regression tests 1
* issue #60 - Double quotation mark is not parsed correctly
* escape_doublequote
* issue #70 - Handle infinity and NaN cases
* NAN value
* infinity
* NAN value
* infinity
* pull request #71 - handle enum type
* issue #76 - dump() / parse() not idempotent
* issue #82 - lexer::get_number return NAN
* issue #89 - nonstandard integer type
* issue #93 reverse_iterator operator inheritance problem
* issue #100 - failed to iterator json object with reverse_iterator
* issue #101 - binary string causes numbers to be dumped as hex
* issue #111 - subsequent unicode chars
* issue #144 - implicit assignment to std::string fails
* issue #146 - character following a surrogate pair is skipped
* issue #171 - Cannot index by key of type static constexpr const char*
* issue #186 miloyip/nativejson-benchmark: floating-point parsing
* issue #228 - double values are serialized with commas as decimal points
* issue #378 - locale-independent num-to-str
* issue #379 - locale-independent str-to-num
* issue #233 - Can't use basic_json::iterator as a base iterator for std::move_iterator
* issue #235 - ambiguous overload for 'push_back' and 'operator+='
* issue #269 - diff generates incorrect patch when removing multiple array elements
* issue #283 - value() does not work with _json_pointer types
* issue #304 - Unused variable warning
* issue #306 - Parsing fails without space at end of file
* issue #310 - make json_benchmarks no longer working in 2.0.4
* issue #323 - add nested object capabilities to pointers
* issue #329 - serialized value not always can be parsed
* issue #360 - Loss of precision when serializing <double>
* issue #366 - json::parse on failed stream gets stuck
* issue #367 - calling stream at EOF
* issue #367 - behavior of operator>> should more closely resemble that of built-in overloads
* (empty)
* (whitespace)
* one value
* one value + whitespace
* whitespace + one value
* three values
* literals without whitespace
* example from #529
* second example from #529
* issue #389 - Integer-overflow (OSS-Fuzz issue 267)
* issue #380 - bug in overflow detection when parsing integers
* issue #405 - Heap-buffer-overflow (OSS-Fuzz issue 342)
* issue #407 - Heap-buffer-overflow (OSS-Fuzz issue 343)
* issue #408 - Heap-buffer-overflow (OSS-Fuzz issue 344)
* issue #411 - Heap-buffer-overflow (OSS-Fuzz issue 366)
* issue #412 - Heap-buffer-overflow (OSS-Fuzz issue 367)
* issue #414 - compare with literal 0)
* issue #416 - Use-of-uninitialized-value (OSS-Fuzz issue 377)
* issue #452 - Heap-buffer-overflow (OSS-Fuzz issue 585)
* issue #454 - doubles are printed as integers
* issue #464 - VS2017 implicit to std::string conversion fix
* issue #465 - roundtrip error while parsing 1000000000000000010E5
* issue #473 - inconsistent behavior in conversion to array type
* std::vector
* std::list
* std::forward_list
* issue #486 - json::value_t can't be a map's key type in VC++ 2015
* issue #494 - conversion from vector<bool> to json fails to build
* issue #504 - assertion error (OSS-Fuzz 856)
* issue #512 - use of overloaded operator '<=' is ambiguous
* issue #575 - heap-buffer-overflow (OSS-Fuzz 1400)
* issue #600 - how does one convert a map in Json back to std::map?
* example 1
* example 2
* issue #602 - BOM not skipped when using json:parse(iterator)
* issue #702 - conversion from valarray<double> to json fails to build
* original example
* full example
* issue #367 - Behavior of operator>> should more closely resemble that of built-in overloads.
* example 1
* issue #714 - throw std::ios_base::failure exception when failbit set to true
* issue #805 - copy constructor is used with std::initializer_list constructor.
* issue #838 - incorrect parse error with binary data in keys
* issue #843 - converting to array not working
* issue #894 - invalid RFC6902 copy operation succeeds
* issue #961 - incorrect parsing of indefinite length CBOR strings
* issue #962 - Timeout (OSS-Fuzz 6034)
* issue #971 - Add a SAX parser - late bug
* issue #972 - Segmentation fault on G++ when trying to assign json string literal to custom json type
* issue #977 - Assigning between different json types
* regression tests, exceptions dependent
* issue #1340 - eof not set on exhausted input stream
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 11.5.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++17
* GNU 8.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++17
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* Clang 18.1.8 with standard gnu++17
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++17
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 12.5.0 with standard gnu++17
* GNU 10.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++17
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++17
* GNU 9.5.0 with standard gnu++11
* GNU 9.5.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 19.1.7 with standard gnu++17
* Clang 17.0.6 with standard gnu++11
* Clang 17.0.6 with standard gnu++17
* Intel 2021.5.0.20211109 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++17
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 7.5.0 with standard gnu++17
* GNU 13.4.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++17
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* Linux-c++ with standard gnu++11
* Linux-c++ with standard gnu++17
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++17
### List of tests in file unit-regression2.cpp
* regression tests 2
* issue #1001 - Fix memory leak during parser callback
* issue #1021 - to/from_msgpack only works with standard typization
* issue #1045 - Using STL algorithms with JSON containers with expected results?
* issue #1292 - Serializing std::variant causes stack overflow
* issue #1299 - compile error in from_json converting to container
* issue #1445 - buffer overflow in dumping invalid utf-8 strings
* a bunch of -1, ensure_ascii=true
* a bunch of -2, ensure_ascii=false
* test case in issue #1445
* issue #1447 - Integer Overflow (OSS-Fuzz 12506)
* issue #1708 - minimum value of int64_t can be outputted
* issue #1727 - Contains with non-const lvalue json_pointer picks the wrong overload
* issue #1647 - compile error when deserializing enum if both non-default from_json and non-member operator== exists for other type
* issue #1715 - json::from_cbor does not respect allow_exceptions = false when input is string literal
* string literal
* string array
* std::string
* issue #1805 - A pair<T1, T2> is json constructible only if T1 and T2 are json constructible
* issue #1825 - A tuple<Args..> is json constructible only if all T in Args are json constructible
* issue #1983 - JSON patch diff for op=add formation is not as per standard (RFC 6902)
* issue #2067 - cannot serialize binary data to text JSON
* PR #2181 - regression bug with lvalue
* issue #2293 - eof doesn't cause parsing to stop
* issue #2315 - json.update and vector<pair>does not work with ordered_json
* issue #2330 - ignore_comment=true fails on multiple consecutive lines starting with comments
* issue #2546 - parsing containers of std::byte
* issue #2574 - Deserialization to std::array, std::pair, and std::tuple with non-default constructable types fails
* std::array
* std::pair
* std::tuple
* issue #4530 - Serialization of empty tuple
* issue #2865 - ASAN detects memory leaks
* issue #2824 - encoding of json::exception::what()
* issue #2825 - Properly constrain the basic_json conversion operator
* issue #2958 - Inserting in unordered json using a pointer retains the leading slash
* issue #2982 - to_{binary format} does not provide a mechanism for specifying a custom allocator for the returned type
* issue #3070 - Version 3.10.3 breaks backward-compatibility with 3.10.2
* issue #3077 - explicit constructor with default does not compile
* issue #3108 - ordered_json doesn't support range based erase
* issue #3343 - json and ordered_json are not interchangeable
* issue #3171 - if class is_constructible from std::string wrong from_json overload is being selected, compilation failed
* issue #3312 - Parse to custom class from unordered_json breaks on G++11.2.0 with C++20
* issue #3428 - Error occurred when converting nlohmann::json to std::any
* issue #3204 - ambiguous regression
* issue #3333 - Ambiguous conversion from nlohmann::basic_json<> to custom class
* issue #3810 - ordered_json doesn't support construction from C array of custom type
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++20
* GNU 11.5.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++17
* GNU 11.5.0 with standard gnu++20
* GNU 8.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++17
* GNU 13.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++20
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* Clang 18.1.8 with standard gnu++17
* Clang 18.1.8 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++17
* GNU 13.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* GNU 14.3.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 12.5.0 with standard gnu++17
* GNU 12.5.0 with standard gnu++20
* GNU 10.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++17
* GNU 10.5.0 with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++17
* GNU 13.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* Clang 20.1.8 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* GNU 9.5.0 with standard gnu++11
* GNU 9.5.0 with standard gnu++17
* GNU 9.5.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++20
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 19.1.7 with standard gnu++17
* Clang 19.1.7 with standard gnu++20
* Clang 17.0.6 with standard gnu++11
* Clang 17.0.6 with standard gnu++17
* Clang 17.0.6 with standard gnu++20
* Intel 2021.5.0.20211109 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++17
* Intel 2021.5.0.20211109 with standard gnu++20
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 7.5.0 with standard gnu++17
* GNU 13.4.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++17
* GNU 13.4.0 with standard gnu++20
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++17
* GNU 9.4.0 with standard gnu++20
* Linux-c++ with standard gnu++11
* Linux-c++ with standard gnu++17
* Linux-c++ with standard gnu++20
* GNU 13.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++17
* GNU 13.3.0 with standard gnu++20
### List of tests in file unit-serialization.cpp
* serialization
* operator<<
* no given width
* given width
* given fill
* operator>>
* no given width
* given width
* given fill
* dump
* invalid character
* ending with incomplete character
* unexpected character
* U+FFFD Substitution of Maximal Subparts
* to_string
* serialization for extreme integer values
* minimum
* maximum
* dump with binary values
* normal
* pretty-printed
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-strings.cpp
* accept
* noncharacter code positions
* overlong sequences
* Examples of an overlong ASCII character
* Maximum overlong sequences
* Overlong representation of the NUL character
* malformed sequences
* Unexpected continuation bytes
* Lonely start characters
* Sequences with last continuation byte missing
* Concatenation of incomplete sequences
* Impossible bytes
* Unicode
* escaped unicode
* unescaped unicode
* escaped utf-16 surrogates
* well-formed
* ill-formed
* parse
* whitespace
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using GNU 11.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 8.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.9.3 with standard gnu++11
* 1 test case was skipped when using Clang 18.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 12.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 10.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 9.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 6.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 19.1.7 with standard gnu++11
* 1 test case was skipped when using Clang 17.0.6 with standard gnu++11
* 1 test case was skipped when using Intel 2021.5.0.20211109 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 7.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 5.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.8.5 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-testsuites.cpp
* compliance tests from json.org
* expected failures
* no failures with trailing literals (relaxed)
* expected passes
* compliance tests from nativejson-benchmark
* doubles
* strings
* roundtrip
* test suite from json-test-suite
* read all sample.json
* json.org examples
* 1.json
* 2.json
* 3.json
* 4.json
* 5.json
* FILE 1.json
* FILE 2.json
* FILE 3.json
* FILE 4.json
* FILE 5.json
* RFC 8259 examples
* 7. Strings
* 8.3 String Comparison
* 13 Examples
* nst's JSONTestSuite
* test_parsing
* y
* n
* n -> y (relaxed)
* i -> y
* i/y -> n (out of range)
* i -> n
* nst's JSONTestSuite (2)
* test_parsing
* y
* n
* n (previously overflowed)
* i -> y
* i -> n
* Big List of Naughty Strings
* parsing blns.json
* roundtripping
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-to_chars.cpp
* digit gen
* single precision
* double precision
* formatting
* single precision
* double precision
* integer
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-type_traits.cpp
* type traits
* is_c_string
* char *
* char[]
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-ubjson.cpp
* UBJSON
* individual values
* discarded
* null
* boolean
* true
* false
* number
* signed
* -9223372036854775808..-2147483649 (int64)
* -2147483648..-32769 (int32)
* -32768..-129 (int16)
* -9263 (int16)
* -128..-1 (int8)
* 0..127 (int8)
* 128..255 (uint8)
* 256..32767 (int16)
* 65536..2147483647 (int32)
* 2147483648..9223372036854775807 (int64)
* unsigned
* 0..127 (int8)
* 128..255 (uint8)
* 256..32767 (int16)
* 65536..2147483647 (int32)
* 2147483648..9223372036854775807 (int64)
* float64
* 3.1415925
* high-precision number
* unsigned integer number
* signed integer number
* floating-point number
* errors
* serialization
* string
* N = 0..127
* N = 128..255
* N = 256..32767
* N = 65536..2147483647
* binary
* N = 0..127
* N = 128..255
* N = 256..32767
* N = 32768..2147483647
* Other Serializations
* No Count No Type
* Yes Count No Type
* array
* empty
* size=false type=false
* size=true type=false
* size=true type=true
* [null]
* size=false type=false
* size=true type=false
* size=true type=true
* [1,2,3,4,5]
* size=false type=false
* size=true type=false
* size=true type=true
* [[[[]]]]
* size=false type=false
* size=true type=false
* size=true type=true
* array with uint16_t elements
* size=false type=false
* size=true type=false
* size=true type=true
* array with uint32_t elements
* size=false type=false
* size=true type=false
* size=true type=true
* object
* empty
* size=false type=false
* size=true type=false
* size=true type=true
* {\
* size=false type=false
* size=true type=false
* size=true type=true
* {\
* size=false type=false
* size=true type=false
* size=true type=true
* errors
* strict mode
* non-strict mode
* strict mode
* excessive size
* array
* object
* SAX aborts
* start_array()
* start_object()
* key() in object
* start_array(len)
* start_object(len)
* key() in object with length
* parsing values
* strings
* number
* float
* array
* optimized version (length only)
* optimized version (type and length)
* parse errors
* empty byte vector
* char
* eof after C byte
* byte out of range
* strings
* eof after S byte
* invalid byte
* array
* optimized array: no size following type
* strings
* sizes
* types
* arrays
* objects
* writing optimized values
* integer
* array of i
* array of U
* array of I
* array of l
* array of L
* unsigned integer
* array of i
* array of U
* array of I
* array of l
* array of L
* discarded
* Universal Binary JSON Specification Examples 1
* Null Value
* No-Op Value
* Boolean Types
* Numeric Types
* Char Type
* String Type
* English
* Russian
* Russian
* Array Type
* size=false type=false
* size=true type=false
* size=true type=true
* Object Type
* size=false type=false
* size=true type=false
* size=true type=true
* Optimized Format
* Array Example
* No Optimization
* Optimized with count
* Optimized with type & count
* Object Example
* No Optimization
* Optimized with count
* Optimized with type & count
* Special Cases (Null, No-Op and Boolean)
* Array
* Object
* all UBJSON first bytes
* UBJSON roundtrips
* input from self-generated UBJSON files
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using GNU 11.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 8.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.9.3 with standard gnu++11
* 1 test case was skipped when using Clang 18.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 12.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 10.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 9.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 6.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 19.1.7 with standard gnu++11
* 1 test case was skipped when using Clang 17.0.6 with standard gnu++11
* 1 test case was skipped when using Intel 2021.5.0.20211109 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 7.5.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.4.0 with standard gnu++11
* 1 test case was skipped when using GNU 5.5.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 4.8.5 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-udl.cpp
* user-defined string literals
* using namespace nlohmann::literals::json_literals
* using namespace nlohmann::json_literals
* using namespace nlohmann::literals
* using namespace nlohmann
* global namespace
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-udt.cpp
* basic usage
* conversion to json via free-functions
* conversion from json via free-functions
* via explicit calls to get
* via explicit calls to get_to
* implicit conversions
* adl_serializer specialization
* partial specialization
* to_json
* from_json
* total specialization
* to_json
* from_json
* even supported types can be specialized
* Non-copyable types
* to_json
* from_json
* custom serializer for pods
* custom serializer that does adl by default
* different basic_json types conversions
* null
* boolean
* discarded
* array
* integer
* float
* unsigned
* string
* binary
* object
* get<custom_json>
* an incomplete type does not trigger a compiler error in non-evaluated context
* Issue #924
* Issue #1237
* compatible array type, without iterator type alias
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-udt_macro.cpp
* Serialization/deserialization via NLOHMANN_DEFINE_TYPE_INTRUSIVE and NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE
* person
* Serialization/deserialization via NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE and NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE
* person
* Serialization/deserialization via NLOHMANN_DEFINE_TYPE_INTRUSIVE_WITH_DEFAULT and NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_WITH_DEFAULT
* person with default values
* Serialization/deserialization via NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_WITH_DEFAULT and NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_WITH_DEFAULT
* derived person with default values
* Serialization/deserialization of classes with 26 public/private member variables via NLOHMANN_DEFINE_TYPE_INTRUSIVE and NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE
* alphabet
* Serialization of non-default-constructible classes via NLOHMANN_DEFINE_TYPE_INTRUSIVE_ONLY_SERIALIZE and NLOHMANN_DEFINE_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE
* person
* Serialization of non-default-constructible classes via NLOHMANN_DEFINE_DERIVED_TYPE_INTRUSIVE_ONLY_SERIALIZE and NLOHMANN_DEFINE_DERIVED_TYPE_NON_INTRUSIVE_ONLY_SERIALIZE
* derived person only serialize
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-unicode1.cpp
* Unicode (1/5)
* \\uxxxx sequences
* correct sequences
* incorrect sequences
* incorrect surrogate values
* incorrect sequences
* high surrogate without low surrogate
* high surrogate with wrong low surrogate
* low surrogate without high surrogate
* read all unicode characters
* check JSON Pointers
* ignore byte-order-mark
* in a stream
* with an iterator
* error for incomplete/wrong BOM
* Markus Kuhn's UTF-8 decoder capability and stress test
* 1 Some correct UTF-8 text
* 2 Boundary condition test cases
* 2.1 First possible sequence of a certain length
* 2.2 Last possible sequence of a certain length
* 2.3 Other boundary conditions
* 3 Malformed sequences
* 3.1 Unexpected continuation bytes
* 3.2 Lonely start characters
* 3.3 Sequences with last continuation byte missing
* 3.4 Concatenation of incomplete sequences
* 3.5 Impossible bytes
* 4 Overlong sequences
* 4.1 Examples of an overlong ASCII character
* 4.2 Maximum overlong sequences
* 4.3 Overlong representation of the NUL character
* 5 Illegal code positions
* 5.1 Single UTF-16 surrogates
* 5.2 Paired UTF-16 surrogates
* 5.3 Noncharacter code positions
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-unicode2.cpp
* Unicode (2/5)
* RFC 3629
* ill-formed first byte
* UTF8-1 (x00-x7F)
* well-formed
* UTF8-2 (xC2-xDF UTF8-tail)
* well-formed
* ill-formed: missing second byte
* ill-formed: wrong second byte
* UTF8-3 (xE0 xA0-BF UTF8-tail)
* well-formed
* ill-formed: missing second byte
* ill-formed: missing third byte
* ill-formed: wrong second byte
* ill-formed: wrong third byte
* UTF8-3 (xE1-xEC UTF8-tail UTF8-tail)
* well-formed
* ill-formed: missing second byte
* ill-formed: missing third byte
* ill-formed: wrong second byte
* ill-formed: wrong third byte
* UTF8-3 (xED x80-9F UTF8-tail)
* well-formed
* ill-formed: missing second byte
* ill-formed: missing third byte
* ill-formed: wrong second byte
* ill-formed: wrong third byte
* UTF8-3 (xEE-xEF UTF8-tail UTF8-tail)
* well-formed
* ill-formed: missing second byte
* ill-formed: missing third byte
* ill-formed: wrong second byte
* ill-formed: wrong third byte
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-unicode3.cpp
* Unicode (3/5)
* RFC 3629
* UTF8-4 (xF0 x90-BF UTF8-tail UTF8-tail)
* well-formed
* ill-formed: missing second byte
* ill-formed: missing third byte
* ill-formed: missing fourth byte
* ill-formed: wrong second byte
* ill-formed: wrong third byte
* ill-formed: wrong fourth byte
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-unicode4.cpp
* Unicode (4/5)
* RFC 3629
* UTF8-4 (xF1-F3 UTF8-tail UTF8-tail UTF8-tail)
* well-formed
* ill-formed: missing second byte
* ill-formed: missing third byte
* ill-formed: missing fourth byte
* ill-formed: wrong second byte
* ill-formed: wrong third byte
* ill-formed: wrong fourth byte
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-unicode5.cpp
* Unicode (5/5)
* RFC 3629
* UTF8-4 (xF4 x80-8F UTF8-tail UTF8-tail)
* well-formed
* ill-formed: missing second byte
* ill-formed: missing third byte
* ill-formed: missing fourth byte
* ill-formed: wrong second byte
* ill-formed: wrong third byte
* ill-formed: wrong fourth byte
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
In the following configuration, however, some test-cases were skipped:
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using GNU 14.3.0 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++17
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++23
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++14
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++20
* 1 test case was skipped when using Clang 20.1.8 with standard gnu++11
* 1 test case was skipped when using GNU 9.4.0 with standard gnu++11
* 1 test case was skipped when using Linux-c++ with standard gnu++11
* 1 test case was skipped when using GNU 13.3.0 with standard gnu++11
### List of tests in file unit-user_defined_input.cpp
* Use arbitrary stdlib container
* Custom container non-member begin/end
* Custom container member begin/end
* Custom iterator
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-windows_h.cpp
* include windows.h
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
### List of tests in file unit-wstring.cpp
* wide strings
* std::wstring
* invalid std::wstring
* std::u16string
* invalid std::u16string
* std::u32string
* invalid std::u32string
All tests in this file were run in the following configurations:
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++20
* GNU 9.4.0 with standard gnu++11
* GNU 11.5.0 with standard gnu++11
* GNU 8.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++14
* GNU 9.4.0 with standard gnu++11
* GNU 4.9.3 with standard gnu++11
* Clang 18.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 14.3.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 12.5.0 with standard gnu++11
* GNU 10.5.0 with standard gnu++11
* GNU 13.3.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++11
* GNU 9.5.0 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++14
* Clang 20.1.8 with standard gnu++23
* GNU 6.4.0 with standard gnu++11
* GNU 14.3.0 with standard gnu++17
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++17
* Clang 19.1.7 with standard gnu++11
* Clang 17.0.6 with standard gnu++11
* Intel 2021.5.0.20211109 with standard gnu++11
* GNU 14.3.0 with standard gnu++23
* Clang 20.1.8 with standard gnu++17
* GNU 7.5.0 with standard gnu++11
* GNU 13.4.0 with standard gnu++11
* GNU 5.5.0 with standard gnu++11
* Clang 20.1.8 with standard gnu++23
* Clang 20.1.8 with standard gnu++14
* GNU 4.8.5 with standard gnu++11
* Clang 20.1.8 with standard gnu++20
* Clang 20.1.8 with standard gnu++11
* GNU 9.4.0 with standard gnu++11
* Linux-c++ with standard gnu++11
* GNU 13.3.0 with standard gnu++11
website: https://github.com/eclipse-score/inc_nlohmann_json/actions
https://github.com/eclipse-score/inc_nlohmann_json/actions
- Github actions page showing that eclipse-score/inc_nlohmann_json is using Github host environment.
Fallacies:
None
Graph:
date-time |
JLS-16 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-17 | Reviewed: ⨯ | Score: 0.0#
Failure rates of CI tests are analysed for both the upstream nlohmann/json and the eclipse-score/inc_nlohmann_json repositories, and any high failure rates are reasonably justified.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Collected data from tests and monitoring of deployed software in eclipse-score/inc_nlohmann_json is analysed according to specified objectives. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
TSF/docs/ci_failure_rate_analysis.md
# Failure rate analysis for nlohmann/json Ubuntu CI
## Scope and data source
- **Repository:** `nlohmann/json`
- **Workflow:** `.github/workflows/ubuntu.yml`
- **Date range:** 2025-01-11 – 2025-04-11 (3 months before the release date of the version v3.12.0)
- **Filter:** `workflow_file_name: ubuntu.yml`
GitHub’s “failure rate” counts all non-successful job runs, including cancelled
jobs (e.g. quickly closed PRs) and infrastructure/tooling problems, not only
true regressions in the JSON library.
## Jobs with the highest reported failure rates
From the Jobs table (sorted by Failure rate):
| Job (short name) | Failure rate | Avg. run time | Job runs |
|:-------------------------------------------|-------------:|--------------:|---------:|
| `ci_test_coverage_clang` | **50 %** | 5m 33s | 6 |
| `ci_static_analysis_clang (ci_clang_tidy)` | **14 %** | 15m 36s | 247 |
| `ci_test_coverage` | **13 %** | 9m 19s | 248 |
| `ci_test_clang` | **12 %** | 6m 12s | 25 |
| `ci_test_documentation` | 4 % | 59s | 232 |
| `ci_static_analysis_clang (ci_test_clang)` | 4 % | 6m 41s | 214 |
| `ci_test_gcc` | 4 % | 6m 27s | 245 |
| `ci_cmake_options` | 3 % | 6m 01s | 248 |
| `ci_test_standards_clang` | 3 % | 2m 50s | 222 |
| `ci_test_compiler_gcc` | 3 % | 3m 28s | 227 |
The remaining jobs are at or below ~3 %, many at ≈0 %.
The remaining data can be found on the GitHub [insights page](https://github.com/nlohmann/json/actions/metrics/performance?dateRangeType=DATE_RANGE_TYPE_CUSTOM&sort=failureRate&tab=jobs&filters=workflow_file_name%3Aubuntu.yml&range=1736553600000-1744329600000).
## Interpretation of the higher failure rates
### `ci_test_coverage_clang`
This job was a short-lived Clang coverage experiment. In the selected time window it only ran 6 times, several of them as part of the "Fix coverage job" pull request where the coverage script was intentionally broken and iteratively fixed. The failures in these runs are due to coverage tooling / CI configuration issues, not failing unit tests or misbehaviour in the JSON library.
### `ci_static_analysis_clang (ci_clang_tidy)`
In the analyzed period, most failed `ci_static_analysis_clang (ci_clang_tidy)` runs come from a few PRs that deliberately changed static analysis or CI tooling, plus real bug fix. In particular, PR [#4654](https://github.com/nlohmann/json/pull/4654) "Fix ~basic_json causing std::terminate", PR [#4663](https://github.com/nlohmann/json/pull/4663) "Add clang-tidy plugin to convert implicit conversions to explicit ones", and the change referenced as [#4701](https://github.com/nlohmann/json/pull/4701) "Suppress clang-analyzer-webkit.NoUncountedMemberChecker" all show multiple failed Ubuntu workflow runs while their clang-tidy / analyzer configuration was being tuned. Because this job treats every clang-tidy or analyzer diagnostic as a hard error, each new or stricter check initially makes the job fail until the warnings are fixed or suppressed. The resulting ~14 % failure rate is therefore explained by intentional tightening and maintenance of the static-analysis pipeline (plus normal PR iteration), rather than by unexplained or unaddressed misbehaviour in the library itself.
### `ci_test_coverage`
The failure rate of `ci_test_coverage` in this period is explained by intentional CI maintenance rather than unstable tests. A significant portion of failures comes from work on PR [#4595](https://github.com/nlohmann/json/pull/4595), where the coverage workflow itself was updated and repeatedly executed on the `fix-coverage` branch and several intermediate runs failed until the configuration of `gcov`, `lcov`, and the Coveralls uploader was corrected. Additional failures stem from PR [#4709](https://github.com/nlohmann/json/pull/4709), which upgraded the minimum CMake version and introduced new CMake and OpenSSL configurations in CI. During this transition, some `ubuntu.yml` workflow runs, including the coverage job, failed until the revised toolchain setup was stabilized. All changes were merged only after `ci_test_coverage` succeeded, and the underlying unit tests remained consistently green throughout, meaning the elevated failure rate reflects CI/tooling evolution rather than regressions in the JSON library.
### `ci_test_clang`
Historically, `ci_test_clang` existed as its own standalone job in the Ubuntu workflow. With PR [#4560](https://github.com/nlohmann/json/pull/4560) (“Clean up and document project files”), the Ubuntu workflow was reorganized and `ci_test_clang` was moved into the matrix of the `ci_static_analysis_clang` job (i.e. `matrix.target: [ci_test_clang, ci_clang_tidy, …]` in `.github/workflows/ubuntu.yml`). GitHub Actions always uses the workflow file from the commit on which a PR is based, so PRs that branch off **before** [#4560](https://github.com/nlohmann/json/pull/4560) and are never rebased still run the *old* workflow, while newer PRs run the *new* workflow where `ci_test_clang` only appears as a matrix target of `ci_static_analysis_clang`. This is why `ci_test_clang` shows up in different ways in the metrics, even though it ultimately refers to the same underlying CMake target.
In this period, `ci_test_clang` ran 25 times and failed 3 times (12%). All failures occurred on PR branches while Clang related code or CI configuration was being adjusted and were resolved before the respective changes were merged. In addition, because `ci_test_clang` was later removed as a standalone Ubuntu job and replaced, the statistics for this job are based on a relatively small number of historical runs. A few legitimate work-in-progress failures therefore translate into a comparatively high percentage, without indicating persistent instability of the Clang test setup.
## Conclusion
For the selected three-month window before the v3.12.0 release, the Ubuntu CI for `nlohmann/json` shows a small set of jobs with noticeably higher failure rates, but in all cases these can be traced back to intentional CI/tooling work or strict analysis settings, not to undetected misbehaviour of the library itself. All of these high failure rates occurred on PR branches, were visible to developers, and were resolved before merging, while the majority of other jobs (including `ci_test_gcc`, `ci_test_standards_*` and most compiler-matrix entries) remain at or near 0 %. Taken together, this indicates a stable CI system that reacts as intended to real issues and configuration changes, with no evidence of systematic, unexplained spikes in test failures for the Ubuntu workflow.
# Failure rate analysis for eclipse-score/inc_nlohmann_json Parent-Workflow/Ubuntu CI
## Scope and data source
- **Repository:** `eclipse-score/inc_nlohmann_json`
- **Workflow:** `Parent Workflow` (top-level CI workflow)
- **Date range:** last 90 days from 08.12.2025
- **Filter:** `workflow_file_name: parent-workflow.yml`
GitHub’s “failure rate” counts all non-successful job runs, including cancelled
jobs (e.g. quickly closed PRs) and infrastructure/tooling problems, not only
true regressions in the JSON library.
## Jobs with the highest reported failure rates
From the Jobs table (sorted by Failure rate):
| Job (short name) | Failure rate | Avg. run time | Job runs |
|:------------------------------------------------------|-------------:|--------------:|---------:|
| `Run Ubuntu Workflow / publish_test_data_success` | **5%** | ≈ 25 s | 38 |
| `Run Labeler Workflow / clone_missing_labels` | **3%** | ≈ 6 s | 38 |
All other Parent-Workflow jobs in the last 90 days report a failure rate of
**0 %** over 38 runs each. The remaining data can be found on the GitHub [insights page](https://github.com/eclipse-score/inc_nlohmann_json/actions/metrics/performance?dateRangeType=DATE_RANGE_TYPE_LAST_90_DAYS&sort=failureRate&tab=jobs&filters=workflow_file_name%3Aparent-workflow.yml).
## Interpretation of the higher failure rates
The job `publish_test_data_success` is the final step of the Ubuntu child
workflow that publishes the persistent test-result database
(`TSF/MemoryEfficientTestResultData.db`) back to the `save_historical_data`
branch. The observed 5.26 % failure rate corresponds to 2 failed runs out of
38, and these failures are limited to this publishing step (e.g. git push /
branch / permission issues) rather than to the execution of the tests
themselves.
The job `clone_missing_labels` belongs to the separate “Labeler Workflow” and
synchronizes GitHub issue/PR labels for this repository with the organisation
defaults. Its 2.63 % failure rate corresponds to 1 failed run out of 38,
and is related to repository/label management rather than to building or
testing the JSON library.
## Conclusion
Taken together, a period of there months show that all test-related
jobs in the Parent Workflow have 0 % failures, while the few non-zero
failure rates are confined to meta-jobs that handle publishing of historical
test data and label synchronization. This indicates a stable CI setup for
`inc_nlohmann_json`, with the only reported failures occurring in
infrastructure-side integration steps rather than in the core test pipeline.
Description: Summary of the CI test failure rates analysis for the upstream nlohmann/json and the eclipse-score/inc_nlohmann_json repositories, including scope, selected time windows, methodology, and interpretation of the results.
Fallacies:
None
Graph:
date-time |
JLS-17 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-18 | Reviewed: ⨯ | Score: 0.0#
Results from automated tests for eclipse-score/inc_nlohmann_json are automatically collected from CI-generated JUnit reports into a persistent test results database together with metadata about the executed tests and workflows.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Data in eclipse-score/inc_nlohmann_json is collected from tests, and from monitoring of deployed software, according to specified objectives. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
TSF/scripts/capture_test_data_memory_sensitive.py
import sys
import sqlite3
import os
import xml.etree.ElementTree as ET
import re
from datetime import datetime, timezone
def setup_environment_variables() -> dict[str, str]:
# Retrieves and validates the necessary environment variables for GitHub workflows.
# Raises a RuntimeError if any required variables are missing.
required_vars = ["GITHUB_RUN_ID", "GITHUB_REPOSITORY", "GITHUB_RUN_ATTEMPT"]
environment = {var: os.getenv(var) for var in required_vars}
missing_vars = [var for var, value in environment.items() if not value]
if missing_vars:
raise RuntimeError(f"Missing required environment variables: {', '.join(missing_vars)}")
return environment
def clean_test_case(testcase: str) -> tuple[str,str]:
# This function expects a testcase of the form "testcase_name_cppxx".
# It returns the tuple ["testcase_name","gnu++xx"].
name, appendix = testcase.rsplit('_',1)
return [name, "gnu++"+appendix.replace('cpp','')]
def read_result_table(input: list[str]) -> dict:
"""
This function expects console output <system-out> of doctest.
It is assumed that this has the following form
<system-out>[doctest] doctest version is "2.4.11"
[doctest] run with "--help" for options
===============================================================================
[doctest] test cases: 1 | 1 passed | 0 failed | 0 skipped
[doctest] assertions: 45 | 45 passed | 0 failed |
[doctest] Status: SUCCESS!
</system-out>
It extracts the number of passed/failed/skipped test cases, and passed/skipped assertions.
"""
metadata = dict()
raw_data = input[0]
data = re.findall(r'(\d+)\s+(passed|failed|skipped)\b', raw_data)
if len(data) < 5:
raise RuntimeError("Fatal Error: Received incomplete or wrong result table.")
metadata["passed test cases"] = int(data[0][0])
metadata["failed test cases"] = int(data[1][0])
metadata["skipped test cases"] = int(data[2][0])
metadata["passed assertions"] = int(data[3][0])
metadata["failed assertions"] = int(data[4][0])
return metadata
def get_metadata(testcase: ET.Element) -> dict:
# expects testcase extracted from a junit xml-file as input
# extracts the data interesting to us
# Assumption of Use: before execution, it is checked if is_unit_test(testcase)==True
metadata = dict()
# from name both name of the test and C++ standard can be extracted
unsplit_name = testcase.get("name", None)
if unsplit_name is None:
raise RuntimeError("Fatal error: Can not read name of test-case!")
name, standard = clean_test_case(unsplit_name)
metadata["name"] = name
metadata["standard"] = standard
metadata["execution time"] = float(testcase.get("time"))
# results are not as easily extracted but must be processed further
metadata = metadata | read_result_table(list(testcase.find("system-out").itertext()))
return metadata
def is_unit_test(testcase: ET.Element) -> bool:
# crude test if the element belongs to a unit-test
return "_cpp" in testcase.get('name')
def get_all_xml_files(directory: str = '.') -> list[str]:
# search the folder containing all the artifacts and list the paths of expected test-reports
result = []
try:
content = os.listdir(directory)
except FileNotFoundError as e:
print(e)
return result
for entry in content:
if os.path.isdir(directory+'/'+entry):
result = result + get_all_xml_files(directory+'/'+entry)
if entry.endswith('.xml'):
file = directory+'/'+entry if directory != '.' else entry
result.append(file)
return result
def get_ctest_target(log_name: str) -> str:
# extracts name of ctest target from junit log
# log_name has the form "path/to/log/file/target_junit.xml", and target is expected
log = log_name.split('/')[-1]
return log.removesuffix("_junit.xml")
def find_most_recent_results(target: str, name: str, compiler: str, cpp_standard: str, database: sqlite3.Connection) -> list[int]:
cursor = database.cursor()
cursor.execute("""
WITH combination AS (
SELECT workflow_info.repo, workflow_info.run_id, workflow_info.run_attempt, workflow_info.time
FROM test_results INNER JOIN workflow_info ON
workflow_info.repo = test_results.repo
AND workflow_info.run_id = test_results.run_id
AND workflow_info.run_attempt = test_results.run_attempt
WHERE test_results.ctest_target = ? AND test_results.name = ? AND test_results.compiler = ? AND test_results.cpp_standard = ?
)
SELECT repo, run_id, run_attempt FROM combination
ORDER BY time DESC, run_id DESC, run_attempt DESC
LIMIT 1;
""",(target,name,compiler,cpp_standard))
result = cursor.fetchone()
if result is None:
# if no recent run is found, data need to be stored
return []
repo, run_id, run_attempt = result
cursor.execute("""
SELECT passed_cases, failed_cases, skipped_cases, passed_assertions, failed_assertions
FROM test_results WHERE
ctest_target = ? AND name = ? AND compiler = ? AND cpp_standard = ? AND repo = ? AND run_id = ? AND run_attempt = ?
""", (target,name,compiler,cpp_standard,repo,run_id,run_attempt))
result = cursor.fetchone()
return [] if result is None else list(result)
##########################
# Below starts the script.
##########################
if __name__ == "__main__":
# check if argument was delivered
if len(sys.argv) != 2:
raise RuntimeError("Expected status of workflow as argument. Aborting!")
# expected argument: status of workflow
# check if the argument has the expected form
status = sys.argv[1]
if status not in ["successful", "failed", "cancelled"]:
raise RuntimeError("The input does not match the expected format! Permissible are 'successful', 'failed' and 'cancelled'. Aborting!")
# get environment variables
try:
environment = setup_environment_variables()
except RuntimeError as e:
raise RuntimeError("Critical error: Can not uniquely identify environment data! Aborting recording of data.")
# Step 1: store metadata of workflow run persistently
# initiate connection to database
connector = sqlite3.connect("TSF/MemoryEfficientTestResultData.db")
connector.execute("PRAGMA foreign_keys = ON")
# load expected tables
# table workflow_info contains metadata of workflow and is updated every time
command = (
"CREATE TABLE IF NOT EXISTS workflow_info(",
"repo TEXT, ", # repository
"run_id INT, ", # ID of workflow run
"run_attempt INT, ", # Attempt-number of workflow run
"status TEXT ", # Termination-status of workflow
"CHECK(status IN ('successful', 'failed', 'cancelled')) DEFAULT 'failed', ",
"time INT, ", # the time that is associated to this workflow run
"PRIMARY KEY(repo, run_id, run_attempt))"
)
connector.execute(''.join(command))
# table test_results contains detailed results for each individual test
command = (
"CREATE TABLE IF NOT EXISTS test_results(",
"ctest_target TEXT, ", # name of the ctest target located in ci.cmake
"name TEXT, ", # name of the test
"execution_time REAL, ", # execution time in seconds
"compiler TEXT, ", # compiler information
"cpp_standard TEXT, ", # cpp-standard
"passed_cases INT, ", # number of passed test-cases
"failed_cases INT, ", # number of failed test-cases
"skipped_cases INT, ", # number of skipped test-cases
"passed_assertions INT, ", # number of passed assertions
"failed_assertions INT, ", # number of failed assertions
"repo TEXT, ", # repository
"run_id INT, ", # ID of workflow run
"run_attempt INT, ", # Attempt-number of workflow run
"FOREIGN KEY(repo, run_id, run_attempt) REFERENCES workflow_info);"
)
connector.execute(''.join(command))
cursor = connector.cursor()
# Count number of rows as heuristic size-checker.
# In case that the update-check fails, and every result is stored, allow for approximately 26 complete results to be stored
cursor.execute("SELECT MAX(COALESCE((SELECT MAX(rowid) FROM workflow_info),0),COALESCE((SELECT MAX(rowid) FROM test_results),0));")
if cursor.fetchone()[0] > 1e5:
connector.close()
raise RuntimeError("The persistent data storage is too large! Please move persistent data to external storage.")
# fill in metadata
# OBSERVE: This script expects the status of the github workflow as argument
repo = environment.get('GITHUB_REPOSITORY')
run_id = environment.get('GITHUB_RUN_ID')
run_attempt = environment.get('GITHUB_RUN_ATTEMPT')
time = int(datetime.now(timezone.utc).timestamp())
command = "INSERT INTO workflow_info VALUES(?,?,?,?,?)"
cursor.execute(command,(repo, run_id, run_attempt, status, time))
# Don't forget to save!
connector.commit()
# Step 2: generate report of most recent test run and update persistent storage if necessary
# initialise database connection
conn = sqlite3.connect("MemoryEfficientTestResults.db")
cur = conn.cursor()
# add the expected table
# the table TestResults.test_results differs from TestResultData.test_results in that the metadata of the commit are not saved.
command = (
"CREATE TABLE IF NOT EXISTS test_results(",
"ctest_target TEXT, ", # name of the ctest target located in ci.cmake
"name TEXT, ", # name of the test
"execution_time REAL, ", # execution time in seconds
"compiler TEXT, ", # compiler information
"cpp_standard TEXT, ", # cpp-standard
"passed_cases INT, ", # number of passed test-cases
"failed_cases INT, ", # number of failed test-cases
"skipped_cases INT, ", # number of skipped test-cases
"passed_assertions INT, ", # number of passed assertions
"failed_assertions INT", # number of failed assertions
")"
)
conn.execute(''.join(command))
# Load my artifacts
junit_logs = get_all_xml_files("./my_artifacts/")
#extract data
for junit_log in junit_logs:
tree = ET.parse(junit_log)
file_root = tree.getroot()
testsuite = next(file_root.iter('testsuite'), None)
if testsuite is None:
print(f"Error: Could not find testsuite data in {junit_log}.")
continue
for testcase in (case for case in file_root.iter('testcase') if is_unit_test(case)):
metadata = get_metadata(testcase)
target = get_ctest_target(junit_log)
compiler = testsuite.get('name')
more_compiler_info = [case for case in file_root.iter('testcase') if case.get("name") == "cmake_target_include_directories_configure"]
if len(more_compiler_info) != 0:
compiler_information = more_compiler_info[0]
information = list(compiler_information.find("system-out").itertext())[0].split('\n')[0]
compiler = information.replace("-- The CXX compiler identification is ","")
name = metadata.get('name')
cpp_standard = metadata.get('standard')
data = (
target,
name,
metadata.get('execution time'),
compiler,
cpp_standard,
metadata.get('passed test cases'),
metadata.get('failed test cases'),
metadata.get('skipped test cases'),
metadata.get('passed assertions'),
metadata.get('failed assertions')
)
command ="INSERT INTO test_results VALUES(?,?,?,?,?,?,?,?,?,?);"
cur.execute(command, data)
conn.commit()
most_recently_stored_results = find_most_recent_results(target,name,compiler,cpp_standard,connector)
current_results = [metadata.get('passed test cases'),metadata.get('failed test cases'),metadata.get('skipped test cases'),metadata.get('passed assertions'),metadata.get('failed assertions')]
if (len(most_recently_stored_results) != 5) or any(most_recently_stored_results[i]!=current_results[i] for i in range(0,5)):
data = data + (repo, run_id, run_attempt)
command ="INSERT INTO test_results VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?);"
cursor.execute(command,data)
# terminate connection to temporary database
# don't forget to commit the changes
conn.commit()
conn.close()
# terminate connection to persistent database
# don't forget to commit the changes again, for good measure
connector.commit()
connector.close()
.github/workflows/ubuntu.yml
name: Ubuntu
on:
workflow_call:
inputs:
artifact_id:
description: 'Unique identifier for artifacts'
required: true
type: string
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
cancel-in-progress: true
jobs:
ci_test_gcc:
runs-on: ubuntu-latest
container: gcc:14
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_gcc
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_gcc_artefact
path: build/my_logs/
ci_infer:
runs-on: ubuntu-latest
container: ghcr.io/nlohmann/json-ci:v2.4.0
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_infer
ci_static_analysis_ubuntu:
runs-on: ubuntu-latest
strategy:
matrix:
target: [ci_test_amalgamation, ci_test_single_header, ci_cppcheck, ci_cpplint, ci_reproducible_tests, ci_non_git_tests, ci_offline_testdata, ci_reuse_compliance, ci_test_valgrind]
steps:
- name: Harden Runner
uses: step-security/harden-runner@c6295a65d1254861815972266d5933fd6e532bdf # v2.11.1
with:
egress-policy: audit
- name: Install Valgrind
run: sudo apt-get update ; sudo apt-get install -y valgrind
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ${{ matrix.target }}
- name: Upload test report
if: ${{ matrix.target != 'ci_reuse_compliance' && matrix.target != 'ci_cpplint' && matrix.target != 'ci_test_amalgamation' && matrix.target != 'ci_cppcheck' }}
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_static_analysis_ubuntu_artefact_${{ matrix.target }}
path: build/my_logs/
ci_static_analysis_clang:
runs-on: ubuntu-latest
container: silkeh/clang:20
strategy:
matrix:
target: [ci_test_clang, ci_clang_tidy, ci_test_clang_sanitizer, ci_clang_analyze, ci_single_binaries]
steps:
- name: Install git, clang-tools, iwyu (ci_single_binaries), and unzip
run: apt-get update ; apt-get install -y git clang-tools iwyu unzip
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
# The default C++ compiler in the docker image is clang++-14 which does not support all compiler flags
run: cmake -S . -B build -DJSON_CI=On -DCLANG_TOOL=clang++-20
- name: Build
run: cmake --build build --target ${{ matrix.target }}
- name: Upload test report
if: ${{ matrix.target != 'ci_clang_analyze' && matrix.target != 'ci_single_binaries' && matrix.target != 'ci_clang_tidy' }}
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_static_analysis_clang_artefact_${{ matrix.target }}
path: build/my_logs/
ci_cmake_options:
runs-on: ubuntu-latest
container: ubuntu:focal
strategy:
matrix:
target: [ci_cmake_flags, ci_test_diagnostics, ci_test_diagnostic_positions, ci_test_noexceptions, ci_test_noimplicitconversions, ci_test_legacycomparison, ci_test_noglobaludls]
steps:
- name: Install build-essential
run: apt-get update ; apt-get install -y build-essential unzip wget git libssl-dev
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ${{ matrix.target }}
- name: Upload test report
if: ${{ matrix.target != 'ci_cmake_flags' }}
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_cmake_options_artefact_${{ matrix.target }}
path: build/my_logs/
ci_test_coverage:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@c6295a65d1254861815972266d5933fd6e532bdf # v2.11.1
with:
egress-policy: audit
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Install dependencies and de_DE locale
run: |
sudo apt-get clean
sudo apt-get update
sudo apt-get install -y build-essential cmake lcov ninja-build make locales gcc-multilib g++-multilib
sudo locale-gen de_DE
sudo update-locale
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_coverage
- name: Archive coverage report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: code-coverage-report
path: ${{ github.workspace }}/build/html
- name: Publish report to Coveralls
uses: coverallsapp/github-action@648a8eb78e6d50909eff900e4ec85cab4524a45b # v2.3.6
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
path-to-lcov: ${{ github.workspace }}/build/json.info.filtered.noexcept
fail-on-error: false # Do not fail the workflow if Coveralls fails
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_coverage_artefact
path: build/my_logs/
ci_test_compilers_gcc_old:
runs-on: ubuntu-latest
strategy:
matrix:
compiler: ['4.8', '4.9', '5', '6']
container: ghcr.io/nlohmann/json-ci:v2.4.0
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Run CMake
run: CXX=g++-${{ matrix.compiler }} cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_compiler_g++-${{ matrix.compiler }}
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_compilers_gcc_old_artefact_${{ matrix.compiler }}
path: build/my_logs/
ci_test_compilers_gcc:
runs-on: ubuntu-latest
strategy:
matrix:
# older GCC docker images (4, 5, 6) fail to check out code
compiler: ['7', '8', '9', '10', '11', '12', '13', '14']
container: gcc:${{ matrix.compiler }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_compiler_default
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_compilers_gcc_clang_artefact_${{ matrix.compiler }}
path: build/my_logs/
ci_test_compilers_clang:
runs-on: ubuntu-latest
strategy:
matrix:
compiler: ['17', '18', '19', '20']
container: silkeh/clang:${{ matrix.compiler }}
steps:
- name: Install unzip and git
run: apt-get update ; apt-get install -y unzip git
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Set env FORCE_STDCPPFS_FLAG for clang 7 / 8 / 9 / 10
run: echo "JSON_FORCED_GLOBAL_COMPILE_OPTIONS=-DJSON_HAS_FILESYSTEM=0;-DJSON_HAS_EXPERIMENTAL_FILESYSTEM=0" >> "$GITHUB_ENV"
if: ${{ matrix.compiler == '7' || matrix.compiler == '8' || matrix.compiler == '9' || matrix.compiler == '10' }}
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_compiler_default
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_compilers_clang_artefact_${{ matrix.compiler }}
path: build/my_logs/
ci_test_standards_gcc:
runs-on: ubuntu-latest
container: gcc:14.3
strategy:
matrix:
standard: [11, 14, 17, 20, 23, 26]
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_gcc_cxx${{ matrix.standard }}
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_standards_gcc_artefact_${{ matrix.standard }}
path: build/my_logs/
ci_test_standards_clang:
runs-on: ubuntu-latest
container: silkeh/clang:20
strategy:
matrix:
standard: [11, 14, 17, 20, 23, 26]
stdlib: [libcxx, libstdcxx]
steps:
- name: Install git and unzip
run: apt-get update ; apt-get install -y git unzip
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build with libc++
run: cmake --build build --target ci_test_clang_libcxx_cxx${{ matrix.standard }}
if: ${{ matrix.stdlib == 'libcxx' }}
- name: Build with libstdc++
run: cmake --build build --target ci_test_clang_cxx${{ matrix.standard }}
if: ${{ matrix.stdlib == 'libstdcxx' }}
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_standards_clang_artefact_${{ matrix.stdlib }}_${{ matrix.standard }}
path: build/my_logs/
ci_cuda_example:
runs-on: ubuntu-latest
container: ghcr.io/nlohmann/json-ci:v2.4.0
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_cuda_example
ci_icpc:
runs-on: ubuntu-latest
container: ghcr.io/nlohmann/json-ci:v2.2.0
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: |
. /opt/intel/oneapi/setvars.sh
cmake --build build --target ci_icpc
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_icpc_artefact
path: build/my_logs/
ci_test_documentation:
runs-on: ubuntu-latest
strategy:
matrix:
target: [ci_test_examples, ci_test_build_documentation]
steps:
- name: Harden Runner
uses: step-security/harden-runner@c6295a65d1254861815972266d5933fd6e532bdf # v2.11.1
with:
egress-policy: audit
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ${{ matrix.target }}
publish_test_data_success:
runs-on: ubuntu-latest
permissions:
contents: write
needs: [ci_test_gcc, ci_infer, ci_static_analysis_ubuntu, ci_static_analysis_clang, ci_cmake_options, ci_test_coverage, ci_test_compilers_gcc_old, ci_test_compilers_gcc, ci_test_compilers_clang, ci_test_standards_gcc, ci_test_standards_clang, ci_cuda_example, ci_icpc, ci_test_documentation]
if: success()
steps:
- name: Check out code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Download all workflow artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: my_artifacts
- name: Get current branch name
id: get_branch
env:
GITHUB_EVENT_NAME: ${{ github.event_name }}
PR_BRANCH_NAME: ${{ github.event.pull_request.head.ref }}
GITHUB_REF_SAFE: ${{ github.ref }}
run: |
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# Use the branch from the pull request safely via environment variable
current_branch="$PR_BRANCH_NAME"
else
# Remove the "refs/heads/" prefix from GITHUB_REF safely
current_branch="${GITHUB_REF_SAFE#refs/heads/}"
fi
# Write the branch name to $GITHUB_ENV securely
echo "branch_name=${current_branch}" >> "$GITHUB_ENV"
- name: Fetch data storage branch
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git fetch --all
git checkout save_historical_data && git pull || git checkout -b save_historical_data
- name: Load persistent data
run: |
if ! git ls-tree --name-only origin/save_historical_data TSF/MemoryEfficientTestResultData.db | grep TSF/MemoryEfficientTestResultData.db; then
touch TSF/MemoryEfficientTestResultData.db
git add TSF/MemoryEfficientTestResultData.db
git commit -m "Create persistent storage"
git push origin save_historical_data
fi
# Handle fork PRs by staying on current working branch instead of switching
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# For PRs, we're already on the right commit from actions/checkout
# Just get the file from save_historical_data branch
git checkout save_historical_data -- TSF/MemoryEfficientTestResultData.db
else
# For push events, switch to the branch and get the file
git checkout $branch_name
git checkout save_historical_data -- TSF/MemoryEfficientTestResultData.db
fi
- name: append test data
run: |
python3 TSF/scripts/capture_test_data_memory_sensitive.py successful
- name: Checkout data storage branch
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
git stash push --include-untracked -- $(git status --porcelain | awk '{print $2}' | grep -v TSF/MemoryEfficientTestResultData.db)
git checkout save_historical_data
- name: Commit and Push persistent data
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
git add TSF/MemoryEfficientTestResultData.db
git commit -m "Save test data" || echo "No test data to be saved"
git push origin save_historical_data
- name: Recover stash
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
# Handle fork PRs by staying on current working branch instead of switching
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# For PRs, we're already on the right branch/commit
git stash apply
else
# For push events, switch back to the working branch
git checkout $branch_name
git stash apply
fi
- name: Move test report
run: |
mkdir test_report
mv MemoryEfficientTestResults.db test_report/MemoryEfficientTestResults.db
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: publish_data_artifact
path: test_report/
publish_test_data_failure:
runs-on: ubuntu-latest
permissions:
contents: write
needs: [ci_test_gcc, ci_infer, ci_static_analysis_ubuntu, ci_static_analysis_clang, ci_cmake_options, ci_test_coverage, ci_test_compilers_gcc_old, ci_test_compilers_gcc, ci_test_compilers_clang, ci_test_standards_gcc, ci_test_standards_clang, ci_cuda_example, ci_icpc, ci_test_documentation]
if: failure()
steps:
- name: Check out code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Download all workflow artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: my_artifacts
- name: Get current branch name
id: get_branch
env:
GITHUB_EVENT_NAME: ${{ github.event_name }}
PR_BRANCH_NAME: ${{ github.event.pull_request.head.ref }}
GITHUB_REF_SAFE: ${{ github.ref }}
run: |
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# Use the branch from the pull request safely via environment variable
current_branch="$PR_BRANCH_NAME"
else
# Remove the "refs/heads/" prefix from GITHUB_REF safely
current_branch="${GITHUB_REF_SAFE#refs/heads/}"
fi
# Write the branch name to $GITHUB_ENV securely
echo "branch_name=${current_branch}" >> "$GITHUB_ENV"
- name: Fetch data storage branch
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git fetch --all
git checkout save_historical_data && git pull || git checkout -b save_historical_data
- name: Load persistent data
run: |
if ! git ls-tree --name-only origin/save_historical_data TSF/MemoryEfficientTestResultData.db | grep TSF/MemoryEfficientTestResultData.db; then
touch TSF/MemoryEfficientTestResultData.db
git add TSF/MemoryEfficientTestResultData.db
git commit -m "Create persistent storage"
git push origin save_historical_data
fi
# Handle fork PRs by staying on current working branch instead of switching
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# For PRs, we're already on the right commit from actions/checkout
# Just get the file from save_historical_data branch
git checkout save_historical_data -- TSF/MemoryEfficientTestResultData.db
else
# For push events, switch to the branch and get the file
git checkout $branch_name
git checkout save_historical_data -- TSF/MemoryEfficientTestResultData.db
fi
- name: append test data
run: |
python3 TSF/scripts/capture_test_data_memory_sensitive.py failed
- name: Checkout data storage branch
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
git stash push --include-untracked -- $(git status --porcelain | awk '{print $2}' | grep -v TSF/MemoryEfficientTestResultData.db)
git checkout save_historical_data
- name: Commit and Push persistent data
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
git add TSF/MemoryEfficientTestResultData.db
git commit -m "Save test data" || echo "No test data to be saved"
git push origin save_historical_data
- name: Recover stash
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
# Handle fork PRs by staying on current working branch instead of switching
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# For PRs, we're already on the right branch/commit
git stash apply
else
# For push events, switch back to the working branch
git checkout $branch_name
git stash apply
fi
- name: Move test report
run: |
mkdir test_report
mv MemoryEfficientTestResults.db test_report/MemoryEfficientTestResults.db
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: publish_data_artifact
path: test_report/
publish_test_data_cancellation:
runs-on: ubuntu-latest
permissions:
contents: write
needs: [ci_test_gcc, ci_infer, ci_static_analysis_ubuntu, ci_static_analysis_clang, ci_cmake_options, ci_test_coverage, ci_test_compilers_gcc_old, ci_test_compilers_gcc, ci_test_compilers_clang, ci_test_standards_gcc, ci_test_standards_clang, ci_cuda_example, ci_icpc, ci_test_documentation]
if: cancelled()
steps:
- name: Check out code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Download all workflow artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: my_artifacts
- name: Get current branch name
id: get_branch
env:
GITHUB_EVENT_NAME: ${{ github.event_name }}
PR_BRANCH_NAME: ${{ github.event.pull_request.head.ref }}
GITHUB_REF_SAFE: ${{ github.ref }}
run: |
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# Use the branch from the pull request safely via environment variable
current_branch="$PR_BRANCH_NAME"
else
# Remove the "refs/heads/" prefix from GITHUB_REF safely
current_branch="${GITHUB_REF_SAFE#refs/heads/}"
fi
# Write the branch name to $GITHUB_ENV securely
echo "branch_name=${current_branch}" >> "$GITHUB_ENV"
- name: Fetch data storage branch
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git fetch --all
git checkout save_historical_data && git pull || git checkout -b save_historical_data
- name: Load persistent data
run: |
if ! git ls-tree --name-only origin/save_historical_data TSF/MemoryEfficientTestResultData.db | grep TSF/MemoryEfficientTestResultData.db; then
touch TSF/MemoryEfficientTestResultData.db
git add TSF/MemoryEfficientTestResultData.db
git commit -m "Create persistent storage"
git push origin save_historical_data
fi
# Handle fork PRs by staying on current working branch instead of switching
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# For PRs, we're already on the right commit from actions/checkout
# Just get the file from save_historical_data branch
git checkout save_historical_data -- TSF/MemoryEfficientTestResultData.db
else
# For push events, switch to the branch and get the file
git checkout $branch_name
git checkout save_historical_data -- TSF/MemoryEfficientTestResultData.db
fi
- name: append test data
run: |
python3 TSF/scripts/capture_test_data_memory_sensitive.py cancelled
- name: Checkout data storage branch
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
git stash push --include-untracked -- $(git status --porcelain | awk '{print $2}' | grep -v TSF/MemoryEfficientTestResultData.db)
git checkout save_historical_data
- name: Commit and Push persistent data
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
git add TSF/MemoryEfficientTestResultData.db
git commit -m "Save test data" || echo "No test data to be saved"
git push origin save_historical_data
- name: Recover stash
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
# Handle fork PRs by staying on current working branch instead of switching
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# For PRs, we're already on the right branch/commit
git stash apply
else
# For push events, switch back to the working branch
git checkout $branch_name
git stash apply
fi
- name: Move test report
run: |
mkdir test_report
mv MemoryEfficientTestResults.db test_report/MemoryEfficientTestResults.db
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: publish_data_artifact
path: test_report/
ubuntu_artifact:
runs-on: ubuntu-latest
needs: [publish_test_data_success, publish_test_data_failure, publish_test_data_cancellation, ci_test_gcc, ci_infer, ci_static_analysis_ubuntu, ci_static_analysis_clang, ci_cmake_options, ci_test_coverage, ci_test_compilers_gcc_old, ci_test_compilers_gcc, ci_test_compilers_clang, ci_test_standards_gcc, ci_test_standards_clang, ci_cuda_example, ci_icpc, ci_test_documentation]
if: ${{ always() && (needs.publish_test_data_success.result == 'success' || needs.publish_test_data_failure.result == 'success' || needs.publish_test_data_cancellation.result == 'success') }}
steps:
- name: Download test report
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: publish_data_artifact
- name: Generate ubuntu artifact
run: |
echo "Generating ubuntu artifact..."
mkdir -p ubuntu
if [ -f MemoryEfficientTestResults.db ]; then
mv MemoryEfficientTestResults.db ubuntu/MemoryEfficientTestResults.db
else
echo "No test-results were captured."
fi
echo "ubuntu processed for ${{ inputs.artifact_id }}" > ubuntu/ubuntu.txt
- name: Upload ubuntu artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ${{ inputs.artifact_id }}
path: ubuntu/
Fallacies:
None
Graph:
date-time |
JLS-18 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-19 | Reviewed: ⨯ | Score: 0.0#
All library components, build dependencies, and build tools in the nlohmann/json repository are declared in build system manifests.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All constructed iterations of the nlohmann/json library include source code, build instructions, tests, results and attestations. |
0.00 |
⨯ Item Reviewed |
|
Construction of releases for the nlohmann/json library is fully repeatable and the results are fully reproducible, with any exceptions documented and justified. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://raw.githubusercontent.com/nlohmann/json/refs/heads/develop/CMakeLists.txt
https://raw.githubusercontent.com/nlohmann/json/refs/heads/develop/CMakeLists.txt
- CMake build manifest file
website: https://raw.githubusercontent.com/nlohmann/json/refs/heads/develop/BUILD.bazel
https://raw.githubusercontent.com/nlohmann/json/refs/heads/develop/BUILD.bazel
- Bazel build manifest file
website: https://raw.githubusercontent.com/nlohmann/json/refs/heads/develop/meson.build
https://raw.githubusercontent.com/nlohmann/json/refs/heads/develop/meson.build
- Meson build manifest file
website: https://raw.githubusercontent.com/nlohmann/json/refs/heads/develop/Package.swift
https://raw.githubusercontent.com/nlohmann/json/refs/heads/develop/Package.swift
- Swift Package Manager manifest file
Fallacies:
None
Graph:
date-time |
JLS-19 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-20 | Reviewed: ⨯ | Score: 0.0#
A github workflow of eclipse-score/inc_nlohmann_json saves the history of scores in the trustable graph to derive trends.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Confidence in the nlohmann/json library is measured based on results of analysis. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
.github/workflows/parent-workflow.yml
name: Parent Workflow
on:
pull_request:
push:
branches:
- main
schedule:
- cron: '0 0 * * *' # Runs daily at midnight UTC
permissions:
contents: read
jobs:
labeler:
permissions:
contents: read
pages: write
pull-requests: write
name: Run Labeler Workflow
uses: ./.github/workflows/labeler.yml
with:
artifact_id: "labeler-${{ github.sha }}"
SME_review_checker:
permissions:
contents: read
pull-requests: read
name: Run SME_review_checker Workflow
if: ${{ github.event_name == 'pull_request' }} # only run SME_review_checker for PRs
uses: ./.github/workflows/SME_review_checker.yml
with:
artifact_id: "SME_review_checker-${{ github.sha }}"
check_amalgamation:
name: Run Amalgamation Workflow
if: ${{ github.event_name == 'pull_request' }} # only run check_amalgamation for PRs
uses: ./.github/workflows/check_amalgamation.yml
with:
artifact_id: "check_amalgamation-${{ github.sha }}"
test_trudag_extensions:
name: Run Test Trudag Extensions Workflow
uses: ./.github/workflows/test_trudag_extensions.yml
with:
artifact_id: "test_trudag_extensions-${{ github.sha }}"
codeql:
permissions:
contents: read
security-events: write
name: Run Codeql analysis Workflow
uses: ./.github/workflows/codeql-analysis.yml
with:
artifact_id: "codeql-${{ github.sha }}"
ubuntu:
name: Run Ubuntu Workflow
permissions:
contents: write
needs: [codeql] # Error if CodeQL and Ubuntu triggered at the same time due to conflicting priorities
uses: ./.github/workflows/ubuntu.yml
with:
artifact_id: "ubuntu-${{ github.sha }}"
dependency_review:
name: Run dependency_review Workflow
if: ${{ github.event_name == 'pull_request' }} # only run dependency_review for PRs
uses: ./.github/workflows/dependency-review.yml
with:
artifact_id: "dependency_review-${{ github.sha }}"
collect_artifacts_pr:
name: "Collect Results & Deploy (PR)"
if: github.event_name == 'pull_request'
needs: [labeler, SME_review_checker, check_amalgamation, test_trudag_extensions, dependency_review, codeql, ubuntu]
runs-on: ubuntu-latest
strategy:
matrix:
target: [labeler, SME_review_checker, check_amalgamation, test_trudag_extensions, dependency_review, codeql, ubuntu]
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Check child workflow results
run: |
echo "=== Checking Child Workflow Results ==="
result="${{ needs[matrix.target].result }}"
echo "${{ matrix.target }} workflow result: $result"
if [[ "$result" != "success" ]]; then
echo "❌ ${{ matrix.target }} workflow failed! Exiting..."
exit 1
fi
echo "✅ Child workflows completed successfully!"
env:
current_workflow: ${{ matrix.target }}
- name: Download artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: "${{ matrix.target }}-${{ github.sha }}"
path: artifacts/
collect_artifacts_non_pr:
name: "Collect Results & Deploy (Non-PR)"
if: github.event_name != 'pull_request'
needs: [labeler, test_trudag_extensions, codeql, ubuntu] # no check_amalgamation, dependency_review or SME_review_checker if non PR
runs-on: ubuntu-latest
strategy:
matrix:
target: [labeler, test_trudag_extensions, codeql, ubuntu]
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Check child workflow results
run: |
echo "=== Checking Child Workflow Results ==="
result="${{ needs[matrix.target].result }}"
echo "${{ matrix.target }} workflow result: $result"
if [[ "$result" != "success" ]]; then
echo "❌ ${{ matrix.target }} workflow failed! Exiting..."
exit 1
fi
echo "✅ Child workflows completed successfully!"
env:
current_workflow: ${{ matrix.target }}
- name: Download artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: "${{ matrix.target }}-${{ github.sha }}"
path: artifacts/
test_publish_documentation:
permissions:
contents: write
pages: write
pull-requests: write
id-token: write
security-events: read
name: Test publish_documentation Workflow
if: github.event_name == 'pull_request' # Whenever on Pull-request, test publication
needs: [collect_artifacts_pr]
uses: ./.github/workflows/test_publication.yml
with:
artifact_id: "ubuntu-${{ github.sha }}"
publish_documentation:
permissions:
contents: write
pages: write
pull-requests: write
id-token: write
security-events: read
name: Run publish_documentation Workflow
if: github.event_name == 'push' # Publish documentation should only run on push to main
needs: [collect_artifacts_non_pr]
uses: ./.github/workflows/publish_documentation.yml
with:
artifact_id: "ubuntu-${{ github.sha }}"
Description: github workflow running on push to main and triggering the workflow publish_documentation
.github/workflows/publish_documentation.yml
# *******************************************************************************
# Copyright (c) 2025 Contributors to the Eclipse Foundation
#
# See the NOTICE file(s) distributed with this work for additional
# information regarding copyright ownership.
#
# This program and the accompanying materials are made available under the
# terms of the Apache License Version 2.0 which is available at
# https://www.apache.org/licenses/LICENSE-2.0
#
# SPDX-License-Identifier: Apache-2.0
# *******************************************************************************
name: Documentation
on:
workflow_call:
inputs:
artifact_id:
description: 'Unique identifier for artifacts to be loaded'
required: true
type: string
permissions:
contents: read
jobs:
run-trudag:
runs-on: ubuntu-latest
permissions:
contents: write
pages: write
pull-requests: write
id-token: write
steps:
- name: Checkout repository
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Load ubuntu-artifact
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: ${{ inputs.artifact_id }}
path: artifacts/
- name: Set up Python
uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
with:
python-version: '3.11.2'
# trudag version v2025.10.22
- name: Install trudag and dependencies
run: |
sudo apt-get update
sudo apt-get install -y graphviz
pip install --upgrade pip
pip install PyYAML requests
pip install git+https://gitlab.com/CodethinkLabs/trustable/trustable@v2025.10.22
- name: Install tools
run: |
sudo apt-get update
sudo apt-get install -y jq
sudo apt install gh -y
- name: Authenticate with GitHub CLI
run: |
echo "${{ secrets.GITHUB_TOKEN }}" | gh auth login --with-token
- name: Get current branch name
id: get_branch
env:
GITHUB_EVENT_NAME: ${{ github.event_name }}
PR_BRANCH_NAME: ${{ github.event.pull_request.head.ref }}
GITHUB_REF_SAFE: ${{ github.ref }}
run: |
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# Use the branch from the pull request safely via environment variable
current_branch="$PR_BRANCH_NAME"
else
# Remove the "refs/heads/" prefix from GITHUB_REF safely
current_branch="${GITHUB_REF_SAFE#refs/heads/}"
fi
# Write the branch name to $GITHUB_ENV securely
echo "branch_name=${current_branch}" >> "$GITHUB_ENV"
- name: Fetch data storage branch
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git fetch --all
git checkout save_historical_data && git pull || git checkout -b save_historical_data
- name: Fetch open issues labelled as bug from nlohmann/json
id: fetch_issues
run: |
echo "${{ secrets.GITHUB_TOKEN }}" | gh auth login --with-token
# Define variables
REPO="nlohmann/json" # Target repository
OUTPUT_FILE="TSF/misbehaviours.md"
# Check if the target repository is reachable
if ! gh repo view "$REPO" --json name -q .name; then
echo "Could not reach the target repository ($REPO). Aborting workflow."
exit 1
fi
# Fetch open issues from the nlohmann/json repository
gh issue list --repo "$REPO" --state open --limit 10000 --json number,title,state,createdAt,url,labels > raw_open_issues.json
# Fetch closed issues from the nlohmann/json repository
gh issue list --repo "$REPO" --state closed --limit 10000 --json number,title,state,createdAt,url,labels > raw_closed_issues.json
# print the list of misbehaviours
python3 TSF/scripts/generate_list_of_misbehaviours.py > $OUTPUT_FILE
# Push misbehaviours file to save_historical_data branch
git add TSF/misbehaviours.md
git commit -m "Updated issues list" || echo "No changes to commit"
git push origin save_historical_data && git pull
- name: Load persistent data
run: |
if ! git ls-tree --name-only origin/save_historical_data TSF/TrustableScoring.db | grep TSF/TrustableScoring.db; then
mkdir -p TSF
touch TSF/TrustableScoring.db
git add TSF/TrustableScoring.db
git commit -m "Initialise persistent data storage"
git push origin save_historical_data
fi
git checkout $branch_name
git checkout save_historical_data -- TSF/TrustableScoring.db
- name: Generate trudag report
run: |
REPO_FULL="${{ github.repository }}"
REPO_NAME="${REPO_FULL#*/}"
OWNER_NAME="${REPO_FULL%%/*}"
TSF/scripts/generate_report.sh "https://${OWNER_NAME}.github.io/${REPO_NAME}/main"
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Checkout data storage branch
run: |
git stash push --include-untracked -- $(git status --porcelain | awk '{print $2}' | grep -v TSF/TrustableScoring.db)
git checkout save_historical_data
git pull
- name: Store persistent data
run: |
git add TSF/TrustableScoring.db
git commit -m "Append data storage" || echo "Historical data already up to date."
git push origin save_historical_data
- name: Recover stash
run: |
git checkout $branch_name
git stash apply
- name: Upload trudag artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: trudag-report-${{ github.sha }}
path: TSF/docs/generated/
if-no-files-found: error
build-docs:
needs: run-trudag
# the eclipse-score/cicd-workflow docs.yml is adjusted to download the trudag report artifact before building the documentation
uses: ./.github/workflows/docs.yml
permissions:
contents: write
pages: write
pull-requests: write
id-token: write
with:
# the bazel-target depends on your repo specific docs_targets configuration (e.g. "suffix")Add commentMore actions
bazel-target: "//:docs -- --github_user=${{ github.repository_owner }} --github_repo=${{ github.event.repository.name }}"
retention-days: 3
Description: github workflow executing calculation and storage of trustable scores
website: https://github.com/eclipse-score/inc_nlohmann_json/blob/save_historical_data/TSF/TrustableScoring.db
https://github.com/eclipse-score/inc_nlohmann_json/blob/save_historical_data/TSF/TrustableScoring.db
- the database containing the trustable scores
Fallacies:
None
Graph:
date-time |
JLS-20 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-22 | Reviewed: ⨯ | Score: 0.0#
A scheduled GitHub workflow in eclipse-score/inc_nlohmann_json triggers the nlohmann/json unit and integration test at least once per day and records their outcomes as time-stamped entries in the persistent test results database.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All specified tests are executed repeatedly, under defined conditions in controlled environments, according to specified objectives. |
0.00 |
✔ Item Reviewed |
Supporting Items:
None
References:
.github/workflows/parent-workflow.yml
name: Parent Workflow
on:
pull_request:
push:
branches:
- main
schedule:
- cron: '0 0 * * *' # Runs daily at midnight UTC
permissions:
contents: read
jobs:
labeler:
permissions:
contents: read
pages: write
pull-requests: write
name: Run Labeler Workflow
uses: ./.github/workflows/labeler.yml
with:
artifact_id: "labeler-${{ github.sha }}"
SME_review_checker:
permissions:
contents: read
pull-requests: read
name: Run SME_review_checker Workflow
if: ${{ github.event_name == 'pull_request' }} # only run SME_review_checker for PRs
uses: ./.github/workflows/SME_review_checker.yml
with:
artifact_id: "SME_review_checker-${{ github.sha }}"
check_amalgamation:
name: Run Amalgamation Workflow
if: ${{ github.event_name == 'pull_request' }} # only run check_amalgamation for PRs
uses: ./.github/workflows/check_amalgamation.yml
with:
artifact_id: "check_amalgamation-${{ github.sha }}"
test_trudag_extensions:
name: Run Test Trudag Extensions Workflow
uses: ./.github/workflows/test_trudag_extensions.yml
with:
artifact_id: "test_trudag_extensions-${{ github.sha }}"
codeql:
permissions:
contents: read
security-events: write
name: Run Codeql analysis Workflow
uses: ./.github/workflows/codeql-analysis.yml
with:
artifact_id: "codeql-${{ github.sha }}"
ubuntu:
name: Run Ubuntu Workflow
permissions:
contents: write
needs: [codeql] # Error if CodeQL and Ubuntu triggered at the same time due to conflicting priorities
uses: ./.github/workflows/ubuntu.yml
with:
artifact_id: "ubuntu-${{ github.sha }}"
dependency_review:
name: Run dependency_review Workflow
if: ${{ github.event_name == 'pull_request' }} # only run dependency_review for PRs
uses: ./.github/workflows/dependency-review.yml
with:
artifact_id: "dependency_review-${{ github.sha }}"
collect_artifacts_pr:
name: "Collect Results & Deploy (PR)"
if: github.event_name == 'pull_request'
needs: [labeler, SME_review_checker, check_amalgamation, test_trudag_extensions, dependency_review, codeql, ubuntu]
runs-on: ubuntu-latest
strategy:
matrix:
target: [labeler, SME_review_checker, check_amalgamation, test_trudag_extensions, dependency_review, codeql, ubuntu]
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Check child workflow results
run: |
echo "=== Checking Child Workflow Results ==="
result="${{ needs[matrix.target].result }}"
echo "${{ matrix.target }} workflow result: $result"
if [[ "$result" != "success" ]]; then
echo "❌ ${{ matrix.target }} workflow failed! Exiting..."
exit 1
fi
echo "✅ Child workflows completed successfully!"
env:
current_workflow: ${{ matrix.target }}
- name: Download artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: "${{ matrix.target }}-${{ github.sha }}"
path: artifacts/
collect_artifacts_non_pr:
name: "Collect Results & Deploy (Non-PR)"
if: github.event_name != 'pull_request'
needs: [labeler, test_trudag_extensions, codeql, ubuntu] # no check_amalgamation, dependency_review or SME_review_checker if non PR
runs-on: ubuntu-latest
strategy:
matrix:
target: [labeler, test_trudag_extensions, codeql, ubuntu]
steps:
- name: Checkout code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Check child workflow results
run: |
echo "=== Checking Child Workflow Results ==="
result="${{ needs[matrix.target].result }}"
echo "${{ matrix.target }} workflow result: $result"
if [[ "$result" != "success" ]]; then
echo "❌ ${{ matrix.target }} workflow failed! Exiting..."
exit 1
fi
echo "✅ Child workflows completed successfully!"
env:
current_workflow: ${{ matrix.target }}
- name: Download artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: "${{ matrix.target }}-${{ github.sha }}"
path: artifacts/
test_publish_documentation:
permissions:
contents: write
pages: write
pull-requests: write
id-token: write
security-events: read
name: Test publish_documentation Workflow
if: github.event_name == 'pull_request' # Whenever on Pull-request, test publication
needs: [collect_artifacts_pr]
uses: ./.github/workflows/test_publication.yml
with:
artifact_id: "ubuntu-${{ github.sha }}"
publish_documentation:
permissions:
contents: write
pages: write
pull-requests: write
id-token: write
security-events: read
name: Run publish_documentation Workflow
if: github.event_name == 'push' # Publish documentation should only run on push to main
needs: [collect_artifacts_non_pr]
uses: ./.github/workflows/publish_documentation.yml
with:
artifact_id: "ubuntu-${{ github.sha }}"
Description: Parent GitHub workflow that is scheduled to run daily and triggers the ubuntu workflow.
.github/workflows/ubuntu.yml
name: Ubuntu
on:
workflow_call:
inputs:
artifact_id:
description: 'Unique identifier for artifacts'
required: true
type: string
permissions:
contents: read
concurrency:
group: ${{ github.workflow }}-${{ github.ref || github.run_id }}
cancel-in-progress: true
jobs:
ci_test_gcc:
runs-on: ubuntu-latest
container: gcc:14
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_gcc
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_gcc_artefact
path: build/my_logs/
ci_infer:
runs-on: ubuntu-latest
container: ghcr.io/nlohmann/json-ci:v2.4.0
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_infer
ci_static_analysis_ubuntu:
runs-on: ubuntu-latest
strategy:
matrix:
target: [ci_test_amalgamation, ci_test_single_header, ci_cppcheck, ci_cpplint, ci_reproducible_tests, ci_non_git_tests, ci_offline_testdata, ci_reuse_compliance, ci_test_valgrind]
steps:
- name: Harden Runner
uses: step-security/harden-runner@c6295a65d1254861815972266d5933fd6e532bdf # v2.11.1
with:
egress-policy: audit
- name: Install Valgrind
run: sudo apt-get update ; sudo apt-get install -y valgrind
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ${{ matrix.target }}
- name: Upload test report
if: ${{ matrix.target != 'ci_reuse_compliance' && matrix.target != 'ci_cpplint' && matrix.target != 'ci_test_amalgamation' && matrix.target != 'ci_cppcheck' }}
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_static_analysis_ubuntu_artefact_${{ matrix.target }}
path: build/my_logs/
ci_static_analysis_clang:
runs-on: ubuntu-latest
container: silkeh/clang:20
strategy:
matrix:
target: [ci_test_clang, ci_clang_tidy, ci_test_clang_sanitizer, ci_clang_analyze, ci_single_binaries]
steps:
- name: Install git, clang-tools, iwyu (ci_single_binaries), and unzip
run: apt-get update ; apt-get install -y git clang-tools iwyu unzip
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
# The default C++ compiler in the docker image is clang++-14 which does not support all compiler flags
run: cmake -S . -B build -DJSON_CI=On -DCLANG_TOOL=clang++-20
- name: Build
run: cmake --build build --target ${{ matrix.target }}
- name: Upload test report
if: ${{ matrix.target != 'ci_clang_analyze' && matrix.target != 'ci_single_binaries' && matrix.target != 'ci_clang_tidy' }}
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_static_analysis_clang_artefact_${{ matrix.target }}
path: build/my_logs/
ci_cmake_options:
runs-on: ubuntu-latest
container: ubuntu:focal
strategy:
matrix:
target: [ci_cmake_flags, ci_test_diagnostics, ci_test_diagnostic_positions, ci_test_noexceptions, ci_test_noimplicitconversions, ci_test_legacycomparison, ci_test_noglobaludls]
steps:
- name: Install build-essential
run: apt-get update ; apt-get install -y build-essential unzip wget git libssl-dev
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ${{ matrix.target }}
- name: Upload test report
if: ${{ matrix.target != 'ci_cmake_flags' }}
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_cmake_options_artefact_${{ matrix.target }}
path: build/my_logs/
ci_test_coverage:
runs-on: ubuntu-latest
steps:
- name: Harden Runner
uses: step-security/harden-runner@c6295a65d1254861815972266d5933fd6e532bdf # v2.11.1
with:
egress-policy: audit
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Install dependencies and de_DE locale
run: |
sudo apt-get clean
sudo apt-get update
sudo apt-get install -y build-essential cmake lcov ninja-build make locales gcc-multilib g++-multilib
sudo locale-gen de_DE
sudo update-locale
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_coverage
- name: Archive coverage report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: code-coverage-report
path: ${{ github.workspace }}/build/html
- name: Publish report to Coveralls
uses: coverallsapp/github-action@648a8eb78e6d50909eff900e4ec85cab4524a45b # v2.3.6
with:
github-token: ${{ secrets.GITHUB_TOKEN }}
path-to-lcov: ${{ github.workspace }}/build/json.info.filtered.noexcept
fail-on-error: false # Do not fail the workflow if Coveralls fails
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_coverage_artefact
path: build/my_logs/
ci_test_compilers_gcc_old:
runs-on: ubuntu-latest
strategy:
matrix:
compiler: ['4.8', '4.9', '5', '6']
container: ghcr.io/nlohmann/json-ci:v2.4.0
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Run CMake
run: CXX=g++-${{ matrix.compiler }} cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_compiler_g++-${{ matrix.compiler }}
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_compilers_gcc_old_artefact_${{ matrix.compiler }}
path: build/my_logs/
ci_test_compilers_gcc:
runs-on: ubuntu-latest
strategy:
matrix:
# older GCC docker images (4, 5, 6) fail to check out code
compiler: ['7', '8', '9', '10', '11', '12', '13', '14']
container: gcc:${{ matrix.compiler }}
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_compiler_default
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_compilers_gcc_clang_artefact_${{ matrix.compiler }}
path: build/my_logs/
ci_test_compilers_clang:
runs-on: ubuntu-latest
strategy:
matrix:
compiler: ['17', '18', '19', '20']
container: silkeh/clang:${{ matrix.compiler }}
steps:
- name: Install unzip and git
run: apt-get update ; apt-get install -y unzip git
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Set env FORCE_STDCPPFS_FLAG for clang 7 / 8 / 9 / 10
run: echo "JSON_FORCED_GLOBAL_COMPILE_OPTIONS=-DJSON_HAS_FILESYSTEM=0;-DJSON_HAS_EXPERIMENTAL_FILESYSTEM=0" >> "$GITHUB_ENV"
if: ${{ matrix.compiler == '7' || matrix.compiler == '8' || matrix.compiler == '9' || matrix.compiler == '10' }}
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_compiler_default
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_compilers_clang_artefact_${{ matrix.compiler }}
path: build/my_logs/
ci_test_standards_gcc:
runs-on: ubuntu-latest
container: gcc:14.3
strategy:
matrix:
standard: [11, 14, 17, 20, 23, 26]
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_test_gcc_cxx${{ matrix.standard }}
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_standards_gcc_artefact_${{ matrix.standard }}
path: build/my_logs/
ci_test_standards_clang:
runs-on: ubuntu-latest
container: silkeh/clang:20
strategy:
matrix:
standard: [11, 14, 17, 20, 23, 26]
stdlib: [libcxx, libstdcxx]
steps:
- name: Install git and unzip
run: apt-get update ; apt-get install -y git unzip
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Get latest CMake and ninja
uses: lukka/get-cmake@28983e0d3955dba2bb0a6810caae0c6cf268ec0c # v4.0.0
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build with libc++
run: cmake --build build --target ci_test_clang_libcxx_cxx${{ matrix.standard }}
if: ${{ matrix.stdlib == 'libcxx' }}
- name: Build with libstdc++
run: cmake --build build --target ci_test_clang_cxx${{ matrix.standard }}
if: ${{ matrix.stdlib == 'libstdcxx' }}
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_test_standards_clang_artefact_${{ matrix.stdlib }}_${{ matrix.standard }}
path: build/my_logs/
ci_cuda_example:
runs-on: ubuntu-latest
container: ghcr.io/nlohmann/json-ci:v2.4.0
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ci_cuda_example
ci_icpc:
runs-on: ubuntu-latest
container: ghcr.io/nlohmann/json-ci:v2.2.0
steps:
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: |
. /opt/intel/oneapi/setvars.sh
cmake --build build --target ci_icpc
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ci_icpc_artefact
path: build/my_logs/
ci_test_documentation:
runs-on: ubuntu-latest
strategy:
matrix:
target: [ci_test_examples, ci_test_build_documentation]
steps:
- name: Harden Runner
uses: step-security/harden-runner@c6295a65d1254861815972266d5933fd6e532bdf # v2.11.1
with:
egress-policy: audit
- uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Run CMake
run: cmake -S . -B build -DJSON_CI=On
- name: Build
run: cmake --build build --target ${{ matrix.target }}
publish_test_data_success:
runs-on: ubuntu-latest
permissions:
contents: write
needs: [ci_test_gcc, ci_infer, ci_static_analysis_ubuntu, ci_static_analysis_clang, ci_cmake_options, ci_test_coverage, ci_test_compilers_gcc_old, ci_test_compilers_gcc, ci_test_compilers_clang, ci_test_standards_gcc, ci_test_standards_clang, ci_cuda_example, ci_icpc, ci_test_documentation]
if: success()
steps:
- name: Check out code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Download all workflow artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: my_artifacts
- name: Get current branch name
id: get_branch
env:
GITHUB_EVENT_NAME: ${{ github.event_name }}
PR_BRANCH_NAME: ${{ github.event.pull_request.head.ref }}
GITHUB_REF_SAFE: ${{ github.ref }}
run: |
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# Use the branch from the pull request safely via environment variable
current_branch="$PR_BRANCH_NAME"
else
# Remove the "refs/heads/" prefix from GITHUB_REF safely
current_branch="${GITHUB_REF_SAFE#refs/heads/}"
fi
# Write the branch name to $GITHUB_ENV securely
echo "branch_name=${current_branch}" >> "$GITHUB_ENV"
- name: Fetch data storage branch
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git fetch --all
git checkout save_historical_data && git pull || git checkout -b save_historical_data
- name: Load persistent data
run: |
if ! git ls-tree --name-only origin/save_historical_data TSF/MemoryEfficientTestResultData.db | grep TSF/MemoryEfficientTestResultData.db; then
touch TSF/MemoryEfficientTestResultData.db
git add TSF/MemoryEfficientTestResultData.db
git commit -m "Create persistent storage"
git push origin save_historical_data
fi
# Handle fork PRs by staying on current working branch instead of switching
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# For PRs, we're already on the right commit from actions/checkout
# Just get the file from save_historical_data branch
git checkout save_historical_data -- TSF/MemoryEfficientTestResultData.db
else
# For push events, switch to the branch and get the file
git checkout $branch_name
git checkout save_historical_data -- TSF/MemoryEfficientTestResultData.db
fi
- name: append test data
run: |
python3 TSF/scripts/capture_test_data_memory_sensitive.py successful
- name: Checkout data storage branch
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
git stash push --include-untracked -- $(git status --porcelain | awk '{print $2}' | grep -v TSF/MemoryEfficientTestResultData.db)
git checkout save_historical_data
- name: Commit and Push persistent data
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
git add TSF/MemoryEfficientTestResultData.db
git commit -m "Save test data" || echo "No test data to be saved"
git push origin save_historical_data
- name: Recover stash
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
# Handle fork PRs by staying on current working branch instead of switching
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# For PRs, we're already on the right branch/commit
git stash apply
else
# For push events, switch back to the working branch
git checkout $branch_name
git stash apply
fi
- name: Move test report
run: |
mkdir test_report
mv MemoryEfficientTestResults.db test_report/MemoryEfficientTestResults.db
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: publish_data_artifact
path: test_report/
publish_test_data_failure:
runs-on: ubuntu-latest
permissions:
contents: write
needs: [ci_test_gcc, ci_infer, ci_static_analysis_ubuntu, ci_static_analysis_clang, ci_cmake_options, ci_test_coverage, ci_test_compilers_gcc_old, ci_test_compilers_gcc, ci_test_compilers_clang, ci_test_standards_gcc, ci_test_standards_clang, ci_cuda_example, ci_icpc, ci_test_documentation]
if: failure()
steps:
- name: Check out code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Download all workflow artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: my_artifacts
- name: Get current branch name
id: get_branch
env:
GITHUB_EVENT_NAME: ${{ github.event_name }}
PR_BRANCH_NAME: ${{ github.event.pull_request.head.ref }}
GITHUB_REF_SAFE: ${{ github.ref }}
run: |
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# Use the branch from the pull request safely via environment variable
current_branch="$PR_BRANCH_NAME"
else
# Remove the "refs/heads/" prefix from GITHUB_REF safely
current_branch="${GITHUB_REF_SAFE#refs/heads/}"
fi
# Write the branch name to $GITHUB_ENV securely
echo "branch_name=${current_branch}" >> "$GITHUB_ENV"
- name: Fetch data storage branch
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git fetch --all
git checkout save_historical_data && git pull || git checkout -b save_historical_data
- name: Load persistent data
run: |
if ! git ls-tree --name-only origin/save_historical_data TSF/MemoryEfficientTestResultData.db | grep TSF/MemoryEfficientTestResultData.db; then
touch TSF/MemoryEfficientTestResultData.db
git add TSF/MemoryEfficientTestResultData.db
git commit -m "Create persistent storage"
git push origin save_historical_data
fi
# Handle fork PRs by staying on current working branch instead of switching
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# For PRs, we're already on the right commit from actions/checkout
# Just get the file from save_historical_data branch
git checkout save_historical_data -- TSF/MemoryEfficientTestResultData.db
else
# For push events, switch to the branch and get the file
git checkout $branch_name
git checkout save_historical_data -- TSF/MemoryEfficientTestResultData.db
fi
- name: append test data
run: |
python3 TSF/scripts/capture_test_data_memory_sensitive.py failed
- name: Checkout data storage branch
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
git stash push --include-untracked -- $(git status --porcelain | awk '{print $2}' | grep -v TSF/MemoryEfficientTestResultData.db)
git checkout save_historical_data
- name: Commit and Push persistent data
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
git add TSF/MemoryEfficientTestResultData.db
git commit -m "Save test data" || echo "No test data to be saved"
git push origin save_historical_data
- name: Recover stash
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
# Handle fork PRs by staying on current working branch instead of switching
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# For PRs, we're already on the right branch/commit
git stash apply
else
# For push events, switch back to the working branch
git checkout $branch_name
git stash apply
fi
- name: Move test report
run: |
mkdir test_report
mv MemoryEfficientTestResults.db test_report/MemoryEfficientTestResults.db
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: publish_data_artifact
path: test_report/
publish_test_data_cancellation:
runs-on: ubuntu-latest
permissions:
contents: write
needs: [ci_test_gcc, ci_infer, ci_static_analysis_ubuntu, ci_static_analysis_clang, ci_cmake_options, ci_test_coverage, ci_test_compilers_gcc_old, ci_test_compilers_gcc, ci_test_compilers_clang, ci_test_standards_gcc, ci_test_standards_clang, ci_cuda_example, ci_icpc, ci_test_documentation]
if: cancelled()
steps:
- name: Check out code
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
- name: Download all workflow artifacts
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
path: my_artifacts
- name: Get current branch name
id: get_branch
env:
GITHUB_EVENT_NAME: ${{ github.event_name }}
PR_BRANCH_NAME: ${{ github.event.pull_request.head.ref }}
GITHUB_REF_SAFE: ${{ github.ref }}
run: |
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# Use the branch from the pull request safely via environment variable
current_branch="$PR_BRANCH_NAME"
else
# Remove the "refs/heads/" prefix from GITHUB_REF safely
current_branch="${GITHUB_REF_SAFE#refs/heads/}"
fi
# Write the branch name to $GITHUB_ENV securely
echo "branch_name=${current_branch}" >> "$GITHUB_ENV"
- name: Fetch data storage branch
run: |
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git fetch --all
git checkout save_historical_data && git pull || git checkout -b save_historical_data
- name: Load persistent data
run: |
if ! git ls-tree --name-only origin/save_historical_data TSF/MemoryEfficientTestResultData.db | grep TSF/MemoryEfficientTestResultData.db; then
touch TSF/MemoryEfficientTestResultData.db
git add TSF/MemoryEfficientTestResultData.db
git commit -m "Create persistent storage"
git push origin save_historical_data
fi
# Handle fork PRs by staying on current working branch instead of switching
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# For PRs, we're already on the right commit from actions/checkout
# Just get the file from save_historical_data branch
git checkout save_historical_data -- TSF/MemoryEfficientTestResultData.db
else
# For push events, switch to the branch and get the file
git checkout $branch_name
git checkout save_historical_data -- TSF/MemoryEfficientTestResultData.db
fi
- name: append test data
run: |
python3 TSF/scripts/capture_test_data_memory_sensitive.py cancelled
- name: Checkout data storage branch
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
git stash push --include-untracked -- $(git status --porcelain | awk '{print $2}' | grep -v TSF/MemoryEfficientTestResultData.db)
git checkout save_historical_data
- name: Commit and Push persistent data
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
git add TSF/MemoryEfficientTestResultData.db
git commit -m "Save test data" || echo "No test data to be saved"
git push origin save_historical_data
- name: Recover stash
if: github.event_name == 'schedule' || (github.event_name == 'push' && github.ref_name == 'main')
run: |
# Handle fork PRs by staying on current working branch instead of switching
if [[ "$GITHUB_EVENT_NAME" == "pull_request" ]]; then
# For PRs, we're already on the right branch/commit
git stash apply
else
# For push events, switch back to the working branch
git checkout $branch_name
git stash apply
fi
- name: Move test report
run: |
mkdir test_report
mv MemoryEfficientTestResults.db test_report/MemoryEfficientTestResults.db
- name: Upload test report
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: publish_data_artifact
path: test_report/
ubuntu_artifact:
runs-on: ubuntu-latest
needs: [publish_test_data_success, publish_test_data_failure, publish_test_data_cancellation, ci_test_gcc, ci_infer, ci_static_analysis_ubuntu, ci_static_analysis_clang, ci_cmake_options, ci_test_coverage, ci_test_compilers_gcc_old, ci_test_compilers_gcc, ci_test_compilers_clang, ci_test_standards_gcc, ci_test_standards_clang, ci_cuda_example, ci_icpc, ci_test_documentation]
if: ${{ always() && (needs.publish_test_data_success.result == 'success' || needs.publish_test_data_failure.result == 'success' || needs.publish_test_data_cancellation.result == 'success') }}
steps:
- name: Download test report
uses: actions/download-artifact@fa0a91b85d4f404e444e00e005971372dc801d16 # v4.1.8
with:
name: publish_data_artifact
- name: Generate ubuntu artifact
run: |
echo "Generating ubuntu artifact..."
mkdir -p ubuntu
if [ -f MemoryEfficientTestResults.db ]; then
mv MemoryEfficientTestResults.db ubuntu/MemoryEfficientTestResults.db
else
echo "No test-results were captured."
fi
echo "ubuntu processed for ${{ inputs.artifact_id }}" > ubuntu/ubuntu.txt
- name: Upload ubuntu artifact
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: ${{ inputs.artifact_id }}
path: ubuntu/
Description: Ubuntu CI workflow that executes the nlohmann/json unit and integration tests in a controlled CI environment.
TSF/scripts/capture_test_data_memory_sensitive.py
import sys
import sqlite3
import os
import xml.etree.ElementTree as ET
import re
from datetime import datetime, timezone
def setup_environment_variables() -> dict[str, str]:
# Retrieves and validates the necessary environment variables for GitHub workflows.
# Raises a RuntimeError if any required variables are missing.
required_vars = ["GITHUB_RUN_ID", "GITHUB_REPOSITORY", "GITHUB_RUN_ATTEMPT"]
environment = {var: os.getenv(var) for var in required_vars}
missing_vars = [var for var, value in environment.items() if not value]
if missing_vars:
raise RuntimeError(f"Missing required environment variables: {', '.join(missing_vars)}")
return environment
def clean_test_case(testcase: str) -> tuple[str,str]:
# This function expects a testcase of the form "testcase_name_cppxx".
# It returns the tuple ["testcase_name","gnu++xx"].
name, appendix = testcase.rsplit('_',1)
return [name, "gnu++"+appendix.replace('cpp','')]
def read_result_table(input: list[str]) -> dict:
"""
This function expects console output <system-out> of doctest.
It is assumed that this has the following form
<system-out>[doctest] doctest version is "2.4.11"
[doctest] run with "--help" for options
===============================================================================
[doctest] test cases: 1 | 1 passed | 0 failed | 0 skipped
[doctest] assertions: 45 | 45 passed | 0 failed |
[doctest] Status: SUCCESS!
</system-out>
It extracts the number of passed/failed/skipped test cases, and passed/skipped assertions.
"""
metadata = dict()
raw_data = input[0]
data = re.findall(r'(\d+)\s+(passed|failed|skipped)\b', raw_data)
if len(data) < 5:
raise RuntimeError("Fatal Error: Received incomplete or wrong result table.")
metadata["passed test cases"] = int(data[0][0])
metadata["failed test cases"] = int(data[1][0])
metadata["skipped test cases"] = int(data[2][0])
metadata["passed assertions"] = int(data[3][0])
metadata["failed assertions"] = int(data[4][0])
return metadata
def get_metadata(testcase: ET.Element) -> dict:
# expects testcase extracted from a junit xml-file as input
# extracts the data interesting to us
# Assumption of Use: before execution, it is checked if is_unit_test(testcase)==True
metadata = dict()
# from name both name of the test and C++ standard can be extracted
unsplit_name = testcase.get("name", None)
if unsplit_name is None:
raise RuntimeError("Fatal error: Can not read name of test-case!")
name, standard = clean_test_case(unsplit_name)
metadata["name"] = name
metadata["standard"] = standard
metadata["execution time"] = float(testcase.get("time"))
# results are not as easily extracted but must be processed further
metadata = metadata | read_result_table(list(testcase.find("system-out").itertext()))
return metadata
def is_unit_test(testcase: ET.Element) -> bool:
# crude test if the element belongs to a unit-test
return "_cpp" in testcase.get('name')
def get_all_xml_files(directory: str = '.') -> list[str]:
# search the folder containing all the artifacts and list the paths of expected test-reports
result = []
try:
content = os.listdir(directory)
except FileNotFoundError as e:
print(e)
return result
for entry in content:
if os.path.isdir(directory+'/'+entry):
result = result + get_all_xml_files(directory+'/'+entry)
if entry.endswith('.xml'):
file = directory+'/'+entry if directory != '.' else entry
result.append(file)
return result
def get_ctest_target(log_name: str) -> str:
# extracts name of ctest target from junit log
# log_name has the form "path/to/log/file/target_junit.xml", and target is expected
log = log_name.split('/')[-1]
return log.removesuffix("_junit.xml")
def find_most_recent_results(target: str, name: str, compiler: str, cpp_standard: str, database: sqlite3.Connection) -> list[int]:
cursor = database.cursor()
cursor.execute("""
WITH combination AS (
SELECT workflow_info.repo, workflow_info.run_id, workflow_info.run_attempt, workflow_info.time
FROM test_results INNER JOIN workflow_info ON
workflow_info.repo = test_results.repo
AND workflow_info.run_id = test_results.run_id
AND workflow_info.run_attempt = test_results.run_attempt
WHERE test_results.ctest_target = ? AND test_results.name = ? AND test_results.compiler = ? AND test_results.cpp_standard = ?
)
SELECT repo, run_id, run_attempt FROM combination
ORDER BY time DESC, run_id DESC, run_attempt DESC
LIMIT 1;
""",(target,name,compiler,cpp_standard))
result = cursor.fetchone()
if result is None:
# if no recent run is found, data need to be stored
return []
repo, run_id, run_attempt = result
cursor.execute("""
SELECT passed_cases, failed_cases, skipped_cases, passed_assertions, failed_assertions
FROM test_results WHERE
ctest_target = ? AND name = ? AND compiler = ? AND cpp_standard = ? AND repo = ? AND run_id = ? AND run_attempt = ?
""", (target,name,compiler,cpp_standard,repo,run_id,run_attempt))
result = cursor.fetchone()
return [] if result is None else list(result)
##########################
# Below starts the script.
##########################
if __name__ == "__main__":
# check if argument was delivered
if len(sys.argv) != 2:
raise RuntimeError("Expected status of workflow as argument. Aborting!")
# expected argument: status of workflow
# check if the argument has the expected form
status = sys.argv[1]
if status not in ["successful", "failed", "cancelled"]:
raise RuntimeError("The input does not match the expected format! Permissible are 'successful', 'failed' and 'cancelled'. Aborting!")
# get environment variables
try:
environment = setup_environment_variables()
except RuntimeError as e:
raise RuntimeError("Critical error: Can not uniquely identify environment data! Aborting recording of data.")
# Step 1: store metadata of workflow run persistently
# initiate connection to database
connector = sqlite3.connect("TSF/MemoryEfficientTestResultData.db")
connector.execute("PRAGMA foreign_keys = ON")
# load expected tables
# table workflow_info contains metadata of workflow and is updated every time
command = (
"CREATE TABLE IF NOT EXISTS workflow_info(",
"repo TEXT, ", # repository
"run_id INT, ", # ID of workflow run
"run_attempt INT, ", # Attempt-number of workflow run
"status TEXT ", # Termination-status of workflow
"CHECK(status IN ('successful', 'failed', 'cancelled')) DEFAULT 'failed', ",
"time INT, ", # the time that is associated to this workflow run
"PRIMARY KEY(repo, run_id, run_attempt))"
)
connector.execute(''.join(command))
# table test_results contains detailed results for each individual test
command = (
"CREATE TABLE IF NOT EXISTS test_results(",
"ctest_target TEXT, ", # name of the ctest target located in ci.cmake
"name TEXT, ", # name of the test
"execution_time REAL, ", # execution time in seconds
"compiler TEXT, ", # compiler information
"cpp_standard TEXT, ", # cpp-standard
"passed_cases INT, ", # number of passed test-cases
"failed_cases INT, ", # number of failed test-cases
"skipped_cases INT, ", # number of skipped test-cases
"passed_assertions INT, ", # number of passed assertions
"failed_assertions INT, ", # number of failed assertions
"repo TEXT, ", # repository
"run_id INT, ", # ID of workflow run
"run_attempt INT, ", # Attempt-number of workflow run
"FOREIGN KEY(repo, run_id, run_attempt) REFERENCES workflow_info);"
)
connector.execute(''.join(command))
cursor = connector.cursor()
# Count number of rows as heuristic size-checker.
# In case that the update-check fails, and every result is stored, allow for approximately 26 complete results to be stored
cursor.execute("SELECT MAX(COALESCE((SELECT MAX(rowid) FROM workflow_info),0),COALESCE((SELECT MAX(rowid) FROM test_results),0));")
if cursor.fetchone()[0] > 1e5:
connector.close()
raise RuntimeError("The persistent data storage is too large! Please move persistent data to external storage.")
# fill in metadata
# OBSERVE: This script expects the status of the github workflow as argument
repo = environment.get('GITHUB_REPOSITORY')
run_id = environment.get('GITHUB_RUN_ID')
run_attempt = environment.get('GITHUB_RUN_ATTEMPT')
time = int(datetime.now(timezone.utc).timestamp())
command = "INSERT INTO workflow_info VALUES(?,?,?,?,?)"
cursor.execute(command,(repo, run_id, run_attempt, status, time))
# Don't forget to save!
connector.commit()
# Step 2: generate report of most recent test run and update persistent storage if necessary
# initialise database connection
conn = sqlite3.connect("MemoryEfficientTestResults.db")
cur = conn.cursor()
# add the expected table
# the table TestResults.test_results differs from TestResultData.test_results in that the metadata of the commit are not saved.
command = (
"CREATE TABLE IF NOT EXISTS test_results(",
"ctest_target TEXT, ", # name of the ctest target located in ci.cmake
"name TEXT, ", # name of the test
"execution_time REAL, ", # execution time in seconds
"compiler TEXT, ", # compiler information
"cpp_standard TEXT, ", # cpp-standard
"passed_cases INT, ", # number of passed test-cases
"failed_cases INT, ", # number of failed test-cases
"skipped_cases INT, ", # number of skipped test-cases
"passed_assertions INT, ", # number of passed assertions
"failed_assertions INT", # number of failed assertions
")"
)
conn.execute(''.join(command))
# Load my artifacts
junit_logs = get_all_xml_files("./my_artifacts/")
#extract data
for junit_log in junit_logs:
tree = ET.parse(junit_log)
file_root = tree.getroot()
testsuite = next(file_root.iter('testsuite'), None)
if testsuite is None:
print(f"Error: Could not find testsuite data in {junit_log}.")
continue
for testcase in (case for case in file_root.iter('testcase') if is_unit_test(case)):
metadata = get_metadata(testcase)
target = get_ctest_target(junit_log)
compiler = testsuite.get('name')
more_compiler_info = [case for case in file_root.iter('testcase') if case.get("name") == "cmake_target_include_directories_configure"]
if len(more_compiler_info) != 0:
compiler_information = more_compiler_info[0]
information = list(compiler_information.find("system-out").itertext())[0].split('\n')[0]
compiler = information.replace("-- The CXX compiler identification is ","")
name = metadata.get('name')
cpp_standard = metadata.get('standard')
data = (
target,
name,
metadata.get('execution time'),
compiler,
cpp_standard,
metadata.get('passed test cases'),
metadata.get('failed test cases'),
metadata.get('skipped test cases'),
metadata.get('passed assertions'),
metadata.get('failed assertions')
)
command ="INSERT INTO test_results VALUES(?,?,?,?,?,?,?,?,?,?);"
cur.execute(command, data)
conn.commit()
most_recently_stored_results = find_most_recent_results(target,name,compiler,cpp_standard,connector)
current_results = [metadata.get('passed test cases'),metadata.get('failed test cases'),metadata.get('skipped test cases'),metadata.get('passed assertions'),metadata.get('failed assertions')]
if (len(most_recently_stored_results) != 5) or any(most_recently_stored_results[i]!=current_results[i] for i in range(0,5)):
data = data + (repo, run_id, run_attempt)
command ="INSERT INTO test_results VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?);"
cursor.execute(command,data)
# terminate connection to temporary database
# don't forget to commit the changes
conn.commit()
conn.close()
# terminate connection to persistent database
# don't forget to commit the changes again, for good measure
connector.commit()
connector.close()
Fallacies:
None
Graph:
date-time |
JLS-22 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-23 | Reviewed: ⨯ | Score: 0.0#
The Eclipse S-CORE organization mirrors the nlohmann/json project in a github fork.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All sources and tools for the nlohmann/json library are mirrored in our controlled environment. |
0.18 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json
https://github.com/nlohmann/json
- Start-page of the original nlohmann/json project
website: https://github.com/eclipse-score/inc_nlohmann_json
https://github.com/eclipse-score/inc_nlohmann_json
- Start-page of the mirror of nlohmann/json within Eclipse S-CORE
Fallacies:
None
Graph:
date-time |
JLS-23 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-24 | Reviewed: ⨯ | Score: 0.0#
The nlohmann/json library recognizes malformed JSON and returns an exception.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Prohibited misbehaviours for the nlohmann/json library are identified, and mitigations are specified, verified and validated based on analysis. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
Item |
Summary |
Score |
Status |
|---|---|---|---|
The service provided by the nlohmann/json library throws an exception on ill-formed UTF-8 data. |
0.00 |
⨯ Item Reviewed |
|
The parsing service provided by the nlohmann/json library throws an exception on ill-formed literal names. |
0.00 |
⨯ Item Reviewed |
|
The parsing service provided by the nlohmann/json library throws an exception on ill-formed numbers. |
0.00 |
⨯ Item Reviewed |
|
The parsing service provided by the nlohmann/json library throws an exception on ill-formed strings. |
0.00 |
⨯ Item Reviewed |
|
The parsing service provided by the nlohmann/json library throws an exception on ill-formed arrays. |
0.00 |
⨯ Item Reviewed |
|
The parsing service provided by the nlohmann/json library throws an exception on ill-formed objects. |
0.00 |
⨯ Item Reviewed |
|
The service provided by the nlohmann/json library recognises ill-formed byte-order marks and throws an exception. |
0.00 |
⨯ Item Reviewed |
References:
None
Fallacies:
None
Graph:
date-time |
JLS-24 |
NPF-07.2 |
TIJ-01 |
TIJ-02 |
TIJ-03 |
TIJ-04 |
TIJ-05 |
TIJ-06 |
|---|---|---|---|---|---|---|---|---|
2025-11-26 12:04:09 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
0.00 |
JLS-25 | Reviewed: ⨯ | Score: 0.0#
Malicious code changes in nlohmann/json are mitigated by code reviews, adhering to the contribution guidelines and security policy specified by nlohmann/json.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Prohibited misbehaviours for the nlohmann/json library are identified, and mitigations are specified, verified and validated based on analysis. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
- OpenSSF Scorecard Report for nlohmann/json, where scores for ‘Security-Policy’ and ‘Code-Review’ reflect this statement.
website: https://github.com/nlohmann/json?tab=contributing-ov-file#readme
https://github.com/nlohmann/json?tab=contributing-ov-file#readme
- Contribution Guidelines for nlohmann/json, where it is indirectly indicated that all changes are reviewed.
Fallacies:
None
Graph:
date-time |
JLS-25 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-26 | Reviewed: ⨯ | Score: 0.0#
Any failed CI pipeline executions in the master branch of the nlohmann/json repository are analyzed and fixed.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Collected data from tests and monitoring of deployed software in eclipse-score/inc_nlohmann_json is analysed according to specified objectives. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
failures on branch master of nlohmann/json
0 workflows failed on branch master of nlohmann/json
website: https://github.com/nlohmann/json/actions?query=branch%3Amaster
https://github.com/nlohmann/json/actions?query=branch%3Amaster
- CI workflow history for the master branch of nlohmann/json.
Fallacies:
None
Graph:
date-time |
JLS-26 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-27 | Reviewed: ⨯ | Score: 0.0#
The test coverage for this version of nlohmann/json is monitored using Coveralls and is not decreasing over time, unless reasonably justified.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Expected or required behaviours for the nlohmann/json library are identified, specified, verified and validated based on analysis. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
None
Fallacies:
None
Graph:
date-time |
JLS-27 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-28 | Reviewed: ✔ | Score: 0.0#
Outstanding bugs and misbehaviours are triaged in the nlohmann/json repository.
aschemmel-tech: no triage done
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
In the nlohmann/json repository, known bugs or misbehaviours are analysed and triaged, and critical fixes or mitigations are implemented or applied. |
0.38 |
✔ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/discussions/4967
https://github.com/nlohmann/json/discussions/4967
- Discussion in nlohmann/json showing that no triage processes for bugs are currently in place
Fallacies:
None
Graph:
date-time |
JLS-28 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-29 | Reviewed: ⨯ | Score: 0.0#
Known bugs, misbehaviours and CVEs are analyzed and either fixed or mitigated in the nlohmann/json repository.
aschemmel-tech: evidences support bugs and CVE fixing, no mitigation actions are offered.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
In the nlohmann/json repository, known bugs or misbehaviours are analysed and triaged, and critical fixes or mitigations are implemented or applied. |
0.38 |
✔ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/releases/tag/v3.12.0
https://github.com/nlohmann/json/releases/tag/v3.12.0
- release notes for v3.12.0, listing bugs, CVEs and warnings which were either fixed or mitigated since last release
website: https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
- OpenSSF Scorecard Report for nlohmann/json, where scores for ‘Vulnerabilities’, ‘Pinned-Dependencies’ and ‘Dangerous-Workflow’ support this statement.
Fallacies:
None
Graph:
date-time |
JLS-29 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-30 | Reviewed: ⨯ | Score: 0.0#
Outstanding CVEs are analyzed within eclipse-score/inc_nlohmann_json to determine whether they can be dismissed, and/or are relevant for S-CORE’s use cases of the nlohmann/json library.
aschemmel-tech: In my understanding bot evidences support the statement completely.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
In the nlohmann/json repository, known bugs or misbehaviours are analysed and triaged, and critical fixes or mitigations are implemented or applied. |
0.38 |
✔ Item Reviewed |
Supporting Items:
None
References:
website: https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
- OpenSSF Scorecard Report for nlohmann/json, where score for ‘Vulnerabilities’ shows that there are no outstanding CVEs.
website: https://github.com/nlohmann/json/discussions/4975
https://github.com/nlohmann/json/discussions/4975
- Screenshot of dismissed code scanning alerts, which can also be dismissed in S-CORE.
Fallacies:
None
Graph:
date-time |
JLS-30 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-31 | Reviewed: ⨯ | Score: 0.0#
The nlohmann/json repository uses a static code analysis tool.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Prohibited misbehaviours for the nlohmann/json library are identified, and mitigations are specified, verified and validated based on analysis. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
- OpenSSF Scorecard Report for nlohmann/json, where score for ‘SAST’ supports this statement.
Fallacies:
None
Graph:
date-time |
JLS-31 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-32 | Reviewed: ✔ | Score: 1.0#
All pull requests to the develop branch in the nlohmann/json repository trigger a request for review from Niels Lohmann (@nlohmann).
aschemmel-tech: nlohman is the repository codeowner
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
nlohmann/json library components, configurations and tools are updated under specified change and configuration management controls. |
0.65 |
✔ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/blob/develop/.github/CODEOWNERS
https://github.com/nlohmann/json/blob/develop/.github/CODEOWNERS
- CODEOWNERS file specifying that changes to any file requests @nlohmann for code review in case of a pull request
Fallacies:
None
Graph:
date-time |
JLS-32 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
1.00 |
2025-12-16 13:14:25 |
1.00 |
2025-12-16 14:03:34.403563 |
1.00 |
JLS-33 | Reviewed: ✔ | Score: 0.75#
Outstanding CVEs are triaged in the nlohmann/json repository.
aschemmel-tech: Security vulnerabilities reporting asks for severities, but this is given by the reporter
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
In the nlohmann/json repository, known bugs or misbehaviours are analysed and triaged, and critical fixes or mitigations are implemented or applied. |
0.38 |
✔ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/security/advisories/new
https://github.com/nlohmann/json/security/advisories/new
- the vulnerability reporting template used in nlohmann/json, which includes a severity section and CVSS scoring
Fallacies:
None
Graph:
date-time |
JLS-33 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.75 |
2025-12-16 13:14:25 |
0.75 |
2025-12-16 14:03:34.403563 |
0.75 |
JLS-34 | Reviewed: ✔ | Score: 0.9#
The nlohmann/json library has no external components or dependencies besides the C++ standard components.
aschemmel-tech: I saw dependency to the “Hedley” header but with fixed version.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All inputs to the nlohmann/json library are assessed, to identify potential risks and issues. |
0.32 |
⨯ Item Reviewed |
|
All sources and tools for the nlohmann/json library are mirrored in our controlled environment. |
0.18 |
⨯ Item Reviewed |
|
nlohmann/json library components, configurations and tools are updated under specified change and configuration management controls. |
0.65 |
✔ Item Reviewed |
Supporting Items:
None
References:
List of included libraries for: single_include/nlohmann/json.hppDescription: file containing all include occurrences of the score-project.
#include <algorithm> #include <cstddef> #include <functional> #include <initializer_list> #include <iosfwd> #include <iterator> #include <memory> #include <string> #include <utility> #include <vector> #include <utility> #include <algorithm> #include <array> #include <forward_list> #include <iterator> #include <map> #include <optional> #include <string> #include <tuple> #include <type_traits> #include <unordered_map> #include <utility> #include <valarray> #include <cstddef> #include <exception> #include <numeric> #include <stdexcept> #include <string> #include <vector> #include <array> #include <cstddef> #include <cstdint> #include <string> #include <utility> #include <type_traits> #include <stdint.h> #include <stdint.h> #include <version> #include <cstdlib> #include <cassert> #include <compare> #include <cstddef> #include <array> #include <cstddef> #include <type_traits> #include <utility> #include <limits> #include <string> #include <tuple> #include <type_traits> #include <utility> #include <iterator> #include <cstdint> #include <map> #include <memory> #include <string> #include <vector> #include <cstring> #include <string> #include <utility> #include <experimental/filesystem> #include <filesystem> #include <optional> #include <algorithm> #include <iterator> #include <string> #include <tuple> #include <type_traits> #include <utility> #include <valarray> #include <vector> #include <cstddef> #include <iterator> #include <tuple> #include <utility> #include <ranges> #include <cstddef> #include <string> #include <cstdint> #include <tuple> #include <utility> #include <cstdint> #include <cstddef> #include <functional> #include <algorithm> #include <array> #include <cmath> #include <cstddef> #include <cstdint> #include <cstdio> #include <cstring> #include <iterator> #include <limits> #include <string> #include <utility> #include <vector> #include <bit> #include <array> #include <cstddef> #include <cstring> #include <iterator> #include <memory> #include <numeric> #include <string> #include <type_traits> #include <utility> #include <cstdio> #include <istream> #include <cstddef> #include <string> #include <type_traits> #include <utility> #include <vector> #include <array> #include <clocale> #include <cstddef> #include <cstdio> #include <cstdlib> #include <initializer_list> #include <string> #include <utility> #include <vector> #include <cstdint> #include <utility> #include <string> #include <cmath> #include <cstdint> #include <functional> #include <string> #include <utility> #include <vector> #include <cstddef> #include <limits> #include <iterator> #include <type_traits> #include <cstddef> #include <iterator> #include <utility> #include <type_traits> #include <algorithm> #include <cctype> #include <cerrno> #include <cstdlib> #include <iosfwd> #include <limits> #include <numeric> #include <string> #include <utility> #include <vector> #include <initializer_list> #include <utility> #include <algorithm> #include <array> #include <map> #include <cmath> #include <cstdint> #include <cstring> #include <limits> #include <string> #include <utility> #include <vector> #include <algorithm> #include <cstddef> #include <iterator> #include <memory> #include <string> #include <vector> #include <ios> #include <ostream> #include <algorithm> #include <array> #include <clocale> #include <cmath> #include <cstddef> #include <cstdint> #include <cstdio> #include <limits> #include <string> #include <iomanip> #include <type_traits> #include <utility> #include <array> #include <cmath> #include <cstdint> #include <cstring> #include <limits> #include <type_traits> #include <functional> #include <initializer_list> #include <iterator> #include <memory> #include <stdexcept> #include <type_traits> #include <utility> #include <vector> #include <any> #include <string_view>
Fallacies:
None
Graph:
date-time |
JLS-34 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.90 |
2025-12-16 13:14:25 |
0.90 |
2025-12-16 14:03:34.403563 |
0.90 |
JLS-35 | Reviewed: ⨯ | Score: 0.0#
Pull requests in the nlohmann/json repository are merged only after running CI-tests and successfully passing the pipeline.
aschemmel-tech: evidences support statement
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
nlohmann/json library components, configurations and tools are updated under specified change and configuration management controls. |
0.65 |
✔ Item Reviewed |
Supporting Items:
None
References:
website: https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
- OpenSSF Scorecard Report for nlohmann/json, where score for ‘CI-Tests’ supports this statement
website: https://github.com/nlohmann/json/pulls?q=is%3Apr+is%3Aclosed+review%3Aapproved
https://github.com/nlohmann/json/pulls?q=is%3Apr+is%3Aclosed+review%3Aapproved
- All approved pull requests in the nlohmann/json repository, with the results of the CI pipeline executions.
Fallacies:
None
Graph:
date-time |
JLS-35 |
|---|---|
2025-11-26 12:04:09 |
0.00 |
2025-12-10 14:14:18 |
0.00 |
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-36 | Reviewed: ⨯ | Score: 0.0#
Updates of the eclipse-score/inc_nlohmann_json repository are carried out in accordance with the defined and documented update process in TSF/README.md.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Manual methodologies applied for the nlohmann/json library by contributors, and their results, are managed according to specified objectives. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
TSF/README.md
# TSF Structure
The TSF-related additions, such as the Trustable Graph and tooling extensions, are primarily organized within the TSF folder:
- the items of the Trustable Graph are in `TSF/trustable`
- the building blocks of the documentation, which is built with the S-CORE docs-as-code tool, is placed in `TSF/docs`
- the report generated by [trudag](https://codethinklabs.gitlab.io/trustable/trustable/trudag/usage.html) is placed in `TSF/docs/generated`. This can either be produced as part of the CI pipeline or manually using the shell script `./TSF/scripts/generate_documentation.sh`. It is strongly recommended that the TSF/docs folder is included in the .gitignore file
- the utility scripts are documented in the `TSF/scripts` folder
The TSF graph (including links, nodes and their hashes) is saved in the `.dotstop.dot` file and the trudag extensions including CPP test references are stored in the `.dotstop_extensions` folder since these locations are required by the trudag tool.
# Forking the repository
In order to fork this repository or set up any repository where the TSF documentation in this repository is to be included, the following settings have to be configured on GitHub. In addition to the below settings, make sure to also configure appropriate branch protection rules.
- In `Settings` > `General` >`Features`:
- Enable `Issues`
- In `Settings` > `Code and automation` > `Actions` > `General` > `Workflow Permissions`:
- Make sure that only the following settings are enabled:
- `Allow all actions and reusable workflows`
- `Require approval for first-time contributors`
- `Read repository contents and packages permissions`
- In `Settings` > `Code and automation` > `Pages`:
- Under `Source`, select `GitHub Actions`
- In `Settings` > `Security` > `Advanced Security`:
- Make sure that only the following settings are enabled:
- `Dependency graph`
- `Dependabot alerts`
- In `Actions tab`:
- Click `I understand my workflows, go ahead and enable them`
- In the left side menu, click `Show more workflows...` and enable any workflows which are labelled as `Disabled`
# Release management
The releases process of this repository shall conform to the [release management plan of Eclipse S-CORE](https://github.com/eclipse-score/score/blob/668bae4d1a704565983d34b8447d5a035696299a/docs/platform_management_plan/release_management.rst#id11). Most notably, the release tags shall follow semantic versioning format.
- The components of the tag shall be incremented both in case of an update to TSF documentation, and in case of updating to a new release of the upstream nlohmann/json library.
- Updates to TSF documentation shall increment the PATCH or MINOR component.
- Updating the version of nlohmann/json to a new release shall increment the appropriate tag component based on the extensiveness and nature of the upstream changes.
- To indicate the version of nlohmann/json in use, the nlohmann/json release tag shall always be clearly included in the release notes of this repository.
- The release notes of this repository shall always indicate whether the release includes changes to only TSF documentation, only the version of nlohmann/json, or both.
To update either the version of nlohmann/json within S-CORE or TSF documentation, please refer to the respective Update Concepts below.
# Update Concept for the version of nlohmann/json within S-CORE
## Assumptions of use
This description of the update process is based on the following structure of the repository.
It is assumed that the repository has a default branch called ``main`` containing the most recent documented version of ``nlohmann/json`` together with its documentation.
The releases of the documented nlohmann/json version are identified by release notes on ``main``.
Note that there is **no automatic information** on the existence of a new release in the original ``nlohmann/json``; instead the possibility to update is detected **manually**.
Note further that, due to the currently relatively limited use of nlohmann/json within S-CORE, there appears currently no inherent need to keep the version up to date.
## Update process of the original nlohmann/json
The releases of ``nlohmann/json`` are collected on the [Release site](https://github.com/nlohmann/json/releases) of the repository ``nlohmann/json``.
Each release announcement is expected to contain the release date, SHA-256 values for json.hpp, include.zip and json.tar.xz, and a brief list containing bug fixes, improvements, further changes and deprecated functions.
The new release is expected to be located within the branch **master** in nlohmann/json, from where the most recent version can be drawn.
## Update process of the S-CORE version
In the following, we shall describe the intricacies of updating the version of ``nlohmann/json`` within Eclipse S-CORE.
This version is not a mere fork of the original master branch of ``nlohmann/json``, but instead enriched with the documentation following the Trustable Software Framework (TSF).
The enrichment with the documentation necessitates some changes to the fork of the original repository.
For the most part, these changes are in-obtrusive, and mere additions.
In particular, the folders ``include`` and ``single-include`` remain unchanged, and should be updated without further adaptations.
In some cases, however, additional tests are run and data are generated and collected, which were not run or generated in the original ``nlohmann/json``, so that obtrusive changes of files were necessary.
For these files, and in particular the workflow files, caution must be exercised, as to not disturb the documentation.
Moreover, some parts of the documentation must be adapted to the new version.
### What can not be updated without further precautions?
* ``cmake/ci.cmake``
This file defines, in particular, the various custom cmake targets; in particular, the various configurations for the execution of the unit- and integration-tests are defined.
The TSF requires, or, at the very least, strongly encourages us to collect test-results.
In order to do this efficiently, the ctest command is adapted to automatically generate the junit-logs of each test-run.
For this, the option ``--output-junit`` is set with output path ``../my_logs/TARGETNAME_junit.xml``, where TARGETNAME is replaced by the name of the respective cmake target; in case that this convention is insufficient to uniquely identify the logs, TARGETNAME is amended by a number.
When updating, it must be ensured that these adaptations are preserved.
Moreover, if the update introduces new cmake targets or new executions of ctest, it must be ensured, that the junit-log is generated and stored with a similar naming convention in the folder "../my_logs/".
Otherwise, it can not be ensured that the test data are accurately captured.
* ``cmake/download_test_data.cmake``
This file is modified to ensure that the test-data are not downloaded from the original test-data repository, but instead from the copy of that repository within the Eclipse S-CORE organisation.
It must be ensured that this change is preserved.
* ``tests/CMakeLists.txt``
This file collects, in particular, the files containing the unit- and integration-tests in a list, which is given to cmake.
Custom tests were added in TSF/tests to document the fulfillment of the expectations.
To ensure that these tests are run, the file tests/CMakeLists.txt has been modified.
During the update, it must be ensured, that the custom tests are still being executed.
* ``.github/workflows/parent-workflow.yml``
To ensure a specific execution order for the individual github workflows, their execution is orchestrated by the parent-workflow.
To guarantee that this order is respected, it must be ensured that every workflow except for ``comment_check_amalgamation.yml``, ``docs-cleanup.yml``, ``parent-workflow.yml``, ``scorecards.yml`` and ``stale.yml`` runs ``on workflow_call``, only.
* ``.github/workflows/ubuntu.yml``
The ubuntu workflow orchestrates the parallel execution of various cmake targets with varying configurations running on the latest version of ubuntu.
The first TSF related adaptation of this workflow is that every step, in which a junit-report is generated, generates an artifact.
It must be ensured, that these artifacts are still generated after the update.
The second adaptation is that the test-results are captured, processed and persistently stored or stored in the ubuntu-artifact.
Therefore, it must be ensured that the jobs ``publish_test_data_success``, ``publish_test_data_failure``, ``publish_test_data_cancellation`` and ``ubuntu_artifact`` are executed.
Moreover, in case that any further job is added by nlohmann, it must be ensured that this job is added to the list of jobs required before the latter workflows are executed.
If any further job added by nlohmann generates a junit-log, it must be ensured that this job generates an artifact containing its junit-logs.
* ``.github/workflows/cifuzz.yml``
This workflow uses Google's oss-fuzz, which is not available to the copy within Eclipse S-CORE.
Therefore, this workflow needs to be disabled in the copy.
Currently, this is done by removing it altogether, which we recommend to do so that no confusion as to why this workflow is not executed arises.
* ``.github/workflows/publish_documentation.yml``
The original version of this workflow is replaced with a completely customised version, which reflects the use of trudag and the integration into the Eclipse S-CORE organisation.
Therefore, it is recommended to not change this workflow.
In particular, the version of publish_documentation.yml in the original repository nlohmann/json must not replace the publish_documentation.yml of the present repository.
* ``.github/workflows/test_trudag_extensions.yml``
This workflow is not present in the original nlohmann/json and must not be removed, or modified (besides updating the versions of tools, if necessary) by the update.
* Other entries of ``.github/workflows``
For every workflow, it must be ensured that the conditions of their execution are unchanged.
The workflows ``check_amalgamation``, ``codeql``, ``dependency_review``, ``labeler`` and ``test_trudag_extensions`` generate an artifact, which must not be changed.
New workflows should be carefully reviewed.
If it is determined that their execution within the project is beneficial, and that they do not interfere with, then they should be integrated within the parent workflow at an appropriate place and their execution condition should be set to on ``workflow``, or their execution should be scheduled appropriately.
It is strongly recommended that the new workflow produces an artifact on success, and that the validator ``check_artifact_exists`` is adapted accordingly.
If nlohmann deletes any of the currently executed workflows, in particular ``check_amalgamation.yml``, ``codeql.yml``, ``dependency_review.yml``, ``labeler.yml``, ``test_trudag_extensions.yml`` and ``ubuntu.yml``, then it is strongly recommended to keep the currently executed version, since the automatic validator ``check_artifact_exists`` depends on the existence of these workflows.
In case that it is determined that these workflows should be deleted also in the documented copy of ``nlohmann/json``, then the validator ``check_artifact_exists`` and all its occurrences must be adapted accordingly.
* ``ChangeLog.md``
It must be ensured that the changes of the new release of nlohmann/json are properly described in the file ``ChangeLog.md``.
### Necessary adaptations
The following adaptation is recommended, and has, unfortunately, not been automated.
* ``TSF/trustable/statements/JLS-02.md``
It must be carefully ensured that this statement and its references are still valid. In particular, it is strongly recommended to refer to a fuzz testing result running on the version that is updated to.
The following adaptations to the documentation have been automated; the python-script TSF/scripts/update_helper.py may be used to assist with these changes.
For the error-free execution is it necessary, however, to adhere to the naming scheme json_version_X_XX_X, and to not change the structure of the directories.
* ``TSF/Trustable/statements/JLS-11.md``
It must be ensured that the correct release date is used.
* ``TSF/trustable/statements/JLS-14.md``
It must be ensured that the release of the correct version is referenced.
Furthermore, the sha-value of the evidence must be adapted to the one provided in that announcement post.
* ``TSF/trustable/docs/introduction/index.rst``
In this file, the version of ``nlohmann/json`` that is documented is explicitly mentioned at two places.
This version must be updated.
* ``TSF/scripts/generate_list_of_misbehaviours.py``
This script contains version and release date hard-coded. Both must be updated.
### Recommended procedure
Based on the above observations, the following steps are recommended for each update to the library.
1. Merge branch master from the original nlohmann/json into an external fork of the eclipse-score/inc_nlohmann_json repository, where steps 2-12 shall be performed.
2. Confirm the deletion of cifuzz.yml, macos.yml and windows.yml.
3. Resolve the potential merge conflict in publish-documentation.yml by rejecting the incoming changes.
4. Update the versions of the github actions, if necessary.
5. Resolve the potential merge conflicts in check_amalgamation.yml, codeql.yml, dependency_review.yml, labeler.yml and test_trudag_extensions.yml to ensure that the artifacts are generated, i.e. the jobs ``Generate XXX artifact`` and ``Upload XXX artifact`` are retained.
6. Resolve the potential merge conflict in ubuntu.yml following the above instructions.
7. Resolve the potential merge conflicts in cmake/download_test_data.cmake and cmake/ci.cmake following the above instructions.
8. Carefully examine the automatically merged changes. If no interference is to be expected, complete the merge.
9. In case any additional workflow has been added, carefully examine and integrate into the parent-workflow or schedule appropriately.
10. Adapt the documentation as described above.
11. Generate the TSF report locally and carefully investigate any change in the trustable score(s).
If any relevant behaviour of the library changes, adapt the documentation.
Additionally, if any additional tests were added, or existing tests were changed, carefully investigate whether these warrant an amendment of the documentation.
12. Merge into the ``main`` branch of eclipse-score/inc_nlohmann_json.
13. Create a new release tag in eclipse-score/inc_nlohmann_json, following semantic versioning.
# Update concept for the TSF documentation
## Assumptions of use
The documentation follows the Trustable Software Framework (TSF), which is documented [here](https://codethinklabs.gitlab.io/trustable/trustable/print_page.html).
Furthermore, the automatic generation of the TSF report and the tracking of changes to the core functionalities of the library uses _trudag_, which is developed by Codethink and located [here](https://gitlab.com/CodethinkLabs/trustable/trustable).
## Version of trudag
The documentation is currently built using trudag version v2025.10.22
In case of new releases of trudag in the future, it is recommended to carefully review the introduced changes and rigorously test it before merging it into eclipse-score/inc_nlohmann_json.
The following should be considered:
* How has the algorithm for the accumulation of the trustable score changed? Ideally, it is not changed, otherwise the necessity for a new review arises.
* How has the data store interface changed? Ideally, it is not changed, but if it does and the expected schema format changes, data_store.py needs to be updated accordingly.
* How has the the expected configuration for the TSF items changed? It is known that this configuration changed (at least) once before. What does the potential change mean?
* Do all custom references and validators as well as the data store interface work as before?
* Has the algorithm for the hashing changed, or are there any changes to the trustable scores? If so, investigate carefully!
## Subject-Matter-Expert-scores
The intention with the SME scores is to find the _true_ trustable score by means of a heuristic law-of-large-numbers argument.
Therefore, it is very much welcome for contributors to add their SME scores to statements for which they feel confident to do so.
While the committer may check SME scores for plausibility, it is recommended to not question SME scores as this interferes with the assumed independence of the SME!
It is highly recommended to not delete SME scores under usual circumstances; most certainly, the SME scores should never be changed by anybody except the original SME.
The following unusual circumstances can, after careful consideration, justify the removal or (much preferably!) the request for re-evaluation by the original SMEs:
* change of references:
If, e.g. due to an update of ``nlohmann/json``, the references of any items (be it tests or code) changes, then this should trigger a re-evaluation of the statement.
In particular if the behaviour changed significantly, it can be justifiable to assume that the old SME scores do not reflect the statement anymore.
* addition of automatic validators:
Recall that the SME-scores have different meanings depending on whether or not an automatic validator is implemented.
In the absence of a validator, the SME shall assess their confidence in the statement based on linked artifacts (references) and their own knowledge.
In the presence of a validator, the SME shall assess only their confidence in the validator as an accurate measure of the truth of the statement.
Some important remarks regarding the process of performing an SME review include:
* Assumptions-of-Use (AoU) items are not to be scored by an SME. These should at some point in time be converted to a regular statement by the integrator, but shall until then remain without SME scores, evidence or references. These automatically contribute with a score of 0 for the time being.
* Any statement that has supporting statements (child nodes) shall not be scored by an SME. The scores for these shall only comprise of the automatic calculation of the mean of the scores of all linked child nodes.
* The TSF report is only published for the default branch of a repository. In order to to see TSF documentation for a non-default branch, the reviewer can checkout the branch in a clone and locally generate the report by running the ``TSF/scripts/generate_documentation.sh`` script.
* Any TSF item that has the ``normative`` parameter set to false shall not be scored by the SME. These are just informational and do not make a statement about the project. Note that the parameter itself shall not be set or changed by the SME, but is defined by the contributor who created the TSF item.
For documentation on how to actually perform a SME review, please refer to the following:
* https://codethinklabs.gitlab.io/trustable/trustable/trudag/usage.html#reviewing-items
* https://codethinklabs.gitlab.io/trustable/trustable/trudag/usage.html#sme-review
## Validators
The automatic validators are intended to calculate a trustable score based on quantifiable data.
In particular the introduction of a validator changes the meaning of the (potential) SME scores associated to a statement.
Therefore, the change or introduction of an automatic validator is most critical.
In case the validator used in a statement has been changed, it is highly recommended to urge the original SME to re-review the statement and adapt their scores, or (at the very least) to enlist additional SMEs to judge the changed statement.
After careful consideration the highly critical decision to remove some SME scores no longer reflecting the statement could be made.
## References
References should be treated in the same way as validators, i.e. any update of a reference should trigger a re-review by the SME.
For references, however, the decision to remove a stale SME score is even more critical unless the reference reveals critical new information, which is highly unlikely, or the change of the reference is triggered by a significant change in the behaviour of the library, which heavily affected the statement.
Description: release management and update process description
Fallacies:
None
Graph:
date-time |
JLS-36 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-37 | Reviewed: ⨯ | Score: 0.0#
High-level statements are decomposed into smaller, recursive statements.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Confidence in the nlohmann/json library is measured based on results of analysis. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://eclipse-score.github.io/inc_nlohmann_json/main/generated/trustable_report_for_Software.html#compliance-for-ta
https://eclipse-score.github.io/inc_nlohmann_json/main/generated/trustable_report_for_Software.html#compliance-for-ta
- Trustable Compliance Report showing scores for different TA items.
website: https://eclipse-score.github.io/inc_nlohmann_json/main/generated/trustable_graph.html
https://eclipse-score.github.io/inc_nlohmann_json/main/generated/trustable_graph.html
- presentation of the full trustable graph in which high-level statements are broken down
Fallacies:
None
Graph:
date-time |
JLS-37 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-38 | Reviewed: ⨯ | Score: 0.0#
Every release of the nlohmann/json library shall provide configuration management for build instructions and infrastructure.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Construction of releases for the nlohmann/json library is fully repeatable and the results are fully reproducible, with any exceptions documented and justified. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://json.nlohmann.me/integration/cmake/
https://json.nlohmann.me/integration/cmake/
- cmake build management documentation for nlohmann/json
website: https://json.nlohmann.me/integration/package_managers/
https://json.nlohmann.me/integration/package_managers/
- package manager documentation for nlohmann/json
Fallacies:
None
Graph:
date-time |
JLS-38 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-40 | Reviewed: ⨯ | Score: 0.0#
The manual process for reporting vulnerabilities in the nlohmann/json library is well defined and documented in the project’s security policy and vulnerability reporting template.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Manual methodologies applied for the nlohmann/json library by contributors, and their results, are managed according to specified objectives. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/security/policy
https://github.com/nlohmann/json/security/policy
- Security policy describing how to report vulnerabilities for the nlohmann/json library
website: https://github.com/nlohmann/json/security/advisories/new
https://github.com/nlohmann/json/security/advisories/new
- Well-defined process for issuing a vulnerability or bug report for the nlohmann/json library
Fallacies:
None
Graph:
date-time |
JLS-40 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-41 | Reviewed: ⨯ | Score: 0.0#
The manual process for extending the nlohmann/json library’s test suite is well-defined and documented, such that code coverage remains at (or close to) 100% when fixes or other non-trivial changes are proposed.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Manual methodologies applied for the nlohmann/json library by contributors, and their results, are managed according to specified objectives. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://json.nlohmann.me/community/contribution_guidelines/#write-tests
https://json.nlohmann.me/community/contribution_guidelines/#write-tests
- Contribution guidelines describing the test policy and requirement to maintain 100% coverage
Fallacies:
None
Graph:
date-time |
JLS-41 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-42 | Reviewed: ⨯ | Score: 0.0#
All manual pull requests to the nlohmann/json repository that introduce non-trivial changes are expected to explain the rationale for the proposed change and to link to an existing issue, in accordance with the project’s contribution guidelines.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Manual methodologies applied for the nlohmann/json library by contributors, and their results, are managed according to specified objectives. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/blob/develop/.github/CONTRIBUTING.md#describe-your-changes
https://github.com/nlohmann/json/blob/develop/.github/CONTRIBUTING.md#describe-your-changes
- Contribution guidelines requiring manual pull requests to describe the rationale behind non-trivial changes
website: https://github.com/nlohmann/json/blob/develop/.github/CONTRIBUTING.md#reference-an-existing-issue
https://github.com/nlohmann/json/blob/develop/.github/CONTRIBUTING.md#reference-an-existing-issue
- Contribution guidelines requiring manual pull requests to link to an existing issue
website: https://github.com/nlohmann/json/pulls
https://github.com/nlohmann/json/pulls
- GitHub pull requests showing review discussions, approvals, and merge/close status
Fallacies:
None
Graph:
date-time |
JLS-42 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-43 | Reviewed: ⨯ | Score: 0.0#
Feature requests for the nlohmann/json repository are raised in the project’s GitHub discussions and are actively reviewed and answered by the maintainer.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Manual methodologies applied for the nlohmann/json library by contributors, and their results, are managed according to specified objectives. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/discussions/categories/ideas
https://github.com/nlohmann/json/discussions/categories/ideas
- Feature request discussions showing that feature requests are actively investigated and answered
website: https://github.com/nlohmann/json/blob/develop/.github/CODEOWNERS
https://github.com/nlohmann/json/blob/develop/.github/CODEOWNERS
- Definition of responsible owners and reviewers for the nlohmann/json repository
Fallacies:
None
Graph:
date-time |
JLS-43 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-44 | Reviewed: ⨯ | Score: 0.0#
The manual process for reporting bugs in the nlohmann/json library is well defined and documented in the project’s contribution guidelines and bug report template.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Manual methodologies applied for the nlohmann/json library by contributors, and their results, are managed according to specified objectives. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/blob/develop/.github/ISSUE_TEMPLATE/bug.yaml
https://github.com/nlohmann/json/blob/develop/.github/ISSUE_TEMPLATE/bug.yaml
- Bug report issue template for the nlohmann/json library
website: https://github.com/nlohmann/json/blob/develop/.github/CONTRIBUTING.md#reporting-issues
https://github.com/nlohmann/json/blob/develop/.github/CONTRIBUTING.md#reporting-issues
- Contribution guidelines describing how to report bugs and issues for the nlohmann/json library
Fallacies:
None
Graph:
date-time |
JLS-44 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-45 | Reviewed: ⨯ | Score: 0.0#
The storage location, retention limits, and intended use of captured test result data for eclipse-score/inc_nlohmann_json are documented so that the scope and reproducibility of the available test data can be understood.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Data in eclipse-score/inc_nlohmann_json is collected from tests, and from monitoring of deployed software, according to specified objectives. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
TSF/scripts/README.md
# Scripts used for the documentation
The following scripts are used in the automatic compilation of the trustable report.
## capture_test_data.py
The python-script [capture_test_data.py](capture_test_data.py) is intended to run at the end of the [ubuntu-workflow](../../.github/workflows/ubuntu.yml). It collects and combines the test-results that were collected in the individual jobs, appends the persistent test data storage and generates a database containing the results of the most recent tests only. Since storage space on the github is limited, the storage of test data is limited to two weeks only. It must be noted that this implementation is only intended as a temporary proof of concept!
## capture_test_data_memory_sensitive.py
The python script [capture_test_data_memory_sensitive.py](capture_test_data_memory_sensitive.py) is an alternative to [capture_test_data.py](capture_test_data.py), which is more memory sensitive.
Instead of capturing the full test-data, a first snapshot of the data is captured, which is then updated whenever a relevant change is detected.
The characterisation of "relevant change" follows the following heuristic argumentation:
* The execution time of each test is of lesser interest, therefore it is not continuously updated.
* A test is uniquely identified by the columns ctest_target, name and cpp_standard.
Moreover, a size-check is performed on the persistent storage, using the following heuristic assumptions
* It is expected, that an update of a test-result only happens, when a test fails, which leads automatically to the failure of the pipeline, which should trigger a manual review by the maintainer, or failed before.
* It is expected, that tests fail rarely, and in particular on average less than 10 different test-cases per workflow-run fail.
* It is expected, that less than 1,000 workflow runs, where a record of the test-result is triggered, happen per year, since these are only triggered once daily and on each push to main.
* It is expected, that a ten-year-record of test-results is sufficient for documentation purposes.
In view of these assumptions, we limit the storage to approximately 100,000 test-results and 100,000 workflow-metadata, which guarantees *en passant* that github's file size limit of 100MB is respected.
In the worst case that every recorded test result is detected as a relevant change, this restraint collects test-results from 27 workflows in total.
Whenever either limit of 100,000 test-results and 100,000 workflow-metadata in the persistent data storage is violated, the script returns the error "The persistent data storage is too large! Please move persistent data to external storage.", thereby failing the workflow, which advises the maintainer to take up action.
## clean_trudag_report.py
The python-script [clean_trudag_report.py](clean_trudag_report.py) runs at the end of generate_report.sh.
By default, the report generated by trudag has the format that is expected by MkDocs.
This format, however, is not properly rendered by sphinx, so that an adaption of the output is necessary.
This adaption happens within this script.
## generate_report.sh
The shell-script [generate_report.sh](generate_report.sh) is used to generate the trustable report and the graphs of which the documentation consists.
In particular, it first cleans up potentially left-over obsolete reports, then generates the report and the full view of the trustable graph using trudag.
Thereafter, plot_partial_graphs.py is called, before finally clean_trudag_report.py is executed.
## plot_partial_graphs.py
The python-script [plot_partial_graphs.py](plot_partial_graphs.py) is used to generate a myriad of partial graphs with links amongst them.
These are then used to augment the full graph, which is, due to its size, quite hard to display, in the documentation.
By the means of the links of the items, a more or less \"navigable\" version of the trustable graph is created, which displays a handful of items at a time, only.
# Helper-scripts for offline development
## fetch_json.cpp
The C++ script [fetch_json.cpp](fetch_json.cpp) is used to extract lists of paths to json testsuites from unit-testsuites.cpp, which are then used in testsuite-references. This script is purely used during the manual compilation of the references.
## generate_documentation.sh
The shell-script [generate_documentation.sh](generate_documentation.sh) generates a locally hosted version of the documentation, which is used for testing purposes.
## generate_list_of_tests.py
The python script [generate_list_of_tests.py](generate_list_of_tests.py) is used to generate the [list_of_test_environments.md](../docs/list_of_test_environments.md) listing the expected test-cases with their execution environments.
The file [list_of_test_environments.md](../docs/list_of_test_environments.md) is then persistently stored within the documentation.
Additionally, this script is utilised within the validator ``check_list_of_tests`` to generate a list of test environments from the most recent test results and to compare this list with the persistently stored one.
In case that new tests are added, tests are removed or test environments change, it is therefore necessary to have an accurate list_of_test_environments.md, which is highly recommended to be generated using this script.
To run the script, the database `MemoryEfficientTestResults.db` must be downloaded from the artifact of the most recent successful ubuntu workflow, and placed in the folder `artifacts` in the root of the repository.
Moreover, it must be ensured that this script is run from within the root of the repository, only.
## generate_subgraph_plots.sh
The shell-script [generate_subgraph_plots.sh](generate_subgraph_plots.sh) is used to generate subgraphs of the trustable graph using trudag plot with the --pick option.
Since the options of adding custom hyperlinks to the individual nodes of the generated subgraphs are not quite as flexible as desired for the \"navigable\" version of the trustable graph, we opted to use the custom script plot_partial_graphs.py.
Description: Explanation of how test-result data is stored, how long it is retained, which size limits apply, and which maintenance actions are required for the persistent storage.
TSF/scripts/capture_test_data_memory_sensitive.py
import sys
import sqlite3
import os
import xml.etree.ElementTree as ET
import re
from datetime import datetime, timezone
def setup_environment_variables() -> dict[str, str]:
# Retrieves and validates the necessary environment variables for GitHub workflows.
# Raises a RuntimeError if any required variables are missing.
required_vars = ["GITHUB_RUN_ID", "GITHUB_REPOSITORY", "GITHUB_RUN_ATTEMPT"]
environment = {var: os.getenv(var) for var in required_vars}
missing_vars = [var for var, value in environment.items() if not value]
if missing_vars:
raise RuntimeError(f"Missing required environment variables: {', '.join(missing_vars)}")
return environment
def clean_test_case(testcase: str) -> tuple[str,str]:
# This function expects a testcase of the form "testcase_name_cppxx".
# It returns the tuple ["testcase_name","gnu++xx"].
name, appendix = testcase.rsplit('_',1)
return [name, "gnu++"+appendix.replace('cpp','')]
def read_result_table(input: list[str]) -> dict:
"""
This function expects console output <system-out> of doctest.
It is assumed that this has the following form
<system-out>[doctest] doctest version is "2.4.11"
[doctest] run with "--help" for options
===============================================================================
[doctest] test cases: 1 | 1 passed | 0 failed | 0 skipped
[doctest] assertions: 45 | 45 passed | 0 failed |
[doctest] Status: SUCCESS!
</system-out>
It extracts the number of passed/failed/skipped test cases, and passed/skipped assertions.
"""
metadata = dict()
raw_data = input[0]
data = re.findall(r'(\d+)\s+(passed|failed|skipped)\b', raw_data)
if len(data) < 5:
raise RuntimeError("Fatal Error: Received incomplete or wrong result table.")
metadata["passed test cases"] = int(data[0][0])
metadata["failed test cases"] = int(data[1][0])
metadata["skipped test cases"] = int(data[2][0])
metadata["passed assertions"] = int(data[3][0])
metadata["failed assertions"] = int(data[4][0])
return metadata
def get_metadata(testcase: ET.Element) -> dict:
# expects testcase extracted from a junit xml-file as input
# extracts the data interesting to us
# Assumption of Use: before execution, it is checked if is_unit_test(testcase)==True
metadata = dict()
# from name both name of the test and C++ standard can be extracted
unsplit_name = testcase.get("name", None)
if unsplit_name is None:
raise RuntimeError("Fatal error: Can not read name of test-case!")
name, standard = clean_test_case(unsplit_name)
metadata["name"] = name
metadata["standard"] = standard
metadata["execution time"] = float(testcase.get("time"))
# results are not as easily extracted but must be processed further
metadata = metadata | read_result_table(list(testcase.find("system-out").itertext()))
return metadata
def is_unit_test(testcase: ET.Element) -> bool:
# crude test if the element belongs to a unit-test
return "_cpp" in testcase.get('name')
def get_all_xml_files(directory: str = '.') -> list[str]:
# search the folder containing all the artifacts and list the paths of expected test-reports
result = []
try:
content = os.listdir(directory)
except FileNotFoundError as e:
print(e)
return result
for entry in content:
if os.path.isdir(directory+'/'+entry):
result = result + get_all_xml_files(directory+'/'+entry)
if entry.endswith('.xml'):
file = directory+'/'+entry if directory != '.' else entry
result.append(file)
return result
def get_ctest_target(log_name: str) -> str:
# extracts name of ctest target from junit log
# log_name has the form "path/to/log/file/target_junit.xml", and target is expected
log = log_name.split('/')[-1]
return log.removesuffix("_junit.xml")
def find_most_recent_results(target: str, name: str, compiler: str, cpp_standard: str, database: sqlite3.Connection) -> list[int]:
cursor = database.cursor()
cursor.execute("""
WITH combination AS (
SELECT workflow_info.repo, workflow_info.run_id, workflow_info.run_attempt, workflow_info.time
FROM test_results INNER JOIN workflow_info ON
workflow_info.repo = test_results.repo
AND workflow_info.run_id = test_results.run_id
AND workflow_info.run_attempt = test_results.run_attempt
WHERE test_results.ctest_target = ? AND test_results.name = ? AND test_results.compiler = ? AND test_results.cpp_standard = ?
)
SELECT repo, run_id, run_attempt FROM combination
ORDER BY time DESC, run_id DESC, run_attempt DESC
LIMIT 1;
""",(target,name,compiler,cpp_standard))
result = cursor.fetchone()
if result is None:
# if no recent run is found, data need to be stored
return []
repo, run_id, run_attempt = result
cursor.execute("""
SELECT passed_cases, failed_cases, skipped_cases, passed_assertions, failed_assertions
FROM test_results WHERE
ctest_target = ? AND name = ? AND compiler = ? AND cpp_standard = ? AND repo = ? AND run_id = ? AND run_attempt = ?
""", (target,name,compiler,cpp_standard,repo,run_id,run_attempt))
result = cursor.fetchone()
return [] if result is None else list(result)
##########################
# Below starts the script.
##########################
if __name__ == "__main__":
# check if argument was delivered
if len(sys.argv) != 2:
raise RuntimeError("Expected status of workflow as argument. Aborting!")
# expected argument: status of workflow
# check if the argument has the expected form
status = sys.argv[1]
if status not in ["successful", "failed", "cancelled"]:
raise RuntimeError("The input does not match the expected format! Permissible are 'successful', 'failed' and 'cancelled'. Aborting!")
# get environment variables
try:
environment = setup_environment_variables()
except RuntimeError as e:
raise RuntimeError("Critical error: Can not uniquely identify environment data! Aborting recording of data.")
# Step 1: store metadata of workflow run persistently
# initiate connection to database
connector = sqlite3.connect("TSF/MemoryEfficientTestResultData.db")
connector.execute("PRAGMA foreign_keys = ON")
# load expected tables
# table workflow_info contains metadata of workflow and is updated every time
command = (
"CREATE TABLE IF NOT EXISTS workflow_info(",
"repo TEXT, ", # repository
"run_id INT, ", # ID of workflow run
"run_attempt INT, ", # Attempt-number of workflow run
"status TEXT ", # Termination-status of workflow
"CHECK(status IN ('successful', 'failed', 'cancelled')) DEFAULT 'failed', ",
"time INT, ", # the time that is associated to this workflow run
"PRIMARY KEY(repo, run_id, run_attempt))"
)
connector.execute(''.join(command))
# table test_results contains detailed results for each individual test
command = (
"CREATE TABLE IF NOT EXISTS test_results(",
"ctest_target TEXT, ", # name of the ctest target located in ci.cmake
"name TEXT, ", # name of the test
"execution_time REAL, ", # execution time in seconds
"compiler TEXT, ", # compiler information
"cpp_standard TEXT, ", # cpp-standard
"passed_cases INT, ", # number of passed test-cases
"failed_cases INT, ", # number of failed test-cases
"skipped_cases INT, ", # number of skipped test-cases
"passed_assertions INT, ", # number of passed assertions
"failed_assertions INT, ", # number of failed assertions
"repo TEXT, ", # repository
"run_id INT, ", # ID of workflow run
"run_attempt INT, ", # Attempt-number of workflow run
"FOREIGN KEY(repo, run_id, run_attempt) REFERENCES workflow_info);"
)
connector.execute(''.join(command))
cursor = connector.cursor()
# Count number of rows as heuristic size-checker.
# In case that the update-check fails, and every result is stored, allow for approximately 26 complete results to be stored
cursor.execute("SELECT MAX(COALESCE((SELECT MAX(rowid) FROM workflow_info),0),COALESCE((SELECT MAX(rowid) FROM test_results),0));")
if cursor.fetchone()[0] > 1e5:
connector.close()
raise RuntimeError("The persistent data storage is too large! Please move persistent data to external storage.")
# fill in metadata
# OBSERVE: This script expects the status of the github workflow as argument
repo = environment.get('GITHUB_REPOSITORY')
run_id = environment.get('GITHUB_RUN_ID')
run_attempt = environment.get('GITHUB_RUN_ATTEMPT')
time = int(datetime.now(timezone.utc).timestamp())
command = "INSERT INTO workflow_info VALUES(?,?,?,?,?)"
cursor.execute(command,(repo, run_id, run_attempt, status, time))
# Don't forget to save!
connector.commit()
# Step 2: generate report of most recent test run and update persistent storage if necessary
# initialise database connection
conn = sqlite3.connect("MemoryEfficientTestResults.db")
cur = conn.cursor()
# add the expected table
# the table TestResults.test_results differs from TestResultData.test_results in that the metadata of the commit are not saved.
command = (
"CREATE TABLE IF NOT EXISTS test_results(",
"ctest_target TEXT, ", # name of the ctest target located in ci.cmake
"name TEXT, ", # name of the test
"execution_time REAL, ", # execution time in seconds
"compiler TEXT, ", # compiler information
"cpp_standard TEXT, ", # cpp-standard
"passed_cases INT, ", # number of passed test-cases
"failed_cases INT, ", # number of failed test-cases
"skipped_cases INT, ", # number of skipped test-cases
"passed_assertions INT, ", # number of passed assertions
"failed_assertions INT", # number of failed assertions
")"
)
conn.execute(''.join(command))
# Load my artifacts
junit_logs = get_all_xml_files("./my_artifacts/")
#extract data
for junit_log in junit_logs:
tree = ET.parse(junit_log)
file_root = tree.getroot()
testsuite = next(file_root.iter('testsuite'), None)
if testsuite is None:
print(f"Error: Could not find testsuite data in {junit_log}.")
continue
for testcase in (case for case in file_root.iter('testcase') if is_unit_test(case)):
metadata = get_metadata(testcase)
target = get_ctest_target(junit_log)
compiler = testsuite.get('name')
more_compiler_info = [case for case in file_root.iter('testcase') if case.get("name") == "cmake_target_include_directories_configure"]
if len(more_compiler_info) != 0:
compiler_information = more_compiler_info[0]
information = list(compiler_information.find("system-out").itertext())[0].split('\n')[0]
compiler = information.replace("-- The CXX compiler identification is ","")
name = metadata.get('name')
cpp_standard = metadata.get('standard')
data = (
target,
name,
metadata.get('execution time'),
compiler,
cpp_standard,
metadata.get('passed test cases'),
metadata.get('failed test cases'),
metadata.get('skipped test cases'),
metadata.get('passed assertions'),
metadata.get('failed assertions')
)
command ="INSERT INTO test_results VALUES(?,?,?,?,?,?,?,?,?,?);"
cur.execute(command, data)
conn.commit()
most_recently_stored_results = find_most_recent_results(target,name,compiler,cpp_standard,connector)
current_results = [metadata.get('passed test cases'),metadata.get('failed test cases'),metadata.get('skipped test cases'),metadata.get('passed assertions'),metadata.get('failed assertions')]
if (len(most_recently_stored_results) != 5) or any(most_recently_stored_results[i]!=current_results[i] for i in range(0,5)):
data = data + (repo, run_id, run_attempt)
command ="INSERT INTO test_results VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?);"
cursor.execute(command,data)
# terminate connection to temporary database
# don't forget to commit the changes
conn.commit()
conn.close()
# terminate connection to persistent database
# don't forget to commit the changes again, for good measure
connector.commit()
connector.close()
Fallacies:
None
Graph:
date-time |
JLS-45 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-47 | Reviewed: ⨯ | Score: 0.0#
The nlohmann/json library only consists of source code in the form of a single header file, and has no binary artifacts.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All inputs to the nlohmann/json library are assessed, to identify potential risks and issues. |
0.32 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp
https://github.com/nlohmann/json/blob/develop/single_include/nlohmann/json.hpp
- Path to the header file that the json library consists of.
Fallacies:
None
Graph:
date-time |
JLS-47 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-48 | Reviewed: ⨯ | Score: 0.0#
The nlohmann/json project transparently lists any third party tools used in building, testing and documenting the project.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All inputs to the nlohmann/json library are assessed, to identify potential risks and issues. |
0.32 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/blob/55f93686c01528224f448c19128836e7df245f72/README.md#used-third-party-tools
https://github.com/nlohmann/json/blob/55f93686c01528224f448c19128836e7df245f72/README.md#used-third-party-tools
- List of third party tools used to build, test and document nlohmann/json v3.12.0.
Fallacies:
None
Graph:
date-time |
JLS-48 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-49 | Reviewed: ⨯ | Score: 0.0#
The eclipse-score/inc_nlohmann_json project mirrors the external nlohmann/json_test_data repository, containing test data used by the unit tests in nlohmann/json, such that the test suite can be executed without internet connectivity.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All inputs to the nlohmann/json library are assessed, to identify potential risks and issues. |
0.32 |
⨯ Item Reviewed |
|
Construction of releases for the nlohmann/json library is fully repeatable and the results are fully reproducible, with any exceptions documented and justified. |
0.00 |
⨯ Item Reviewed |
|
All sources and tools for the nlohmann/json library are mirrored in our controlled environment. |
0.18 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/blob/55f93686c01528224f448c19128836e7df245f72/README.md?plain=1#L1842
https://github.com/nlohmann/json/blob/55f93686c01528224f448c19128836e7df245f72/README.md?plain=1#L1842
- Instructions on how to run test suite without internet connectivity.
cmake/download_test_data.cmake
set(JSON_TEST_DATA_URL https://github.com/eclipse-score/inc_nlohmann_json)
set(JSON_TEST_DATA_BRANCH json_test_data_version_3_1_0_mirror)
# if variable is set, use test data from given directory rather than downloading them
if(JSON_TestDataDirectory)
message(STATUS "Using test data in ${JSON_TestDataDirectory}.")
add_custom_target(download_test_data)
file(WRITE ${CMAKE_BINARY_DIR}/include/test_data.hpp "#define TEST_DATA_DIRECTORY \"${JSON_TestDataDirectory}\"\n")
else()
find_package(Git)
# target to download test data
add_custom_target(download_test_data
COMMAND test -d json_test_data || ${GIT_EXECUTABLE} clone --branch ${JSON_TEST_DATA_BRANCH} ${JSON_TEST_DATA_URL}.git --quiet --depth 1 json_test_data
COMMENT "Downloading test data from ${JSON_TEST_DATA_URL} (branch: ${JSON_TEST_DATA_BRANCH})"
WORKING_DIRECTORY ${CMAKE_BINARY_DIR}
)
# create a header with the path to the downloaded test data
file(WRITE ${CMAKE_BINARY_DIR}/include/test_data.hpp "#define TEST_DATA_DIRECTORY \"${CMAKE_BINARY_DIR}/json_test_data\"\n")
endif()
# determine the operating system (for debug and support purposes)
find_program(UNAME_COMMAND uname)
find_program(VER_COMMAND ver)
find_program(LSB_RELEASE_COMMAND lsb_release)
find_program(SW_VERS_COMMAND sw_vers)
set(OS_VERSION_STRINGS "${CMAKE_SYSTEM}")
if (VER_COMMAND)
execute_process(COMMAND ${VER_COMMAND} OUTPUT_VARIABLE VER_COMMAND_RESULT OUTPUT_STRIP_TRAILING_WHITESPACE)
set(OS_VERSION_STRINGS "${OS_VERSION_STRINGS}; ${VER_COMMAND_RESULT}")
endif()
if (SW_VERS_COMMAND)
execute_process(COMMAND ${SW_VERS_COMMAND} OUTPUT_VARIABLE SW_VERS_COMMAND_RESULT OUTPUT_STRIP_TRAILING_WHITESPACE ERROR_QUIET)
string(REGEX REPLACE "[ ]*\n" "; " SW_VERS_COMMAND_RESULT "${SW_VERS_COMMAND_RESULT}")
set(OS_VERSION_STRINGS "${OS_VERSION_STRINGS}; ${SW_VERS_COMMAND_RESULT}")
endif()
if (LSB_RELEASE_COMMAND)
execute_process(COMMAND ${LSB_RELEASE_COMMAND} -a OUTPUT_VARIABLE LSB_RELEASE_COMMAND_RESULT OUTPUT_STRIP_TRAILING_WHITESPACE ERROR_QUIET)
string(REGEX REPLACE "[ ]*\n" "; " LSB_RELEASE_COMMAND_RESULT "${LSB_RELEASE_COMMAND_RESULT}")
set(OS_VERSION_STRINGS "${OS_VERSION_STRINGS}; ${LSB_RELEASE_COMMAND_RESULT}")
endif()
if (UNAME_COMMAND)
execute_process(COMMAND ${UNAME_COMMAND} -a OUTPUT_VARIABLE UNAME_COMMAND_RESULT OUTPUT_STRIP_TRAILING_WHITESPACE ERROR_QUIET)
set(OS_VERSION_STRINGS "${OS_VERSION_STRINGS}; ${UNAME_COMMAND_RESULT}")
endif()
message(STATUS "Operating system: ${OS_VERSION_STRINGS}")
# determine the compiler (for debug and support purposes)
if (MSVC)
execute_process(COMMAND ${CMAKE_CXX_COMPILER} OUTPUT_VARIABLE CXX_VERSION_RESULT OUTPUT_STRIP_TRAILING_WHITESPACE ERROR_VARIABLE CXX_VERSION_RESULT ERROR_STRIP_TRAILING_WHITESPACE)
set(CXX_VERSION_RESULT "${CXX_VERSION_RESULT}; MSVC_VERSION=${MSVC_VERSION}; MSVC_TOOLSET_VERSION=${MSVC_TOOLSET_VERSION}")
else()
execute_process(COMMAND ${CMAKE_CXX_COMPILER} --version OUTPUT_VARIABLE CXX_VERSION_RESULT OUTPUT_STRIP_TRAILING_WHITESPACE)
endif()
string(REGEX REPLACE "[ ]*\n" "; " CXX_VERSION_RESULT "${CXX_VERSION_RESULT}")
message(STATUS "Compiler: ${CXX_VERSION_RESULT}")
Description: File specifying that test data should be downloaded from given directory instead of external repository, if provided.
website: https://github.com/nlohmann/json_test_data
https://github.com/nlohmann/json_test_data
- External repository containing test data.
website: https://github.com/eclipse-score/inc_nlohmann_json/tree/json_test_data_version_3_1_0_mirror
https://github.com/eclipse-score/inc_nlohmann_json/tree/json_test_data_version_3_1_0_mirror
- Branch in eclipse-score/inc_nlohmann_json mirroring v3.1.0 of the test data.
Fallacies:
None
Graph:
date-time |
JLS-49 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-50 | Reviewed: ⨯ | Score: 0.0#
All third party tools used in nlohmann/json for building, testing and documenting the project are assessed within eclipse-score/inc_nlohmann_json.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All inputs to the nlohmann/json library are assessed, to identify potential risks and issues. |
0.32 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/blob/55f93686c01528224f448c19128836e7df245f72/README.md#used-third-party-tools
https://github.com/nlohmann/json/blob/55f93686c01528224f448c19128836e7df245f72/README.md#used-third-party-tools
- List of third party tools used to build, test and document nlohmann/json v3.12.0.
TSF/docs/third_party_tools_assessment.md
# Assessment of Third-Party Tools Used in nlohmann/json
This file provides an assessment of all third-party tools used in the development, testing, and documentation of nlohmann/json version 3.12.0, as required by TA-INPUTS in the Trustable Software Framework (TSF).
**Assessment Date**: November 25, 2025
**Assessors**: @erikhu1
**Target nlohmann/json Version**: v3.12.0
## Tool Assessment Summary
### amalgamate.py
- **Role**: Third-party Python script, mirrored into the nlohmann/json repository and is used to generate the single-header distribution file `json.hpp` from the modular source tree.
- **Potential Misbehaviours**: The upstream README (https://github.com/edlund/amalgamate/blob/master/README.md) explicitly states that `amalgamate.py` is "quite dumb" and may produce "weird results" for non-trivial code. In particular, it only understands very simple `#include` directives and does not correctly handle:
- macro-based includes (e.g. `#define HEADER_PATH "x.h"` / `#include HEADER_PATH`),
- certain assumptions about file endings (missing final newline, backslash-escaped newline).
As a result, it could incorrectly merge files (wrong order, missing or duplicated code), fail to include required headers, or corrupt the generated header in subtle ways.
The script originates from an external project that is not under the direct control of the nlohmann/json maintainers and shows limited public maintenance activity.
- **Severity**: High - Any undetected error in the amalgamated header would directly affect the library's functionality.
- **Detectability**: High - The generated `json.hpp` is stored in version control and built in CI, where it is compiled and run with the full unit test suite. This means that most structural or include-related problems show up as build or test failures, and more subtle issues are further reduced by the use of fuzz testing.
- **Mitigation**:
- The script is mirrored into the repository and thus fully auditable.
- Compilation tests and unit tests are run on the amalgamated `json.hpp`.
- Changes to the mirrored script and to the generated `json.hpp` are manually reviewed before releases.
### American fuzzy lop (AFL)
- **Role**: Fuzz testing tool used to generate many random and mutated inputs for the library in order to find crashes and hangs.
- **Potential Misbehaviours**: AFL can only explore the parts of the code that are reachable from the chosen fuzz targets and input seeds. It may miss important edge cases or code paths, so existing bugs can remain undiscovered. This can create a false sense of robustness.
- **Severity**: Medium - bugs that are not found by AFL can still reach users, but AFL itself never changes the library code or the released header.
- **Detectability**: Low - missed edge cases are usually only noticed later, for example through other tests, other fuzzers, or bug reports from users.
- **Mitigation**: AFL is only one part of a broader testing strategy. It is complemented by:
- continuous fuzzing via OSS-Fuzz and libFuzzer,
- extensive unit tests with a 100% coverage target,
- dynamic analysis with tools such as Valgrind.
### AppVeyor
- **Role**: Continuous integration service used to build and test the library on Windows.
- **Potential Misbehaviours**: AppVeyor itself can fail (e.g. service outages, misconfigured jobs, flaky Windows images), which can lead to:
- build failures that are unrelated to the code,
- tests not running or reporting wrong results,
- Windows-specific issues not being exercised properly.
- **Severity**: Medium - if Windows CI is broken or misconfigured, Windows-specific problems might not be detected, but the source code and `json.hpp` are not modified by AppVeyor.
- **Detectability**: High - CI results are visible for every commit and pull request, and unexpected failures or missing runs are noticed by maintainers. Suspicious results can be cross-checked with local Windows builds. Any misbehaviours can also be reported by the users.
- **Mitigation**: Multiple CI platforms (GitHub Actions, Cirrus CI), cross-platform testing
### Artistic Style
- **Role**: Tool used to automatically format and indent the C++ source code.
- **Potential Misbehaviours**: Artistic Style can change whitespace and line breaks in a way that makes the code harder to read. In unusual cases or with a wrong configuration, it could also reformat code so that the structure is broken (for example, by changing braces in a way that affects parsing).
- **Severity**: Low - The tool is only used for formatting. if it ever breaks the code, the problem is detected by compilation or tests before a release.
- **Detectability**: High - Formatting changes are visible in code review and any structural problems show up as compiler errors or test failures.
- **Mitigation**:
- After reformatting, the code is compiled and tested in CI.
- Non-trivial or suspicious formatting changes are checked manually during review.
### Clang
- **Role**: One of the main compilers used to build and test the library. In CI it is also used with sanitizers such as AddressSanitizer and UndefinedBehaviorSanitizer. These sanitizers are special compiler modes that make the program crash with a clear error message when it does things like reading/writing invalid memory or relying on undefined behaviour, so such bugs are easier to find during testing.
- **Potential Misbehaviours**: Clang itself, or its sanitizers, can have bugs. This can lead to:
- wrong binary code being generated,
- some memory or undefined-behaviour problems not being detected,
- false reports that claim there is a problem when there is none.
- **Severity**: High - If Clang or its sanitizers miss real issues, tests can still pass even though bugs are present. This reduces confidence in the test results.
- **Detectability**: Medium - problems that only appear with one compiler are partly caught by:
- also building and testing with other compilers (e.g. GCC, MSVC) and
- bug reports from users who build with different toolchains.
- **Mitigation**:
- The library is built and tested with multiple compilers and versions in CI (Clang, GCC, MSVC).
- Static analysis tools (e.g. Coverity, cppcheck, Codacy) and dynamic tools (sanitizers, Valgrind) are used in addition to normal compilation and unit tests, which increases the chance of finding real issues even if one compiler or sanitizer misses them.
### CMake
- **Role**: Primary build system generator used to configure how the library's internal targets are built on different platforms. CMake reads the project’s `CMakeLists.txt` files and generates platform-specific build files (e.g. Makefiles, Ninja files, Visual Studio projects) that control the compilation and linking of the unit test and optional helper tools (e.g. sanitizer), including the selection of compilers and the compiler flags used.
- **Potential Misbehaviours**: CMake can be misconfigured or behave differently between versions, which can lead to:
- wrong or incomplete build configurations (e.g. tests not being built or run),
- missing or incorrectly detected dependencies and features,
- wrong compiler or linker flags (e.g. no debug info, missing warnings, wrong standard version).
In such cases, the code itself may be correct, but the way it is built and tested is not what the developers expect.
- **Severity**: High - If CMake generates a wrong build configuration, all builds that rely on it can be affected
- **Detectability**: High - most CMake-related problems show up as:
- build failures in CI or on user systems,
- obviously missing targets,
- inconsistent results between different platforms or compilers.
Such issues are usually noticed quickly when running builds on multiple systems.
- **Mitigation**:
- The root `CMakeLists.txt` in the nlohmann/json repository declares a minimum required CMake version (via `cmake_minimum_required(...)`), so too old or unsupported CMake versions are rejected at configure time rather than producing silently broken build files.
- The CMake-based build configuration is exercised in continuous integration on multiple platforms and compilers (Linux, macOS, Windows with Clang, GCC, and MSVC), as documented on the project's “Quality assurance” page (https://json.nlohmann.me/community/quality_assurance/?utm_source=chatgpt.com#simple-integration).
- The same CMake setup is used to configure and build the internal unit tests (via `enable_testing()` / `add_subdirectory(test)` in `CMakeLists.txt`) and the small example/demo programs described in the documentation, so incorrect build options or missing dependencies usually cause test or example targets to fail.
- All changes to the CMake files are tracked in version control and go through pull-request review. They must pass the full CI matrix before being merged, which reduces the risk that a broken CMake configuration is used for a release.
### Codacy
- **Role**: Codacy is a hosted code quality and static analysis service that is connected to the nlohmann/json repository. It automatically analyzes new commits and pull requests with various C++ linters and rules, and then reports findings such as style issues, potential bugs, complexity problems, and code smells in a web dashboard and as comments on pull requests.
- **Potential Misbehaviours**: Codacy can:
- report false positives,
- miss real issues (false negatives),
- change its rules or analyzers over time, so that the same code can produce different warnings at different points in time even if the project itself has not changed.
- **Severity**: Low - Codacy is purely informational. It does not modify any code.
- **Detectability**: High - Codacy's findings are visible alongside results from other tools (e.g. Coverity, cppcheck, compiler warnings), and discrepancies or obviously wrong reports are easy to spot during code review.
- **Mitigation**: Multiple static analysis tools (Coverity, cppcheck, clang-tidy) and manual code review.
### Coveralls
- **Role**: Coveralls is a hosted service for code coverage measurement and reporting. In the nlohmann/json project, coverage data produced during the Ubuntu CI workflow is uploaded to Coveralls from the GitHub Actions workflow `.github/workflows/ubuntu.yml` using the `coverallsapp/github-action` step (“Publish report to Coveralls”). The resulting coverage information is shown on the project's Coveralls page and is linked as a badge in the README.
- **Potential Misbehaviours**: Coveralls can:
- display incorrect coverage percentages or mark lines as covered/uncovered incorrectly,
- lose or mix up coverage history, which can make trends look better or worse than they are,
- be temporarily unavailable.
- **Severity**: Medium - Misleading coverage information can cause important parts of the code to remain untested, even when the coverage dashboard looks “green”.
- **Detectability**: Medium - Inconsistencies can be compared against local coverage runs.
- **Mitigation**:
- Coverage is also generated locally and in CI using `lcov` and viewed as HTML reports, so Coveralls results can be cross-checked against these local reports.
### clang-tidy
- **Role**: Clang-Tidy is used as static analysis tool to detect bug patterns, unsafe constructs, and style/performance issues in the code base. In nlohmann/json it is explicitly part of the project’s “Static analysis” quality assurance activities, and it is configured via a repository-wide [.clang-tidy](https://github.com/nlohmann/json/blob/develop/.clang-tidy) configuration file.
- **Potential Misbehaviours**:
- False positives (reporting issues that are not real problems) or false negatives (missing real issues).
- Results can vary between Clang-Tidy versions and depending on compile flags / include paths.
- **Severity**: Medium - Clang-Tidy findings are advisory and incorrect or missing findings can reduce confidence and leave issues undiscovered, but the tool does not modify shipped artefacts by itself.
- **Detectability**: Medium - False positives are usually spotted during review and false negatives are harder to detect and may only surface through other analyzers, tests, fuzzing, or user reports.
- **Mitigation**:
- Clang-Tidy is not the only analysis gate, results are complemented by other static analyzers (e.g. cppcheck, Coverity, Clang Static Analyzer) and dynamic testing (unit tests, fuzzing, sanitizers).
- The .clang-tidy configuration is version-controlled and reviewed like normal code changes.
- CI uses a controlled toolchain environment (e.g. the project CI image), reducing version drift and improving reproducibility.
### Coverity Scan
- **Role**: Coverity Scan is a hosted static analysis service that regularly analyzes the nlohmann/json code base for potential defects. The nlohmann/json repo has a dedicated Coverity Scan entry (linked via the “Coverity Scan Build Status” badge in `README.md`), where findings are listed and tracked in a web dashboard.
- **Potential Misbehaviours**: Could miss bugs (false negatives) or report non-issues (false positives)
- **Severity**: Medium - Missed bugs or misinterpreted reports can affect the quality of the library.
- **Detectability**: High - Coverity Scan findings are compared with results from other static analyzers (such as cppcheck and Codacy), compiler warnings, and the behaviour observed in tests and fuzzing.
- **Mitigation**:
- Multiple static analyzers (cppcheck, clang-tidy, Codacy), extensive testing.
- Coverity Scan is used within S-CORE, where tool qualification evidence is maintained.
### cppcheck
- **Role**: Cppcheck is a static analysis tool for C++ that is used to scan the nlohmann/json library code base for potential problems such as null dereferences, uninitialized variables, dead code, or suspicious constructs.
- **Potential Misbehaviours**: False positives could waste developer time, false negatives could miss bugs.
- **Severity**: Medium - Missed issues (false negatives) can affect users if they are not caught by other tools or tests. However, its output is advisory, and problems only enter the code base if humans misinterpret or ignore the results.
- **Detectability**: High - Cppcheck's findings are viewed together with other static analyzers (such as Coverity and Codacy), compiler warnings from different compilers and the behaviour observed in unit tests and fuzzing.
- **Mitigation**: Multiple static analysis tools (Coverity, clang-tidy), compiler warnings enabled, extensive unit tests.
### doctest
- **Role**: Doctest is the C++ open source unit testing framework used by nlohmann/json for its internal test suite. The project’s unit tests are implemented in source files under `tests/src/` (for example `tests/src/unit-*.cpp`), where test cases are written using doctest macros such as `TEST_CASE`, `CHECK`, and `REQUIRE`.
- **Potential Misbehaviours**: Doctest itself can have bugs or limitations. This can lead to situations of:
- false passes, for example if an assertion macro does not correctly detect a failing condition or if failures are swallowed by the framework,
- false failures, for example due to issues with test registration, command-line handling, or interaction with specific compilers and optimisation levels,
- some test cases are never discovered or executed (for example if certain `TEST_CASE` definitions are not registered correctly in some build configurations).
- **Severity**: High - The doctest-based unit tests are a key mechanism for ensuring correctness of the library before changes are merged or releases are made. If doctest hides real failures or silently skips tests, bugs can remain undetected and be shipped to users.
- **Detectability**: High - Framework problems are partly detectable because the same doctest-based test suite is compiled and executed with multiple compilers (such as Clang, GCC, and MSVC) and on different platforms, so framework or configuration issues often show up as differences between environments or as unexpected crashes in the test binaries.
- **Mitigation**:
- The doctest-based tests are run in a broad CI matrix (different operating systems, compilers, and standard libraries), which helps reveal framework or configuration issues that only appear under certain toolchains.
- For critical functionality and previously fixed defects, test coverage is reinforced by additional checks such as fuzzing, sanitizer runs, and targeted manual tests.
- When a discrepancy is suspected, new test cases are added or existing ones are refined, improving both test coverage and the chance to notice framework-related issues.
- All changes to the tests or to the way doctest is integrated (e.g. via CMake) go through normal pull requests, are reviewed by maintainers, and must pass the full CI test matrix.
### GitHub Changelog Generator
- **Role**: GitHub Changelog Generator is a tool that uses the GitHub API to collect issues, pull requests, and tags from the nlohmann/json repository and then generates a `ChangeLog.md` file from this information. It is used to create a human-readable list of changes for releases.
- **Potential Misbehaviours**: The tool can miss entries, include wrong or misleading information or produce poorly formatted output (e.g. broken Markdown or duplicated entries).
- **Severity**: Low - Even if the generated changelog is incomplete or inaccurate, this only affects documentation and release notes. It does not change the any code of the library or its functionality.
- **Detectability**: High - Incorrect or missing entries, odd wording, or broken formatting in `ChangeLog.md` are usually easy to spot when maintainers review the file before a release. Users may also report inconsistencies if they notice them.
- **Mitigation**:
- The generated `ChangeLog.md` is committed to version control and reviewed and edited manually by maintainers before a release.
- If the generator's configuration leads to repeated problems (e.g. systematically missing certain types of issues), the configuration or the generation process can be adjusted, and older changelog entries can be corrected in the repository.
- Because the changelog is under version control, any mistakes can be fixed later, and the history of changes to `ChangeLog.md` is traceable.
### Google Benchmark
- **Role**: Google Benchmark is a C++ microbenchmarking framework used in nlohmann/json to implement standalone performance benchmarks for the library. The benchmark program in `tests/benchmarks/src/benchmarks.cpp` uses the Google Benchmark API to measure operations such as `json::parse`, `dump`, and conversions to binary formats (e.g. CBOR) on representative JSON files, so maintainers can track performance and detect regressions between versions.
- **Potential Misbehaviours**: Google Benchmark can:
- produce noisy or misleading measurements if the environment is unstable (e.g. different CPUs, background load, power-saving modes),
- encourage microbenchmarks that are not representative of real-world usage, leading to optimizations that improve benchmark numbers but not actual user workloads,
- be misconfigured, which can bias results or exaggerate small differences.
- **Severity**: Low - Incorrect or misleading benchmark results can waste time or lead to suboptimal performance decisions, but they do not affect the functional correctness or safety of the library.
- **Detectability**: High - suspicious benchmark results can usually be identified by rerunning the benchmarks on the same and on different machines and by comparing results across compilers, optimization levels, and configurations.
- **Mitigation**:
- Performance changes suggested by Google Benchmark results are not accepted blindly, maintainers manually interpret the data and consider how realistic the benchmarked scenarios are for typical users.
- Functional tests and fuzzing remain the primary gate for correctness.
### Hedley
- **Role**: Hedley is a third-party header that provides portability and feature-detection macros for different compilers and platforms. nlohmann/json ships a bundled copy in `include/nlohmann/thirdparty/hedley/hedley.hpp`, which is included from `json.hpp` and used to define macros for attributes, warning suppression, and compiler feature checks.
- **Potential Misbehaviours**: Wrong or incomplete Hedley macros can:
- assume compiler features or attributes that are not actually supported,
- trigger compilation errors or excessive warnings on some compilers,
- silently disable useful attributes or diagnostics, which may lead to performance issues or missed warnings.
- **Severity**: High - Hedley is used throughout the headers and affects how the library is compiled on all platforms. Incorrect macros can break builds or cause subtle differences in behaviour across compilers.
- **Detectability**: High - most problems show up as build failures or unusual warnings on specific compilers or platforms, and are likely to be caught by the project's CI matrix.
- **Mitigation**:
- A specific version of Hedley is used in the nlohman/json repository and is fully under version control and updates are done via pull requests and must pass all CI checks before being included in a release
- The library is built and tested with multiple compilers (Clang, GCC, MSVC) and on multiple platforms in CI.
### lcov
- **Role**: lcov is used in the nlohmann/json project as part of the coverage setup introduced together with the CMake-based build system (see 'Added CMake and lcov' in the project's changelog and PR #6). It processes gcov output from the doctest-based unit tests to generate local HTML reports, which can be cross-checked against the coverage information published via Coveralls.
- **Potential Misbehaviours**: lcov can misinterpret or partially ignore `gcov` data, so that some lines are shown as covered or uncovered incorrectly and generate incomplete or inconsistent HTML reports.
- **Severity**: Low - lcov only reads coverage data and creates reports; it does not affect any code or the functionality of the lirary.
- **Detectability**: High - Inconsistencies can be detected by comparing lcov's HTML output with the coverage data uploaded to Coveralls.
- **Mitigation**:
- Mitigation through cross-validation with Coveralls and manual inspection of coverage data.
- lcov is used within S-CORE, where tool qualification evidence is maintained.
### libFuzzer
- **Role**: libFuzzer is LLVM's fuzzing engine that runs inside the test binary and uses code-coverage feedback to generate new inputs. In nlohmann/json it is used as the fuzzing backend for the fuzz harnesses under `tests/src/` (e.g. `fuzzer-parse_json.cpp`), each implementing `LLVMFuzzerTestOneInput` and calling APIs such as `json::parse` and the `from_*` functions for CBOR, MessagePack, UBJSON, and BJData. The same libFuzzer-based targets are also built and run by Google OSS-Fuzz, which executes them continuously with sanitizers to find crashes and undefined behaviour.
- **Potential Misbehaviours**: libFuzzer can only explore code that is reachable through the fuzz harness and within the time and resources it is given, so it can miss important edge cases or provide misleading coverage.
- **Severity**: Medium - Bugs that are not found by libFuzzer (false negatives) can still reach users if they are not caught by other fuzzers, tests, or reviews.
- **Detectability**: Low - Missed edge cases are they are typically discovered later through other fuzzers, unit and regression tests, or bug reports from users.
- **Mitigation**:
- libFuzzer is used together with other fuzzing approaches (AFL, OSS-Fuzz,..).
- Crashes and sanitizer findings from fuzzing are treated as defects and, where applicable, fixed and accompanied by additional regression tests.
- High unit-test coverage.
### Material for MkDocs
- **Role**: The third-party theme used by the mkdocs-based documentation of nlohmann/json. The documentation sources live under `docs/mkdocs` and are built with MkDocs to produce the public site at <https://json.nlohmann.me>. The theme controls the visual appearance, navigation, and layout of this site.
- **Potential Misbehaviours**: Visual issues, broken layouts, accessibility problems
- **Severity**: Low - Problems in the theme can make the documentation harder to read or navigate, but they do not change the library sources.
- **Detectability**: High - Visual or layout issues are usually easy to spot when viewing the documentation site during local builds or on the deployed site, and users can report any problems they encounter.
- **Mitigation**:
- The documentation is built and viewed locally during development (following the documented Contributing Guidlines).
- The site can be checked in multiple browsers and on different devices to catch theme-related layout or responsiveness problems.
- The mkdocs and Material for MkDocs versions used in `docs/mkdocs` are pinned and updated via pull requests, which are reviewed and must pass CI before being merged.
### MkDocs
- **Role**: MkDocs is the static site generator used to build the nlohmann/json documentation at <https://json.nlohmann.me>, using sources in `docs/mkdocs/docs` and the configuration in `docs/mkdocs/mkdocs.yml`. Maintainers build and preview the site locally with `make install_venv -C docs/mkdocs` and `make serve -C docs/mkdocs`, and the same MkDocs setup is also run in CI to build the documentation.
- **Potential Misbehaviours**: MkDocs could fail to build the documentation (for example due to configuration or plugin issues), generate pages with broken links or missing sections, or render Markdown in a way that causes formatting or navigation problems.
- **Severity**: Low - Even if the documentation is incomplete or incorrectly rendered, this only affects the documentation site. MkDocs does not modify the library sources.
- **Detectability**: High - Build failures are immediately visible when running the MkDocs commands locally or in CI, and visual or navigation issues can be spotted when reviewing the rendered site.
- **Mitigation**:
- The documentation is built and viewed locally during development, so maintainers can manually review the generated pages before changes are published.
- The MkDocs configuration and its dependencies (including the theme and plugins) are kept under version control in `docs/mkdocs`, and updates are applied via pull requests.
- Broken links or formatting problems reported by users can be fixed by updating the Markdown sources or the MkDocs configuration, with all changes tracked and reviewed like normal code changes.
### OSS-Fuzz
- **Role**: OSS-Fuzz is Google's hosted service for continuous fuzzing of open-source projects. nlohmann/json is enrolled there as the `json` project, where OSS-Fuzz builds the repository together with its libFuzzer-based fuzz harnesses under `tests/src/` (e.g. `fuzzer-parse_json.cpp`, `fuzzer-parse_cbor.cpp`,..`) and runs the resulting fuzzers continuously with various sanitizer configurations against all supported parsers. The integration is documented in the project's documentation/README, and several GitHub issues (e.g. [#389](https://github.com/nlohmann/json/issues/389), [#409](https://github.com/nlohmann/json/issues/409), [#452](https://github.com/nlohmann/json/issues/452)) show concrete bugs that were reported by OSS-Fuzz and then fixed in the library.
- **Potential Misbehaviours**: OSS-Fuzz can experience service outages or build/configuration issues, meaning that fuzzing temporarily stops or some fuzz targets are not executed as expected. It can also report false positives, which may consume maintainer time, while still missing other edge cases that do not get reached by the current fuzz setup.
- **Severity**: Medium - Continuous fuzzing from OSS-Fuzz is an important safety net for catching parser bugs and undefined behaviour over time. If it stops working or misses certain paths, bugs may remain undetected longer, but OSS-Fuzz itself never modifies the source code or the shipped `json.hpp`.
- **Detectability**: Medium - problems with OSS-Fuzz (such as build failures or missing runs) are visible on the OSS-Fuzz project dashboards and through email notifications, but the absence of new reports does not guarantee the absence of bugs.
- **Mitigation**:
- The same fuzz harnesses used by OSS-Fuzz can also be built and run locally (with libFuzzer or AFL).
- Fuzzing is complemented by extensive unit tests with high coverage, sanitizer builds, and manual review of critical parsing and binary-format code.
- Crashes and sanitizer findings reported by OSS-Fuzz are treated as defects and, where applicable, fixed and covered by regression tests.
### Probot
- **Role**: Probot is a framework for GitHub Apps that automate repository workflows. In the nlohmann/json repo it is configured via `.github/config.yml` to use apps such as Sentiment Bot and Request Info. These bots post automated comments on issues and pull requests, for example reminding users of the Code of Conduct or asking for more information. Their behaviour can be seen in GitHub issues where `@sentiment-bot` has commented.
- **Potential Misbehaviours**: Probot-based apps can misclassify comments as toxic and respond inappropriately, repeatedly ask for more information on already well-described issues, or automatically label and close issues that are in fact valid. Such behaviour can confuse contributors or temporarily hide relevant discussions, but it only affects issue and PR handling on GitHub.
- **Severity**: Low - Probot automation influences how issues and pull requests are managed, not how the library is built or executed.
- **Detectability**: High - All Probot actions appear as comments, labels, or status changes in the GitHub UI, so misbehaviour is visible to maintainers and contributors.
- **Mitigation**:
- The Probot configuration in `.github/config.yml` is under control and can be adjusted or reverted via pull requests if a bot behaves undesirably.
- Maintainers monitor issue and PR activity and can reopen, relabel, or manually respond to threads that were mishandled by automation.
- If a Probot app causes persistent problems, it can be disabled or its scope reduced without impacting the library code or release process.
### Valgrind
- **Role**: Valgrind is a runtime analysis tool used in nlohmann/json to run the test binaries under a memory checker in order to detect leaks and invalid memory accesses. According to the quality assurance documentation and the CMake setup, the test suite can be executed under Valgrind via the `JSON_Valgrind` CMake option, which configures CTest to run the tests with Valgrind and to treat reported memory errors as test failures (using options such as `--leak-check=full` and a non-zero `--error-exitcode`).
- **Potential Misbehaviours**: Valgrind can miss some memory errors or report false positives. It also slows down execution significantly, so runs may use smaller inputs or be executed less frequently.
- **Severity**: High - Memory errors, invalid accesses, or leaks are critical, and if Valgrind fails to reveal them and they are not caught by other tools, they can affect applications using nlohmann/json.
- **Detectability**: High - Valgrind's limitations are partly mitigated because the same test suite is also run with compiler sanitizers (e.g. AddressSanitizer).
- **Mitigation**:
- Valgrind is used alongside compiler sanitizers, fuzzing (including OSS-Fuzz), and a high-coverage unit test suite.
- Memory related findings from Valgrind or sanitizers are treated as defects and, where appropriate, fixed and covered by regression tests.
## Assessment Methodology
Each tool was evaluated based on:
1. **Role in Release Process**: How the tool contributes to building, testing, or documenting nlohmann/json
2. **Potential Misbehaviours**: Ways the tool could malfunction or produce incorrect results
3. **Impact Assessment**:
- **Severity**: Impact on library correctness, safety, or usability (Low/Medium/High)
- **Detectability**: Likelihood that misbehaviour would be caught by other tools or processes (Low/Medium/High)
4. **Mitigation Measures**: Existing safeguards that reduce risk from tool misbehaviour
## Risk Categorization
The highest-severity tools are those that directly affect the build and test artefact, namely amalgamate.py, CMake, Hedley, Clang and doctest, while the fuzzing tools AFL, libFuzzer and OSS-Fuzz are assessed as medium severity with low detectability, and the documentation and automation tools (such as MkDocs, Material for MkDocs, GitHub Changelog Generator and Probot) are considered low severity because they do not influence the library's behaviour.
| Tool | Severity | Detectability |
|----------------------------|----------|---------------|
| amalgamate.py | High | High |
| CMake | High | High |
| Hedley | High | High |
| doctest | High | High |
| Clang | High | Medium |
| Coverity Scan | Medium | High |
| cppcheck | Medium | High |
| AppVeyor | Medium | High |
| Coveralls | Medium | Medium |
| clang-tidy | Medium | Medium |
| AFL | Medium | Low |
| libFuzzer | Medium | Low |
| OSS-Fuzz | Medium | Low |
| MkDocs | Low | High |
| Material for MkDocs | Low | High |
| GitHub Changelog Generator | Low | High |
| Probot | Low | High |
| Google Benchmark | Low | High |
| lcov | Low | High |
| Codacy | Low | High |
## Conclusion
The nlohmann/json project employs a multi validation strategy with multiple redundant tools for critical functions (testing, analysis, memory checking). High-severity tools are well-qualified through extensive CI testing, cross-validation, and manual review processes. The risk from tool misbehaviour is assessed as **LOW** due to comprehensive mitigation measures.
## References
### Project documentation
- [nlohmann/json README - Used third-party tools](https://github.com/nlohmann/json/blob/v3.12.0/README.md#used-third-party-tools)
- [nlohmann/json Quality Assurance](https://json.nlohmann.me/community/quality_assurance)
- [CII Best Practices Badge](https://bestpractices.coreinfrastructure.org/projects/289)
- [OpenSSF Scorecard](https://scorecard.dev/viewer/?uri=github.com/nlohmann/json)
- [Design goals: Testing, Valgrind, Clang Sanitizers, OSS-Fuzz](https://json.nlohmann.me/home/design_goals/)
### Source and build configuration
- [`cmake/test.cmake` - CTest and `JSON_Valgrind` configuration](https://github.com/nlohmann/json/blob/master/cmake/test.cmake)
- [`CMakeLists.txt` (root) - CMake integration and test setup](https://github.com/nlohmann/json/blob/master/CMakeLists.txt)
- [Ubuntu CI workflow `.github/workflows/ubuntu.yml` (Coveralls upload)](https://github.com/nlohmann/json/blob/master/.github/workflows/ubuntu.yml)
- [Docs build configuration `docs/mkdocs/mkdocs.yml`](https://github.com/nlohmann/json/blob/master/docs/mkdocs/mkdocs.yml)
### Fuzzing
- [OSS-Fuzz project profile `json`](https://introspector.oss-fuzz.com/project-profile?project=json)
- [OSS-Fuzz project config for nlohmann/json](https://github.com/google/oss-fuzz/tree/master/projects/json)
- [Integer-overflow (OSS-Fuzz issue 267)](https://github.com/nlohmann/json/issues/389)
- [Heap-buffer-overflow / Stack-overflow reports](https://github.com/nlohmann/json/issues/577)
### Static analysis, coverage, external services
- [Coverity Scan project page for nlohmann/json](https://scan.coverity.com/projects/nlohmann-json)
- [Coveralls page for nlohmann/json](https://coveralls.io/github/nlohmann/json)
- [Codacy project for nlohmann/json](https://app.codacy.com/gh/nlohmann/json)
- [lcov project documentation](https://github.com/linux-test-project/lcov)
- [clang-tidy](https://clang.llvm.org/extra/clang-tidy/)
### Tool-specific external documentation
- [doctest documentation](https://github.com/doctest/doctest)
- [Google Benchmark documentation](https://github.com/google/benchmark)
- [Hedley documentation](https://github.com/nemequ/hedley)
- [libFuzzer documentation](https://llvm.org/docs/LibFuzzer.html)
- [Valgrind documentation](https://valgrind.org/docs/manual/manual.html)
- [MkDocs documentation](https://www.mkdocs.org/)
- [Material for MkDocs documentation](https://squidfunk.github.io/mkdocs-material/)
- [GitHub Changelog Generator](https://github.com/github-changelog-generator/github-changelog-generator)
- [Probot framework](https://probot.github.io/)
Description: Safety assessment of third party tools used in nlohmann/json v3.12.0.
Fallacies:
None
Graph:
date-time |
JLS-50 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-51 | Reviewed: ⨯ | Score: 0.0#
All source code of the nlohmann/json library is mirrored within eclipse-score/inc_nlohmann_json.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All constructed iterations of the nlohmann/json library include source code, build instructions, tests, results and attestations. |
0.00 |
⨯ Item Reviewed |
|
All sources and tools for the nlohmann/json library are mirrored in our controlled environment. |
0.18 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json
https://github.com/nlohmann/json
- Start-page of the original nlohmann/json project
website: https://github.com/eclipse-score/inc_nlohmann_json
https://github.com/eclipse-score/inc_nlohmann_json
- Start-page of the mirror of nlohmann/json within Eclipse S-CORE
website: https://github.com/eclipse-score/inc_nlohmann_json/blob/main/single_include/nlohmann/json.hpp
https://github.com/eclipse-score/inc_nlohmann_json/blob/main/single_include/nlohmann/json.hpp
- The single header file that contains all the source code downstream users need.
website: https://github.com/eclipse-score/inc_nlohmann_json/tree/main/include/nlohmann
https://github.com/eclipse-score/inc_nlohmann_json/tree/main/include/nlohmann
- The modular header files, containing all the source code which is amalgamated into the single header file.
Fallacies:
None
Graph:
date-time |
JLS-51 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-52 | Reviewed: ⨯ | Score: 0.0#
Releases of the nlohmann/json library contain a comprehensive set of attestations.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All constructed iterations of the nlohmann/json library include source code, build instructions, tests, results and attestations. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Releases of the nlohmann/json library contain commit sign-offs. |
0.00 |
⨯ Item Reviewed |
|
Releases of the nlohmann/json library contain Github pull request approvals. |
0.00 |
⨯ Item Reviewed |
|
Releases of the nlohmann/json library contain continuous-integration test reports such as coverage reports, fuzzing reports & OpenSSF scorecard reports. |
0.00 |
⨯ Item Reviewed |
References:
None
Fallacies:
None
Graph:
date-time |
JLS-52 |
JLS-63 |
JLS-64 |
JLS-65 |
|---|---|---|---|---|
2025-12-16 13:14:25 |
0.00 |
0.00 |
0.00 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
0.00 |
0.00 |
0.00 |
JLS-53 | Reviewed: ⨯ | Score: 0.0#
Given the same version of the repository, configuration, and documented environment, the build process shall produce identical single-include header files.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Construction of releases for the nlohmann/json library is fully repeatable and the results are fully reproducible, with any exceptions documented and justified. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
Item |
Summary |
Score |
Status |
|---|---|---|---|
The SHA value of the nlohmann/json library in use within eclipse-score/inc_nlohmann_json coincides with the SHA value provided by Niels Lohmann for that version. |
0.00 |
⨯ Item Reviewed |
References:
None
Fallacies:
None
Graph:
date-time |
JLS-53 |
JLS-14 |
|---|---|---|
2025-12-16 13:14:25 |
0.00 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
0.00 |
JLS-61 | Reviewed: ⨯ | Score: 0.0#
Any Github Actions used in the testing process are pinned to specific commit SHAs to ensure consistent action behavior across runs.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All tests for the nlohmann/json library, and its build and test environments, are constructed from controlled/mirrored sources and are reproducible, with any exceptions documented. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/blob/55f93686c01528224f448c19128836e7df245f72/.github/workflows/ubuntu.yml
https://github.com/nlohmann/json/blob/55f93686c01528224f448c19128836e7df245f72/.github/workflows/ubuntu.yml
- Ubuntu workflow of nlohmann/json release v3.12.0, containing specific commit SHAs for all Github Actions, in e.g., lines 24, 36, 118, etc.
Fallacies:
None
Graph:
date-time |
JLS-61 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-62 | Reviewed: ⨯ | Score: 0.0#
All tests except the ones which are excluded when using the flag “ctest -LE not_reproducible” are reproducible.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All tests for the nlohmann/json library, and its build and test environments, are constructed from controlled/mirrored sources and are reproducible, with any exceptions documented. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/blob/55f93686c01528224f448c19128836e7df245f72/README.md?plain=1#L1862
https://github.com/nlohmann/json/blob/55f93686c01528224f448c19128836e7df245f72/README.md?plain=1#L1862
- README file of the nlohmann/json repository stating that ‘ctest -LE not_reproducible’ excludes non-reproducible tests.
website: https://github.com/nlohmann/json/issues/2324
https://github.com/nlohmann/json/issues/2324
- Issue in the nlohmann/json repository addressing tests that were non-reproducible.
TSF/docs/non_reproducible_tests.md
# Non-Reproducible Tests in nlohmann/json
## Overview
The nlohmann/json test suite includes 8 CMake integration tests (comprising 10 individual test executions) that are labeled as `not_reproducible`. These tests verify different methods of consuming the library in CMake-based projects but create build artifacts with timestamps and cached state that prevent bit-for-bit reproducible builds.
The non-reproducibility of all these tests is reasonable and justified.
These can be identified and found by searching for "LABELS not_reproducible" in the project.
## The 8 Non-Reproducible CMake Integration Tests
### 1. cmake_import (2 tests)
**Location:** `tests/cmake_import/`
**Test Names:**
- `cmake_import_configure`
- `cmake_import_build`
**What it tests:**
Verifies that the library can be found and used via CMake's `find_package(nlohmann_json)` mechanism after installation. This simulates how end users would consume an installed version of the library.
**Why non-reproducible:**
- Requires `JSON_Install=ON` to generate CMake config files (`nlohmann_jsonConfig.cmake`)
- Uses `find_package()` which searches for and reads installed CMake configuration files
- Generates CMakeCache.txt with absolute paths and timestamps in the test build directory
- Creates compiled executables (`with_namespace_target`, `without_namespace_target`) with timestamps
- Multiple runs produce different timestamps on all generated artifacts
---
### 2. cmake_import_minver (2 tests)
**Location:** `tests/cmake_import_minver/`
**Test Names:**
- `cmake_import_minver_configure`
- `cmake_import_minver_build`
**What it tests:**
Similar to `cmake_import`, but specifically tests that CMake's version constraint mechanism works correctly with `find_package(nlohmann_json 3.2.0 REQUIRED)`. Ensures the library properly declares its version in the CMake config files.
**Why non-reproducible:**
- Same reasons as `cmake_import`
- Additionally verifies version metadata in generated config files
- Creates timestamped build artifacts and CMake cache files
---
### 3. cmake_add_subdirectory (2 tests)
**Location:** `tests/cmake_add_subdirectory/`
**Test Names:**
- `cmake_add_subdirectory_configure`
- `cmake_add_subdirectory_build`
**What it tests:**
Verifies that the library can be embedded directly into a CMake project using `add_subdirectory()`. This is the most common integration method for projects that vendor their dependencies.
**Why non-reproducible:**
- Runs a nested CMake configuration in an isolated test directory
- Generates CMakeCache.txt with absolute source/binary paths and timestamps
- Builds multiple executables (`with_namespace_target`, `without_namespace_target`, `without_exceptions`)
- Each build creates object files, executables with embedded timestamps
- Subsequent runs create new artifacts with different timestamps
- CMake's incremental build cache persists between runs, causing different behavior
---
### 4. cmake_target_include_directories (2 tests)
**Location:** `tests/cmake_target_include_directories/`
**Test Names:**
- `cmake_target_include_directories_configure`
- `cmake_target_include_directories_build`
**What it tests:**
Verifies that the library headers can be used via direct `target_include_directories()` calls with both `PRIVATE` and `SYSTEM` variants. Also tests a specific regression from [discussion #2281](https://github.com/nlohmann/json/discussions/2281) regarding transitive include directories.
**Why non-reproducible:**
- Creates a full CMake build with libraries (Foo, Bar) and executables
- Generates CMakeCache.txt with absolute paths to source includes
- All compiled artifacts have timestamps
- Static libraries and executables accumulate with different modification times on each run
---
## Assertions and Reproducibility
Additionally, the nlohmann/json README notes that **assertions must be disabled** for fully reproducible builds (see [discussion #4494](https://github.com/nlohmann/json/discussions/4494)). This is because:
- Assertions often use `__FILE__` which embeds absolute source paths
- Different build machines have different paths
- This embeds environment-specific information into binaries
To ensure reproducible builds, compile with:
```bash
cmake -DCMAKE_BUILD_TYPE=Release # Typically defines NDEBUG
# or explicitly:
cmake -DCMAKE_CXX_FLAGS="-DNDEBUG"
```
## Running Reproducible Tests Only
To run only reproducible tests:
```bash
# Exclude non-reproducible tests
ctest -LE not_reproducible
# Exclude both non-reproducible and git-required tests
ctest -LE "not_reproducible|git_required"
```
The project has a dedicated CI target that automatically excludes these tests:
```bash
cmake --build build --target ci_reproducible_tests
```
Description: Overview and description of 8 non-reproducible CMake integration tests.
Fallacies:
None
Graph:
date-time |
JLS-62 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-63 | Reviewed: ⨯ | Score: 0.0#
Releases of the nlohmann/json library contain commit sign-offs.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Releases of the nlohmann/json library contain a comprehensive set of attestations. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://www.bestpractices.dev/en/projects/289?criteria_level=1#section_project_oversight
https://www.bestpractices.dev/en/projects/289?criteria_level=1#section_project_oversight
- OpenSSF Best Practices Report for nlohmann/json, where within ‘Project Oversight’ it states that the Developer Certificate of Origin (DCO) is enforced via the DCO GitHub app (https://github.com/settings/installations/58991705) as of December 30, 2024.
website: https://developercertificate.org/
https://developercertificate.org/
- The Developer Certificate of Origin in its original formulation.
website: https://github.com/nlohmann/json/discussions/4578
https://github.com/nlohmann/json/discussions/4578
- Announcement by Niels Lohmann that as of January 1, 2025, the project will require DCO (https://developercertificate.org) for all merge requests.
Fallacies:
None
Graph:
date-time |
JLS-63 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-64 | Reviewed: ⨯ | Score: 0.0#
Releases of the nlohmann/json library contain Github pull request approvals.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Releases of the nlohmann/json library contain a comprehensive set of attestations. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://github.com/nlohmann/json/discussions/5022
https://github.com/nlohmann/json/discussions/5022
- Clarification by @nlohmann that all PRs not created by @nlohmann will be reviewed by him, while not all of his PRs are reviewed in practice.
website: https://github.com/nlohmann/json/pulls?q=is%3Apr+is%3Aclosed+is%3Amerged
https://github.com/nlohmann/json/pulls?q=is%3Apr+is%3Aclosed+is%3Amerged
- List of all merged Pull Requests of the nlohmann/json repository.
Fallacies:
None
Graph:
date-time |
JLS-64 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-65 | Reviewed: ⨯ | Score: 0.0#
Releases of the nlohmann/json library contain continuous-integration test reports such as coverage reports, fuzzing reports & OpenSSF scorecard reports.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
Releases of the nlohmann/json library contain a comprehensive set of attestations. |
0.00 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
https://scorecard.dev/viewer/?uri=github.com%2Fnlohmann%2Fjson
- OpenSSF Scorecard Report for the nlohmann/json library.
website: https://coveralls.io/github/nlohmann/json
https://coveralls.io/github/nlohmann/json
- Coverage Report for the nlohmann/json library.
website: https://introspector.oss-fuzz.com/project-profile?project=json
https://introspector.oss-fuzz.com/project-profile?project=json
- Most recent report for fuzzing introspection of the nlohmann/json library with historical plots.
Fallacies:
None
Graph:
date-time |
JLS-65 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |
JLS-66 | Reviewed: ⨯ | Score: 0.0#
The mirror of nlohmann/json is configured via infrastructure under direct control, and rejects history rewrites.
Supported Requests:
Item |
Summary |
Score |
Status |
|---|---|---|---|
All sources and tools for the nlohmann/json library are mirrored in our controlled environment. |
0.18 |
⨯ Item Reviewed |
Supporting Items:
None
References:
website: https://otterdog.eclipse.org/projects/automotive.score/repos/inc_nlohmann_json#rulesets
https://otterdog.eclipse.org/projects/automotive.score/repos/inc_nlohmann_json#rulesets
- Dashboard of the infrastructure configuration tool ‘Otterdog’, showing that the main branch of eclipse-score/inc_nlohmann_json is protected and requires code reviews for all merge requests.
Fallacies:
None
Graph:
date-time |
JLS-66 |
|---|---|
2025-12-16 13:14:25 |
0.00 |
2025-12-16 14:03:34.403563 |
0.00 |