feat(parsers): SPDX, CSAF 2.0 and OpenVEX interchange-format parsers - #15414
Open
devGregA wants to merge 5 commits into
Open
feat(parsers): SPDX, CSAF 2.0 and OpenVEX interchange-format parsers#15414devGregA wants to merge 5 commits into
devGregA wants to merge 5 commits into
Conversation
DefectDojo supports CycloneDX but not SPDX, the other widely-used SBOM format and the one
named in several public-sector SBOM requirements. Any tool that emits SPDX now becomes
importable.
Behaviour mirrors the CycloneDX parser deliberately, so the same SBOM does not behave
differently depending on which format it was exported in:
* packages[] become component/dependency inventory recorded on the test as
LocationData.dependency (purl, name, version, checksums, license expression). They do
NOT become findings. An installed package is not a weakness, and one finding per
package would fill a product with rows nobody can remediate.
* findings come only from a real weakness signal.
SPDX 2.x has no vulnerability container at all, so the only weakness signal the format
offers is a SECURITY external reference of an advisory type (advisory/fix/url/swid) whose
locator or comment names a CVE or GHSA.
The central judgement call: SPDX also files CPE identifiers under referenceCategory
SECURITY, as cpe22Type/cpe23Type. A CPE states what a package IS - it is an identifier,
not a vulnerability. Treating the category alone as a weakness signal would turn every
inventoried package into a bogus finding; the real syft output used for these fixtures
contains 68 such refs across 15 packages. CPEs are therefore recorded in the description
for identification and never create findings.
test_cpe_references_never_become_findings is the regression guard, and it asserts the
fixture really contains cpe23Type so it cannot pass vacuously.
Severity is Medium for every finding, because an SPDX advisory reference carries no
severity, score or vector. Medium rather than Info is deliberate: a real, named CVE must
not be filtered out of sight by a minimum-severity setting.
Licenses go onto the component's location as its license expression (concluded preferred
over declared), the same field CycloneDX populates. A license is a fact about a component,
not a weakness, so no license findings are manufactured.
SPDX 3.0 uses a different JSON-LD serialisation and is REJECTED with an actionable
message rather than silently parsing zero packages out of a valid SBOM - a silent empty
import would look like success.
Tag-value support converts the document into the JSON shape and calls the same mapping
code, so both serialisations are guaranteed identical; a test asserts that.
Fixtures are real tool output: syft 1.50.0, `syft alpine:3.10 -o spdx-json`, against a
locally pulled image via the local Docker daemon only. Sanitised by rewriting
documentNamespace to example.com, pinning the timestamp, dropping 61 files/110
relationships the parser never reads, and trimming to a few packages with their real
field values intact. The advisory references were added by hand, spec-conformant and
pointing at example.com, because syft is an SBOM generator and emits no advisory refs -
disclosed rather than presented as scanner output.
ruff (pinned 0.15.22) clean. 36 mapping assertions verified locally.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
OpenVEX implements the VEX model: a producer stating whether their product is actually
affected by a known vulnerability.
VEX IS A SUPPRESSION SIGNAL, NOT A FINDING SOURCE.
The purpose of a VEX document is to tell a consumer which vulnerabilities they can stop
worrying about. A not_affected statement is the producer saying that a scanner flagged
something and they are telling you it is not exploitable in their product. Importing that
as an active finding would invert the document's meaning - reassurance would arrive as new
work - and would leave DefectDojo worse off than not reading VEX at all.
not_affected and fixed statements are therefore imported INACTIVE so they suppress
matching findings. test_not_affected_is_never_active is the regression guard.
Full status matrix:
not_affected inactive; out_of_scope / false_p / is_mitigated by justification
fixed inactive; is_mitigated
affected ACTIVE - the producer asserts real exposure
under_investigation active but verified=False, Info severity - undecided
unrecognised ACTIVE - a status a future spec revision adds must never silently
suppress a real vulnerability
affected is deliberately the one status that stays active: it is the producer stating the
product IS exposed, and its action_statement is the only actionable instruction the format
carries. Suppressing it would discard the most useful statement in VEX.
The five not_affected justifications say materially different things and DefectDojo has
distinct fields that match them, so each maps individually rather than collapsing into one
disposition: inline_mitigations_already_exist maps to is_mitigated because a real
mitigation exists; vulnerable_code_not_present, vulnerable_code_not_in_execute_path and
vulnerable_code_cannot_be_controlled_by_adversary map to false_p because the match is not
real exposure; component_not_present maps to out_of_scope. A missing justification defaults
to out_of_scope and is called out in the description as an incomplete document. The
justification and any impact_statement are always recorded so the reasoning stays
auditable.
Severity: VEX states exploitability, not impact, so it carries none. affected is Medium so
a producer-asserted exposure is not filtered out of sight by a minimum-severity setting;
everything else is Info, because those findings exist to suppress and ranking them would
push suppressed rows in front of real ones.
Both spec shapes are read: v0.0.1 used bare strings for vulnerability and products, v0.2.0
uses objects. Older producers still emit v0.0.1, and failing to read it would silently drop
suppression statements - the worst failure mode for a VEX parser. Aliases on a v0.2.0
vulnerability object join the primary identifier.
Fixtures are real tool output: vexctl v0.4.4 (the OpenVEX project reference CLI, built and
run in a golang:1.26 container), three vexctl create invocations for the not_affected,
affected and fixed statuses. Sanitised by pinning the random @id digest and timestamps and
replacing the default author string with "Example Org". The under_investigation statement,
the inline-mitigations statement and the v0.0.1 bare-string statement were added by hand
because vexctl has no flag to produce them; all are spec-conformant and this is disclosed
rather than presented as tool output.
ruff (pinned 0.15.22) clean. 32 mapping assertions verified locally, covering every status
and every justification.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CSAF 2.0 is the OASIS standard for machine-readable vendor security advisories. One finding per (vulnerability, PRODUCT) pair. A single advisory routinely covers many products in different states - one patched, another exposed, a third explicitly unaffected - so collapsing to one finding per vulnerability would lose exactly the information the advisory exists to convey. known_not_affected products are imported INACTIVE (false_p). That bucket is the vendor explicitly stating a product is not exposed; importing it as active would turn a statement of non-exposure into work for the reader. Full matrix: known_affected / first_affected / last_affected ACTIVE known_not_affected inactive, false_p fixed / first_fixed / recommended inactive, is_mitigated under_investigation active, verified=False unrecognised bucket ACTIVE (never silently suppress) Product-tree resolution is the substantial part, in its own module. A CSAF product id is never defined in one place, and the parser indexes all three mechanisms: the flat full_product_names list, the recursive branches tree, and relationships (which synthesise a NEW product id for a component-of relationship). For branch-shaped trees the version lives on a product_version branch rather than the leaf, so the tree is walked carrying the nearest enclosing version down. Where the leaf name repeats that version it is stripped, so the two shapes converge on the same component_name and component_version - otherwise the same product would deduplicate differently depending on how the advisory happened to describe it. A test asserts the convergence. scores[] entries are scoped to a product list, so the same CVE can be scored differently per product; each finding takes its own product score, vector and baseSeverity. remediations[] are likewise filtered per product. An advisory carrying NO score becomes Medium, not Info - a published vendor advisory is not informational, and Info would hide it behind a minimum-severity setting. A vulnerability with no CVE yet (embargoed) falls back to its ids[] tracking id. CSAF 1.x used the older CVRF schema and is rejected with a message naming CVRF, rather than mis-parsed. Fixture provenance differs from the SBOM parsers and is worth stating plainly: no scanner generates CSAF. A CSAF document is authored by a product security team and published, not produced by running a tool over a target, so there is no run-the-tool step available. The three fixtures are authored against the CSAF 2.0 schema and verified with csaf_validator from github.com/gocsaf/csaf/v3 (the reference Go implementation) - all three pass schema validation. The docs page explains how to obtain real advisories via csaf_downloader instead. That validator also corrected a spec assumption: CSAF 2.0 sets minItems=1 on vulnerabilities, so an advisory with none must OMIT the key rather than carry an empty array. The no_vuln fixture reflects that. ruff (pinned 0.15.22) clean. 30 mapping assertions verified locally, covering every product_status bucket, both product-tree shapes and the relationship case. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…types SPDX Scan, OpenVEX Scan and CSAF Scan all use hash_code over [vuln_id_from_tool, component_name, component_version], matching CycloneDX. For SPDX that means the same SBOM deduplicates identically whether it was exported as CycloneDX or SPDX. For OpenVEX it is load-bearing: a VEX statement has to deduplicate ONTO the SBOM finding for the same component and CVE, which is exactly how a suppression is meant to land. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
github-actions
Bot
added
settings_changes
…3_FEATURE_LOCATIONS Fixes the five CI failures on this branch. Both were defects in the supporting files, not in the parser mapping logic, which CI otherwise exercised clean. 1. unittests.test_parsers.TestParsers.test_file_existence failed for all three parsers with category='docs'. The docs path was correct; the content assertion was not. test_parsers.py requires the literal heading "### Sample Scan Data" via a case-sensitive assertRegex, and all three pages used "### Sample scan data". Renamed the heading in each. 2. test_no_vuln_still_records_component_inventory and test_checksums_become_location_hashes failed because both assert on test.unsaved_metadata, which the parser only populates when settings.V3_FEATURE_LOCATIONS is enabled - matching how the CycloneDX parser gates the same behaviour. The rest-framework workflow runs the suite with that flag both on and off (docker-compose.override.unit_tests_cicd.yml defaults it to False), so the two tests failed in the off half of the matrix. Both are now decorated with the existing skip_unless_v3 helper from unittests.dojo_test_case rather than weakening the assertions, since the behaviour under test genuinely only exists when the flag is on. The parsers themselves are unchanged. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three interchange-format parsers: SPDX, CSAF 2.0, OpenVEX
Three parsers for open interchange formats. Between them they make a large number of tools importable,
because any tool that emits one of these formats becomes supported without further work. Modelled on the
existing
dojo/tools/cyclonedx/anddojo/tools/sarif/parsers.Base branch is
devperreadme-docs/CONTRIBUTING.md("Base your PR against thedevorbugfixbranch") — these are new features.
No registry edit is needed:
dojo/tools/factory.pyauto-discoversdojo/tools/<dir>/parser.py. The onlyshared files touched are the two dicts in
settings.dist.py.I confirmed none of the three already exist under another name:
dojo/tools/contains nospdx,csaf,vexorsbomdirectory, and no existing parser referencesspdxVersion,SPDXRef,csaf_versionoropenvexinternally.1. SPDX —
dojo/tools/spdx/— scan typeSPDX ScanDefectDojo supports CycloneDX but not SPDX, the other widely-used SBOM format and the one named in
several public-sector SBOM requirements.
Supported: SPDX 2.2 / 2.3 in JSON (
.json) and tag-value (any other extension). Bothserialisations go through one mapping function, so a document exported either way produces identical
findings — asserted by
test_tag_value_matches_json.SPDX 3.0 is rejected with an actionable message. 3.0 uses a JSON-LD
@graphserialisation this parserdoes not understand; silently parsing zero packages out of a valid SBOM would look like a clean import of
an empty document, which is worse than an error.
Packages are inventory, not findings
SPDX 2.x has no vulnerability container at all. This parser follows the CycloneDX parser exactly
(verified, not assumed — its JSON parser creates findings only from
vulnerabilities[]and routescomponents to
test.unsaved_metadataasLocationData.dependency):packages[]→ component inventory recorded on the test (PURL, name, version, checksums, licenseexpression). Not findings. An installed package is not a weakness, and one finding per package
would fill a product with rows nobody can remediate.
advisory,fix,url,swid) whose locator or comment names aCVE-…orGHSA-….CPEs are identity, not vulnerabilities
This is the parser's main judgement call and the reason for a dedicated regression test.
SPDX files CPE identifiers under
referenceCategory: SECURITYas well, using reference typescpe22Type/cpe23Type. A CPE states what a package is — it is an identifier, not a weakness.Treating the category alone as a weakness signal would turn every inventoried package into a bogus
finding: the real syft output these fixtures came from contains 68 CPE references across 15 packages,
so a clean SBOM would have produced 68 false findings.
CPEs are therefore recorded in the finding description for identification and never create findings.
test_cpe_references_never_become_findingsguards this, and it asserts the fixture really does containcpe23Typeso the test cannot pass vacuously.Licenses are carried on the component's location as its license expression (
licenseConcludedpreferred over
licenseDeclared), the same field CycloneDX populates. No license findings aremanufactured — a license is a fact about a component, not a weakness.
Severity is Medium for every finding, because an SPDX advisory reference carries no severity, score or
vector. Medium rather than Info is deliberate: a real, named CVE must not be filtered out of sight by a
minimum-severity setting.
NOASSERTION/NONEare SPDX placeholders meaning "unknown" and are treated as absent throughout.Sample converted finding
Fixture provenance
Real tool output. syft 1.50.0:
docker run --rm -v /var/run/docker.sock:/var/run/docker.sock \ anchore/syft:latest alpine:3.10 -o spdx-json > sbom.spdx.jsonTarget was a locally-pulled
alpine:3.10image, scanned through the local Docker daemon only.Sanitisation:
documentNamespacerewritten to anexample.comURI (syft embeds a random UUID and thelocal hostname);
creationInfo.createdpinned so fixtures are byte-stable; the 61filesand 110relationshipsentries the parser never reads dropped so the fixture stays reviewable; trimmed to 3–4packages with their real field values verbatim.
Disclosed: the advisory external references were added by hand, not emitted by syft. syft is an
SBOM generator and emits only
cpe23Typeidentity references, so a real advisory ref could not becaptured. The added refs are spec-conformant (SPDX 2.3 §11.21) and point at
example.com. Flagging thisrather than presenting it as scanner output.
2. CSAF 2.0 —
dojo/tools/csaf/— scan typeCSAF ScanThe OASIS standard for machine-readable vendor security advisories. CSAF 1.x used the older CVRF schema
and is rejected with a message naming CVRF.
One finding per (vulnerability, PRODUCT) pair. A single advisory routinely covers many products in
different states — one patched, another exposed, a third explicitly unaffected. Collapsing to one finding
per vulnerability would discard exactly the information the advisory exists to convey.
known_not_affectedis imported inactiveproduct_statusbucketknown_affected,first_affected,last_affectedknown_not_affectedfalse_pfixed,first_fixed,recommendedis_mitigatedunder_investigationverified=FalseThat bucket is the vendor explicitly stating a product is not exposed. Importing it as an active finding
would turn "you are not affected" into work for the reader.
Product-tree resolution (its own module)
A CSAF product id is never defined in one place.
dojo/tools/csaf/product_tree.pyindexes all threemechanisms: the flat
full_product_names[], the recursivebranches[]tree, andrelationships[](which synthesise a new product id for "library as a component of app").
For branch-shaped trees the version lives on a
product_versionbranch, not the leaf, so the tree iswalked carrying the nearest enclosing version down. Where the leaf name repeats that version
("Generic App 1.0.0") it is stripped, so both shapes converge on the same
component_name/component_version— otherwise the same product would deduplicate differentlydepending on how the advisory happened to describe it.
test_branch_shaped_product_tree_resolves_versionsasserts the convergence.scores[]andremediations[]are both scoped to product lists, so each finding takes its own product'sscore, vector,
baseSeverityand remediations. An advisory carrying no score becomes Medium, notInfo — a published vendor advisory is not informational, and Info would hide it behind a minimum-severity
setting. A vulnerability with no CVE yet (embargoed) falls back to its
ids[]tracking id.Sample converted finding
Fixture provenance — different from the other two
No scanner generates CSAF. A CSAF document is authored by a product security team and published; it is
not produced by running a tool over a target. There is no "run the tool" step available for this parser.
The three fixtures are therefore authored against the CSAF 2.0 schema and verified with
csaf_validatorfromgithub.com/gocsaf/csaf/v3, the reference Go implementation, run in a container.All three pass schema validation. The docs page explains how to obtain real advisories with
csaf_downloaderfrom a vendor's/.well-known/csaf/feed.That validator also corrected a spec assumption: CSAF 2.0 sets
minItems: 1onvulnerabilities, so anadvisory with none must omit the key rather than carry an empty array. The
no_vulnfixture reflectsthat. (The validator additionally warns that a CSAF filename should match
document/tracking/id; theDefectDojo
<tool>_no_vuln.jsonfixture convention takes precedence in the test suite.)3. OpenVEX —
dojo/tools/openvex/— scan typeOpenVEX ScanVEX is a suppression signal, not a finding source
The purpose of a VEX document is to tell a consumer which vulnerabilities they can stop worrying about. A
not_affectedstatement is the producer saying "your scanner flagged this, and we are telling you it isnot exploitable here." Importing that as an active finding would invert the document's meaning —
reassurance would arrive as new work — and would leave DefectDojo worse off than not reading VEX at all.
not_affectedandfixedare therefore imported inactive, so they suppress rather than report.test_not_affected_is_never_activeis the regression guard.not_affectedout_of_scope/false_p/is_mitigated, by justificationfixedis_mitigatedaffectedunder_investigationverified=False, Infoaffectedis deliberately the one status that stays active: it is the producer stating the product isexposed, and its
action_statementis the only actionable instruction VEX carries. Suppressing it woulddiscard the most useful statement in the format.
Justifications map individually
The five permitted justifications say materially different things, and DefectDojo has distinct fields that
match them, so collapsing all five into one disposition would throw away information the producer took the
trouble to state:
justificationinline_mitigations_already_existis_mitigatedvulnerable_code_not_presentfalse_pvulnerable_code_not_in_execute_pathfalse_pvulnerable_code_cannot_be_controlled_by_adversaryfalse_pcomponent_not_presentout_of_scopeout_of_scopeSeverity: VEX states exploitability, not impact, so it carries none.
affectedis Medium so aproducer-asserted exposure is not filtered out of sight; everything else is Info, because those findings
exist to suppress and ranking them would push suppressed rows in front of real ones.
Both spec shapes are read. v0.0.1 used bare strings for
vulnerabilityandproducts; v0.2.0 usesobjects. Older producers still emit v0.0.1, and failing to read it would silently drop suppression
statements — the worst failure mode for a VEX parser.
Sample converted finding
Fixture provenance
Real tool output.
vexctlv0.4.4, the OpenVEX project's own reference CLI, built and run in agolang:1.26container:(plus
affectedandfixedvariants — all three commands are in the docs page).Sanitisation: the random
@iddigest and all timestamps pinned; vexctl's"Unknown Author"replacedwith
"Example Org". Statement bodies keep vexctl's real field names and values.Disclosed: the
under_investigationstatement, theinline_mitigations_already_existstatement andthe v0.0.1 bare-string statement were added by hand — vexctl has no flag to produce them. All are
spec-conformant.
Shared settings
dojo/settings/settings.dist.py, both dicts:All three match CycloneDX deliberately. For SPDX it means the same SBOM deduplicates identically in either
format. For OpenVEX it is load-bearing: a VEX statement must deduplicate onto the SBOM finding for the
same component and CVE, which is exactly how a suppression is meant to land.
Validation
ruffat the repo's pinned 0.15.22 — clean across all three parsers, all three test modules andsettings.dist.py. (Run viauvx, because a newer locally-installed ruff cannot parse this repo'sruff.toml.)bucket, every justification, both SBOM serialisations, both product-tree shapes and every rejection
path.
csaf_validatorschema validation.example.com, "Example Corp", "Generic App"), no credentialsand no PII. The only non-
example.comURLs are real upstream OSS project homepages(
alpinelinux.org,busybox.net) that appear in syft's own output, plus specification URLs.Not run:
./run-unittest.sh, which needs the full docker-compose stack in dev mode. The 46DojoTestCasetests are written but have not executed under the official runner — the assertions abovewere verified with a harness that stubs only
dojo.modelsanddjango.conf.settingsand loads the realdojo.tools.locations.LocationData, so the mapping code exercised is the shipped code. Please treat CIas the first authoritative run of the DojoTestCase suite.
Bugs the tests caught during development
("Generic App 1.0.0", "1.0.0")for branch-shaped trees — the versionwas left duplicated in the component name, which would have deduplicated the same product differently
depending on the advisory's shape.
no_vulnfixture was schema-invalid (vulnerabilities: []violatesminItems: 1) — caught bythe reference validator.
🤖 Generated with Claude Code