CBOMkit vs CRYPTAGION: two CBOMs of the same repository, compared line by line

Published 12 October 2026 · 8 min read · By Ali Korsi Comparison pages usually rest on feature tables. This one rests on two CBOMs of the same commit, published in full, so you can check every number — including the ones that do not flatter us.

In short

• Both tools produced a schema-valid CycloneDX 1.6 CBOM, and both gave identical results on a repeat run.
• They looked at different things: CBOMkit skipped the test suite and scanned the Java and Go helper scripts; our run covered every Python file, tests included, and tagged each finding by scope.
• They found different things: only one source line appears in both CBOMs. In the library’s own code, CBOMkit reported 4 lines in 2 files; CRYPTAGION reported 46 detections in 11 files.
• Each caught something the other missed, and each mislabelled DSA as Diffie-Hellman once. Neither count is a quality score; the details below are what matter.

What we ran

The target is pyca/cryptography at commit d45b793, the same commit as our published pyca CBOM example. CBOMkit is the open-source CBOM toolset maintained under the Post-Quantum Cryptography Alliance (PQCA) at the Linux Foundation, Apache-2.0 licensed. We ran release 2.3.0 (September 2026) as a service with its default settings, the way its quickstart describes, and called its scan API twice. The CBOMkit service checks out branches or tags, so we pinned the commit through a local read-only mirror with a branch at d45b793; both CBOMs record that commit.

Everything is published: CBOMkit CBOM (and the repeat), progress logs, the deployment file, the scan and comparison scripts, the comparison output and a methodology note with image digests and limits. Our side is the existing CRYPTAGION CBOM and its validation record.

Results at a glance

Same file, different findings

src/cryptography/hazmat/primitives/serialization/ssh.py, which parses and writes SSH keys, is the only library file both tools reported on, and it shows how differently they work:

• CBOMkit reported the RSA key objects being built from SSH data — the RSAPublicNumbers and RSAPrivateNumbers calls on lines 317, 336 and 337 — each with the related public or private key as its own component. CRYPTAGION did not report those lines; it saw RSA only at the import on line 18.
• CRYPTAGION reported the ECDSA curves (P-256, P-384, P-521), ECDSA signing and verification, RSA PKCS#1 v1.5 padding and the SHA-1, SHA-256, SHA-384 and SHA-512 selections — 15 detections in the same file. CBOMkit reported none of them.

The same pattern holds across the run. CBOMkit resolves calls into cryptographic APIs and attaches what it can infer about them; CRYPTAGION records every place the code names an algorithm, including imports, curve tables and capability checks. In the documentation helpers, CBOMkit also covered the Java and Go scripts — for example the five Cipher.getInstance("RSA/ECB/OAEP…") calls, recorded with OAEP padding and a 2,048-bit key — which our Python-only run did not scan.

What CBOMkit does well

• It is free and open. Apache-2.0, maintained in the open under PQCA, with a CycloneDX output any tool can read — including ours.
• Call-level context. Where it resolves a call, it records parameters such as key size, padding and the operations used, and links keys to the algorithms that use them.
• Several languages in one pass. Java, Python and Go modules were indexed and scanned together.
• Less noise by default. Leaving test code out keeps the CBOM short; for many uses that is the right default.
• Fast, and easy to wire in. The scan itself took 5 to 7 seconds here, and there is a GitHub Action and a SonarQube plugin for pipelines.

Where each fell short on this target

• CBOMkit saw little of the library’s own code: 4 lines in 2 of its 104 source files. It labelled the DSA key generation in dsa.py as FFDH. It does not scan test code, so cryptography that only appears there is not in the CBOM.
• CRYPTAGION missed the RSA key construction lines CBOMkit caught, did not cover the Java and Go helpers in this run, and produced a raw count dominated by tests (1,746 of 1,804) that only makes sense with its scope tags. It also labelled one raw DSA test call as Diffie-Hellman, which our validation record already lists.

When to use which

CBOMkit is a good choice for a free first CBOM of an application, especially in Java or Python, in CI, when you have the cryptographic expertise to interpret the output and the time to run and maintain it.

CRYPTAGION is built for regulated organisations that need more than the inventory: production, test and vendored code kept apart, certificates and live TLS endpoints alongside code, a per-asset risk score with HNDL exposure, a migration roadmap and reports mapped to DORA, NIS2 and the CRA, delivered with support.

They are not exclusive. Both speak CycloneDX 1.6, and two CBOMs that disagree are worth more than one that nobody has checked. For the broader picture, see CRYPTAGION vs open-source CBOM tools, and to run the same test on any vendor, our CBOM tool evaluation checklist.

Limits of this comparison

• One repository, one commit, one machine. pyca/cryptography is a cryptographic library rather than an application that calls one, which gives call-resolving tools less to work with. Results on an application will differ, and we plan to repeat the exercise on one.
• CBOMkit ran as a service with default settings and without building the target. Its own documentation says a build (through the SonarQube plugin or the GitHub Action) improves completeness, particularly for Java.
• The CRYPTAGION run was Python-only by design and is the one published on 10 October; it was not re-run for this page.
• We maintain CRYPTAGION. We have published every input and output so that you do not have to take our reading on trust. Corrections are welcome at contact@cryptagion.io.

Sources

• CBOMkit repository (release 2.3.0, commit feace38)
• Methodology note, image digests and files for the CBOMkit run
• CRYPTAGION pyca/cryptography runbook and validation record
• CycloneDX 1.6 JSON reference

Independent comparison by CRYPTAGION. CBOMkit, PQCA and the pyca project have not reviewed or endorsed it. Last verified 12 October 2026 against CBOMkit 2.3.0.

← All resources · Home