vuln-cpe/README.md

vuln-cpe – NVD Official CPE Dictionary + CPE Match Criteria

Standalone raw-data mirror for Gitea Actions, using a persistent Git checkout on the existing vuln-runner host.

Sources

• https://services.nvd.nist.gov/rest/json/cpes/2.0 — product CPE dictionary; max 10,000/page.
• https://services.nvd.nist.gov/rest/json/cpematch/2.0 — match criteria and version ranges; max 500/page.

The mirror stores one full API record wrapper per UUID at data/cpes/<prefix>/<uuid>.json and data/cpematch/<prefix>/<uuid>.json. No flattening or interpretation is done. Deprecated records are retained as such. Source entries removed without a modification event require a periodically scheduled full reconciliation; this implementation does not delete absent entries automatically.

Gitea setup

1. Create empty VULN/vuln-cpe repo and push these files to main.
2. Create /var/lib/act_runner/cache/vuln-cpe owned by act_runner (0700).
3. Runner label vuln-runner; normal Gitea secrets.GITEA_TOKEN needs write permission.
4. Put the same NVD_API_KEY as for vuln-nvd in the org Actions Secrets (recommended). Without key the requests are slower.
5. Manually start the workflow, then it runs every four hours (UTC cron).

Bootstrap and updates

Both feeds are separate, checkpointed and capped at 20 pages per run. The initial full import takes multiple runs, committing progress after every run; it does not mean all data are available after the first successful action. Watch state/cpes.json and state/cpematch.json: both must show "mode":"delta" before declaring the initial mirror complete.

After a full import, changes since the bootstrap start are replayed in 110-day windows with a 2-minute safety lag, to avoid gaps. Individual pages have a durable checkpoint; a crash during a page causes harmless replay on the next run. The HTTP API window is limited to 120 days. No data is deleted merely because it wasn't in a page.

Test

python3 -m unittest discover -s tests -v

Important limitations

• The NVD API imposes request limits; repeated runs for a large bootstrap can take days. Do not run multiple jobs in parallel with the same API key.
• A shallow Git checkout with thousands of records still requires disk space and pushes can be large.
• NVD CVE applicability tree (AND, OR, negate) remains in vuln-nvd; this repository stores only match criteria and CPE dictionary, not the applicability tree itself.
• A standalone full reconciliation/retirement audit should be scheduled periodically. NVD deprecations are retained with their status.