# vuln-builder – SQLite Starter (v1)

First functional, **conservative** builder for the Gitea organization `VULN`.

## Scope

- Clones/fetches all named repositories in a persistent Docker volume.
- Keeps the source Git commit for **every** mirror in `sources`.
- Indexes **CVEProject (`cveMetadata`)**, **NVD (`cve`)** and **OSV (`id`/`affected`)** into SQLite.
- Saves OSV package/version range events without guessing at version comparisons.
- Saves only simple non-negated NVD CPE disjunctions; complex AND/NOT expressions are **not** converted into potentially incorrect matches.
- Other vendors and distro data are **not yet normalized**. This is not yet a complete Trivy-compatible scanner DB.
- Writes `master.db` (local staging), compressed `master.db.zst`, and on subsequent builds `delta-OLD-to-NEW.jsonl.zst`, plus `manifest.json`.
- **No automated release publishing yet**: files remain on the Docker host volume at `/vuln-cache/vuln-builder/releases`.

## Install

1. Create empty `VULN/vuln-builder` in Gitea, then push this folder to branch `main`.
2. Ensure Docker runner with label `vulnerability` is online and the named volume `gitea-vuln-cache` is mounted in **job** containers at `/vuln-cache`.
3. Configure Gitea Actions read permission for the Gitea token across all `VULN` repositories (a job token may be scoped to a single repo; if cross-repo reads fail, create a separate read-only organization credential named `VULN_READ_TOKEN`, and change only the workflow's secret reference). Do not log credentials.
4. Start `.gitea/workflows/build.yml`. Cron: daily 03:35 UTC. Initial download can be *very* large. Make sure all source mirrors finished their first import.

## Manual release inspection

On Docker host:

```sh
docker run --rm -v gitea-vuln-cache:/vuln-cache:ro alpine ls -lh /vuln-cache/vuln-builder/releases
```

## Safety and limitations

- Input source repositories need valid Git checkout and the three core sources must exist.
- A successful first-build **does not** imply all feeds are fully imported; check their source-specific completion markers before starting.
- SQLite v1 is an index, not a final vulnerability scanner: no Debian/RPM version comparison, no complete CPE boolean-expression evaluator, no CSAF product-tree traversal yet.
- Delta JSONL updates have `{table, op, key}` for deletion and `{table, op, row}` for upserts. A client must verify the starting version, apply within one SQLite transaction with foreign key constraints managed correctly, and only mark the new version after integrity checks. This package does **not** yet implement that client.
- The source data can require many GB in the persistent Git cache. Avoid resetting/deleting the volume to save time.
- `master.db` remains uncompressed on the server for diffing against the next build.
- The master artifact and manifest are replaced one after another; consumers must only use `manifest.json` after it is written and verify the published SHA-256.

## Tests

```sh
python3 -m unittest discover -s tests -v
```