# vuln-nvd — Gitea NVD mirror

NVD CVE API 2.0 mirror. Stores one complete **NVD `cve` JSON object** per `data/YYYY/CVE-YYYY-NNNN.json`, preserving metrics, configurations/CPEs, descriptions, references, weaknesses and status. No summary projection is performed.

## Requirements

- Python >= 3.10, Git, Linux Gitea host runner (`vuln-runner`)
- Access to `services.nvd.nist.gov`
- Recommended: NVD API key stored as Gitea Actions Secret **`NVD_API_KEY`** in `VULN/vuln-nvd → Settings → Actions → Secrets`. Without it requests are slower. Never commit the key.
- Gitea Actions job token must have repository write access for automatic push.

## Gitea installation

```bash
cd vuln-nvd
git init
git branch -M main
git add .
git commit -m 'Initial NVD mirror'
git remote add origin https://git.apex-project.de/VULN/vuln-nvd.git
git push -u origin main
```

Enable Actions in the Gitea repository. Run `.gitea/workflows/update.yml` manually once from Actions. The schedule then runs every two hours at minute 27 (server's configured scheduler timezone). Make sure your Gitea version recognizes `.gitea/workflows/` (otherwise move to `.github/workflows/`).

## Synchronization logic

- **First run**: publication-date windows (30 days each) beginning January 1, 2017, all pages (up to 2,000 records per page). Earlier CVE IDs published after 2017 are excluded based on ID year, by design.
- **Later runs**: last-modified windows with 48-hour overlap, so failures and recent updates are caught. The script filters on CVE ID year >= 2017.
- Retains rejected records with `vulnStatus=Rejected` (do not silently delete them; the central SQLite builder must treat them as non-actionable).
- Saves `state/sync.json` **only after all API requests complete**. A failed job will not advance the checkpoint. Because a failed job does not commit files, the next run may fetch some records again; it is idempotent.
- A missing record from an incremental API result is **not** proof of deletion. The builder must apply explicit rejected status. An occasional full reconciliation is recommended separately, not via ordinary hourly jobs.
- Gitea Git repository size will increase substantially over time, especially with frequent modifications. Monitor disk usage and Gitea limits.

## Local use

```bash
export NVD_API_KEY='your-key' # optional; do not put into your shell history if avoidable
python3 scripts/sync.py --since-year 2017
python3 scripts/validate.py --changed-only
```

Full reconciliation (may take hours):

```bash
python3 scripts/sync.py --mode full --since-year 2017
```

Full mode re-fetches all publication windows but does **not** remove disappeared files; deleting unmatched records requires a separate, explicitly verified reconciliation step.

## API and rate limiting

- API: `https://services.nvd.nist.gov/rest/json/cves/2.0`
- Max 120 days per date window; this mirror uses 30 days.
- Up to 2000 items per page.
- Default delays 6.5 s/request without a key, 0.8 s with one; backoff for transient errors.
- API key should be requested from NVD. Gitea token is separate.

## Data contract for SQLite builder

Input files: `data/<year>/CVE-<year>-<number>.json` containing the value of the `cve` object from each NVD API vulnerability item. `state/sync.json` provides the successful extraction watermark.