Initial Red Hat CSAF mirror

2026-11-10 21:28:10 +01:00
commit e6a108b4b0
9 changed files with 340 additions and 0 deletions
@@ -0,0 +1,45 @@
+# vuln-redhat – Red Hat CSAF mirror
+
+Mirrors the **original Red Hat CSAF JSON** from [the official advisory feed](https://security.access.redhat.com/data/csaf/v2/advisories/). The repository retains advisories by directory year starting in 2017. No transformation is applied to the JSON bytes. The authoritative `index.txt` is fetched every run; `changes.csv` is used as an update hint, and a rotating set of 250 entries is always rechecked as additional reconciliation.
+
+## Server installation (Debian/Ubuntu, host-based Gitea runner)
+
+```bash
+sudo apt install -y python3 git util-linux ca-certificates
+sudo install -d -o act_runner -g act_runner -m 700 /var/lib/act_runner/cache/vuln-redhat
+```
+
+Create `VULN/vuln-redhat` in Gitea and push the contents to `main`. The workflow runs on the already-registered **`vuln-runner`**. It uses the built-in `${{ secrets.GITEA_TOKEN }}`; no additional PAT is required if this already works for other repos. Allow job token repository write access.
+
+## Run
+
+Open `https://git.apex-project.de/VULN/vuln-redhat/actions`, select **Red Hat CSAF Mirror**, and run manually. Subsequent runs are scheduled every four hours at minute 53 (Gitea scheduler time zone).
+
+```bash
+# Direct sync for development, without Git:
+python3 scripts/sync.py --since-year 2017
+python3 scripts/validate.py
+
+# Only first 5 records, without advancing stored state:
+python3 scripts/sync.py --limit 5 --rotate 0
+```
+
+## Output
+
+```text
+data/csaf/2017/rhsa-2017_001.json
+...
+state/sync.json
+```
+
+### Behavior and limitations
+
+- The first sync can download tens of thousands of JSON documents and create a very large Git commit; leave enough disk space and allow several hours.
+- New advisory files are discovered using `index.txt`. Changed entries are identified opportunistically via `changes.csv` plus a rotating audit. **This is not a guaranteed near-real-time change feed**: a historical advisory can take multiple runs to be revisited if it is not recognizable in the changes feed.
+- Files missing from a successfully retrieved authoritative index are pruned only when the count is below an explicit mass-deletion safety threshold. A suspicious deletion count fails the job instead.
+- Any HTTP/JSON error stops the run, and neither the state nor Git commit advances. The persistent checkout is retained for investigation on failures; after fixing, revert only unintended local edits before re-running.
+- Run jobs on trusted branches only, because a host runner executes scripts as its local Linux account.
+- License: Red Hat security data is CC BY 4.0. Preserve attribution and link to the original source if you redistribute it.
+- Full live download is not part of the provided local tests.
+
+Source: https://access.redhat.com/security/data