1 line
14 KiB
JSON
1 line
14 KiB
JSON
{"schema_version":"1.9.2","id":"GHSA-73v3-wf97-vpgj","published":"2026-11-12T22:54:59Z","modified":"2026-11-12T23:00:05.611968338Z","aliases":["CVE-2026-126366"],"summary":"msgpack-ruby JRuby decoder makes uncapped header-count allocations; a 5-byte message forces multi-GiB OOM","details":"### Summary\n\nThe **JRuby** implementation of msgpack (`msgpack` gem, `java` platform — pure Java decoder under `ext/java/`) sizes containers directly from the element count written in the message header, with **no cap at all**: no equivalent of the C unpacker's `SHRT_MAX` clamp (since 1.7.3) and none of the buffered-bytes cap that commit c8d42d8 added for CRuby in 1.8.6. A single 5-byte message decides how much memory a JVM process tries to allocate:\n\n* `array32` header declaring `0x7FFFFFFF` elements → `new IRubyObject[0x7FFFFFFF]` (~8 GiB of reference slots) attempted immediately → `java.lang.OutOfMemoryError: Requested array size exceeds VM limit`.\n* Counts *below* the VM's max-array-size but above available heap (e.g. 5×10⁸ with a 512 MiB heap) produce a **real allocation attempt** — `OutOfMemoryError: Java heap space` — the JVM genuinely tries to take the memory before failing.\n* `str32`/`bin32` headers declaring `0x7FFFFFFF` bytes do the same via `readBytes(size)` → `new byte[size]`.\n* Below the failure threshold there is a **silent band**: one streaming unpacker fed only an `array32` header with count 2×10⁷ (5 wire bytes, zero elements) grows used heap **+78 MiB immediately**, no exception (`each` swallows the underflow), and the footprint stays elevated after `Unpacker#reset` + full GC. Linear in declared count (~4 B/element); nothing bounds it — not delivered bytes, not a per-container cap.\n\nOnly counts ≥ 2³¹ are refused (a clean `MessagePack::UnderflowError` from `getUnsignedInt()`), and only **maps** are safe (`consumeHash` grows per element instead of pre-sizing). Verified on the released gem `msgpack 1.8.6 java`, JRuby 9.4.12 / OpenJDK 25, 2026-10-30 — i.e. **unfixed in the current release**. Proposed fix mirrors c8d42d8: cap initial allocation by bytes actually buffered and grow as data arrives; read strings in bounded chunks instead of allocating the declared length up front.\n\n\n### Details\n\n### 1. The four dispatch arms, and which pre-allocate (line numbers at tag v1.8.6 — current release; `ext/java` has no commits after it)\n\n| header | dispatch ([Decoder.java](https://github.com/msgpack/msgpack-ruby/blob/v1.8.6/ext/java/org/msgpack/jruby/Decoder.java)) | behaviour on the declared count | measured (released 1.8.6-java, Xmx512m) |\n|---|---|---|---|\n| `array32` `\\xdd` | [:296](https://github.com/msgpack/msgpack-ruby/blob/v1.8.6/ext/java/org/msgpack/jruby/Decoder.java#L296) → [`consumeArray(int size)` :139-145](https://github.com/msgpack/msgpack-ruby/blob/v1.8.6/ext/java/org/msgpack/jruby/Decoder.java#L139-L145) | `new IRubyObject[size]` **immediately**, before reading any element | count `0x7FFFFFFF`: OOME \"Requested array size exceeds VM limit\"; count 5×10⁸: OOME \"**Java heap space**\" (real allocation attempt); count 2×10⁷: **+78 MiB, silent** |\n| `str32` `\\xdb` | [:294](https://github.com/msgpack/msgpack-ruby/blob/v1.8.6/ext/java/org/msgpack/jruby/Decoder.java#L294) → [`consumeString` :129](https://github.com/msgpack/msgpack-ruby/blob/v1.8.6/ext/java/org/msgpack/jruby/Decoder.java#L129) → [`readBytes(int size)` :186-190](https://github.com/msgpack/msgpack-ruby/blob/v1.8.6/ext/java/org/msgpack/jruby/Decoder.java#L186-L190) | `new byte[size]` **immediately** | len `0x7FFFFFFF`: OOME \"array size exceeds VM limit\" |\n| `bin32` `\\xc6` | [:269](https://github.com/msgpack/msgpack-ruby/blob/v1.8.6/ext/java/org/msgpack/jruby/Decoder.java#L269) → same `consumeString`/`readBytes` | same | len `0x7FFFFFFF`: OOME \"array size exceeds VM limit\" |\n| `map32` `\\xdf` | [:298](https://github.com/msgpack/msgpack-ruby/blob/v1.8.6/ext/java/org/msgpack/jruby/Decoder.java#L298) → [`consumeHash(int size)` :147+](https://github.com/msgpack/msgpack-ruby/blob/v1.8.6/ext/java/org/msgpack/jruby/Decoder.java#L147) | **no pre-allocation** — inserts per element, underflows on first missing byte | count `0x7FFFFFFF`: no exception (swallowed), no large allocation |\n\nThe only guard is [`getUnsignedInt()` :88-94](https://github.com/msgpack/msgpack-ruby/blob/v1.8.6/ext/java/org/msgpack/jruby/Decoder.java#L88-L94): counts with the top bit set (≥ 2³¹) raise `MessagePack::UnderflowError: Size too large (limited to 2**31 for the Java version)`. That bounds the *value* but leaves the entire positive band `(available heap, 2³¹)` unbounded — and that band is where real allocation attempts happen.\n\n### 2. Contrast with the C implementation (same repo)\n\nThe CRuby unpacker has had a per-container cap since 1.7.3 (`INITIAL_BUFFER_CAPACITY_MAX = SHRT_MAX`) and, since [c8d42d8](https://github.com/msgpack/msgpack-ruby/commit/c8d42d8) in **1.8.6**, additionally caps initial size by bytes actually buffered (\"an element needs at least one byte…\"). Strings on C are bounded too (`read_raw_body_cont` allocates `min(remaining, buffered)`). The Java decoder has **neither generation of cap** — it is strictly behind the CRuby behaviour this repo already considers correct enough to ship.\n\n### 3. Silent retention + mechanism control (measured 2026-10-30, released gem)\n\nOne `MessagePack::Unpacker` per process, fed only a header (`\"\\xdd\" + [count].pack('N')`), zero elements; heap = `Runtime.total_memory - free_memory` after `GC.start` on both sides:\n\n| declared count | used heap before → after | delta | exception? |\n|---|---|---|---|\n| 1,000,000 | 26.4 → 31.1 MiB | +4.7 | none (`each` swallows the underflow) |\n| 5,000,000 | 53.3 → 74.0 MiB | +20.7 | none |\n| 20,000,000 | 51.3 → 129.3 MiB | **+78.0** | none |\n\nMechanism control: after the feed, `Unpacker#reset` + three `GC.start` cycles left used heap at **+79.4 MiB above baseline** (2×10⁷ arm) — the elevation is *not* unpacker-held state; it is JVM heap already spent on header-declared allocation and not returned. Either way the process footprint grows per attack and stays grown; the peak cost is paid synchronously during parsing regardless of GC behaviour afterwards. (Honest scope note: this is unbounded **allocation**, not a memory leak.)\n\n### 4. Why JVM makes availability impact worse than the CRuby analogue\n\nIn CRuby, allocation failure raises `NoMemoryError`/fails inside one worker process. In a JVM, an OOME raised on the parser thread can land anywhere in the host application's shared runtime — long-lived JRuby servers (Rails, Logstash-class pipelines, JDBC apps) share heap across all threads and connections; a single crafted message can destabilize or kill the whole JVM, not just the request. The silent band means no exception at all is even required to spend memory.\n\n### 5. Not claimed / scope\n\n* **Fluentd is NOT affected** (and this report deliberately does not involve it): Fluentd ships CRuby-only — CI pins `engine: cruby`, gemspec requires ruby >= 3.3 while JRuby 9.4 implements Ruby 3.1 semantics, and there are zero JRUBY guards in lib/. This is an upstream msgpack-ruby report only.\n* Maps (`map32`) do not pre-allocate — measured; the class applies to `array32`/`str32`/`bin32`.\n* Counts ≥ 2³¹ are cleanly refused — measured; no claim there.\n* Older java-gem releases were not individually probed; `<= 1.8.6` means \"every release up to the current one carries this code\".\n\n\n### Proof of concept\n\nVerified on the **released gem** `msgpack 1.8.6 java`, JRuby 9.4.12.0, OpenJDK 25.0.4.1, Linux x86-64 (artifacts in `evidence_186_recheck/`). Always run with a small `-Xmx` so oversized requests fail fast instead of wrecking the box.\n\n### Setup\n\n```console\n$ curl -fsSLO https://repo1.maven.org/maven2/org/jruby/jruby-dist/9.4.12.0/jruby-dist-9.4.12.0-bin.tar.gz\n$ tar xzf jruby-dist-9.4.12.0-bin.tar.gz && export PATH=$PWD/jruby-9.4.12.0/bin:$PATH\n$ jruby -S gem install msgpack --no-document # resolves the java-platform gem: \"msgpack (1.8.6 java)\"\n```\n\n### 1. Header shapes (fresh process per arm, `-Xmx512m`)\n\n```console\narray32 count=0xFFFFFFFF -> MessagePack::UnderflowError: Size too large (limited to 2**31 for the Java version)\narray32 count=0x7FFFFFFF -> Java::JavaLang::OutOfMemoryError: Requested array size exceeds VM limit\narray32 count=500000000 -> Java::JavaLang::OutOfMemoryError: Java heap space # real allocation attempt\nstr32 len=0x7FFFFFFF -> Java::JavaLang::OutOfMemoryError: Requested array size exceeds VM limit\nbin32 len=0x7FFFFFFF -> Java::JavaLang::OutOfMemoryError: Requested array size exceeds VM limit\nmap32 count=0x7FFFFFFF -> no exception, no large allocation (consumeHash grows per element)\n```\n\nEach payload is five bytes, e.g. `\"\\xdd\" + [count].pack('N')` fed to `MessagePack::Unpacker#feed_each`. Scripts: `jr_probe.rb`, `jr_probe_extra.rb` (attached in kit).\n\n### 2. Silent band — retention from one header (Xmx1g)\n\n```console\n$ for n in 1000000 5000000 20000000; do jruby -J-Xmx1g jr_one.rb $n; done\ncount= 1000000 heap 26.4 -> 31.1 MiB (delta 4.7) | no exception (underflow swallowed)\ncount= 5000000 heap 53.3 -> 74.0 MiB (delta 20.7) | no exception (underflow swallowed)\ncount= 20000000 heap 51.3 -> 129.3 MiB (delta 78.0) | no exception (underflow swallowed)\n\n$ jruby -J-Xmx1g jr_reset_control.rb 20000000\ncount=20000000 heap base=49.9 after_feed=129.3 (delta 79.4) after_reset_gc=129.3 (retained_vs_base 79.4)\n```\n\nA streaming server that keeps one unpacker per connection (the pattern any socket-fed msgpack service uses) pays this per connection with five wire bytes each and produces no exception, no log line.\n\n### 3. Fix verification path\n\nApply the C-side idea from c8d42d8: in `consumeArray`, cap the initial array by bytes actually buffered (an element needs ≥ 1 byte) and grow as elements arrive; in `readBytes`/`consumeString`, read in bounded chunks instead of allocating the declared length up front. Then re-run this PoC: the silent band should track delivered bytes, and oversized declarations should raise `UnderflowError`-class errors without touching multi-GiB allocations. msgpack-ruby CI runs jruby-9.4/truffleruby, so a patch is verifiable as-is; I can prepare one against `ext/java/org/msgpack/jruby/Decoder.java`.\n\n\n### Impact\n\n**Type.** Uncontrolled resource consumption driven by *declared* sizes rather than delivered bytes — CWE-770 (Allocation of Resources Without Limits or Throttling), precisely CWE-1325 (Independent Resource Usage Limiting Not Applied). Availability only; no confidentiality/integrity impact, no code execution.\n\n**Who is impacted.** Any JRuby application that feeds untrusted msgpack to `MessagePack.unpack`/`Unpacker` — the java-platform gem's whole user base (JRuby-based services, log/RPC pipelines, JVM polyglot stacks using this gem). For them: remote OOM of a shared-heap JVM from a 5-byte message, or silent heap spend below the exception band.\n\n**Severity.** Proposed **High / 7.5**, `CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H` — remote, trivially crafted, default settings, process death; same metric string as the accepted fluentd OOM family (e.g. GHSA-h3xv-5jpx-r4j2) and as this repo's own C-side analogue (reported separately, patched at 1.8.6). If triage weighs the platform scope (java gem only), lower framings read against that — we flag it honestly; the maintainers' call on scope shouldn't delay the patch.\n\n**Relationship to the C-side report.** Same flaw class, different implementation and fix state: CRuby's pre-allocation was capped by c8d42d8 (released 1.8.6, separate advisory with affected `<= 1.8.5`, patched `1.8.6`). This JRuby variant is **unfixed in 1.8.6** and needs its own patch + CVE. Cross-reference both ways once filed.\n\n**What this is not.** Not a Fluentd issue (Fluentd is CRuby-only — CI pins cruby, gemspec ruby >= 3.3 vs JRuby's Ruby 3.1 semantics); no fluentd advisory should carry this. Not a memory leak (measured: footprint elevation persists after reset+GC without retained objects). Not exploitable via maps or counts ≥ 2³¹ (both measured safe/cleanly refused).","affected":[{"package":{"name":"msgpack","ecosystem":"RubyGems","purl":"pkg:gem/msgpack"},"ranges":[{"type":"ECOSYSTEM","events":[{"introduced":"0"},{"fixed":"1.8.7"}]}],"versions":["0.0.1","0.2.0","0.2.1","0.2.2","0.3.0","0.3.1","0.3.2","0.3.3","0.3.4","0.3.5","0.3.6","0.3.7","0.3.8","0.3.9","0.4.0","0.4.1","0.4.2","0.4.3","0.4.4","0.4.5","0.4.6","0.4.7","0.5.0","0.5.1","0.5.10","0.5.11","0.5.12","0.5.2","0.5.3","0.5.4","0.5.5","0.5.6","0.5.7","0.5.8","0.5.9","0.6.0","0.6.0pre1","0.6.1","0.6.2","0.7.0","0.7.0dev1","0.7.1","0.7.2","0.7.2dev1","0.7.3","0.7.4","0.7.5","0.7.6","1.0.0","1.0.1","1.0.2","1.0.3","1.1.0","1.2.0","1.2.1","1.2.10","1.2.2","1.2.3","1.2.4","1.2.5","1.2.6","1.2.7","1.2.8","1.2.9","1.3.0","1.3.1","1.3.2","1.3.3","1.4.0","1.4.0.pre1","1.4.1","1.4.2","1.4.3","1.4.4","1.4.4.pre1","1.4.5","1.5.0","1.5.1","1.5.2","1.5.3","1.5.4","1.5.5","1.5.6","1.6.0","1.6.1","1.7.0","1.7.1","1.7.2","1.7.3","1.7.4","1.7.5","1.8.0","1.8.1","1.8.2","1.8.3","1.8.4","1.8.5","1.8.6"],"database_specific":{"last_known_affected_version_range":"<= 1.8.6","source":"https://github.com/github/advisory-database/blob/main/advisories/github-reviewed/2026/11/GHSA-73v3-wf97-vpgj/GHSA-73v3-wf97-vpgj.json"}}],"references":[{"type":"WEB","url":"https://github.com/msgpack/msgpack-ruby/security/advisories/GHSA-73v3-wf97-vpgj"},{"type":"WEB","url":"https://github.com/msgpack/msgpack-ruby/commit/ca0efc1256c687b428c46dd68ce000ff515d53f2"},{"type":"PACKAGE","url":"https://github.com/msgpack/msgpack-ruby"},{"type":"WEB","url":"https://github.com/msgpack/msgpack-ruby/releases/tag/v1.8.6"},{"type":"WEB","url":"https://github.com/msgpack/msgpack-ruby/releases/tag/v1.8.7"}],"database_specific":{"cwe_ids":["CWE-1325","CWE-770"],"github_reviewed":true,"github_reviewed_at":"2026-11-12T22:54:59Z","nvd_published_at":null,"severity":"HIGH"},"severity":[{"type":"CVSS_V3","score":"CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H"}]} |