Skip to content

fix(graphics): cap RLE decoder dimensions to prevent OOM from adversarial input - #1962

Open
Greg Lamberson (glamberson) wants to merge 1 commit into
Devolutions:masterfrom
lamco-admin:fix/rle-decompress-dimension-cap
Open

fix(graphics): cap RLE decoder dimensions to prevent OOM from adversarial input#1962
Greg Lamberson (glamberson) wants to merge 1 commit into
Devolutions:masterfrom
lamco-admin:fix/rle-decompress-dimension-cap

Conversation

@glamberson

Copy link
Copy Markdown
Contributor

Summary

  • ironrdp_graphics::rle::decompress_helper allocated its output buffer as dst.resize(row_delta * height, 0) with no cap on width/height, both of which come directly from TS_BITMAP_DATA's wire-decoded, unvalidated fields (MS-RDPBCGR 2.2.9.1.1.3.1.2.2, both 16-bit unsigned integers). Worst case: 65535 * 65535 * 3 bytes (24 bpp) is roughly 12.6 GB from a few attacker-controlled bytes.
  • Added a per-axis MAX_DECODE_DIM = 8192 cap, checked before the allocation, matching the existing cap already used by ironrdp-graphics's ClearCodec decoder and MS-RDPBCGR's own documented maximum desktop width for current Windows RDP server versions (section 3.3.5.3.3, note 46).
  • New RleError::DimensionsTooLarge variant carries the offending width/height for diagnostics.

Validation

cargo xtask check fmt/lints/tests/typos/locks all pass. Added three regression tests: width over the limit, height over the limit, and a boundary test confirming MAX_DECODE_DIM itself is still accepted (not an off-by-one).

Notes

Filed as part of the audit tracked in #1315. The existing fuzz/fuzz_targets/rle_decompression.rs target's BitmapInput generator currently caps width/height at u8 (max 255), well under both the old and new limits, so it would not have found this on its own; widening that generator to the wire's actual u16 range is a natural follow-up but is out of scope here since it also feeds three other oracles (rdp6_encode_bitmap_stream, rdp6_decode_bitmap_stream_to_rgb24, and one more) that have not been audited for the same class of issue.

…rial input

decompress_helper allocated its output buffer as
dst.resize(row_delta * height, 0) with no cap on width/height, both
of which come directly from TS_BITMAP_DATA's wire-decoded, unvalidated
fields (MS-RDPBCGR 2.2.9.1.1.3.1.2.2, both 16-bit unsigned integers).
Worst case: 65535 * 65535 * 3 bytes (24 bpp) is roughly 12.6 GB from a
few attacker-controlled bytes.

Added a per-axis MAX_DECODE_DIM = 8192 cap, checked before the
allocation, matching the existing cap already used by
ironrdp-graphics's ClearCodec decoder and MS-RDPBCGR's own documented
maximum desktop width for current Windows RDP server versions
(section 3.3.5.3.3, note 46). New RleError::DimensionsTooLarge variant
carries the offending width/height for diagnostics.
@github-actions github-actions Bot added maintainer-required Maintainer review or intervention is required risk/unknown Risk could not be determined automatically; needs maintainer-level scrutiny scope/core Touches the core architectural tier size/S Size: up to 199 counted lines and 5 files; exceeds XS in either measure labels Sep 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

maintainer-required Maintainer review or intervention is required risk/unknown Risk could not be determined automatically; needs maintainer-level scrutiny scope/core Touches the core architectural tier size/S Size: up to 199 counted lines and 5 files; exceeds XS in either measure

Development

Successfully merging this pull request may close these issues.

1 participant