fix(graphics): cap RLE decoder dimensions to prevent OOM from adversarial input - #1962
Open
Greg Lamberson (glamberson) wants to merge 1 commit into
Open
Conversation
…rial input decompress_helper allocated its output buffer as dst.resize(row_delta * height, 0) with no cap on width/height, both of which come directly from TS_BITMAP_DATA's wire-decoded, unvalidated fields (MS-RDPBCGR 2.2.9.1.1.3.1.2.2, both 16-bit unsigned integers). Worst case: 65535 * 65535 * 3 bytes (24 bpp) is roughly 12.6 GB from a few attacker-controlled bytes. Added a per-axis MAX_DECODE_DIM = 8192 cap, checked before the allocation, matching the existing cap already used by ironrdp-graphics's ClearCodec decoder and MS-RDPBCGR's own documented maximum desktop width for current Windows RDP server versions (section 3.3.5.3.3, note 46). New RleError::DimensionsTooLarge variant carries the offending width/height for diagnostics.
Greg Lamberson (glamberson)
deployed
to
llm-providers
September 12, 2026 22:33 — with
GitHub Actions
Active
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ironrdp_graphics::rle::decompress_helperallocated its output buffer asdst.resize(row_delta * height, 0)with no cap onwidth/height, both of which come directly fromTS_BITMAP_DATA's wire-decoded, unvalidated fields (MS-RDPBCGR 2.2.9.1.1.3.1.2.2, both 16-bit unsigned integers). Worst case: 65535 * 65535 * 3 bytes (24 bpp) is roughly 12.6 GB from a few attacker-controlled bytes.MAX_DECODE_DIM = 8192cap, checked before the allocation, matching the existing cap already used byironrdp-graphics's ClearCodec decoder and MS-RDPBCGR's own documented maximum desktop width for current Windows RDP server versions (section 3.3.5.3.3, note 46).RleError::DimensionsTooLargevariant carries the offending width/height for diagnostics.Validation
cargo xtask check fmt/lints/tests/typos/locksall pass. Added three regression tests: width over the limit, height over the limit, and a boundary test confirmingMAX_DECODE_DIMitself is still accepted (not an off-by-one).Notes
Filed as part of the audit tracked in #1315. The existing
fuzz/fuzz_targets/rle_decompression.rstarget'sBitmapInputgenerator currently caps width/height atu8(max 255), well under both the old and new limits, so it would not have found this on its own; widening that generator to the wire's actualu16range is a natural follow-up but is out of scope here since it also feeds three other oracles (rdp6_encode_bitmap_stream,rdp6_decode_bitmap_stream_to_rgb24, and one more) that have not been audited for the same class of issue.