Uniq: improve performances #13249
Open
sylvestre wants to merge 2 commits into
Open
Conversation
is_c_locale() was called on every line inside key_end_index() when -w/--check-chars is set, doing up to 3 std::env::var_os() lookups each time. Locale env vars can't change mid-process, so this was pure per-line overhead, causing uniq -w to be ~5x slower than GNU uniq even for small -w values. Compute is_c_locale() once at startup and cache it on the Uniq struct instead. Fixes uutils#13199
write_line() issued two separate write_all() calls per output line (line content, then the terminator byte), each going through the dynamically-dispatched Box<dyn Write> from open_output_file(). Merge them into a single write via a reused scratch buffer. Also match the input BufReader's capacity to the existing 128KB output buffer (previously the 8KB std default), for consistency. Measured on a 20x-repeated /usr/share/dict/words (~80MB, pinned to one CPU core to reduce noise): -w 1 dropped from 429.7ms to 395.5ms (~8%), -w 512 from 578.9ms to 497.8ms (~14%).
|
GNU testsuite comparison: |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes: #13199