Skip to content

Read .overcommit.yml as UTF-8 in the hook template - #910

Merged
sds merged 2 commits into
sds:mainfrom
ChandraChaan:fix/hook-encoding-pr
Nov 4, 2026
Merged

sds merged 2 commits into
sds:mainfrom
ChandraChaan:fix/hook-encoding-pr

Conversation

@ChandraChaan

@ChandraChaan ChandraChaan commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #868.

Every hook in template-dir/hooks starts by reading .overcommit.yml to look for a gemfile: setting:

File.read('.overcommit.yml') =~ /gemfile: (?:false|['"]?(.*)['"]?)/

File.read without an encoding uses Encoding.default_external. Under a non-UTF-8 locale such as LANG=C that is US-ASCII, so if .overcommit.yml contains any non-ASCII character (even in a comment) the match raises ArgumentError: invalid byte sequence in US-ASCII before Overcommit loads, and every git hook fails.

This passes encoding: 'UTF-8' in all 10 hook copies (the template directory spec requires them to be byte-identical).

The regression spec in spec/integration/template_dir_spec.rb initializes a temporary repository using the hook templates and writes a .overcommit.yml containing a non-ASCII comment. It runs the installed pre-commit hook with LC_ALL=C LANG=C and -E US-ASCII, then checks for a successful exit and output from the configured AuthorName check. This verifies that the hook actually runs instead of passing on an unrelated startup failure or an early exit. The encoding is set explicitly because the CI runners use a UTF-8 locale by default.

Initial validation of the original spec on the fork:

  • Spec only, without the fix: Tests fails on every Ruby from 2.6 to 4.0 with a single failure out of 1667 examples, template_dir_spec.rb:31, expected "…/template-dir/hooks/pre-commit:30:in '<main>': invalid byte s…" not to include "invalid byte sequence".
  • Spec with the fix: Tests passes on all eight Ruby versions (1667 examples, 0 failures), and Lint passes.

The original fix commit is the squashed form of the fix branch; a separate follow-up commit strengthens the regression test. I left CHANGELOG.md alone since it looks like it is written when a version is cut.

Follow-up validation: Tests passes on all eight Ruby versions (2.6–4.0), and Lint passes. The strengthened spec also passes locally on Ruby 2.6 and fails both without the encoding fix and with hook execution disabled. The full local suite has four macOS timestamp precision failures, all reproduced on the original PR commit.

ChandraChaan and others added 2 commits November 4, 2026 03:31
The hook template reads .overcommit.yml with File.read and no encoding,
so the string takes Encoding.default_external. Under a non-UTF-8 locale
(for example LANG=C, as used by some GUI git clients) that is US-ASCII,
and matching the gemfile regex against a file containing any non-ASCII
character, even in a comment, raises "invalid byte sequence in
US-ASCII" and every git hook fails.

Pass encoding: 'UTF-8' in all hook copies, which the template directory
spec requires to be identical, and add a spec that runs a hook with
LC_ALL=C against a non-ASCII .overcommit.yml.
Run the installed pre-commit hook in an initialized temporary repository with an explicit US-ASCII external encoding. Require a successful exit and output from the configured AuthorName check so unrelated startup failures and disabled hooks cannot satisfy the regression test.
@sds sds added the bug label Nov 4, 2026
@sds
sds merged commit 51a541d into sds:main Nov 4, 2026
19 checks passed
@sds

sds commented Nov 4, 2026

Copy link
Copy Markdown
Owner

Thanks!

sds added a commit that referenced this pull request Nov 5, 2026
Windows CI has been disabled since #836. Restore the full test matrix on
Windows for every supported Ruby (2.6–4.0), alongside Linux, and remove
the Windows and AppVeyor exclusions from the specs. Give the slower
Windows jobs 30 minutes to complete the integration suite while
retaining the 15-minute Linux limit. Both platforms now run the same
1,679 examples; no examples are removed or newly skipped.

The restored matrix exposed two production failures:

- Reading merge state with `File.open(...).read` leaves handles open
until garbage collection, preventing Windows from deleting `MERGE_MODE`.
Use `File.read` so the handles close immediately. Also close readers and
temporary files in the Gerrit specs, and retain commit-message fixtures
until each example finishes, to prevent Windows finalizer errors.
- Parent-command detection depends on WMIC, which is unavailable on
current Windows runners. Query `Win32_Process` through PowerShell's
`Get-CimInstance`, preserving the `nil` fallback when the command cannot
launch or the parent has exited.

Use Ruby's native symlink APIs on Windows instead of invoking `mklink`
and parsing `dir` output. This supports directory links, raises on
creation failures, and lets the restored specs exercise real symlinks
with `core.symlinks=true`. Ten additional examples cover files,
directories, missing targets, spaces, and creation errors.

Repair the parallelism regression spec: its previous built-in hook
ignored the configured sleep command. An ad hoc serial hook now actually
sleeps and writes a marker; the spec requires a successful commit and
verifies the marker. Allow 30 seconds for Git and interpreter startup on
slower Windows runners instead of the flaky five-second deadline. Two
additional examples cover unavailable Windows process information.

Validation:

- Local Ruby 3.4.7: full suite with 1,679 examples and zero failures;
the strengthened parallelism example and all 52 commit-message context
examples also pass independently.
- [Tests](https://github.com/sds/overcommit/actions/runs/41940143707):
all 16 Linux/Windows jobs pass, each with 1,679 examples, zero failures,
and zero pending examples. All Windows logs are free of tempfile
finalizer errors.
- [Lint](https://github.com/sds/overcommit/actions/runs/41940143568),
coverage finalization, and all 16 Coveralls statuses pass. All 34 PR
checks are green on `03e86dc5`.

Refs #836 and #910.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Invoke Bundler Context: File.read() doesn't use the right encoding

2 participants