Also fail on Pester block/container failures (not just failed tests)#128
Merged
tablackburn merged 2 commits intoJul 7, 2026
Merged
Conversation
In Pester 5 a run has three independent failure counts. Gating only on FailedCount misses FailedBlocksCount (BeforeAll/AfterAll) and FailedContainersCount (files that fail to discover/load), so a failing BeforeAll or an unloadable test file passes the build green. This adds those counts to the gate. Generated with AI (GitHub Copilot CLI). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
|
@nohwnd should we also make this change to ? |
Contributor
Author
|
Yes. You can also check the result property. |
Contributor
|
@nohwnd do you want to include that change in this PR or do you want me to take care of it? I'm good either way. |
tablackburn
pushed a commit
that referenced
this pull request
Jul 7, 2026
Test-PSBuildPester gated only on FailedCount, so a BeforeAll/AfterAll that threw or a test file that errored during discovery left the count at zero and the build passed despite tests never running. Gate on the run's aggregate Result property instead, which Pester derives from all failure categories (failed tests, failed blocks, failed containers). This applies the shipped-function half of the fix discussed in #128; that PR covers the repository's own psakeFile.ps1. Verified against crash fixtures: a discovery-crash file yields FailedCount=0 / Result=Failed and now throws; a BeforeAll crash still throws; a healthy suite still passes. Full suite: 301 passed, 0 failed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011semwa5HU1BUKKL4RoWjKa
3 tasks
5 tasks
tablackburn
reviewed
Jul 7, 2026
tablackburn
left a comment
tablackburn
left a comment
Contributor
There was a problem hiding this comment.
Thanks for the fix! This is correct for the repo's own build file, and we've verified the bug class is real here — while migrating this repo's test suite to Pester 6 (#132), a test file died during discovery and the build still reported success (FailedCount=0, Container failed: 1, exit 0).
Per the discussion above, the shipped-function half (Test-PSBuildPester.ps1) is now covered in #133 using the Result property you suggested, so this PR can stay scoped as-is. Left one optional inline suggestion if you'd like the two gates to match.
Generated by Claude Code
tablackburn
pushed a commit
that referenced
this pull request
Jul 7, 2026
Address review feedback on #133: the Unreleased entry linked #128, but this change lands via #133. Point the entry at #133 and note the companion relationship to #128 explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011semwa5HU1BUKKL4RoWjKa
tablackburn
approved these changes
Jul 7, 2026
tablackburn
pushed a commit
that referenced
this pull request
Jul 7, 2026
Replace the three-counter sum merged in #128 with the aggregate Result property, matching the gate in Test-PSBuildPester. Same behavior, one source of truth for what "failed" means. Suggested by nohwnd in the #128 discussion. Verified both directions locally: a discovery-crash container fails the build (exit 1), and a healthy run under Pester 5.8.0 passes (301 passed, exit 0). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_011semwa5HU1BUKKL4RoWjKa
tablackburn
added a commit
that referenced
this pull request
Jul 7, 2026
Test-PSBuildPester gated pass/fail only on FailedCount, so a BeforeAll/AfterAll that threw or a test file that errored during discovery left the count at zero and the build passed despite tests never running. Gate on the run's aggregate Result property instead, which Pester derives from all failure categories. Also switch this repo's own psakeFile.ps1 Pester gate (the three-counter sum from #128) to the same Result check so both gates share one definition of "failed". Companion to #128. Same bug lineage as #31, #52, and #57.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I've been seeing this commonly across Pester 5 build scripts and I'm fixing it where it looks like it could hide real failures.
In Pester 5 a run reports three independent failure counts:
FailedCount— failed tests (It)FailedBlocksCount— failedBeforeAll/AfterAllFailedContainersCount— files that error during discovery / fail to loadGating on only
FailedCountmeans aBeforeAllthat throws, or a test file that fails to load, leavesFailedCount = 0and the build passes green despite real failures. Pester itself derives the overall pass/fail from the sum of all three.This change includes all three counts in the gate. Display/log messages are left unchanged.