[docs] Fix false claim that specific manifest rules override directory rules - #7446
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit intoAug 5, 2026
Conversation
…y rules `docs/edit/manifest.md` stated "More specific rules override directory rules" and illustrated it with an example that does the opposite of what the surrounding text claims. Manifest declarations accumulate. `Manifest.get_declarations` appends the declarations of every rule whose path-segment prefix matches the nodeid (`match_rule` is a pure prefix match), and `conftest.py` adds one pytest marker per collected declaration. Nothing picks a "most specific" winner. The root cause is that no entry ever declares a test enabled: a version entry like `v2.0.0` is turned by `process_inline` into a condition with `excluded_component_version: >=2.0.0` carrying a `missing_feature` declaration, so a *satisfied* version entry emits nothing at all. Only disabling declarations exist, which is why a narrower entry has nothing to override a broader one with. Verified with the doc's own example: at python 3.0.0, the nodeid tests/appsec/iast/sink/test_specific.py::Test_X::test_y still resolves to missing_feature inherited from the directory rule, despite satisfying the more specific v2.0.0 entry. Replace the misleading example with the same YAML annotated with what actually happens, and document the ways to genuinely enable a test that a broader entry disables: narrowing that entry with `component_version`, `excluded_weblog`/`weblog` or `weblog_declaration`, or restructuring it. Also document the corollary that weblog lists need not be mirrored onto narrower entries when a broader `weblog_declaration` already gates unlisted weblogs via `'*'`. Every YAML block in the new section was executed against a throwaway manifest directory to confirm it behaves as documented. Co-Authored-By: Claude <noreply@anthropic.com>
Contributor
|
|
darccio
marked this pull request as ready for review
August 4, 2026 14:56
nccatoni
approved these changes
Aug 5, 2026
nccatoni
left a comment
Collaborator
There was a problem hiding this comment.
Great catch! Thank for the correction
gh-worker-dd-mergequeue-cf854d
Bot
deleted the
dario.castane/festive-herschel-a4bf79
branch
August 5, 2026 14:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
docs/edit/manifest.mdcontained a factually incorrect statement about manifest ruleresolution, in the section people read precisely when they are trying to activate a test:
Manifest declarations accumulate — they never override.
Manifest.get_declarations(
utils/manifest/_internal/core.py) appends the declarations of every rule whosepath-segment prefix matches the nodeid (
match_ruleinrule.pyis a pure prefix match),and
conftest.pycallsadd_pytest_markeronce per collected declaration. Nothing anywherepicks a "most specific" winner.
Worse, the example given to illustrate the claim demonstrates the opposite. Verified with
that exact snippet: at python
3.0.0, the nodeidtests/appsec/iast/sink/test_specific.py::Test_X::test_ystill resolves tomissing_feature, inherited from the directory rule, even though it satisfies the morespecific
v2.0.0entry. Following the doc as written leaves you with a test you believe isenabled and that is silently still disabled.
Changes
Docs only — no framework or test behaviour is touched.
Split the old section into
### Directory-Level Rules(prefix matching only) and a new### How Entries Combine, which:accumulate, and that a narrower entry can only add restrictions.
derivable: no entry ever declares a test enabled.
process_inline(parser.py) turnsv2.0.0into a condition withexcluded_component_version: >=2.0.0carrying amissing_featuredeclaration, so a satisfied version entry emits nothing at all. Onlydisabling declarations exist — hence there is nothing for a narrower entry to override a
broader one with.
that entry by version (
component_version) or by weblog (excluded_weblog/weblog) —plus a note to restructure the broader entry when neither fits.
when a broader
weblog_declarationalready gates unlisted weblogs through'*', a bareversion entry below it is safe.
Notes for the reviewer
Every YAML block in the new section was executed against a throwaway manifest directory
via
Manifest(components, weblog, path=...)before being written down — 13 cases in total,including the negative ones (unlisted weblogs such as
fastapi/express4staying gated by'*', and versions below the narrower entry still being gated by it). All matched thedocumented behaviour.
I also grepped
docs/for other statements about override/precedence in manifestresolution; the claim was confined to this one section, and
docs/edit/enable-test.mddoesnot repeat it.
./format.shpasses. Two failures show up on a first local run and are pre-existing andenvironmental, not from this change (both already known on this machine):
yamlfmtreformattedmanifests/python.yml(85 lines of*django:→*django :).Cause:
format.shpins yamlfmt 0.16.0 but only installs it whenwhich yamlfmtfindsnothing, so a newer global v0.20.0 shadows it. Reverted; confirmed the pinned 0.16.0
leaves
manifests/untouched and lint-clean, so nothing of it is in this PR.utils/scripts/shellcheck.shdies withline 74: @: unbound variableunder macOS bash3.2 (empty
"${@}"withset -u). Passes under bash 5, as in CI.Workflow
🚀 Once your PR is reviewed and the CI green, you can merge it!
🛟 #apm-shared-testing 🛟
Reviewer checklist
tests/ormanifests/is modified ? I have the approval from R&P team — applies here: this changesdocs/, so R&P approval is required.build-XXX-imagelabel is present🤖 Generated with Claude Code