You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Several live Python modules fail rather than skip when the attached graph was emitted below the analysis level they need. The suite already knows how to do this correctly in places, so the inconsistency is the bug.
Measured on a graph of odoo-slim-19 emitted by codeanalyzer-python 1.5.3 at analysis_level=1 (1,626 modules, 15,549 callables) — a graph that answers every level-1 query correctly:
test_e2e_neo4j_live.py::test_get_config_keys_and_config_uses_line_up asserts get_config_uses() is non-empty whenever get_config_keys() is. The two are not tied: config keys come from artifact extraction at level 1, while config uses are resolved at analysis_level >= 2 (codeanalyzer-python's core.py, the literal tier). The graph has 93 :ConfigKey nodes and zero PY_USES_CONFIG edges. Both numbers are correct for a level-1 emit.
Six failures across test_resolve.py raise SelectorNotInGraph ('invoice_id', 'vals', 'AccessError'). resolve_value addresses formal_in ports, which are a level-4 overlay.
The assertion message compounds it: "came back empty on a graph with PY_USES_CONFIG edges" states as fact something the test never checked, and it is false on this graph. I believed that message over the graph and reported the wrong finding once already.
The correct shape is already in the same file. test_get_config_readers_finds_the_callable_that_reads_the_key queries for PY_USES_CONFIG edges first and skips cleanly when there are none.
against a level-1 graph: 8 failed, 100 passed, 17 skipped. The same run on release/2.0 before #412 gives the identical failure set, so this predates that change.
Expected behavior
A live module states the analysis level it needs and skips with that reason when the attached graph is below it, the way the config-readers test already does. A test that needs an overlay checks the overlay is present, not a neighbouring one. The skip reason names the level, so a green run with skips is readable rather than reassuring.
Additional context
Scope boundary
Test-suite only. No facade change, no query change, no new accessor.
Not about whether a level-1 graph should answer these queries. It should not, and it does not; this is about how the suite reports that.
The eighth failure in the same run is not in scope: :PyModule now carries a source property, so the module_source_unavailable divergence the live twin pins is genuinely stale. That is the read-path uptake tracked in Incorporate codeanalyzer-python v1.5.2 #396.
Caveats and known risks
A guard that is too broad silently stops testing something. python-sdk#362 is the precedent: a retired env-var spelling made a run end green having verified nothing. The skip reason must name the level and the overlay it probed, so a reader can tell "not applicable here" from "quietly stopped checking".
The probe must read the data (are there PY_USES_CONFIG edges? are there formal_in ports?), never an analyzer version string. A graph's level is not recorded on it, and :PyApplication carries no analysis_level property — verified on this graph.
Definition of done
Against a level-1 graph the four modules report skips with reasons naming the missing overlay and zero failures; against a level-4 graph they run and assert as they do today; the misleading assertion message is gone.
Describe the bug
Several live Python modules fail rather than skip when the attached graph was emitted below the analysis level they need. The suite already knows how to do this correctly in places, so the inconsistency is the bug.
Measured on a graph of
odoo-slim-19emitted by codeanalyzer-python 1.5.3 atanalysis_level=1(1,626 modules, 15,549 callables) — a graph that answers every level-1 query correctly:test_e2e_neo4j_live.py::test_get_config_keys_and_config_uses_line_upassertsget_config_uses()is non-empty wheneverget_config_keys()is. The two are not tied: config keys come from artifact extraction at level 1, while config uses are resolved atanalysis_level >= 2(codeanalyzer-python'score.py, the literal tier). The graph has 93:ConfigKeynodes and zeroPY_USES_CONFIGedges. Both numbers are correct for a level-1 emit.test_resolve.pyraiseSelectorNotInGraph('invoice_id','vals','AccessError').resolve_valueaddressesformal_inports, which are a level-4 overlay.The assertion message compounds it: "came back empty on a graph with
PY_USES_CONFIGedges" states as fact something the test never checked, and it is false on this graph. I believed that message over the graph and reported the wrong finding once already.The correct shape is already in the same file.
test_get_config_readers_finds_the_callable_that_reads_the_keyqueries forPY_USES_CONFIGedges first and skips cleanly when there are none.To Reproduce
against a level-1 graph: 8 failed, 100 passed, 17 skipped. The same run on
release/2.0before #412 gives the identical failure set, so this predates that change.Expected behavior
A live module states the analysis level it needs and skips with that reason when the attached graph is below it, the way the config-readers test already does. A test that needs an overlay checks the overlay is present, not a neighbouring one. The skip reason names the level, so a green run with skips is readable rather than reassuring.
Additional context
Scope boundary
:PyModulenow carries asourceproperty, so themodule_source_unavailabledivergence the live twin pins is genuinely stale. That is the read-path uptake tracked in Incorporate codeanalyzer-python v1.5.2 #396.Caveats and known risks
PY_USES_CONFIGedges? are thereformal_inports?), never an analyzer version string. A graph's level is not recorded on it, and:PyApplicationcarries noanalysis_levelproperty — verified on this graph.Definition of done
Against a level-1 graph the four modules report skips with reasons naming the missing overlay and zero failures; against a level-4 graph they run and assert as they do today; the misleading assertion message is gone.