Skip to content

HDDS-17739. Fix snapshot diff dependency ordering for reoccupied paths. - #12014

Draft
SaketaChalamchala wants to merge 2 commits into
apache:masterfrom
SaketaChalamchala:HDDS-17739
Draft

SaketaChalamchala wants to merge 2 commits into
apache:masterfrom
SaketaChalamchala:HDDS-17739

Conversation

@SaketaChalamchala

Copy link
Copy Markdown
Contributor

Found by TLA+ model checking and code review of the OM optimized snapshot diff paths under HDDS-15926, on commit d3b75c7.
The patch is generated with Specula (Claude Opus 5.5).

Should be merged after #12013

What changes were proposed in this pull request?

FSO snapshot diff dependency ordering (SnapDiffDependencyGraph) incorrectly applied a reoccupied-path skip when adding to-snapshot path-prefix edges. That skip broke ordering in two ways:

  1. Wrong “ordered” reports that do not replay — it dropped required edges for RENAME(target) under a reoccupied ancestor.
    Example A: Wrong order (skip too aggressive for RENAME)
    Snapshot 1 → 2:
mkdir a
create a/a
snapshot 1
mv a b
mkdir a
mv b/a a/a    # file ends at a/a in snapshot 2
snapshot 2

Report entries:

RENAME a/a -> a/a
RENAME a -> b
CREATE a

This leaves S2 with b/a instead of a/a, b`

After this patch: the graph detects a cycle (true constraint conflict). With HDDS-17738, ordered jobs should fall back to an unordered report instead of FAILED.

  1. False cycles — it kept spurious edges from to-snapshot ancestor ops to MODIFY entries (and missed ancestor-freed paths), so jobs failed even when a valid order existed.
    Example B: False cycle (skip too weak for MODIFY + nested recreate)
    Snapshot 1 → 2:
mkdir -p a/a
create a/a/a
snapshot 1
mv a b
mkdir -p a/a
overwrite b/a/a          # MODIFY at from-snapshot path a/a/a
snapshot 2

Report entries (valid replay order exists):

MODIFY a/a/a
RENAME a -> b
CREATE a
CREATE a/a

MODIFY a/a/a is on the old tree (path in snapshot 1). It must run before RENAME a -> b. It must not depend on CREATE a, / CREATE a/a on the new tree.

The old logic still added (or failed to skip) a CREATE a → MODIFY a/a/a style edge when the path was freed via an ancestor rename (a, not a/a/a itself). That closed a false cycle → ordered job FAILED, while unordered worked.

After this patch: no to-snapshot edges for MODIFY;
expected order: MODIFY a/a/a, RENAME a -> b, CREATE a, CREATE a/a

The fix aligns edge rules with the path namespace contract: MODIFY never gets to-snapshot prefix edges, and CREATE / RENAME(target) always keep those edges. The reoccupied-path skip is removed (buildReoccupiedPaths, shouldSkipReoccupiedPathToSnapshotEdge).

  1. addPathPrefixToSnapshotEdges — skip MODIFY (and DELETE) entirely; only CREATE and RENAME(target) get to-snapshot prefix edges.
  2. Remove buildReoccupiedPaths, shouldSkipReoccupiedPathToSnapshotEdge, and isStrictPathPrefix.
  3. Always add CREATE/RENAME(target) ancestor → descendant edges (no skip).

Rationale in one line: MODIFY is from-snapshot only for hierarchy; CREATE/RENAME(target) are to-snapshot only. Mixing MODIFY into to-snapshot prefix edges (with heuristics) caused both bugs.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-17739

How was this patch tested?

  • Unit: TestSnapDiffDependencyGraph
    • New: testRenameSourceReuseWithDescendantRenameIntoNewOccupierDetectsCycle, testRenameSourceReuseByNestedCreateWithDescendantModify, testDeleteAndRecreateNestedPathWithDescendantModifyAndRename
    • Updated: testDeleteAndRecreateWithOldTreeChildModifyAndRename now expects a cycle (cross-parent rename into new occupier)
    • Existing reoccupied-path MODIFY cases updated / still pass with the new edge rules

@SaketaChalamchala SaketaChalamchala added snapshot https://issues.apache.org/jira/browse/HDDS-6517 AI-gen labels Nov 11, 2026
@github-actions github-actions Bot added the om label Nov 11, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI-gen om snapshot https://issues.apache.org/jira/browse/HDDS-6517

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant