Skip to content

HDDS-16734. Gate conditional delete and multipart upload completion - #11879

Open
rich7420 wants to merge 5 commits into
apache:masterfrom
rich7420:HDDS-16734
Open

rich7420 wants to merge 5 commits into
apache:masterfrom
rich7420:HDDS-16734

Conversation

@rich7420

@rich7420 rich7420 commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Conditional delete and conditional MPU completion can silently run unconditionally against an older OM, which ignores the condition fields it does not know. RpcClient now refuses them when the OM is too old. Conditional delete needs the new pre-ZDU CONDITIONAL_DELETE version, so it stays available before ZDU finalization. Conditional MPU completion needs ATOMIC_CREATE_IF_NOT_EXISTS, which shipped with it in 2.2.0.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-16734

How was this patch tested?

On JDK 21 at af061868e0:

  • TestRpcClient (190) passed. Its conditional delete and MPU completion cases fail without the gates and pass with them.
  • TestOMVersionManager (26) passed, and so did the OM request tests for create, delete and MPU completion.
  • Checkstyle and PMD passed for ozone-client and ozone-manager.

Copilot AI balanced review requested due to automatic review settings October 31, 2026 17:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.

@smengcl
smengcl marked this pull request as ready for review October 31, 2026 19:57

@smengcl smengcl left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good

@adoroszlai
adoroszlai marked this pull request as draft November 1, 2026 09:03
@adoroszlai

Copy link
Copy Markdown
Contributor

@errose28 please take a look

@rich7420
rich7420 marked this pull request as ready for review November 1, 2026 19:23

@chungen0126 chungen0126 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @rich7420 for working on this. Could you please clarify the purpose of each test? Although they are quick unit tests, having this many variations feels redundant.

Comment on lines +706 to +707
@EnumSource(value = OzoneManagerVersion.class, names = {"DEFAULT_VERSION", "S3_LIST_MULTIPART_UPLOADS_PAGINATION",
"ATOMIC_CREATE_IF_NOT_EXISTS", "S3_DERIVED_KEY", "CONDITIONAL_DELETE", "ZDU", "UNKNOWN_VERSION"})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need to test all the versions?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trimmed to the versions just before and at the gate.

List<OMRequest> requests = new ArrayList<>();
RpcClient client = createConditionalRequestClient(requests, version);
try {
for (String etag : Arrays.asList(null, "etag", "*", "")) {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to test all four senarios? I think testing Arrays.asList(null, "etag") is enough.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done, thanks!

Comment on lines +740 to +741
@EnumSource(value = OzoneManagerVersion.class, names = {"DEFAULT_VERSION", "S3_LIST_MULTIPART_UPLOADS_PAGINATION",
"ATOMIC_CREATE_IF_NOT_EXISTS", "S3_DERIVED_KEY", "CONDITIONAL_DELETE", "ZDU", "UNKNOWN_VERSION"})

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same here. Do we really need to test all the versions?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Trimmed to the versions just before and at the gate.

@rich7420

rich7420 commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Good point, a lot of them were testing the same thing. Trimmed them down in d45b89f, so each case now covers a different check.

@rich7420

rich7420 commented Nov 9, 2026

Copy link
Copy Markdown
Contributor Author

Trimmed this down to the client-side gates. The OM-side checkFeatureEnabled calls only read ozone.om.features.disabled, which no test sets for these features, so they don't protect against an older OM anyway. HDDS-17754 removes calls like that.

# Conflicts:
#	hadoop-ozone/client/src/main/java/org/apache/hadoop/ozone/client/rpc/RpcClient.java
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants