HDDS-14661. Handle split table writes for Multipart Upload by spacemonkd · Pull Request #10588 · apache/ozone

spacemonkd · 2026-06-23T06:21:28Z

What changes were proposed in this pull request?

HDDS-14661. Handle split table writes for Multipart Upload

Please describe your PR in detail:

Implements the runtime support for schemaVersion 1 multipart uploads using the split multipartPartsTable.

This patch covers the MPU lifecycle after an upload is already marked as schemaVersion 1:

CommitPart writes part metadata into multipartPartsTable instead of growing MultipartInfoTable.
Complete MPU reads parts from multipartPartsTable, assembles the final key, and deletes all consumed part rows.
Abort MPU scans split-table parts, releases quota, moves part blocks to the deleted table, and removes part rows.
Expired MPU cleanup follows the same split-table cleanup path for abandoned uploads.
ListParts reads ordered part metadata from multipartPartsTable.
Legacy schemaVersion 0 behavior remains unchanged.

This patch intentionally does not decide when new uploads become schemaVersion 1.
PR #10062 owns the upgrade/finalization and initiate-MPU schema selection. Once that branch starts creating schemaVersion 1 MPUs, this patch provides the runtime read/write/cleanup behavior needed to handle them end to end.

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-14661

How was this patch tested?

Patch was tested using unit tests

spacemonkd · 2026-06-25T19:04:22Z

@errose28 @ivandika3 @kerneltime could you please take a look at this PR?
I plan on merging this PR first to put the wiring in place and then simultaneously merge the upgrade PR as they are co-dependent.

Having both of these ready to merge so we can do it in quick succession would be great!

ivandika3

Thanks @spacemonkd for the patch. Overall looks good. Left some comments.

Let's get this in asap, this feature has been outstanding quite a while.

ivandika3 · 2026-06-26T13:49:21Z

+/**
+ * Cache-aware scanner for multipart parts table rows.
+ */
+public final class MultipartPartScanUtil {


Nit: I'd suggest to move this to OMMultipartUploadUtils so we only need a single util class for MPU.

ivandika3 · 2026-06-26T13:54:23Z

+    try (TableIterator<OmMultipartPartKey,
+        ? extends Table.KeyValue<OmMultipartPartKey, OmMultipartPartInfo>>
+        iterator = omMetadataManager.getMultipartPartsTable().iterator(prefix)) {
+      while (iterator.hasNext()) {
+        Table.KeyValue<OmMultipartPartKey, OmMultipartPartInfo> kv =
+            iterator.next();
+        if (kv == null) {
+          continue;
+        }
+        OmMultipartPartKey key = kv.getKey();
+        if (!uploadId.equals(key.getUploadId())) {
+          break;
+        }
+        if (key.hasPartNumber()) {
+          parts.put(key.getPartNumber(), kv.getValue());
+        }
+      }
+    }
+
+    Iterator<Map.Entry<CacheKey<OmMultipartPartKey>,
+        CacheValue<OmMultipartPartInfo>>> cacheIterator =
+        omMetadataManager.getMultipartPartsTable().cacheIterator();
+    while (cacheIterator.hasNext()) {
+      Map.Entry<CacheKey<OmMultipartPartKey>, CacheValue<OmMultipartPartInfo>>
+          cacheEntry = cacheIterator.next();
+      OmMultipartPartKey key = cacheEntry.getKey().getCacheKey();
+      if (!uploadId.equals(key.getUploadId()) || !key.hasPartNumber()) {
+        continue;
+      }
+      OmMultipartPartInfo value = cacheEntry.getValue().getCacheValue();
+      if (value == null) {
+        parts.remove(key.getPartNumber());
+      } else {
+        parts.put(key.getPartNumber(), value);
+      }
+    }
+
+    return parts;


Usually, in OmMetadataReader listKeys and getMultipartUploadKeys, we will iterate the cache first before the DB. Any reason here is reversed? Not saying it's wrong, just want to standardize.

ivandika3 · 2026-06-26T14:00:15Z

+      if (multipartKeyInfo.getSchemaVersion()
+          == OmMultipartKeyInfo.LEGACY_SCHEMA_VERSION
+          && null != oldPartKeyInfo) {


Nit: A lot of unnecessary line wraps here which AI agent usually does. We can try to fix unnecessary wraps introduced in this patch as much as possible (within reason).

ivandika3 · 2026-06-26T14:03:45Z

+  private OmMultipartPartKey getMultipartPartKey(String uploadId,
+      int partNumber) {
+    return OmMultipartPartKey.of(uploadId, partNumber);
+  }


Nit: Let's make this inline instead of introducing another method.

ivandika3 · 2026-06-26T14:08:00Z

+  private void validateSplitPartInfo(OmKeyInfo omKeyInfo, int partNumber)
+      throws OMException {
+    if (StringUtils.isBlank(omKeyInfo.getMetadata().get(OzoneConsts.ETAG))) {
+      throw new OMException("Missing ETag for multipart upload part "
+          + partNumber, OMException.ResultCodes.INVALID_REQUEST);
+    }
+    if (omKeyInfo.getKeyLocationVersions() == null
+        || omKeyInfo.getKeyLocationVersions().isEmpty()
+        || omKeyInfo.getLatestVersionLocations().getLocationList().isEmpty()) {
+      throw new OMException("Missing block locations for multipart upload part "
+          + partNumber, OMException.ResultCodes.INVALID_REQUEST);
+    }
+  }


Do we have this logic in the schema version 0? OM might allow uploading an empty part (although S3G might fail it early).

ivandika3 · 2026-06-26T14:10:29Z

+        if (eTagBasedValidationAvailable) {
+          requestPartId = part.getETag();
+          omPartId = multipartPartInfo.getETag();
+        } else {
+          requestPartId = part.getPartName();
+          omPartId = multipartPartInfo.getPartName();
+        }
+        requestPart = new MultipartCommitRequestPart(
+            requestPartId, omPartId, StringUtils.equals(requestPartId, omPartId));


Are we going to call the ETag validation here?

ivandika3 · 2026-06-26T14:12:26Z

+        for (OmKeyInfo currentKeyPartInfo :
+            abortInfo.getPartsKeyInfoToDelete()) {
+          addPartToDeletedTable(omMetadataManager, batchOperation,
+              omBucketInfo, abortInfo, currentKeyPartInfo,
+              omMultipartKeyInfo.getUpdateID());
+        }

-        // multi-part key format is volumeName/bucketName/keyName/uploadId
-        String deleteKey = omMetadataManager.getOzoneDeletePathKey(
-            currentKeyPartInfo.getObjectID(), abortInfo.getMultipartKey());
-
-        omMetadataManager.getDeletedTable().putWithBatch(batchOperation,
-            deleteKey, repeatedOmKeyInfo);
+        for (OmMultipartPartKey partKey :
+            abortInfo.getPartsTableKeysToDelete()) {
+          omMetadataManager.getMultipartPartsTable().deleteWithBatch(
+              batchOperation, partKey);
+        }


Just a thought, but if it's possible we can use deleteRange in the future to reduce the tombstones. However, deleteRange need to be done carefully since it can invalidate valid DB entries.

ivandika3 · 2026-06-26T14:17:42Z

+    this.multipartPartKey = multipartPartKey;
    this.openKey = openKey;
    this.omMultipartKeyInfo = omMultipartKeyInfo;
+    this.omMultipartPartInfo = omMultipartPartInfo;


We might want to add a precondition here that if mutlipartPartKey and omMultipartPartInfo need to either be both null or both not null.

HDDS-14661. Handle split table writes for Multipart Upload

3431784

spacemonkd self-assigned this Jun 23, 2026

spacemonkd added the s3 S3 Gateway label Jun 23, 2026

spacemonkd requested a review from ivandika3 June 23, 2026 06:21

spacemonkd assigned errose28 and rakeshadr and unassigned rakeshadr and errose28 Jun 23, 2026

spacemonkd requested review from amaliujia, errose28, kerneltime and rakeshadr June 23, 2026 06:21

spacemonkd mentioned this pull request Jun 23, 2026

HDDS-14665. Add upgrade handling to multipart requests #10062

Open

spacemonkd marked this pull request as ready for review June 25, 2026 19:03

ivandika3 reviewed Jun 26, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

HDDS-14661. Handle split table writes for Multipart Upload#10588

HDDS-14661. Handle split table writes for Multipart Upload#10588
spacemonkd wants to merge 1 commit into
apache:masterfrom
spacemonkd:HDDS-14661

spacemonkd commented Jun 23, 2026

Uh oh!

spacemonkd commented Jun 25, 2026

Uh oh!

ivandika3 left a comment

Uh oh!

ivandika3 Jun 26, 2026

Uh oh!

ivandika3 Jun 26, 2026

Uh oh!

ivandika3 Jun 26, 2026

Uh oh!

ivandika3 Jun 26, 2026

Uh oh!

ivandika3 Jun 26, 2026

Uh oh!

ivandika3 Jun 26, 2026

Uh oh!

ivandika3 Jun 26, 2026

Uh oh!

ivandika3 Jun 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Uh oh!

Conversation

spacemonkd commented Jun 23, 2026

What changes were proposed in this pull request?

What is the link to the Apache JIRA

How was this patch tested?

Uh oh!

spacemonkd commented Jun 25, 2026

Uh oh!

ivandika3 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants