Implement filesystem foundation milestones#996
Conversation
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a filesystem layer on top of Elastickv, implementing a core filesystem service, a FUSE session adapter, key formatting utilities, and manual split boundary snapping with associated metrics. The review feedback highlights several critical issues: a bug in extracting gRPC status codes from wrapped errors, atomicity violations in SetAttr, Release, and lease reaping due to split transactions, a performance bottleneck in StatFS caused by scanning the entire keyspace, and an optimization opportunity in the Write path to avoid redundant reads on full chunk overwrites.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
Addressed the latest-head filesystem review blockers. Changes:
Validation run on this head:
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4f140605b5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Addressed the latest-head filesystem review findings on Changes:
Validation:
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a7528bdd5f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if replacedEntry.Type == TypeDirectory { | ||
| return nil, ErrIsDir | ||
| } |
There was a problem hiding this comment.
When the source entry is a directory and newName already names a regular file, this helper treats the target as replaceable because it only rejects directory targets. The caller then overwrites the file's dirent with the directory and unlinks/GCs the file, so rename(dir, file) succeeds instead of returning a type error; please compare the source and target types before allowing replacement.
Useful? React with 👍 / 👎.
| err := s.scanExistingChunks(ctx, meta, ts, start, func(pair *store.KVPair) error { | ||
| delta = delta.merge(usageDelta{bytesSub: uint64(len(pair.Value))}) | ||
| elems = append(elems, &kv.Elem[kv.OP]{Op: kv.Del, Key: pair.Key}) | ||
| readKeys = append(readKeys, pair.Key) |
There was a problem hiding this comment.
For truncating a file with more than 10,000 stored chunks (about 40 GiB at the default 4 MiB chunk size), this loop appends every chunk key to ReadKeys; the coordinator rejects transactions once len(ReadKeys) > 10_000, so large-file truncation can fail instead of deleting the tail. The same unbounded read set is used by file GC below, so unlink/release/rename-over-target can hit the same limit; please page these deletes/usage updates or avoid a single read set that grows with the whole file.
Useful? React with 👍 / 👎.
| if user := fskeys.ExtractRouteKey(key); user != nil { | ||
| return user | ||
| } |
There was a problem hiding this comment.
This branch makes the data path route !fs|chk|... through the virtual !fs|route|chk|... domain, but DistributionServer.GetRoute still calls engine.GetRoute(req.Key) directly. After a filesystem boundary split, a GetRoute request for a raw chunk key can return the pre-boundary route or miss because !fs|chk|... sorts before !fs|route|chk|..., while reads and writes go to the normalized route; please reuse the same normalization for the distribution RPC so control-plane lookups match data-path routing.
Useful? React with 👍 / 👎.
| } | ||
| end := prefixEnd(prefix) | ||
| for { | ||
| page, err := s.store.ScanAt(ctx, start, end, statFSScanPageSize, ts) |
There was a problem hiding this comment.
When this service is backed by kv.ShardStore, chunk writes are stored on the group selected by routeKey(fskeys.ChunkKey(...)) (the virtual !fs|route|chk|... key), but this raw !fs|chk|... range scan is routed by ShardStore.ScanAt using the raw scan bounds. In a multi-range deployment where the raw chunk prefix and the virtual chunk-route prefix resolve to different groups, truncate/GC scans return no chunks, leaving deleted file data behind and usage deltas wrong; route these chunk prefix scans through the same filesystem route domain or scan the owning group directly.
Useful? React with 👍 / 👎.
1 participant
Summary
Design doc
Validation
Author: bootjp