Skip to content

[codex] Improve mesh segmentation post-processing#780

Merged
fernandotonon merged 4 commits into
masterfrom
codex/improve-mesh-segmentation
Jun 30, 2026
Merged

[codex] Improve mesh segmentation post-processing#780
fernandotonon merged 4 commits into
masterfrom
codex/improve-mesh-segmentation

Conversation

@fernandotonon

@fernandotonon fernandotonon commented Jun 30, 2026

Copy link
Copy Markdown
Owner

Summary

This PR improves mesh part segmentation quality without changing or retraining the model weights.

It updates the inference/post-processing harness to:

  • sample vertices across the whole mesh instead of biasing toward the first N vertices
  • correctly scatter sampled model predictions back to all mesh vertices
  • smooth predicted labels across mesh topology to reduce noisy fragmented selections
  • preserve rig/bone-derived hints when a skeleton exists, while still improving unrigged meshes through better sampling and topology smoothing
  • pass available rig labels into GUI, CLI, and MCP segmentation entry points

Why

The previous segmentation path often produced fragmented or incomplete regions. On unrigged meshes, a sampling/scatter issue could leave large parts of the mesh with weak/default labels. The topology smoothing pass makes selections more coherent, which improves whole-part selection even when no skeleton is available.

This does not solve every semantic mistake. For example, adjacent regions such as head/arm or belly/leg can still merge when model confidence is weak. That should be addressed in a follow-up with stronger boundary/geodesic constraints or retrained weights.

Validation

  • Built QtMeshEditor locally in the x86_64 macOS build tree.
  • Manually tested segmentation in the app on an unrigged mesh; selections are more coherent than before, though still not perfect.

Summary by CodeRabbit

  • New Features

    • Segmentation now uses rig-derived prior labels to improve predictions and training-data exports.
    • Added topology-based smoothing for more consistent segment labels.
  • Bug Fixes

    • Improved handling for smaller meshes by ensuring every vertex can be sampled directly.
    • Better preserves rig-related label hints during segmentation processing.
  • Chores

    • Training-data export now includes more complete resolved mesh information.

@coderabbitai

coderabbitai Bot commented Jun 30, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@fernandotonon, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 45 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 102e6888-14b6-4742-bbba-e5a585583546

📥 Commits

Reviewing files that changed from the base of the PR and between 60dd4cd and 5fba81e.

📒 Files selected for processing (1)
  • src/MeshSegmenter.cpp
📝 Walkthrough

Walkthrough

AutoRig::rigPriorPartLabels results are now computed and forwarded into MeshSegmenter::predict from CLIPipeline::cmdSegment, EditModeController::selectByPart, and MCPServer::toolSegmentMesh. MeshSegmenter gains applyBoneHints and smoothLabelsByTopology helpers, fixes deterministic ONNX sampling for small meshes, and changes post-processing order to: apply bone hints → smooth → re-apply bone hints.

Changes

Rig-prior labels wired into segmentation

Layer / File(s) Summary
MeshSegmenter helpers and ONNX post-processing
src/MeshSegmenter.cpp
Adds applyBoneHints() and smoothLabelsByTopology() internal helpers. Fixes deterministic ONNX point sampling for vertexCount <= N to map index i directly to vertex i. Fixes scatter-back to assign vertexLabels[i] = pointLabel[i] directly. Changes post-processing order to: apply bone hints → topology smooth (1 iter) → re-apply bone hints.
rigPriorPartLabels computed and passed at all call sites
src/CLIPipeline.cpp, src/EditModeController.cpp, src/MCPServer.cpp
Each call site computes rigLabels via AutoRig::rigPriorPartLabels and passes rigLabels.data() (or nullptr) to MeshSegmenter::predict. CLIPipeline additionally validates rigResolved density and uses rigLabels/rigResolved for training-data JSON serialization. EditModeController moves rigLabels out of the inner block and captures it in the worker-thread lambda.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • fernandotonon/QtMeshEditor#771: Directly related — establishes the MeshSegmenter integration and cmdSegment/selectByPart/toolSegmentMesh call sites that this PR extends with rig-prior label passing.

Poem

🐇 A rabbit hops through vertices with glee,
bone hints applied, then smoothed so carefully,
rig labels passed where nullptr once stood,
topology votes as a good neighbor should,
then bone hints reign again — just as they should! 🎉

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 44.44% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title is concise and accurately summarizes the main change: improving mesh segmentation post-processing.
Description check ✅ Passed The description covers the summary, motivation, and validation, and includes the key technical points even though it does not use every template section.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/improve-mesh-segmentation

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@fernandotonon fernandotonon marked this pull request as ready for review June 30, 2026 03:35

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e3b13a5474

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/MeshSegmenter.cpp Outdated
Comment on lines +107 to +108
if (bp >= 0 && bp < count)
labels[v] = bp;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Do not treat unresolved rig labels as hints

In meshes with a skeleton but sparse or missing bone assignments, AutoRig::rigPriorPartLabels returns a non-empty vector and leaves unassigned vertices as Unknown (0). Since the CLI/MCP/Edit paths now pass that vector into predict, this condition accepts 0 as an authoritative bone hint and overwrites the model/geometric prediction with Unknown; with --no-model or a missing model, a sparsely weighted skinned mesh can therefore come back mostly/all Unknown instead of retaining spatial labels. Ignore the Unknown sentinel here (or only pass sufficiently resolved labels) so unresolved vertices keep the predictor result.

Useful? React with 👍 / 👎.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
src/MeshSegmenter.cpp (1)

123-133: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Deduplicate topology neighbours before voting.

neighbours[v] can contain the same adjacent vertex multiple times across shared triangles, so smoothing becomes tessellation-weighted instead of a majority over adjacent vertices.

Proposed fix
     for (int i = 0; i + 2 < indexCount; i += 3) {
         const int a = static_cast<int>(indices[i]);
         const int b = static_cast<int>(indices[i + 1]);
         const int c = static_cast<int>(indices[i + 2]);
         if (a < 0 || b < 0 || c < 0 || a >= vertexCount || b >= vertexCount || c >= vertexCount)
             continue;
         neighbours[a].push_back(b); neighbours[a].push_back(c);
         neighbours[b].push_back(a); neighbours[b].push_back(c);
         neighbours[c].push_back(a); neighbours[c].push_back(b);
     }
+    for (auto& ns : neighbours) {
+        std::sort(ns.begin(), ns.end());
+        ns.erase(std::unique(ns.begin(), ns.end()), ns.end());
+    }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/MeshSegmenter.cpp` around lines 123 - 133, The neighbour accumulation in
MeshSegmenter::segmentMesh currently adds duplicate adjacent vertices into
neighbours[v], which makes the smoothing vote depend on tessellation density
instead of unique adjacency. Deduplicate each vertex’s neighbour list before the
voting step in the same MeshSegmenter logic, using a per-vertex set or by
uniquing the accumulated neighbours after triangle traversal, while keeping the
rest of the segmentation flow unchanged.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/MeshSegmenter.cpp`:
- Around line 123-133: The neighbour accumulation in MeshSegmenter::segmentMesh
currently adds duplicate adjacent vertices into neighbours[v], which makes the
smoothing vote depend on tessellation density instead of unique adjacency.
Deduplicate each vertex’s neighbour list before the voting step in the same
MeshSegmenter logic, using a per-vertex set or by uniquing the accumulated
neighbours after triangle traversal, while keeping the rest of the segmentation
flow unchanged.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 614afa73-6acc-486f-ac8e-9f5d969346c4

📥 Commits

Reviewing files that changed from the base of the PR and between 2b93d62 and 60dd4cd.

📒 Files selected for processing (4)
  • src/CLIPipeline.cpp
  • src/EditModeController.cpp
  • src/MCPServer.cpp
  • src/MeshSegmenter.cpp

@sonarqubecloud

Copy link
Copy Markdown

@fernandotonon fernandotonon merged commit e0d522e into master Jun 30, 2026
21 checks passed
@fernandotonon fernandotonon deleted the codex/improve-mesh-segmentation branch June 30, 2026 05:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant