fix(table): server-side column tag filter across all pages (#25063) (#29324) [1.13 backport]#29844
Merged
Merged
Conversation
…29324) * fix(table): server-side column tag filter across all pages (#25063) Column tag filtering ran client-side on the loaded page only, so columns matching a tag on a later page stayed hidden once columns paginate server-side. Push the filter down to the same layer as pagination. Backend: add tags/glossaryTerms query params to the existing /columns/search endpoints (matching the /columns/grid vocabulary) and filter the full matching column set before sort/pagination in searchTableColumnsInternal. Matching uses a single tag_usage prefix lookup (getTagsByPrefix) instead of hydrating tags onto every column. Frontend: route the SchemaTable tag/glossary dropdown through the search call, reset to page 1 on change, drop the client-side onFilter, and source the dropdown options from the full table columns so every tag is selectable regardless of page. * fix(table): update delegating overloads and TagSource import for tag filter The shorter searchTableColumns* overloads still called the new arity without the filterTagFQNs argument (openmetadata-service compile failure), and the IT used the wrong TagSource package. Pass null through the delegators and import the nested TagLabel.TagSource. * fix(table): match column tag filter to UI tag set (derived + certification tags) filterColumnsByTags resolved column tags via getTagsByPrefix, which returns only direct tag_usage rows and strips certification-classification tags. The column tag filter dropdown is built from column.tags (direct + glossary-derived tags), so selecting a derived or certification tag returned zero matches. Resolve column tags the same way the column list responses do: raw tag_usage prefix query (no certification stripping) enriched with glossary-derived tags via batchFetchDerivedTags. Derived-tag fetch failures degrade to direct tags. Add an IT covering the glossaryTerms param, the derived classification tag, and the by-id /columns/search endpoint. * fix(table): AND tags and glossaryTerms column filters instead of OR parseColumnTagFilters unioned tags and glossaryTerms into one set, so a column matched if it had either filter's tag (OR). The UI exposes these as two independent AntD column filters, which AntD combines with AND across columns — matching the pre-existing client-side onFilter behavior. Selecting both a classification tag and a glossary term therefore returned a superset. Thread the two groups separately via a ColumnTagFilter holder and match (any tag in tags) AND (any term in glossaryTerms), OR-ing within each group. Add an IT asserting tags-only and glossaryTerms-only each match their columns while the combination matches only the column carrying both. * fix(ui): use null filteredValue when no column tag filter is active Match the convention used by every sibling columns table (TopicSchema, ContainerDataModel, FileColumnsTable, etc.) which pass filteredValue ?? null. Passing an always-present array deviated from that pattern. * fix(ui): use number-based pagination for tag-filtered columns A tag/glossary filter uses the same offset-based /columns/search endpoint (with paging.total) as text search, but isNumberBased was derived from searchText only, so a tag-only filter fell back to cursor-based pagination and could disable Next/Previous. Include hasTagFilter in the check. * fix(table): preserve column tree nesting in server-side tag filter Column tag/glossary filtering routes through /columns/search, which previously flattened the column tree and returned matching nested columns as top-level rows, dropping their ancestor path. Reconstruct a pruned tree server-side instead: keep every matched column at its real depth together with its ancestor chain, pruning siblings/children that do not match (mirrors the UI's getFilteredTagsData, now applied across all pages). The search path operates on a deep copy of the column tree so pruning and field population never mutate the loaded entity object as a side effect of a read. Semantic change for reviewers: filtered `paging.total` now counts top-level columns that contain a match rather than the flattened count of matching columns. This is the correct unit once rows are nested, and it is what the number-based pager renders. UI: - Expand all ancestors of matches while a search/tag filter is active so deep matches are visible at their level. - Source the Tags/Glossary filter options from the union of the full table columns and the loaded page, so a tag applied to a column on the current page appears as a filter option immediately (the context table.columns is not refreshed on a column-level tag edit). getAllTags dedupes by tag. Isolated in one commit so it can be reverted wholesale if reviewers prefer the flattened behaviour. (cherry picked from commit 3a66dae)
Contributor
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
github-actions
Bot
added
backend
safe to test
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
938
to
+946
|
|
||
| useEffect(() => { | ||
| setExpandedRowKeys((prev) => { | ||
| const autoKeys = getExpandAllKeysToDepth(tableColumns ?? [], 1); | ||
| const depth = searchText || hasTagFilter ? Number.MAX_SAFE_INTEGER : 1; | ||
| const autoKeys = getExpandAllKeysToDepth(tableColumns ?? [], depth); | ||
|
|
||
| return [...new Set([...autoKeys, ...prev])]; | ||
| }); | ||
| }, [tableColumns]); | ||
| }, [tableColumns, searchText, hasTagFilter]); |
Contributor
There was a problem hiding this comment.
When a schema has a large or deeply nested struct column, any search or tag filter now expands every returned descendant by using Number.MAX_SAFE_INTEGER as the depth. A broad filter that returns a large nested root can generate thousands of expanded row keys and make the schema table slow or unresponsive even though only matched ancestor paths need to open.
Code Review ✅ ApprovedBackports the server-side column tag filter fix to the 1.13 release, ensuring consistent search results across all paginated table pages. No issues found in this cherry-picked change. OptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Gitar |
Contributor
|
The Python checkstyle failed. Please run You can install the pre-commit hooks with |
Contributor
❌ UI Checkstyle Failed❌ I18n SyncTranslation locale files are out of sync with Affected files
Fix locally (fast - only checks files changed in this branch): make ui-checkstyle-changed |
|
Contributor
🟡 Playwright Results — all passed (12 flaky)✅ 3674 passed · ❌ 0 failed · 🟡 12 flaky · ⏭️ 26 skipped
🟡 12 flaky test(s) (passed on retry)
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-picks #29324 into
1.13.Original:
fix(table): server-side column tag filter across all pages (#25063)— merge commit3a66dae8eb9ea1dc60ae45e9d8e9e73a829af167onmain.Why
On the table Schema view, columns paginate server-side but tag filtering ran client-side over the loaded page only, so matches on later pages were hidden. This moves tag / glossary-term filtering into the same server-side layer as pagination, so it spans the whole table.
What changed (identical to #29324)
tags+glossaryTermsquery params on both/columns/searchendpoints; server-side tree pruning that keeps matches at their real nested depth;totalcounts top-level columns containing a match; within-group OR, across-group AND semantics.Verification
git diff 3a66dae8 HEADover the 6 changed source files is empty — byte-identical to the original.mvn clean install -pl openmetadata-service,openmetadata-integration-tests -am -DskipTests→ BUILD SUCCESS (both modules SUCCESS).🤖 Generated with Claude Code
Greptile Summary
This PR moves schema-column tag filtering into the server-side column search path.
tagsandglossaryTermsquery params to both column search endpoints.Confidence Score: 4/5
The backend column search path needs fixes before merging.
openmetadata-service/src/main/java/org/openmetadata/service/jdbi3/TableRepository.java
Important Files Changed
Sequence Diagram
%%{init: {'theme': 'neutral'}}%% sequenceDiagram participant UI as SchemaTable UI participant API as TableResource columns/search participant Repo as TableRepository participant Tags as tag usage and derived tags UI->>API: q, limit, offset, tags, glossaryTerms API->>Repo: ColumnTagFilter Repo->>Tags: load direct column tags Repo->>Tags: load derived tags Repo->>Repo: prune matching column tree Repo-->>API: TableColumnList with paging API-->>UI: filtered columns%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% sequenceDiagram participant UI as SchemaTable UI participant API as TableResource columns/search participant Repo as TableRepository participant Tags as tag usage and derived tags UI->>API: q, limit, offset, tags, glossaryTerms API->>Repo: ColumnTagFilter Repo->>Tags: load direct column tags Repo->>Tags: load derived tags Repo->>Repo: prune matching column tree Repo-->>API: TableColumnList with paging API-->>UI: filtered columnsReviews (1): Last reviewed commit: "fix(table): server-side column tag filte..." | Re-trigger Greptile
Context used: