Fixes #29493: expand entity_extension.extension from VARCHAR(256) to VARCHAR(512)#29496
Fixes #29493: expand entity_extension.extension from VARCHAR(256) to VARCHAR(512)#29496ivnvMkhl wants to merge 3 commits into
Conversation
…CHAR(256) to VARCHAR(512) FullyQualifiedName.buildHash joins one MD5 hash (32 chars) per FQN segment with dots. Tables with 8+ levels of nested columns produce keys of 263+ chars (8*32 + 7 dots), exceeding the VARCHAR(256) limit and causing a DB error. VARCHAR(512) safely covers up to 15 nesting levels. Migration added for both PostgreSQL and MySQL; bootstrap schemas updated to match.
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
Thanks for your contribution @ivnvMkhl. This won't solve the underlying issue. It simply move the bound higher up. If we are seeing this kind of error in production let's solve the root cause |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
|
Hi there 👋 Thanks for your contribution! The OpenMetadata team will review the PR shortly! Once it has been labeled as Let us know if you need any help! |
Code Review ✅ ApprovedExpands the entity_extension.extension column to VARCHAR(512) to accommodate nested FQN segments. Schema migrations and bootstrap files have been updated, and no issues were found. OptionsDisplay: compact → Showing less information. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
Description of Changes
Fixes #29493
FullyQualifiedName.buildHashbuilds the extension key by MD5-hashing each FQN segment and joining with dots. For deeply nested columns (8+ levels) this produces keys longer than 256 chars (8 segments × 32 chars + 7 dots = 263), exceeding theentity_extension.extensioncolumn limit and causing a DB error:This PR expands the column to
VARCHAR(512), which safely covers up to 15 nesting levels (15×32 + 14 = 494 chars).Changes:
2.0.3for PostgreSQL and MySQLpostgres.sql,mysql.sql) updated to matchType of Change
High-level Design
N/A — schema-only change, no logic changes.
Testing Coverage
UI Evidence
Not applicable.
Checklist
Fixes #<issue>: <description>formatGreptile Summary
This PR fixes a database overflow error by widening
entity_extension.extensionfromVARCHAR(256)toVARCHAR(512)in both MySQL and PostgreSQL, covering the case where deeply nested struct columns (8+ levels) generate hash-based extension keys exceeding 256 characters.2.0.3for both MySQL and PostgreSQL with correct dialect-specific syntax (MODIFY COLUMNvsALTER COLUMN TYPE) and well-documented reasoning for not usingTEXT(PK constraint).mysql.sql,postgres.sql) are updated in sync with the migrations.entity_extension_time_series.extension— a separate table also present in both schemas — uses a different, shorter key pattern (varchar(100)in MySQL) and is unaffected by this change.Confidence Score: 5/5
Schema-only change with no logic impact; widening a VARCHAR column is a non-destructive DDL operation on both MySQL and PostgreSQL.
All four changed files are in sync: migration scripts and bootstrap schemas agree on VARCHAR(512). The MySQL migration correctly carries over the ascii/ascii_bin collation. The PostgreSQL ALTER COLUMN TYPE syntax preserves NOT NULL and the composite PK automatically. No data is at risk and no rollback complexity is introduced.
No files require special attention.
Important Files Changed
Entity Relationship Diagram
%%{init: {'theme': 'neutral'}}%% erDiagram entity_extension { varchar_36 id PK varchar_512 extension PK "was VARCHAR(256)" varchar_256 jsonSchema json json } entity_extension_time_series { varchar_768 entityFQNHash varchar_100 extension "unchanged" varchar_50 jsonSchema json json bigint timestamp }%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% erDiagram entity_extension { varchar_36 id PK varchar_512 extension PK "was VARCHAR(256)" varchar_256 jsonSchema json json } entity_extension_time_series { varchar_768 entityFQNHash varchar_100 extension "unchanged" varchar_50 jsonSchema json json bigint timestamp }Reviews (3): Last reviewed commit: "Merge branch 'main' into fix/entity-exte..." | Re-trigger Greptile