Skip to content

Fixes #29493: expand entity_extension.extension from VARCHAR(256) to VARCHAR(512)#29496

Open
ivnvMkhl wants to merge 3 commits into
open-metadata:mainfrom
ivnvMkhl:fix/entity-extension-varchar-overflow
Open

Fixes #29493: expand entity_extension.extension from VARCHAR(256) to VARCHAR(512)#29496
ivnvMkhl wants to merge 3 commits into
open-metadata:mainfrom
ivnvMkhl:fix/entity-extension-varchar-overflow

Conversation

@ivnvMkhl

@ivnvMkhl ivnvMkhl commented Jun 25, 2026

Copy link
Copy Markdown

Description of Changes

Fixes #29493

FullyQualifiedName.buildHash builds the extension key by MD5-hashing each FQN segment and joining with dots. For deeply nested columns (8+ levels) this produces keys longer than 256 chars (8 segments × 32 chars + 7 dots = 263), exceeding the entity_extension.extension column limit and causing a DB error:

value too long for type character varying(256)

This PR expands the column to VARCHAR(512), which safely covers up to 15 nesting levels (15×32 + 14 = 494 chars).

Changes:

  • Migration 2.0.3 for PostgreSQL and MySQL
  • Bootstrap schemas (postgres.sql, mysql.sql) updated to match

Type of Change

  • Bug fix

High-level Design

N/A — schema-only change, no logic changes.

Testing Coverage

  • Manual testing: reproduced the overflow with a table containing 8-level nested struct columns, confirmed the migration resolves the error.

UI Evidence

Not applicable.

Checklist

Greptile Summary

This PR fixes a database overflow error by widening entity_extension.extension from VARCHAR(256) to VARCHAR(512) in both MySQL and PostgreSQL, covering the case where deeply nested struct columns (8+ levels) generate hash-based extension keys exceeding 256 characters.

  • Migration files are added under 2.0.3 for both MySQL and PostgreSQL with correct dialect-specific syntax (MODIFY COLUMN vs ALTER COLUMN TYPE) and well-documented reasoning for not using TEXT (PK constraint).
  • Bootstrap schemas (mysql.sql, postgres.sql) are updated in sync with the migrations.
  • entity_extension_time_series.extension — a separate table also present in both schemas — uses a different, shorter key pattern (varchar(100) in MySQL) and is unaffected by this change.

Confidence Score: 5/5

Schema-only change with no logic impact; widening a VARCHAR column is a non-destructive DDL operation on both MySQL and PostgreSQL.

All four changed files are in sync: migration scripts and bootstrap schemas agree on VARCHAR(512). The MySQL migration correctly carries over the ascii/ascii_bin collation. The PostgreSQL ALTER COLUMN TYPE syntax preserves NOT NULL and the composite PK automatically. No data is at risk and no rollback complexity is introduced.

No files require special attention.

Important Files Changed

Filename Overview
bootstrap/sql/migrations/native/2.0.3/mysql/schemaChanges.sql New migration that widens entity_extension.extension from VARCHAR(256) to VARCHAR(512) with correct CHARACTER SET ascii COLLATE ascii_bin preservation; well-commented explaining the PK constraint that prevents using TEXT.
bootstrap/sql/migrations/native/2.0.3/postgres/schemaChanges.sql New migration using correct PostgreSQL ALTER COLUMN TYPE syntax; preserves NOT NULL and PK constraints automatically; comments correctly note the MySQL TEXT/PK constraint reasoning even though it doesn't apply to Postgres.
bootstrap/sql/schema/mysql.sql Bootstrap schema updated to VARCHAR(512) to match migration; all other column attributes (CHARACTER SET, COLLATE, NOT NULL, PRIMARY KEY) preserved correctly.
bootstrap/sql/schema/postgres.sql Bootstrap schema updated to character varying(512); entity_extension_pkey constraint on (id, extension) is untouched and valid since PostgreSQL VARCHAR widening doesn't require index rebuilds.

Entity Relationship Diagram

%%{init: {'theme': 'neutral'}}%%
erDiagram
    entity_extension {
        varchar_36 id PK
        varchar_512 extension PK "was VARCHAR(256)"
        varchar_256 jsonSchema
        json json
    }
    entity_extension_time_series {
        varchar_768 entityFQNHash
        varchar_100 extension "unchanged"
        varchar_50 jsonSchema
        json json
        bigint timestamp
    }
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
erDiagram
    entity_extension {
        varchar_36 id PK
        varchar_512 extension PK "was VARCHAR(256)"
        varchar_256 jsonSchema
        json json
    }
    entity_extension_time_series {
        varchar_768 entityFQNHash
        varchar_100 extension "unchanged"
        varchar_50 jsonSchema
        json json
        bigint timestamp
    }
Loading

Reviews (3): Last reviewed commit: "Merge branch 'main' into fix/entity-exte..." | Re-trigger Greptile

…CHAR(256) to VARCHAR(512)

FullyQualifiedName.buildHash joins one MD5 hash (32 chars) per FQN segment
with dots. Tables with 8+ levels of nested columns produce keys of 263+ chars
(8*32 + 7 dots), exceeding the VARCHAR(256) limit and causing a DB error.

VARCHAR(512) safely covers up to 15 nesting levels. Migration added for both
PostgreSQL and MySQL; bootstrap schemas updated to match.
@github-actions

Copy link
Copy Markdown
Contributor

❌ PR checklist incomplete

This PR cannot be merged until the following are addressed on its linked issue:

The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically.

Maintainers can bypass this check by adding the skip-pr-checks label.

@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

Comment thread bootstrap/sql/migrations/native/2.0.3/mysql/schemaChanges.sql
@TeddyCr

TeddyCr commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Thanks for your contribution @ivnvMkhl. This won't solve the underlying issue. It simply move the bound higher up. If we are seeing this kind of error in production let's solve the root cause

@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@github-actions

Copy link
Copy Markdown
Contributor

Hi there 👋 Thanks for your contribution!

The OpenMetadata team will review the PR shortly! Once it has been labeled as safe to test, the CI workflows
will start executing and we'll be able to make sure everything is working as expected.

Let us know if you need any help!

@gitar-bot

gitar-bot Bot commented Jun 26, 2026

Copy link
Copy Markdown
Code Review ✅ Approved

Expands the entity_extension.extension column to VARCHAR(512) to accommodate nested FQN segments. Schema migrations and bootstrap files have been updated, and no issues were found.

Options

Display: compact → Showing less information.

Comment with these commands to change:

Compact
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

entity_extension.extension VARCHAR(256) overflow for deeply nested columns (8+ levels)

2 participants