fix(ui): make PNG lineage export work for very large graphs - #29480
fix(ui): make PNG lineage export work for very large graphs#29480chirag-madlani wants to merge 4 commits into
Conversation
Two layered fixes to stop "Invalid string length" / frozen renderer on PNG
export for lineage graphs with hundreds of nodes.
1. Use canvas.toBlob() instead of canvas.toDataURL().
- toCanvas + canvas.toBlob is the native async path that returns a Blob
directly. toPng / toDataURL synchronously allocates a base64 JS string
proportional to the physical pixel count — for a 500-node graph that
string exceeds V8's ~512MB max string length and throws "Invalid
string length", which the existing catch path surfaced as a toast but
left the export broken.
- The renderEdgesOverlay composite path now drawImage's the nodes
canvas directly onto the composite (no Image + data URL round-trip),
and the composite emits a Blob via toBlob as well.
- Replaces downloadImageFromBase64 with downloadBlob — same anchor
pattern, takes a Blob.
2. Adaptive pixelRatio cap.
- Even with toBlob, a 500-node graph at pixelRatio:3 would request a
~17946×12792 canvas which exceeds Chrome's 16K dim cap and 64MP area
limit. computeSafePixelRatio drops pixelRatio for very large graphs
so physical dims stay ≤16384 per side and ≤64MP total. Small graphs
still get pixelRatio:3 for sharpness.
Verified the bug end-to-end against the running app at localhost:3001
with 500 nodes (logical 5982×4264 → physical 17946×12792 at pr=3, base64
estimate ~400MB). After the fix, computed pixelRatio drops to ~1.58 and
the canvas + Blob path completes without freezing the renderer.
Update ExportUtils.test.tsx (28 tests, all pass) to cover the new
downloadBlob helper, the adaptive pixelRatio cap (asserts both area and
dim bounds), and the toCanvas / toBlob composite path.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
github-actions
Bot
added
safe to test
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| const canvasToBlob = (canvas: HTMLCanvasElement): Promise<Blob> => | ||
| new Promise((resolve, reject) => { | ||
| canvas.toBlob( | ||
| (blob) => | ||
| blob ? resolve(blob) : reject(new Error('canvas.toBlob returned null')), | ||
| 'image/png', | ||
| 1.0 | ||
| ); | ||
| }); |
There was a problem hiding this comment.
canvas.toBlob can still return null in the simple (non-composite) path with no targeted error message
canvasToBlob correctly rejects when blob is null, and the outer catch in exportPNGImageFromElement will surface a generic error toast. However, a null blob typically signals a cross-origin-tainted canvas, which has a distinct cause from a general export failure. The existing "Invalid string length" branch in the error handler won't match, so the user sees the same generic message regardless of the actual cause. This isn't a regression — the old code had the same gap — but the new wrapper makes the null case more visible, so a more informative error message (or at minimum a dedicated test assertion) for toBlob === null might be worth a follow-up.
| quality: 1.0, | ||
| style: { |
There was a problem hiding this comment.
quality option is ignored by toCanvas — it is only consumed by lossy encoders (toJpeg, toWebp). Leaving it in the options object is harmless but misleading, since the PNG output quality is always lossless regardless of this value. Removing it avoids confusion for future maintainers.
| quality: 1.0, | |
| style: { | |
| style: { |
Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!
Jest test CoverageUI tests summary
|
🔴 Playwright Results — 2 failure(s), 29 flaky✅ 4472 passed · ❌ 2 failed · 🟡 29 flaky · ⏭️ 38 skipped
Genuine Failures (failed on all attempts)❌
|
… at 500
The previous route.fulfill forwarded response.headers() verbatim, which
kept Content-Encoding: gzip and Content-Length from the upstream response
even though we replaced the body with plain (unencoded) JSON. The browser
either failed to gunzip our plain body or read the wrong number of bytes,
so the intercepted "capped at 500" payload never actually reached the
frontend and the test silently ran against the full upstream data.
Switch to Playwright's { response, json } shortcut (same pattern used in
Table.spec.ts:779) so headers stay valid after the body change — the
shortcut auto-strips Content-Encoding and recomputes Content-Length.
Also log original → filtered node counts inside the handler so it's
obvious in the Playwright output whether the cap actually applied for
each of the service / domain / dataProduct views.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
![gitar-bot[bot]](https://avatars.githubusercontent.com/in/827041?s=60&v=4){kind=small}
| console.log( | ||
| `[PlatformLineage route] ${route | ||
| .request() | ||
| .url()} — nodes: ${originalNodeCount} → ${filteredNodeCount}` | ||
| ); |
There was a problem hiding this comment.
💡 Quality: Committed console.log adds noise to CI test output
This commit adds a console.log diagnostic inside the route interceptor. It runs once per intercepted view (service/domain/dataProduct), so it will emit multiple lines into every CI run of this spec. While the accompanying comment explains it is a deliberate diagnostic to confirm the slice is applied, persistent logging in committed test code adds noise and is usually removed once the interceptor is verified. Consider removing it before merge, or gating it behind a debug flag (e.g. process.env.DEBUG_PLATFORM_LINEAGE) so normal runs stay quiet. This is a minor, non-blocking concern; the node-counting logic itself is correct and null-safe.
Was this helpful? React with 👍 / 👎
|
Code Review 👍 Approved with suggestions 0 resolved / 1 findingsEnables successful PNG exports for large lineage graphs by replacing base64-string conversions with 💡 Quality: Committed console.log adds noise to CI test output📄 openmetadata-ui/src/main/resources/ui/playwright/e2e/Flow/PlatformLineage.spec.ts:56-60 This commit adds a 🤖 Prompt for agentsOptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Gitar |
Describe your changes:
Follow-up to #29362 / #29250. On lineage graphs with hundreds of nodes (e.g., the 500-node platform lineage at
/lineage?fullscreen=true), PNG export throwsInvalid string lengthand the existing catch path surfaces the toast "PNG image could not be generated due to an excessively large Lineage" — the user cannot actually export.Verified end-to-end against the running app: 500 nodes → logical viewport 5982×4264 → at
pixelRatio:3, physical canvas would be 17946×12792 (229 MP), well over Chrome's 16K dim cap, andcanvas.toDataURL()would allocate a base64 string ~400 MB, hitting V8's ~512 MB max string length. The renderer freezes for 30+ s before the error path runs.Two layered fixes:
canvas.toBlob()instead ofcanvas.toDataURL()— switch fromtoPngtotoCanvasfromhtml-to-image, then usecanvas.toBlob(native async API that returns aBlobdirectly, no JS-string intermediate). The composite edges path nowdrawImages the nodes canvas directly onto the composite — noImage+ data-URL round-trip.downloadImageFromBase64is replaced withdownloadBlob(same anchor pattern, takes aBlob).Adaptive
pixelRatiocap —computeSafePixelRatiokeeps physical dimensions ≤ 16384 per side and total area ≤ 64 MP. For 500 nodes,pixelRatiodrops from 3 to ~1.58 (logical 5982×4264 → physical ~9460×6748, ~64 MP). Small graphs still get the fullpixelRatio:3for sharpness.Type of change:
High-level design:
N/A — small change.
Tests:
Use cases covered
pixelRatio:3renderEdgesOverlay) still produces a white-background image with edges behind nodesUnit tests
src/utils/Export/ExportUtils.test.tsx— 28 tests, all pass. Covers:downloadBlobanchor lifecycle (replaces thedownloadImageFromBase64describe block)exportPNGImageFromElementusestoCanvas+canvas.toBlob(no base64 string)pixelRatio— asserts both area bound (≤64 MP) and dim bound (≤16384 per side)renderEdgesOverlay— draws edges before nodes, usescomposite.toBlobBackend integration tests
Ingestion integration tests
Playwright (UI) tests
LineageExportPNGSnapshot.spec.ts(edge-presence via file size) andPlatformLineage.spec.ts.Manual testing performed
http://localhost:3001/lineage?fullscreen=true— 500 nodespixelRatiofor the 500-node graph drops from 3 → ~1.58, physical canvas stays under 16K per sideUI screen recording / screenshots:
Not applicable — backend logic change inside the export utility. Exported PNG content is unchanged for small/medium graphs; large graphs now succeed at a slightly lower DPI instead of failing.
Checklist:
Greptile Summary
This PR fixes PNG export failures on very large lineage graphs (500+ nodes) by replacing the
toPng→ base64 string pipeline inExportUtils.tswithtoCanvas→canvas.toBlob, eliminating the V8 max-string-length and Chrome canvas-dimension crashes that previously made large exports impossible.toCanvas+canvas.toBlob: swapshtml-to-image'stoPngfortoCanvas, then converts directly to aBlobviacanvasToBlob; the composite-edges path nowdrawImages the nodes canvas directly instead of round-tripping through a data URL and anImageelement.downloadImageFromBase64is replaced with the simplerdownloadBlob.pixelRatiocap (computeSafePixelRatio): keeps physical canvas dimensions ≤ 16 384 px per side and ≤ 64 MP total; small graphs still render at the full 3× ratio while a 500-node graph drops to ~1.58× to stay within browser limits.Confidence Score: 5/5
Safe to merge — the fix directly eliminates two well-understood browser limits that were causing export failures, with no regressions on small/medium graphs.
Both the toCanvas+Blob path and the adaptive pixelRatio computation are mechanically sound, manually verified end-to-end, and backed by 28 unit tests covering the adaptive capping bounds, composite drawing order, and error branches. The only minor open point is a floating-point rounding alignment between the html-to-image canvas and the composite canvas at non-integer pixel ratios, which is imperceptible in practice.
No files require special attention; ExportUtils.ts is the critical path but the logic is well-tested and the PR author verified it against the 500-node production graph.
Important Files Changed
Flowchart
%%{init: {'theme': 'neutral'}}%% flowchart TD A[exportPNGImageFromElement] --> B[querySelector] B -- not found --> C[throw Error] B -- found --> D[computeSafePixelRatio\nlogicalW × logicalH] D --> E{pixelRatio ≤ 3?} E -- large graph → capped --> F[toCanvas with reduced pixelRatio] E -- small graph → full 3× --> F F --> G{renderEdgesOverlay?} G -- yes composite path --> H[renderEdgesOverlay → edgesCanvas] H --> I[create composite canvas\nphysicalW × physicalH] I --> J[fillRect white] J --> K{edgesCanvas != null?} K -- yes --> L[drawImage edgesCanvas] K -- no --> M[drawImage nodesCanvas] L --> M M --> N[canvasToBlob composite] G -- no simple path --> O[canvasToBlob nodesCanvas] N --> P[downloadBlob] O --> P P --> Q[createObjectURL → anchor click → revokeObjectURL] F -- throws --> R{Invalid string length?} R -- yes --> S[showErrorToast invalid-string-length] R -- no --> T[showErrorToast generic]%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%% flowchart TD A[exportPNGImageFromElement] --> B[querySelector] B -- not found --> C[throw Error] B -- found --> D[computeSafePixelRatio\nlogicalW × logicalH] D --> E{pixelRatio ≤ 3?} E -- large graph → capped --> F[toCanvas with reduced pixelRatio] E -- small graph → full 3× --> F F --> G{renderEdgesOverlay?} G -- yes composite path --> H[renderEdgesOverlay → edgesCanvas] H --> I[create composite canvas\nphysicalW × physicalH] I --> J[fillRect white] J --> K{edgesCanvas != null?} K -- yes --> L[drawImage edgesCanvas] K -- no --> M[drawImage nodesCanvas] L --> M M --> N[canvasToBlob composite] G -- no simple path --> O[canvasToBlob nodesCanvas] N --> P[downloadBlob] O --> P P --> Q[createObjectURL → anchor click → revokeObjectURL] F -- throws --> R{Invalid string length?} R -- yes --> S[showErrorToast invalid-string-length] R -- no --> T[showErrorToast generic]Reviews (3): Last reviewed commit: "Merge branch 'main' into fix-lineage-png..." | Re-trigger Greptile