Skip to content

Add studio site create --from static source imports - #3952

Draft
chubes4 wants to merge 43 commits into
trunkfrom
feat/site-artifact-import-cli
Draft

Add studio site create --from static source imports#3952
chubes4 wants to merge 43 commits into
trunkfrom
feat/site-artifact-import-cli

Conversation

@chubes4

@chubes4 chubes4 commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds studio site create --from <source> for creating a Studio site from a Static Site Importer source.
  • Supports server-rendered URLs, directories, individual files, ZIP archives, and website artifact JSON.
  • Installs SSI as temporary import tooling, runs the import outside the dependency Blueprint, activates the generated theme, then removes SSI after success.
  • Makes large URL imports resumable across bounded WP-CLI invocations instead of relying on a longer PHP timeout.
  • Browser capture removes fixed hosting-provider acquisition chrome and its matching body reservation.
  • Responsive export emits one authoring body when desktop/mobile captures differ only in presentation, while retaining dual snapshots for genuinely different responsive content.

Usage

studio site create --from ./site-directory --name "Imported Site" --skip-browser
studio site create --from ./index.html --name "Imported Site" --skip-browser
studio site create --from ./site.zip --name "Imported Site" --skip-browser
studio site create --from ./artifact.json --name "Imported Site" --skip-browser
studio site create --from https://example.com --name "Imported Site" --skip-browser

Import lifecycle

  1. Studio creates and provisions the site through its existing Blueprint path.
  2. Studio shapes the selected source for SSI and runs the import through WP-CLI.
  3. URL imports request one effective SSI batch per invocation with a 180-second cooperative deadline.
  4. Studio reads SSI continuation state and reinvokes the import until complete.
  5. Studio deactivates and removes SSI while retaining the generated active theme.
  6. Failed work remains resumable through the persisted import identity and SSI work directory.

Scope and limitations

  • Server-rendered and browser-captured URL sources are supported.
  • Responsive snapshots collapse only under deterministic semantic equivalence; genuinely different responsive markup remains preserved.
  • The cooperative continuation contract depends on Static Site Importer PR 809, tracked by issue 808.
  • Studio owns site lifecycle and capture orchestration; SSI owns source resolution and theme generation.

Evidence

  • Cara Jane browser capture removed Weebly promotion chrome from all 8 routes.
  • 7 of 8 routes collapsed to one responsive authoring body; contact retained dual markup because its form DOM genuinely differs.
  • Captured HTML reduced from 532,705 to 375,171 bytes.
  • Imported Gutenberg output reduced from 1,552 to 791 blocks, 899 to 458 Groups, and 127 to 76 empty Groups.
  • All 791 imported blocks remain valid under wp.blocks.validateBlock.
  • Fresh source geometry on image-heavy routes is close after normalization: newborn 0.2% desktop / 1.5% mobile height delta; family 4.5% / 3.4%; boudoir 4.7% / 3.0%; love 8.8% / 3.9%.
  • Home, About, Prices, and Contact still have visible typography/navigation/form parity gaps; this iteration improves capture ownership and editor structure but does not claim complete visual parity.

Testing

  • npx vitest run src/lib/capture-export.test.ts src/lib/screenshot/page-helpers.overlay.test.ts (31 tests)
  • npx tsc --noEmit
  • npm run build
  • Full package suite: 2,923 passed; five unrelated tests failed under parallel load. Both affected files pass serially: 29/29 and 59/59.

AI assistance

  • Model/tool: OpenAI GPT-5.6 Sol via OpenCode
  • Used for: Architecture investigation, implementation, responsive capture normalization, tests, live WordPress import diagnosis, and browser verification.
  • Chris Huber reviewed and remains responsible for the change.

@chubes4

chubes4 commented Jul 23, 2026

Copy link
Copy Markdown
Contributor Author

CI follow-up pushed in 7d7f45d8c:

  • Unit lane now runs npm run test -- --tagsFilter='!e2e', keeping tagged CLI integration suites in the dedicated serialized CLI E2E lane.
  • Filtered Unit verification passed: 211 files, 2747 passed, 60 skipped.
  • Focused start-stop.e2e.test.ts passed: 2/2.
  • The full local CLI E2E lane remains machine-sensitive under concurrent Playground/PHP load; isolated lifecycle coverage passes and the existing dedicated lane remains serialized.

AI assistance: openai/gpt-5.6-sol via OpenCode diagnosed the CI lane overlap, drafted the one-line test-selection change, and ran verification. Chris Huber remains responsible for the change.

@chubes4

chubes4 commented Aug 2, 2026

Copy link
Copy Markdown
Contributor Author

1,145-route scale validation

The latest branch head (ca6191a59) is pushed and the end-to-end Richlynn run completed successfully through this Studio CLI path.

  • Source: https://www.richlynngroup.com/
  • Routes: 1,145/1,145 completed
  • Batches: 124/124 completed
  • Final state: zero failed or pending batches
  • WordPress routes: 1,148 unique page URIs, no duplicate paths
  • Cache: 5,364 network requests avoided
  • Recovery: resumed across compiler interruption, encoded Unicode paths, transient HTML classification, and canonical route collisions

This proves the Studio orchestration and dependent SSI batching path can handle the target scale. It does not mark the generated site as fidelity-complete: Richlynn still has 13,055 core/html blocks, 1,235 reported fallbacks, 396 retained external assets, and no exact visual/editor parity proof. The generated site remains evidence under the zero-fallback gate.

Dependency PRs:

Validation on the pushed Studio head: npx vitest run apps/cli/commands/site/tests/create.test.ts apps/cli/lib/tests/run-wp-cli-command.test.ts (66/66 passed).

AI assistance: openai/gpt-5.6-sol via OpenCode was used for scale-run diagnosis, implementation support, verification, and drafting this evidence update. Chris Huber remains responsible for the submitted changes.

@chubes4 chubes4 changed the title Experiment: create Studio sites from static artifacts Add studio create --from static source imports Aug 3, 2026
@borkweb

borkweb commented Aug 3, 2026

Copy link
Copy Markdown
Member

The /liberate command already imports static sites into Studio via the blocks-engine 🕺

You can do so via: /liberate ~/path/to/static/site

That being said, switching out liberate's tooling to a different set of direct calls would make sense if the php-transformer has caught up to the JS approach! I also like the direct CLI command that you have for it.

@chubes4

chubes4 commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

@borkweb I have not compared the outputs side by side, but I was hoping the approaches could coexist initially. This PR intentionally avoids disrupting existing behavior.

Longer term, I think Data Liberation could own the browser-dependent stages: client-side rendering, runtime behavior, and the visual feedback loop, and use the Studio CLI to materialize the site.

The shared contract would be a clean HTML-based website artifact, regardless of the original source.

I think that refactoring can be explored separately from this PR. Redirecting /liberate is not necessary for this primitive to exist.

Comment thread packages/data-liberation-agent/src/lib/screenshot/screenshotter.ts Fixed
Comment on lines +382 to +385
const mhtml = ( await capturePageHtml( page ) ).replace(
/<script\b[^>]*>[\s\S]*?<\/script>/gi,
''
);
if ( ! isDesktop && plan.captureMobileHtml ) {
try {
const mhtml = ( await capturePageHtml( page ) ).replace(
/<script\b[^>]*>[\s\S]*?<\/script>/gi,
chubes4 and others added 23 commits August 13, 2026 16:16
Keep the site create change focused on generic static source imports while preserving the Figma implementation on its dedicated branch.

AI assistance: OpenAI gpt-5.6-sol via OpenCode removed the isolated Figma handoff scope and verified the focused CLI tests, lint, typecheck, and build. Chris Huber reviewed and remains responsible for the change.
AI assistance: OpenAI gpt-5.6-sol via OpenCode reduced the fixture after reproducing and timing the test locally; Chris Huber reviewed and remains responsible for the change.
AI assistance: OpenAI gpt-5.6-sol via OpenCode verified the Homeboy release asset and updated Studio's default importer URL. Chris Huber remains responsible for the change.
@chubes4
chubes4 force-pushed the feat/site-artifact-import-cli branch from f837702 to 1cf64f6 Compare August 13, 2026 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants