Design Exploration: Improve visual annotation workflows

[shaunandrews]

Related issues

• Related to STU-2183

How AI was used in this PR

Built through an iterative Codex session with continuous human UX feedback. Codex implemented the feature, hardened the data flow, and added tests; the interaction design was manually reviewed in an isolated Electron build.

Proposed Changes

• Makes Studio preview annotations a focused batch workflow: select an element, write a note, save and continue, then send the batch to chat.
• Adds a compact draggable composer, scrim, persistent outlines and numbered markers, overlapping-layer selection, keyboard controls, scroll locking, and viewport-safe markers.
• Keeps unsent annotations across page navigation while rendering markers and outlines only on the page where each note was created.
• Sends each annotation as its own chat message with an attached element chip containing useful tag, ID/class, and nearby-text context.
• Uses one shared inspector runtime for Studio and the standalone CLI browser, with thin host adapters for Studio's secure bridge and the CLI's local handoff.
• Adds payload validation, size/count limits, safer browser lifecycle handling, and bounded persisted state.

Screenshots

1. Enter annotation mode — light

The preview toolbar exposes a clear Annotate action before any element is selected.

2. Select an element and compose a note — light and dark

The selected element stays isolated by the scrim while the composer exposes element context, overlapping-layer navigation, cancel, save, and send actions.

Light	Dark

3. Save several notes and keep annotating — dark

Saved elements retain numbered outlines while annotation mode stays active and Send to chat becomes available.

4. Keep edge markers visible — light

When the selected element extends beyond the viewport, its numbered marker clamps into view.

5. Send annotations to chat — light

Each submitted note becomes a separate user message with its element context attached underneath.

Review guide

Please focus on the shared inspector lifecycle, cross-page batch persistence, structured annotation payload, and the security boundary between previewed pages and Studio. Transcript styling and screenshots are comparatively self-contained.

⚠️ Visual change: needs human review in light + dark mode.

Testing Instructions

1. In Studio, annotate normal and overlapping elements, drag the composer, save several notes across multiple pages, and send them to chat.
2. Confirm only the current page's saved elements are outlined and each note appears as a separate chat message with element context.
3. Build the CLI, open its standalone annotation browser, and repeat the select/save/navigate/send flow.
4. Run:
   • npm run typecheck
   • npm test -- apps/ui/src/components/site-preview/inspector-script.test.ts apps/ui/src/components/site-preview/index.test.tsx apps/ui/src/ui-classic/components/session-view/annotations.test.ts apps/ui/src/ui-classic/components/session-view/conversation/index.test.ts apps/ui/src/data/queries/use-agent-run.test.tsx apps/cli/ai/inspector/annotation-result.test.ts apps/cli/commands/ai/sessions/tests/resume.test.ts packages/common/ai/inspector-annotations.test.ts packages/common/ai/visual-annotations.test.ts
   • npm run cli:build
   • npm run build --workspace @studio/ui

Pre-merge Checklist

• Typecheck passes
• Focused tests pass
• CLI and production UI builds pass
• Human visual review in light and dark mode
• Full CI passes

[wpmobilebot]

📊 Performance Test Results

Comparing 651a413 vs trunk

app-size

Metric	trunk	651a413	Diff	Change
App Size (Mac)	1406.61 MB	1406.74 MB	+0.13 MB	⚪ 0.0%

site-editor

Metric	trunk	651a413	Diff	Change
load	1074 ms	1070 ms	4 ms	⚪ 0.0%

site-startup

Metric	trunk	651a413	Diff	Change
siteCreation	7544 ms	7539 ms	5 ms	⚪ 0.0%
siteStartup	2871 ms	2872 ms	+1 ms	⚪ 0.0%

---

Results are median values from multiple test runs.

Legend: 🟢 Improvement (faster) | 🔴 Regression (slower) | ⚪ No change (<50ms diff)