Skip to content

feat(assets): add explain-video skill - #197

Closed
Prithpal-Sooriya wants to merge 4 commits into
mainfrom
feature/create-pr-to-move-my-ybx
Closed

Prithpal-Sooriya wants to merge 4 commits into
mainfrom
feature/create-pr-to-move-my-ybx

Conversation

@Prithpal-Sooriya

@Prithpal-Sooriya Prithpal-Sooriya commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

🎬 explain-video: PR and doc walkthroughs as short narrated videos (Phase 1)

What this PR does

Moves the custom explain-video agent skill into domains/assets/skills/explain-video/, restructured to this repo's bundle layout (skill.md + references/ + scripts/ + assets/) so it survives tools/install intact:

  • skill.md — the router: profile inference (code-change / technical-doc / product-doc), depth, script style, theme, approval gate, production pipeline.
  • references/ — facts ledger rules, script style, PR triage, pronunciation lexicon, and the three content profiles.
  • scripts/ — the local pipeline: Kokoro speech prep, synthesis, segment timing, Whisper audio verification, facts verification, one-time setup, and the pinned single-file renderer (render/vw.ts, a hash-verified copy of tiny-video-renderer v0.2.3, MIT, derived from Videowright) plus its Python tests.
  • assets/styles/ — the three themes: MetaMask, MetaMask Developer, Motion Engineering.

Marked maturity: experimental — opt-in, like acceptance-criteria-doc. Verified locally: skill linter 0 errors, 43/43 Python tests pass from the new layout, and an installer smoke test ships the full bundle (renderer, styles, profiles, lexicon) to Claude / Cursor / Agents with all paths resolving.

And yes — the video at the bottom of this description was generated by the very skill this PR adds. We dogfooded it for the pitch.

Video Walkthrough

Generated by the skill this PR adds — a pipeline walkthrough in the metamask-developer theme, rendered end-to-end by the v0.2.3 renderer bundled in this PR.

explain-video-v023.mp4

🤖 AI-generated walkthrough — script, animation, and voice made by an agent

1. The problem, and the philosophy

Review fatigue is real. Agents and humans are generating more PRs and longer descriptions every sprint. Plain text is the slowest medium for human comprehension — we are drowning in walls of text, and a 600-line diff description is a cognitive bottleneck before anyone reads a single line of code.

The Rung 4 shift. Per Andrej Karpathy's framing of LLM output mediums, bespoke video walkthroughs sit above plain text: a 90-second to 2-minute narrated video lets a reviewer passively digest the intent of a change before diving into the raw diff. Video is the cognitive primer — the mental map. It also serves visual and auditory learners on the team who get nothing extra from another paragraph.

The PR complexity benchmark. Here's the part engineers will like: if the agent struggles to generate a clear, concise 2-minute walkthrough of a PR, that is a flashing red sign that the PR is too complex and needs to be split. The video is a complexity smoke test.

2. How we use it — scope and guardrails

  • Supplementary, not a replacement. Code diffs and PR text remain the searchable single source of truth. The video is a fast on-ramp, never the record.
  • Author accountability. The author self-reviews the generated video before it is posted — no video slop, no misrepresentations. The skill will not upload or attach anything until the plan is approved.
  • Nothing is invented. Every number, date, and name that appears on screen or in the narration is traced to its source in a facts ledger and machine-checked before render. Whisper STT validates the spoken audio in a closed loop, so captions follow the voice and mispronunciations get flagged and repaired.
  • When NOT to use it. Small PRs (<50 lines), emergency hotfixes, simple copy or variable renames. Text is fine there; the video adds nothing.

3. Hardened answers to the skepticism

  • "Is rendering slow?" No. Generation takes 10–15 minutes and runs asynchronously in the background, alongside our existing 20-minute CI pipeline or autonomous agent jobs. Same-day, zero-friction delivery.
  • "Is local setup annoying?" No — it is local-only, and an agent drives the entire install. scripts/setup.sh provisions the Python environment, the local Kokoro TTS model (82M parameters, no cloud voice, no voice cloning) and ffmpeg; the renderer needs no npm install: Node 22.18+, Chrome, ffmpeg. Nothing requires Docker or a CI runner.
  • "Can we trust the narrative?" Three layers: the facts ledger blocks unsourced claims, Whisper STT closed-loop validation checks what was actually said against the script, and the author does a final self-review before posting. Every video carries an AI-generated disclosure note.

4. Try it (engineers and PMs)

./tools/install --repo <your-repo> --target /path/to/your-repo --domain assets --include explain-video

Then ask your agent:

  • "Make an explain video of this PR" (or a branch / diff)
  • "Turn this PRD / ADR / strategy doc into a video" — it infers the right profile and depth on its own

The Phase 1 ask: try it on your next substantive PR or your next doc readout, and tell us where it breaks. That feedback decides whether this graduates from experimental.

Comment thread domains/assets/skills/explain-video/scripts/render/vw.ts Fixed
Comment thread domains/assets/skills/explain-video/scripts/render/vw.ts Fixed
`export const fps = ${fps};`,
`export const timelineUrl = ${JSON.stringify(url(timelinePath))};`,
`export const configUrl = ${JSON.stringify(url(join(projectRoot, "videowright.config.ts")))};`,
`export const segmentLoaders = {\n${ids.map((id) => ` ${JSON.stringify(id)}: () => import(${JSON.stringify(url(join(segmentsDir, id, "index.ts")))})`).join(",\n")}\n};`,
Comment thread domains/assets/skills/explain-video/scripts/render/vw.ts Fixed
Sync vw.ts from tiny-video-renderer v0.2.2 (upstream:
Prithpal-Sooriya/tiny-video-renderer#2), fixing the four CodeQL alerts
on the vendored renderer:

- uncontrolled data in path expressions (x2): request paths no longer
  reach the filesystem; the server indexes servable files at startup
  (with the TS import fallbacks) and answers requests by exact map
  lookup, so traversal and absolute paths simply miss
- improper code sanitization: segment directory names are allowlisted
  ([A-Za-z0-9_-]+) before being interpolated into the generated virtual
  module
- information exposure: 500 responses no longer echo raw error
  messages; the reason goes to the server log only

Rendering output is unchanged: a 610-frame render of a real project is
bit-identical (video and audio stream hashes) before and after, and the
skill's tests still pass. skill.md pin updated to v0.2.2
(sha256 116d1dba33e2b91aebb2c5afa407ff299977d473fb7729c3789e6909a64b7fc9).
@Prithpal-Sooriya

Copy link
Copy Markdown
Contributor Author

Two updates to this PR:

1. Corrected the pitch body. The "Is local setup annoying?" answer wrongly claimed the pipeline "runs inside standardized Docker containers or CI runners." That was false — everything is local-only, and an agent drives the entire install (scripts/setup.sh provisions the Python env, the local Kokoro model and ffmpeg; the renderer needs Node 22.18+, Chrome, ffmpeg — no npm install). The body now says exactly that. Nothing else in the body changed (video walkthrough and AI disclosure intact).

2. Fixed the four CodeQL alerts on scripts/render/vw.ts (uncontrolled data in path expressions ×2, improper code sanitization, information exposure through a stack trace) — pushed in ad1627f. Following the skill's own rule ("do not edit it in place"), the fix was authored upstream and re-vendored:

  • Upstream: Harden the module server: serve from a startup index, allowlist segment ids, keep error detail out of responses Prithpal-Sooriya/tiny-video-renderer#2 — the module server now indexes every servable file at startup (with the TS import fallbacks) and answers requests by exact map lookup, so no request-derived string ever reaches the filesystem; segment directory names are allowlisted ([A-Za-z0-9_-]+) before being interpolated into the generated virtual module; 500 responses no longer echo raw error messages (the reason goes to the server log only).
  • Here: render/vw.ts synced to v0.2.2 and the pinned hash in skill.md updated (sha256 116d1dba…); node --test upstream passes 13/13, the skill's Python tests pass, and a 610-frame render of a real project is bit-identical before and after (video and audio stream hashes match) — the hardening changes no rendering behavior.

Once the upstream PR merges it will be tagged v0.2.2, matching this pin.

…deo-renderer v0.2.2

The style.css is unchanged between v0.2.1 and v0.2.2 (pinned sha
c2f5f7e9… still verifies); the URL now matches the release the
renderer is vendored from.
Same sync as Cloud-Agent-Memory#296 / MetaMask-TODOs#28: code-scan
fixes from Prithpal-Sooriya/tiny-video-renderer#3 (log-forging closed,
jsLiteral escaping for generated code, virtual module import via a
variable). Rendering output is bit-identical; pin and style URL move
to the v0.2.3 tag.
@Prithpal-Sooriya

Copy link
Copy Markdown
Contributor Author

Re-rendered the walkthrough with the v0.2.3 renderer bundled in this PR — the video above is new.

What changed since the first render:

  • Renderer proof: the entire video (scenes → segments → 3,980 frames → h264/AAC mux) was produced by the exact scripts/render/vw.ts at commit be7565a — the v0.2.3 copy with the hardened module server (map-based file serving, SEGMENT_ID allowlist, generic 500s, log-forging fixes). A 6fps preview + frame audit preceded the final 60fps render.
  • Theme: metamask-developer this time (the first render used the consumer theme) — dot-grid #0A0A0A canvas, developer fox lockup, and the syntax tones (orange = focus, purple = secondary, lime = result, blue = structure) introduced as a legend in scene 1.
  • New pipeline scene: the six steps the skill actually runs — Analyse → Script + Facts → Voice (Kokoro) → Verify (Whisper, closed loop) → Subtitles + Timing → Render — alongside the same steps as real terminal commands, ending in node scripts/render/vw.ts render timeline.ts --output video.mp4.
  • Guardrails scene with real artifacts: an actual facts.json entry and the verify_audio.py output (OK: all chunks match — the closed loop passing on this very video's audio).

Dogfood ledger for this render: verify_facts OK (every number sourced), Whisper round-trip zero repairs, 66.3s / 4 scenes / 3,980 frames. 🤖 AI-generated walkthrough — script, animation, and voice made by an agent.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants