screen-demo: add synthetic-terminal mode via Remotion TerminalScene
Make synthetic screen recording a first-class, discoverable capability alongside real OS capture. For CLI / terminal / install-flow demos where commands and output are predictable, author a `terminal_scene` cut instead of driving a real screen recorder — deterministic, privacy-safe, pixel- perfect, and frame-accurate to narration cues. Components: - TerminalScene.tsx: window chrome, char-by-char typing, blinking cursor, scrolling output, non-blocking floating pills, spring-based reveals - ProviderChip.tsx: rotating badge overlay that cycles through provider names (used in AI-generated-motion scenes) - BackgroundVideoLayer + source_in_seconds + backgroundVideo props on every scene type — supports video-behind-component composition Discovery chain (six layers, so the next agent finds this without reading source code): 1. pipeline_defs/screen-demo.yaml — bump to 2.1, declare production_modes (real_capture, synthetic_terminal) with required_tools, scene_type, and agent_skills pointers 2. skills/pipelines/screen-demo/idea-director.md — mode-selection table at brief time; brief.metadata.production_mode contract 3. skills/pipelines/screen-demo/asset-director.md — reads production_mode and branches asset production (capture+overlays vs steps+narration+ pacing check) 4. .agents/skills/synthetic-screen-recording/SKILL.md — Layer 3 skill with step kinds (cmd/out/pause/pill), pacing rule, and the frozen-terminal failure mode captured from the showcase v3 retune 5. AGENT_GUIDE.md — TerminalScene added to Remotion routing; links SCENE_TYPES.md as the authoritative cut-type registry 6. remotion-composer/SCENE_TYPES.md — new cheat sheet of every cut.type and overlay.type with required fields, plus a "how to add a new scene type" section (candidates: ChatTranscript, EditorScene, PrReview, SlackThread, TicketBoard) Guardrail: - lib/verify_scene_pacing.py — reusable trace() and assert_alignment() helpers that mimic the TerminalScene frame math exactly. Fail loudly before render if steps burn through too fast or leave the scene frozen.
This commit is contained in:
@@ -0,0 +1,176 @@
|
||||
# Synthetic Screen Recording (Remotion TerminalScene)
|
||||
|
||||
**Decision this skill answers:** When the user wants a screen-recording-looking demo of a terminal, CLI tool, or coding workflow — do I **capture the real desktop** (OS screen recording via `screen_recorder`, Windows-MCP, Cap, or Playwright), or do I **synthesize it in Remotion** with the `TerminalScene` component?
|
||||
|
||||
> **Heuristic:** If the agent can author the exact command/output sequence in advance, synthesize. Only capture live when the real behavior is unpredictable, needs a real app UI, or the user explicitly asked for a real recording.
|
||||
|
||||
## Why this exists
|
||||
|
||||
v3 of the OpenMontage showcase tried to use Windows-MCP + `screen_recorder` to drive a Git-Bash window for the install walkthrough. It stalled on window positioning, focus races, and taskbar privacy concerns. We pivoted to **pure Remotion rendering** — a React component named `TerminalScene` that draws a fake terminal and types commands character-by-character. The output is visually indistinguishable from a real screen recording (same traffic-light window chrome, blinking cursor, scrolling output) but deterministic, privacy-safe, pixel-perfect at 1080p, and pace-controllable to the frame.
|
||||
|
||||
That component + pattern is the capability this skill makes discoverable.
|
||||
|
||||
## When to use synthetic (TerminalScene)
|
||||
|
||||
**YES, synthesize when:**
|
||||
- The demo is a **terminal / CLI / coding session** where commands and outputs are predictable
|
||||
- The user wants a polished tutorial feel (clean typography, floating pills, cursor blink)
|
||||
- Install walkthroughs, setup demos, API key config, `make` targets, `git clone` flows
|
||||
- You need tight sync with narration — every command must land on a specific beat
|
||||
- You want the result reproducible (re-render gets identical pixels)
|
||||
- The user's actual desktop has private apps/windows visible you'd otherwise have to crop
|
||||
|
||||
**NO, capture a real screen when:**
|
||||
- The demo is **a real app UI** that can't be faked (Figma, Photoshop, a web app with live state, a browser flow)
|
||||
- The user explicitly asked for a recording of *their* actual screen
|
||||
- The behavior depends on timing you can't script (streaming LLM output, real network latency)
|
||||
- There's a visual quirk (a cursor effect, a plugin pop-up) that only appears in the live environment
|
||||
|
||||
**For a browser demo** → `playwright-recording` skill, not this one.
|
||||
**For a real desktop** → `screen_recorder` tool or Cap via `cap_recorder`.
|
||||
|
||||
## The component — `TerminalScene`
|
||||
|
||||
Located at: `remotion-composer/src/components/TerminalScene.tsx`
|
||||
Exported from: `remotion-composer/src/components/index.ts`
|
||||
Wired in dispatch: `remotion-composer/src/Explainer.tsx` (`if (cut.type === "terminal_scene")`)
|
||||
|
||||
**Props:**
|
||||
```ts
|
||||
interface TerminalSceneProps {
|
||||
title?: string; // shown in the window title bar
|
||||
steps: TerminalStep[]; // the timeline
|
||||
prompt?: string; // "$", ">", etc.
|
||||
accentColor?: string; // pill + prompt glow
|
||||
backgroundColor?: string;
|
||||
}
|
||||
```
|
||||
|
||||
**Step kinds:**
|
||||
```ts
|
||||
{ kind: "cmd", text: string, typeSpeed?: number, holdSeconds?: number }
|
||||
{ kind: "out", text: string, holdSeconds?: number }
|
||||
{ kind: "pause", seconds: number }
|
||||
{ kind: "pill", text: string, color?: string, durationSeconds?: number }
|
||||
```
|
||||
|
||||
- `cmd` — prints the prompt, types the text character-by-character (`typeSpeed` is seconds per character, default 0.035), then holds for `holdSeconds` (default 0.3)
|
||||
- `out` — a line of program output, reveals instantly with a short fade-in
|
||||
- `pause` — dead time. Terminal holds on last visible state. USE THIS TO SYNC WITH NARRATION.
|
||||
- `pill` — non-blocking floating badge (top-right). Spring-in, hold, spring-out. Does NOT advance the cursor — the next step runs in parallel.
|
||||
|
||||
## Authoring pattern
|
||||
|
||||
Author a new scene by adding a cut to `build_composition.py` (or your equivalent props builder):
|
||||
|
||||
```python
|
||||
install_steps = [
|
||||
{"kind": "pause", "seconds": 7.0}, # wait for intro narration
|
||||
{"kind": "cmd", "text": "git clone https://github.com/calesthio/OpenMontage.git",
|
||||
"typeSpeed": 0.045, "holdSeconds": 0.3},
|
||||
{"kind": "out", "text": "Cloning into 'OpenMontage'..."},
|
||||
{"kind": "out", "text": "remote: Enumerating objects: 2847, done."},
|
||||
{"kind": "pill", "text": "repo cloned", "color": "#34D399", "durationSeconds": 2.6},
|
||||
{"kind": "pause", "seconds": 3.8}, # bridge to next narration cue
|
||||
# ...
|
||||
]
|
||||
|
||||
cuts.append({
|
||||
"id": "install-terminal",
|
||||
"type": "terminal_scene",
|
||||
"terminalTitle": "bash — OpenMontage setup",
|
||||
"prompt": "$",
|
||||
"accentColor": "#22D3EE",
|
||||
"steps": install_steps,
|
||||
"in_seconds": 50.0,
|
||||
"out_seconds": 110.0,
|
||||
})
|
||||
```
|
||||
|
||||
## THE RULE: pace with narration, never ahead
|
||||
|
||||
**The #1 failure mode:** steps run continuously and burn through all content in the first 40% of the scene, leaving the terminal frozen for the remaining 60%. This is what killed the v3 first pass — the capability menu rendered at t=80s but narration didn't announce it until t=92s.
|
||||
|
||||
**Do this instead:**
|
||||
|
||||
1. **Know your narration cues** — for each scene, write down the exact video-time each narration segment starts.
|
||||
2. **Start with a pause** that reaches the first narration cue before any command types.
|
||||
3. **Time each command to land with its narration line** — `cmd` should start typing the moment narration says its line, not before.
|
||||
4. **Put pauses between command groups** that bridge to the next narration cue.
|
||||
5. **End with a closer hold** — a pause long enough that the final state is readable after narration ends.
|
||||
|
||||
**Sanity-check your steps before rendering** — every minute of Remotion render is precious. Sum the step durations and verify they equal scene duration:
|
||||
|
||||
```python
|
||||
import math
|
||||
def trace(steps, scene_start, fps=30):
|
||||
t = 0.0
|
||||
for s in steps:
|
||||
k = s["kind"]
|
||||
if k == "cmd":
|
||||
tf = math.ceil(len(s["text"]) * s.get("typeSpeed", 0.035) * fps)
|
||||
t += tf / fps + s.get("holdSeconds", 0.3)
|
||||
elif k == "out":
|
||||
t += max(2, math.ceil(0.08 * fps)) / fps + s.get("holdSeconds", 0.15)
|
||||
elif k == "pause":
|
||||
t += s["seconds"]
|
||||
# "pill" is non-blocking — does NOT advance cursor
|
||||
print(f" {t + scene_start:6.2f}s {k}: {s.get('text', '')[:40]}")
|
||||
trace(install_steps, 50)
|
||||
```
|
||||
|
||||
Look at the output column. Each narration cue's video-time must appear adjacent to the command/output it announces. If a command lands 10s before or after its cue, adjust pauses.
|
||||
|
||||
See `lib/verify_scene_pacing.py` for a reusable version of this script.
|
||||
|
||||
## Design rules (inherited from the v3 retune)
|
||||
|
||||
- **Intro pause** — every terminal scene opens with at least 2s of empty-terminal-with-blinking-cursor before anything types. The viewer needs to register the window.
|
||||
- **Pill timing** — a pill should fire at the exact moment its named event completes on screen (e.g., `repo cloned` immediately after the last `Receiving objects` line). Pills are your substitute for real-world UI notifications.
|
||||
- **Command hold after typing** — keep `holdSeconds` ≥ 0.3 on every `cmd` so viewers register the completed command before the first output scrolls in.
|
||||
- **Output cadence** — space `holdSeconds` on output lines between 0.4 and 1.0. Output that flies too fast feels like a bug; output that crawls feels boring.
|
||||
- **Auto-scroll works** — the terminal holds the most recent 18 lines. Don't worry about off-screen content.
|
||||
- **Cursor blinks only on the latest command line while typing + a ~0.2s tail** after typing completes.
|
||||
|
||||
## `ProviderChip` (companion component)
|
||||
|
||||
The `.agents/skills/synthetic-screen-recording` pattern also owns `ProviderChip` — a rotating badge overlay that cycles through a list of provider names at a fixed cadence. Used in the v3 showcase to cycle through all 11 AI video-gen providers during the "generated motion" section.
|
||||
|
||||
```python
|
||||
overlays.append({
|
||||
"type": "provider_chip",
|
||||
"providers": ["Veo 3.1", "Seedance 2.0", "Kling 2.5", ...],
|
||||
"cycleSeconds": 2.5,
|
||||
"position": "bottom-right",
|
||||
"accentColor": "#22D3EE",
|
||||
"label": "generated with",
|
||||
"in_seconds": 195.0,
|
||||
"out_seconds": 222.5,
|
||||
})
|
||||
```
|
||||
|
||||
Wired in dispatch at: `remotion-composer/src/Explainer.tsx` overlay renderer (`overlay.type === "provider_chip"`).
|
||||
|
||||
## Adding new synthetic-UI components
|
||||
|
||||
The pattern generalizes. When you need to fake another UI surface (Claude Code chat bubbles, a Jira ticket view, a GitHub PR diff, a Slack message, a VS Code status bar):
|
||||
|
||||
1. Copy `TerminalScene.tsx` as a template.
|
||||
2. Define a `steps` interface for the relevant timeline primitives.
|
||||
3. Render each step by interpolating `frame` against cumulative start/end times.
|
||||
4. Wire it into `Explainer.tsx`'s `SceneRenderer` dispatch with a new `cut.type`.
|
||||
5. Add the type to the `Cut` interface in `Explainer.tsx` and to `components/index.ts`.
|
||||
6. Add a section to this skill documenting it.
|
||||
7. Update `remotion-composer/SCENE_TYPES.md` with the new cut type.
|
||||
|
||||
## Related skills
|
||||
|
||||
- `.agents/skills/remotion` — general Remotion authoring (hooks, springs, sequences)
|
||||
- `.agents/skills/playwright-recording` — real browser-flow capture for web apps
|
||||
- `tools/capture/screen_recorder` — ffmpeg-based desktop capture
|
||||
- `tools/capture/cap_recorder` — Cap.so polished desktop capture
|
||||
- `skills/pipelines/screen-demo/asset-director.md` — chooses between synthetic and real for a screen-demo project
|
||||
|
||||
## Provenance
|
||||
|
||||
Introduced: OpenMontage showcase v3 render (2026-04-16). Original motivation: the v3 setup walkthrough section needed a 60-second install demo where every command aligned to Chirp 3 HD narration cues, and Windows-MCP-driven real capture was too flaky in practice. See `projects/openmontage-showcase/build_composition.py` for the reference implementation.
|
||||
@@ -356,6 +356,12 @@ For these requests:
|
||||
- Explainer videos with `flat-motion-graphics` playbook -> Remotion animated scenes, not Ken Burns
|
||||
- Data-driven videos -> Remotion stat cards and charts, not static image screenshots
|
||||
- Any pipeline using still images -> Remotion spring animations, not FFmpeg pan-and-zoom
|
||||
- **Screen demos of a CLI/terminal/install flow -> `TerminalScene` (synthetic screen recording), not OS-level capture.** See `.agents/skills/synthetic-screen-recording/SKILL.md`. Faster, deterministic, privacy-safe. Use real capture (`screen_recorder`, `cap_recorder`, `playwright-recording`) only when the demo is a real app UI or requires unpredictable live behavior.
|
||||
|
||||
### Remotion scene types available in `remotion-composer/`
|
||||
|
||||
See `remotion-composer/SCENE_TYPES.md` for the authoritative list and their cut schemas. Current scene types usable via `cut.type`:
|
||||
`text_card`, `stat_card`, `callout`, `comparison`, `hero_title`, `terminal_scene`, `anime_scene`, `bar_chart`, `line_chart`, `pie_chart`, `kpi_grid`, `progress_bar`. Overlay types include `section_title`, `stat_reveal`, `hero_title`, `provider_chip`.
|
||||
|
||||
**When Remotion is NOT available**, `video_compose` falls back to FFmpeg Ken Burns motion on still images. This still works but produces less engaging visuals. Mention this tradeoff in the proposal.
|
||||
|
||||
|
||||
@@ -0,0 +1,134 @@
|
||||
"""Verify that a TerminalScene `steps` list paces with narration cues.
|
||||
|
||||
Use from any build_composition.py or synthetic-UI builder:
|
||||
|
||||
from lib.verify_scene_pacing import trace, assert_alignment
|
||||
|
||||
trace(install_steps, scene_start=50.0)
|
||||
assert_alignment(
|
||||
install_steps,
|
||||
scene_start=50.0,
|
||||
scene_end=110.0,
|
||||
narration_cues=[
|
||||
(57.0, "seg07 Clone the repo"),
|
||||
(65.5, "seg08 Run make setup"),
|
||||
(83.0, "seg09 Open the folder"),
|
||||
(92.0, "seg10 agent reads guide"),
|
||||
],
|
||||
tolerance=1.0,
|
||||
)
|
||||
|
||||
The tracer mimics the frame math inside TerminalScene.tsx so video-time
|
||||
estimates are exact to 1/fps. Fails loudly if any narration cue has no
|
||||
matching command/output within `tolerance` seconds.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
|
||||
def step_duration(step: dict[str, Any], fps: int = 30) -> float:
|
||||
"""Return the cursor-advancement for a single step (frame-accurate).
|
||||
|
||||
Pills DO NOT advance the cursor — they're non-blocking overlays.
|
||||
"""
|
||||
k = step["kind"]
|
||||
if k == "cmd":
|
||||
type_frames = math.ceil(len(step["text"]) * step.get("typeSpeed", 0.035) * fps)
|
||||
return type_frames / fps + step.get("holdSeconds", 0.3)
|
||||
if k == "out":
|
||||
reveal_frames = max(2, math.ceil(0.08 * fps))
|
||||
return reveal_frames / fps + step.get("holdSeconds", 0.15)
|
||||
if k == "pause":
|
||||
return float(step["seconds"])
|
||||
if k == "pill":
|
||||
return 0.0
|
||||
raise ValueError(f"Unknown step kind: {k!r}")
|
||||
|
||||
|
||||
@dataclass
|
||||
class Landmark:
|
||||
video_time: float
|
||||
kind: str
|
||||
text: str
|
||||
|
||||
|
||||
def trace(steps: list[dict[str, Any]], scene_start: float = 0.0, fps: int = 30, *, quiet: bool = False) -> list[Landmark]:
|
||||
"""Walk the step list and print a video-time landmark for each visible event.
|
||||
|
||||
Returns the list of landmarks (useful for alignment checks).
|
||||
"""
|
||||
cursor = 0.0
|
||||
out: list[Landmark] = []
|
||||
for s in steps:
|
||||
k = s["kind"]
|
||||
vt = round(cursor + scene_start, 2)
|
||||
if k in ("cmd", "out", "pill"):
|
||||
text = s.get("text", "")
|
||||
out.append(Landmark(video_time=vt, kind=k.upper(), text=text))
|
||||
if not quiet:
|
||||
prefix = {"CMD": "CMD ", "OUT": "OUT ", "PILL": "PILL "}[k.upper()]
|
||||
print(f" {vt:7.2f}s {prefix}{text[:60]}")
|
||||
cursor += step_duration(s, fps)
|
||||
|
||||
end_vt = round(cursor + scene_start, 2)
|
||||
if not quiet:
|
||||
print(f" {end_vt:7.2f}s -- steps end --")
|
||||
return out
|
||||
|
||||
|
||||
def assert_alignment(
|
||||
steps: list[dict[str, Any]],
|
||||
scene_start: float,
|
||||
scene_end: float,
|
||||
narration_cues: list[tuple[float, str]],
|
||||
*,
|
||||
tolerance: float = 1.0,
|
||||
fps: int = 30,
|
||||
) -> None:
|
||||
"""Validate that every narration cue has a visual landmark within tolerance.
|
||||
|
||||
Also checks that total step duration does not overflow scene_end.
|
||||
Raises AssertionError on any mismatch.
|
||||
"""
|
||||
landmarks = trace(steps, scene_start, fps, quiet=True)
|
||||
errors: list[str] = []
|
||||
|
||||
for cue_time, cue_desc in narration_cues:
|
||||
# Find closest landmark by video-time
|
||||
if not landmarks:
|
||||
errors.append(f"cue {cue_time:.2f}s ({cue_desc}): no landmarks at all")
|
||||
continue
|
||||
closest = min(landmarks, key=lambda lm: abs(lm.video_time - cue_time))
|
||||
delta = closest.video_time - cue_time
|
||||
if abs(delta) > tolerance:
|
||||
errors.append(
|
||||
f"cue {cue_time:.2f}s ({cue_desc}) has no visual within ±{tolerance:.1f}s — "
|
||||
f"closest is {closest.kind} at {closest.video_time:.2f}s ({delta:+.2f}s off): {closest.text[:40]}"
|
||||
)
|
||||
|
||||
# Overflow check
|
||||
cursor = sum(step_duration(s, fps) for s in steps)
|
||||
end_vt = scene_start + cursor
|
||||
scene_duration = scene_end - scene_start
|
||||
if cursor > scene_duration + 0.5:
|
||||
errors.append(
|
||||
f"steps overflow scene: cursor ends at {end_vt:.2f}s but scene_end is {scene_end:.2f}s "
|
||||
f"(overflow {cursor - scene_duration:.2f}s)"
|
||||
)
|
||||
if cursor < scene_duration - 5.0:
|
||||
errors.append(
|
||||
f"steps underfill scene by {scene_duration - cursor:.2f}s — last visible step holds "
|
||||
f"frozen from {end_vt:.2f}s to {scene_end:.2f}s. Add a closer pause."
|
||||
)
|
||||
|
||||
if errors:
|
||||
raise AssertionError(
|
||||
"Scene pacing check failed:\n - " + "\n - ".join(errors)
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["step_duration", "trace", "assert_alignment", "Landmark"]
|
||||
@@ -1,14 +1,45 @@
|
||||
name: screen-demo
|
||||
version: "2.0"
|
||||
version: "2.1"
|
||||
description: >
|
||||
Screen recording pipeline. Takes raw screen capture (app demo, browser walkthrough,
|
||||
coding tutorial) and produces a polished demo video with callouts, zoom crops,
|
||||
subtitles, and cleaned audio. EP orchestration adds quality gates for legibility,
|
||||
audio clarity, and pacing.
|
||||
Screen recording pipeline. Two production modes:
|
||||
1. REAL CAPTURE — takes raw screen capture (app demo, browser walkthrough,
|
||||
coding tutorial) via screen_recorder, cap_recorder, or playwright-recording
|
||||
and produces a polished demo video with callouts, zoom crops, subtitles,
|
||||
and cleaned audio.
|
||||
2. SYNTHETIC (Remotion TerminalScene) — for CLI/terminal/install-flow demos,
|
||||
renders a deterministic terminal animation with typed commands, scrolling
|
||||
output, floating command pills, and blinking cursor. See
|
||||
.agents/skills/synthetic-screen-recording/SKILL.md. Preferred when the
|
||||
demo content is predictable (install walkthroughs, make targets, git clone,
|
||||
API key config) because it's faster to iterate, privacy-safe, pixel-perfect,
|
||||
and frame-accurate to narration cues.
|
||||
The idea-director picks the mode at brief time based on whether the demo surface
|
||||
is a real app UI (real capture) or a scriptable terminal session (synthetic).
|
||||
category: screen_recording
|
||||
stability: production
|
||||
default_checkpoint_policy: guided
|
||||
|
||||
# Production modes this pipeline supports
|
||||
production_modes:
|
||||
- name: real_capture
|
||||
description: "OS screen recording of a real desktop/browser session"
|
||||
required_tools: [screen_recorder]
|
||||
optional_tools: [cap_recorder]
|
||||
best_for:
|
||||
- Real app UIs (Figma, Photoshop, web apps with live state)
|
||||
- Demos that depend on unpredictable live behavior
|
||||
- User explicitly asked for a recording of their own screen
|
||||
- name: synthetic_terminal
|
||||
description: "Remotion TerminalScene — synthesized terminal animation"
|
||||
required_tools: [video_compose] # Remotion path
|
||||
scene_type: terminal_scene
|
||||
agent_skills: [synthetic-screen-recording]
|
||||
best_for:
|
||||
- CLI / terminal / coding session demos
|
||||
- Install walkthroughs, setup demos, config flows
|
||||
- Tight narration-synced pacing where every command lands on a beat
|
||||
- Reproducible renders (same input = identical pixels)
|
||||
|
||||
orchestration:
|
||||
mode: executive-producer
|
||||
skill: pipelines/screen-demo/executive-producer
|
||||
|
||||
@@ -0,0 +1,64 @@
|
||||
# Remotion Composer — Scene & Overlay Cheat Sheet
|
||||
|
||||
Authoritative list of `cut.type` and `overlay.type` values the `Explainer` composition accepts. Each row maps to a dispatch case in `src/Explainer.tsx`.
|
||||
|
||||
When you add a new component, append it here and in `src/components/index.ts`.
|
||||
|
||||
---
|
||||
|
||||
## Cut types (`cut.type`)
|
||||
|
||||
| `type` | Component | Required fields | Common fields | Purpose |
|
||||
|---|---|---|---|---|
|
||||
| *(none — video)* | `OffthreadVideo` | `source` (path to mp4) | `source_in_seconds`, `animation` (zoom-in, ken-burns), `in_seconds`, `out_seconds` | Play an MP4 clip directly |
|
||||
| *(none — image)* | `Img` | `source` (path to png/jpg) | `animation`, `in_seconds`, `out_seconds` | Play a still with Ken Burns |
|
||||
| `text_card` | `TextCard` | `text` | `fontSize`, `backgroundVideo`, `backgroundOverlay`, `color` | Large-typography beat |
|
||||
| `hero_title` | `HeroTitle` | `text` | `heroSubtitle`, `backgroundVideo`, `backgroundOverlay` | Title/end card |
|
||||
| `stat_card` | `StatCard` | `stat` | `subtitle`, `accentColor`, `backgroundVideo` | A single big number |
|
||||
| `callout` | `CalloutBox` | `text` | `callout_type` (info/warning/tip/quote), `title`, `backgroundVideo` | Boxed message with bullets |
|
||||
| `comparison` | `ComparisonCard` | `leftLabel`, `leftValue`, `rightLabel`, `rightValue` | `title`, `backgroundColor` | Side-by-side compare |
|
||||
| `bar_chart` | `BarChart` | `chartData` | `chartAnimation`, `showValues`, `showGrid`, `backgroundVideo` | Animated bars |
|
||||
| `line_chart` | `LineChart` | `chartSeries` | `chartAnimation`, `xLabel`, `yLabel`, `showMarkers` | Animated line |
|
||||
| `pie_chart` | `PieChart` | `chartData` | `donut`, `centerLabel`, `centerValue`, `showLegend` | Pie / donut |
|
||||
| `kpi_grid` | `KPIGrid` | `chartData` | `title`, `columns`, `chartAnimation` | 2–4 column KPI grid |
|
||||
| `progress_bar` | `ProgressBar` | `progress` | `progressLabel`, `progressColor`, `progressSegments` | Animated progress |
|
||||
| `anime_scene` | `AnimeScene` | `images` (list) | `particles`, `lightingFrom`, `lightingTo`, `vignette` | Still-image anime scene with particles + camera motion |
|
||||
| **`terminal_scene`** | **`TerminalScene`** | **`steps`** (list of cmd/out/pause/pill) | **`terminalTitle`, `prompt`, `accentColor`** | **Synthetic terminal animation — NO real capture needed. See [`.agents/skills/synthetic-screen-recording/SKILL.md`](../.agents/skills/synthetic-screen-recording/SKILL.md)** |
|
||||
|
||||
---
|
||||
|
||||
## Overlay types (`overlay.type`)
|
||||
|
||||
| `type` | Component | Required fields | Common fields | Purpose |
|
||||
|---|---|---|---|---|
|
||||
| `section_title` | `SectionTitle` | `text` | `accentColor`, `position` (top-left, etc.) | Tiny section label |
|
||||
| `stat_reveal` | `StatReveal` | `text` | `subtitle`, `accentColor`, `position` | Corner stat badge |
|
||||
| `hero_title` | `HeroTitle` (as overlay) | `text` | `subtitle` | Full-frame title overlay |
|
||||
| **`provider_chip`** | **`ProviderChip`** | **`providers`** (list of strings) | **`cycleSeconds`, `position`, `accentColor`, `label`** | **Rotating badge that cycles through provider names — used in AI-generated-motion scenes to show which model produced the clip** |
|
||||
|
||||
---
|
||||
|
||||
## Adding a new scene type
|
||||
|
||||
1. Create the React component in `src/components/MyScene.tsx`. Use `interpolate(frame, [inFrame, outFrame], [from, to])` and `spring(...)` for motion. Read `useCurrentFrame()` and `useVideoConfig()`.
|
||||
2. Export it in `src/components/index.ts`.
|
||||
3. Add the `type` to the `Cut` interface in `src/Explainer.tsx` (and any new prop fields).
|
||||
4. Add a dispatch case in `SceneRenderer`:
|
||||
```tsx
|
||||
if (cut.type === "my_scene" && cut.mySceneData) {
|
||||
return maybeWrapWithBg(<MyScene ... />);
|
||||
}
|
||||
```
|
||||
5. Document it in this file. That's what makes it discoverable to the next agent.
|
||||
|
||||
## Existing synthetic-UI components
|
||||
|
||||
Currently only `TerminalScene` exists. The pattern generalizes — likely candidates to add next, if a pipeline needs them:
|
||||
|
||||
- `ChatTranscript` — Claude/Cursor/GPT chat-bubble timeline with typing animation
|
||||
- `EditorScene` — VS Code-style code editor with syntax highlight + cursor motion
|
||||
- `PrReview` — GitHub PR diff view with inline-comment reveals
|
||||
- `SlackThread` — Slack thread with avatars + reaction pops
|
||||
- `TicketBoard` — Jira / Linear card moving across columns
|
||||
|
||||
Pattern: follow `TerminalScene.tsx` — a `steps` list of timeline primitives, cursor-advancing durations, spring-based reveals, optional non-blocking pills/badges.
|
||||
@@ -41,6 +41,9 @@ import { StatReveal } from "./components/StatReveal";
|
||||
import { HeroTitle } from "./components/HeroTitle";
|
||||
import { AnimeScene } from "./components/AnimeScene";
|
||||
import type { CameraMotion } from "./components/AnimeScene";
|
||||
import { TerminalScene } from "./components/TerminalScene";
|
||||
import type { TerminalStep } from "./components/TerminalScene";
|
||||
import { ProviderChip } from "./components/ProviderChip";
|
||||
import type { ParticleType } from "./components/ParticleOverlay";
|
||||
import { resolveTheme, type ThemeConfig, DEFAULT_THEME } from "./Root";
|
||||
|
||||
@@ -197,6 +200,9 @@ interface Cut {
|
||||
subtitle?: string;
|
||||
callout_type?: "info" | "warning" | "tip" | "quote";
|
||||
title?: string;
|
||||
// Video source trim — seek to this point in the source before playback.
|
||||
// Defaults to 0 (play from beginning). Use this instead of in_seconds for source trimming.
|
||||
source_in_seconds?: number;
|
||||
// Comparison props
|
||||
leftLabel?: string;
|
||||
rightLabel?: string;
|
||||
@@ -228,7 +234,9 @@ interface Cut {
|
||||
// Styling overrides
|
||||
backgroundColor?: string;
|
||||
backgroundImage?: string; // AI-generated or stock image rendered behind the component
|
||||
backgroundOverlay?: number; // Opacity of dark overlay on backgroundImage (0-1, default 0.55)
|
||||
backgroundVideo?: string; // Video clip rendered behind the component (takes priority over backgroundImage)
|
||||
backgroundVideoStart?: number; // Seek position in seconds for background video (default 0)
|
||||
backgroundOverlay?: number; // Opacity of dark overlay on backgroundImage/backgroundVideo (0-1, default 0.55)
|
||||
color?: string;
|
||||
accentColor?: string;
|
||||
fontSize?: number;
|
||||
@@ -250,16 +258,24 @@ interface Cut {
|
||||
vignette?: boolean;
|
||||
lightingFrom?: string;
|
||||
lightingTo?: string;
|
||||
// Terminal scene props (type: "terminal_scene")
|
||||
steps?: TerminalStep[];
|
||||
terminalTitle?: string;
|
||||
prompt?: string;
|
||||
}
|
||||
|
||||
interface Overlay {
|
||||
type: "section_title" | "stat_reveal" | "hero_title";
|
||||
type: "section_title" | "stat_reveal" | "hero_title" | "provider_chip";
|
||||
in_seconds: number;
|
||||
out_seconds: number;
|
||||
text: string;
|
||||
text?: string;
|
||||
subtitle?: string;
|
||||
accentColor?: string;
|
||||
position?: string;
|
||||
// provider_chip
|
||||
providers?: string[];
|
||||
cycleSeconds?: number;
|
||||
label?: string;
|
||||
}
|
||||
|
||||
interface AudioLayer {
|
||||
@@ -472,9 +488,54 @@ const BackgroundImageLayer: React.FC<{
|
||||
);
|
||||
};
|
||||
|
||||
// Background video layer — plays a looping video behind component content with dark overlay
|
||||
const BackgroundVideoLayer: React.FC<{
|
||||
src: string;
|
||||
startFrom?: number;
|
||||
overlayOpacity?: number;
|
||||
children: React.ReactNode;
|
||||
}> = ({ src, startFrom = 0, overlayOpacity = 0.55, children }) => {
|
||||
const { fps } = useVideoConfig();
|
||||
|
||||
return (
|
||||
<AbsoluteFill style={{ overflow: "hidden" }}>
|
||||
{/* Background video */}
|
||||
<OffthreadVideo
|
||||
src={resolveAsset(src)}
|
||||
startFrom={Math.round(startFrom * fps)}
|
||||
style={{
|
||||
width: "100%",
|
||||
height: "100%",
|
||||
objectFit: "cover",
|
||||
}}
|
||||
muted
|
||||
/>
|
||||
{/* Dark overlay for readability */}
|
||||
<AbsoluteFill
|
||||
style={{
|
||||
background: `rgba(15, 23, 42, ${overlayOpacity})`,
|
||||
}}
|
||||
/>
|
||||
{/* Component content on top */}
|
||||
{children}
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
|
||||
const SceneRenderer: React.FC<{ cut: Cut; theme: ThemeConfig }> = ({ cut, theme }) => {
|
||||
// Wrap component with background image if specified
|
||||
const maybeWrapWithBgImage = (element: React.ReactElement) => {
|
||||
// Wrap component with background video or image if specified
|
||||
const maybeWrapWithBg = (element: React.ReactElement) => {
|
||||
if (cut.backgroundVideo) {
|
||||
return (
|
||||
<BackgroundVideoLayer
|
||||
src={cut.backgroundVideo}
|
||||
startFrom={cut.backgroundVideoStart ?? 0}
|
||||
overlayOpacity={cut.backgroundOverlay ?? 0.55}
|
||||
>
|
||||
{element}
|
||||
</BackgroundVideoLayer>
|
||||
);
|
||||
}
|
||||
if (cut.backgroundImage) {
|
||||
return (
|
||||
<BackgroundImageLayer
|
||||
@@ -491,24 +552,24 @@ const SceneRenderer: React.FC<{ cut: Cut; theme: ThemeConfig }> = ({ cut, theme
|
||||
// Resolve the scene element based on cut type, then wrap with backgroundImage if set
|
||||
// Use transparent bg so the animated gradient background shows through
|
||||
// When no explicit backgroundColor on the cut, inherit from theme
|
||||
const rawBg = cut.backgroundImage ? "transparent" : (cut.backgroundColor || theme.surfaceColor);
|
||||
const rawBg = (cut.backgroundImage || cut.backgroundVideo) ? "transparent" : (cut.backgroundColor || theme.surfaceColor);
|
||||
const bgColor = (rawBg === theme.backgroundColor || rawBg === "#0F172A" || rawBg === "#0f172a") ? "transparent" : rawBg;
|
||||
const textColor = cut.color || theme.textColor;
|
||||
const accent = cut.accentColor || theme.accentColor;
|
||||
|
||||
// Explicit component types — use theme-derived defaults for colors
|
||||
if (cut.type === "text_card" && cut.text) {
|
||||
return maybeWrapWithBgImage(
|
||||
return maybeWrapWithBg(
|
||||
<TextCard text={cut.text} fontSize={cut.fontSize} color={textColor} backgroundColor={bgColor} />
|
||||
);
|
||||
}
|
||||
if (cut.type === "stat_card" && cut.stat) {
|
||||
return maybeWrapWithBgImage(
|
||||
return maybeWrapWithBg(
|
||||
<StatCard stat={cut.stat} subtitle={cut.subtitle} accentColor={accent} backgroundColor={bgColor} />
|
||||
);
|
||||
}
|
||||
if (cut.type === "callout" && cut.text) {
|
||||
return maybeWrapWithBgImage(
|
||||
return maybeWrapWithBg(
|
||||
<CalloutBox
|
||||
text={cut.text} type={cut.callout_type} title={cut.title}
|
||||
borderColor={accent} backgroundColor={cut.backgroundColor || theme.surfaceColor}
|
||||
@@ -517,7 +578,7 @@ const SceneRenderer: React.FC<{ cut: Cut; theme: ThemeConfig }> = ({ cut, theme
|
||||
);
|
||||
}
|
||||
if (cut.type === "comparison" && cut.leftLabel && cut.rightLabel && cut.leftValue && cut.rightValue) {
|
||||
return maybeWrapWithBgImage(
|
||||
return maybeWrapWithBg(
|
||||
<ComparisonCard
|
||||
leftLabel={cut.leftLabel} rightLabel={cut.rightLabel}
|
||||
leftValue={cut.leftValue} rightValue={cut.rightValue}
|
||||
@@ -526,14 +587,25 @@ const SceneRenderer: React.FC<{ cut: Cut; theme: ThemeConfig }> = ({ cut, theme
|
||||
);
|
||||
}
|
||||
if (cut.type === "hero_title" && cut.text) {
|
||||
return maybeWrapWithBgImage(
|
||||
return maybeWrapWithBg(
|
||||
<HeroTitle title={cut.text} subtitle={cut.heroSubtitle || cut.subtitle} />
|
||||
);
|
||||
}
|
||||
if (cut.type === "terminal_scene" && cut.steps) {
|
||||
return maybeWrapWithBg(
|
||||
<TerminalScene
|
||||
title={cut.terminalTitle || "Terminal"}
|
||||
steps={cut.steps as TerminalStep[]}
|
||||
prompt={cut.prompt}
|
||||
accentColor={accent}
|
||||
backgroundColor={bgColor || theme.backgroundColor}
|
||||
/>
|
||||
);
|
||||
}
|
||||
|
||||
// --- Chart types — use theme.chartColors as default palette ---
|
||||
if (cut.type === "bar_chart" && cut.chartData) {
|
||||
return maybeWrapWithBgImage(
|
||||
return maybeWrapWithBg(
|
||||
<BarChart
|
||||
data={cut.chartData} title={cut.title} colors={cut.chartColors || theme.chartColors}
|
||||
animationStyle={(cut.chartAnimation as any) || "grow-up"}
|
||||
@@ -542,7 +614,7 @@ const SceneRenderer: React.FC<{ cut: Cut; theme: ThemeConfig }> = ({ cut, theme
|
||||
);
|
||||
}
|
||||
if (cut.type === "line_chart" && cut.chartSeries) {
|
||||
return maybeWrapWithBgImage(
|
||||
return maybeWrapWithBg(
|
||||
<LineChart
|
||||
series={cut.chartSeries} title={cut.title} colors={cut.chartColors || theme.chartColors}
|
||||
animationStyle={(cut.chartAnimation as any) || "draw"}
|
||||
@@ -552,7 +624,7 @@ const SceneRenderer: React.FC<{ cut: Cut; theme: ThemeConfig }> = ({ cut, theme
|
||||
);
|
||||
}
|
||||
if (cut.type === "pie_chart" && cut.chartData) {
|
||||
return maybeWrapWithBgImage(
|
||||
return maybeWrapWithBg(
|
||||
<PieChart
|
||||
data={cut.chartData} title={cut.title} colors={cut.chartColors || theme.chartColors}
|
||||
animationStyle={(cut.chartAnimation as any) || "expand"}
|
||||
@@ -562,7 +634,7 @@ const SceneRenderer: React.FC<{ cut: Cut; theme: ThemeConfig }> = ({ cut, theme
|
||||
);
|
||||
}
|
||||
if (cut.type === "kpi_grid" && cut.chartData) {
|
||||
return maybeWrapWithBgImage(
|
||||
return maybeWrapWithBg(
|
||||
<KPIGrid
|
||||
metrics={cut.chartData} title={cut.title} columns={cut.columns}
|
||||
colors={cut.chartColors || theme.chartColors} animationStyle={(cut.chartAnimation as any) || "count-up"}
|
||||
@@ -571,7 +643,7 @@ const SceneRenderer: React.FC<{ cut: Cut; theme: ThemeConfig }> = ({ cut, theme
|
||||
);
|
||||
}
|
||||
if (cut.type === "progress_bar" && cut.progress !== undefined) {
|
||||
return maybeWrapWithBgImage(
|
||||
return maybeWrapWithBg(
|
||||
<AbsoluteFill
|
||||
style={{
|
||||
background: bgColor || theme.surfaceColor,
|
||||
@@ -620,16 +692,16 @@ const SceneRenderer: React.FC<{ cut: Cut; theme: ThemeConfig }> = ({ cut, theme
|
||||
const animation = cut.animation || cut.transform?.animation;
|
||||
|
||||
if (cut.source && isImage(cut.source)) {
|
||||
return <ImageScene src={cut.source} animation={animation} />;
|
||||
return maybeWrapWithBg(<ImageScene src={cut.source} animation={animation} />);
|
||||
}
|
||||
|
||||
if (cut.source && isVideo(cut.source)) {
|
||||
return <VideoScene src={cut.source} startFrom={cut.in_seconds} />;
|
||||
return maybeWrapWithBg(<VideoScene src={cut.source} startFrom={cut.source_in_seconds ?? 0} />);
|
||||
}
|
||||
|
||||
// Final fallback — try as image if source exists, otherwise show text_card
|
||||
if (cut.source) {
|
||||
return <ImageScene src={cut.source} animation={animation} />;
|
||||
return maybeWrapWithBg(<ImageScene src={cut.source} animation={animation} />);
|
||||
}
|
||||
|
||||
// No source, no type — render as text card with cut id as fallback
|
||||
@@ -664,6 +736,17 @@ const OverlayRenderer: React.FC<{ overlay: Overlay }> = ({ overlay }) => {
|
||||
if (overlay.type === "hero_title") {
|
||||
return <HeroTitle title={overlay.text} subtitle={overlay.subtitle} />;
|
||||
}
|
||||
if (overlay.type === "provider_chip" && overlay.providers) {
|
||||
return (
|
||||
<ProviderChip
|
||||
providers={overlay.providers as string[]}
|
||||
cycleSeconds={overlay.cycleSeconds}
|
||||
position={(overlay.position as any) || "bottom-right"}
|
||||
accentColor={overlay.accentColor}
|
||||
label={overlay.label}
|
||||
/>
|
||||
);
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
|
||||
@@ -0,0 +1,101 @@
|
||||
import { AbsoluteFill, interpolate, spring, useCurrentFrame, useVideoConfig } from "remotion";
|
||||
|
||||
/**
|
||||
* ProviderChip — rotating pill of AI video provider names that cycle through.
|
||||
* Positioned in a corner over background video.
|
||||
*/
|
||||
|
||||
interface ProviderChipProps {
|
||||
providers: string[];
|
||||
cycleSeconds?: number;
|
||||
position?: "top-left" | "top-right" | "bottom-left" | "bottom-right";
|
||||
accentColor?: string;
|
||||
label?: string;
|
||||
}
|
||||
|
||||
const POS_STYLES: Record<string, React.CSSProperties> = {
|
||||
"top-left": { top: 48, left: 48 },
|
||||
"top-right": { top: 48, right: 48 },
|
||||
"bottom-left": { bottom: 96, left: 48 }, // avoid caption zone
|
||||
"bottom-right": { bottom: 96, right: 48 },
|
||||
};
|
||||
|
||||
export const ProviderChip: React.FC<ProviderChipProps> = ({
|
||||
providers,
|
||||
cycleSeconds = 2.5,
|
||||
position = "bottom-right",
|
||||
accentColor = "#22D3EE",
|
||||
label = "generated with",
|
||||
}) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps } = useVideoConfig();
|
||||
|
||||
const cycleFrames = Math.max(1, Math.round(cycleSeconds * fps));
|
||||
const idx = Math.floor(frame / cycleFrames) % providers.length;
|
||||
const current = providers[idx];
|
||||
const framesIntoCycle = frame % cycleFrames;
|
||||
|
||||
// Spring in on cycle start
|
||||
const springIn = spring({
|
||||
frame: framesIntoCycle,
|
||||
fps,
|
||||
config: { damping: 14, stiffness: 200 },
|
||||
durationInFrames: Math.ceil(fps * 0.35),
|
||||
});
|
||||
|
||||
// Fade out before cycle end
|
||||
const fadeOut =
|
||||
framesIntoCycle > cycleFrames - fps * 0.25
|
||||
? interpolate(framesIntoCycle, [cycleFrames - fps * 0.25, cycleFrames], [1, 0], { extrapolateRight: "clamp" })
|
||||
: 1;
|
||||
|
||||
const alpha = Math.min(springIn, fadeOut);
|
||||
const translateY = interpolate(springIn, [0, 1], [12, 0]);
|
||||
|
||||
return (
|
||||
<AbsoluteFill pointerEvents="none">
|
||||
<div
|
||||
style={{
|
||||
position: "absolute",
|
||||
...POS_STYLES[position],
|
||||
display: "flex",
|
||||
flexDirection: "column",
|
||||
alignItems: position.includes("right") ? "flex-end" : "flex-start",
|
||||
gap: 8,
|
||||
opacity: alpha,
|
||||
transform: `translateY(${translateY}px)`,
|
||||
}}
|
||||
>
|
||||
<div
|
||||
style={{
|
||||
fontSize: 16,
|
||||
color: "rgba(255,255,255,0.6)",
|
||||
fontFamily: "Inter, sans-serif",
|
||||
fontWeight: 500,
|
||||
letterSpacing: 1.5,
|
||||
textTransform: "uppercase",
|
||||
}}
|
||||
>
|
||||
{label}
|
||||
</div>
|
||||
<div
|
||||
style={{
|
||||
padding: "14px 26px",
|
||||
background: "rgba(11, 15, 26, 0.82)",
|
||||
border: `2px solid ${accentColor}`,
|
||||
borderRadius: 999,
|
||||
color: accentColor,
|
||||
fontFamily: "'Space Grotesk', Inter, sans-serif",
|
||||
fontWeight: 700,
|
||||
fontSize: 28,
|
||||
letterSpacing: 0.3,
|
||||
backdropFilter: "blur(8px)",
|
||||
boxShadow: `0 8px 32px ${accentColor}30`,
|
||||
}}
|
||||
>
|
||||
{current}
|
||||
</div>
|
||||
</div>
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
@@ -0,0 +1,266 @@
|
||||
import { AbsoluteFill, interpolate, spring, useCurrentFrame, useVideoConfig } from "remotion";
|
||||
|
||||
/**
|
||||
* TerminalScene — animated terminal with typed commands and scrolling output.
|
||||
*
|
||||
* Each "step" is either:
|
||||
* { kind: "cmd", text: "git clone ...", typeSpeed?: number } — typed char-by-char with prompt
|
||||
* { kind: "out", text: "cloning into 'OpenMontage'..." } — reveals instantly
|
||||
* { kind: "pause", seconds: number } — silent dwell
|
||||
* { kind: "pill", text: "Piper TTS installed", color?: string } — floating badge
|
||||
*
|
||||
* Steps execute in order at the specified durations. Terminal auto-scrolls when
|
||||
* it fills up.
|
||||
*/
|
||||
|
||||
export type TerminalStep =
|
||||
| { kind: "cmd"; text: string; typeSpeed?: number; holdSeconds?: number }
|
||||
| { kind: "out"; text: string; holdSeconds?: number }
|
||||
| { kind: "pause"; seconds: number }
|
||||
| { kind: "pill"; text: string; color?: string; durationSeconds?: number };
|
||||
|
||||
interface TerminalSceneProps {
|
||||
title?: string;
|
||||
steps: TerminalStep[];
|
||||
prompt?: string;
|
||||
accentColor?: string;
|
||||
backgroundColor?: string;
|
||||
}
|
||||
|
||||
interface RenderedLine {
|
||||
text: string;
|
||||
isCmd: boolean;
|
||||
startFrame: number;
|
||||
endFrame: number; // frame at which typing completes
|
||||
}
|
||||
|
||||
interface RenderedPill {
|
||||
text: string;
|
||||
color: string;
|
||||
startFrame: number;
|
||||
endFrame: number;
|
||||
}
|
||||
|
||||
export const TerminalScene: React.FC<TerminalSceneProps> = ({
|
||||
title = "Terminal",
|
||||
steps,
|
||||
prompt = "$",
|
||||
accentColor = "#22D3EE",
|
||||
backgroundColor = "#0B0F1A",
|
||||
}) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps } = useVideoConfig();
|
||||
|
||||
// Lay out timing in frames
|
||||
const lines: RenderedLine[] = [];
|
||||
const pills: RenderedPill[] = [];
|
||||
let cursorFrame = 0;
|
||||
|
||||
for (const step of steps) {
|
||||
if (step.kind === "cmd") {
|
||||
const speed = step.typeSpeed ?? 0.035; // seconds per char
|
||||
const typeFrames = Math.ceil(step.text.length * speed * fps);
|
||||
const hold = Math.ceil((step.holdSeconds ?? 0.3) * fps);
|
||||
lines.push({
|
||||
text: step.text,
|
||||
isCmd: true,
|
||||
startFrame: cursorFrame,
|
||||
endFrame: cursorFrame + typeFrames,
|
||||
});
|
||||
cursorFrame += typeFrames + hold;
|
||||
} else if (step.kind === "out") {
|
||||
const revealFrames = Math.max(2, Math.ceil(0.08 * fps));
|
||||
const hold = Math.ceil((step.holdSeconds ?? 0.15) * fps);
|
||||
lines.push({
|
||||
text: step.text,
|
||||
isCmd: false,
|
||||
startFrame: cursorFrame,
|
||||
endFrame: cursorFrame + revealFrames,
|
||||
});
|
||||
cursorFrame += revealFrames + hold;
|
||||
} else if (step.kind === "pause") {
|
||||
cursorFrame += Math.ceil(step.seconds * fps);
|
||||
} else if (step.kind === "pill") {
|
||||
const dur = Math.ceil((step.durationSeconds ?? 2.2) * fps);
|
||||
pills.push({
|
||||
text: step.text,
|
||||
color: step.color ?? accentColor,
|
||||
startFrame: cursorFrame,
|
||||
endFrame: cursorFrame + dur,
|
||||
});
|
||||
// pill is non-blocking — don't advance cursor
|
||||
}
|
||||
}
|
||||
|
||||
// Only render lines that have started
|
||||
const visibleLines = lines.filter(l => frame >= l.startFrame);
|
||||
|
||||
// Auto-scroll: keep last N lines in view
|
||||
const MAX_VISIBLE = 18;
|
||||
const scrollStart = Math.max(0, visibleLines.length - MAX_VISIBLE);
|
||||
const renderedLines = visibleLines.slice(scrollStart);
|
||||
|
||||
// Cursor blinks on most recent command
|
||||
const blinkPhase = Math.floor(frame / (fps * 0.55)) % 2 === 0;
|
||||
|
||||
// Terminal window frame fade-in
|
||||
const windowOpacity = spring({ frame, fps, config: { damping: 25, stiffness: 100 } });
|
||||
|
||||
return (
|
||||
<AbsoluteFill
|
||||
style={{
|
||||
background: backgroundColor,
|
||||
justifyContent: "center",
|
||||
alignItems: "center",
|
||||
padding: "80px",
|
||||
fontFamily: "'JetBrains Mono', 'Consolas', 'Monaco', monospace",
|
||||
}}
|
||||
>
|
||||
<div
|
||||
style={{
|
||||
width: "85%",
|
||||
maxWidth: 1600,
|
||||
height: "80%",
|
||||
opacity: windowOpacity,
|
||||
transform: `scale(${interpolate(windowOpacity, [0, 1], [0.97, 1])})`,
|
||||
borderRadius: 16,
|
||||
overflow: "hidden",
|
||||
boxShadow: "0 40px 120px rgba(0,0,0,0.6), 0 0 1px rgba(255,255,255,0.2) inset",
|
||||
background: "#12151F",
|
||||
position: "relative",
|
||||
}}
|
||||
>
|
||||
{/* Title bar */}
|
||||
<div
|
||||
style={{
|
||||
display: "flex",
|
||||
alignItems: "center",
|
||||
gap: 8,
|
||||
padding: "14px 18px",
|
||||
background: "#1A1F2E",
|
||||
borderBottom: "1px solid rgba(255,255,255,0.05)",
|
||||
}}
|
||||
>
|
||||
<div style={{ width: 12, height: 12, borderRadius: "50%", background: "#FF5F56" }} />
|
||||
<div style={{ width: 12, height: 12, borderRadius: "50%", background: "#FFBD2E" }} />
|
||||
<div style={{ width: 12, height: 12, borderRadius: "50%", background: "#27C93F" }} />
|
||||
<div
|
||||
style={{
|
||||
flex: 1,
|
||||
textAlign: "center",
|
||||
color: "#8E8E93",
|
||||
fontSize: 16,
|
||||
fontFamily: "Inter, sans-serif",
|
||||
}}
|
||||
>
|
||||
{title}
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Terminal body */}
|
||||
<div
|
||||
style={{
|
||||
padding: "32px 40px",
|
||||
fontSize: 26,
|
||||
lineHeight: 1.55,
|
||||
color: "#E5E7EB",
|
||||
height: "calc(100% - 46px)",
|
||||
overflow: "hidden",
|
||||
}}
|
||||
>
|
||||
{renderedLines.map((line, idx) => {
|
||||
if (line.isCmd) {
|
||||
// Char-by-char typed command
|
||||
const progress = interpolate(
|
||||
frame,
|
||||
[line.startFrame, line.endFrame],
|
||||
[0, line.text.length],
|
||||
{ extrapolateRight: "clamp" }
|
||||
);
|
||||
const typed = line.text.slice(0, Math.floor(progress));
|
||||
const isLatest = idx === renderedLines.length - 1;
|
||||
const isActive = frame <= line.endFrame + fps * 0.2;
|
||||
return (
|
||||
<div key={`${line.startFrame}-${idx}`} style={{ display: "flex", alignItems: "baseline" }}>
|
||||
<span style={{ color: accentColor, marginRight: 12, fontWeight: 600 }}>{prompt}</span>
|
||||
<span style={{ color: "#F1F5F9" }}>{typed}</span>
|
||||
{isLatest && isActive && blinkPhase && (
|
||||
<span
|
||||
style={{
|
||||
display: "inline-block",
|
||||
width: 12,
|
||||
height: 26,
|
||||
background: "#F1F5F9",
|
||||
marginLeft: 2,
|
||||
transform: "translateY(4px)",
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
} else {
|
||||
// Instant-reveal output line with fade-in
|
||||
const alpha = interpolate(
|
||||
frame,
|
||||
[line.startFrame, line.endFrame],
|
||||
[0, 1],
|
||||
{ extrapolateRight: "clamp" }
|
||||
);
|
||||
return (
|
||||
<div
|
||||
key={`${line.startFrame}-${idx}`}
|
||||
style={{ color: "#9CA3AF", opacity: alpha, paddingLeft: 4 }}
|
||||
>
|
||||
{line.text}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
})}
|
||||
</div>
|
||||
|
||||
{/* Floating command pills */}
|
||||
{pills
|
||||
.filter(p => frame >= p.startFrame && frame <= p.endFrame)
|
||||
.map((pill, idx) => {
|
||||
const lifeProgress = (frame - pill.startFrame) / Math.max(1, pill.endFrame - pill.startFrame);
|
||||
// spring in (0 → 1), hold, spring out (0.8 → 1.0)
|
||||
const inAlpha = spring({
|
||||
frame: frame - pill.startFrame,
|
||||
fps,
|
||||
config: { damping: 14, stiffness: 180 },
|
||||
durationInFrames: Math.ceil(fps * 0.35),
|
||||
});
|
||||
const outAlpha =
|
||||
lifeProgress > 0.82
|
||||
? interpolate(lifeProgress, [0.82, 1], [1, 0], { extrapolateRight: "clamp" })
|
||||
: 1;
|
||||
const alpha = Math.min(inAlpha, outAlpha);
|
||||
const translateY = interpolate(inAlpha, [0, 1], [14, 0]);
|
||||
return (
|
||||
<div
|
||||
key={`${pill.startFrame}-${idx}`}
|
||||
style={{
|
||||
position: "absolute",
|
||||
top: 28 + idx * 62,
|
||||
right: 32,
|
||||
padding: "12px 20px",
|
||||
background: pill.color,
|
||||
color: "#0B0F1A",
|
||||
borderRadius: 999,
|
||||
fontFamily: "Inter, sans-serif",
|
||||
fontWeight: 700,
|
||||
fontSize: 20,
|
||||
letterSpacing: 0.2,
|
||||
opacity: alpha,
|
||||
transform: `translateY(${translateY}px)`,
|
||||
boxShadow: `0 10px 30px ${pill.color}40`,
|
||||
}}
|
||||
>
|
||||
{pill.text}
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
@@ -10,5 +10,8 @@ export { StatReveal } from "./StatReveal";
|
||||
export { HeroTitle } from "./HeroTitle";
|
||||
export { ParticleOverlay } from "./ParticleOverlay";
|
||||
export { AnimeScene } from "./AnimeScene";
|
||||
export { TerminalScene } from "./TerminalScene";
|
||||
export { ProviderChip } from "./ProviderChip";
|
||||
export type { ParticleType } from "./ParticleOverlay";
|
||||
export type { CameraMotion, AnimeSceneProps } from "./AnimeScene";
|
||||
export type { TerminalStep } from "./TerminalScene";
|
||||
|
||||
@@ -4,6 +4,23 @@
|
||||
|
||||
This stage produces the minimal but high-leverage assets that make a screen demo easier to follow: subtitles, audio cleanup, reusable overlays, masks, and optional light-weight support cards.
|
||||
|
||||
## Two production modes — pick before generating assets
|
||||
|
||||
**Read the brief's `production_mode` field.** If `idea` didn't set one, decide here:
|
||||
|
||||
| Mode | When | Asset production looks like |
|
||||
|---|---|---|
|
||||
| **`real_capture`** | Real app UI (browser, design tool, IDE with plugins); live behavior; user asked for their own screen recorded | Clean audio + subtitles + callout overlays (arrows, highlight masks) applied on top of the captured MP4 |
|
||||
| **`synthetic_terminal`** | CLI, terminal, install flow, make targets, git/npm commands, `.env` config — anything scriptable | **No capture at all.** Author a `terminal_scene` cut for `video_compose` (Remotion). Commands type char-by-char, output scrolls, pills announce completions. See `.agents/skills/synthetic-screen-recording/SKILL.md`. |
|
||||
|
||||
**Mode selection heuristic:** *"Can I predict every command and its output before shooting?"* If yes → synthetic. If no → real capture.
|
||||
|
||||
For synthetic mode, the asset stage produces:
|
||||
- **narration (tts_selector)** aligned to the exact video-time each command should type
|
||||
- **a `steps` list** (cmd/out/pause/pill primitives) that paces with narration cues
|
||||
- **a pacing verification** via `lib.verify_scene_pacing.assert_alignment(...)` — must pass before render
|
||||
- **no** screen-recorder footage, no callout arrows, no zoom-crop regions (those are real-capture concepts)
|
||||
|
||||
## Prerequisites
|
||||
|
||||
| Layer | Resource | Purpose |
|
||||
|
||||
@@ -2,9 +2,20 @@
|
||||
|
||||
## When To Use
|
||||
|
||||
Use this pipeline when the source is already a screen recording: product walkthrough, software tutorial, coding demo, browser flow, or troubleshooting capture.
|
||||
Use this pipeline whenever the deliverable is a screen-recording-style demo. There are **two production modes** — pick one in the brief:
|
||||
|
||||
Your job is to turn raw capture into a clear procedural video. The main deliverable at this stage is a schema-valid `brief`, with pipeline-specific detail stored in `brief.metadata`.
|
||||
| Mode | Source material | Pick when |
|
||||
|---|---|---|
|
||||
| **`real_capture`** | An actual screen recording (MP4) captured via `screen_recorder`, `cap_recorder`, or `playwright-recording` | Real app UI, live behavior, browser flows, IDE plugins, user asked for their own screen |
|
||||
| **`synthetic_terminal`** | None — nothing is captured. You author a `terminal_scene` cut for Remotion | CLI / terminal / install flow / make targets / git clone / API key config — anything scriptable where every command and output is predictable |
|
||||
|
||||
**Decision question:** *"Can I predict every command and its output before shooting?"* If yes → synthetic. If no → real capture.
|
||||
|
||||
**Record the mode in `brief.metadata.production_mode`.** The asset-director reads this field to choose between capture+overlay assets vs a `steps` list paced with narration.
|
||||
|
||||
For `synthetic_terminal`, also read `.agents/skills/synthetic-screen-recording/SKILL.md` before proceeding — it encodes the pacing rule that killed an earlier showcase render (commands burned through in 40% of scene time, then terminal froze for the remaining 60%).
|
||||
|
||||
Your job at this stage is to turn the user's request into a clear procedural video plan. The main deliverable is a schema-valid `brief`, with pipeline-specific detail stored in `brief.metadata`.
|
||||
|
||||
## Operating Principles
|
||||
|
||||
|
||||
Reference in New Issue
Block a user