docs(prompting): adopt 5-aspect video specification across skills
Incorporate the structured taxonomy from Lin et al. "Building a Precise Video Language with Human-AI Oversight" (CMU/Harvard, arXiv 2604.21718v2). The paper proves prompts structured around five aspects (Subject / Subject Motion / Scene / Spatial Framing / Camera) unlock controllable cinematography in fine-tuned video generation models. Off-the-shelf VLMs already nail subject and scene; the gains live in motion, spatial, and camera, which prompts routinely omit. Universal layer (skills/creative/video-gen-prompting.md, +125 lines): - 5-aspect prompt skeleton replaces flat formula - Camera movements regrouped (translation / rotation / lens-only) with dolly!=zoom, pan!=truck, bird's-eye!=aerial disambiguations - New primitive tables: camera height, camera angle, POV, lens distortion (fisheye vs barrel), focus / DoF (rack / pull / tracking), playback speed (6 modes), subject transitions - Order-matters and self-contained-prompt rules - Identity anchoring rule for multi-shot - Strict static-shot rule, anti-subjective callout, overlays-not-depth - Per-model word-count guidance Per-model guides (sora, veo, hunyuan, ltx, seedance): - Add the primitives each model honors literally - Word-count sweet spots per model - Strengthen seedance verbatim-identity and subject-transition guidance Pipeline directors (cinematic / explainer / animation scene-director, cinematic / explainer asset-director): - 5-aspect scene-plan checklist (per-pipeline adapted) - Overlays-not-depth callout - Pre / critique / post self-review loop for generation prompts Reviewer (skills/meta/reviewer.md): - CHAI critique-quality rules: accurate / complete / constructive - Critical findings now require a proposed_fix Storytelling, cinematic, broll, video-reference-analyst: - Anti-subjective rule (replace mood adjectives with visual causes) - Camera-intent-per-beat for script writers - POV column in stock-footage query templates - 5-aspect structured output mandatory for reference-video analysis skills/INDEX.md: video-gen-prompting marked as canonical 5-aspect spec.
This commit is contained in:
+3
-1
@@ -58,6 +58,7 @@ Key capability families to look for in the output:
|
||||
| `audio_processing` | — | FFmpeg-based local tools |
|
||||
| `enhancement` | — | Mixed providers |
|
||||
| `analysis` | — | Mixed providers |
|
||||
| `character_animation` | — | Local character specs, SVG rigs, pose libraries, action timelines, previews, and QA |
|
||||
| `graphics` | — | Local rendering tools |
|
||||
| `music_generation` | — | Single-provider |
|
||||
| `subtitle` | — | Pure Python |
|
||||
@@ -92,7 +93,7 @@ Key capability families to look for in the output:
|
||||
| Enhancement Strategy | `creative/enhancement-strategy.md` | Overlay placement and density | `ffmpeg` |
|
||||
| Data Visualization | `creative/data-visualization.md` | Chart type selection, animation, label placement | `d3-viz`, `remotion-best-practices` |
|
||||
| Video Stitching | `creative/video-stitching.md` | Multi-clip assembly, AI clip chaining, spatial composition | `ffmpeg`, `video_toolkit` |
|
||||
| Video Gen Prompting | `creative/video-gen-prompting.md` | Universal video generation prompt vocabulary | `ai-video-gen`, `ltx2`, `create-video` |
|
||||
| Video Gen Prompting | `creative/video-gen-prompting.md` | Universal video generation prompt vocabulary; **canonical 5-aspect spec** (Subject / Motion / Scene / Spatial / Camera); ~200 cinematography primitives | `ai-video-gen`, `ltx2`, `create-video` |
|
||||
| ↳ Seedance Prompting | `creative/prompting/seedance-prompting.md` | **Preferred premium default.** Seedance 2.0 8-component structure, multi-shot, lip-sync, reference-to-video | `seedance-2-0`, `ai-video-gen` |
|
||||
| ↳ Grok Prompting | `creative/prompting/grok-prompting.md` | Grok image/video prompting, edit flows, reference-image video | `grok-media` |
|
||||
| ↳ Sora Prompting | `creative/prompting/sora-prompting.md` | Sora 2 structured template, advanced fields | `ai-video-gen` |
|
||||
@@ -127,6 +128,7 @@ Pipeline type skills provide production guidance for specific video formats, ind
|
||||
| Long-Form | `creative/long-form.md` | YouTube 10+ min â€" chapters, retention, end screens |
|
||||
| Screen Recording | `creative/screen-recording.md` | Code walkthroughs, tutorials, software demos |
|
||||
| Animation Pipeline | `creative/animation-pipeline.md` | Motion graphics, easing, transitions, composition |
|
||||
| Character Animation Pipeline | `pipelines/character-animation/` | Rigged local cartoon characters, pose libraries, action timelines, SVG/Canvas/Remotion/HyperFrames rendering |
|
||||
| Cinematic | `creative/cinematic.md` | Letterbox, film pacing, layered audio, color grading |
|
||||
|
||||
## Pipeline Stage Director Skills
|
||||
|
||||
@@ -70,14 +70,18 @@ Fallback: AI-generated image of server racks
|
||||
|
||||
### Query Templates by Scene Type
|
||||
|
||||
| Scene Type | Query Template | Example |
|
||||
Add a **POV keyword** to every query. Stock libraries (Pexels, Pixabay, Storyblocks, Artgrid) explicitly index POV terms — drone, aerial, OTS (over-the-shoulder), macro, top-down, dashcam, FPV, handheld, locked-off — and adding the POV often unlocks better matches than refining the subject. The CMU/Harvard CHAI taxonomy treats POV as a first-class Scene aspect for the same reason: it changes which library shelf you're searching.
|
||||
|
||||
| Scene Type | Query Template | Example with POV |
|
||||
|-----------|---------------|---------|
|
||||
| Establishing | `[place] [time of day]` | "tokyo skyline night" |
|
||||
| Activity | `[person] [action]` | "scientist microscope" |
|
||||
| Object | `[object] [style]` | "circuit board closeup" |
|
||||
| Nature | `[element] [quality]` | "ocean waves aerial" |
|
||||
| Abstract motion | `[movement] [style]` | "light trails timelapse" |
|
||||
| Workplace | `[setting] [activity]` | "modern office meeting" |
|
||||
| Establishing | `[place] [time of day] [POV]` | "tokyo skyline night drone" |
|
||||
| Activity | `[person] [action] [POV]` | "scientist microscope OTS" |
|
||||
| Object | `[object] [style] [POV]` | "circuit board macro top-down" |
|
||||
| Nature | `[element] [quality] [POV]` | "ocean waves aerial drone" |
|
||||
| Abstract motion | `[movement] [style] [POV]` | "light trails timelapse locked-off" |
|
||||
| Workplace | `[setting] [activity] [POV]` | "modern office meeting handheld" |
|
||||
|
||||
If the scene description doesn't already imply a POV, ask the script/scene director — don't default to "no POV." A wrong-POV match (handheld when the scene needs drone) is harder to fix than a wrong color grade.
|
||||
|
||||
## Evaluating Stock Footage Quality
|
||||
|
||||
@@ -89,6 +93,7 @@ When the stock tool returns results, evaluate before using:
|
||||
- **Style compatibility:** Doesn't clash with the playbook's visual style
|
||||
- **No watermarks:** Pexels/Pixabay are license-free, but verify
|
||||
- **Composition:** Subject is well-framed, not cut off awkwardly
|
||||
- **POV match:** Does the footage's actual POV (drone, OTS, macro, handheld, locked-off, etc.) match what the scene needs? A wrong POV — e.g., handheld when the scene wants drone — is **more costly to fix than a wrong color grade**. Reject and re-query rather than try to crop your way out of it.
|
||||
|
||||
### Video Criteria (all image criteria plus)
|
||||
- **Duration:** At least as long as the scene needs (can trim, can't extend)
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
> Sources: No Film School editorial guides, StudioBinder filmmaking resources, Film Riot
|
||||
> production tutorials, CinematographyDB shot databases, Walter Murch "In the Blink of an Eye"
|
||||
|
||||
> For the universal cinematography vocabulary (camera, lens, motion, focus primitives, plus the 5-aspect Subject / Subject Motion / Scene / Spatial Framing / Camera spec), see `skills/creative/video-gen-prompting.md`. This file layers cinematic-specific conventions on top of those primitives — it does not redefine them.
|
||||
|
||||
## Quick Reference Card
|
||||
|
||||
```
|
||||
@@ -16,6 +18,19 @@ MUSIC: 60-90 BPM, orchestral or ambient, dynamic (not loop-based)
|
||||
TARGET LUFS: -14 LUFS integrated, -24 LUFS for quiet moments
|
||||
```
|
||||
|
||||
## Replace Mood Adjectives with Visual Causes
|
||||
|
||||
> **"Cinematic" and "epic" don't constrain pixels.** The cinematic look comes from concrete choices: aspect ratio, lens, lighting key, color grade, shot duration, and audio layer count. State those — the rest is decoration.
|
||||
>
|
||||
> The CMU/Harvard CHAI study showed subjective phrasing varies wildly across annotators and model interpretations, which means a beat tagged "moody" routes to a different visual every render. Replace the adjective with the lighting + grade + shot-duration combination that produces moodiness. See `skills/creative/storytelling.md` "Anti-Subjective Rule" for the script-side equivalent.
|
||||
>
|
||||
> | Mood adjective | Cinematic translation |
|
||||
> |---|---|
|
||||
> | "epic" | 2.39:1 letterbox, 24fps, 8s+ shot duration, orchestral score with crescendo |
|
||||
> | "moody" | `moody_dark` grade at 0.6, key light at 1/8 fill, 6s+ contemplative shots, ambient bed at -28dB |
|
||||
> | "intimate" | 1.85:1, 40-50mm equivalent, shallow DoF, 2 audio layers (dialogue + room tone), no music under dialogue |
|
||||
> | "cinematic" | (this word is banned — pick one of the above or describe the actual choices) |
|
||||
|
||||
## Aspect Ratios
|
||||
|
||||
| Ratio | Resolution (in 1080p frame) | Feel | When to Use |
|
||||
@@ -113,6 +128,7 @@ Add a subtle ambient layer to fill silence and create depth:
|
||||
- Highlights should be slightly rolled off (never pure white)
|
||||
- Skin tones must stay on the vectorscope skin tone line
|
||||
- Consistency across all clips — one LUT/profile for the entire video
|
||||
- **If a beat is described as "moody," rewrite as the lighting + grade + shot-duration combination that produces moodiness.** Don't pass mood adjectives to the asset/edit stages.
|
||||
|
||||
## Applying to OpenMontage
|
||||
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
> Source: [Tencent Prompt Handbook](https://github.com/Tencent-Hunyuan/HunyuanVideo-1.5/blob/main/assets/HunyuanVideo_1_5_Prompt_Handbook_EN.md)
|
||||
> For universal vocabulary, see: `skills/creative/video-gen-prompting.md`
|
||||
|
||||
**Word count:** Hunyuan 1.5 reads well at 80–200 words; doesn't reward 400-word essays.
|
||||
|
||||
## HunyuanVideo Prompt Formula
|
||||
|
||||
### Text-to-Video
|
||||
@@ -34,15 +36,28 @@ Describe lighting with multiple dimensions:
|
||||
|
||||
| Movement | Type | HunyuanVideo Prompt |
|
||||
|----------|------|-------------------|
|
||||
| Crane / Pedestal | Vertical | "camera rises vertically" |
|
||||
| Truck / Tracking | Horizontal | "camera tracks left alongside subject" |
|
||||
| Dolly In | Push | "camera pushes forward toward subject" |
|
||||
| Dolly Out | Pull | "camera pulls back from subject" |
|
||||
| Pan | Rotation | "camera pans right across the scene" |
|
||||
| Crane / Pedestal | Translation (vertical) | "camera rises vertically" |
|
||||
| Truck / Tracking | Translation (horizontal) | "camera tracks left alongside subject" |
|
||||
| Dolly In | Translation (push) | "camera pushes forward toward subject" |
|
||||
| Dolly Out | Translation (pull) | "camera pulls back from subject" |
|
||||
| Pan | Rotation (yaw) | "camera pans right across the scene" |
|
||||
| Tilt | Rotation (pitch) | "camera tilts upward to follow the rocket" |
|
||||
| Roll | Rotation (Z-axis / Dutch) | "camera rolls clockwise into a Dutch tilt" |
|
||||
| Orbit | Circular | "camera orbits around subject" |
|
||||
| Follow | Lock-on | "camera follows subject from behind" |
|
||||
| Zoom | Lens-only (focal length) | "camera slowly zooms in on the figure" |
|
||||
| Rack focus | Lens-only (focal plane, snap) | "rack focus from the foreground bottle to the figure in the background" |
|
||||
| Pull focus | Lens-only (focal plane, gradual) | "camera shifts focus from foreground X to background Y" |
|
||||
| Static | Fixed | "static camera, no movement" |
|
||||
|
||||
### Focal-plane labels at start AND end of dynamic-DoF shots
|
||||
|
||||
Hunyuan benefits when both endpoints of focus-changing shots are stated. State where focus starts AND where it lands — don't leave one implicit.
|
||||
|
||||
Example: "shallow DoF; focus on the foreground bottle at start; focus pulls to the figure in the background by end."
|
||||
|
||||
Without both endpoints, Hunyuan often defaults to deep focus or holds on the wrong plane.
|
||||
|
||||
### Style Keywords
|
||||
|
||||
**Photorealistic / Cinematic**:
|
||||
@@ -65,6 +80,10 @@ When using image-to-video, the input image defines appearance. Your prompt shoul
|
||||
|
||||
**Bad I2V prompt**: "A beautiful woman in a red dress standing in a forest" — this repeats what the image already shows.
|
||||
|
||||
### Order motions temporally
|
||||
|
||||
Describe motion in temporal order; if multiple movements occur, separate them ("first the camera pans right, then tilts upward"). Hunyuan executes motion in the order it appears in the prompt — bundling two movements into one clause causes one of them to be dropped or blended.
|
||||
|
||||
## Example (T2V)
|
||||
|
||||
```
|
||||
|
||||
@@ -11,9 +11,13 @@ LTX-2 uses a clean, focused prompt structure:
|
||||
2. **Set the scene** — lighting, color palette, textures, atmosphere
|
||||
3. **Describe the action** — natural sequence flowing from beginning to end
|
||||
4. **Define the character(s)** — physical cues (age, hair, clothes), not abstract labels
|
||||
5. **Camera movement(s)** — specify how and when; describe what appears AFTER the movement
|
||||
5. **Camera movement(s)** — specify how and when; describe what appears AFTER the movement. (LTX honors the translation/rotation/lens distinction: `dolly` ≠ `zoom`, `pan` ≠ `truck`. Pick the right family — translation moves the rig, rotation pivots it, lens-only changes focal length or focal plane without moving the camera.)
|
||||
6. **Describe the audio** — ambient sound, music, speech, or singing
|
||||
|
||||
### Strict-Static-Shot rule
|
||||
|
||||
If you write "static camera," the shot must have NO movement, NO focus change, NO zoom. LTX takes "static" literally — adding any motion verb later in the prompt will either be ignored or will produce a glitch where the camera contradicts itself. Pick one: static, OR a single named movement.
|
||||
|
||||
## LTX-Specific Tips
|
||||
|
||||
### Post-Movement Description
|
||||
@@ -51,6 +55,7 @@ LTX organizes styles into three families:
|
||||
| Overloaded scenes | Many characters/actions reduces coherence |
|
||||
| Conflicting lighting descriptions | Pick one setup, commit to it |
|
||||
| Starting complex | Build up: simple prompt first, add layers |
|
||||
| Prompts over ~80 words | LTX-2 degrades past that. Pick the most important 5–6 elements. |
|
||||
|
||||
## LTX Technical Notes
|
||||
|
||||
|
||||
@@ -44,6 +44,8 @@ Seedance is unusually literal about camera language, multi-shot cuts, and quoted
|
||||
|
||||
## Multi-shot pattern
|
||||
|
||||
> **Repeat identity verbatim across every shot.** "the same character" / pronouns / "Aang again" do not work. Repeat the 3–6 disambiguating visual attributes verbatim in every shot block. Seedance treats each shot as if you said it cold.
|
||||
|
||||
Seedance honors explicit shot lists:
|
||||
|
||||
```
|
||||
@@ -63,6 +65,22 @@ Style: anamorphic lens, teal-orange cinematic grade, 35mm film grain.
|
||||
Audio: rising orchestral swell with low taiko pulse, wind, distant wingbeats.
|
||||
```
|
||||
|
||||
### Subject transition primitives in multi-shot
|
||||
|
||||
Seedance handles four distinct ways a subject can enter or exit a shot. Naming the primitive explicitly helps the model build the right transition between shots.
|
||||
|
||||
- **Subject revealing** (by camera move OR subject move) — the subject becomes visible mid-shot.
|
||||
Example: `Shot 2 (slow truck right): empty corridor at first; the camera trucks right to reveal Aang — bald, blue arrow tattoo, orange robes — pressed flat against the wall.`
|
||||
|
||||
- **Subject disappearing** — the subject leaves frame, by motion or occlusion.
|
||||
Example: `Shot 4 (static wide): Aang — bald, blue arrow tattoo, orange robes — sprints into the temple doorway and is swallowed by shadow; camera holds on the empty threshold.`
|
||||
|
||||
- **Subject switching** (rack focus / camera move) — focus or framing transfers from one subject to another.
|
||||
Example: `Shot 5 (close-up, rack focus): rack focus from Aang's glowing arrow tattoo in foreground to Sokka — dark hair, blue tunic, boomerang on back — emerging from the mist behind.`
|
||||
|
||||
- **Complex alternating focus** — focus oscillates between two subjects within one shot.
|
||||
Example: `Shot 7 (medium two-shot, alternating rack focus): focus on Aang — bald, blue arrow tattoo, orange robes — as he speaks, then pulls to Katara — long brown hair, blue water-tribe parka — as she answers, then back to Aang on the final beat.`
|
||||
|
||||
## Lip-sync pattern
|
||||
|
||||
```
|
||||
@@ -84,6 +102,7 @@ Sokka, half a step behind, replies: "Then we fight."
|
||||
| `generate_audio` | Keep `true` — sync audio is the moat. Strip in compose if unused. |
|
||||
| `model_variant` | `standard` for hero + multi-shot + camera-heavy. `fast` for b-roll, previews, latency-capped jobs. |
|
||||
| `seed` | Lock once a shot composition reads; iterate variants with the same seed. |
|
||||
| `prompt length` | 200–400 words for hero shots; 80–150 for inserts. Seedance is one of the few models that rewards long, structured 5-aspect prompts. |
|
||||
|
||||
## Iteration strategy
|
||||
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
> Source: [OpenAI Sora 2 Cookbook](https://developers.openai.com/cookbook/examples/sora/sora2_prompting_guide)
|
||||
> For universal vocabulary, see: `skills/creative/video-gen-prompting.md`
|
||||
|
||||
**Word count:** Sora 2 plateaus around 100–250 words. Past 250, additional detail rarely improves output.
|
||||
|
||||
## Sora-Specific Prompt Template
|
||||
|
||||
Sora responds best to a structured format with prose + cinematography block + action beats:
|
||||
@@ -40,6 +42,9 @@ Sora uniquely responds to these production-level details that most models ignore
|
||||
| **Wardrobe** | "navy coat, sleeves rolled, suspenders loose" |
|
||||
| **Finishing** | "fine-grain overlay, mild halation, gate weave, soft vignette" |
|
||||
| **Shutter** | "180° shutter angle" |
|
||||
| **Playback speed** | "speed ramp from 1x to 0.25x mid-shot", "stop-motion staccato", "time-reversed exhale" |
|
||||
| **Lens distortion** | "fisheye barrel distortion at the edges", "subtle barrel curvature on straight lines" |
|
||||
| **Focus mode** | "rack focus from foreground bottle to background figure", "deep focus, FG to BG sharp" |
|
||||
|
||||
## What Sora Does Differently
|
||||
|
||||
|
||||
@@ -3,6 +3,8 @@
|
||||
> Source: [Vertex AI Video Gen Prompt Guide](https://cloud.google.com/vertex-ai/generative-ai/docs/video/video-gen-prompt-guide)
|
||||
> For universal vocabulary, see: `skills/creative/video-gen-prompting.md`
|
||||
|
||||
**Word count:** VEO 3.1 sweet spot is 100–250 words; longer prompts stop helping.
|
||||
|
||||
## VEO-Specific 14-Component Structure
|
||||
|
||||
VEO responds to the most comprehensive prompt structure of any model:
|
||||
@@ -29,16 +31,30 @@ VEO responds to the most comprehensive prompt structure of any model:
|
||||
- **Negative prompts**: Explicitly supported — "no text overlays, no watermarks, no lens flare"
|
||||
- **Editing vocabulary**: Understands "match cut", "jump cut", "montage", "split diopter" as prompt terms.
|
||||
|
||||
### Camera vocabulary VEO honors literally
|
||||
|
||||
VEO 3.1 distinguishes the three camera-motion families and treats their tokens as separate primitives. Mixing them up (e.g. asking for a "zoom" when you mean a "dolly") will produce the wrong move.
|
||||
|
||||
- **Translation (rig physically moves):** `dolly` (in/out along the lens axis), `truck` (left/right laterally), `pedestal` (up/down vertically)
|
||||
- **Rotation (rig stays put, camera rotates):** `pan` (yaw, left/right), `tilt` (pitch, up/down), `roll` (Dutch / Z-axis)
|
||||
- **Lens-only (rig and body don't move):** `zoom` (focal length change), `rack focus` / `pull focus` / `focus tracking` (focal-plane change)
|
||||
|
||||
dolly ≠ zoom; pan ≠ truck. VEO follows whichever token leads.
|
||||
|
||||
## VEO Lens Effects (Unique)
|
||||
|
||||
VEO specifically responds to optical effects most models ignore:
|
||||
|
||||
| Effect | Prompt Language |
|
||||
|--------|----------------|
|
||||
| **Rack focus** | "rack focus from foreground flower to background figure" |
|
||||
| **Rack focus** | "rack focus from foreground flower to background figure" (snap shift) |
|
||||
| **Pull focus** | "slow pull focus from the candle in the foreground to the doorway behind" (gradual, slower than rack) |
|
||||
| **Focus tracking** | "focus tracks the runner as she crosses frame; background stays soft" (focus follows a moving subject) |
|
||||
| **Dolly zoom (vertigo)** | "vertigo effect as character realizes the truth" |
|
||||
| **Fisheye** | "fisheye lens distortion, skatepark POV" |
|
||||
| **Lens flare** | "anamorphic lens flare from setting sun" |
|
||||
| **Anamorphic lens flare** | "anamorphic lens flare streaking horizontally from setting sun" |
|
||||
|
||||
These three focus modes (rack, pull, tracking) are different — VEO 3.1 honors the distinction per the paper.
|
||||
|
||||
## VEO Art Movement References
|
||||
|
||||
|
||||
@@ -57,6 +57,32 @@ For a **3-minute explainer video** (scale proportionally for other lengths):
|
||||
| 3 min | 3-5 | 8s | 22s | 100s | 30s | 15s |
|
||||
| 5 min | 5-8 | 10s | 30s | 180s | 50s | 20s |
|
||||
|
||||
## Anti-Subjective Rule
|
||||
|
||||
> Hooks, beats, and section descriptions in OpenMontage scripts must describe the **visual cause** of the emotion, not the emotion itself. The CMU/Harvard CHAI study showed that subjective phrasing varies wildly across annotators and across model interpretations — so it does not constrain pixels and it doesn't reliably guide downstream generation tools.
|
||||
>
|
||||
> | Avoid | Use instead |
|
||||
> |---|---|
|
||||
> | "epic reveal" | "wide aerial pull-back; subject silhouetted against rising sun" |
|
||||
> | "inspiring moment" | "low angle on the subject's face; light catches the edge of a tear" |
|
||||
> | "moody atmosphere" | "low-key key light, lifted shadows by 2 stops, fog volumetrics" |
|
||||
> | "powerful music swell" | "music drops out at 0:42, holds 1.5s of silence, returns with low taiko at half tempo" |
|
||||
>
|
||||
> The rule applies to script narration AND to the metadata fields scene-director consumes. For the universal vocabulary that names these visual primitives, see `skills/creative/video-gen-prompting.md`.
|
||||
|
||||
## Subject Transitions in the Script
|
||||
|
||||
When a script beat introduces a new subject, kills one off, or hands focus from one subject to another, **name the transition explicitly** so the scene-director doesn't have to infer it. The CMU/Harvard taxonomy uses four labels:
|
||||
|
||||
| Label | What it means |
|
||||
|---|---|
|
||||
| **revealing** | A new subject enters frame or is uncovered (door opens, camera pans to find them, fog clears). |
|
||||
| **disappearing** | An existing subject leaves frame or is removed (walks out, fades, eclipsed). |
|
||||
| **switching** | Focus jumps from subject A to subject B (cut, rack focus, camera whip). |
|
||||
| **complex-alternating** | Multiple subjects trade focus repeatedly within a beat (debate cross-cutting, ensemble action). |
|
||||
|
||||
Add 1-2 sentences in the beat describing the mechanism (cut, pan, reveal-by-light, etc.). This propagates into the scene_plan as a transition primitive.
|
||||
|
||||
## Hook Types
|
||||
|
||||
| Type | Pattern | Best For |
|
||||
@@ -114,6 +140,20 @@ Don't explain the answer. **Reconstruct the reasoning path** so the viewer feels
|
||||
**Progressive Revelation:** Never show the full picture at once. Build visuals layer by layer.
|
||||
Each layer arrives exactly when the narration references it.
|
||||
|
||||
## Camera Intent Per Beat
|
||||
|
||||
When writing a beat, attach one line of camera intent so the scene-director doesn't have to invent it from a blank slate. Use the universal vocabulary in `skills/creative/video-gen-prompting.md` (Subject / Subject Motion / Scene / Spatial Framing / Camera). One line is enough — the scene-director will expand it.
|
||||
|
||||
Example beat:
|
||||
|
||||
```
|
||||
[0:30] Concept 1 — atoms aren't tiny planets
|
||||
Narration: "We grew up imagining electrons as tiny planets orbiting the nucleus..."
|
||||
Camera intent: medium shot of stylized atom; slow rotation; deep focus.
|
||||
```
|
||||
|
||||
The camera-intent line is consumed verbatim by the scene-director's 5-aspect spec — keep it concrete, no mood adjectives.
|
||||
|
||||
## Pacing Rules
|
||||
|
||||
| Rule | Value | Source |
|
||||
|
||||
@@ -24,18 +24,50 @@ For model-specific tips, see the linked guides below.
|
||||
| **Kling 2.6** | [Kling Prompt Guide](https://fal.ai/learn/devs/kling-2-6-pro-prompt-guide) | 4-part structure. Supports `++emphasis++` syntax for key elements. |
|
||||
| **Wan 2.1 / CogVideoX** | Use this generic guide | No official prompt guide. Standard cinematographic vocabulary works well. |
|
||||
|
||||
## Order Matters
|
||||
|
||||
When listing multiple subjects or events:
|
||||
|
||||
- **Temporal order** when events unfold over time ("First X enters, then Y reacts").
|
||||
- **Prominence order** when temporal isn't relevant — humans before objects, largest/most-centered first, then secondary subjects.
|
||||
|
||||
## Self-Contained Prompt
|
||||
|
||||
> Write the prompt so that someone who has never seen the intended video could picture the subjects, scene, motion, and camera work from your text alone. If a reader could not picture it, a generation model will not render it.
|
||||
|
||||
## Universal Prompt Formula
|
||||
|
||||
All video generation models respond to this structure. Include what's relevant, omit what's not.
|
||||
Prior work (CMU/Harvard, "Building a Precise Video Language with Human-AI Oversight") shows VLMs reliably describe subject + scene but fail on motion, spatial, and camera. **Forcing prompts to fill all five slots is the highest-leverage change.**
|
||||
|
||||
The OpenMontage canonical 5-aspect skeleton:
|
||||
|
||||
```
|
||||
[Shot type/framing] + [Camera movement] + [Subject description] +
|
||||
[Action/motion in beats] + [Setting/environment] + [Lighting] +
|
||||
[Style/aesthetic] + [Audio/atmosphere]
|
||||
[Subject] type + key visual attributes + how to disambiguate when multiple
|
||||
[Subject Motion] actions in temporal order; subject↔object and subject↔subject interactions; group action
|
||||
[Scene] overlays (separately!) + POV + setting + time of day + scene dynamics
|
||||
[Spatial] shot size + position-in-frame + depth (FG/MG/BG) + camera-height-relative
|
||||
— and how those CHANGE during the clip
|
||||
[Camera] playback speed → lens distortion → height → angle → focus/DoF → steadiness → movement
|
||||
```
|
||||
|
||||
**Shorter prompts = more creative freedom. Longer prompts = more control.**
|
||||
|
||||
### Prompt Length by Model
|
||||
|
||||
Empirical sweet spots from the paper's Section 6 findings — different models reward different prompt densities:
|
||||
|
||||
| Model | Sweet Spot | Notes |
|
||||
|---|---|---|
|
||||
| Seedance 2.0 | 200–400 words for hero shots, 80–150 for inserts | Reward long, structured 5-aspect prompts |
|
||||
| Wan 2.2 | 200–400 words | Fine-tuned on long captions |
|
||||
| Sora 2 / VEO 3.1 | 100–250 words | Plateau past ~250 |
|
||||
| LTX-2 | ≤ 80 words | Degrades past that, keep tight |
|
||||
| Runway Gen-4 | ≤ 60 words | "Focus on motion, not appearance" |
|
||||
|
||||
### Overlays Are Not Scene Depth
|
||||
|
||||
> Overlays (titles, HUD, subtitles, watermarks, framing graphics) are NOT part of the scene's foreground/midground/background depth axis. List them separately with content and placement. Never say "overlay in the foreground."
|
||||
|
||||
---
|
||||
|
||||
## Camera Shot Types
|
||||
@@ -58,23 +90,56 @@ All video generation models respond to this structure. Include what's relevant,
|
||||
|
||||
## Camera Movements
|
||||
|
||||
| Movement | What It Does | Best For |
|
||||
|----------|-------------|----------|
|
||||
| **Static / fixed** | No movement | Dialogue, contemplation, stability |
|
||||
| **Pan** (left/right) | Rotates horizontally | Revealing a scene, following action |
|
||||
| **Tilt** (up/down) | Rotates vertically | Revealing height, slow reveal |
|
||||
| **Dolly in / out** | Physically moves toward/away | Building tension, emphasis |
|
||||
| **Truck** (left/right) | Moves sideways | Parallels subject movement |
|
||||
| **Pedestal** (up/down) | Moves vertically | Smooth elevation changes |
|
||||
| **Crane shot** | Sweeping vertical arcs | Epic reveals, transitions |
|
||||
| **Tracking / follow** | Follows subject | Action sequences, walk-and-talk |
|
||||
| **Arc shot** | Circles around subject | Dramatic emphasis, 360° reveal |
|
||||
| **Zoom** (in/out) | Lens focal length change | Quick emphasis (cheaper than dolly) |
|
||||
| **Whip pan** | Extremely fast pan (blurs) | Transitions, energy, surprise |
|
||||
| **Handheld / shaky cam** | Unstable, human feel | Documentary, urgency, realism |
|
||||
| **Aerial / drone** | High altitude, smooth | Landscapes, establishing shots |
|
||||
| **Slow push-in** | Gradual forward movement | Building intimacy or tension |
|
||||
| **Dolly zoom (vertigo)** | Dolly one way, zoom opposite | Disorientation, revelation |
|
||||
The paper shows current models confuse translation, rotation, and lens-only changes — group your prompts so the model can't conflate them:
|
||||
|
||||
| Group | Primitives | Rule |
|
||||
|---|---|---|
|
||||
| **Translation** (camera physically moves) | dolly in/out, truck left/right, pedestal up/down | "dolly forward toward subject" |
|
||||
| **Rotation** (camera pivots in place) | pan left/right, tilt up/down, roll CW/CCW | "pan right across the room" |
|
||||
| **Lens-only** (no camera move) | zoom in/out, rack focus, pull focus, focus tracking | "zoom in" ≠ "dolly in" |
|
||||
| **Hybrid / signature** | dolly zoom (vertigo), arc/orbit, crane, whip pan, tracking/follow, handheld | "vertigo" only at moments of revelation |
|
||||
| **Stillness states** | static (NO movement at all — strict), micro-shake, locked-off | "static" requires zero movement, focus change, or zoom |
|
||||
|
||||
> **dolly ≠ zoom.** dolly is camera translation; zoom is focal-length change. Models follow whichever token dominates. **pan ≠ truck.** pan rotates, truck translates laterally.
|
||||
|
||||
> **Static shot is strict.** A static shot has zero movement, zero focus change, zero zoom. If any of those occur, do NOT write "static camera" — pick the right movement primitive.
|
||||
|
||||
## Camera Height (relative to ground)
|
||||
|
||||
| Primitive | Example |
|
||||
|---|---|
|
||||
| Aerial-level | "drone-altitude wide of the city" |
|
||||
| Overhead-level | "rooftop height looking across the street" |
|
||||
| Eye-level | "framed at eye level" |
|
||||
| Hip-level | "hip-height tracking shot" |
|
||||
| Ground-level | "low to the ground, ankle height" |
|
||||
| Water-level | "skimming the water surface" |
|
||||
| Underwater | "submerged below the surface" |
|
||||
|
||||
## Camera Angle (relative to subject)
|
||||
|
||||
| Primitive | Definition |
|
||||
|---|---|
|
||||
| **Bird's-eye** | strict top-down. Not the same as aerial. |
|
||||
| High angle | looking down on subject |
|
||||
| Level angle | camera and subject at same height |
|
||||
| Low angle | looking up at subject |
|
||||
| Worm's-eye | looking straight up |
|
||||
| **Dutch angle (fixed)** | tilted horizon held steady |
|
||||
| **Dutch angle (rolling)** | horizon tilt changes during shot |
|
||||
|
||||
> **bird's-eye = strict top-down. aerial = altitude.** A drone shot at 45° looking down is a high angle from aerial height, NOT bird's-eye.
|
||||
|
||||
## Point of View (POV)
|
||||
|
||||
| POV | Example |
|
||||
|---|---|
|
||||
| First-person | "the camera follows the character's viewpoint as they walk" |
|
||||
| Drone | "aerial drone footage of city skyline" |
|
||||
| Over-the-shoulder | "OTS framing of the laptop screen" |
|
||||
| Top-down oblique | "top-down view of the chess board, tilted slightly" |
|
||||
| Dashcam | "vehicle dashcam framing of the road" |
|
||||
| Objective / Neutral | (default — use when no specific POV) |
|
||||
|
||||
## Lighting Vocabulary
|
||||
|
||||
@@ -100,14 +165,51 @@ All video generation models respond to this structure. Include what's relevant,
|
||||
|
||||
| Effect | Result |
|
||||
|--------|--------|
|
||||
| **Shallow depth of field** | Subject sharp, background bokeh |
|
||||
| **Deep focus** | Everything sharp, foreground to background |
|
||||
| **Wide-angle lens** (24-35mm) | Broader view, exaggerated perspective |
|
||||
| **Telephoto** (85mm+) | Compressed perspective, subject isolation |
|
||||
| **Anamorphic** | Stretched aspect, signature lens flares |
|
||||
| **Lens flare** | Streaks from bright light hitting lens |
|
||||
| **Rack focus** | Shift focus between subjects in-shot |
|
||||
| **Fisheye** | Ultra-wide, barrel distortion |
|
||||
|
||||
### Lens Distortion
|
||||
|
||||
The paper distinguishes two primitives that models honor as separate effects — they are NOT interchangeable:
|
||||
|
||||
| Primitive | Effect |
|
||||
|---|---|
|
||||
| **Fisheye** | extreme curvature, edges bent strongly outward |
|
||||
| **Barrel** | mild distortion, straight lines bow slightly outward |
|
||||
|
||||
### Focus / Depth of Field
|
||||
|
||||
| Primitive | Definition |
|
||||
|---|---|
|
||||
| Deep focus | everything sharp, FG to BG |
|
||||
| Shallow DoF | subject sharp, background bokeh |
|
||||
| Extremely shallow DoF | razor-thin focal plane |
|
||||
| Rack focus | shifts focus between two subjects mid-shot |
|
||||
| Pull focus | gradual focus shift (slower than rack) |
|
||||
| Focus tracking | focus follows a moving subject |
|
||||
|
||||
When DoF changes during a shot, label start AND end focal plane (FG/MG/BG/out-of-focus).
|
||||
|
||||
## Subject Transitions
|
||||
|
||||
When subjects enter, leave, or hand off focus, name the transition explicitly:
|
||||
|
||||
| Primitive | When |
|
||||
|---|---|
|
||||
| **Subject revealing** | a new subject enters frame (by subject movement OR camera movement) |
|
||||
| **Subject disappearing** | a subject exits frame |
|
||||
| **Subject switching** | focus shifts from one subject to another (often via rack focus or camera move) |
|
||||
| **Complex alternating** | subjects alternate focus multiple times |
|
||||
|
||||
Always name the cause: "by subject movement" or "by camera movement". This unlocks reveal-style camerawork in multi-shot prompts.
|
||||
|
||||
## Identity Anchoring for Multi-Shot Prompts
|
||||
|
||||
> Models lose character identity across cuts unless you re-state it. In every shot of a multi-shot prompt, repeat the same 3–6 disambiguating visual attributes for each named subject verbatim. Pronouns and "the same character" do not work.
|
||||
>
|
||||
> Example: "Aang — bald, blue arrow tattoo on forehead, orange-and-yellow robes — plants his staff. … Aang — bald, blue arrow tattoo on forehead, orange-and-yellow robes — turns to camera."
|
||||
|
||||
## Style & Aesthetic References
|
||||
|
||||
@@ -137,10 +239,23 @@ All video generation models respond to this structure. Include what's relevant,
|
||||
|
||||
## Temporal Effects
|
||||
|
||||
### Playback Speed
|
||||
|
||||
The paper defines six explicit playback-speed primitives. Use the right one — they're not synonymous:
|
||||
|
||||
| Primitive | Definition |
|
||||
|---|---|
|
||||
| Time-lapse | events significantly faster than real time (clouds racing) |
|
||||
| Fast-motion | slightly faster than real (1x–3x) |
|
||||
| Slow-motion | slower than real |
|
||||
| Stop-motion | frame-by-frame discrete movements |
|
||||
| Speed-ramp | mix of fast and slow within the same shot |
|
||||
| Time-reversed | plays in reverse |
|
||||
|
||||
### Other Temporal Devices
|
||||
|
||||
| Effect | Use |
|
||||
|--------|-----|
|
||||
| **Slow motion** | Emphasis, beauty, impact |
|
||||
| **Time-lapse** | Passage of time, processes |
|
||||
| **Freeze-frame** | Dramatic pause |
|
||||
| **Rapid cuts** | Energy, urgency |
|
||||
| **Continuous / long take** | Immersion, tension |
|
||||
@@ -160,6 +275,15 @@ Put dialogue in quotation marks: `Character says: "Hello world."`
|
||||
|
||||
## What to Avoid
|
||||
|
||||
> **Replace emotional adjectives with the visual cause of the emotion.**
|
||||
> - "sad character" → "tears on cheek, shoulders slumped, staring at empty chair"
|
||||
> - "cinematic mood" → "low-key Rembrandt key + 35mm anamorphic + crushed shadows, lifted-by-2-stops shadow detail"
|
||||
> - "epic" → "low-angle, 24mm wide, sun directly behind subject, lens flare on the rim"
|
||||
>
|
||||
> "Inspiring," "powerful," "moody," "epic" do not constrain pixels.
|
||||
|
||||
> **Static shot is strict.** A static shot has zero movement, zero focus change, zero zoom. If any of those occur, do NOT write "static camera" — pick the right movement primitive.
|
||||
|
||||
| Don't | Why | Do Instead |
|
||||
|-------|-----|-----------|
|
||||
| "Beautiful scene" | Too vague, no visual info | "Wet cobblestone street, warm streetlamp glow reflecting in puddles" |
|
||||
|
||||
+16
-3
@@ -6,6 +6,18 @@ After completing any pipeline stage's work — before checkpointing. You are the
|
||||
|
||||
Every stage gets reviewed. No exceptions. The review quality determines whether the final video is worth watching.
|
||||
|
||||
## Critique Quality (CHAI Rules)
|
||||
|
||||
> Findings ≠ critiques. A finding identifies a problem; a critique tells the next stage how to fix it. The CMU/Harvard CHAI study ("Building a Precise Video Language with Human-AI Oversight", arXiv 2604.21718v2) showed that critique quality, measured on three axes, directly governs downstream output quality. Apply all three to every reviewer pass.
|
||||
>
|
||||
> **Accurate.** Every finding must reference a concrete artifact field, line number, or visible asset frame. Forbid hallucinated criticism — if you cannot point to where the problem is, you are guessing.
|
||||
>
|
||||
> **Complete.** A reviewer pass that catches one mistake while missing a second is worse than scoring "needs another pass" and continuing. If you find one critical issue, scan for the rest of the same class before returning. Pattern-match: where else in this artifact could the same mistake be hiding?
|
||||
>
|
||||
> **Constructive.** Every "critical" finding MUST propose a concrete fix, not just identify the problem. "Caption is wrong" → "Caption says 'man on the right'; the man is on the left of the frame. Replace with 'the man on the left of the frame.'" If you cannot propose a fix, label the finding as "investigation" not "critical."
|
||||
>
|
||||
> Removing any of these three properties measurably hurts pipeline output. The reviewer is the choke point — be rigorous.
|
||||
|
||||
## Protocol
|
||||
|
||||
### Step 1: Load Review Context
|
||||
@@ -27,9 +39,10 @@ First, the non-negotiable check:
|
||||
For each `review_focus` item from the manifest:
|
||||
1. Evaluate the artifact against this specific criterion
|
||||
2. Assign a severity:
|
||||
- **critical** — Must fix before proceeding. The artifact is broken, incomplete, or dangerously wrong.
|
||||
- **suggestion** — Should fix. Improves quality significantly but doesn't block progress.
|
||||
- **nitpick** — Could fix. Minor polish that's nice-to-have.
|
||||
- **critical** — Must fix before proceeding. The artifact is broken, incomplete, or dangerously wrong. **Per CHAI rules, every critical finding MUST carry a `proposed_fix` (concrete replacement text, exact field value, or specific corrective action). A critical finding without a proposed fix is downgraded to `investigation`.**
|
||||
- **suggestion** — Should fix. Improves quality significantly but doesn't block progress. **Suggestions MUST carry a `proposed_change` describing how to improve.**
|
||||
- **nitpick** — Could fix. Minor polish that's nice-to-have. May stand alone without a proposed change.
|
||||
- **investigation** — A real concern but you cannot pinpoint the fix. Surface it for the next round; do not block on it.
|
||||
3. Write a specific, actionable finding (not vague)
|
||||
|
||||
**Good finding:** "Section 3 narration is 180 words for a 10-second window — that's 1080 wpm, impossible to speak. Cut to 25 words."
|
||||
|
||||
@@ -38,7 +38,8 @@ video_analyzer.execute({
|
||||
```
|
||||
|
||||
Read the resulting VideoAnalysisBrief. Before proceeding, present a summary to the
|
||||
user. This is NOT a raw dump. It's a conversational interpretation:
|
||||
user. This is NOT a raw dump. It's a conversational interpretation, and it MUST be
|
||||
structured by the 5 aspects so downstream stages can lift fields directly:
|
||||
|
||||
```
|
||||
"I've watched the video. Here's what I see:
|
||||
@@ -48,12 +49,22 @@ user. This is NOT a raw dump. It's a conversational interpretation:
|
||||
**Structure:** [X scenes over Y seconds, pacing style]
|
||||
**Motion:** [N of M scenes are motion clips / animated stills / static images.
|
||||
This video uses [AI-generated video clips / still images with pan-zoom / a mix].]
|
||||
|
||||
**5-aspect breakdown (per shot or per shot-group):**
|
||||
- Subject: [type, count, attributes; subject transitions across shots: revealing / disappearing / switching / complex-alternating; or N/A]
|
||||
- Subject Motion: [actions in temporal order; interactions; or N/A]
|
||||
- Scene: [overlays (text/graphics) listed separately; POV (drone/OTS/macro/etc.); setting; time of day; dynamics]
|
||||
- Spatial Framing: [shot size; subject position; depth; height-relative; how it changes]
|
||||
- Camera: [playback speed; lens; height; angle; focus/DoF; steadiness; movement]
|
||||
|
||||
**What makes it work:** [2-3 specific things — the hook technique, the pacing,
|
||||
the visual transitions, the narration style]
|
||||
|
||||
Now let me check what I can do with your current setup..."
|
||||
```
|
||||
|
||||
The 5-aspect block above is the **canonical form** that `proposal-director`, `script-director`, and `scene-director` will read. Do not collapse it back into prose — keep the labels.
|
||||
|
||||
**Motion classification is critical.** The VideoAnalysisBrief now includes per-scene
|
||||
`motion_type` ("motion_clip", "animated_still", "static_image") and `flow_variance`.
|
||||
Use this to determine the production approach:
|
||||
@@ -81,6 +92,22 @@ Update the brief's `content_analysis`, `style_profile`, and `replication_guidanc
|
||||
fields with your visual observations. This is where the analysis becomes truly
|
||||
comprehensive — the tools provide structure; your vision provides understanding.
|
||||
|
||||
### 5-Aspect Structured Output (MANDATORY)
|
||||
|
||||
The analyst's report MUST break down the reference video into the **five aspects** from the CMU/Harvard CHAI study (also the canonical structure used in `skills/creative/video-gen-prompting.md`). A narrative-only summary is no longer sufficient — downstream stages (proposal, script, scene-director) ingest the 5-aspect form directly without re-parsing prose.
|
||||
|
||||
**Decision-tree captioning policy.** For each detected shot, walk all five aspects in order:
|
||||
|
||||
> - **Subject:** type, attributes (count, age, role, costume, distinguishing features), multiple-subject disambiguation, transitions across shots (revealing / disappearing / switching / complex-alternating).
|
||||
> - **Subject Motion:** actions in temporal order; group/interaction patterns (parallel, sequential, reactive); locomotion vs gesture vs facial.
|
||||
> - **Scene:** **overlays separately** (text, lower thirds, graphics, watermark — call these out as their own layer, do not merge into setting) + POV (drone, aerial, OTS, macro, top-down, dashcam, FPV, handheld, locked-off) + setting + time of day + dynamics (weather, particles, crowd movement).
|
||||
> - **Spatial Framing:** shot size (ECU/CU/MS/WS/EWS), subject position in frame, depth (foreground/midground/background usage), height-relative (above/at/below subject) — and how each of these **changes** across the shot if the camera or subject moves.
|
||||
> - **Camera:** playback speed (real-time / slow-mo / time-lapse), lens distortion (anamorphic, fish-eye, tilt-shift), height (ground / eye / overhead), angle (high / low / Dutch), focus / DoF (rack focus, deep focus, shallow), steadiness (locked / handheld / gimbal), movement (push / pull / pan / tilt / dolly / truck / crane / orbit).
|
||||
>
|
||||
> **Mark any aspect explicitly as N/A** if it doesn't apply (e.g., "Subject: N/A — pure scenery shot," or "Scene overlays: N/A — no graphics"). **Silent omission is the most common analyst failure** and produces ambiguous downstream prompts.
|
||||
|
||||
See `skills/creative/video-gen-prompting.md` for primitive definitions and the canonical vocabulary used at every aspect.
|
||||
|
||||
### Step 2: Capability Audit
|
||||
|
||||
Run standard preflight:
|
||||
|
||||
@@ -85,9 +85,25 @@ Recommended metadata keys:
|
||||
- `tool_path_map`
|
||||
- `reusable_motifs`
|
||||
|
||||
### 5. Quality Gate
|
||||
### 5. 5-Aspect Scene-Plan Checklist
|
||||
|
||||
> Every scene must specify all five aspects, BUT the load shifts with the scene's `animation_mode`. Manim and other diagrammatic/programmatic scenes care most about **Subject** and **Spatial Framing** — Camera and Subject Motion in the cinematographic sense often map to N/A or to abstract equivalents. AI-video / `image_animation` / `anime_scene` scenes care about all five and behave like cinematic shots. Marking an aspect as N/A is allowed but must be explicit per scene; silent omission is forbidden.
|
||||
>
|
||||
> 1. **Subject** — type + key visual attributes; for Manim, the equation/object/graph being foregrounded; for `anime_scene`, the character or environment in focus.
|
||||
> 2. **Subject Motion** — for Manim, the order of `Create`/`Transform`/`FadeIn` and what each animation conveys; for AI-video, the actions and interactions in temporal order.
|
||||
> 3. **Scene** — overlays (separately!) + POV + setting + time of day + scene dynamics. For Manim, "setting" is the canvas background + axis style; for `anime_scene`, the environment + lighting gradient.
|
||||
> 4. **Spatial Framing** — shot size + position-in-frame + depth (FG/MG/BG) + camera-height-relative; and how those CHANGE. Manim cares about layout grid + element positions; AI-video cares about full cinematographic framing.
|
||||
> 5. **Camera** — playback speed → lens distortion → height → angle → focus/DoF → steadiness → movement. For Manim and pure motion-graphics, default to N/A unless using a virtual camera move (`MoveCamera`, `self.frame`). For `anime_scene` and AI-video, specify fully.
|
||||
>
|
||||
> Tie this back to `animation_mode` in scene metadata: a Manim scene that lists Camera fully is over-specified; an AI-video scene that omits Camera is under-specified. See `skills/creative/video-gen-prompting.md` for the primitive vocabulary.
|
||||
|
||||
> **Overlays callout.** Overlays (titles, subtitles, HUD, watermarks, framing graphics, lower-thirds, `hero_title`, `section_title`, `provider_chip`) are NOT part of the scene's foreground/midground/background depth axis. List them separately in scene metadata (`overlays: [...]`) with content and placement. Never describe an overlay as "in the foreground" — that confuses both downstream tools and any video-understanding model that re-analyzes the output.
|
||||
|
||||
### 6. Quality Gate
|
||||
|
||||
- every scene has a clear timing intent,
|
||||
- the 5-aspect checklist is satisfied for the scene's `animation_mode` (with explicit N/A where appropriate),
|
||||
- overlays live under `overlays:`, never inside the framing description,
|
||||
- the transition system is limited and meaningful,
|
||||
- the tool path is explicit,
|
||||
- the sequence feels like one designed system.
|
||||
|
||||
@@ -101,6 +101,22 @@ Recommended metadata keys:
|
||||
- `generated_support_assets`
|
||||
- `rights_notes`
|
||||
|
||||
### Pre/Post Self-Review for Generation Prompts
|
||||
|
||||
> Before sending a prompt to any image or video generation tool, run a three-step self-review modeled on the CHAI oversight loop ("Building a Precise Video Language with Human-AI Oversight", arXiv 2604.21718v2). Cost is small (no extra tool calls); benefit is large (avoids wasted generations). For cinematic, this matters most for **hero-frame prompts** — one bad hero frame ruins the piece, and hero frames are the most expensive shots to regenerate.
|
||||
>
|
||||
> **Step 1 — Pre-caption pass.** Write the prompt the way you'd write it today. Do not over-edit; aim for a complete first draft.
|
||||
>
|
||||
> **Step 2 — Critique pass.** Score the draft against the 5-aspect checklist (Subject / Subject Motion / Scene / Spatial Framing / Camera). For each aspect:
|
||||
> - Is it specified? If not, is the omission deliberate (e.g., "no subject — scenery shot") or accidental?
|
||||
> - Are confusable terms disambiguated? (dolly vs zoom, pan vs truck, bird's-eye vs aerial, fisheye vs barrel, full shot vs close-up)
|
||||
> - Are emotional adjectives ("epic", "moody", "cinematic") replaced with their visual causes (low-key lighting, slow push-in, anamorphic flare, deep shadows)?
|
||||
> - For multi-shot prompts and identity-anchored hero frames: is identity anchored verbatim across shots?
|
||||
>
|
||||
> **Step 3 — Post-caption pass.** Rewrite filling the missing aspects, fixing confusable terms, and replacing subjective language. The post-caption is what gets sent to the generation tool.
|
||||
>
|
||||
> Log the (pre, critique, post) triplet in the asset metadata for traceability. This mirrors the CHAI workflow and creates a record the reviewer can audit.
|
||||
|
||||
### 5. Quality Gate
|
||||
|
||||
- source and support assets are clearly distinguished,
|
||||
|
||||
@@ -49,11 +49,26 @@ Recommended metadata keys:
|
||||
- `title_card_rules`
|
||||
- `support_insert_rules`
|
||||
|
||||
### 5. Quality Gate
|
||||
### 5. 5-Aspect Scene-Plan Checklist
|
||||
|
||||
> Every scene beat — and especially every hero frame — must specify all five aspects. Cinematic relies on a small number of memorable frames; vague hero-frame specs are the single most common failure mode and produce unpredictable model output. Marking an aspect as N/A is allowed but must be explicit (e.g., "no subject — establishing scenery shot"). Silent omission is forbidden.
|
||||
>
|
||||
> 1. **Subject** — type + key visual attributes; if multiple, how to disambiguate. For hero frames, identity must be anchored verbatim across shots.
|
||||
> 2. **Subject Motion** — actions in temporal order; subject↔object / subject↔subject interactions.
|
||||
> 3. **Scene** — overlays (separately!) + POV + setting + time of day + scene dynamics.
|
||||
> 4. **Spatial Framing** — shot size + position-in-frame + depth (FG/MG/BG) + camera-height-relative; and how those CHANGE across the beat.
|
||||
> 5. **Camera** — playback speed → lens distortion → height → angle → focus/DoF → steadiness → movement.
|
||||
>
|
||||
> See `skills/creative/video-gen-prompting.md` for the primitive vocabulary.
|
||||
|
||||
> **Overlays callout.** Overlays (titles, subtitles, HUD, watermarks, framing graphics, lower-thirds, name plates, end-tag cards) are NOT part of the scene's foreground/midground/background depth axis. List them separately in scene metadata (`overlays: [...]`) with content and placement. Never describe an overlay as "in the foreground" — that confuses both downstream tools and any video-understanding model that re-analyzes the output.
|
||||
|
||||
### 6. Quality Gate
|
||||
|
||||
- every beat has a scene treatment,
|
||||
- hero frames are identifiable,
|
||||
- hero frames are identifiable AND fully specified across all 5 aspects,
|
||||
- support inserts are justified,
|
||||
- overlays are recorded under `overlays:`, never inside the depth/framing description,
|
||||
- the visual language stays consistent across the piece.
|
||||
|
||||
## Common Pitfalls
|
||||
|
||||
@@ -178,6 +178,22 @@ Assemble all generated assets into the manifest:
|
||||
}
|
||||
```
|
||||
|
||||
### Pre/Post Self-Review for Generation Prompts
|
||||
|
||||
> Before sending a prompt to any generation tool — `image_selector`, `diagram_gen`, `video_selector`, even `code_snippet` styling prompts — run a three-step self-review modeled on the CHAI oversight loop ("Building a Precise Video Language with Human-AI Oversight", arXiv 2604.21718v2). Cost is small (no extra tool calls); benefit is large (avoids wasted generations). This applies just as strongly to `diagram_gen` Mermaid prompts and `image_selector` illustration prompts as it does to video generation — bad explainer visuals fail in the same way: missing subject, missing framing, vague "make it look educational."
|
||||
>
|
||||
> **Step 1 — Pre-caption pass.** Write the prompt the way you'd write it today. Do not over-edit; aim for a complete first draft.
|
||||
>
|
||||
> **Step 2 — Critique pass.** Score the draft against the 5-aspect checklist (Subject / Subject Motion / Scene / Spatial Framing / Camera). For each aspect:
|
||||
> - Is it specified? If not, is the omission deliberate (e.g., "Camera N/A — Remotion native scene", "no subject motion — static diagram") or accidental?
|
||||
> - Are confusable terms disambiguated? (dolly vs zoom, pan vs truck, bird's-eye vs aerial, fisheye vs barrel, full shot vs close-up; for diagrams: flowchart vs sequence vs state diagram, top-down vs left-right)
|
||||
> - Are emotional adjectives ("clean", "professional", "modern") replaced with their visual causes (sans-serif typography, generous whitespace, monochromatic palette with one accent)?
|
||||
> - For multi-shot prompts: is identity anchored verbatim across shots? For `image_selector` prompts that recur (a character or world appearing in multiple scenes), are consistency anchors specified verbatim?
|
||||
>
|
||||
> **Step 3 — Post-caption pass.** Rewrite filling the missing aspects, fixing confusable terms, and replacing subjective language. The post-caption is what gets sent to the generation tool.
|
||||
>
|
||||
> Log the (pre, critique, post) triplet in the asset metadata for traceability. This mirrors the CHAI workflow and creates a record the reviewer can audit.
|
||||
|
||||
### Step 7: Verify All Assets
|
||||
|
||||
**Existence check:**
|
||||
|
||||
@@ -156,6 +156,20 @@ If the video includes narration, the script **must** be written to fit the video
|
||||
- If narration exceeds video by >1s, either trim the script and regenerate, or extend the video's closing scene.
|
||||
- Always run `composition_validator` before rendering to catch mismatches automatically.
|
||||
|
||||
### Step 4c: 5-Aspect Scene-Plan Checklist
|
||||
|
||||
> Every scene must specify all five aspects. For diagram, chart, and Remotion-native scenes, "Subject" can map to a foregrounded data element and "Camera" can be marked N/A — but only EXPLICITLY (e.g., `"camera": "N/A — Remotion native scene, no virtual camera"`). Silent omission is the most common failure mode and produces unpredictable model output, brittle prompts, and reviewer churn.
|
||||
>
|
||||
> 1. **Subject** — type + key visual attributes; if multiple, how to disambiguate. For diagram/chart scenes, this is the foregrounded data element (the node, the bar, the KPI being highlighted). For generated images, it's the person/object/concept being illustrated.
|
||||
> 2. **Subject Motion** — actions in temporal order; for animated diagrams, the order in which nodes/edges/values appear or change.
|
||||
> 3. **Scene** — overlays (separately!) + POV + setting + time of day + scene dynamics. For Remotion scenes, "setting" maps to background treatment + theme.
|
||||
> 4. **Spatial Framing** — shot size + position-in-frame + depth (FG/MG/BG) + camera-height-relative; and how those CHANGE. For static Remotion scenes, document the layout grid + which element occupies the visual center.
|
||||
> 5. **Camera** — playback speed → lens distortion → height → angle → focus/DoF → steadiness → movement. Mark N/A for native-Remotion scenes; specify fully for `generated`/`broll`/`image_animation` scenes.
|
||||
>
|
||||
> See `skills/creative/video-gen-prompting.md` for the primitive vocabulary.
|
||||
|
||||
> **Overlays callout.** Overlays (titles, subtitles, HUD, watermarks, framing graphics, lower-thirds, section_title bars, stat_reveal chips, hero_title overlays, provider chips) are NOT part of the scene's foreground/midground/background depth axis. List them separately in scene metadata (`overlays: [...]`) with content and placement. Never describe an overlay as "in the foreground" — that confuses both downstream tools and any video-understanding model that re-analyzes the output.
|
||||
|
||||
### Step 5: Validate Against Playbook
|
||||
|
||||
The style playbook constrains your visual choices:
|
||||
|
||||
Reference in New Issue
Block a user