Upgrade Remotion composition engine with cinematic enhancements
- Upgrade Remotion packages to 4.0.441, add transitions/captions/fonts/media - Add spring-animated image scenes, stat reveals, section titles, hero cards - Add TikTok-style word-by-word captions with highlight effect - Add Google Fonts (Space Grotesk) and dynamic duration via calculateMetadata - Fix Remotion false-positive: check node_modules/ in _remotion_available() - Add project directory convention, music library, tool naming docs - Add music transparency to proposal stage, subtitle pipeline to compose stage - Add README showcase montage and Windows npm troubleshooting note
This commit is contained in:
calesthio
+10
@@ -21,6 +21,12 @@ output/
|
||||
pipeline/
|
||||
tests/qa/output/
|
||||
|
||||
# Project workspaces (generated assets, renders — all regenerable)
|
||||
projects/
|
||||
|
||||
# User music library (personal royalty-free tracks — not part of repo)
|
||||
music_library/
|
||||
|
||||
# Environment
|
||||
.env
|
||||
.env.local
|
||||
@@ -42,3 +48,7 @@ Thumbs.db
|
||||
|
||||
# Node (for remotion-composer/)
|
||||
node_modules/
|
||||
|
||||
# Remotion build output and copied project assets (regenerable at render time)
|
||||
remotion-composer/out/
|
||||
remotion-composer/public/
|
||||
|
||||
@@ -46,6 +46,40 @@ Infrastructure files:
|
||||
- `tools/cost_tracker.py` — budget governance
|
||||
- `lib/pipeline_loader.py` — manifest loading and helpers
|
||||
|
||||
## Project Directory Convention
|
||||
|
||||
Every production run creates a project workspace under `projects/`. This directory is gitignored — all generated assets are regenerable.
|
||||
|
||||
```
|
||||
projects/<project-name>/
|
||||
├── artifacts/ # JSON artifacts from each stage (research_brief, script, scene_plan, etc.)
|
||||
├── assets/
|
||||
│ ├── images/ # Generated images (PNG)
|
||||
│ ├── video/ # Generated video clips (MP4)
|
||||
│ ├── audio/ # Narration segments + final mix (MP3/WAV)
|
||||
│ ├── music/ # Background music track (MP3)
|
||||
│ └── subtitles.srt # Generated subtitles
|
||||
└── renders/
|
||||
└── final.mp4 # Final rendered video (the deliverable)
|
||||
```
|
||||
|
||||
**Naming convention**: Use kebab-case derived from the video title (e.g., `hidden-math-of-nature`, `how-music-rewires-brain`).
|
||||
|
||||
Create the project directory at pipeline initialization, before any stage runs. All tools and agents should write outputs to these paths — never to the repo root or ad-hoc locations.
|
||||
|
||||
## Music Library
|
||||
|
||||
Users can place royalty-free music tracks in `music_library/` (gitignored). The asset director will check this folder before falling back to API-based music generation.
|
||||
|
||||
```
|
||||
music_library/
|
||||
├── ambient_track.mp3
|
||||
├── cinematic_epic.mp3
|
||||
└── ...
|
||||
```
|
||||
|
||||
If the folder has tracks, the proposal and asset stages should present them as options alongside generated music. See the proposal-director and asset-director skills for details.
|
||||
|
||||
## Available Pipelines
|
||||
|
||||
| Pipeline | Best For | Stability |
|
||||
@@ -237,6 +271,23 @@ Key capability families to look for in the output:
|
||||
|
||||
Each tool in the registry declares `best_for`, `install_instructions`, `runtime` (LOCAL, API, LOCAL_GPU, HYBRID), and `status`. Read these fields — do not assume tool strengths from memory.
|
||||
|
||||
### Tool Class Naming Convention
|
||||
|
||||
All tool classes use **PascalCase without a "Tool" suffix**. When importing tools in Python:
|
||||
|
||||
| Module | Class Name | NOT |
|
||||
|--------|-----------|-----|
|
||||
| `tools.audio.music_gen` | `MusicGen` | ~~MusicGenTool~~ |
|
||||
| `tools.video.video_compose` | `VideoCompose` | ~~VideoComposeTool~~ |
|
||||
| `tools.audio.audio_mixer` | `AudioMixer` | ~~AudioMixerTool~~ |
|
||||
| `tools.tts.elevenlabs_tts` | `ElevenLabsTTS` | ~~ElevenLabsTTSTool~~ |
|
||||
| `tools.analysis.transcriber` | `Transcriber` | ~~TranscriberTool~~ |
|
||||
| `tools.subtitle.subtitle_gen` | `SubtitleGen` | ~~SubtitleGenTool~~ |
|
||||
|
||||
When in doubt, check: `grep "^class " tools/<path>.py`
|
||||
|
||||
All tools call via `.execute(params_dict)` (returns `ToolResult` with `.success`, `.data`, `.error`), NOT `.run()`.
|
||||
|
||||
### Selector Pattern
|
||||
|
||||
Three selector tools abstract multi-provider capabilities. **Selectors auto-discover providers from the registry.** Adding a new provider tool automatically makes it available through the selector — no selector code changes needed.
|
||||
|
||||
@@ -12,6 +12,14 @@ No video editing experience required. No complex UI to learn. Just describe your
|
||||
|
||||
Works with **Claude Code, Cursor, Copilot, Windsurf, Codex** — any AI coding assistant that can read files and run code.
|
||||
|
||||
<p align="center">
|
||||
<img src="assets/showcase.jpg" alt="Nature's Source Code — an AI-produced explainer video" width="100%">
|
||||
</p>
|
||||
|
||||
> **"Nature's Source Code"** — a cinematic explainer produced from a single prompt. The agent researched the topic across dozens of sources, wrote the script, generated every visual, composed narration and music, added word-by-word captions, and rendered the final video. Total human input: one sentence and a few approvals.
|
||||
>
|
||||
> OpenMontage produces **10+ minute videos** with minimal interaction — describe your vision, approve the creative direction, and get a finished production.
|
||||
|
||||
---
|
||||
|
||||
## Why OpenMontage?
|
||||
@@ -112,7 +120,7 @@ Edit your own talking-head footage. Generate a fully animated explainer from scr
|
||||
### Composition & Rendering
|
||||
| Engine | Type | What It Does |
|
||||
|--------|------|-------------|
|
||||
| **Remotion** | Local (Node.js) | React-based programmatic video — animated text cards, stat cards, charts, spring physics transitions, image-to-video with motion. **When no video generation providers are configured, the agent generates still images and Remotion turns them into fully animated video.** |
|
||||
| **Remotion** | Local (Node.js) | React-based programmatic video — spring-animated image scenes, stat reveals, section titles, hero cards, TikTok-style word-by-word captions, scene transitions (fade/slide/wipe/flip), Google Fonts, and audio with fade curves. **When no video generation providers are configured, the agent generates still images and Remotion turns them into fully animated video.** |
|
||||
| **FFmpeg** | Local | Core video assembly, encoding, subtitle burn, audio muxing, color grading |
|
||||
|
||||
---
|
||||
@@ -207,6 +215,8 @@ That's it. `make setup` installs Python dependencies, the Remotion composer, fre
|
||||
> pip install piper-tts
|
||||
> cp .env.example .env
|
||||
> ```
|
||||
>
|
||||
> **Windows users:** If `npm install` fails with `ERR_INVALID_ARG_TYPE`, use `npx --yes npm install` instead.
|
||||
|
||||
### 3. Add API Keys (optional)
|
||||
|
||||
|
||||
Binary file not shown.
|
After Width: | Height: | Size: 386 KiB |
Generated
+2942
File diff suppressed because it is too large
Load Diff
@@ -8,11 +8,15 @@
|
||||
"upgrade": "npx remotion upgrade"
|
||||
},
|
||||
"dependencies": {
|
||||
"@remotion/cli": "4.0.0",
|
||||
"@remotion/player": "4.0.0",
|
||||
"@remotion/captions": "^4.0.441",
|
||||
"@remotion/cli": "^4.0.441",
|
||||
"@remotion/google-fonts": "^4.0.441",
|
||||
"@remotion/media": "^4.0.441",
|
||||
"@remotion/player": "^4.0.441",
|
||||
"@remotion/transitions": "^4.0.441",
|
||||
"react": "^18.2.0",
|
||||
"react-dom": "^18.2.0",
|
||||
"remotion": "4.0.0"
|
||||
"remotion": "^4.0.441"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/react": "^18.2.0",
|
||||
|
||||
+208
-100
@@ -6,13 +6,35 @@ import {
|
||||
Sequence,
|
||||
interpolate,
|
||||
spring,
|
||||
staticFile,
|
||||
useCurrentFrame,
|
||||
useVideoConfig,
|
||||
} from "remotion";
|
||||
import { loadFont } from "@remotion/google-fonts/SpaceGrotesk";
|
||||
|
||||
// Resolve asset path — use staticFile() for local paths, passthrough URLs
|
||||
function resolveAsset(src: string): string {
|
||||
if (src.startsWith("http://") || src.startsWith("https://") || src.startsWith("data:")) {
|
||||
return src;
|
||||
}
|
||||
// Strip any file:// prefix
|
||||
const clean = src.replace(/^file:\/\/\/?/, "");
|
||||
return staticFile(clean);
|
||||
}
|
||||
import { TextCard } from "./components/TextCard";
|
||||
import { StatCard } from "./components/StatCard";
|
||||
import { CalloutBox } from "./components/CalloutBox";
|
||||
import { ComparisonCard } from "./components/ComparisonCard";
|
||||
import { CaptionOverlay, WordCaption } from "./components/CaptionOverlay";
|
||||
import { SectionTitle } from "./components/SectionTitle";
|
||||
import { StatReveal } from "./components/StatReveal";
|
||||
import { HeroTitle } from "./components/HeroTitle";
|
||||
|
||||
// Load Space Grotesk font for cinematic typography
|
||||
const { fontFamily } = loadFont("normal", {
|
||||
weights: ["400", "700"],
|
||||
subsets: ["latin"],
|
||||
});
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Types — aligned with edit_decisions artifact schema
|
||||
@@ -43,35 +65,42 @@ interface Cut {
|
||||
transform?: {
|
||||
animation?: string;
|
||||
scale?: number;
|
||||
position?: { x: number; y: number };
|
||||
position?: string | { x: number; y: number };
|
||||
};
|
||||
}
|
||||
|
||||
interface Overlay {
|
||||
type: "section_title" | "stat_reveal" | "hero_title";
|
||||
in_seconds: number;
|
||||
out_seconds: number;
|
||||
text: string;
|
||||
subtitle?: string;
|
||||
accentColor?: string;
|
||||
position?: string;
|
||||
}
|
||||
|
||||
interface AudioLayer {
|
||||
src: string;
|
||||
volume?: number;
|
||||
}
|
||||
|
||||
interface SfxCue {
|
||||
src: string;
|
||||
start_seconds: number;
|
||||
volume?: number;
|
||||
}
|
||||
|
||||
interface AudioConfig {
|
||||
narration?: AudioLayer;
|
||||
music?: AudioLayer;
|
||||
sfx?: SfxCue[];
|
||||
music?: AudioLayer & {
|
||||
fadeInSeconds?: number;
|
||||
fadeOutSeconds?: number;
|
||||
};
|
||||
}
|
||||
|
||||
interface ExplainerProps {
|
||||
export interface ExplainerProps {
|
||||
cuts: Cut[];
|
||||
subtitles?: { enabled: boolean; src?: string };
|
||||
overlays?: Overlay[];
|
||||
captions?: WordCaption[];
|
||||
audio?: AudioConfig;
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Image scene — spring / interpolate animations (replaces FFmpeg Ken Burns)
|
||||
// Image Extensions
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const IMAGE_EXTENSIONS = [".png", ".jpg", ".jpeg", ".bmp", ".tiff", ".tif", ".webp"];
|
||||
@@ -87,6 +116,24 @@ function isVideo(source: string): boolean {
|
||||
return VIDEO_EXTENSIONS.some((ext) => lower.endsWith(ext));
|
||||
}
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Cinematic vignette overlay
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const Vignette: React.FC = () => (
|
||||
<AbsoluteFill
|
||||
style={{
|
||||
background:
|
||||
"radial-gradient(ellipse at center, transparent 50%, rgba(0,0,0,0.6) 100%)",
|
||||
pointerEvents: "none",
|
||||
}}
|
||||
/>
|
||||
);
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Enhanced Image Scene — spring physics, parallax, variety
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const ImageScene: React.FC<{ src: string; animation?: string }> = ({
|
||||
src,
|
||||
animation,
|
||||
@@ -94,102 +141,101 @@ const ImageScene: React.FC<{ src: string; animation?: string }> = ({
|
||||
const frame = useCurrentFrame();
|
||||
const { fps, durationInFrames } = useVideoConfig();
|
||||
|
||||
// Smooth fade-in on entrance
|
||||
const fadeIn = spring({ frame, fps, config: { damping: 20 } });
|
||||
// Smooth spring fade-in
|
||||
const fadeIn = spring({ frame, fps, config: { damping: 18, stiffness: 80 } });
|
||||
|
||||
// Fade-out for crossfade effect
|
||||
const fadeOutStart = durationInFrames - 8;
|
||||
const fadeOut = interpolate(frame, [fadeOutStart, durationInFrames], [1, 0.3], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
|
||||
let scale = 1;
|
||||
let translateX = 0;
|
||||
let translateY = 0;
|
||||
const anim = animation || "zoom-in";
|
||||
|
||||
// Progress with easing — smoother than linear
|
||||
const progress = interpolate(frame, [0, durationInFrames], [0, 1], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
|
||||
if (anim === "zoom-in") {
|
||||
scale = interpolate(frame, [0, durationInFrames], [1, 1.15], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
scale = 1 + progress * 0.18;
|
||||
} else if (anim === "zoom-out") {
|
||||
scale = interpolate(frame, [0, durationInFrames], [1.15, 1], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
scale = 1.18 - progress * 0.18;
|
||||
} else if (anim === "pan-left") {
|
||||
translateX = interpolate(frame, [0, durationInFrames], [30, -30], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
translateX = interpolate(progress, [0, 1], [40, -40]);
|
||||
scale = 1.15;
|
||||
} else if (anim === "pan-right") {
|
||||
translateX = interpolate(frame, [0, durationInFrames], [-30, 30], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
translateX = interpolate(progress, [0, 1], [-40, 40]);
|
||||
scale = 1.15;
|
||||
} else if (anim === "ken-burns") {
|
||||
scale = interpolate(frame, [0, durationInFrames], [1, 1.2], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
translateX = interpolate(frame, [0, durationInFrames], [0, -20], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
translateY = interpolate(frame, [0, durationInFrames], [0, -10], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
} else if (anim === "ken-burns" || anim === "ken-burns-slow-zoom") {
|
||||
// Cinematic Ken Burns: gentle zoom + diagonal drift
|
||||
scale = 1 + progress * 0.22;
|
||||
translateX = interpolate(progress, [0, 1], [0, -25]);
|
||||
translateY = interpolate(progress, [0, 1], [0, -15]);
|
||||
} else if (anim === "parallax") {
|
||||
// Subtle parallax — foreground moves faster
|
||||
translateY = interpolate(progress, [0, 1], [15, -15]);
|
||||
scale = 1.1;
|
||||
}
|
||||
// "static" or "none" → no motion, just fade in
|
||||
// "static" or "none" → just display
|
||||
|
||||
return (
|
||||
<AbsoluteFill style={{ overflow: "hidden", backgroundColor: "#000" }}>
|
||||
<AbsoluteFill style={{ overflow: "hidden", backgroundColor: "#0F172A" }}>
|
||||
<Img
|
||||
src={src}
|
||||
src={resolveAsset(src)}
|
||||
style={{
|
||||
width: "100%",
|
||||
height: "100%",
|
||||
objectFit: "cover",
|
||||
opacity: fadeIn,
|
||||
opacity: fadeIn * fadeOut,
|
||||
transform: `scale(${scale}) translate(${translateX}px, ${translateY}px)`,
|
||||
willChange: "transform, opacity",
|
||||
}}
|
||||
/>
|
||||
<Vignette />
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Video scene — OffthreadVideo for frame-accurate rendering
|
||||
// Enhanced Video Scene
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const VideoScene: React.FC<{ src: string; startFrom?: number }> = ({
|
||||
src,
|
||||
startFrom = 0,
|
||||
}) => {
|
||||
const { fps } = useVideoConfig();
|
||||
return (
|
||||
<AbsoluteFill>
|
||||
<OffthreadVideo
|
||||
src={src}
|
||||
startFrom={Math.round(startFrom * fps)}
|
||||
style={{ width: "100%", height: "100%", objectFit: "cover" }}
|
||||
/>
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Fade transition wrapper
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const FadeIn: React.FC<{
|
||||
children: React.ReactNode;
|
||||
durationFrames?: number;
|
||||
}> = ({ children, durationFrames = 10 }) => {
|
||||
const frame = useCurrentFrame();
|
||||
const opacity = interpolate(frame, [0, durationFrames], [0, 1], {
|
||||
const { fps, durationInFrames } = useVideoConfig();
|
||||
|
||||
const fadeIn = spring({ frame, fps, config: { damping: 20 } });
|
||||
const fadeOutStart = durationInFrames - 8;
|
||||
const fadeOut = interpolate(frame, [fadeOutStart, durationInFrames], [1, 0.3], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
return <AbsoluteFill style={{ opacity }}>{children}</AbsoluteFill>;
|
||||
|
||||
return (
|
||||
<AbsoluteFill style={{ backgroundColor: "#0F172A" }}>
|
||||
<OffthreadVideo
|
||||
src={resolveAsset(src)}
|
||||
startFrom={Math.round(startFrom * fps)}
|
||||
style={{
|
||||
width: "100%",
|
||||
height: "100%",
|
||||
objectFit: "cover",
|
||||
opacity: fadeIn * fadeOut,
|
||||
}}
|
||||
muted
|
||||
/>
|
||||
<Vignette />
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
@@ -227,7 +273,6 @@ const SceneRenderer: React.FC<{ cut: Cut }> = ({ cut }) => {
|
||||
);
|
||||
}
|
||||
|
||||
// Auto-detect from source file extension
|
||||
const animation = cut.animation || cut.transform?.animation;
|
||||
|
||||
if (isImage(cut.source)) {
|
||||
@@ -238,59 +283,122 @@ const SceneRenderer: React.FC<{ cut: Cut }> = ({ cut }) => {
|
||||
return <VideoScene src={cut.source} startFrom={cut.in_seconds} />;
|
||||
}
|
||||
|
||||
// Fallback: treat as image
|
||||
return <ImageScene src={cut.source} animation={animation} />;
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Overlay renderer
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
const OverlayRenderer: React.FC<{ overlay: Overlay }> = ({ overlay }) => {
|
||||
if (overlay.type === "section_title") {
|
||||
return (
|
||||
<SectionTitle
|
||||
title={overlay.text}
|
||||
subtitle={overlay.subtitle}
|
||||
accentColor={overlay.accentColor}
|
||||
position={(overlay.position as any) || "top-left"}
|
||||
/>
|
||||
);
|
||||
}
|
||||
if (overlay.type === "stat_reveal") {
|
||||
return (
|
||||
<StatReveal
|
||||
stat={overlay.text}
|
||||
label={overlay.subtitle}
|
||||
accentColor={overlay.accentColor}
|
||||
position={(overlay.position as any) || "bottom-right"}
|
||||
/>
|
||||
);
|
||||
}
|
||||
if (overlay.type === "hero_title") {
|
||||
return <HeroTitle title={overlay.text} subtitle={overlay.subtitle} />;
|
||||
}
|
||||
return null;
|
||||
};
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Main composition
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const Explainer: React.FC<ExplainerProps> = ({ cuts, audio }) => {
|
||||
const { fps } = useVideoConfig();
|
||||
export const Explainer: React.FC<ExplainerProps> = ({
|
||||
cuts,
|
||||
overlays,
|
||||
captions,
|
||||
audio,
|
||||
}) => {
|
||||
const { fps, durationInFrames } = useVideoConfig();
|
||||
|
||||
return (
|
||||
<AbsoluteFill style={{ backgroundColor: "#000" }}>
|
||||
{/* Visual layers */}
|
||||
<AbsoluteFill style={{ backgroundColor: "#0F172A", fontFamily }}>
|
||||
{/* Layer 1: Visual scenes */}
|
||||
{cuts.map((cut) => {
|
||||
const from = Math.round(cut.in_seconds * fps);
|
||||
const duration = Math.round(
|
||||
(cut.out_seconds - cut.in_seconds) * fps
|
||||
);
|
||||
|
||||
const scene = <SceneRenderer cut={cut} />;
|
||||
const wrapped =
|
||||
cut.transition_in === "fade" ? (
|
||||
<FadeIn>{scene}</FadeIn>
|
||||
) : (
|
||||
scene
|
||||
);
|
||||
const duration = Math.round((cut.out_seconds - cut.in_seconds) * fps);
|
||||
|
||||
return (
|
||||
<Sequence key={cut.id} from={from} durationInFrames={duration}>
|
||||
<AbsoluteFill>{wrapped}</AbsoluteFill>
|
||||
<SceneRenderer cut={cut} />
|
||||
</Sequence>
|
||||
);
|
||||
})}
|
||||
|
||||
{/* Audio layers */}
|
||||
{/* Layer 2: Overlays (section titles, stat reveals, hero titles) */}
|
||||
{overlays?.map((overlay, i) => {
|
||||
const from = Math.round(overlay.in_seconds * fps);
|
||||
const duration = Math.round(
|
||||
(overlay.out_seconds - overlay.in_seconds) * fps
|
||||
);
|
||||
|
||||
return (
|
||||
<Sequence key={`overlay-${i}`} from={from} durationInFrames={duration}>
|
||||
<OverlayRenderer overlay={overlay} />
|
||||
</Sequence>
|
||||
);
|
||||
})}
|
||||
|
||||
{/* Layer 3: Captions (word-by-word highlight) */}
|
||||
{captions && captions.length > 0 && (
|
||||
<CaptionOverlay
|
||||
words={captions}
|
||||
wordsPerPage={6}
|
||||
fontSize={42}
|
||||
highlightColor="#22D3EE"
|
||||
backgroundColor="rgba(15, 23, 42, 0.7)"
|
||||
/>
|
||||
)}
|
||||
|
||||
{/* Layer 4: Audio — narration */}
|
||||
{audio?.narration?.src && (
|
||||
<Audio src={audio.narration.src} volume={audio.narration.volume ?? 1} />
|
||||
<Audio src={resolveAsset(audio.narration.src)} volume={audio.narration.volume ?? 1} />
|
||||
)}
|
||||
|
||||
{/* Layer 4: Audio — music with fade in/out */}
|
||||
{audio?.music?.src && (
|
||||
<Audio
|
||||
src={audio.music.src}
|
||||
volume={audio.music.volume ?? 0.06}
|
||||
src={resolveAsset(audio.music.src)}
|
||||
volume={(f) => {
|
||||
const baseVol = audio.music!.volume ?? 0.1;
|
||||
const fadeInDur = (audio.music!.fadeInSeconds ?? 2) * fps;
|
||||
const fadeOutDur = (audio.music!.fadeOutSeconds ?? 3) * fps;
|
||||
const totalFrames = durationInFrames;
|
||||
|
||||
// Fade in
|
||||
const fadeIn = interpolate(f, [0, fadeInDur], [0, baseVol], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
// Fade out
|
||||
const fadeOut = interpolate(
|
||||
f,
|
||||
[totalFrames - fadeOutDur, totalFrames],
|
||||
[baseVol, 0],
|
||||
{ extrapolateLeft: "clamp", extrapolateRight: "clamp" }
|
||||
);
|
||||
return Math.min(fadeIn, fadeOut);
|
||||
}}
|
||||
/>
|
||||
)}
|
||||
{audio?.sfx?.map((cue, i) => (
|
||||
<Sequence
|
||||
key={`sfx-${i}`}
|
||||
from={Math.round(cue.start_seconds * fps)}
|
||||
>
|
||||
<Audio src={cue.src} volume={cue.volume ?? 0.5} />
|
||||
</Sequence>
|
||||
))}
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
|
||||
@@ -1,5 +1,17 @@
|
||||
import { Composition } from "remotion";
|
||||
import { Explainer } from "./Explainer";
|
||||
import { Composition, CalculateMetadataFunction } from "remotion";
|
||||
import { Explainer, ExplainerProps } from "./Explainer";
|
||||
|
||||
const calculateMetadata: CalculateMetadataFunction<ExplainerProps> = async ({
|
||||
props,
|
||||
}) => {
|
||||
const cuts = props.cuts || [];
|
||||
if (cuts.length === 0) {
|
||||
return { durationInFrames: 30 * 60 };
|
||||
}
|
||||
const lastEnd = Math.max(...cuts.map((c) => c.out_seconds || 0));
|
||||
// Add 1 second padding for final fade
|
||||
return { durationInFrames: Math.ceil((lastEnd + 1) * 30) };
|
||||
};
|
||||
|
||||
export const Root: React.FC = () => {
|
||||
return (
|
||||
@@ -7,15 +19,17 @@ export const Root: React.FC = () => {
|
||||
<Composition
|
||||
id="Explainer"
|
||||
component={Explainer}
|
||||
durationInFrames={30 * 60} // default 60s at 30fps, overridden by props
|
||||
durationInFrames={30 * 60}
|
||||
fps={30}
|
||||
width={1920}
|
||||
height={1080}
|
||||
defaultProps={{
|
||||
cuts: [],
|
||||
subtitles: { enabled: false },
|
||||
overlays: [],
|
||||
captions: [],
|
||||
audio: {},
|
||||
}}
|
||||
calculateMetadata={calculateMetadata}
|
||||
/>
|
||||
</>
|
||||
);
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
import {
|
||||
AbsoluteFill,
|
||||
Sequence,
|
||||
interpolate,
|
||||
spring,
|
||||
useCurrentFrame,
|
||||
useVideoConfig,
|
||||
} from "remotion";
|
||||
|
||||
// Word-level caption for TikTok-style highlight display
|
||||
export interface WordCaption {
|
||||
word: string;
|
||||
startMs: number;
|
||||
endMs: number;
|
||||
}
|
||||
|
||||
interface CaptionOverlayProps {
|
||||
words: WordCaption[];
|
||||
// How many words to show at once in a "page"
|
||||
wordsPerPage?: number;
|
||||
fontSize?: number;
|
||||
color?: string;
|
||||
highlightColor?: string;
|
||||
backgroundColor?: string;
|
||||
fontFamily?: string;
|
||||
}
|
||||
|
||||
interface CaptionPage {
|
||||
words: WordCaption[];
|
||||
startMs: number;
|
||||
endMs: number;
|
||||
}
|
||||
|
||||
function buildPages(words: WordCaption[], wordsPerPage: number): CaptionPage[] {
|
||||
const pages: CaptionPage[] = [];
|
||||
for (let i = 0; i < words.length; i += wordsPerPage) {
|
||||
const pageWords = words.slice(i, i + wordsPerPage);
|
||||
if (pageWords.length === 0) continue;
|
||||
pages.push({
|
||||
words: pageWords,
|
||||
startMs: pageWords[0].startMs,
|
||||
endMs: pageWords[pageWords.length - 1].endMs,
|
||||
});
|
||||
}
|
||||
return pages;
|
||||
}
|
||||
|
||||
const PageRenderer: React.FC<{
|
||||
page: CaptionPage;
|
||||
fontSize: number;
|
||||
color: string;
|
||||
highlightColor: string;
|
||||
backgroundColor: string;
|
||||
fontFamily: string;
|
||||
}> = ({ page, fontSize, color, highlightColor, backgroundColor, fontFamily }) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps } = useVideoConfig();
|
||||
|
||||
const currentMs = page.startMs + (frame / fps) * 1000;
|
||||
|
||||
// Spring entrance
|
||||
const entrance = spring({
|
||||
frame,
|
||||
fps,
|
||||
config: { damping: 18, stiffness: 120 },
|
||||
});
|
||||
|
||||
return (
|
||||
<AbsoluteFill
|
||||
style={{
|
||||
justifyContent: "flex-end",
|
||||
alignItems: "center",
|
||||
paddingBottom: 80,
|
||||
}}
|
||||
>
|
||||
<div
|
||||
style={{
|
||||
opacity: entrance,
|
||||
transform: `translateY(${interpolate(entrance, [0, 1], [20, 0])}px)`,
|
||||
backgroundColor,
|
||||
borderRadius: 12,
|
||||
padding: "14px 28px",
|
||||
maxWidth: "80%",
|
||||
textAlign: "center",
|
||||
}}
|
||||
>
|
||||
<span
|
||||
style={{
|
||||
fontSize,
|
||||
fontWeight: 700,
|
||||
fontFamily,
|
||||
lineHeight: 1.4,
|
||||
whiteSpace: "pre-wrap",
|
||||
}}
|
||||
>
|
||||
{page.words.map((w, i) => {
|
||||
const isActive = w.startMs <= currentMs && w.endMs > currentMs;
|
||||
const isPast = w.endMs <= currentMs;
|
||||
return (
|
||||
<span
|
||||
key={`${w.startMs}-${i}`}
|
||||
style={{
|
||||
color: isActive ? highlightColor : isPast ? color : `${color}99`,
|
||||
transition: "none", // CSS transitions forbidden in Remotion
|
||||
textShadow: isActive
|
||||
? `0 0 20px ${highlightColor}66, 0 2px 4px rgba(0,0,0,0.5)`
|
||||
: "0 2px 4px rgba(0,0,0,0.5)",
|
||||
}}
|
||||
>
|
||||
{w.word}
|
||||
</span>
|
||||
);
|
||||
})}
|
||||
</span>
|
||||
</div>
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
|
||||
export const CaptionOverlay: React.FC<CaptionOverlayProps> = ({
|
||||
words,
|
||||
wordsPerPage = 6,
|
||||
fontSize = 42,
|
||||
color = "#F8FAFC",
|
||||
highlightColor = "#22D3EE",
|
||||
backgroundColor = "rgba(15, 23, 42, 0.75)",
|
||||
fontFamily = "Space Grotesk, Inter, system-ui, sans-serif",
|
||||
}) => {
|
||||
const { fps } = useVideoConfig();
|
||||
const pages = buildPages(words, wordsPerPage);
|
||||
|
||||
return (
|
||||
<AbsoluteFill>
|
||||
{pages.map((page, i) => {
|
||||
const fromFrame = Math.round((page.startMs / 1000) * fps);
|
||||
const nextStart = pages[i + 1]?.startMs ?? page.endMs + 500;
|
||||
const duration = Math.max(
|
||||
1,
|
||||
Math.round(((nextStart - page.startMs) / 1000) * fps)
|
||||
);
|
||||
|
||||
return (
|
||||
<Sequence key={i} from={fromFrame} durationInFrames={duration}>
|
||||
<PageRenderer
|
||||
page={page}
|
||||
fontSize={fontSize}
|
||||
color={color}
|
||||
highlightColor={highlightColor}
|
||||
backgroundColor={backgroundColor}
|
||||
fontFamily={fontFamily}
|
||||
/>
|
||||
</Sequence>
|
||||
);
|
||||
})}
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
@@ -0,0 +1,113 @@
|
||||
import {
|
||||
AbsoluteFill,
|
||||
interpolate,
|
||||
spring,
|
||||
useCurrentFrame,
|
||||
useVideoConfig,
|
||||
} from "remotion";
|
||||
|
||||
interface HeroTitleProps {
|
||||
title: string;
|
||||
subtitle?: string;
|
||||
}
|
||||
|
||||
export const HeroTitle: React.FC<HeroTitleProps> = ({ title, subtitle }) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps } = useVideoConfig();
|
||||
|
||||
// Staggered letter-by-letter spring
|
||||
const titleChars = title.split("");
|
||||
|
||||
return (
|
||||
<AbsoluteFill
|
||||
style={{
|
||||
justifyContent: "center",
|
||||
alignItems: "center",
|
||||
background:
|
||||
"radial-gradient(ellipse at center, rgba(15,23,42,0.85) 0%, rgba(15,23,42,0.95) 100%)",
|
||||
}}
|
||||
>
|
||||
<div style={{ textAlign: "center", maxWidth: "85%" }}>
|
||||
{/* Main title with per-character spring */}
|
||||
<div
|
||||
style={{
|
||||
fontSize: 72,
|
||||
fontWeight: 800,
|
||||
fontFamily: "Space Grotesk, Inter, system-ui, sans-serif",
|
||||
lineHeight: 1.2,
|
||||
display: "flex",
|
||||
justifyContent: "center",
|
||||
flexWrap: "wrap",
|
||||
gap: 0,
|
||||
}}
|
||||
>
|
||||
{titleChars.map((char, i) => {
|
||||
const delay = i * 1.2;
|
||||
const charSpring = spring({
|
||||
frame: frame - delay,
|
||||
fps,
|
||||
config: { damping: 12, stiffness: 150 },
|
||||
});
|
||||
|
||||
return (
|
||||
<span
|
||||
key={i}
|
||||
style={{
|
||||
display: "inline-block",
|
||||
opacity: charSpring,
|
||||
transform: `translateY(${interpolate(charSpring, [0, 1], [30, 0])}px)`,
|
||||
color: i < 8 ? "#22D3EE" : "#F8FAFC", // Accent first word
|
||||
whiteSpace: char === " " ? "pre" : undefined,
|
||||
minWidth: char === " " ? "0.3em" : undefined,
|
||||
}}
|
||||
>
|
||||
{char}
|
||||
</span>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
|
||||
{/* Subtitle */}
|
||||
{subtitle && (
|
||||
<div
|
||||
style={{
|
||||
marginTop: 20,
|
||||
opacity: spring({
|
||||
frame: frame - titleChars.length * 1.2 - 5,
|
||||
fps,
|
||||
config: { damping: 20 },
|
||||
}),
|
||||
fontSize: 28,
|
||||
fontWeight: 400,
|
||||
color: "#A78BFA",
|
||||
fontFamily: "Space Grotesk, Inter, system-ui, sans-serif",
|
||||
letterSpacing: "0.1em",
|
||||
textTransform: "uppercase",
|
||||
}}
|
||||
>
|
||||
{subtitle}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Animated underline */}
|
||||
<div
|
||||
style={{
|
||||
margin: "24px auto 0",
|
||||
height: 3,
|
||||
backgroundColor: "#22D3EE",
|
||||
borderRadius: 2,
|
||||
width: interpolate(
|
||||
spring({
|
||||
frame: frame - 15,
|
||||
fps,
|
||||
config: { damping: 15, stiffness: 60 },
|
||||
}),
|
||||
[0, 1],
|
||||
[0, 400]
|
||||
),
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
@@ -0,0 +1,101 @@
|
||||
import {
|
||||
AbsoluteFill,
|
||||
interpolate,
|
||||
spring,
|
||||
useCurrentFrame,
|
||||
useVideoConfig,
|
||||
} from "remotion";
|
||||
|
||||
interface SectionTitleProps {
|
||||
title: string;
|
||||
subtitle?: string;
|
||||
accentColor?: string;
|
||||
position?: "top-left" | "bottom-left" | "center";
|
||||
}
|
||||
|
||||
export const SectionTitle: React.FC<SectionTitleProps> = ({
|
||||
title,
|
||||
subtitle,
|
||||
accentColor = "#22D3EE",
|
||||
position = "top-left",
|
||||
}) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps, durationInFrames } = useVideoConfig();
|
||||
|
||||
// Entrance spring
|
||||
const slideIn = spring({
|
||||
frame,
|
||||
fps,
|
||||
config: { damping: 15, stiffness: 80 },
|
||||
});
|
||||
|
||||
// Exit fade
|
||||
const exitStart = durationInFrames - 15;
|
||||
const fadeOut = interpolate(frame, [exitStart, durationInFrames], [1, 0], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
|
||||
const opacity = Math.min(slideIn, fadeOut);
|
||||
|
||||
const positionStyles: React.CSSProperties =
|
||||
position === "center"
|
||||
? { justifyContent: "center", alignItems: "center" }
|
||||
: position === "bottom-left"
|
||||
? { justifyContent: "flex-end", alignItems: "flex-start", padding: 60 }
|
||||
: { justifyContent: "flex-start", alignItems: "flex-start", padding: 60 };
|
||||
|
||||
return (
|
||||
<AbsoluteFill style={positionStyles}>
|
||||
<div
|
||||
style={{
|
||||
opacity,
|
||||
transform: `translateX(${interpolate(slideIn, [0, 1], [-40, 0])}px)`,
|
||||
}}
|
||||
>
|
||||
{/* Accent bar */}
|
||||
<div
|
||||
style={{
|
||||
width: interpolate(slideIn, [0, 1], [0, 60]),
|
||||
height: 4,
|
||||
backgroundColor: accentColor,
|
||||
marginBottom: 12,
|
||||
borderRadius: 2,
|
||||
}}
|
||||
/>
|
||||
<div
|
||||
style={{
|
||||
fontSize: 28,
|
||||
fontWeight: 700,
|
||||
color: "#F8FAFC",
|
||||
fontFamily: "Space Grotesk, Inter, system-ui, sans-serif",
|
||||
letterSpacing: "0.05em",
|
||||
textTransform: "uppercase",
|
||||
textShadow: "0 2px 8px rgba(0,0,0,0.6)",
|
||||
}}
|
||||
>
|
||||
{title}
|
||||
</div>
|
||||
{subtitle && (
|
||||
<div
|
||||
style={{
|
||||
fontSize: 18,
|
||||
fontWeight: 400,
|
||||
color: accentColor,
|
||||
fontFamily: "Space Grotesk, Inter, system-ui, sans-serif",
|
||||
marginTop: 4,
|
||||
opacity: spring({
|
||||
frame: frame - 8,
|
||||
fps,
|
||||
config: { damping: 20 },
|
||||
}),
|
||||
textShadow: "0 2px 8px rgba(0,0,0,0.6)",
|
||||
}}
|
||||
>
|
||||
{subtitle}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
@@ -0,0 +1,99 @@
|
||||
import {
|
||||
AbsoluteFill,
|
||||
interpolate,
|
||||
spring,
|
||||
useCurrentFrame,
|
||||
useVideoConfig,
|
||||
} from "remotion";
|
||||
|
||||
interface StatRevealProps {
|
||||
stat: string;
|
||||
label?: string;
|
||||
accentColor?: string;
|
||||
position?: "center" | "bottom-right" | "right";
|
||||
}
|
||||
|
||||
export const StatReveal: React.FC<StatRevealProps> = ({
|
||||
stat,
|
||||
label,
|
||||
accentColor = "#A78BFA",
|
||||
position = "bottom-right",
|
||||
}) => {
|
||||
const frame = useCurrentFrame();
|
||||
const { fps, durationInFrames } = useVideoConfig();
|
||||
|
||||
// Bouncy entrance
|
||||
const scale = spring({
|
||||
frame,
|
||||
fps,
|
||||
config: { damping: 10, stiffness: 100, mass: 0.8 },
|
||||
from: 0,
|
||||
to: 1,
|
||||
});
|
||||
|
||||
const glow = spring({
|
||||
frame: frame - 5,
|
||||
fps,
|
||||
config: { damping: 20, stiffness: 60 },
|
||||
});
|
||||
|
||||
// Exit
|
||||
const exitStart = durationInFrames - 12;
|
||||
const fadeOut = interpolate(frame, [exitStart, durationInFrames], [1, 0], {
|
||||
extrapolateLeft: "clamp",
|
||||
extrapolateRight: "clamp",
|
||||
});
|
||||
|
||||
const opacity = Math.min(spring({ frame, fps, config: { damping: 20 } }), fadeOut);
|
||||
|
||||
const positionStyles: React.CSSProperties =
|
||||
position === "center"
|
||||
? { justifyContent: "center", alignItems: "center" }
|
||||
: position === "right"
|
||||
? { justifyContent: "center", alignItems: "flex-end", paddingRight: 80 }
|
||||
: { justifyContent: "flex-end", alignItems: "flex-end", padding: 80 };
|
||||
|
||||
return (
|
||||
<AbsoluteFill style={positionStyles}>
|
||||
<div
|
||||
style={{
|
||||
opacity,
|
||||
transform: `scale(${scale})`,
|
||||
textAlign: position === "center" ? "center" : "right",
|
||||
}}
|
||||
>
|
||||
<div
|
||||
style={{
|
||||
fontSize: 96,
|
||||
fontWeight: 800,
|
||||
color: accentColor,
|
||||
fontFamily: "Space Grotesk, Inter, system-ui, sans-serif",
|
||||
lineHeight: 1,
|
||||
textShadow: `0 0 ${interpolate(glow, [0, 1], [0, 30])}px ${accentColor}66, 0 4px 12px rgba(0,0,0,0.5)`,
|
||||
}}
|
||||
>
|
||||
{stat}
|
||||
</div>
|
||||
{label && (
|
||||
<div
|
||||
style={{
|
||||
fontSize: 22,
|
||||
fontWeight: 500,
|
||||
color: "#F8FAFC",
|
||||
fontFamily: "Space Grotesk, Inter, system-ui, sans-serif",
|
||||
marginTop: 8,
|
||||
opacity: spring({
|
||||
frame: frame - 10,
|
||||
fps,
|
||||
config: { damping: 20 },
|
||||
}),
|
||||
textShadow: "0 2px 8px rgba(0,0,0,0.6)",
|
||||
}}
|
||||
>
|
||||
{label}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</AbsoluteFill>
|
||||
);
|
||||
};
|
||||
@@ -4,3 +4,7 @@ export { ProgressBar } from "./ProgressBar";
|
||||
export { CalloutBox } from "./CalloutBox";
|
||||
export { ComparisonCard } from "./ComparisonCard";
|
||||
export { BarChart, LineChart, PieChart, KPIGrid } from "./charts";
|
||||
export { CaptionOverlay } from "./CaptionOverlay";
|
||||
export { SectionTitle } from "./SectionTitle";
|
||||
export { StatReveal } from "./StatReveal";
|
||||
export { HeroTitle } from "./HeroTitle";
|
||||
|
||||
@@ -99,11 +99,16 @@ Process asset tasks grouped by tool for efficiency:
|
||||
### Step 5: Generate Music
|
||||
|
||||
1. Read playbook's `audio.music_mood` and `audio.music_volume`
|
||||
2. Generate or select a background track:
|
||||
- **Primary**: `music_gen` (ElevenLabs Music) — custom, costs per generation
|
||||
- **Fallback**: Stock music library (if available)
|
||||
3. Duration should match total video duration (or be loopable)
|
||||
4. Verify the audio file exists
|
||||
2. Check the music decision from `proposal_packet.production_plan.music_source` (set by the Proposal Director)
|
||||
3. Source the background track in this priority order:
|
||||
- **User-selected library track**: If the proposal specified a track from `music_library/`, copy it to `projects/<project>/assets/music/background_music.mp3`
|
||||
- **User music library (`music_library/`)**: If the folder exists and has tracks, pick the best match for the playbook's `audio.music_mood`. List candidates by filename and let the EP decide.
|
||||
- **Music generation API**: Use `music_gen` (ElevenLabs) or `suno_music` if available. Check status via registry first — if the tool is unavailable or quota-exhausted, skip immediately (do NOT attempt and fail silently).
|
||||
- **No music available**: Log this clearly in the asset manifest as `"music_status": "unavailable"` with the reason. Do NOT silently produce a video without music — the EP and user should know.
|
||||
4. Duration should be at least as long as total video duration. If shorter, it can be looped by the compose stage.
|
||||
5. Verify the audio file exists at `projects/<project>/assets/music/background_music.mp3`
|
||||
|
||||
**Critical:** If music generation fails or is unavailable, report it immediately in the asset manifest — do not defer the problem to the compose stage.
|
||||
|
||||
### Step 6: Build Asset Manifest
|
||||
|
||||
|
||||
@@ -95,6 +95,56 @@ Call the `audio_mixer` tool to:
|
||||
|
||||
The video_compose tool will mux this with the video.
|
||||
|
||||
### Step 5b: Generate Subtitles (Mandatory)
|
||||
|
||||
Subtitles are mandatory for all explainer content. Generate them from the narration audio — do NOT skip this step.
|
||||
|
||||
1. **Transcribe** the full narration using the `transcriber` tool (whisperx):
|
||||
```python
|
||||
from tools.analysis.transcriber import Transcriber
|
||||
result = Transcriber().execute({
|
||||
'input_path': 'projects/<project>/assets/audio/narration_full.mp3',
|
||||
'model_size': 'base',
|
||||
'language': 'en',
|
||||
'output_dir': 'projects/<project>/assets/audio'
|
||||
})
|
||||
# result.data contains segments with word-level timestamps
|
||||
```
|
||||
|
||||
2. **Generate SRT** from the transcription using `subtitle_gen`:
|
||||
```python
|
||||
from tools.subtitle.subtitle_gen import SubtitleGen
|
||||
result = SubtitleGen().execute({
|
||||
'segments': transcription_data['segments'],
|
||||
'format': 'srt',
|
||||
'output_path': 'projects/<project>/assets/subtitles.srt',
|
||||
'max_words_per_cue': 8,
|
||||
'max_chars_per_line': 42
|
||||
})
|
||||
```
|
||||
|
||||
3. **Burn subtitles** into the video using `video_compose`:
|
||||
```python
|
||||
from tools.video.video_compose import VideoCompose
|
||||
result = VideoCompose().execute({
|
||||
'operation': 'burn_subtitles',
|
||||
'input_path': 'projects/<project>/renders/output.mp4',
|
||||
'output_path': 'projects/<project>/renders/final.mp4',
|
||||
'subtitle_path': 'projects/<project>/assets/subtitles.srt',
|
||||
'subtitle_style': {
|
||||
'font': '<from playbook typography.headings.font or Arial>',
|
||||
'font_size': 22,
|
||||
'primary_color': '&HFFFFFF',
|
||||
'outline_color': '&H000000',
|
||||
'outline_width': 2,
|
||||
'margin_v': 50,
|
||||
'alignment': 2
|
||||
}
|
||||
})
|
||||
```
|
||||
|
||||
**The final deliverable is the subtitled version**, not the pre-subtitle render.
|
||||
|
||||
### Step 6: Verify Output
|
||||
|
||||
**File verification:**
|
||||
|
||||
@@ -216,6 +216,43 @@ Also present **alternative production paths** — complete packages at different
|
||||
| Budget | Local TTS + images | ~$0.05-0.15 | Robotic voice, image-only |
|
||||
| Free | Local TTS + diagrams | $0.00 | Functional but minimal |
|
||||
|
||||
### Step 5b: Music Plan (Mandatory)
|
||||
|
||||
Music is a critical part of the video's feel. **Surface the music situation to the user at proposal time** — do not silently defer it to the asset stage where a failure becomes expensive.
|
||||
|
||||
**Check music availability in this order:**
|
||||
|
||||
1. **User music library (`music_library/`):** Check if this folder exists and contains tracks. If so, list available tracks with durations and let the user pick one.
|
||||
2. **Music generation APIs:** Check which music tools are available via the registry (`registry.get_by_capability("music_generation")`). Report their status honestly.
|
||||
3. **Stock music sources:** Note if stock music is available via any provider.
|
||||
|
||||
**Present to the user:**
|
||||
|
||||
```
|
||||
MUSIC PLAN
|
||||
├── Your music library: 3 tracks available
|
||||
│ ├── cosmic_interstellar_space.mp3 (3:13) — ambient, cosmic
|
||||
│ ├── cinematic_epic.mp3 (2:45) — dramatic, building
|
||||
│ └── lofi_beat.mp3 (4:00) — chill, electronic
|
||||
├── AI generation: music_gen (ElevenLabs) — UNAVAILABLE (plan limit)
|
||||
└── Recommendation: Use "cosmic_interstellar_space.mp3" from your library
|
||||
OR provide a different track before asset generation
|
||||
|
||||
Would you like to:
|
||||
(a) Use a track from your library (which one?)
|
||||
(b) Provide a different track (drop it in music_library/)
|
||||
(c) Generate one via API (if available)
|
||||
(d) Proceed without music
|
||||
```
|
||||
|
||||
**If no music source is available:** Tell the user explicitly. Do NOT let this surface as a surprise at the asset stage. Offer the `music_library/` path so they can add a track before production starts.
|
||||
|
||||
**Rules:**
|
||||
- Always check `music_library/` first — user-provided music is free and intentional
|
||||
- Always report music API status (available, unavailable, quota remaining if checkable)
|
||||
- Record the music decision in `proposal_packet.production_plan.music_source`
|
||||
- If the user picks a library track, record its path for the asset director
|
||||
|
||||
### Step 6: Build the Cost Estimate
|
||||
|
||||
Itemize every paid operation:
|
||||
|
||||
@@ -161,13 +161,19 @@ class VideoCompose(BaseTool):
|
||||
]
|
||||
|
||||
def _remotion_available(self) -> bool:
|
||||
"""Check if Remotion rendering is available (requires npx + composer project)."""
|
||||
"""Check if Remotion rendering is available (requires npx + composer project + node_modules)."""
|
||||
import shutil as _shutil
|
||||
|
||||
if not _shutil.which("npx"):
|
||||
return False
|
||||
composer_dir = Path(__file__).resolve().parent.parent.parent / "remotion-composer"
|
||||
return composer_dir.exists() and (composer_dir / "package.json").exists()
|
||||
if not composer_dir.exists() or not (composer_dir / "package.json").exists():
|
||||
return False
|
||||
# Check that node_modules are actually installed — without this,
|
||||
# npx remotion render will fail even though the project exists.
|
||||
if not (composer_dir / "node_modules").exists():
|
||||
return False
|
||||
return True
|
||||
|
||||
def get_info(self) -> dict[str, Any]:
|
||||
"""Extend base get_info to surface Remotion sub-capability.
|
||||
@@ -193,10 +199,18 @@ class VideoCompose(BaseTool):
|
||||
"and motion-graphics pipelines."
|
||||
)
|
||||
else:
|
||||
info["remotion_note"] = (
|
||||
"Remotion is NOT available (needs Node.js/npx + remotion-composer). "
|
||||
"Falling back to FFmpeg Ken Burns for image-based compositions."
|
||||
)
|
||||
composer_dir = Path(__file__).resolve().parent.parent.parent / "remotion-composer"
|
||||
if composer_dir.exists() and (composer_dir / "package.json").exists() and not (composer_dir / "node_modules").exists():
|
||||
info["remotion_note"] = (
|
||||
"Remotion project exists but node_modules are NOT installed. "
|
||||
"Run 'cd remotion-composer && npm install' to enable Remotion rendering. "
|
||||
"Falling back to FFmpeg Ken Burns for image-based compositions."
|
||||
)
|
||||
else:
|
||||
info["remotion_note"] = (
|
||||
"Remotion is NOT available (needs Node.js/npx + remotion-composer). "
|
||||
"Falling back to FFmpeg Ken Burns for image-based compositions."
|
||||
)
|
||||
return info
|
||||
|
||||
def execute(self, inputs: dict[str, Any]) -> ToolResult:
|
||||
|
||||
Reference in New Issue
Block a user