ARCHITECTURE.md said `npx @hyperframes/cli` but that scoped package
is a 404. The published npm package is `hyperframes`, consumed as
`npx hyperframes`. hyperframes_compose.py already documents this
distinction in its _NPM_PACKAGE constant and install_instructions.
ToolRegistry exposes get(name) but source_media_review called the
non-existent get_tool(name) in four places — audio_probe (×2),
frame_sampler, and transcriber. All four renamed to .get().
Lines 219 and 227 imported upload_image_fal from _shared but called
the non-existent upload_image_to_fal, raising a NameError whenever
image_to_video or reference_to_video used local image paths.
Adds a transcript_comparison check to VideoCompose._run_final_review
that word-diffs the whisper/whisperx transcript against script.txt and
fails loudly when the TTS engine literally voiced punctuation tokens
(dot, dots, ellipsis, comma, dash, hyphen, period). Chirp3-HD did this
to ellipses in a production run and it slipped past review; now it
cannot.
Also corrects the "tiny background video" gotcha in
skills/core/hyperframes.md. After six renders of blaming HyperFrames
CSS, the real root cause was 640x360 Pexels sources combined with a
fit-and-pad pre-transform — HyperFrames was rendering the letterboxed
input faithfully. Gotcha now walks through the ffprobe diagnostic and
the scale-to-cover fix, and keeps the wrapper-div pattern for the
right reasons (aspect mismatch handling, not framework bug workaround).
Four regression tests cover the new check: punctuation-leak detection,
clean-audio false-positive guard, graceful skip when inputs missing,
and always-present transcript_comparison section.
Surfaces six real traps an agent learns the expensive way: full-frame
videos need BOTH HTML width/height attrs AND `!important` CSS because
the runtime applies inline size from intrinsic dimensions; preview-scrub
matters more than a 60-minute render for visual bugs; legibility comes
from scrims + text-shadow, not from lowering video opacity; downloaded
stock needs dense keyframes (-g 30) before render or parallel capture
times out; --workers 1 is mandatory for video-heavy compositions; the
deterministic-font compiler silently replaces unmapped families like
Space Grotesk, use Outfit/Inter/JetBrains Mono/Poppins instead. Each
gotcha documented with the fix-at-author-time pattern so the next agent
ships a clean first render.
Separates creative grammar (renderer_family) from technical engine
(render_runtime) so HyperFrames can stand alongside Remotion as a
first-class runtime instead of masquerading as a Remotion sub-case.
Locks runtime choice at proposal stage and enforces it end-to-end: the
schemas require it, video_compose routes by it, the reviewer fails
closed on silent swaps, and a parametrized contract test walks every
pipeline manifest to ensure each planning-stage skill explains the
conversation to the user. Adds hyperframes_compose (scaffold/lint/
validate/render/doctor/add_block), a playbook -> CSS style bridge, and
vendored HyperFrames Layer 3 skills from commit d291358, pinned via
PROVENANCE.md for future re-sync. Final_review now records
render_runtime_used and runtime_swap_detected so compose lies are
catchable after the fact.
Expands the Seedance 2.0 surface from fal.ai-only to every gateway the
OpenMontage toolset can reach, so users on Replicate / Runway /
Higgsfield automatically route cinematic briefs through Seedance 2.0
instead of the gateway's prior default.
- tools/video/seedance_replicate (new): REPLICATE_API_TOKEN adapter for
bytedance/seedance-2.0 and bytedance/seedance-2.0-fast. Same provider
family ("seedance"), quality_score=0.95, premium capability flags.
- tools/video/runway_video: add seedance_2.0 / seedance_2.0_fast to the
model enum as the new default, flag native_audio + multi_shot +
camera_direction + lip_sync in supports, point agent_skills at
seedance-2-0 Layer 3 skill. Note Runway Unlimited/Enterprise + non-US
gating in the model description.
- tools/video/higgsfield_video: same pattern. seedance_2.0 is now the
default model. Cost estimates calibrated to Higgsfield credit pricing
(~$0.50-$1.20 per 5s).
- tools/video/_shared: annotate HEYGEN_PROVIDERS seedance_pro/lite as
Seedance 1.x (2.0 on HeyGen flows through Video Agent / Avatar Shots
which need a separate wrapper — tracked as follow-up).
- .agents/skills/seedance-2-0: update provider matrix to show which
gateways are wrapped (fal.ai, Replicate, Runway, Higgsfield) vs. which
are still roadmap (HeyGen Video Agent, BytePlus, Freepik, aggregators).
The scoring engine already deduplicates by provider="seedance" so
whichever gateway the user has configured wins automatically. Agents
passing preferred_provider="seedance" to video_selector route to the
best available Seedance 2.0 path without naming a specific gateway.
Seedance 2.0 is now routed as the top-ranked video generator whenever a
premium gateway is available. Touches the tool layer, scoring engine,
cinematic pipeline, and both skill layers so discovery works from every
entry point.
- tools/video/seedance_video: BETA stability, quality_score=0.95, add
reference_to_video operation plus 9 img + 3 vid + 3 audio ceilings,
fix pre-existing upload_image_fal import
- tools/base_tool: surface optional quality_score / success_rate /
latency fields in get_info so the scorer can read them
- lib/scoring: fix reliability enum-vs-string bug that was pinning every
available tool to 0.0, switch to overlap coefficient so rich best_for
descriptions aren't penalized, add premium-cinematic feature bonus
- pipeline_defs/cinematic + cinematic asset-director: add pixabay_music
and freesound_music, restore pixabay-first music default
- cinematic compose-director: mandatory Remotion preflight at stage entry
- New Layer 3 .agents/skills/seedance-2-0/SKILL.md (8-part prompt
structure, multi-shot, lip-sync, reference-to-video, provider landscape)
- New Layer 2 skills/creative/prompting/seedance-prompting.md
- Update ai-video-gen, video-gen-prompting, AGENT_GUIDE, INDEX to flag
Seedance 2.0 as the preferred premium default and make the skill
discoverable from every routing path
Declaratively wire the new Layer 2 animation-runtime dispatcher into the
four pipelines that do meaningful animation authoring:
- animated-explainer (generated explainer scenes, kinetic typography)
- animation (motion graphics, logo morphs, motion paths)
- cinematic (title cards, curved camera moves, prestige easings)
- screen-demo (synthetic terminal timing + callout overlays)
Before this, skills/meta/animation-runtime-selector.md was referenced only
in asset-director prose — a fresh-context agent running preflight on the
manifest alone would miss it. Adding it to required_skills: closes the
discovery loop: the reviewer + checkpoint systems now enforce that it
was consulted before authoring animated components.
No schema change required — follows the existing pattern of listing meta
skills (reviewer, checkpoint-protocol, skill-creator) as required.
Mirror the official GSAP AI skills (greensock/gsap-skills, MIT) into
.agents/skills/ and add the Layer 2 wiring that makes them discoverable
from a fresh context.
GSAP covers animation needs that Remotion primitives strain at:
per-character text reveals (SplitText), SVG shape morphs (MorphSVG),
curved camera paths (MotionPath), stroke-reveal line drawing (DrawSVG),
layout-to-layout flight (Flip), and custom bezier easings (CustomEase).
Also becomes mandatory day-1 knowledge if we wire in HyperFrames later
(HF uses GSAP timelines as its native animation runtime).
Layer 3 adds (.agents/skills/):
- gsap-core, gsap-timeline, gsap-plugins, gsap-utils
- gsap-react, gsap-performance
- gsap-scrolltrigger, gsap-frameworks (situational)
- gsap/README.md — OpenMontage-specific framing and Remotion-safe usage
Layer 2 wiring (the discovery triggers):
- skills/meta/animation-runtime-selector.md — NEW routing meta-skill.
Decision matrix covering Remotion primitives, GSAP plugins, framer-
motion, Lottie, Manim, D3, TerminalScene. Enforces the "keep it
simple" bias: reach for GSAP only when the plugin genuinely earns
its bundle weight.
- skills/pipelines/explainer/asset-director.md — references GSAP for
kinetic typography, multi-step choreography, SVG line draws.
- skills/pipelines/animation/asset-director.md — references GSAP for
logo morphs, motion paths, FLIP transitions, custom easings.
- skills/pipelines/cinematic/asset-director.md — references GSAP for
cinematic camera moves, per-char title reveals, prestige easings.
- AGENT_GUIDE.md — adds a categorized Layer 3 skills table so a fresh-
context agent can find the right skill by what they're trying to do,
plus a pointer to animation-runtime-selector.md for routing.
Determinism: every GSAP use inside Remotion must drive timeline progress
from useCurrentFrame(), never requestAnimationFrame. Three Remotion-safe
patterns are documented in both the gsap/README and the selector skill.
Attribution: https://github.com/greensock/gsap-skills (MIT).
Make synthetic screen recording a first-class, discoverable capability
alongside real OS capture. For CLI / terminal / install-flow demos where
commands and output are predictable, author a `terminal_scene` cut instead
of driving a real screen recorder — deterministic, privacy-safe, pixel-
perfect, and frame-accurate to narration cues.
Components:
- TerminalScene.tsx: window chrome, char-by-char typing, blinking cursor,
scrolling output, non-blocking floating pills, spring-based reveals
- ProviderChip.tsx: rotating badge overlay that cycles through provider
names (used in AI-generated-motion scenes)
- BackgroundVideoLayer + source_in_seconds + backgroundVideo props on
every scene type — supports video-behind-component composition
Discovery chain (six layers, so the next agent finds this without reading
source code):
1. pipeline_defs/screen-demo.yaml — bump to 2.1, declare production_modes
(real_capture, synthetic_terminal) with required_tools, scene_type, and
agent_skills pointers
2. skills/pipelines/screen-demo/idea-director.md — mode-selection table
at brief time; brief.metadata.production_mode contract
3. skills/pipelines/screen-demo/asset-director.md — reads production_mode
and branches asset production (capture+overlays vs steps+narration+
pacing check)
4. .agents/skills/synthetic-screen-recording/SKILL.md — Layer 3 skill with
step kinds (cmd/out/pause/pill), pacing rule, and the frozen-terminal
failure mode captured from the showcase v3 retune
5. AGENT_GUIDE.md — TerminalScene added to Remotion routing; links
SCENE_TYPES.md as the authoritative cut-type registry
6. remotion-composer/SCENE_TYPES.md — new cheat sheet of every cut.type
and overlay.type with required fields, plus a "how to add a new scene
type" section (candidates: ChatTranscript, EditorScene, PrReview,
SlackThread, TicketBoard)
Guardrail:
- lib/verify_scene_pacing.py — reusable trace() and assert_alignment()
helpers that mimic the TerminalScene frame math exactly. Fail loudly
before render if steps burn through too fast or leave the scene frozen.
Teach the agent to automatically use Pixabay AI-generated fantasy clips
for children's content. scene-director.md gets query rewriting rules
(10-row table) and source routing. asset-director.md gets source lock,
visual consistency check, and fallback rules. Root.tsx adds a
CaptionOverlayOnly Remotion composition for transparent caption
rendering used in the TikTok-style subtitle pipeline.
6 API-based adapters: Pixabay Video, Coverr, NARA (U.S. National Archives),
Library of Congress, Pond5 Public Domain, Videvo. 5 scraper-based adapters
for sites without APIs: ESA, NOAA, Mixkit, Dareful, JAXA. All follow the
StockSource protocol and are auto-discovered — no tool code changes needed.
Updated scene-director, asset-director, and idea-director skills with
source routing guidance so agents know which provider to use for which
content type (e.g. nara for historical, noaa for ocean, esa for space).
Also includes: grok_video capability updates (native audio, 1-15s duration,
new aspect ratios), seedance_video tool (Seedance 2.0 via fal.ai),
ProductReveal Remotion composition registration.
Add HeroTitle Remotion composition (used for documentary title cards).
Gitignore now covers youtube tokens, stray pexels downloads, and
scratch dirs. Removed leftover test media and credential files.
Adds a lightweight alternative to the corpus_builder + clip_search pipeline
that skips CLIP embeddings, motion scores, and index files. Uses the same
StockSource adapter protocol so it works with all providers (Pexels,
Archive.org, NASA, Wikimedia, Unsplash). Asset-director skill now documents
both fast path (direct search) and standard path (corpus + CLIP retrieval).
Pipeline manifest updated to make corpus tools optional.
End-tag now composited over final body footage (ProRes 4444 with alpha)
instead of concatenated as a black card. Updated idea-director,
edit-director, compose-director, and pipeline manifest with overlay-first
instructions. Default Google TTS voice changed from Neural2-D to
Chirp3-HD-Orus across tool code and docs.
All of these changes came out of running the P1 and P2 documentary-montage
audits end-to-end and watching specific things break. Grouping them into
one commit because they share a theme: making the corpus builder and its
stock source adapters robust enough that a real brief can produce a
real corpus without special-casing.
corpus_builder.py + new clip_cache.py + test_clip_cache.py
Phase 1 of the shared-corpus architecture. Adds
~/.openmontage/clips_cache/ — a process-safe, LRU-evicted cache of
downloaded clip files keyed by clip_id. Before each candidate download,
corpus_builder asks the cache whether the bytes already exist on disk
from a previous run; on a hit it hard-links (or copies on cross-drive)
the blob into the caller's corpus dir and skips the network fetch. On
a miss it downloads as usual and ingests the fresh file. Motivation:
re-running the P1 audit after every tool fix was re-downloading gigs
of archive.org footage that had already been fetched in the prior run.
Cache faults never block the pipeline — they degrade gracefully to
normal downloads. The cache bubbles counters into the corpus_builder
return payload so the production report can show hit/miss/bytes-saved.
Default 20 GB cap, overridable via OPENMONTAGE_CACHE_MAX_GB.
Full test coverage: try_link, ingest, stats, LRU eviction, manifest
persistence, lock behavior. 23 tests, tmp_path-scoped.
stock_sources/archive_org.py
1. Three-strategy query cascade (phrase_prox_10 → distinctive_and →
distinctive_or). Motivation: natural-language documentary queries
against archive.org Solr were zeroing out — "1950s family watching
television" returned 0 hits because Solr's default multi-term AND
over-intersects. Walks strict to loose and returns the first
non-empty strategy. Stop words, source hints ("prelinger",
"archive", "footage"), and year tokens ("1950s") are excluded from
the distinctive-token picks since they don't correlate with
Prelinger title tokens.
2. 150 MB per-rendition size cap. archive.org routinely hosts
multi-hundred-megabyte h.264 masters and one 2 GB Prelinger item
poisons corpus build wall-time and disk. Within a format bucket
we now pick the largest rendition under the cap; if nothing fits
we fall through to the next format rather than dropping the item.
3. 180 s default max-duration ceiling when the caller hasn't set one
— archive.org is the only source that routinely hosts feature-
length material and a naive fan-out pulls them into corpora that
only ever want a few seconds per clip.
stock_sources/wikimedia.py
Parallel 3-strategy cascade (full → top2_or → single_best). Motivation:
Commons CirrusSearch also defaults to AND across multi-word queries;
our first P2 diagnostic pass returned 0 video results for 10/10
queries. Same stop-word / source-hint / year-token stripping as
archive_org so the two adapters stay symmetric.
test_stock_source_adapters.py
Rewrote the wikimedia query-builder tests against the new cascade
API. Added coverage for multi-word fallback + source-hint/year
stripping.
video_compose.py
Two small fixes for the Remotion renderer on Windows:
1. Resolve output_path to absolute before invoking the CLI so the
binary can write wherever the caller asked regardless of cwd.
2. Pass cwd=composer_dir to run_command so npx can find the local
Remotion binary under node_modules/.bin. Without this, Windows
npx returns "could not determine executable to run" because it
resolves .bin relative to the process cwd rather than the
script's parent.
User override during the audit runs made music and an end-tag MANDATORY
defaults for documentary-montage (narration stays optional). This commit
codifies those defaults into the pipeline manifest + director skills and
ships the Remotion end-tag component that the compose stage now
concatenates after the FFmpeg body.
Changes:
- pipeline_defs/documentary-montage.yaml: idea/edit/compose stages now
require music_plan and end_tag_plan to be present; opt-out requires
an explicit user note recorded in metadata.
- skills/pipelines/documentary-montage/idea-director.md: rewrote sections
4 (music MANDATORY), 5 (end-tag MANDATORY with shape
{text,palette,duration_seconds,render_engine:remotion,component:EndTag}),
6 (narration OPTIONAL), 7 (updated brief JSON shape), 8 (quality gate).
Common Pitfalls grew two entries covering silent-contract drift and
end-tag omission.
- skills/pipelines/documentary-montage/compose-director.md: section 4
now stops on music-contract violation; new section 4b documents the
two-engine flow (FFmpeg body -> npx remotion render EndTag -> ffmpeg
concat). Quality gate checks render_report.metadata.music_mixed and
end_tag_rendered.
- remotion-composer/src/components/EndTag.tsx (new): bold uppercase
typographic end-card with animated 0->100% underline draw-in and a
single left-to-right shimmer sweep. Supports cool_offwhite_on_black
and warm_ivory_on_black palettes. Renders at 1920x1080 @ 30fps.
- remotion-composer/src/Root.tsx: register EndTag as a first-class
composition (durationInFrames=165, 5.5s hold) so it can be rendered
standalone via the Remotion CLI with --props overrides.
New end-to-end pipeline for building thematic documentary montages from
a locally-indexed corpus of free stock footage (Pexels, Archive.org,
NASA). The agent builds a project-local corpus, CLIP-ranks candidates
per scene slot, edits with motion-aware arc logic, and composes via
ffmpeg. No paid APIs required for the full path.
Pipeline definition and director skills:
- pipeline_defs/documentary-montage.yaml: 5-stage manifest
(idea -> scene_plan -> assets -> edit -> compose)
- skills/pipelines/documentary-montage/: 6 director skills
(executive-producer + idea/scene/asset/edit/compose directors)
Corpus and retrieval infrastructure:
- tools/video/corpus_builder.py: multi-source stock fan-out with
resumable append-only corpus index
- tools/video/clip_search.py: CLIP ViT-B/32 retrieval —
rank_for_slot, find_similar_set, diversify, stats
- tools/video/stock_sources/: base + pexels + archive_org + nasa
adapters with a pluggable BaseStockSource contract
- lib/clip_embedder.py: CLIP wrapper
- lib/corpus.py: corpus schema, jsonl append/read, motion-score
caching
video_compose fix rolled in because any concat-based pipeline depends
on it:
- Replace ambiguous -to with -t duration (was double-trimming cuts)
- Force re-encode + normalize to 1920x1080 @ 30fps (was keyframe-
snapping with -c copy and breaking concat on mixed-source corpora)
- Add silent-audio anullsrc fallback for clips without an audio
stream
README: add Documentary Montage row to the pipeline table and bump
the pipeline count from 11 to 12.
- New: Higgsfield video provider with multi-model routing (Kling 3.0, Veo 3.1, Sora 2, WAN 2.5, Soul Cinema) and Soul ID character consistency
- Updated: Runway provider with gen4_aleph and gen3a_turbo models, proper pixel-ratio mapping, probe_output, watermark param, RUNWAYML_API_SECRET env var support
- Docs: Updated provider counts (12→13), tool counts (51→52), added Higgsfield setup/pricing sections across README, AGENT_GUIDE, ARCHITECTURE, and PROVIDERS
New capture layer for the screen-demo pipeline: FFmpeg for quick CLI-driven
recording, Cap integration for polished recordings with webcam overlay and
cursor effects. Selector presents both options and routes based on availability.
Route Chirp and Journey voice families to the v1beta1 API endpoint
automatically. Adds beta voice detection, dynamic API version
selection, and Chirp3-HD cost estimation ($30/1M chars).
Compose-director had contradictory instructions: Step 2 described Remotion
captions/audio, but Steps 5/5b gave detailed FFmpeg code that agents followed
instead. This caused three failures in production: FFmpeg subtitles instead of
Remotion CaptionOverlay, missing audio (mixed externally but never embedded in
Remotion props), and skipped audio verification in post-render review.
Changes:
- compose-director: Remotion is now DEFAULT for audio, captions, text overlays;
FFmpeg is labeled FALLBACK only. Post-render review has mandatory ffprobe gate
and audio transcription with explicit stop conditions.
- remotion.md: routing table updated (captions/audio → Remotion), added universal
Post-Render Verification Protocol for all pipelines (only 2/10 had one).
- scene-director, asset-director: added pitfall for AI-generated text in CTA
screens — must use Remotion text_card for verbatim text.
- image-provider-usage: added Recraft V4 caveat (style param causes 422 on fal.ai).
- recraft_image.py: documented the style parameter 422 issue inline.
- AGENT_GUIDE.md: add "never read source code" rule — skills are the
interface, not .py files
- animation.yaml: add Layer 2 skill-first guardrail in assets stage
- video-reference-analyst.md: Step 4b now mandates Layer 2 before Layer 3,
explicitly forbids reading implementation code
- CinematicRenderer: add TikTok-style CaptionOverlay and separate music
track support (narration + music as independent audio layers)
- cinematic/types.ts: add CinematicCaptionConfig and music prop types
z