Code-Generated Promo Videos (1): Tech Stack Overview & Remotion Footage

This article is based on hands-on experience from the MiBee NVR open-source 45-second promo video project. You will learn how to generate video footage by code (Remotion), produce AI voiceovers (edge-tts), synthesize BGM offline (numpy), and mux everything into a final video with ffmpeg. All steps are ready to follow.

What This Technology Does

A multilingual promo project consists of three independent production stages, finally muxed by ffmpeg:

The overall pipeline looks like this:

flowchart TD
    A[Footage<br/>Remotion] --> D[ffmpeg Mux]
    B[Voiceover<br/>edge-tts] --> D
    C[BGM<br/>numpy] --> D
    D --> E[Final MP4]

    classDef blue fill:#2196F3,stroke:#1565C0,color:#fff;
    classDef green fill:#4CAF50,stroke:#2E7D32,color:#fff;
    classDef orange fill:#FF9800,stroke:#E65100,color:#fff;
    classDef purple fill:#9C27B0,stroke:#6A1B9A,color:#fff;

    class A blue;
    class B green;
    class C orange;
    class D purple;
    class E green;

Flow: Three independent stages produce silent footage, voiceover MP3s, and BGM WAV respectively. ffmpeg muxes them into the final MP4. Each stage can be iterated independently without affecting the others.

Why Code Instead of GUI Editors

Traditional NLE software (Premiere Pro, DaVinci Resolve, CapCut) is efficient for creating a single video. But when you need bulk production — translating the same promo into 6 languages, each with male and female voice versions — manual editing effort grows linearly. The code approach offers:

• Versionable: Script = strings, footage = React components, voiceover = voice ID, BGM = synthesis parameters, muxing = ffmpeg filter graph. Everything is text — in Git, diffable, revertible.
• Reproducible: The same parameters always produce the same result. Remotion’s pure-function frame model guarantees deterministic output.
• Totally free: edge-tts uses Microsoft’s public endpoint, BGM is computed live with numpy — zero cost.
• Zero marginal cost for multilingual: Switching a voice parameter yields English or Cantonese versions; the footage is reused verbatim across all languages — render once, mux 6 audio tracks.
• Debuggable: Each stage is independent. Problems are isolated to the culprit stage.

Tech Stack & Directory Layout

The recommended project layout (set your project root as BASE; all paths below are relative to it):

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21

project-root/
├── remotion-videos/                # Remotion project (footage)
│   ├── package.json                # Dependencies: remotion ^4
│   ├── remotion.config.ts          # Chrome executable config
│   ├── chrome-dl/                  # Manually downloaded chrome-headless-shell
│   │   └── chrome-headless-shell-win64/chrome-headless-shell.exe
│   └── src/
│       ├── Root.tsx                # Composition registration
│       └── compositions/promo/
│           ├── VideoComposition.tsx # Sequence-based scene stitching
│           ├── Scene1~Scene7*.tsx   # One component per scene
│           ├── Icons.tsx            # Inline SVG icons
│           ├── Backgrounds.tsx      # Background components
│           └── constants.ts         # Colors, copy constants
├── voiceover/                      # Audio pipeline
│   ├── gen_all_voices.py           # edge-tts batch generation
│   ├── gen_bgm.py                  # numpy BGM synthesis
│   ├── mix_all.py                  # ffmpeg muxing
│   ├── bgm.wav                     # Generated BGM
│   └── voice_zh_f/ zh_m/ en_f/ ... # Voiceover clips per variant
└── MiBeeNVR_Promo_*.mp4            # Final output

You only need Node.js (18+ recommended) and Python 3.8+. All code in this series follows this layout.

Remotion Footage Generation

Installation & Chrome Setup

1
2

cd remotion-videos
npm install

Remotion requires a browser engine to render. In sandboxed or offline environments, Remotion’s default Chromium download often fails. The most reliable approach is to manually download chrome-headless-shell and point to it explicitly.

Download the latest chrome-headless-shell for your OS from Chrome for Testing, extract it to remotion-videos/chrome-dl/. Then configure in remotion.config.ts:

1
2
3
4
5
6
7
8

import { Config } from "@remotion/cli/config";

Config.setVideoImageFormat("jpeg");
Config.setOverwriteOutput(true);
Config.setConcurrency(4);
Config.setBrowserExecutable(
  "./chrome-dl/chrome-headless-shell-win64/chrome-headless-shell.exe"
);

chrome-headless-shell is Chrome stripped of UI, extensions, printing — only the rendering engine remains. It is smaller and suited for server/sandbox environments.

Registering a Composition

A Composition represents a complete video. Register it in src/Root.tsx:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15

import React from "react";
import { Composition } from "remotion";
import { PromoComposition } from "./compositions/promo/VideoComposition";

export const RemotionRoot: React.FC = () => (
  <Composition
    id="promo"             // Render target ID
    component={PromoComposition}
    durationInFrames={1350} // 45s × 30fps
    fps={30}
    width={1920}
    height={1080}
    defaultProps={{}}
  />
);

Key parameters:

• durationInFrames: total frames = duration(seconds) × fps
• fps: frame rate, typically 30
• width / height: output resolution, 1920×1080 is standard

Stitching Scenes with Sequence

Each scene is an independent component. Stitch them with Remotion’s <Sequence>, which takes from (start frame) and durationInFrames:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24

import { AbsoluteFill, Sequence, useCurrentFrame, interpolate } from "remotion";

// Cross-fade overlay between scenes
const Overlay = ({ duration }: { duration: number }) => {
  const frame = useCurrentFrame();      // local frame, starts at 0
  let opacity = 0;
  if (frame < 15)
    opacity = interpolate(frame, [0, 15], [1, 0], { extrapolateRight: "clamp" });
  else if (frame > duration - 15)
    opacity = interpolate(frame, [duration - 15, duration], [0, 1], { extrapolateLeft: "clamp" });
  return <AbsoluteFill style={{ backgroundColor: "#0a0e1a", opacity, pointerEvents: "none", zIndex: 100 }} />;
};

export const PromoComposition = () => (
  <AbsoluteFill style={{ backgroundColor: "#0a0e1a" }}>
    <Sequence from={0}    durationInFrames={150}><Scene1 /><Overlay duration={150} /></Sequence>
    <Sequence from={150}  durationInFrames={180}><Scene2 /><Overlay duration={180} /></Sequence>
    <Sequence from={330}  durationInFrames={240}><Scene3 /><Overlay duration={240} /></Sequence>
    <Sequence from={570}  durationInFrames={210}><Scene4 /><Overlay duration={210} /></Sequence>
    <Sequence from={780}  durationInFrames={210}><Scene5 /><Overlay duration={210} /></Sequence>
    <Sequence from={990}  durationInFrames={210}><Scene6 /><Overlay duration={210} /></Sequence>
    <Sequence from={1200} durationInFrames={150}><Scene7 /><Overlay duration={150} /></Sequence>
  </AbsoluteFill>
);

useCurrentFrame() frame model：<Sequence from={150}> applies a frame offset to its children. When the renderer processes global frame 150, the child component’s useCurrentFrame() returns 0 (local frame), not global 150. Therefore, every interpolate() call inside a Scene uses local frame ranges. In the Overlay example, frame < 15 means “the first 15 frames of this scene” — exactly the convenience that local frame offset provides.

The cross-fade overlay component works as follows: at the start of each scene it fades from the background color to transparent; at the end it fades back to the background color.

Core Animation API

Remotion provides a few animation primitives that cover most motion needs:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30

import { AbsoluteFill, useCurrentFrame, useVideoConfig, interpolate, spring } from "remotion";

export const Scene1 = () => {
  const frame = useCurrentFrame();
  const { fps } = useVideoConfig();

  // 1) interpolate: map frame number to any property value
  const opacity = interpolate(frame, [40, 70], [0, 1], {
    extrapolateLeft: "clamp",
    extrapolateRight: "clamp",
  });

  // 2) spring: physics-based elastic entry
  const scale = spring({
    frame: frame - 40,   // start bouncing at frame 40
    fps,
    config: { damping: 12, stiffness: 100, mass: 0.8 },
  });

  // 3) trigonometry for floating/breathing
  const float = Math.sin(frame * 0.08) * 8;

  return (
    <AbsoluteFill style={{ justifyContent: "center", alignItems: "center" }}>
      <div style={{ transform: `scale(${scale}) translateY(${float}px)`, opacity }}>
        {/* Your content */}
      </div>
    </AbsoluteFill>
  );
};

Three core tools:

• interpolate(input, [inMin,inMax], [outMin,outMax], options): The universal mapper. extrapolateLeft / extrapolateRight control behavior outside the range (clamp to hold, extend to continue).
• spring({frame, fps, config}): Physics-based elastic animation, more lively than linear tweening. damping, stiffness, mass control the elastic behavior.
• Trigonometry: Math.sin(frame * frequency) * amplitude creates floating, breathing, or pulsing effects — lightweight and dependency-free.

Backgrounds & Inline SVG Icons

⚠️ headless Chrome has no color emoji fonts：Emoji characters like 🐝🫐📷⚡ render as empty boxes — they look like missing icons. All icons must be drawn as inline SVG, never as emoji.

Each icon is a React component returning an <svg> element:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22

import React from "react";

type IconProps = { size?: number; color?: string; glowColor?: string; glow?: number };

// Glow: two stacked drop-shadows
const withGlow = (glowColor: string, g: number) =>
  g > 0
    ? `drop-shadow(0 0 ${6 * g}px ${glowColor}) drop-shadow(0 0 ${14 * g}px ${glowColor}aa)`
    : "none";

// Camera icon (bullet-style surveillance)
export const CameraIcon: React.FC<IconProps> = ({
  size = 64, color = "#4CAF50", glowColor = "#4CAF50", glow = 0.6,
}) => (
  <svg width={size} height={size} viewBox="0 0 100 100"
    style={{ filter: withGlow(glowColor, glow), overflow: "visible" }}>
    <rect x="22" y="34" width="46" height="30" rx="8" fill="#0d1526" stroke={color} strokeWidth="2.5" />
    <circle cx="68" cy="49" r="15" fill="#0d1526" stroke={color} strokeWidth="2.5" />
    <circle cx="68" cy="49" r="8" fill={color} opacity="0.35" />
    <circle cx="68" cy="49" r="3.5" fill={color} />
  </svg>
);

Also package backgrounds as reusable components. Common types:

• Gradient base fill: Eliminates the “floating in black void” feeling.
• Floating hexagons / particle fields: Adds tech aesthetic and depth.
• Glow effects: CSS drop-shadow chained in layers creates luminous accents.

Font size hint: 1920×1080 scales down to ~0.21× on mobile. Text at 16–28px shrinks to 3–6px — illegible mush. If your video targets mobile viewers, keep body text ≥ 40px, or apply a global font-size scale factor.

Render Command

Once the footage is ready, render a silent MP4:

1
2

cd remotion-videos
npx remotion render promo "../MiBeeNVR_Promo_Full.mp4" --log=error

• promo is the id registered in Root.tsx
• Output is a silent MP4 (audio is handled later by ffmpeg)
• --log=error suppresses everything except errors
• Render speed reference: a 45-second video takes ~2 minutes (varies by CPU/memory and concurrency)
• Config.setConcurrency(4) controls how many screenshot processes run in parallel

How Remotion Works: React Becomes Video

Understanding the principles lets you modify, debug, and extend with confidence.

Frame Model: Video as Pure Function

A video is a deterministic sequence of frames. A web page is interactive — nothing changes without user input. A video is not — frame 100 should always look the same. Remotion models “what frame N looks like” as a pure function: frame number in, deterministic DOM/SVG out.

Where does useCurrentFrame() get its value? Although it’s a React Hook, its value is not driven by user interaction — the renderer injects frame=N when generating frame N. Similarly, useVideoConfig() injects fps, width, height, and durationInFrames. Therefore, interpolate(frame, [0,30], [0,1]) always yields 0.5 at frame 15 — deterministic, reproducible, testable.

Per-Frame Screenshot Pipeline

flowchart TD
    A[React Components] --> B[Headless Chrome<br/>render each frame DOM]
    B --> C[Capture frames<br/>as JPEG/PNG]
    C --> D[ffmpeg encode]
    D --> E[Silent MP4]

    classDef blue fill:#2196F3,stroke:#1565C0,color:#fff;
    classDef green fill:#4CAF50,stroke:#2E7D32,color:#fff;
    classDef orange fill:#FF9800,stroke:#E65100,color:#fff;
    classDef purple fill:#9C27B0,stroke:#6A1B9A,color:#fff;

    class A orange;
    class B blue;
    class C blue;
    class D purple;
    class E green;

Render pipeline: React components are rendered into real DOM inside headless Chrome frame by frame. The screenshot engine exports each frame as JPEG/PNG. Finally ffmpeg encodes the image sequence into an H.264 video stream.

Step by step:

1. Remotion launches a headless browser (Chromium or chrome-headless-shell).
2. The browser renders each frame as real DOM — full support for CSS gradients, Web fonts, SVG filters, drop-shadow, and other modern Web features.
3. The browser’s screenshot capability exports the frame as JPEG or PNG.
4. Repeat 1350 times (45s × 30fps) → 1350 images.
5. Internal ffmpeg encodes the image sequence into an H.264 video stream, outputting a silent MP4.

Why Chrome Is Necessary

Only a modern GPU-accelerated renderer can correctly handle SVG filters, CSS drop-shadow, mix-blend-mode, Web fonts, and similar features. Chrome’s chrome-headless-shell strips the UI, extensions, printing, etc. — keeping only the rendering engine. It is smaller and consumes fewer resources, making it ideal for server and sandbox environments.

Why It Scales (Serverless / Distributed)

Frame N’s rendering does not depend on any other frame. This is the key benefit of the pure-function model — 1350 frames can be arbitrarily partitioned and rendered in parallel across multiple machines, then stitched together. Remotion Lambda (the official serverless offering) is built on exactly this principle: your concurrency is limited only by your budget, not by a single machine. Locally, Config.setConcurrency(4) simply runs 4 screenshot processes in parallel on the same machine.

Series: ① Overview & Remotion ② edge-tts Voiceover ③ numpy BGM ④ ffmpeg Muxing