Analyze a video's mood and add AI-generated BGM. Optionally speed up/slow down. Uses Gemini for video analysis and fal.ai Lyria2 for music generation. Trigge...
Analyze a video's content and mood, generate matching BGM via AI, and mix it in.
Input Video → Gemini Analysis → Lyria2 BGM → FFmpeg Mix → Output
(mood/style) (generate) (speed + volume + fade)
python3 (or activate your project venv if available)GOOGLE_GENAI_API_KEY (video understanding)FAL_API_KEY (Lyria2 music generation)# Install required packages
pip install google-generativeai httpx
# Set API keys via environment variables
export GOOGLE_GENAI_API_KEY="your_google_api_key"
export FAL_API_KEY="your_fal_api_key"
/video-bgm <path-to-video>
/video-bgm <path-to-video> --speed 1.1
/video-bgm <path-to-video> --speed 1.1 --volume 5
/video-bgm <path-to-video> --style "lo-fi chill"
| Arg | Default | Description |
|---|---|---|
path | (required) | Path to input video file |
--speed | 1.0 | Speed multiplier (e.g. 1.1 = 10% faster) |
--volume | 5.0 | BGM volume multiplier (Lyria2 output is quiet) |
--fade-in | 1.5 | Fade in duration in seconds |
--fade-out | 3.0 | Fade out duration in seconds |
--style | (auto) | Override music style (skip Gemini analysis) |
Upload video to Gemini 2.0 Flash and get deep mood analysis.
import google.generativeai as genai
import os
GOOGLE_API_KEY = os.environ.get("GOOGLE_GENAI_API_KEY")
genai.configure(api_key=GOOGLE_API_KEY)
Use this analysis prompt (acts as music supervisor, not generic):
Watch this video very carefully. You are a music supervisor for commercials.
Tell me:
1. What BRAND POSITIONING does this video convey? (luxury? affordable? aspirational?)
2. What is the EMOTIONAL JOURNEY of the viewer? Be specific at each moment.
3. What REAL commercial music references would fit? Name specific ad styles
(Four Seasons resort? Apple reveal? Volvo? Pottery Barn? Nike?)
4. What is the ENERGY LEVEL? Contemplative/still or forward momentum?
5. What tempo, instruments, and production style would ACTUALLY work?
Be honest - classical piano is often too stuffy. Consider modern alternatives.
Then provide a SINGLE music generation prompt (2-3 sentences) that captures
the ideal BGM. Focus on: instruments, tempo BPM, mood adjectives, production style.
Format: MUSIC_PROMPT: <your prompt here>
Parse the Gemini response to find the MUSIC_PROMPT: line. This becomes the Lyria2 prompt.
Always APPEND these constraints to any Lyria2 prompt:
No vocals, no drums, no percussion hits, no sound effects.
import httpx
import os
FAL_API_KEY = os.environ.get("FAL_API_KEY")
resp = httpx.post(
"https://fal.run/fal-ai/lyria2",
headers={
"Authorization": f"Key {FAL_API_KEY}",
"Content-Type": "application/json",
},
json={"prompt": music_prompt},
timeout=120.0,
)
audio_url = resp.json()["audio"]["url"]
Lyria2 generates ~32s of audio. Output is WAV, 48kHz stereo.
# Step 4a: Speed up video (if requested) and strip any existing audio
ffmpeg -y -i INPUT.mp4 \
-filter:v "setpts=PTS/{speed}" \
-an \
-c:v libx264 -preset medium -crf 18 \
OUTPUT_speedup.mp4
# Step 4b: Get sped-up duration
DURATION=$(ffprobe -v quiet -show_entries format=duration -of csv=p=0 OUTPUT_speedup.mp4)
# Step 4c: Mix BGM with volume boost, fade in/out, trim to video length
ffmpeg -y \
-i OUTPUT_speedup.mp4 \
-i bgm.wav \
-filter_complex "[1:a]volume={volume},atrim=0:{duration},afade=t=in:st=0:d={fade_in},afade=t=out:st={duration-fade_out}:d={fade_out}[a]" \
-map 0:v -map "[a]" \
-c:v copy -c:a aac -b:a 192k \
-shortest \
OUTPUT_final.mp4
Open the final video for user review. Also open the BGM separately so they can evaluate the music alone.
-an) — some videos have unwanted audio tracksAll files are saved next to the input video:
input_video.mp4 → original
input_video_speedup.mp4 → sped up, no audio
input_video_bgm.wav → generated BGM
input_video_final.mp4 → final output with BGM
# Basic: analyze and add BGM
/video-bgm ~/Desktop/product_video.mp4
# Speed up 10% and add BGM
/video-bgm ~/Desktop/product_video.mp4 --speed 1.1
# Override style (skip Gemini analysis)
/video-bgm ~/Desktop/ad.mp4 --style "upbeat modern pop, synth pads, 100 BPM"
# Adjust volume
/video-bgm ~/Desktop/quiet_video.mp4 --volume 8
ZIP package — ready to use