Production pipeline for ASCII art video — any format. Converts video/audio/images/generative input into colored ASCII character video output (MP4, GIF, image sequence). Covers: video-to-ASCII conversion, audio-reactive music visualizers, generative ASCII art animations, hybrid video+audio reactive, text/lyrics overlays, real-time terminal rendering. Use when users request: ASCII video, text art video, terminal-style video, character art animation, retro text visualization, audio visualizer in ASCII, converting video to ASCII art, matrix-style effects, or any animated ASCII output.
Production pipeline for ASCII art video — any format. Converts video/audio/images/generative input into colored ASCII character video output (MP4, GIF, image sequence). Covers: video-to-ASCII conversion, audio-reactive music visualizers, generative ASCII art animations, hybrid video+audio reactive, text/lyrics overlays, real-time terminal rendering. Use when users request: ASCII video, text art video, terminal-style video, character art animation, retro text visualization, audio visualizer in ASCII, converting video to ASCII art, matrix-style effects, or any animated ASCII output.
ASCII Video Production Pipeline
Full production pipeline for rendering any content as colored ASCII character video.
Modes
Mode
Input
Output
Read
Video-to-ASCII
Video file
ASCII recreation of source footage
references/inputs.md § Video Sampling
Audio-reactive
Audio file
Generative visuals driven by audio features
references/inputs.md § Audio Analysis
Generative
None (or seed params)
Procedural ASCII animation
references/effects.md
Hybrid
Video + audio
ASCII video with audio-reactive overlays
Both input refs
Lyrics/text
Audio + text/SRT
Timed text with visual effects
references/inputs.md § Text/Lyrics
TTS narration
Text quotes + TTS API
Narrated testimonial/quote video with typed text
references/inputs.md § TTS Integration
Stack
Single self-contained Python script per project. No GPU.
Layer
Tool
Purpose
Core
Python 3.10+, NumPy
Math, array ops, vectorized effects
Signal
SciPy
FFT, peak detection (audio modes only)
Imaging
Pillow (PIL)
Font rasterization, video frame decoding, image I/O
Generate narration clips for quote/testimonial videos
Optional
OpenCV
Video frame sampling, edge detection, optical flow
Pipeline Architecture (v2)
Every mode follows the same 6-stage pipeline. See references/architecture.md for implementation details, references/scenes.md for scene protocol, and references/composition.md for multi-grid composition and tonemap.
INPUT — Load/decode source material (video frames, audio samples, images, or nothing)
ANALYZE — Extract per-frame features (audio bands, video luminance/edges, motion vectors)
SCENE_FN — Scene function renders directly to pixel canvas (uint8 H,W,3). May internally compose multiple character grids via _render_vf() + pixel blend modes. See references/composition.md
TONEMAP — Percentile-based adaptive brightness normalization with per-scene gamma. Replaces linear brightness multipliers. See references/composition.md § Adaptive Tonemap
SHADE — Apply post-processing ShaderChain + FeedbackBuffer. See references/shaders.md
ENCODE — Pipe raw RGB frames to ffmpeg for H.264/GIF encoding
Creative Direction
Every project should look and feel different. The references provide a vocabulary of building blocks — don't copy them verbatim. Combine, modify, and invent.
Never hardcode worker counts, resolution, or CRF. Always detect and adapt.
Step 3: Build the Script
Write as a single Python file. Major components:
Hardware detection + quality profile — see references/optimization.md
Input loader — mode-dependent; see references/inputs.md
Feature analyzer — audio FFT, video luminance, or pass-through
Grid + renderer — multi-density character grids with bitmap cache; _render_vf() helper for value/hue field → canvas
Character palettes — multiple palettes chosen per project theme; see references/architecture.md
Color system — HSV + discrete RGB palettes as needed; see references/architecture.md
Scene functions — each returns canvas (uint8 H,W,3) directly. May compose multiple grids internally via pixel blend modes. See references/scenes.md + references/composition.md
Tonemap — adaptive brightness normalization with per-scene gamma; see references/composition.md
Shader pipeline — ShaderChain + FeedbackBuffer per-section config; see references/shaders.md
Scene table + dispatcher — maps time ranges to scene functions + shader/feedback configs; see references/scenes.md
Parallel encoder — N-worker batch clip rendering with ffmpeg pipes
Main — orchestrate full pipeline
Step 4: Handle Critical Bugs
Font Cell Height (macOS Pillow)
textbbox() returns wrong height. Use font.getmetrics():
Brightness — Use tonemap(), Not Linear Multipliers
ASCII on black is inherently dark. This is the #1 visual issue. Do NOT use linear * N brightness multipliers — they clip highlights and wash out the image. Instead, use the adaptive tonemap function from references/composition.md:
deftonemap(canvas, gamma=0.75):
"""Percentile-based adaptive normalization + gamma. Replaces all brightness multipliers."""
f = canvas.astype(np.float32)
lo = np.percentile(f, 1) # black point (1st percentile)
hi = np.percentile(f, 99.5) # white point (99.5th percentile)if hi - lo < 1: hi = lo + 1
f = (f - lo) / (hi - lo)
f = np.clip(f, 0, 1) ** gamma # gamma < 1 = brighter midsreturn (f * 255).astype(np.uint8)
Dense animated backgrounds — never flat black, always fill the grid
Vignette minimum clamped to 0.15 (not 0.12)
Bloom threshold lowered to 130 (not 170) so more pixels contribute to glow
Use screen blend mode (not overlay) when compositing dark ASCII layers — overlay squares dark values: 2 * 0.12 * 0.12 = 0.03
Font Compatibility
Not all Unicode characters render in all fonts. Validate palettes at init:
for c in palette:
img = Image.new("L", (20, 20), 0)
ImageDraw.Draw(img).text((0, 0), c, fill=255, font=font)
if np.array(img).max() == 0:
log(f"WARNING: char '{c}' (U+{ord(c):04X}) not in font, removing from palette")
Brightness verification: sample 5-10 frames across video, check mean > 8 for ASCII content.
References
File
Contents
references/architecture.md
Grid system, font selection, character palettes (library of 20+), color system (HSV + discrete RGB), _render_vf() helper, compositing, v2 effect function contract
references/inputs.md
All input sources: audio analysis, video sampling, image conversion, text/lyrics, TTS integration (ElevenLabs, voice assignment, audio mixing)
references/effects.md
Effect building blocks: 12 value field generators (vf_sinefield through vf_noise_static), 8 hue field generators (hf_fixed through hf_plasma), radial/wave/fire effects, particles, composing guide
