relic-video-world-model

name	relic-video-world-model
title	RELIC: Interactive Video World Models with Long-Horizon Memory
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.04040
keywords	["world-models","video-generation","spatial-memory","robotics","long-horizon-planning"]
description	Compressed historical latents with camera poses in KV cache (4× compression), extended teacher training (20-second sequences), and replayed back-propagation (block-wise differentiation) enabling real-time interactive video generation with long-range spatial consistency.

Summary

RELIC is a unified framework for interactive video world modeling combining three innovations: memory-efficient spatial memory using highly compressed historical latents in KV cache, extended teacher training enabling 20-second sequence generation, and replayed back-propagation for memory-efficient distillation. Together these enable real-time 16 FPS video generation with precise action control and long-horizon consistency.

Core Technique

Compressed Spatial Memory: Store historical latents with camera pose information:

memory = [(latent_t1, camera_pose_t1), (latent_t2, camera_pose_t2), ...]

Compress latents via quantization and efficient encodings, achieving 4× compression.

Extended Teacher Training: Fine-tune base video model to generate longer sequences (20 seconds vs. 5 seconds), providing stronger supervision for long-range consistency learning.

Replayed Back-Propagation: Instead of computing gradients over full 20-second sequences (memory prohibitive), use block-wise differentiation:

# Full sequence: backward through all 400 frames
# Blocked: backward through 20-frame blocks independently

Implementation

Spatial memory structure:

class SpatialMemory:
    def __init__(self, max_history=100):
        self.latents = []  # Compressed feature maps
        self.camera_poses = []  # Camera extrinsics
        self.cache_size = 0

    def add(self, latent, pose):
        # Compress latent via quantization
        compressed = quantize(latent, bits=8)
        self.latents.append(compressed)
        self.camera_poses.append(pose)

    def retrieve(self, current_pose, k=10):
        
        scores = similarity(current_pose, .camera_poses)
        indices = topk(scores, k)
         .latents[indices]