ovie-monocular-novel-view-synthesis

name	ovie-monocular-novel-view-synthesis
title	OVIE: Monocular Novel-View Synthesis via Unpaired Internet Images
version	0.0.3
engine	skillxiv-v0.0.3-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.23488
keywords	["Novel-View Synthesis","Monocular Depth","Unpaired Images","Geometry-Free Inference","3D Scaffolding"]
description	A single insight eliminates multi-view requirements for novel-view synthesis: monocular depth acts as a training-time geometric scaffold to generate synthetic view pairs from unpaired internet images, but can be discarded at inference. This reframes the problem from needing paired multi-view data to leveraging abundant 2D internet imagery. Trigger: When limited to monocular video or single-image novel-view synthesis, use depth as training scaffold on unpaired data—the model learns geometry without needing it at inference.
category	Insight-Driven Papers

The Breakthrough Insight

The observation: Monocular depth estimation can act as a training-time geometric scaffold to generate synthetic view pairs from single unpaired images, but the trained model works without depth at inference—eliminating multi-view training data requirements.

Why this matters: Previous novel-view synthesis required multi-view dataset collection (expensive, limited in diversity). The insight reveals that depth can generate pseudo-views from unpaired internet images during training, transforming a 30-million-image internet corpus into synthetic paired data. At inference, depth is discarded—geometry is baked into the model.

Why Was This Hard?

Novel-view synthesis traditionally required paired multi-view data: multiple cameras capturing the same scene. This is expensive to collect and limits dataset diversity. Unsupervised approaches existed, but required complex geometric assumptions or were inefficient.

The hidden assumption was that multi-view structure was necessary throughout the pipeline. But the authors discovered depth can be a temporary tool: generate pairs during training, discard at inference. This insight requires recognizing that the model learns geometry sufficiently to extrapolate without explicit depth at inference time.

Why nobody discovered this before: The idea of using depth asymmetrically—only during training—wasn't standard. Most works either use depth throughout or abandon it entirely. The possibility of training with depth but discarding it for inference wasn't obvious.

How the Insight Reframes the Problem

Before the insight:

• Problem seemed to require: Multi-view video captures with camera calibration
• Bottleneck was: Expensive paired multi-view data collection
• Complexity was at: Learning geometry from limited paired views

After the insight:

• Problem reduces to: Run monocular depth on unpaired internet images to generate synthetic pairs, train on those pairs
• Bottleneck moves to: Quality of monocular depth estimation (errors propagate to training)
• New framing enables: Training on 30 million unpaired internet images instead of thousands of paired captures

Shift type: Data-property insight. The paper discovered that monocular depth provides sufficient signal to generate realistic synthetic pairs from unpaired images, unlocking training on large-scale unstructured data.

Minimal Recipe

The key approach uses depth as a training-time scaffold:

# Monocular depth generates synthetic paired views from unpaired images.
# Visibility masking handles disocclusions during view synthesis.


 ():
    
     image  unpaired_internet_images:
        
        depth = monocular_depth_estimator(image)

        
        
        camera_transform = sample_random_camera_pose()
        pseudo_target = project_with_camera(image, depth, camera_transform)

        
        valid_mask = compute_visibility_mask(depth, camera_transform)

        
        
        train_step(image, pseudo_target, valid_mask)

 ():
    
    novel_view = model.forward(source_image, target_camera_pose)
     novel_view