prism-hypothesis

name	prism-hypothesis
title	The Prism Hypothesis: Harmonizing Semantic and Pixel Representations
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.19693
keywords	["vision","representation-learning","semantic","pixel-fidelity","tokenization"]
description	Unify semantic understanding and pixel-level detail in a single representation by decomposing features into frequency bands. Low frequencies encode semantics while high frequencies capture pixels—enabling one tokenizer for both understanding and generation through frequency-based modulation and semantic-wise alignment.

Overview

The Prism Hypothesis addresses the fundamental tension between semantic abstraction and pixel fidelity in foundation models. Rather than maintaining separate semantic and pixel encoders that conflict, this approach uses a unified frequency-decomposed representation where different frequency bands serve different purposes.

Core Technique

The key insight is that different frequency components naturally separate semantic and pixel concerns.

Frequency-Based Decomposition: Data modalities are viewed as projections onto a shared feature spectrum, where semantic meaning lives in low frequencies and fine details in high frequencies.

# Frequency-based latent decomposition
import numpy as np

class UnifiedAutoencoder:
    def __init__(self, num_bands=4):
        self.num_bands = num_bands

    def decompose_spectrum(self, features):
        """
        Split latent representations into K frequency bands
        using FFT-based projection.
        """
        # FFT to frequency domain
        fft_features = np.fft.fft(features, axis=-1)

        # Divide into bands
        bands = []
        band_size = fft_features.shape[-1] // self.num_bands
        for i in range(self.num_bands):
            start = i * band_size
            end = (i + 1) * band_size
            band = fft_features[..., start:end]
            bands.append(band)

        return bands  # Low freq → semantic, High freq → pixel detail

Unified Autoencoding (UAE) Architecture: Three mechanisms process multi-band latents:

# Multi-stage frequency processing pipeline
class :
     ():
        .noise_injection = NoiseLayer()
        .spectral_transforms = SpectralTransformBlocks()
        .decoder = Decoder()

     ():
        
        
        bands = [.noise_injection(b)  b  bands]

        
        transformed = [.spectral_transforms(b)  b  bands]

        
        output = .decoder(transformed)
         output