slotcurri-video-object-centric-learning

name	slotcurri-video-object-centric-learning
title	SlotCurri: Reconstruction-Guided Slot Curriculum for Video Object-Centric Learning
version	0.0.3
engine	skillxiv-v0.0.3-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.22758
keywords	["Slot Attention","Curriculum Learning","Video Object-Centric Learning","Over-Fragmentation","Progressive Allocation"]
description	Replace fixed full-capacity slot initialization with progressive curriculum-based slot expansion to reduce over-fragmentation in video object-centric learning. Improves FG-ARI by +6.8 on YouTube-VIS and +8.3 on MOVi-C. Use when training slot attention models on videos with variable object counts and sizes.
category	Component Innovation

What This Skill Does

Replace the standard practice of allocating all slot attention slots upfront with a progressive curriculum that starts with minimal slots and expands them reconstruction-guided. This eliminates over-fragmentation where single objects split across multiple slots, improving object discovery by +6.8 to +8.3 FG-ARI on real-world video datasets.

The Component Swap

The old SlotContrast baseline initializes all K slots at training start, forcing the model to use all slots regardless of whether the scene contains K objects:

# Old approach: allocate all slots upfront
K = 10  # Fixed capacity
slots = nn.Parameter(torch.randn(1, K, slot_dim))
# Model pressured to fragment objects across slots to minimize loss

The new SlotCurri approach starts with minimal slots and progressively expands them through curriculum stages. Slots are spawned reconstruction-guided from high-error regions:

# New: progressive slot curriculum
K_init = 2  # Start small
K_max = 10  # Final capacity
num_stages = 3

# Stage-dependent slot count: K^(m) = K_init + m*sigma + 3m(m-1)/2
def slot_count_schedule(stage_m, sigma=1):
    return K_init + stage_m * sigma + 3 * stage_m * (stage_m - 1) // 2

# Slot spawning: duplicate from high-reconstruction-error parents
reconstruction_error = ||video_recon - video_real||_2  # Per-slot error
high_error_slots = torch.argsort(reconstruction_error)[:top_k]
new_slots = slots[high_error_slots] + noise_perturbation(scale=0.1)

slots = torch.cat([slots, new_slots], dim=1)  # Expand capacity

Augment the reconstruction loss with structure-aware SSIM to better capture object boundaries:

# Old loss: pixel-level MSE only
loss_old = F.mse_loss(video_recon, video_real)


loss_mse = F.mse_loss(video_recon, video_real)
loss_ssim =  - ssim(video_recon, video_real, window_size=)
loss_new = loss_mse +  * loss_ssim