| name | canvit-active-vision-foundation-model |
| title | CanViT: The First Active-Vision Foundation Model |
| version | 0.0.3 |
| engine | skillxiv-v0.0.3-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2603.22570 |
| keywords | ["Active Vision","Foundation Models","Retinotopic Vision Transformer","Canvas Attention","Selective Attention"] |
| description | Establishes Active-Vision Foundation Models (AVFM) as a new problem class and proposes CanViT: a retinotopic ViT backbone with Canvas Attention that decouples thinking (glimpse processing) from memory (scene canvas). Dense latent distillation from DINOv3 enables unsupervised pretraining on 1B random glimpses. Achieves 81.2% ImageNet accuracy with frozen probes—proving foundation models can be adapted to active-vision tasks. Trigger: When building systems requiring selective visual attention to scenes (robotics, surveillance, embodied AI), apply the active-vision paradigm with retinotopic architecture and canvas memory to scale beyond single-glimpse models. |
| category | Field Foundation |
The Problem Statement
What is this problem?
Active vision—the problem of intelligently selecting where to look in an image or scene—has historically been studied as a specialized subtask, separate from general vision foundation models. Most vision models (ResNets, ViTs, DINOv2) process entire images uniformly, treating the viewing problem as solved. But in robotics, surveillance, or embodied AI, agents must learn how and where to look given computational constraints.
Before CanViT, no scalable, general-purpose architecture existed for active vision that could:
- Learn from diverse viewpoints (not task-specific gaze patterns)
- Adapt to new tasks via transfer learning (foundation model paradigm)
- Scale to large scenes without memory explosion
- Work across embodied AI domains (robotics, surveillance, navigation)
Why is this important?
Most real-world vision systems are resource-constrained: robotics arms, mobile agents, or edge deployment. Selective attention is fundamental—you can't process high-resolution images of entire environments in real-time. Yet active vision research was fragmented, domain-specific, and lacked unified foundation models.
CanViT establishes that foundation models can be built for active vision, opening a new research direction with broad applicability.
What existing approaches are inadequate?
- Standard ViTs/CNNs: Process full images, waste computation on irrelevant regions
- Policy-specific active vision models: Hand-designed for particular tasks (object detection, semantic segmentation); don't transfer
- Attention mechanisms: Soft spatial attention, but don't model sequential eye movements or sampling patterns
- No foundation model paradigm: Active vision lacked the pretraining scale and transfer-learning infrastructure of generic vision
The Paradigm Shift: Active-Vision Foundation Models
What new way of thinking does CanViT introduce?
Active-Vision Foundation Models (AVFM) reframe vision as a sequential decision problem: given computational constraints, what regions should the model examine, and in what order? Foundation models enable transfer of learned visual-attentional strategies to new domains.
Key conceptual innovations:
-
Retinotopic representation: Process visual input as a sequence of foveated glimpses, mimicking biological vision. Each glimpse is high-resolution near a fixation point, lower-resolution periphery.
