shamisa-self-supervised-image-quality

name	shamisa-self-supervised-image-quality
title	SHAMISA: Self-Supervised No-Reference Image Quality Assessment via Structured Relational Supervision
version	0.0.3
engine	skillxiv-v0.0.3-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.13669
keywords	["Self-Supervised Learning","Image Quality Assessment","Graph-Weighted Invariance","Contrastive Learning","No-Reference"]
description	Replace standard VICReg invariance loss with graph-weighted learnable adjacency matrix to enable self-supervised no-reference image quality assessment without human labels. Improves SRCC by +0.017 (2% relative) on six-dataset average and shows stronger cross-dataset transfer. Use when training quality assessment models without paired quality labels.
category	Component Innovation

What This Skill Does

Replace the binary-constraint invariance term in VICReg (self-supervised learning framework) with a graph-weighted formulation using learnable soft adjacency. This enables self-supervised no-reference IQA using only a compositional distortion engine and dual-source relation graphs, improving generalization by 2% relative without requiring human quality annotations.

The Component Swap

The old VICReg uses a rigid binary set of augmentation-positive pairs with fixed relationships:

# Old: standard VICReg invariance (fixed positive pairs)
# A = set of fixed positive pairs from data augmentation
# Loss = Σ_(i,j)∈A ||Z_i - Z_j||²₂

invariance_loss_old = 0
for i, j in positive_pairs:  # Binary: (i,j) are paired or not
    invariance_loss_old += torch.norm(embeddings[i] - embeddings[j])**2

The new SHAMISA approach replaces fixed binary constraints with learnable weighted adjacency:

# New: graph-weighted VICReg invariance (soft adjacency)
# G = learnable adjacency matrix in [0,1]^(N×N)
# Loss = Σᵢ Σⱼ G_{i,j} ||Z_i - Z_j||²₂

# Learnable adjacency matrix
G = torch.nn.Parameter(
    torch.ones(batch_size, batch_size) * 0.5  # Initialize to uniform
)
G.data = torch.clamp(G, min=0, max=1)  # Constrain to [0,1]

# Weighted invariance loss
invariance_loss_new = 0
for i in range(batch_size):
    for j in range(batch_size):
        weight = G[i, j]  # Soft weight, learnable
        invariance_loss_new += weight * torch.norm(embeddings[i] - embeddings[j])**2

The graph G is constructed from dual relation sources that capture quality-relevant structure: