| name | cv-sentence-transformer-target-encoding |
| description | Encode text prompts into fixed-length dense vectors using SentenceTransformer for cosine-similarity evaluation in image-to-text retrieval tasks |
SentenceTransformer Target Encoding
Overview
In image-to-prompt competitions, the target is not raw text but its SentenceTransformer embedding. Models predict embedding vectors and are scored by cosine similarity against the ground-truth prompt embeddings. Understanding this encoding step is essential: predictions must live in the same embedding space as the evaluation targets.
Quick Start
from sentence_transformers import SentenceTransformer
import numpy as np
st_model = SentenceTransformer("all-MiniLM-L6-v2")
prompts = ["a painting of a sunset over mountains", "cyberpunk city at night"]
embeddings = st_model.encode(prompts, normalize_embeddings=True)
similarity = np.dot(embeddings[0], embeddings[1])
Workflow
- Load a SentenceTransformer model matching the competition's evaluation model
- Encode all ground-truth prompts into dense vectors (the target)
- Any prediction pipeline must output vectors in this same space
- Score predictions via cosine similarity:
np.dot(pred, target) / (norm(pred) * norm(target))
- For submission, flatten the embedding matrix row-wise into a single column
Key Decisions
- Model choice: must match evaluation exactly โ
all-MiniLM-L6-v2 (384-dim) is common; check competition description
- Normalization: enable
normalize_embeddings=True to simplify cosine similarity to dot product
- Batch encoding: use
batch_size=32 for large datasets to avoid OOM
- Style keywords: appending style tokens ("fine details, masterpiece") to captions before encoding can bias embeddings toward the target distribution
- vs. CLIP text embeddings: SentenceTransformer captures semantic meaning; CLIP captures visual-textual alignment โ different spaces
References
