Use this skill when working with task arithmetic for editing neural network models, including creating task vectors from pre-trained and fine-tuned checkpoints, combining them via arithmetic operations (negation, addition, analogies), and applying them to CLIP vision models for multi-task learning.
Use this skill when working with task arithmetic for editing neural network models, including creating task vectors from pre-trained and fine-tuned checkpoints, combining them via arithmetic operations (negation, addition, analogies), and applying them to CLIP vision models for multi-task learning.
Task Vectors — Editing Models with Task Arithmetic
When to Use
Activate this skill when you need to:
Edit pre-trained model behavior without retraining from scratch
Combine multiple fine-tuned models into a single multi-task model
Negate unwanted capabilities or biases from a pre-trained model
Perform task analogies across model weight spaces
Work with CLIP ViT-B/32, ViT-B/16, or ViT-L/14 checkpoints
Apply parameter-space arithmetic to neural network weights
Trigger keywords: task vectors, task arithmetic, model editing, weight space, fine-tuned checkpoints, negation, model merging, multi-task model, CLIP editing
"""
Demonstrate negating a task vector and applying it to a pre-trained model.
Negating a task vector decreases performance on the target task while
having little effect on other tasks.
Args:
task_vector: A TaskVector instance.
pretrained_checkpoint: Path to the pre-trained (zero-shot) .pt file.
scaling_coef: Scaling coefficient controlling the step size (0–1).
Returns:
Image encoder with the negated task vector applied.
"""
"""
Demonstrate adding multiple task vectors and applying the sum.
Adding task vectors combines capabilities so the resulting model
performs well on all included tasks simultaneously.
Args:
pretrained_checkpoint: Path to the pre-trained (zero-shot) .pt file.
finetuned_checkpoints: Dict mapping dataset name -> finetuned .pt path.
scaling_coef: Scaling coefficient (0–1).
Returns:
Image encoder with summed task vector applied.
"""
from
import
# noqa: PLC0415
print
f"\n[3] Adding task vectors for datasets: {list(finetuned_checkpoints.keys())}"
for
in
# Use Python's built-in sum (relies on TaskVector.__add__ and __radd__)
sum
print
f" Combined {len(task_vectors)} task vectors via sum()."
print
f" Sum task vector applied (scaling_coef={scaling_coef})."
"""
Demonstrate a task analogy: new_vector = C + B - A.
If 'A is to B as C is to D', combining three task vectors can improve
performance on the fourth task D without any training data for D.
Args:
pretrained_checkpoint: Path to the pre-trained (zero-shot) .pt file.
checkpoint_a: Fine-tuned checkpoint for task A.
checkpoint_b: Fine-tuned checkpoint for task B.
checkpoint_c: Fine-tuned checkpoint for task C.
scaling_coef: Scaling coefficient (0–1).
Returns:
Image encoder with the analogy task vector applied.
"""
"""
Evaluate an image encoder on a given dataset using the repo's eval module.
Args:
image_encoder: The modified image encoder returned by apply_to().
dataset_name: Name of the dataset (e.g., 'MNIST', 'ImageNet').
args: Parsed argument namespace (from parse_arguments()).
"""
from
eval
import
# noqa: PLC0415
print
f"\n[5] Evaluating on dataset: {dataset_name}"
print
f" Results on {dataset_name}: {metrics}"
return
def
main
"""
Run a full task vector workflow using placeholder checkpoint paths.
To run end-to-end:
1. Download checkpoints from the Google Drive link in README.
2. Set CHECKPOINT_DIR and DATA_DIR below.
3. Run: python task_vectors_demo.py
"""
