task-vectors

name	task-vectors
description	Use this skill when working with task arithmetic for editing neural network models, including creating task vectors from pre-trained and fine-tuned checkpoints, combining them via arithmetic operations (negation, addition, analogies), and applying them to CLIP vision models for multi-task learning.

Task Vectors — Editing Models with Task Arithmetic

When to Use

Activate this skill when you need to:

• Edit pre-trained model behavior without retraining from scratch
• Combine multiple fine-tuned models into a single multi-task model
• Negate unwanted capabilities or biases from a pre-trained model
• Perform task analogies across model weight spaces
• Work with CLIP ViT-B/32, ViT-B/16, or ViT-L/14 checkpoints
• Apply parameter-space arithmetic to neural network weights

Trigger keywords: task vectors, task arithmetic, model editing, weight space, fine-tuned checkpoints, negation, model merging, multi-task model, CLIP editing

Quick Reference

• Paper: Editing Models with Task Arithmetic (ICLR 2023)
• Repository: https://github.com/mlfoundations/task_vectors
• Checkpoints: Google Drive
• Core module: src/task_vectors.py

Installation / Setup

Prerequisites

• conda (Anaconda or Miniconda)
• Python (version managed by conda environment)

Step 1: Create and activate conda environment

conda env create
conda activate task-vectors

Step 2: Add source directory to PYTHONPATH

cd task_vectors
export PYTHONPATH="$PYTHONPATH:$PWD"

Step 3: Download checkpoints

Download CLIP ViT-B/32, ViT-B/16, and ViT-L/14 checkpoints (pre-trained zero-shot + fine-tuned on 8 tasks) from: https://drive.google.com/drive/folders/1u_Tva6x0p6oxu5Eo0ZZsf-520Cc_3MKw?usp=share_link

Expected checkpoint layout:

Demo Scripts

scripts/task_vectors_demo.py

#!/usr/bin/env python3
"""
Task Vectors Demo: Editing Models with Task Arithmetic

This script demonstrates how to use the task_vectors library to:
1. Create task vectors from pre-trained and fine-tuned CLIP checkpoints
2. Negate a task vector to degrade performance on a specific task
3. Add multiple task vectors together for multi-task performance
4. Perform task analogies

Requirements:
- conda activate task-vectors
- export PYTHONPATH="$PYTHONPATH:/path/to/task_vectors"
- Download checkpoints from:
  https://drive.google.com/drive/folders/1u_Tva6x0p6oxu5Eo0ZZsf-520Cc_3MKw

Usage:
    python task_vectors_demo.py --model ViT-L-14 --data_location /path/to/data
"""

import sys
import os

# ---------------------------------------------------------------------------
# NOTE: Add the src/ directory to the path before importing task_vectors.
# Adjust this path to match your local clone of mlfoundations/task_vectors.
# ---------------------------------------------------------------------------
REPO_SRC = os.path.join(os.path.dirname(__file__), "..", "src")
if os.path.isdir(REPO_SRC):
    sys.path.insert(0, os.path.abspath(REPO_SRC))


def demo_create_task_vector(
    pretrained_checkpoint: str,
    finetuned_checkpoint: str,
):
    """
    Demonstrate creating a TaskVector from two checkpoints.

    Args:
        pretrained_checkpoint: Path to the pre-trained (zero-shot) .pt file.
        finetuned_checkpoint:  Path to the fine-tuned .pt file.

    Returns:
        TaskVector instance.
    """
    from task_vectors import TaskVector  # noqa: PLC0415  (lazy import after path setup)

    print(f"[1] Creating task vector from:\n"
          f"    pretrained : {pretrained_checkpoint}\n"
          f"    finetuned  : {finetuned_checkpoint}")

    task_vector = TaskVector(pretrained_checkpoint, finetuned_checkpoint)
    ()
     task_vector


 ():
    
    ()
    neg_task_vector = -task_vector
    image_encoder = neg_task_vector.apply_to(pretrained_checkpoint, scaling_coef=scaling_coef)
    ()
     image_encoder


 ():
    
     task_vectors  TaskVector  

    ()

    task_vectors = [
        TaskVector(pretrained_checkpoint, ckpt)
         ckpt  finetuned_checkpoints.values()
    ]

    
    task_vector_sum = (task_vectors)
    ()

    image_encoder = task_vector_sum.apply_to(pretrained_checkpoint, scaling_coef=scaling_coef)
    ()
     image_encoder


 ():
    
     task_vectors  TaskVector  

    ()
    tv_a = TaskVector(pretrained_checkpoint, checkpoint_a)
    tv_b = TaskVector(pretrained_checkpoint, checkpoint_b)
    tv_c = TaskVector(pretrained_checkpoint, checkpoint_c)

    analogy_vector = tv_c + tv_b - tv_a
    image_encoder = analogy_vector.apply_to(pretrained_checkpoint, scaling_coef=scaling_coef)
    ()
     image_encoder


 ():
    
       eval_single_dataset  

    ()
    metrics = eval_single_dataset(image_encoder, dataset_name, args)
    ()
     metrics


 ():
    
    
    
    
    CHECKPOINT_DIR =    
    DATA_DIR       =            
    MODEL          =                
    DATASETS       = [, ]
    

    pretrained = os.path.join(CHECKPOINT_DIR, MODEL, )
    finetuned  = {
        ds: os.path.join(CHECKPOINT_DIR, MODEL, ds, )
         ds  DATASETS
    }

    
      os.path.exists(pretrained):
        ( * )
        ()
        ()
        ( * )
        _run_structural_demo()
        

    
    :
         args  parse_arguments  
        args = parse_arguments()
        args.data_location = DATA_DIR
        args.model = MODEL
        args.save  = os.path.join(CHECKPOINT_DIR, MODEL)
     Exception  exc:
        ()
        

    
    tv_mnist = demo_create_task_vector(pretrained, finetuned[])

    
    neg_encoder = demo_negate_task_vector(tv_mnist, pretrained, scaling_coef=)
    demo_evaluate(neg_encoder, , args)
    demo_evaluate(neg_encoder, , args)

    
    multi_encoder = demo_add_task_vectors(pretrained, finetuned, scaling_coef=)
     ds  DATASETS:
        demo_evaluate(multi_encoder, ds, args)


 ():
    
    ()

    
     :
        

         ():
            .name = name
            .vector = {: }

         ():
            result = MockVector()
             result

         () -> :
             MockVector()

         ():
            
             other == :
                 
             .__add__(other)

         () -> :
             .__add__(-other)

         ():
            ()
            ()
             

         ():
             

    tv_a = MockVector()
    tv_b = MockVector()
    tv_c = MockVector()

    ()
    ()
    ()
    ()

    (, -tv_a)
    (, tv_a + tv_b)
    (, ([tv_a, tv_b, tv_c]))
    (, tv_c + tv_b - tv_a)

    ()
    neg_tv = -tv_a
    neg_tv.apply_to(, scaling_coef=)

    ()
    summed = ([tv_a, tv_b])
    summed.apply_to(, scaling_coef=)

    ()


 __name__ == :
    main()