representation-engineering

name	representation-engineering
description	Use this skill when working with Representation Engineering (RepE) for AI transparency, monitoring, or controlling internal representations of large language models including truthfulness detection, emotion control, harmlessness steering, and memorization analysis.

Representation Engineering (RepE)

When to Use

Activate this skill when you need to:

• Monitor or manipulate internal representations of LLMs for transparency
• Detect or control truthfulness, honesty, or deception in language models
• Steer model behavior (emotions, fairness, harmlessness) using representation vectors
• Implement RepReading (classification via internal representations) or RepControl (generation steering)
• Analyze memorization or power-seeking behaviors in LLMs
• Use contrast vectors for safety-relevant interventions
• Build upon Hugging Face pipelines with representation-level control

Keywords: representation engineering, RepE, RepReading, RepControl, AI transparency, contrast vectors, LAT (Linear Artificial Tomography), honesty detection, emotion control, LLM steering, internal representations, cognitive neuroscience AI

Quick Reference

• Paper: https://arxiv.org/abs/2310.01405
• Website & Demo: https://www.ai-transparency.org/
• GitHub: https://github.com/andyzoujm/representation-engineering
• Examples directory: examples/ (honesty, emotions, fairness, memorization, harmless/harmful)
• HuggingFace pipelines docs: https://huggingface.co/docs/transformers/main_classes/pipelines

Installation

Prerequisites

• Python 3.8+
• PyTorch
• Hugging Face transformers

Install from GitHub

git clone https://github.com/andyzoujm/representation-engineering.git
cd representation-engineering
pip install -e .

Core Features

• RepReading Pipeline: Classifies internal representations across model layers using linear probes (PCA-based direction finding)
• RepControl Pipeline: Steers generation by injecting representation directions into model hidden states
• HuggingFace Integration: Both pipelines inherit from HuggingFace's pipeline API for compatibility
• RepE_eval Framework: Evaluation framework based on RepReading as an alternative to zero-shot/few-shot baselines
• Multi-concept Support: Honesty, emotions, fairness, memorization, harmlessness, power-seeking
• LoRRA Finetuning: Representation-aware finetuning support via lorra_finetune/
• Built-in Datasets: Emotions, facts, memorization data in data/

Usage Examples

Register Pipelines and Initialize

from repe import repe_pipeline_registry  # register 'rep-reading' and 'rep-control' tasks into Hugging Face pipelines
repe_pipeline_registry()

# ... initializing model and tokenizer ....

rep_reading_pipeline = pipeline("rep-reading", model=model, tokenizer=tokenizer)
rep_control_pipeline = pipeline("rep-control", model=model, tokenizer=tokenizer, **control_kwargs)

Honesty Detection Example (from examples/honesty)

from repe import repe_pipeline_registry
from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM
from examples.honesty.utils import honesty_function_dataset

repe_pipeline_registry()

model_name = "meta-llama/Llama-2-13b-chat-hf"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name, device_map="auto")

# Build dataset of honest vs dishonest statements
train_data, test_data = honesty_function_dataset(
    data_path="data/facts/facts_true_false.csv",
    tokenizer=tokenizer,
    user_tag="[INST]",
    assistant_tag="[/INST]"
)

rep_reading_pipeline = pipeline("rep-reading", model=model, tokenizer=tokenizer)

# Train a rep reader (linear probe) on honesty direction
rep_reader = rep_reading_pipeline.get_directions(
    train_data["data"],
    rep_token=-1,
    hidden_layers=list(range(-1, -model.config.num_hidden_layers, -1)),
    n_difference=1,
    train_labels=train_data["labels"],
    direction_method="pca",
)

Emotion Control Example

from examples.primary_emotions.utils import primary_emotions_concept_dataset

# Build emotion concept dataset
emotions_data = primary_emotions_concept_dataset(
    data_dir="data/emotions",
    user_tag="[INST]",
    assistant_tag="[/INST]"
)

# Use rep_control_pipeline to steer generation toward a target emotion
rep_control_pipeline = pipeline(
    "rep-control",
    model=model,
    tokenizer=tokenizer,
    layers=list(range(-5, -18, -1)),
    block_name="decoder_block"
)

Key APIs / Models

Pipeline Tasks

Task	Description
"rep-reading"	Read/classify internal representations
"rep-control"	Steer generation via representation injection

Supported Models (from examples)

• meta-llama/Llama-2-13b-chat-hf
• meta-llama/Llama-2-7b-chat-hf
• mistralai/Mistral-7B-Instruct-v0.1
• meta-llama/Meta-Llama-3-8B-Instruct
• Any HuggingFace CausalLM with decoder layers

Direction Methods

• "pca" — Principal Component Analysis (default, most common)
• "cluster_mean" — Cluster mean difference

Key Parameters for get_directions

• rep_token (int): Token position to extract representation from (e.g., -1 for last token)
• hidden_layers (list): Layer indices to extract from
• n_difference (int): Number of contrastive pairs
• train_labels (list): Labels for supervised direction finding
• direction_method (str): Method for finding direction ("pca")

RepControl Parameters

• layers (list): Layers at which to inject the control vector
• block_name (str): Name of the transformer block (e.g., "decoder_block")
• control_coeff (float): Coefficient scaling the control vector injection

Data Formats

Honesty Dataset (data/facts/facts_true_false.csv)

CSV with columns for statement text and true/false label.

Emotions Dataset (data/emotions/*.json)

JSON files per emotion with prompt-completion pairs.

Memorization Dataset

• data/memorization/quotes/ — Popular quotes, unseen quotes, completions
• data/memorization/literary_openings/ — Real vs. fake literary openings

Common Patterns & Best Practices

1. Tag Format: Always match user_tag/assistant_tag to your model's chat template (e.g., [INST]/[/INST] for LLaMA-2 chat).
2. Layer Selection: Typically use middle-to-later layers for semantic concepts; iterate over range(-1, -num_layers, -1).
3. PCA Direction: Use direction_method="pca" for robust concept directions; the sign of the direction may need flipping depending on dataset ordering.
4. Control Coefficient: Start with small values (e.g., ±10 to ±20) and tune; large values can degrade fluency.
5. Contrastive Pairs: Dataset should contain paired positive/negative examples for clean direction extraction.
6. RepE_eval: Use repe_eval/ as an additional evaluation baseline on standard benchmarks alongside zero-shot/few-shot.

Demo Scripts

scripts/repe_demo.py

#!/usr/bin/env python3
"""
Representation Engineering (RepE) Demo Script

Demonstrates how to use the RepReading and RepControl pipelines from the
`repe` library for monitoring and steering LLM internal representations.

Requires:
    pip install -e . (from repository root)
    pip install transformers torch accelerate

Usage:
    python repe_demo.py

Note: Set HF_TOKEN environment variable if accessing gated models like LLaMA-2.
      Replace MODEL_NAME with a model you have access to.
"""

import os
import json
from typing import List, Tuple, Dict, Any

# ---------------------------------------------------------------------------
# Step 1: Register RepE pipelines into HuggingFace
# ---------------------------------------------------------------------------
from repe import repe_pipeline_registry
repe_pipeline_registry()

from transformers import pipeline, AutoTokenizer, AutoModelForCausalLM
import torch


# ---------------------------------------------------------------------------
# Configuration — replace with your model and data paths
# ---------------------------------------------------------------------------
MODEL_NAME = "meta-llama/Llama-2-7b-chat-hf"  # or any causal LM you have access to
DATA_PATH = "data/facts/facts_true_false.csv"   # path relative to repo root
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

# Chat template tags for LLaMA-2-chat
USER_TAG = "[INST]"
ASSISTANT_TAG = "[/INST]"


def load_model_and_tokenizer(model_name: str):
    
    ()
    tokenizer = AutoTokenizer.from_pretrained(model_name, use_fast=)
    tokenizer.padding_side = 
     tokenizer.pad_token  :
        tokenizer.pad_token = tokenizer.eos_token

    ()
    model = AutoModelForCausalLM.from_pretrained(
        model_name,
        torch_dtype=torch.float16  torch.cuda.is_available()  torch.float32,
        device_map=,
    )
    model.()
    ()
     model, tokenizer


 () -> [, ]:
    
     csv

    honest_prefix = 
    dishonest_prefix = 

    data, labels = [], []

    :
         (data_path, )  f:
            reader = csv.DictReader(f)
            rows = (reader)
     FileNotFoundError:
        ()
        rows = [
            {: , : },
            {: , : },
            {: , : },
            {: , : },
        ] *   

     row  rows[:n_train]:
        statement = row.get(, row.get(, ))
        label = row.get(, )

        
        honest_prompt = (
            
            
            
        )

        
        dishonest_prompt = (
            
            
            
        )

        data.append([honest_prompt, dishonest_prompt])
        labels.append([, ])  

    split = ( * (data))
    train = {: data[:split], : labels[:split]}
    test = {: data[split:], : labels[split:]}
     train, test


 ():
    
    ()

    rep_reading_pipeline = pipeline(
        ,
        model=model,
        tokenizer=tokenizer,
    )

    num_layers = model.config.num_hidden_layers
    
    hidden_layers = ((-, -num_layers // , -))

    ()

    
    train_inputs = [item  pair  train_data[]  item  pair]
    train_labels = [label  pair  train_data[]  label  pair]

    rep_reader = rep_reading_pipeline.get_directions(
        train_inputs,
        rep_token=-,               
        hidden_layers=hidden_layers,
        n_difference=,
        train_labels=train_labels,
        direction_method=,     
        direction_kwargs={},
    )

    ()

    
    test_inputs = [pair[]  pair  test_data[][:]]  

    scores = rep_reading_pipeline(
        test_inputs,
        rep_token=-,
        hidden_layers=hidden_layers,
        rep_reader=rep_reader,
        batch_size=,
    )

    ()
     i, (text, score_dict)  ((test_inputs[:], scores[:])):
        avg_score = (score_dict.values()) / (score_dict)
        ()
        ()

     rep_reader, hidden_layers


 ():
    
    ()

    
    control_layers = hidden_layers[:]  

    activations = {}
     layer  control_layers:
        
        
         (rep_reader, )  layer  rep_reader.directions:
            direction = rep_reader.directions[layer]
            activations[layer] = torch.tensor(
                direction * ,  
                dtype=torch.float16  torch.cuda.is_available()  torch.float32,
            ).to(DEVICE)

      activations:
        ()
        ()
        

    rep_control_pipeline = pipeline(
        ,
        model=model,
        tokenizer=tokenizer,
        layers=control_layers,
        block_name=,
        control_method=,
    )

    test_prompt = (
        
    )

    ()
    ()

    :
        output_honest = rep_control_pipeline(
            test_prompt,
            activations=activations,
            max_new_tokens=,
            do_sample=,
        )
        ()
     Exception  e:
        ()

    ()
    :
        baseline_pipeline = pipeline(
            ,
            model=model,
            tokenizer=tokenizer,
        )
        output_baseline = baseline_pipeline(
            test_prompt,
            max_new_tokens=,
            do_sample=,
        )
        ()
     Exception  e:
        ()


 ():
    
    ()
    :
         sys
        sys.path.insert(, )
         utils  primary_emotions_concept_dataset, primary_emotions_function_dataset

        concept_data = primary_emotions_concept_dataset(
            data_dir=,
            user_tag=USER_TAG,
            assistant_tag=ASSISTANT_TAG,
        )
        ()
         emotion, samples  (concept_data.items())[:]:
            ()
             samples:
                ()

        function_data = primary_emotions_function_dataset(
            data_dir=,
            user_tag=USER_TAG,
            assistant_tag=ASSISTANT_TAG,
        )
        ()

     ImportError  e:
        ()
     FileNotFoundError  e:
        ()


 ():
    
    ( * )
    ()
    ( * )

    
    demo_emotion_dataset()

    
    run_model_demo = os.environ.get(, ) == 

     run_model_demo:
        model, tokenizer = load_model_and_tokenizer(MODEL_NAME)

        
        ()
        train_data, test_data = build_honesty_dataset(
            data_path=DATA_PATH,
            tokenizer=tokenizer,
            user_tag=USER_TAG,
            assistant_tag=ASSISTANT_TAG,
        )
        ()

        
        rep_reader, hidden_layers = demo_rep_reading(model, tokenizer, train_data, test_data)

        
        demo_rep_control(model, tokenizer, rep_reader, hidden_layers)
    :
        ()
        ()
        ()
        ()

    ()


 __name__ == :
    main()