Method to identify and manipulate interpretable directions in model activation space corresponding to personality traits. Enables real-time monitoring of trait drift and preventive steering to mitigate unwanted behavioral shifts in language models.
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Um comando direto ignora o prompt de revisão. Verifique a origem antes de executá-lo.
Method to identify and manipulate interpretable directions in model activation space corresponding to personality traits. Enables real-time monitoring of trait drift and preventive steering to mitigate unwanted behavioral shifts in language models.
Persona Vectors: Monitoring and Controlling LLM Character Traits
Persona Vectors provide a novel approach to understanding and controlling language model behavior by identifying interpretable directions in activation space that correspond to specific personality traits. This technique enables both monitoring and controlled modification of model behavior at scale, addressing critical safety and alignment challenges.
Core Concept
The fundamental insight is that language models develop consistent personality traits (e.g., honesty, helpfulness, harmfulness) that manifest as coherent directions in their hidden activation space. By identifying these "persona vectors," practitioners can:
Monitor personality drift during deployment by tracking projections onto trait vectors
Predict behavioral shifts before they occur during training
Apply steering interventions to prevent undesirable personality changes
Flag problematic training data that induces unwanted trait shifts
Control model behavior through post-hoc activation manipulation
Architecture Overview
The persona vector framework consists of:
Trait Description Collection: Natural language descriptions of personality traits
Activation Space Mapping: Extraction of hidden layer activations for trait-labeled examples
"""
Register a personality trait with natural language description.
Args:
trait_name: e.g., "harmfulness", "honesty", "sycophancy"
description: What this trait means and how it manifests
"""
"""
Add an example with trait intensity score (0-1).
Args:
trait_name: Which trait this example demonstrates
text: Model output or example text
intensity: 0=trait absent, 1=extreme trait manifestation
"""
"""
Create pairs of examples with low vs high trait intensity.
These pairs train the direction identification.
"""
self
for
in
if
for
in
if
# Pair each high-intensity example with a random low-intensity example
for
in
for
in
return
This creates the training data for identifying trait vectors from naturally labeled examples.
Step 2: Extract activations and identify trait vectors
Extract hidden layer activations and compute the direction that maximizes trait variation:
classPersonaVectorExtractor:
"""Identifies personality trait directions in activation space"""def__init__(self, model, layer_name: str = "transformer.h.10"):
self.model = model
self.layer_name = layer_name # Which layer to extract fromself.hook_handle = Nonedefextract_activations(self, text: str, token_idx: int = -1) -> np.ndarray:
"""
Extract hidden activations from specified layer for given text.
Args:
text: Input text to process
token_idx: Which token position to extract (-1 for last token)
Returns:
Activation vector of shape (hidden_size,)
"""
activations = Nonedefhook_fn(module, input, output):
nonlocal activations
# output is (batch_size, seq_len, hidden_size)
activations = output[0, token_idx, :].detach().cpu().numpy()
# Register forward hookfor name, module inself.model.named_modules():
if name == self.layer_name:
self.hook_handle = module.register_forward_hook(hook_fn)
break# Forward passwith torch.no_grad():
self.model(text)
# Remove hookifself.hook_handle:
self.hook_handle.remove()
return activations
defcompute_trait_vector_pca(self, low_texts: List[str],
high_texts: List[str]) -> np.ndarray:
"""
Compute trait vector as primary direction of variation
between low and high trait intensity examples.
Uses PCA: fits PCA to centered difference vectors.
"""
low_activations = np.array([self.extract_activations(text) for text in low_texts])
high_activations = np.array([self.extract_activations(text) for text in high_texts])
# Center and compute mean difference
low_mean = low_activations.mean(axis=0)
high_mean = high_activations.mean(axis=0)
mean_diff = high_mean - low_mean
# Combine for PCA: all examples centered at origin
combined = np.vstack([
low_activations - low_mean,
high_activations - high_mean
])
# PCA: first component is trait direction
cov = combined.T @ combined
eigvals, eigvecs = np.linalg.eigh(cov)
# Largest eigenvalue corresponds to trait direction
trait_vector = eigvecs[:, -1]
# Ensure it points toward high intensityif np.dot(trait_vector, mean_diff) < 0:
trait_vector = -trait_vector
return trait_vector
defcompute_trait_vector_contrastive(self, low_texts: List[str],
high_texts: List[str]) -> np.ndarray:
"""
Alternative: compute trait vector as regression direction
that separates high from low intensity examples.
"""
low_acts = np.array([self.extract_activations(text) for text in low_texts])
high_acts = np.array([self.extract_activations(text) for text in high_texts])
# Simple approach: maximize difference magnitude using least squares
X = np.vstack([low_acts, high_acts])
y = np.hstack([np.zeros(len(low_acts)), np.ones(len(high_acts))])
# Solve: min ||Xw - y||^2
w = np.linalg.lstsq(X, y, rcond=None)[0]
w = w / np.linalg.norm(w) # Normalizereturn w
These methods identify vectors pointing toward trait manifestation in activation space.
Step 3: Implement monitoring and drift detection
Track how trait vectors evolve during training or deployment:
classTraitMonitor:
"""Monitors personality trait drift in deployed models"""def__init__(self, trait_vectors: Dict[str, np.ndarray],
baseline_projections: Dict[str, float]):
self.trait_vectors = trait_vectors
self.baseline_projections = baseline_projections
self.history: List[Dict[str, float]] = []
defmeasure_traits(self, text: str, extractor: PersonaVectorExtractor) -> Dict[str, float]:
"""
Measure trait intensities for given output.
Args:
text: Model output to evaluate
extractor: PersonaVectorExtractor instance
Returns:
Dict mapping trait names to projection magnitudes (0-1)
"""
activation = extractor.extract_activations(text)
projections = {}
for trait_name, trait_vector inself.trait_vectors.items():
# Project activation onto trait vector
projection = np.dot(activation, trait_vector)
# Normalize relative to baseline
baseline = self.baseline_projections.get(trait_name, 0.0)
normalized = (projection - baseline) / (np.linalg.norm(trait_vector) + 1e-8)
projections[trait_name] = normalized
return projections
defdetect_drift(self, measurements: Dict[str, float],
threshold: float = 0.2) -> List[str]:
"""
Detect if traits have drifted significantly from baseline.
Returns:
List of traits that have drifted beyond threshold
"""
drifted_traits = []
for trait_name, projection in measurements.items():
baseline = self.baseline_projections.get(trait_name, 0.0)
drift_magnitude = abs(projection - baseline)
if drift_magnitude > threshold:
drifted_traits.append(trait_name)
return drifted_traits
deflog_measurement(self, measurements: Dict[str, float],
timestamp: str = None):
"""Record trait measurements for trend analysis"""
entry = {"timestamp": timestamp or"now", **measurements}
self.history.append(entry)
This enables real-time monitoring of personality shifts.
Step 4: Implement steering interventions
Apply controlled activation manipulations to shift traits:
classPersonaSteering:
"""Applies activation interventions to control personality traits"""def__init__(self, model, trait_vectors: Dict[str, np.ndarray],
layer_name: str = "transformer.h.10"):
self.model = model
self.trait_vectors = trait_vectors
self.layer_name = layer_name
self.steering_strength = {} # Map trait -> intervention strengthdefset_steering_strength(self, trait_name: str, strength: float):
"""
Configure how strongly to steer toward/away from trait.
Args:
trait_name: Which trait to control
strength: Positive to increase trait, negative to decrease
"""self.steering_strength[trait_name] = strength
defcreate_steering_hook(self):
"""
Create a forward hook that applies trait steering during inference.
"""
trait_vectors = self.trait_vectors
steering_strength = self.steering_strength
defhook_fn(module, input, output):
# output: (batch_size, seq_len, hidden_size)
modified_output = output.clone()
for i inrange(output.size(0)): # For each example in batchfor j inrange(output.size(1)): # For each token
activation = output[i, j, :]
# Apply steering for each traitfor trait_name, trait_vector in trait_vectors.items():
strength = steering_strength.get(trait_name, 0.0)
ifabs(strength) > 1e-6:
# Shift activation along trait vector
trait_vec_tensor = torch.from_numpy(trait_vector).float()
shift = strength * trait_vec_tensor.to(activation.device)
modified_output[i, j, :] = activation + shift
return modified_output
return hook_fn
defapply_steering(self, generate_fn, prompt: str,
steering_config: Dict[str, float]) -> str:
"""
Generate text with steering applied.
Args:
generate_fn: Model's generation function
prompt: Input prompt
steering_config: Dict mapping trait names to steering strengths
Returns:
Generated text with trait steering applied
"""self.steering_strength = steering_config
# Register hook
hook_handle = Nonefor name, module inself.model.named_modules():
if name == self.layer_name:
hook_handle = module.register_forward_hook(self.create_steering_hook())
break# Generate with steering
output = generate_fn(prompt)
# Remove hookif hook_handle:
hook_handle.remove()
return output
This enables post-hoc control of model behavior through activation manipulation.
Step 5: Data curation via trait-based filtering
Identify problematic training examples by detecting unexpected trait shifts:
classTrainingDataCurator:
"""Identifies problematic training examples based on trait shifts"""def__init__(self, trait_vectors: Dict[str, np.ndarray],
extractor: PersonaVectorExtractor):
self.trait_vectors = trait_vectors
self.extractor = extractor
defdetect_anomalous_examples(self, training_examples: List[str],
expected_trait_profile: Dict[str, float],
anomaly_threshold: float = 0.5) -> List[Tuple[str, str]]:
"""
Find training examples that induce unexpected trait shifts.
Args:
training_examples: List of training texts
expected_trait_profile: Expected trait intensities for this dataset
anomaly_threshold: How different from expected to flag
Returns:
List of (example, anomalous_trait) tuples
"""
flagged = []
for example in training_examples:
measurements = {}
for trait_name, trait_vec inself.trait_vectors.items():
activation = self.extractor.extract_activations(example)
projection = np.dot(activation, trait_vec)
measurements[trait_name] = projection
# Compare to expected profilefor trait_name, expected_value in expected_trait_profile.items():
if trait_name in measurements:
diff = abs(measurements[trait_name] - expected_value)
if diff > anomaly_threshold:
flagged.append((example, trait_name))
return flagged
This enables filtering of training data that would introduce undesired personality shifts.
Practical Guidance
When to use Persona Vectors:
Monitoring production LLM systems for personality drift
Preventing specific undesirable traits (harmfulness, dishonesty, sycophancy)
Curating training data to maintain consistent model personality
Fine-tuning systems where trait control is important
