safety-at-one-shot-lm-repair

name	safety-at-one-shot-lm-repair
title	Safety at One Shot: Patching Fine-Tuned LLMs with A Single Instance
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2601.01887
keywords	["LLM Safety","Model Alignment","Efficient Fine-Tuning","Low-Rank Adaptation"]
description	Recover safety alignment in fine-tuned LLMs using only a single safety example, without sacrificing utility. Leverages low-rank structure of safety gradients to enable minimal-cost correction even when models have been compromised by extensive harmful training data.

When to Use This Skill

• Recovering safety in accidentally fine-tuned models
• Rapid safety patching when new safety issues emerge
• Minimal-resource safety corrections in resource-constrained settings
• Multi-model safety updates across model portfolios
• Scenarios where full retraining is impractical

When NOT to Use This Skill

• Initial model development (use comprehensive safety training instead)
• Situations requiring extensive safety certification
• Adversarially compromised models (one example insufficient)

Problem Summary

Fine-tuning safety-aligned LLMs on domain data can substantially compromise safety properties. Prior solutions required extensive safety training data (hundreds of examples) and computational overhead, often degrading model utility. This creates a dilemma: practitioners need practical safety corrections without rebuilding entire models or sacrificing performance on useful tasks.

Solution: Low-Rank Safety Gradient Exploitation

Safety corrections employ low-rank structure of safety gradients, enabling single-example recovery.

class OneShot SafetyPatch:
    def __init__(self, model, reference_model):
        self.model = model  # Compromised model
        self.reference = reference_model  # Original safe model

    def patch_with_single_example(self, safety_example, learning_rate=0.01):
        """Recover safety with one reference example"""

        # Step 1: Compute safety gradient
        # Forward pass with compromised model
        output = self.model(safety_example)
        safe_target = self.reference(safety_example)

        # Compute alignment loss (KL divergence)
        alignment_loss = kl_divergence(output.logits, safe_target.logits)

        # Compute gradient
        gradients = torch.autograd.grad(
            alignment_loss, self.model.parameters(), retain_graph=
        )

        
        
        gradient_matrix = torch.cat([g.flatten()  g  gradients])

        
        U, S, Vt = torch.linalg.svd(gradient_matrix, full_matrices=)

        
        k_rank =   
        U_low = U[:, :k_rank]
        S_low = S[:k_rank]

        
        correction = U_low @ torch.diag(S_low) @ Vt[:k_rank, :]

        
         param, grad_correction  (.model.parameters(), correction):
            param.data -= learning_rate * grad_correction

     ():
        
        harmful_outputs = 
        utility_score = 

         prompt  test_prompts:
            output = .model.generate(prompt)

            
             is_harmful(output):
                harmful_outputs += 

            
             is_useful(output):
                utility_score += 

         {
            :  - (harmful_outputs / (test_prompts)),
            : utility_score / (test_prompts)
        }