tars-token-adaptive-hallucination-reduction

Token-adaptive preference optimization framework using min-max formulation to reduce multimodal LLM hallucination. Achieves 50% hallucination reduction using min-max distributional robustness while preserving visual grounding.

الانتقال إلى التثبيت

المستودع: ADu2021/skillXiv
آخر نشاط في المصدر: ٢٤ مارس ٢٠٢٦ في ١٩:٤٢
لغة SKILL.md المكتشفة: الإنجليزية
النجوم: ٦
التفرعات: ٠

خيارات التثبيت

يُحدَّد Prompt الذي يراجع المصدر أولًا بشكل افتراضي. يمكنك التبديل إلى أمر مباشر أو تنزيل نسخة محلية.

التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.

يتجاوز الأمر المباشر Prompt المخصّص للمراجعة. افحص المصدر قبل تشغيله.

npx skills add https://github.com/ADu2021/skillXiv --skill tars-token-adaptive-hallucination-reduction

يبقى الأمر في سطر واحد. مرّر أفقيًا لمراجعته كاملًا قبل النسخ.

تفضّل نسخة محلية؟ نزّل الملفات المتاحة حاليًا لدى SkillsMP.

مراجعة ملفات المصدر

اقرأ SKILL.md وأي ملفات مرافقة يعرضها SkillsMP قبل أن تقرر التثبيت.

SKILL.md

تعليمات المصدر · معاينة للقراءة فقط

العودة إلى البحث العودة إلى التثبيت

meaningful-kebab-case-name

Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.

2026-03-26

action-quantization-behavior-cloning

Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.

2026-03-26

adaptive-lora-personalized-ranks

Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.

2026-03-26

عرض الكل من هذا المستودع

import torch import torch.nn.functional as F from typing import List, Tuple, Dict class PreferenceEncoder: """Encodes human preferences as token-level distributions""" def __init__(): .tokenizer = tokenizer vocab_size = vocabulary_size () -> [torch.Tensor, torch.Tensor]: preferred_tokens = .tokenizer.encode(preferred_text) dispreferred_tokens = .tokenizer.encode(dispreferred_text) max_len = ((preferred_tokens), (dispreferred_tokens)) preferred_dist = torch.zeros(max_len, .vocab_size) dispreferred_dist = torch.zeros(max_len, .vocab_size) i, token_id (preferred_tokens): preferred_dist[i, token_id] = i, token_id (dispreferred_tokens): dispreferred_dist[i, token_id] = preferred_dist, dispreferred_dist () -> torch.Tensor: all_prefs = [] preferred, dispreferred pairs: pref_dist, _ = .encode_preference_pair(preferred, dispreferred) all_prefs.append(pref_dist) max_len = (p.shape[] p all_prefs) batch_prefs = torch.zeros((all_prefs), max_len, .vocab_size) i, pref (all_prefs): batch_prefs[i, :pref.shape[], :] = pref batch_prefs

class MinMaxPreferenceOptimizer: """Min-max optimization for robust preference learning""" def __init__(self, model, tokenizer, epsilon: float = 0.1): self.model = model self.tokenizer = tokenizer self.epsilon = epsilon # Perturbation budget self.vocab_size = model.config.vocab_size def compute_preference_loss(self, logits: torch.Tensor, target_dist: torch.Tensor, positions: torch.Tensor) -> torch.Tensor: """ Compute loss between predicted logits and preference targets. Args: logits: Model output logits, shape (batch, seq_len, vocab_size) target_dist: Target probability distribution over tokens positions: Which positions correspond to preference targets Returns: Scalar loss value """ # Extract logits at preference positions pred_probs = F.softmax(logits, dim=-1) # KL divergence from model distribution to target loss = F.kl_div( F.log_softmax(logits, dim=-1), target_dist.clamp(min=1e-8), reduction='batchmean' ) return loss def perturbation_step(self, target_dist: torch.Tensor, step_size: float = 0.01) -> torch.Tensor: """ Inner optimization: find worst-case perturbation within semantic budget. Args: target_dist: Current preference target step_size: Optimization step size Returns: Perturbed distribution that remains semantically valid """ perturbed = target_dist.clone().detach().requires_grad_(True) optimizer = torch.optim.Adam([perturbed], lr=step_size) for _ in range(5): # Inner loop iterations # Compute model loss on perturbed targets logits = self.model(...) # Forward pass loss = self.compute_preference_loss(logits, perturbed, mask=None) # Maximize loss (adversarial): find worst perturbation loss_adv = -loss optimizer.zero_grad() loss_adv.backward() optimizer.step() # Project back to semantic constraint set perturbed.data = self._project_to_semantic_budget(perturbed.data) return perturbed.detach() def _project_to_semantic_budget(self, dist: torch.Tensor) -> torch.Tensor: """ Project distribution back to semantic constraint set. Keep similar tokens with higher weight. """ # Reproject to valid probability distribution dist = F.softmax(dist, dim=-1) # Constraint: perturbed distribution cannot exceed epsilon away from original # (in terms of TV divergence or other semantic distance metric) # Simplified: clamp large changes return dist def min_max_training_step(self, images: torch.Tensor, prompts: torch.Tensor, preference_targets: torch.Tensor) -> Tuple[torch.Tensor, float]: """ Single min-max training step: maximize robustness to preference perturbations. Args: images: Visual input prompts: Text prompts preference_targets: Target distributions for next tokens Returns: (updated_model_state, loss_value) """ # Outer loop: optimize model parameters optimizer = torch.optim.AdamW(self.model.parameters(), lr=1e-5) for outer_step in range(3): # Outer iterations # Inner loop: find worst perturbation perturbed_targets = self.perturbation_step(preference_targets) # Forward pass with model outputs = self.model(images, prompts) logits = outputs.logits # Compute loss on perturbed targets loss = self.compute_preference_loss(logits, perturbed_targets, positions=None) # Outer optimization: minimize loss on worst-case perturbation optimizer.zero_grad() loss.backward() optimizer.step() return self.model.state_dict(), loss.item()

class TokenAdaptivePreference: """Adaptively applies preference steering per token""" def __init__(self, model): self.model = model def compute_token_importance(self, logits: torch.Tensor, target_dist: torch.Tensor) -> torch.Tensor: """ Compute importance weight for each token based on preference strength. Args: logits: Model predictions target_dist: Preference targets Returns: Importance weights of shape (batch, seq_len) """ pred_probs = F.softmax(logits, dim=-1) # Importance = entropy difference (high-entropy targets need stronger steering) target_entropy = -(target_dist * torch.log(target_dist + 1e-8)).sum(dim=-1) pred_entropy = -(pred_probs * torch.log(pred_probs + 1e-8)).sum(dim=-1) importance = (target_entropy - pred_entropy).clamp(min=0) # Normalize to [0, 1] importance = importance / (importance.max() + 1e-8) return importance def apply_adaptive_preference(self, logits: torch.Tensor, target_dist: torch.Tensor, base_strength: float = 1.0) -> torch.Tensor: """ Shift logits toward preference targets with token-adaptive strength. Args: logits: Model logits target_dist: Preference targets base_strength: Base preference steering strength Returns: Modified logits """ # Compute per-token importance weights importance = self.compute_token_importance(logits, target_dist) # Compute preference direction pred_probs = F.softmax(logits, dim=-1) pref_direction = target_dist - pred_probs # Shape: (batch, seq_len, vocab) # Apply adaptive steering batch_size, seq_len, vocab_size = logits.shape for b in range(batch_size): for t in range(seq_len): # Scale steering by token importance strength = base_strength * importance[b, t].item() # Move logits toward preference targets logits[b, t, :] = logits[b, t, :] + strength * pref_direction[b, t, :] return logits

class VisualGroundingPreserver: """Ensures preference optimization preserves visual grounding""" def __init__(self, model, vision_encoder): self.model = model self.vision_encoder = vision_encoder def compute_visual_consistency(self, image: torch.Tensor, output_tokens: torch.Tensor) -> float: """ Measure how well output is grounded in visual content. Compute correlation between image features and output token embeddings. """ # Extract visual features image_features = self.vision_encoder(image) # (image_dim,) # Extract output token embeddings token_embeddings = self.model.get_token_embeddings(output_tokens) # (seq_len, emb_dim) # Compute attention-weighted consistency batch_size, seq_len, vocab_size = output_tokens.shape similarities = [] for t in range(seq_len): # Similarity between this token's embedding and image features token_emb = token_embeddings[t] similarity = F.cosine_similarity( image_features.unsqueeze(0), token_emb.unsqueeze(0) ) similarities.append(similarity) # Average similarity across tokens visual_consistency = torch.mean(torch.stack(similarities)) return visual_consistency.item() def grounding_aware_loss(self, logits: torch.Tensor, target_dist: torch.Tensor, image: torch.Tensor, lambda_grounding: float = 0.1) -> torch.Tensor: """ Combine preference loss with visual grounding preservation. Args: logits: Model predictions target_dist: Preference targets image: Visual input lambda_grounding: Weight for grounding constraint Returns: Total loss that preserves grounding """ # Preference alignment loss preference_loss = F.kl_div( F.log_softmax(logits, dim=-1), target_dist, reduction='batchmean' ) # Visual grounding penalty predicted_tokens = torch.argmax(logits, dim=-1) grounding_score = self.compute_visual_consistency(image, predicted_tokens) # Total loss: preference optimization + grounding preservation total_loss = preference_loss - lambda_grounding * grounding_score return total_loss

class TARSTrainer: """Complete TARS training with preference, perturbation, and grounding""" def __init__(self, model, vision_encoder, tokenizer, epsilon: float = 0.1, lambda_grounding: float = 0.1): self.model = model self.vision_encoder = vision_encoder self.pref_encoder = PreferenceEncoder(tokenizer, model.config.vocab_size) self.optimizer = MinMaxPreferenceOptimizer(model, tokenizer, epsilon) self.token_adaptive = TokenAdaptivePreference(model) self.grounding = VisualGroundingPreserver(model, vision_encoder) self.lambda_grounding = lambda_grounding def train_step(self, batch: Dict) -> float: """ Single TARS training step. Args: batch: Contains 'images', 'prompts', 'preferred_text', 'dispreferred_text' Returns: Loss value """ images = batch['images'] prompts = batch['prompts'] preferred = batch['preferred_text'] dispreferred = batch['dispreferred_text'] # Encode preferences as distributions pref_targets = self.pref_encoder.create_preference_labels( list(zip(preferred, dispreferred)) ) # Forward pass outputs = self.model(images, prompts) logits = outputs.logits # Apply token-adaptive preference steering adapted_logits = self.token_adaptive.apply_adaptive_preference( logits, pref_targets, base_strength=1.0 ) # Compute loss with visual grounding preservation loss = self.grounding.grounding_aware_loss( adapted_logits, pref_targets, images, lambda_grounding=self.lambda_grounding ) # Min-max optimization step self.model.train() loss.backward() torch.nn.utils.clip_grad_norm_(self.model.parameters(), 1.0) optimizer = torch.optim.AdamW(self.model.parameters(), lr=1e-5) optimizer.step() optimizer.zero_grad() return loss.item() def train_epoch(self, dataloader, num_epochs: int = 3): """Train for multiple epochs""" for epoch in range(num_epochs): total_loss = 0.0 for batch_idx, batch in enumerate(dataloader): loss = self.train_step(batch) total_loss += loss if batch_idx % 100 == 0: print(f"Epoch {epoch}, Batch {batch_idx}, Loss: {loss:.4f}") avg_loss = total_loss / len(dataloader) print(f"Epoch {epoch} average loss: {avg_loss:.4f}")

name	tars-token-adaptive-hallucination-reduction
title	TARS Min-Max Token-Adaptive Preference Strategy
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2507.21584
keywords	["multimodal-llm","hallucination-reduction","preference-optimization","token-adaptive","robustness"]
description	Token-adaptive preference optimization framework using min-max formulation to reduce multimodal LLM hallucination. Achieves 50% hallucination reduction using min-max distributional robustness while preserving visual grounding.

name	tars-token-adaptive-hallucination-reduction
title	TARS Min-Max Token-Adaptive Preference Strategy
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2507.21584
keywords	["multimodal-llm","hallucination-reduction","preference-optimization","token-adaptive","robustness"]
description	Token-adaptive preference optimization framework using min-max formulation to reduce multimodal LLM hallucination. Achieves 50% hallucination reduction using min-max distributional robustness while preserving visual grounding.

tars-token-adaptive-hallucination-reduction

خيارات التثبيت

مراجعة ملفات المصدر

TARS: MinMax Token-Adaptive Preference Strategy for Hallucination Reduction

Core Concept

Architecture Overview

Implementation Steps

TARS: MinMax Token-Adaptive Preference Strategy for Hallucination Reduction

Core Concept

Architecture Overview

Implementation Steps

Practical Guidance

Reference

tars-token-adaptive-hallucination-reduction

معلومات المصدر

خيارات التثبيت

مراجعة ملفات المصدر

المزيد من هذا المستودع

المزيد من هذا المستودع

TARS: MinMax Token-Adaptive Preference Strategy for Hallucination Reduction

Core Concept

Architecture Overview

Implementation Steps

TARS: MinMax Token-Adaptive Preference Strategy for Hallucination Reduction

Core Concept

Architecture Overview

Implementation Steps

Practical Guidance

Reference