promptquine-evolving-in-context-prompts

Automatically discovers optimal in-context learning prompts through evolutionary token pruning that removes redundant demonstrations to create effective 'gibberish' prompts. Matches state-of-the-art optimization with low-data regimes. Use for automated prompt discovery without manual tuning or human expertise.

インストールへ移動

リポジトリ: ADu2021/skillXiv
ソースの最終更新活動: 2026年3月24日 19:42
検出された SKILL.md の言語: 英語
スター: 6
フォーク: 0

インストール方法

デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。

Codex または Claude でインストールこの Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。

直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。

npx skills add https://github.com/ADu2021/skillXiv --skill promptquine-evolving-in-context-prompts

コマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。

ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。

ソースファイルを確認

インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。

SKILL.md

ソースの指示 · 読み取り専用プレビュー

検索に戻るインストールに戻る

PromptQuine: Evolutionary Token Pruning for Optimal In-Context Learning

Conventional wisdom says that good in-context learning prompts must be well-formed, semantically coherent examples that clearly demonstrate the task. PromptQuine challenges this by discovering that removing tokens from coherent demonstrations to create seemingly incoherent sequences often improves performance. This counterintuitive finding suggests language models respond to minimal features rather than full linguistic structure. PromptQuine formalizes this as an evolutionary search problem: given a prompt with n tokens, discover which subset of m tokens maximizes task performance. The result is a framework that discovers effective prompts faster and cheaper than human optimization or prior automatic methods.

The insight is that natural language prompts contain redundant features. Strategic token removal eliminates this redundancy while preserving core task-relevant information that models actually use. This "Partial Context Hypothesis" explains why seemingly garbled prompts work—they contain just enough signal.

Core Concept

PromptQuine treats prompt optimization as an evolutionary search where the population is binary token masks. Each mask represents which tokens to keep from an original demonstration. Genetic algorithms with regularized evolution search this high-dimensional space, using low-data task performance as the fitness function. The framework includes:

Token Mask Representation: Binary vector indicating which tokens are retained (1) or pruned (0)

Bit-Flip Mutations: Only allow 1→0 operations (removing tokens), preventing expansion

Calibration-then-Selection: Re-rank candidates to reduce overfitting to small validation sets

Tournament Selection: Probabilistic selection favoring high-fitness masks

Regularized Evolution: Balance exploration vs. exploitation to avoid premature convergence

The search landscape exhibits surprising properties: sparse pruned prompts often plateau at excellent performance, suggesting models extract sufficient information from minimal tokens.

Architecture Overview

Genetic Algorithm Core: Population of binary masks evolving through mutation and selection

Fitness Function: Task performance on small validation set (low-data regime)

Mutation Strategy: Token removal (bit-flip 1→0) only

Calibration Module: Re-ranking to mitigate overfitting

Tournament Selection: Fitness-biased probabilistic selection

Stopping Criteria: Convergence when elite fitness plateaus

PromptQuine: Evolutionary Token Pruning for Optimal In-Context Learning

Core Concept

Token Mask Representation: Binary vector indicating which tokens are retained (1) or pruned (0)

Bit-Flip Mutations: Only allow 1→0 operations (removing tokens), preventing expansion

Calibration-then-Selection: Re-rank candidates to reduce overfitting to small validation sets

Tournament Selection: Probabilistic selection favoring high-fitness masks

Regularized Evolution: Balance exploration vs. exploitation to avoid premature convergence

The search landscape exhibits surprising properties: sparse pruned prompts often plateau at excellent performance, suggesting models extract sufficient information from minimal tokens.

Architecture Overview

Genetic Algorithm Core: Population of binary masks evolving through mutation and selection

Fitness Function: Task performance on small validation set (low-data regime)

Mutation Strategy: Token removal (bit-flip 1→0) only

Calibration Module: Re-ranking to mitigate overfitting

Tournament Selection: Fitness-biased probabilistic selection

Stopping Criteria: Convergence when elite fitness plateaus

import numpy as np from typing import List, Tuple, Callable import torch class TokenMask: """Represents a binary mask over prompt tokens.""" def __init__(self, prompt_tokens: List[str]): self.prompt_tokens = prompt_tokens self.num_tokens = len(prompt_tokens) # Start with all tokens (mask = all 1s) self.mask = np.ones(self.num_tokens, dtype=int) def apply_mask(self) -> str: """Apply mask to tokens, returning pruned prompt.""" masked_tokens = [ token for token, keep in zip(self.prompt_tokens, self.mask) if keep == 1 ] return ' '.join(masked_tokens) def mutate(self) -> 'TokenMask': """ Bit-flip mutation: randomly remove one token (1→0). Prevents prompt expansion. """ new_mask = self.TokenMask(self.prompt_tokens) new_mask.mask = self.mask.copy() # Find indices where token is present (mask==1) present_indices = np.where(self.mask == 1)[0] if len(present_indices) > 1: # Remove one present token randomly idx_to_remove = np.random.choice(present_indices) new_mask.mask[idx_to_remove] = 0 return new_mask def copy(self) -> 'TokenMask': """Create independent copy.""" new_mask = TokenMask(self.prompt_tokens) new_mask.mask = self.mask.copy() return new_mask class PromptQuineEvolutionarySearch: """ Evolutionary algorithm for discovering optimal token pruning patterns. Searches 2^n space (n = token count) for high-performance masks. """ def __init__( self, initial_prompt: str, eval_function: Callable[[str], float], population_size: int = 128, mutation_rate: float = 0.1, max_iterations: int = 200 ): """ Args: initial_prompt: Original well-formed in-context example eval_function: Takes pruned_prompt → accuracy score population_size: Number of masks in population mutation_rate: Fraction of population mutated per generation max_iterations: Maximum generations before stopping """ self.initial_prompt = initial_prompt self.eval_function = eval_function self.population_size = population_size self.mutation_rate = mutation_rate self.max_iterations = max_iterations # Parse prompt into tokens self.prompt_tokens = initial_prompt.split() # Initialize population: all masks start as identity self.population = [ TokenMask(self.prompt_tokens) for _ in range(population_size) ] # Track elite candidates (best masks seen) self.elite = [] def evaluate_population(self) -> np.ndarray: """ Evaluate fitness (task accuracy) for entire population. Returns array of fitness scores. """ fitness = np.zeros(len(self.population)) for idx, mask in enumerate(self.population): pruned_prompt = mask.apply_mask() # Evaluate on validation set fitness[idx] = self.eval_function(pruned_prompt) return fitness def calibrate_ranking(self, fitness: np.ndarray, num_evals: int = 1000): """ Re-rank candidates to reduce overfitting. For top candidates, re-evaluate on larger held-out set. """ # Sort by fitness sorted_indices = np.argsort(-fitness)[:10] # Top 10 # Re-evaluate top candidates with more samples recalibrated_fitness = fitness.copy() for idx in sorted_indices: # Run more evaluations mask = self.population[idx] pruned_prompt = mask.apply_mask() robust_fitness = self.eval_function(pruned_prompt) recalibrated_fitness[idx] = robust_fitness return recalibrated_fitness def tournament_selection(self, fitness: np.ndarray) -> int: """ Tournament selection: select best from random subset. Probability proportional to fitness. """ tournament_size = 4 candidates = np.random.choice( len(self.population), size=tournament_size, replace=False ) # Return index of best candidate in tournament best_in_tournament = candidates[np.argmax(fitness[candidates])] return best_in_tournament def evolve(self) -> Tuple[str, float]: """ Run evolutionary search for optimal token pruning. Returns: best_prompt: Pruned prompt with highest validation accuracy best_fitness: Corresponding fitness score """ best_fitness_overall = -1.0 best_mask_overall = None plateau_counter = 0 convergence_threshold = 10 # Stop if no improvement for 10 iterations for iteration in range(self.max_iterations): # Evaluate population fitness = self.evaluate_population() # Calibrate rankings to reduce overfitting fitness = self.calibrate_ranking(fitness) # Track elite current_best_idx = np.argmax(fitness) current_best_fitness = fitness[current_best_idx] if current_best_fitness > best_fitness_overall: best_fitness_overall = current_best_fitness best_mask_overall = self.population[current_best_idx].copy() plateau_counter = 0 else: plateau_counter += 1 # Stopping criterion: convergence if plateau_counter >= convergence_threshold: print(f"Converged at iteration {iteration}") break # Mutation and selection: create next generation next_population = [] # Elitism: keep best individuals elite_size = max(1, int(self.population_size * 0.1)) elite_indices = np.argsort(-fitness)[:elite_size] for elite_idx in elite_indices: next_population.append(self.population[elite_idx].copy()) # Fill rest via tournament selection + mutation while len(next_population) < self.population_size: # Select parent via tournament parent_idx = self.tournament_selection(fitness) parent = self.population[parent_idx] # Mutate: remove tokens child = parent.mutate() next_population.append(child) self.population = next_population if iteration % 20 == 0: print( f"Iteration {iteration}: Best fitness = {best_fitness_overall:.4f}, " f"Pruned tokens = {np.sum(best_mask_overall.mask)}/{len(self.prompt_tokens)}" ) # Return best discovered prompt best_prompt = best_mask_overall.apply_mask() return best_prompt, best_fitness_overall def evaluate_prompt_on_task(pruned_prompt: str, test_samples: List[dict]) -> float: """ Evaluate prompt performance on a specific task. Returns accuracy as fitness metric. Args: pruned_prompt: Token-pruned demonstration test_samples: List of {'input': ..., 'label': ...} Returns: accuracy: Fraction correct predictions """ correct = 0 total = len(test_samples) for sample in test_samples: # Build full prompt: pruned_demonstration + input full_prompt = pruned_prompt + "\n" + sample['input'] # Get LLM prediction (simplified) prediction = get_llm_response(full_prompt) if prediction == sample['label']: correct += 1 accuracy = correct / total return accuracy

name	promptquine-evolving-in-context-prompts
title	Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2506.17930
keywords	["PromptOptimization","EvolutionarySearch","InContextLearning","AutomatedPrompting"]
description	Automatically discovers optimal in-context learning prompts through evolutionary token pruning that removes redundant demonstrations to create effective 'gibberish' prompts. Matches state-of-the-art optimization with low-data regimes. Use for automated prompt discovery without manual tuning or human expertise.

name	promptquine-evolving-in-context-prompts
title	Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2506.17930
keywords	["PromptOptimization","EvolutionarySearch","InContextLearning","AutomatedPrompting"]
description	Automatically discovers optimal in-context learning prompts through evolutionary token pruning that removes redundant demonstrations to create effective 'gibberish' prompts. Matches state-of-the-art optimization with low-data regimes. Use for automated prompt discovery without manual tuning or human expertise.

Aspect	Value	Notes
Search Space	2^n (n=token count)	Exponential but explorable with evolution
Population Size	128	Typical setting
Convergence	50-200 iterations	Task-dependent
Token Compression	40-70%	Effective prompts often highly pruned
Classification Accuracy	77.5% avg (1-shot)	Matches or exceeds Promptbreeder
Search Time	Minutes	Much faster than human optimization
Improvement Over Baseline	+2-8%	Task-dependent

promptquine-evolving-in-context-prompts

インストール方法

ソースファイルを確認

PromptQuine: Evolutionary Token Pruning for Optimal In-Context Learning

Core Concept

Architecture Overview

PromptQuine: Evolutionary Token Pruning for Optimal In-Context Learning

Core Concept

Architecture Overview

Implementation

Practical Guidance

Reference

promptquine-evolving-in-context-prompts

ソース情報

インストール方法

ソースファイルを確認

このリポジトリの他の Skills

このリポジトリの他の Skills

PromptQuine: Evolutionary Token Pruning for Optimal In-Context Learning

Core Concept

Architecture Overview

PromptQuine: Evolutionary Token Pruning for Optimal In-Context Learning

Core Concept

Architecture Overview

Implementation

Practical Guidance

Reference