promptquine-evolving-in-context-prompts

Automatically discovers optimal in-context learning prompts through evolutionary token pruning that removes redundant demonstrations to create effective 'gibberish' prompts. Matches state-of-the-art optimization with low-data regimes. Use for automated prompt discovery without manual tuning or human expertise.

Jump to install

Repository: ADu2021/skillXiv
Last source activity: March 24, 2026 at 19:42
Detected SKILL.md language: English
Stars: 6
Forks: 0

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

A direct command skips the review prompt. Inspect the source before running it.

npx skills add https://github.com/ADu2021/skillXiv --skill promptquine-evolving-in-context-prompts

The command stays on one line. Scroll horizontally to inspect it before copying.

Prefer a local copy? Download the files currently available to SkillsMP.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

SKILL.md

Source instructions · Read-only preview

Back to search Back to installation

meaningful-kebab-case-name

Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.

2026-03-26

action-quantization-behavior-cloning

Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.

2026-03-26

adaptive-lora-personalized-ranks

Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.

2026-03-26

View all from this repository

PromptQuine: Evolutionary Token Pruning for Optimal In-Context Learning

Conventional wisdom says that good in-context learning prompts must be well-formed, semantically coherent examples that clearly demonstrate the task. PromptQuine challenges this by discovering that removing tokens from coherent demonstrations to create seemingly incoherent sequences often improves performance. This counterintuitive finding suggests language models respond to minimal features rather than full linguistic structure. PromptQuine formalizes this as an evolutionary search problem: given a prompt with n tokens, discover which subset of m tokens maximizes task performance. The result is a framework that discovers effective prompts faster and cheaper than human optimization or prior automatic methods.

The insight is that natural language prompts contain redundant features. Strategic token removal eliminates this redundancy while preserving core task-relevant information that models actually use. This "Partial Context Hypothesis" explains why seemingly garbled prompts work—they contain just enough signal.

Core Concept

PromptQuine treats prompt optimization as an evolutionary search where the population is binary token masks. Each mask represents which tokens to keep from an original demonstration. Genetic algorithms with regularized evolution search this high-dimensional space, using low-data task performance as the fitness function. The framework includes:

Token Mask Representation: Binary vector indicating which tokens are retained (1) or pruned (0)

Bit-Flip Mutations: Only allow 1→0 operations (removing tokens), preventing expansion

Calibration-then-Selection: Re-rank candidates to reduce overfitting to small validation sets

Tournament Selection: Probabilistic selection favoring high-fitness masks

Regularized Evolution: Balance exploration vs. exploitation to avoid premature convergence

The search landscape exhibits surprising properties: sparse pruned prompts often plateau at excellent performance, suggesting models extract sufficient information from minimal tokens.

Architecture Overview

Genetic Algorithm Core: Population of binary masks evolving through mutation and selection

Fitness Function: Task performance on small validation set (low-data regime)

Mutation Strategy: Token removal (bit-flip 1→0) only

Calibration Module: Re-ranking to mitigate overfitting

Tournament Selection: Fitness-biased probabilistic selection

Stopping Criteria: Convergence when elite fitness plateaus

PromptQuine: Evolutionary Token Pruning for Optimal In-Context Learning

Core Concept

Token Mask Representation: Binary vector indicating which tokens are retained (1) or pruned (0)

Bit-Flip Mutations: Only allow 1→0 operations (removing tokens), preventing expansion

Calibration-then-Selection: Re-rank candidates to reduce overfitting to small validation sets

Tournament Selection: Probabilistic selection favoring high-fitness masks

Regularized Evolution: Balance exploration vs. exploitation to avoid premature convergence

The search landscape exhibits surprising properties: sparse pruned prompts often plateau at excellent performance, suggesting models extract sufficient information from minimal tokens.

Architecture Overview

Genetic Algorithm Core: Population of binary masks evolving through mutation and selection

Fitness Function: Task performance on small validation set (low-data regime)

Mutation Strategy: Token removal (bit-flip 1→0) only

Calibration Module: Re-ranking to mitigate overfitting

Tournament Selection: Fitness-biased probabilistic selection

Stopping Criteria: Convergence when elite fitness plateaus

import numpy as np from typing import List, Tuple, Callable import torch class TokenMask: """Represents a binary mask over prompt tokens.""" def __init__(self, prompt_tokens: List[str]): self.prompt_tokens = prompt_tokens self.num_tokens = len(prompt_tokens) # Start with all tokens (mask = all 1s) self.mask = np.ones(self.num_tokens, dtype=int) def apply_mask(self) -> str: """Apply mask to tokens, returning pruned prompt.""" masked_tokens = [ token for token, keep in zip(self.prompt_tokens, self.mask) if keep == 1 ] return ' '.join(masked_tokens) def mutate(self) -> 'TokenMask': """ Bit-flip mutation: randomly remove one token (1→0). Prevents prompt expansion. """ new_mask = self.TokenMask(self.prompt_tokens) new_mask.mask = self.mask.copy() # Find indices where token is present (mask==1) present_indices = np.where(self.mask == 1)[0] if len(present_indices) > 1: # Remove one present token randomly idx_to_remove = np.random.choice(present_indices) new_mask.mask[idx_to_remove] = 0 return new_mask def copy(self) -> 'TokenMask': """Create independent copy.""" new_mask = TokenMask(self.prompt_tokens) new_mask.mask = self.mask.copy() return new_mask class PromptQuineEvolutionarySearch: """ Evolutionary algorithm for discovering optimal token pruning patterns. Searches 2^n space (n = token count) for high-performance masks. """ def __init__( self, initial_prompt: str, eval_function: Callable[[str], float], population_size: int = 128, mutation_rate: float = 0.1, max_iterations: int = 200 ): """ Args: initial_prompt: Original well-formed in-context example eval_function: Takes pruned_prompt → accuracy score population_size: Number of masks in population mutation_rate: Fraction of population mutated per generation max_iterations: Maximum generations before stopping """ self.initial_prompt = initial_prompt self.eval_function = eval_function self.population_size = population_size self.mutation_rate = mutation_rate self.max_iterations = max_iterations # Parse prompt into tokens self.prompt_tokens = initial_prompt.split() # Initialize population: all masks start as identity self.population = [ TokenMask(self.prompt_tokens) for _ in range(population_size) ] # Track elite candidates (best masks seen) self.elite = [] def evaluate_population(self) -> np.ndarray: """ Evaluate fitness (task accuracy) for entire population. Returns array of fitness scores. """ fitness = np.zeros(len(self.population)) for idx, mask in enumerate(self.population): pruned_prompt = mask.apply_mask() # Evaluate on validation set fitness[idx] = self.eval_function(pruned_prompt) return fitness def calibrate_ranking(self, fitness: np.ndarray, num_evals: int = 1000): """ Re-rank candidates to reduce overfitting. For top candidates, re-evaluate on larger held-out set. """ # Sort by fitness sorted_indices = np.argsort(-fitness)[:10] # Top 10 # Re-evaluate top candidates with more samples recalibrated_fitness = fitness.copy() for idx in sorted_indices: # Run more evaluations mask = self.population[idx] pruned_prompt = mask.apply_mask() robust_fitness = self.eval_function(pruned_prompt) recalibrated_fitness[idx] = robust_fitness return recalibrated_fitness def tournament_selection(self, fitness: np.ndarray) -> int: """ Tournament selection: select best from random subset. Probability proportional to fitness. """ tournament_size = 4 candidates = np.random.choice( len(self.population), size=tournament_size, replace=False ) # Return index of best candidate in tournament best_in_tournament = candidates[np.argmax(fitness[candidates])] return best_in_tournament def evolve(self) -> Tuple[str, float]: """ Run evolutionary search for optimal token pruning. Returns: best_prompt: Pruned prompt with highest validation accuracy best_fitness: Corresponding fitness score """ best_fitness_overall = -1.0 best_mask_overall = None plateau_counter = 0 convergence_threshold = 10 # Stop if no improvement for 10 iterations for iteration in range(self.max_iterations): # Evaluate population fitness = self.evaluate_population() # Calibrate rankings to reduce overfitting fitness = self.calibrate_ranking(fitness) # Track elite current_best_idx = np.argmax(fitness) current_best_fitness = fitness[current_best_idx] if current_best_fitness > best_fitness_overall: best_fitness_overall = current_best_fitness best_mask_overall = self.population[current_best_idx].copy() plateau_counter = 0 else: plateau_counter += 1 # Stopping criterion: convergence if plateau_counter >= convergence_threshold: print(f"Converged at iteration {iteration}") break # Mutation and selection: create next generation next_population = [] # Elitism: keep best individuals elite_size = max(1, int(self.population_size * 0.1)) elite_indices = np.argsort(-fitness)[:elite_size] for elite_idx in elite_indices: next_population.append(self.population[elite_idx].copy()) # Fill rest via tournament selection + mutation while len(next_population) < self.population_size: # Select parent via tournament parent_idx = self.tournament_selection(fitness) parent = self.population[parent_idx] # Mutate: remove tokens child = parent.mutate() next_population.append(child) self.population = next_population if iteration % 20 == 0: print( f"Iteration {iteration}: Best fitness = {best_fitness_overall:.4f}, " f"Pruned tokens = {np.sum(best_mask_overall.mask)}/{len(self.prompt_tokens)}" ) # Return best discovered prompt best_prompt = best_mask_overall.apply_mask() return best_prompt, best_fitness_overall def evaluate_prompt_on_task(pruned_prompt: str, test_samples: List[dict]) -> float: """ Evaluate prompt performance on a specific task. Returns accuracy as fitness metric. Args: pruned_prompt: Token-pruned demonstration test_samples: List of {'input': ..., 'label': ...} Returns: accuracy: Fraction correct predictions """ correct = 0 total = len(test_samples) for sample in test_samples: # Build full prompt: pruned_demonstration + input full_prompt = pruned_prompt + "\n" + sample['input'] # Get LLM prediction (simplified) prediction = get_llm_response(full_prompt) if prediction == sample['label']: correct += 1 accuracy = correct / total return accuracy

name	promptquine-evolving-in-context-prompts
title	Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2506.17930
keywords	["PromptOptimization","EvolutionarySearch","InContextLearning","AutomatedPrompting"]
description	Automatically discovers optimal in-context learning prompts through evolutionary token pruning that removes redundant demonstrations to create effective 'gibberish' prompts. Matches state-of-the-art optimization with low-data regimes. Use for automated prompt discovery without manual tuning or human expertise.

name	promptquine-evolving-in-context-prompts
title	Evolving Prompts In-Context: An Open-ended, Self-replicating Perspective
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2506.17930
keywords	["PromptOptimization","EvolutionarySearch","InContextLearning","AutomatedPrompting"]
description	Automatically discovers optimal in-context learning prompts through evolutionary token pruning that removes redundant demonstrations to create effective 'gibberish' prompts. Matches state-of-the-art optimization with low-data regimes. Use for automated prompt discovery without manual tuning or human expertise.

Aspect	Value	Notes
Search Space	2^n (n=token count)	Exponential but explorable with evolution
Population Size	128	Typical setting
Convergence	50-200 iterations	Task-dependent
Token Compression	40-70%	Effective prompts often highly pruned
Classification Accuracy	77.5% avg (1-shot)	Matches or exceeds Promptbreeder
Search Time	Minutes	Much faster than human optimization
Improvement Over Baseline	+2-8%	Task-dependent

promptquine-evolving-in-context-prompts

Install options

Review the source files

PromptQuine: Evolutionary Token Pruning for Optimal In-Context Learning

Core Concept

Architecture Overview

PromptQuine: Evolutionary Token Pruning for Optimal In-Context Learning

Core Concept

Architecture Overview

Implementation

Practical Guidance

Reference

promptquine-evolving-in-context-prompts

Source facts

Install options

Review the source files

More from this repository

More from this repository

PromptQuine: Evolutionary Token Pruning for Optimal In-Context Learning

Core Concept

Architecture Overview

PromptQuine: Evolutionary Token Pruning for Optimal In-Context Learning

Core Concept

Architecture Overview

Implementation

Practical Guidance

Reference