メニュー
Unify LLM reasoning with embedding generation by training models to output embeddings as direct outputs of step-by-step reasoning. Trigger: improve performance on complex retrieval tasks requiring deep reasoning.
デフォルトでは、最初にソースを確認する Prompt が選択されています。直接コマンドに切り替えるか、ローカルコピーをダウンロードすることもできます。
インストールを決める前に、SKILL.md と SkillsMP に表示されている付属ファイルをお読みください。
Codex または Claude でインストール この Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。
直接コマンドでは確認用 Prompt が省略されます。実行前にソースを確認してください。
npx skills add https://github.com/ADu2021/skillXiv --skill search-r3-reasoningコマンドは1行のまま表示されます。コピー前に横へスクロールして全体を確認してください。
ローカルで確認しますか？SkillsMP が現在取得できるファイルをダウンロードできます。
Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
SOC 職業分類に基づく
SKILL.md を表示中
| name | search-r3-reasoning |
| title | Search-R3: Unifying Reasoning and Embedding Generation for Retrieval |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2510.07048 |
| keywords | ["retrieval-augmented-generation","reasoning","embedding-generation","post-training","knowledge-intensive-tasks"] |
| description | Unify LLM reasoning with embedding generation by training models to output embeddings as direct outputs of step-by-step reasoning. Trigger: improve performance on complex retrieval tasks requiring deep reasoning. |
Search-R3 addresses a critical gap: Large language models excel at reasoning but have been underutilized for retrieval tasks. Current systems treat reasoning and retrieval as separate pipelines. Search-R3 unifies these by training LLMs to generate search embeddings as direct outputs of their reasoning process. Through combined supervised learning and RL, models learn to produce embeddings that simultaneously capture both reasoning depth and retrieval relevance, improving performance on knowledge-intensive tasks requiring complex reasoning.
The key insight: Embeddings are not fixed representations; they can be outputs of reasoning, making retrieval and reasoning co-dependent and co-optimized.
Define how embeddings emerge from reasoning.
class ReasoningEmbeddingGenerator:
"""
Generate embeddings through explicit reasoning steps.
"""
def __init__(self, model, embedding_dim=768):
self.model = model
self.embedding_dim = embedding_dim
def generate_reasoning_and_embedding(self, query):
"""
Chain-of-thought reasoning that concludes with embedding generation.
Args:
query: Query string or question
Returns:
(reasoning_trace, embedding_vector)
"""
# Step 1: Generate reasoning
reasoning_prompt = (
f"Query: {query}\n\n"
f"Think through this step-by-step:\n"
)
reasoning = .model.generate(
reasoning_prompt,
max_tokens=,
temperature=
)
embedding_prompt = (
)
embedding_text = .model.generate(
embedding_prompt,
max_tokens=,
temperature=
)
embedding = .parse_embedding_output(embedding_text)
reasoning, embedding
():
re
numbers = re.findall(, embedding_text)
(numbers) >= .embedding_dim:
embedding = torch.tensor([(n) n numbers[:.embedding_dim]])
:
embedding = torch.zeros(.embedding_dim)
i, num (numbers):
embedding[i] = (num)
embedding = embedding / (torch.norm(embedding) + )
embedding
():
doc_embeddings = .model.encode_documents(relevant_docs)
similarities = []
doc_emb doc_embeddings:
sim = torch.dot(embedding, doc_emb) / (
torch.norm(embedding) * torch.norm(doc_emb) +
)
similarities.append(sim.item())
recall_at_1 = (similarities) >
mean_similarity = np.mean(similarities)
{
: recall_at_1,
: mean_similarity,
: recall_at_1 * + mean_similarity *
}
Train models to generate quality embeddings via SFT.
class SupervisedEmbeddingTraining:
"""
Supervised fine-tuning for embedding generation.
"""
def __init__(self, model, corpus):
self.model = model
self.corpus = corpus # Document collection
def create_training_pairs(self, query_doc_pairs, num_negatives=5):
"""
Create (query, reasoning, embedding, labels) tuples.
Args:
query_doc_pairs: List of (query, relevant_doc) pairs
num_negatives: Negative documents per query
Returns:
Training examples
"""
training_examples = []
for query, relevant_doc in query_doc_pairs:
# Generate reasoning for this query
reasoning_prompt = (
f"Query: {query}\n"
f"Relevant document: {relevant_doc[:200]}...\n\n"
f"Reasoning for retrieval: "
)
reasoning = self.model.generate(
reasoning_prompt,
max_tokens=200
)
# Generate positive (relevant) embedding
positive_embedding = self.model.encode_document(relevant_doc)
# Sample negative (irrelevant) documents
negative_docs = self._sample_negatives(
query,
relevant_doc,
num_negatives
)
negative_embeddings = [
self.model.encode_document(doc) for doc in negative_docs
]
# Create training example
example = {
"query": query,
"reasoning": reasoning,
"positive_embedding": positive_embedding,
"negative_embeddings": negative_embeddings,
"target_quality": 1.0 # Positive example
}
training_examples.append(example)
return training_examples
def _sample_negatives(self, query, relevant_doc, num_negatives):
"""Sample hard negatives: relevant but not the specific document."""
# BM25 or semantic search to get candidates
candidate_docs = self.corpus.search(query, top_k=50)
negatives = [
doc for doc in candidate_docs
if doc != relevant_doc
][:num_negatives]
return negatives
def train_supervised(self, training_examples, num_epochs=5):
"""
SFT on reasoning + embedding generation.
"""
optimizer = torch.optim.Adam(self.model.parameters(), lr=1e-5)
for epoch in range(num_epochs):
epoch_loss = 0
for example in training_examples:
query = example["query"]
reasoning = example["reasoning"]
positive_emb = example["positive_embedding"]
# Generate embedding output
embedding_prompt = (
f"Query: {query}\n"
f"Reasoning: {reasoning}\n"
f"Embedding: "
)
generated_embedding_text = self.model.generate(
embedding_prompt,
max_tokens=200
)
generated_embedding = parse_embedding(generated_embedding_text)
# Contrastive loss: pull closer to positive, push away from negatives
loss = compute_contrastive_loss(
generated_embedding,
positive_emb,
example["negative_embeddings"]
)
# Backward
optimizer.zero_grad()
loss.backward()
optimizer.step()
epoch_loss += loss.item()
print(f"Epoch {epoch}: SFT loss = {epoch_loss / len(training_examples):.4f}")
return self.model
RL for embedding optimization without corpus re-encoding.
class EmbeddingRLEnvironment:
"""
RL environment for embedding optimization.
Handles evolving embeddings without re-encoding corpus.
"""
def __init__(self, model, corpus_embeddings, ground_truth_labels):
self.model = model
self.corpus_embeddings = corpus_embeddings # Pre-encoded documents
self.ground_truth = ground_truth_labels # Relevance labels
self.cache = {} # Cache computed similarities
def compute_reward(self, query_embedding, query, relevant_docs):
"""
Reward function: how well does embedding retrieve relevant documents?
"""
similarities = []
for doc_id, doc_embedding in enumerate(self.corpus_embeddings):
similarity = cosine_similarity(query_embedding, doc_embedding)
similarities.append((doc_id, similarity))
# Rank documents
rankings = sorted(similarities, key=lambda x: x[1], reverse=True)
# Compute MRR (Mean Reciprocal Rank) for relevant docs
mrr = 0
for rank, (doc_id, _) in enumerate(rankings):
if doc_id in self.ground_truth[query]:
mrr = 1.0 / (rank + 1)
break
# Compute NDCG
dcg = 0
for rank, (doc_id, _) in enumerate(rankings[:10]):
if doc_id in self.ground_truth[query]:
dcg += 1.0 / np.log2(rank + 2)
ideal_dcg = sum(
1.0 / np.log2(i + 2)
for i in range(min(len(self.ground_truth[query]), 10))
)
ndcg = dcg / (ideal_dcg + 1e-8)
# Combined reward
reward = 0.5 * mrr + 0.5 * ndcg
return reward
def step(self, query, reasoning, query_embedding):
"""
Execute one RL step: evaluate embedding quality and return reward.
"""
reward = self.compute_reward(query_embedding, query, self.ground_truth)
# Truncated to [-1, 1] for stability
reward = np.clip(reward, -1, 1)
return reward
Optimize embeddings through reinforcement learning.
class EmbeddingRLTraining:
"""
RL fine-tuning for embedding generation.
"""
def __init__(self, model, env):
self.model = model
self.env = env
def train_rl(self, query_list, num_epochs=10):
"""
RL training for embedding optimization.
"""
optimizer = torch.optim.Adam(self.model.parameters(), lr=5e-6)
for epoch in range(num_epochs):
epoch_reward = 0
for query in query_list:
# Generate reasoning
reasoning_prompt = f"Query: {query}\nReasoning: "
reasoning = self.model.generate(
reasoning_prompt,
max_tokens=200
)
# Generate embedding
embedding_prompt = (
f"Query: {query}\n"
f"Reasoning: {reasoning}\n"
f"Embedding: "
)
embedding_text = self.model.generate(
embedding_prompt,
max_tokens=200
)
query_embedding = parse_embedding(embedding_text)
# Evaluate in RL environment
reward = self.env.step(query, reasoning, query_embedding)
# Policy gradient loss
log_prob = self.model.compute_log_prob(embedding_text)
pg_loss = -reward * log_prob
# Entropy bonus for diversity
entropy = compute_entropy_bonus(
self.model.get_logits(embedding_prompt)
)
total_loss = pg_loss - 0.01 * entropy
# Update
optimizer.zero_grad()
total_loss.backward()
torch.nn.utils.clip_grad_norm_(self.model.parameters(), 1.0)
optimizer.step()
epoch_reward += reward
avg_reward = epoch_reward / len(query_list)
print(f"Epoch {epoch}: RL reward = {avg_reward:.4f}")
return self.model
Combine SFT and RL stages.
def train_search_r3(
base_model,
corpus,
training_queries,
test_queries,
config
):
"""
Complete Search-R3 training pipeline.
"""
# Pre-encode corpus for RL efficiency
corpus_embeddings = base_model.encode_documents(corpus)
# Create ground truth labels (via BM25 or manual)
ground_truth = create_retrieval_labels(training_queries, corpus)
# Stage 1: Supervised Learning
print("Stage 1: Supervised Fine-Tuning")
sft_trainer = SupervisedEmbeddingTraining(base_model, corpus)
training_pairs = create_training_pairs(training_queries, corpus)
training_examples = sft_trainer.create_training_pairs(training_pairs)
sft_model = sft_trainer.train_supervised(training_examples, num_epochs=5)
# Stage 2: Reinforcement Learning
print("\nStage 2: Reinforcement Learning")
env = EmbeddingRLEnvironment(
sft_model,
corpus_embeddings,
ground_truth
)
rl_trainer = EmbeddingRLTraining(sft_model, env)
final_model = rl_trainer.train_rl(
training_queries,
num_epochs=10
)
# Evaluation
print("\nEvaluation")
metrics = evaluate_search_r3(final_model, test_queries, corpus)
return final_model, metrics
Measure performance on knowledge-intensive tasks.
def evaluate_search_r3(model, test_queries, corpus):
"""
Evaluate Search-R3 on retrieval + reasoning tasks.
"""
results = {
"mrr": [],
"ndcg": [],
"reasoning_quality": []
}
for query in test_queries:
# Generate reasoning + embedding
reasoning, embedding = model.generate_reasoning_and_embedding(query)
# Retrieve documents
corpus_embeddings = model.encode_documents(corpus)
similarities = [
torch.dot(embedding, doc_emb).item()
for doc_emb in corpus_embeddings
]
# Evaluate retrieval
rankings = sorted(
enumerate(similarities),
key=lambda x: x[1],
reverse=True
)
# MRR computation
relevant_docs = get_relevant_documents(query, corpus)
for rank, (doc_id, _) in enumerate(rankings):
if doc_id in relevant_docs:
results["mrr"].append(1.0 / (rank + 1))
break
# Evaluate reasoning quality
reasoning_quality = score_reasoning(reasoning, query)
results["reasoning_quality"].append(reasoning_quality)
avg_mrr = np.mean(results["mrr"])
avg_reasoning = np.mean(results["reasoning_quality"])
print(f"MRR: {avg_mrr:.3f}")
print(f"Avg reasoning quality: {avg_reasoning:.3f}")
return results
Hyperparameters:
When to Use:
When NOT to Use:
Search-R3: Unifying Reasoning and Embedding Generation for Retrieval — arXiv:2510.07048