Skip to main content Inicio Creadores adu2021 skillxiv thinking-to-recall
thinking-to-recall Demonstrates that chain-of-thought reasoning improves LLM factual retrieval through computational buffering and self-priming. Improves single-hop factual accuracy by enabling models to generate contextual bridge facts before recalling answers.
Ir a la instalación Skills Marketplace Descubre y explora habilidades de IA creadas por la comunidad.
Instalar con Codex o Claude Copia este prompt, pégalo en Codex, Claude u otro asistente, y deja que revise la página de la skill y la instale por ti.
Mostrar detalles del prompt Un comando directo omite el prompt de revisión. Revisa el origen antes de ejecutarlo.
npx skills add https://github.com/ADu2021/skillXiv --skill thinking-to-recallEl comando permanece en una sola línea. Desplázate horizontalmente para revisarlo antes de copiarlo.
¿Prefieres una copia local? Descarga los archivos que SkillsMP tiene disponibles ahora.
Descargar Zip Descargando... Más de este repositorio meaningful-kebab-case-name Convert arXiv papers into ready-to-use agent skills using category-aware extraction. First classifies the paper into one or more of 11 research categories, then applies a specialized extraction pipeline for each category — because different types of papers produce different types of usable knowledge. A single paper can yield multiple skills if it spans categories. Use this skill whenever the user wants to turn a paper into a skill, extract practical techniques from research, build a skill library from papers, convert arXiv papers into reusable agent instructions, or batch-process multiple papers into skills. Also trigger when someone asks about extracting actionable knowledge from papers, making research practical for LLM agents, or systematically converting academic contributions into structured agent capabilities.
action-quantization-behavior-cloning Establish regret bounds for behavior cloning with discretized actions combining statistical error and quantization error terms. Prove smoothness requirements for safe quantizer design, show that learning-based quantizers fail these requirements, and propose model-based augmentation to reduce error dependence from H² to H.
adaptive-lora-personalized-ranks Dynamically allocate LoRA ranks per-layer during fine-tuning instead of using fixed uniform ranks. Learn optimal rank for each layer and subject via variational framework with discretized exponential distribution, reducing memory footprint while maintaining fidelity and text-alignment.
Ocupaciones relacionadas SOC
Basado en la clasificación ocupacional SOC
name thinking-to-recall title Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs version 0.0.2 engine skillxiv-v0.0.2-claude-opus-4.6 license MIT url https://arxiv.org/abs/2603.09906 keywords ["Reasoning","Parametric Knowledge","Inference Time Scaling","Prompt Engineering","Fact Retrieval"] description Demonstrates that chain-of-thought reasoning improves LLM factual retrieval through computational buffering and self-priming. Improves single-hop factual accuracy by enabling models to generate contextual bridge facts before recalling answers.
Thinking to Recall: Unlocking Parametric Knowledge Through Reasoning
LLMs perform well on multi-hop reasoning tasks but surprisingly struggle on simple single-hop factual questions that require no complex logic. Enabling chain-of-thought reasoning dramatically improves performance on these supposedly simple queries. The question: why does reasoning help when the task requires no reasoning?
Through controlled experiments isolating different reasoning effects, this skill reveals three mechanisms: computational buffering (generating tokens to "warm up"), factual priming (recalling related facts that enable main answer), and verification (intermediate facts provide checkpoints). By understanding these mechanisms, you can optimize prompting strategies for pure factual retrieval tasks.
Core Concept
Reasoning doesn't just help complex tasks—it unlocks parametric knowledge on simple factual questions through:
Computational Buffer : Model generates reasoning tokens independently of semantic content; this computation alone improves main answer quality (even with random text of same length)
Factual Priming : Model generates topically related facts during reasoning that serve as "bridges" to the target answer, like priming in human memory
Hallucination Detection : Intermediate facts in reasoning traces can be verified, identifying when the trajectory will fail
Architecture Overview
Chain-of-Thought Prompting : Request thinking before answering for simple factsIntermediate Fact Extraction : Parse generated reasoning to identify stated factsFact Verification : Cross-check extracted facts against knowledge basesTrajectory Quality Assessment : Predict answer correctness from intermediate fact validityCalibration : Only prioritize high-confidence reasoning trajectories
Implementation Steps
Implement fact-aware reasoning with intermediate verification to improve single-hop factual recall.
Compute Reasoning Effect Isolation
import torch
from typing import List , Tuple
class ReasoningEffectAnalyzer :
"""Analyze what component of reasoning improves factual recall."""
def
__init__
self, model, tokenizer, knowledge_base
self
self
self
def
measure_reasoning_benefit
self, question: str , ground_truth: str
dict
"""
Measure three components of reasoning benefit.
Args:
question: factual question
ground_truth: correct answer
Returns:
effects: dict with {computational_buffer, factual_priming, verification}
"""
self
self
self
len
" "
"filler"
f"{question} \n{filler_text} \nAnswer:"
self
'computational_buffer'
and
not
self
for
in
if
self
'factual_priming'
len
0
self
for
in
'verification_accuracy'
sum
len
1e-6
'baseline_correct'
'reasoning_correct'
'improvement'
and
not
return
def
_get_direct_answer
self, question: str
str
"""Get model's direct answer without reasoning."""
f"{question} \nAnswer:"
return
self
def
_get_reasoning_output
self, question: str
str
"""Get model's reasoning trajectory."""
f"{question} \nLet me think about this step by step:"
return
self
def
_extract_answer
self, reasoning_output: str
str
"""Extract final answer from reasoning output."""
if
"Answer:"
in
return
"Answer:"
1
'\n'
0
return
'\n'
1
def
_extract_facts
self, text: str
List
str
"""Extract factual statements from text."""
for
in
'.'
if
return
def
_is_related_to_question
self, fact: str , question: str
bool
"""Check if fact is related to question topic."""
set
set
len
return
2
def
_verify_fact
self, fact: str
bool
"""Verify if fact is true according to knowledge base."""
return
self
def
_get_answer_from_prompt
self, prompt: str
str
"""Get model's response to prompt."""
self
'pt'
with
self
50
0.1
return
self
0
Fact-Aware Reasoning Prompting
class FactAwareReasoningPrompt :
"""Generate prompts that leverage factual priming."""
def __init__ (self, knowledge_base, model ):
self .kb = knowledge_base
self .model = model
def construct_priming_prompt (self, question: str , related_entity: str = None ) -> str :
"""
Construct prompt with factual priming.
Args:
question: factual question
related_entity: optional entity to seed priming
Returns:
prompt: formatted prompt with priming hints
"""
if related_entity is None :
related_entity = self ._extract_main_entity(question)
related_facts = self .kb.get_facts_about(related_entity)[:3 ]
prompt = f"""Context facts:
{chr (10 ).join(related_facts)}
Question: {question}
Let me think step by step:"""
return prompt
def construct_verification_prompt (self, question: str , reasoning_trace: str ) -> str :
"""
Construct prompt to verify reasoning trace.
Args:
question: original question
reasoning_trace: model-generated reasoning
Returns:
prompt: verification prompt
"""
facts = self ._extract_sentences(reasoning_trace)
verification_prompt = f"""Evaluate the following reasoning for the question: {question}
Reasoning steps:
"""
for i, fact in enumerate (facts):
verification_prompt += f"\n{i+1 } . {fact} \n"
verification_prompt += " Is this fact correct? (yes/no): "
return verification_prompt
def _extract_main_entity (self, question: str ) -> str :
"""Extract primary entity from question."""
words = question.split()
return max (words, key=len ) if words else ""
def _extract_sentences (self, text: str ) -> List [str ]:
"""Extract sentences from text."""
return [s.strip() for s in text.split('.' ) if s.strip()]
class ReasoingCalibrationModule :
"""Calibrate confidence in reasoning trajectories."""
def __init__ (self, model, knowledge_base ):
self .model = model
self .kb = knowledge_base
def calibrate_reasoning_quality (self, question: str , reasoning_trace: str ) -> float :
"""
Estimate correctness probability of reasoning trajectory.
Args:
reasoning_trace: model's generated reasoning
Returns:
confidence: probability that final answer is correct (0-1)
"""
facts = self ._extract_facts(reasoning_trace)
verification_scores = []
for fact in facts:
is_true = self .kb.is_true(fact)
verification_scores.append(1.0 if is_true else 0.0 )
if not verification_scores:
return 0.5
weighted_verification = sum (
(1 - 0.1 * i) * score
for i, score in enumerate (verification_scores)
) / len (verification_scores)
return weighted_verification
def _extract_facts (self, text: str ) -> List [str ]:
"""Extract factual claims."""
return [s.strip() for s in text.split('.' ) if s.strip()]
def filter_by_reasoning_quality (
self,
candidates: List [Tuple [str , str ]],
threshold: float = 0.7
) -> List [Tuple [str , str ]]:
"""
Filter candidates by reasoning quality.
Args:
candidates: list of (reasoning_trace, answer) pairs
threshold: minimum quality score
Returns:
filtered: high-quality candidates only
"""
filtered = []
for reasoning, answer in candidates:
quality = self .calibrate_reasoning_quality("" , reasoning)
if quality >= threshold:
filtered.append((reasoning, answer))
return filtered
def improved_factual_qa (
question: str ,
model,
knowledge_base,
num_samples: int = 3 ,
use_calibration: bool = True
) -> str :
"""
Improved factual QA leveraging reasoning effects.
Args:
question: factual question
model: language model
knowledge_base: verified fact database
num_samples: number of reasoning trajectories to generate
use_calibration: whether to filter by reasoning quality
Returns:
answer: best answer considering reasoning quality
"""
prompter = FactAwareReasoningPrompt(knowledge_base, model)
calibrator = ReasoingCalibrationModule(model, knowledge_base)
prompt = prompter.construct_priming_prompt(question)
candidates = []
for _ in range (num_samples):
reasoning = _generate_with_temperature(model, prompt, temperature=0.7 )
answer = extract_answer_from_reasoning(reasoning)
candidates.append((reasoning, answer))
if use_calibration:
high_quality = calibrator.filter_by_reasoning_quality(candidates, threshold=0.6 )
candidates = high_quality if high_quality else candidates
answers = [ans for _, ans in candidates]
most_common = max (set (answers), key=answers.count)
return most_common
Practical Guidance
Reasoning length: 2-4 sentences works well; longer shows diminishing returns
Fact priming count: 3 related facts optimal
Verification threshold: 0.6-0.7 (allows some flexibility)
Candidate generation: 3-5 samples for majority voting
Single-hop factual questions where baseline performance is poor
Questions about well-covered topics in training data
Scenarios where you can verify facts through external KB
Improving calibration/confidence scores
Multi-hop reasoning (standard chain-of-thought works)
Questions about rare/recent facts
No access to knowledge base for verification
Real-time applications where reasoning adds latency
Priming facts too different from question—doesn't help
Verification against wrong KB—false confidence
Reasoning trajectories too long—diminishing returns
Not using multiple samples—single trajectory may be unlucky
Integration Notes : Works as a prompting strategy; requires knowledge base for fact verification; can be combined with retrieval-augmented generation for external fact access.
Evidence : Improves single-hop factual accuracy 5-15% through reasoning; computational buffering alone provides 2-3% improvement; fact verification enables proper calibration, reducing hallucination rates.
