Replace parallel self-consistency with sequential reasoning where chains iteratively build on previous attempts, weighted by inverse entropy to prioritize confident solutions, achieving 46.7 pp accuracy gains over parallel approaches.
Replace parallel self-consistency with sequential reasoning where chains iteratively build on previous attempts, weighted by inverse entropy to prioritize confident solutions, achieving 46.7 pp accuracy gains over parallel approaches.
Title: Build Reasoning Chains Sequentially for Superior Accuracy
The dominant paradigm for improving language model reasoning is parallel self-consistency: run N independent reasoning chains, then vote. Sequential Edge shows this is suboptimal. By running chains sequentially where each explicitly builds on previous attempts, models achieve 95.6% superiority rates over parallel approaches. Further, weighting solutions by inverse entropy (favoring low-entropy/confident chains) outperforms majority voting universally.
This is a pure inference-time technique requiring no model retraining.
Core Concept
Sequential Reasoning With Entropy-Weighted Aggregation:
Sequential Scaling: Each chain reads previous attempts and tries to improve them
Iterative Refinement: Three mechanisms available: error correction, context accumulation, verification
Inverse-Entropy Voting: Weight each answer by how certain the model was (low entropy = high confidence)
No Training Required: Works with base models or fine-tuned variants
Universal Superiority: Better than parallel approaches across all tested models and benchmarks
Architecture Overview
Sequential Chain Generation: Causal generation where chain k reads output of chain k-1
Confidence Estimation: Compute Shannon entropy from token-level probability distributions
Entropy-Weighted Aggregation: Answers weighted inversely to their generation entropy
Stopping Criterion: Continue sequencing until convergence or budget exhausted
Hybrid Architecture: Can mix sequential and parallel chains for different problem types
Implementation Steps
1. Implement Sequential Chain Generation
Generate reasoning chains that explicitly reference previous attempts.
"""Generate chains sequentially with cross-chain reference"""
# First chain: independent reasoning
f"""Solve this problem step by step:
{question}
Solution:"""
self
# Subsequent chains: build on previous
for
in
range
1
# Acknowledge previous attempt
1
f"""Previous attempt at this problem:
{previous_solution}
Reconsider the problem {question}
Try a different approach to check if the previous solution is correct.
New solution:"""
self
return
def
generate_chain_with_entropy
self, prompt
"""Generate reasoning chain and compute entropy"""
self
'pt'
# Generate with return_dict_in_generate to get log probabilities
self
300
True
True
self
# Decode generated tokens
0
1
self
True
# Compute entropy over generated tokens
# outputs.scores: tuple of tensors [batch_size, vocab_size] for each position
self
return
def
compute_sequence_entropy
self, scores
"""Compute entropy of token probability distribution"""
for
in
# Get probabilities from logits
1
# Shannon entropy: -sum(p * log(p))
1e-10
sum
1
# Return average entropy over sequence
return
if
else
0.0
2. Implement Inverse-Entropy Voting
Weight answers by confidence (inverse of entropy).
classInverseEntropyVoter:
def__init__(self, extraction_fn):
"""extraction_fn: function that extracts answer from solution text"""self.extract_answer = extraction_fn
defextract_answers_and_weights(self, chains, entropies):
"""Extract final answers and compute entropy-based weights"""
answers = []
weights = []
for chain, entropy inzip(chains, entropies):
# Extract answer from chain
answer = self.extract_answer(chain)
answers.append(answer)
# Weight inversely to entropy: low entropy -> high weight# Use exponential scaling for sharper differences
weight = np.exp(-entropy)
weights.append(weight)
# Normalize weights to sum to 1
total_weight = sum(weights)
weights = [w / total_weight for w in weights]
return answers, weights
defweighted_vote(self, chains, entropies, answer_similarity_fn=None):
"""Aggregate answers weighted by inverse entropy"""
answers, weights = self.extract_answers_and_weights(chains, entropies)
# Group similar answersif answer_similarity_fn:
answer_groups = self.group_similar_answers(answers, answer_similarity_fn)
else:
answer_groups = {ans: [i for i, a inenumerate(answers) if a == ans]
for ans inset(answers)}
# Compute weighted score for each answer group
group_scores = {}
for answer, indices in answer_groups.items():
score = sum(weights[i] for i in indices)
group_scores[answer] = score
# Return highest-scoring answer
final_answer = max(group_scores, key=group_scores.get)
confidence = group_scores[final_answer]
return final_answer, confidence, group_scores
defgroup_similar_answers(self, answers, similarity_fn):
"""Group answers that are semantically similar"""
groups = {}
for i, answer inenumerate(answers):
# Find most similar existing group
best_group = None
best_similarity = 0for group_answer in groups:
sim = similarity_fn(answer, group_answer)
if sim > best_similarity:
best_similarity = sim
best_group = group_answer
if best_group and best_similarity > 0.8:
groups[best_group].append(i)
else:
groups[answer] = [i]
return groups
3. Implement Hybrid Sequential-Parallel Strategy
Combine approaches for complex problems.
classHybridReasoningStrategy:
def__init__(self, model, tokenizer, num_sequential=3, num_parallel=3):
self.sequential_gen = SequentialReasoningGenerator(model, tokenizer)
self.voter = InverseEntropyVoter(self.extract_final_answer)
self.num_seq = num_sequential
self.num_par = num_parallel
defsolve_with_budget(self, question, token_budget=10000):
"""Solve using hybrid strategy within token budget"""# Phase 1: Sequential reasoning (focus on quality)
seq_chains, seq_entropies = self.sequential_gen.generate_sequential_chains(
question, self.num_seq
)
tokens_used = sum(len(c.split()) for c in seq_chains) * 1.3# Estimate# Phase 2: Parallel chains if budget remainsif tokens_used < token_budget * 0.7:
parallel_chains = []
for _ inrange(self.num_par):
prompt = f"Solve: {question}\nSolution:"
chain, _ = self.sequential_gen.generate_chain_with_entropy(prompt)
parallel_chains.append(chain)
# Combine all chains
all_chains = seq_chains + parallel_chains
all_entropies = seq_entropies + [0.0] * len(parallel_chains) # Parallel have no reference entropyelse:
all_chains = seq_chains
all_entropies = seq_entropies
# Aggregate
answer, confidence, scores = self.voter.weighted_vote(all_chains, all_entropies)
return answer, confidence
defextract_final_answer(self, solution_text):
"""Extract final answer from solution"""# Simple strategy: last numeric value or final sentenceimport re
numbers = re.findall(r'-?\d+\.?\d*', solution_text)
if numbers:
returnfloat(numbers[-1])
# Fallback: last sentence
sentences = solution_text.split('.')
return sentences[-2] iflen(sentences) > 1else solution_text
entropy_temperature: 1.0 (controls sharpness of entropy-based weighting)
token_budget: Adjust based on latency constraints
When NOT to Use:
Real-time applications (sequential generation is slower than parallel)
Streaming scenarios where latency is critical
Models without reliable probability distributions
Pitfalls:
Answer extraction brittleness: Regex/heuristic extraction fails on varied formats; use semantic matching
Entropy unreliability: Some models produce low-entropy nonsense; validate with baseline
Sequential dependency: If first chain is very wrong, subsequent chains may not recover; use restart strategies
Key Insight: This is one of the few inference-time tricks that consistently beats the dominant approach. The mechanism is simple: sequential reasoning enables error correction that parallel approaches miss.
