progressive-citation-grounded-dialogue

name	progressive-citation-grounded-dialogue
title	Progressive Training for Citation-Grounded Dialogue: Reducing Hallucination in English-Hindi LLMs
version	0.0.3
engine	skillxiv-v0.0.3-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.18911
keywords	["Citation-Grounded Generation","Hallucination Reduction","Dialogue Systems","Multilingual Models","Structured Generation"]
description	Eliminate hallucination via four-stage progressive training: multilingual adaptation → English dialogue SFT → bilingual SFT → GRPO alignment. Achieve 0.0% hallucination rate for encoder-decoder models using structured citation markers and knowledge-source attribution, with automatic transfer of citation format across languages.

Progressive Citation-Grounded Dialogue

Ranked Findings

Finding 1 (Highest Impact): Citation-Grounded SFT Eliminates Hallucination

• Stages 2+ achieve 0.0% hallucination (encoder-decoder models under NLI-based evaluation)
• Structured format (citations to source passages) forces models to ground outputs
• No downstream GRPO alignment needed for hallucination elimination

Finding 2: Encoder-Decoder Architecture Superior to Decoder-Only

• Flan-T5 models: consistent 0.0% hallucination
• LLaMA-3.2-1B: language-selective citation failure (English works, Hindi fails)
• Mistral: larger decoders (7B) partially recover but never reach encoder-decoder parity
• Implication: model architecture matters as much as training procedure

Finding 3: Smaller Models Can Match Larger Ones with Structured Training

• Flan-T5-250M reaches English metrics of Flan-T5-780M after Stage 2
• Structured citation task requires <250M capacity if properly formatted
• Training recipe supersedes raw model size for hallucination control

Finding 4: GRPO Alignment Provides Marginal Gains

• Well-tuned SFT already achieves strong performance
• GRPO adds <2% improvement on citation-F1 and factuality scores
• Resource cost (reward modeling, RL training) not justified for this task
• Implication: structured generation tasks may not need RL fine-tuning

Finding 5: Cross-Lingual Citation Transfer is Automatic

• Models trained on English citation format automatically apply format to Hindi
• No need for language-specific instruction tuning for citation structure
• Citation formatting is learned as abstract pattern, not language-dependent

Decision Checklist for Practitioners

1. Choose Encoder-Decoder Foundation: Start with encoder-decoder models (Flan-T5, mBART)

   • ✓ Encoder-decoder models exhibit 0.0% hallucination
   • ✗ Avoid decoder-only for critical applications unless scaling to 7B+

2. Prepare Citation-Grounded Data: Format SFT data with explicit source attributions

   • Format: "[Response text] [CITE: source_passage_id, source_passage_id]"
   • Verify source passages are included in context; no floating citations
   • Target: 1,000+ citation-grounded training examples minimum

3. Run Stage 2 (English Dialogue SFT) First: