goedel-prover-formal-theorem-proving

name	goedel-prover-formal-theorem-proving
title	Goedel-Prover-V2 - Scaling Formal Theorem Proving with Expert Iteration
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2508.03613
keywords	["formal-verification","theorem-proving","lean","reinforcement-learning"]
description	Train language models for formal theorem proving via expert iteration with verifier-guided self-correction and checkpoint merging.

Goedel-Prover-V2: Expert Iteration for Formal Theorem Proving

Goedel-Prover-V2 trains LLMs to prove mathematical theorems in the Lean proof assistant through expert iteration: iteratively sampling proofs, checking them against the Lean compiler, and using corrections as training data. The breakthrough is recognizing that verifier feedback (compiler errors) enables models to self-correct, eliminating need for human demonstrations beyond base training.

Core Concept

Formal theorem proving requires exact syntax and logical correctness—tasks where LLMs typically fail without examples. Rather than requiring humans to write proofs, Goedel-Prover leverages the Lean compiler as an automatic teacher: generate a proof, get compiler feedback (error messages), revise, and learn from successful iterations. This "verifier-guided self-correction" is both more scalable and more aligned with actual proving workflows.

Architecture Overview

• Expert Iteration Pipeline: Sample → Verify → Correct → Train loop with Lean compiler as oracle
• Scaffolded Data Synthesis: Generate synthetic theorems of increasing difficulty to create curriculum
• Verifier-Guided Refinement: Use Lean compiler error messages to iteratively fix proofs
• Checkpoint Merging: Average model checkpoints during training to preserve output diversity and prevent mode collapse in later RL stages
• Multi-Scale Models: Train 8B and 32B variants; smaller well-trained models outperform larger untrained ones

Implementation Steps

Step 1: Set Up Lean Environment and Proof Checker

import subprocess
import json
from typing import Tuple, Optional

class LeanProofChecker:
    """Interface to Lean compiler for proof verification."""

    def __init__(self, lean_path: str = "lean"):
        self.lean_path = lean_path

    def check_proof(self, theorem_statement: str, proof_code: str) -> Tuple[bool, str]:
        """
        Check if proof is valid in Lean.

        Returns: (is_valid, feedback)
        """
        full_code = f"""
theorem problem : {theorem_statement} := by
{proof_code}
"""
        try:
            result = subprocess.run(
                [self.lean_path, "--stdin"],
                input=full_code,
                capture_output=True,
                text=True,
                timeout=10
            )

            # Check if compilation succeeded
            if result.returncode == 0:
                return True, "Proof verified"
            else:
                # Extract error messages
                error_msg = result.stderr + result.stdout
                return False, error_msg

         subprocess.TimeoutExpired:
             , 
         Exception  e:
             , (e)


checker = LeanProofChecker()
is_valid, feedback = checker.check_proof(
    ,
    
)

Step 2: Implement Scaffolded Data Synthesis

def generate_synthetic_theorems(base_theorems: list, complexity_levels: int = 5) -> dict:
    """
    Generate theorems of increasing difficulty from base set.
    Curriculum learning: start with simple, progress to complex.
    """
    synthetic_data = {}

    for level in range(complexity_levels):
        theorems_at_level = []

        for base_theorem in base_theorems:
            # Level 0: Original theorems
            if level == 0:
                theorems_at_level.append(base_theorem)
            else:
                # Levels 1+: Generalize and extend theorems
                generalized = generalize_theorem(base_theorem, level)
                theorems_at_level.append(generalized)

        synthetic_data[f'level_{level}'] = theorems_at_level

    return synthetic_data

def generalize_theorem(theorem: str, complexity_level: int) -> str:
    """Make theorems more complex by adding parameters, conditions."""
    # Example: transform ∀ n, n+0=n into ∀ a b, a*(n+0) = a*n
    if complexity_level == 1:
        return theorem.replace("n", "(a + b * n)")
    elif complexity_level == 2:
        return theorem.replace("n", "(f (g n))")  # Compose functions
    else:
         

Step 3: Expert Iteration Loop

import random

class ExpertIteration:
    """Expert iteration: sample → check → correct → train."""

    def __init__(self, model, checker: LeanProofChecker, learning_rate=1e-5):
        self.model = model
        self.checker = checker
        self.training_data = []
        self.lr = learning_rate

    def sample_proofs(self, theorem: str, num_samples: int = 5) -> list:
        """Generate multiple proof attempts."""
        proofs = []

        for _ in range(num_samples):
            # Sample from model (use temperature > 0 for diversity)
            proof_text = self.model.generate(
                f"Prove: {theorem}\nProof:",
                temperature=0.8,
                max_tokens=500
            )
            proofs.append(proof_text)

        return proofs

    def verify_and_filter(self, theorem: str, proofs: list) -> Tuple[list, list]:
        """Check proofs, separate correct from incorrect."""
        correct_proofs = []
        incorrect_proofs = []

        for proof in proofs:
            is_valid, feedback = self.checker.check_proof(theorem, proof)

             is_valid:
                correct_proofs.append((proof, feedback))
            :
                incorrect_proofs.append((proof, feedback))

         correct_proofs, incorrect_proofs

     () -> :
        
        corrected = []

         wrong_proof, error_msg  incorrect_proofs:
            
            correction_prompt = 

            fixed_proof = .model.generate(correction_prompt, max_tokens=)
            is_valid, feedback = .checker.check_proof(theorem, fixed_proof)

             is_valid:
                corrected.append(fixed_proof)

         corrected

     ():
        
         proof, theorem  (proofs, theorems):
            
            prompt = 
            
            loss = .model.compute_loss(prompt, proof)
            loss.backward()

        .model.optimizer.step()

     ():
        
        all_correct = []

         theorem  theorems:
            
            proofs = .sample_proofs(theorem, num_samples)

            
            correct, incorrect = .verify_and_filter(theorem, proofs)
            all_correct.extend([p  p, _  correct])

            
             _  (max_iters):
                  incorrect:
                    

                corrected = .self_correct(theorem, incorrect)
                all_correct.extend(corrected)

                
                correct, incorrect = .verify_and_filter(
                    theorem,
                    [p  p, _  incorrect]
                )

        
         all_correct:
            .train_on_successful_proofs(all_correct, theorems)

         (all_correct)

Step 4: Implement Checkpoint Merging

def merge_checkpoints(checkpoints: list, weights: list = None) -> dict:
    """
    Average model checkpoints to prevent mode collapse.
    RL training can reduce output diversity; averaging preserves it.
    """
    if weights is None:
        weights = [1.0 / len(checkpoints)] * len(checkpoints)

    merged_state = {}

    # Average weights across checkpoints
    for param_name in checkpoints[0].keys():
        merged_state[param_name] = sum(
            w * ckpt[param_name] for w, ckpt in zip(weights, checkpoints)
        )

    return merged_state

class CheckpointManager:
    """Manage checkpoints and periodic averaging."""

    def __init__(self, model, averaging_frequency: int = 100):
        self.model = model
        self.checkpoints = []
        self.averaging_frequency = averaging_frequency
        self.step = 0

    def save_checkpoint(self):
        """Save current model state."""
        self.checkpoints.append(self.model.state_dict().copy())

    def maybe_merge():
        
        .step += 

         .step % .averaging_frequency ==   (.checkpoints) > :
            
            recent = .checkpoints[-:]
            merged = merge_checkpoints(recent)
            .model.load_state_dict(merged)

            
            .checkpoints = [merged]

Step 5: Full Training Pipeline

def train_goedel_prover(
    model,
    theorem_dataset: list,
    num_epochs: int = 3,
    samples_per_theorem: int = 5
):
    """
    Complete training pipeline: expert iteration with curriculum.
    """
    checker = LeanProofChecker()
    expert_iter = ExpertIteration(model, checker)
    checkpoint_mgr = CheckpointManager(model)

    # Sort by difficulty (curriculum)
    difficulties = compute_theorem_difficulty(theorem_dataset)
    sorted_theorems = sorted(zip(theorem_dataset, difficulties), key=lambda x: x[1])

    for epoch in range(num_epochs):
        total_proofs = 0

        for theorem, difficulty in sorted_theorems:
            # Adjust samples based on difficulty
            num_samples = max(3, samples_per_theorem - difficulty // 10)

            num_correct = expert_iter.run_iteration(
                [theorem],
                num_samples=num_samples,
                max_iters=3
            )

            total_proofs += num_correct

            # Periodic checkpoint averaging
            checkpoint_mgr.save_checkpoint()
            checkpoint_mgr.maybe_merge()

        print(f"Epoch {epoch}: {total_proofs} verified proofs")

def compute_theorem_difficulty(theorems: list) -> list:
    """Heuristic: theorem length correlates with difficulty."""
    return [len(t.split()) for t  theorems]

Practical Guidance

When to Use:

• Formal verification tasks with checkable proofs (Lean, Coq, Isabelle)
• Scenarios with large theorem libraries for curriculum learning
• Applications where proof correctness is mandatory
• Cases where iterative refinement is preferred to human demonstrations

When NOT to Use:

• Informal mathematical reasoning without formal verification
• Real-time inference (proof generation is slow)
• Domains without reliable proof checkers
• Scenarios with <100 training theorems (insufficient curriculum)

Hyperparameters:

Parameter	Default	Impact
samples_per_theorem	5	More samples = better coverage, higher compute cost
max_correction_iters	3	Iterations of self-correction; diminishing returns after 3
checkpoint_averaging_freq	100	More frequent averaging = higher diversity, slower training
curriculum_start_difficulty	0	Begin with easiest theorems; increase gradually

Reference

Paper: Goedel-Prover-V2: Scaling Formal Theorem Proving (2508.03613)

• Expert iteration with verifier feedback
• 32B model achieves SOTA on formal theorem benchmarks
• Self-correction via compiler error messages
• Checkpoint merging prevents mode collapse