critic-guided-formalization

name	critic-guided-formalization
title	CriticLean: Critic-Guided Reinforcement Learning for Mathematical Formalization
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2507.06181
keywords	["Theorem Proving","Formal Verification","Reinforcement Learning","Mathematical Reasoning","Semantic Correctness"]
description	Improve formal theorem proofs by treating criticism—evaluation of semantic correctness—as a learning signal. Train critic models to distinguish correct from incorrect formalizations, then use their feedback to guide RL-based proof generation.

CriticLean: Elevating Critique as a Learning Signal for Formal Verification

Translating informal mathematics into formal, executable code (e.g., Lean 4) requires not just generating syntactically correct proofs but ensuring they capture the original mathematical intent. Prior work focused on generation and compilation; CriticLean shifts focus to the critic phase—the evaluation of whether a formalization is semantically correct. By training critic models to assess semantic accuracy and using their feedback as a reinforcement learning signal, CriticLean improves both the quality of generated proofs and the reliability of the evaluation process itself.

The core problem is that compiling without semantic verification produces proofs that are technically valid but miss the mathematical meaning. A proof might compile and be "correct" in isolation yet fail to capture what the original problem asked for.

Core Concept

CriticLean operates on three interconnected components:

1. CriticLeanGPT: A critic model trained via supervised fine-tuning and RL to assess whether a Lean 4 formalization semantically matches a natural language problem
2. CriticLeanBench: A benchmark measuring critic ability to distinguish truly correct from subtly incorrect formalizations
3. FineLeanCorpus: A dataset of 285,000+ formalization problems with human evaluation of correctness

The framework elevates criticism from a post-hoc filter to an active learning component that guides proof generation toward semantic fidelity.

Architecture Overview

• Semantic critic model: Classifies whether formal code captures mathematical intent
• Semantic correctness benchmark: Tests critic on challenging cases where proofs compile but are semantically wrong
• RL training loop: Uses critic feedback as reward signal for proof generation
• Human evaluation dataset: 285K problems with domain-diverse and human-validated correctness labels
• Feedback integration: Critic signals guide generation toward semantically correct proofs

Implementation

Build the critic model by fine-tuning on semantic correctness judgments:

import torch
import torch.nn as nn
from transformers import AutoModelForSequenceClassification, AutoTokenizer
from lean.parser import LeanCodeParser


critic_base = AutoModelForSequenceClassification.from_pretrained(
    ,
    num_labels=  
)
tokenizer = AutoTokenizer.from_pretrained()


 criticlean.data  FineLeanCorpus

corpus = FineLeanCorpus(split=)

 ():
    

    
    text = 

    encoding = tokenizer(
        text,
        max_length=,
        truncation=,
        return_tensors=
    )

     {
        : encoding[],
        : encoding[],
        : torch.tensor(label)  
    }


 transformers  Trainer, TrainingArguments

training_args = TrainingArguments(
    output_dir=,
    num_train_epochs=,
    per_device_train_batch_size=,
    learning_rate=,
    eval_strategy=,
    eval_steps=,
    save_strategy=,
    save_steps=,
)

train_dataset = [
    prepare_criticism_example(p, p[], p[])
     p  corpus
]

trainer = Trainer(
    model=critic_base,
    args=training_args,
    train_dataset=train_dataset,
)

critic_model = trainer.train()
()