multi-agent-evolve

name	multi-agent-evolve
title	Multi-Agent Evolve: LLM Self-Improve through Co-evolution
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2510.23595
keywords	["Self-Play","Co-evolution","Reasoning","RL","LLM Training"]
description	Enables LLM self-improvement without external verification through multi-agent co-evolution. Proposer generates questions, Solver attempts solutions, Judge evaluates both. All agents evolve together via RL, achieving 4.54% improvement on reasoning benchmarks without human supervision.

Multi-Agent Evolve: Self-Improvement via Co-evolutionary RL

LLM reasoning improvement typically requires external verifiers (Python, human feedback). Multi-Agent Evolve eliminates this dependency through three co-evolving agents that provide internal supervision.

The proposer-solver-judge framework creates a closed learning loop where agents improve by interacting with each other.

Core Concept

Three specialized agents from single base LLM:

• Proposer: generates diverse questions
• Solver: attempts to solve them
• Judge: evaluates both question quality and solution correctness

All three undergo RL simultaneously, creating adaptive curriculum as proposer generates harder questions.

Architecture Overview

• Three agent roles instantiated from same LLM
• Shared parameter backbone with role-specific prompting
• Reward signals derived from agent interactions
• Co-evolutionary training loop with alternating optimization

Implementation Steps

Define agent roles with role-specific prompting. Each agent is a view of the same LLM with different prompts:

class MultiAgentSystem:
    def __init__(self, base_llm, model_size='3B'):
        self.base_llm = base_llm
        self.model_size = model_size

        # Role-specific system prompts
        self.proposer_prompt = (
            "You are a question generator. Generate challenging questions "
            "that test reasoning. Format: QUESTION: [question]"
        )

        self.solver_prompt = (
            "You are a problem solver. Given a question, provide step-by-step "
            "reasoning and a final answer. Format: REASONING: [steps]\nANSWER: [answer]"
        )

        self.judge_prompt = (
            "You are an evaluator. Rate question quality (1-5) and solution "
            "correctness (1-5). Format: QUESTION_SCORE: [score]\nSOLUTION_SCORE: [score]"
        )

     ():
        
        prompt = .proposer_prompt + 
        question = .base_llm.generate(prompt, max_tokens=)
         question

     ():
        
        prompt = .solver_prompt + 
        solution = .base_llm.generate(prompt, max_tokens=)
         solution

     ():
        
        prompt = .judge_prompt
        prompt += 
        evaluation = .base_llm.generate(prompt, max_tokens=)

        
        scores = ._parse_scores(evaluation)
         scores

     ():
        
        
        question_score =   
        solution_score = 

         re
        q_match = re.search(, evaluation)
        s_match = re.search(, evaluation)

         q_match:
            question_score = (q_match.group())
         s_match:
            solution_score = (s_match.group())

         question_score, solution_score