hyperagents-self-improvement

name	hyperagents-self-improvement
title	Hyperagents: Open-Ended AI Systems via Recursive Self-Modification
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2603.19461
keywords	["Self-Improvement","Meta-Learning","Program Synthesis","Open-Ended Learning"]
description	Enable AI systems to recursively improve themselves by making the meta-level modification procedure itself editable, achieving open-ended capability growth.

Hyperagents: Recursive Self-Improvement Through Editable Meta-Procedures

Most self-improving AI systems hit a fundamental limit: they have a fixed meta-level improvement mechanism. A reinforcement learning agent can improve its policy, but its reward function is static. A program synthesis agent can generate code, but the synthesis procedure itself never changes.

Hyperagents solve this through a simple but profound insight: make everything—including the improvement mechanism itself—subject to modification. This creates a recursive structure where the system improves not just its task performance, but also how it searches for improvements. The result is open-ended capability growth: each iteration improves the system's ability to improve further.

Core Concept

Hyperagents implement a Darwin-Gödel Machine (DGM) with full editability:

Task Agent: Solves the target problem, can be modified.

Meta Agent: Modifies itself and the task agent, can also be modified.

Key Innovation: The meta-modification procedure is itself editable. Rather than having humans design the self-improvement algorithm, the system evolves its own improvement strategies.

The system generates variants of itself (including different meta-level procedures), evaluates which variant performs best, and keeps the improvement. This creates a recursive loop where the system literally improves its own source code.

Architecture Overview

• Unified Program Representation: Task and meta agents stored as editable programs
• Self-Variant Generation: Systematically mutate and recombine program code
• Evaluation Framework: Test each variant to measure performance improvement
• Persistent Memory: Store successful modifications for reuse across runs
• Metacognitive Loop: Meta-improvements compound across iterations
• Domain-Agnostic: Works across any domain with computable task and evaluation

Implementation Steps

Step 1: Represent Agents as Editable Programs

Store both task and meta logic as modifiable code.

from dataclasses import dataclass
from typing import Callable, List, Dict, Optional, Any
import hashlib
import copy

@dataclass
class AgentProgram:
    """
    Editable representation of an agent.
    Can be task logic or meta-improvement logic.
    """
    code: str  # Python code defining the agent
    metadata: Dict[str, Any]  # Task params, performance history
    version: int  # Iteration when created
    performance: float = 0.0  # Cached performance score
    creation_time: Optional[str] = None

    def compute_hash(self) -> str:
        """Unique identifier for this program."""
        return hashlib.sha256(self.code.encode()).hexdigest()[:8]

    def execute(self, task_input: Any, env_context: Dict = None) -> Any:
        """Run this agent program on a task."""
        # Create execution namespace
        namespace = {'task_input': task_input}
        if env_context:
            namespace.update(env_context)

        :
            (.code, namespace)
             namespace.get(, )
         Exception  e:
             {: (e), : }

     () -> :
        
        new_code = mutation  mutation  .code
        new_program = AgentProgram(
            code=new_code,
            metadata=copy.deepcopy(.metadata),
            version=.version + ,
            performance=
        )
         new_program

 :
    

     ():
        .task_agent = initial_task_agent
        .meta_agent = initial_meta_agent
        .improvement_history = []  
        .persistent_modifications = {}  

     () - :
        

        
        task_code = 

        
        meta_code = 

        .task_agent = AgentProgram(
            code=task_code,
            metadata={: task_description},
            version=
        )

        .meta_agent = AgentProgram(
            code=meta_code,
            metadata={: },
            version=
        )

Step 2: Generate Self-Variants

Create mutations of task and meta agents.

import random
import re

class VariantGenerator:
    """Generate program variants through mutation."""

    def __init__(self):
        self.mutation_operators = [
            self.add_parameter,
            self.refactor_logic,
            self.add_early_exit,
            self.add_memoization,
            self.modify_constants
        ]

    def generate_variants(self, program: AgentProgram,
                         num_variants: int = 5) -> List[AgentProgram]:
        """Create multiple variants of a program."""
        variants = []

        for _ in range(num_variants):
            mutation_fn = random.choice(self.mutation_operators)
            mutated_code = mutation_fn(program.code)
            variant = program.clone(mutation=mutated_code)
            variants.append(variant)

        return variants

    def add_parameter(self, code: str) -> str:
        """Introduce tunable parameters."""
        # Find return statements and add a parameter multiplier
        modified = re.sub(
            r'return ([^;\n]+)',
            r'return param * (\1)',
            code
        )
        # Initialize parameter
        modified = 'param = 1.0\n' + modified
        return modified

    def () -> :
        
        
           code    code:
            
            modified = code.replace(, )
             modified
         code

     () -> :
        
        modified = code.replace(
            ,
            
        )
        modified = re.sub(
            ,
            ,
            modified
        )
         modified

     () -> :
        
        modified =  + code
        modified = modified.replace(
            ,
            
        )
        modified = modified.replace(
            ,
            
        )
         modified

     () -> :
        
         ():
            num = (.group())
            
            adjusted = num * ( +  * random.random())
             (adjusted)

        modified = re.sub(, adjust_number, code)
         modified

Step 3: Evaluate Variants

Test variants and identify improvements.

class PerformanceEvaluator:
    """Measure agent performance on tasks."""

    def __init__(self, test_tasks: List[Dict], success_metric: Callable):
        self.test_tasks = test_tasks
        self.success_metric = success_metric

    def evaluate_agent(self, agent: AgentProgram,
                      num_evals: int = 10) -> Dict[str, float]:
        """
        Run agent on test tasks and measure performance.
        Returns: {accuracy, speed, stability, etc.}
        """

        results = {
            'success_rate': 0.0,
            'avg_latency': 0.0,
            'error_count': 0,
            'consistency': 0.0
        }

        successes = 0
        latencies = []
        outputs = []

        for task in self.test_tasks[:num_evals]:
            import time
            start = time.time()

            try:
                output = agent.execute(task)
                latency = time.time() - start

                if self.success_metric(output, task):
                    successes += 1

                latencies.append(latency)
                outputs.append(output)

            except Exception as e:
                results['error_count'] += 1

        
        results[] = successes / (num_evals, )
        results[] = (latencies) / ((latencies), )

        
        consistency_score =   (((o)  o  outputs)) ==   
        results[] = consistency_score

        
        results[] = (
             * results[] +
             * ( / (results[], )) +
             * results[]
        )

         results

     () -> AgentProgram:
        
        best_variant = 
        best_score = -()

         variant  variants:
            results = .evaluate_agent(variant)
            score = results[]

             score > best_score:
                best_score = score
                best_variant = variant

         best_variant:
            best_variant.performance = best_score

         best_variant

Step 4: Recursive Meta-Improvement

Let the meta-agent improve itself.

class RecursiveImprover:
    """
    Enables the meta-agent to improve itself.
    This is the key to open-ended growth.
    """

    def __init__(self, evaluator: PerformanceEvaluator):
        self.evaluator = evaluator
        self.meta_improvement_history = []
        self.variant_generator = VariantGenerator()

    def improve_task_agent(self, task_agent: AgentProgram,
                          meta_agent: AgentProgram) -> AgentProgram:
        """Use meta-agent to improve task-agent."""

        # Get meta-agent's suggested improvement
        improvement_suggestion = meta_agent.execute(
            task_input=task_agent.code,
            env_context={'evaluation_results': {}}
        )

        if improvement_suggestion and 'error' not in improvement_suggestion:
            improved_code = improvement_suggestion
        else:
            # Fallback: use variant generator
            improved_code = self.variant_generator.add_parameter(task_agent.code)

        improved_agent = task_agent.clone(mutation=improved_code)
        return improved_agent

    def improve_meta_agent(self, meta_agent: AgentProgram,
                          improvement_history: List[Dict]) -> AgentProgram:
        """
        CRITICALLY: Improve the meta-agent itself.
        This enables open-ended improvement.
        """

        # Generate variants of the meta-agent
        meta_variants = self.variant_generator.generate_variants(
            meta_agent, num_variants=3
        )

        # Evaluate each meta-variant by using it to improve a task agent
        

        best_meta_variant = 
        best_meta_score = -()

         meta_variant  meta_variants:
            
            

            suggested_modification = meta_variant.execute(
                task_input={: },
                env_context={}
            )

            
            is_valid = suggested_modification     (suggested_modification)
            meta_score =   is_valid  

             meta_score > best_meta_score:
                best_meta_score = meta_score
                best_meta_variant = meta_variant

         best_meta_variant  best_meta_score > :
             best_meta_variant

         meta_agent

     () -> [, AgentProgram]:
        

        current_task = task_agent
        current_meta = meta_agent

         iteration  (num_iterations):
            ()

            
            improved_task = .improve_task_agent(current_task, current_meta)
            task_perf = .evaluator.evaluate_agent(improved_task)
            ()

            
            improved_meta = .improve_meta_agent(current_meta, .meta_improvement_history)
            ()

            
            .meta_improvement_history.append({
                : iteration,
                : task_perf[],
                : improved_task.compute_hash(),
                : improved_meta.compute_hash()
            })

            current_task = improved_task
            current_meta = improved_meta

         {
            : current_task,
            : current_meta,
            : .meta_improvement_history
        }

Practical Guidance

Hyperparameters:

• Number of variants per iteration: 3-7 (balance exploration vs. compute)
• Mutation types: use 3-5 different operators (diversity improves search)
• Evaluation budget per variant: 10-50 test cases (faster evals allow more iterations)
• Meta-improvement frequency: every 2-5 task iterations

When to Use:

• Long-running systems where continuous improvement is valuable
• Domains where the improvement strategy itself can vary
• Research environments exploring open-ended learning
• When you have compute budget for recursive evaluation

When NOT to Use:

• Real-time systems (recursive improvement adds latency)
• Safety-critical domains (uncontrolled self-modification is risky)
• Single-shot tasks (improvement overhead not justified)
• Systems requiring formal verification (self-modification hard to analyze)

Pitfalls:

• Runaway mutations: without checks, code can diverge into nonsense; validate structure
• Evaluation noise: small performance differences lead to random direction; use multiple runs
• Positive feedback loops: once an improvement strategy works, it dominates; diversify
• Unbounded code growth: programs can bloat; track code size and penalize if necessary

Reference

Paper: arxiv.org/abs/2603.19461