hyperagents-self-improving-ai

# HyperAgents Self-Improving AI Skill > Skill by [ara.so](https://ara.so) — AI Agent Skills collection. ## Overview HyperAgents is a framework for building self-referential self-improving AI agents that can optimize for any computable task. The system uses a meta-agent to iteratively improve a task-agent by generating and evaluating code modifications. The framework supports multiple domains (code generation, reasoning, math, etc.) and uses foundation models to drive the self-improvement loop. **Key Capabilities:** - Self-referential meta-learning where agents modify their own code - Multi-domain support (code, math, reasoning tasks) - Iterative improvement through generation-evaluation loops - Integration with OpenAI, Anthropic, and Google Gemini models - Docker-based safe execution environment ## Installation ### Prerequisites ```bash # Install system dependencies (Fedora/RHEL) sudo dnf install -y python3.12-devel graphviz graphviz-devel cmake ninja-build bzip2-devel zlib-devel ncurses-devel libffi-devel # For Ubuntu/Debian: # sudo apt-get install -y python3.12-dev graphviz libgraphviz-dev cmake ninja-build libbz2-dev zlib1g-dev libncurses-dev libffi-dev ``` ### Setup ```bash # Clone the repository git clone https://github.com/facebookresearch/HyperAgents.git cd HyperAgents # Create virtual environment python3.12 -m venv venv_nat source venv_nat/bin/activate # Install dependencies pip install -r requirements.txt pip install -r requirements_dev.txt # Build Docker container for safe execution docker build --network=host -t hyperagents . ``` ### Environment Configuration Create a `.env` file with your API keys: ```bash # .env file OPENAI_API_KEY=your_openai_key_here ANTHROPIC_API_KEY=your_anthropic_key_here GEMINI_API_KEY=your_gemini_key_here ``` ### Initialize Agents ```bash # Setup initial agent implementations bash ./setup_initial.sh ``` ## Core Concepts ### Architecture 1. **Task Agent**: Solves domain-specific tasks (code generation, math, etc.) 2. **Meta Agent**: Observes task agent performance and generates improvements 3. **Generation Loop**: Iteratively evolves agents through self-improvement cycles ### File Structure ``` HyperAgents/ ├── agent/ # Foundation model interfaces ├── domains/ # Task-specific implementations ├── utils/ # Common utilities ├── meta_agent.py # Meta-agent implementation ├── task_agent.py # Task-agent implementation ├── generate_loop.py # Main entry point └── run_meta_agent.py # Meta-agent execution script ``` ## Usage ### Running the Self-Improvement Loop ```bash # Basic usage with default settings python generate_loop.py --domains code_generation # Multiple domains python generate_loop.py --domains math reasoning # Custom configuration python generate_loop.py \ --domains code_generation \ --max_iterations 10 \ --output_dir ./my_outputs \ --model_name gpt-4 ``` ### Key Command-Line Arguments ```python # Common arguments for generate_loop.py --domains # Domain(s) to optimize (code_generation, math, reasoning, etc.) --max_iterations # Maximum improvement iterations --output_dir # Directory for outputs (default: outputs/) --model_name # Foundation model to use --baseline # Baseline agent to compare against --temperature # Sampling temperature for generation --num_samples # Number of samples per iteration ``` ## Working with Task Agents ### Creating a Custom Task Agent ```python # task_agent.py - Basic structure from typing import Any, Dict, List from agent.base_agent import BaseAgent class MyTaskAgent(BaseAgent): """Custom task agent for specific domain.""" def __init__(self, config: Dict[str, Any]): super().__init__(config) self.domain = config.get('domain', 'custom') def solve_task(self, task_input: str) -> str: """ Main method to solve a task. Args: task_input: Input task specification Returns: Solution to the task """ # Generate prompt for the model prompt = self._create_prompt(task_input) # Get model response response = self.model.generate( prompt=prompt, temperature=self.config.get('temperature', 0.7), max_tokens=self.config.get('max_tokens', 2048) ) # Post-process response solution = self._parse_solution(response) return solution def _create_prompt(self, task_input: str) -> str: """Create prompt for the model.""" return f"""Solve the following task: Task: {task_input} Solution:""" def _parse_solution(self, response: str) -> str: """Extract solution from model response.""" # Custom parsing logic return response.strip() def evaluate(self, task_input: str, solution: str) -> float: """ Evaluate solution quality. Returns: Score between 0 and 1 """ # Domain-specific evaluation return self._compute_score(task_input, solution) ``` ### Using the Task Agent ```python from task_agent import MyTaskAgent # Initialize agent config = { 'domain': 'custom', 'model_name': 'gpt-4', 'temperature': 0.7, 'max_tokens': 2048 } agent = MyTaskAgent(config) # Solve a task task = "Write a function to compute Fibonacci numbers" solution = agent.solve_task(task) score = agent.evaluate(task, solution) print(f"Solution: {solution}") print(f"Score: {score}") ``` ## Working with Meta Agents ### Meta Agent Structure ```python # meta_agent.py - Core implementation from typing import Dict, List, Any import difflib class MetaAgent: """Meta-agent that improves task agents.""" def __init__(self, config: Dict[str, Any]): self.config = config self.model = self._initialize_model() self.history = [] def generate_improvement( self, current_code: str, performance_data: List[Dict[str, Any]] ) -> str: """ Generate improved version of task agent. Args: current_code: Current task agent implementation performance_data: Performance metrics from recent runs Returns: Improved code implementation """ # Analyze performance insights = self._analyze_performance(performance_data) # Generate improvement prompt prompt = self._create_meta_prompt(current_code, insights) # Generate new code improved_code = self.model.generate( prompt=prompt, temperature=self.config.get('meta_temperature', 0.8), max_tokens=self.config.get('meta_max_tokens', 4096) ) # Validate and extract code validated_code = self._validate_code(improved_code) # Store in history self.history.append({ 'original': current_code, 'improved': validated_code, 'insights': insights }) return validated_code def _analyze_performance( self, performance_data: List[Dict[str, Any]] ) -> Dict[str, Any]: """Analyze performance metrics to identify improvement areas.""" # Compute statistics scores = [d['score'] for d in performance_data] avg_score = sum(scores) / len(scores) # Identify failure patterns failures = [d for d in performance_data if d['score'] < 0.5] return { 'average_score': avg_score, 'num_failures': len(failures), 'failure_patterns': self._extract_patterns(failures) } def _create_meta_prompt( self, current_code: str, insights: Dict[str, Any] ) -> str: """Create prompt for meta-level improvement.""" return f"""You are a meta-agent tasked with improving an AI task agent. Current Implementation: ```python {current_code} ``` Performance Analysis: - Average Score: {insights['average_score']:.2f} - Failures: {insights['num_failures']} - Common Issues: {insights.get('failure_patterns', 'None identified')} Generate an improved version that addresses these issues. Output only the complete improved code. Improved Implementation: ```python""" def _validate_code(self, code: str) -> str: """Validate and extract code from response.""" # Extract code block if '```python' in code: code = code.split('```python')[1].split('```')[0] # Basic syntax validation try: compile(code, '<string>', 'exec') except SyntaxError as e: raise ValueError(f"Generated code has syntax error: {e}") return code.strip() def compute_diff(self, old_code: str, new_code: str) -> List[str]: """Compute diff between code versions.""" diff = difflib.unified_diff( old_code.splitlines(keepends=True), new_code.splitlines(keepends=True), fromfile='old_agent.py', tofile='new_agent.py' ) return list(diff) ``` ### Running Meta Agent ```python # run_meta_agent.py - Example usage from meta_agent import MetaAgent from task_agent import MyTaskAgent import json def run_meta_improvement_cycle( initial_agent_code: str, test_tasks: List[str], num_iterations: int = 5 ): """Run multiple iterations of meta-improvement.""" # Initialize meta-agent meta_config = { 'model_name': 'gpt-4', 'meta_temperature': 0.8, 'meta_max_tokens': 4096 } meta_agent = MetaAgent(meta_config) current_code = initial_agent_code for iteration in range(num_iterations): print(f"\n=== Iteration {iteration + 1} ===") # Evaluate current agent performance_data = evaluate_agent(current_code, test_tasks) avg_score = sum(d['score'] for d in performance_data) / len(performance_data) print(f"Current Performance: {avg_score:.3f}") # Generate improvement improved_code = meta_agent.generate_improvement( current_code, performance_data ) # Show diff diff = meta_agent.compute_diff(current_code, improved_code) print("Changes:") print(''.join(diff[:20])) # Show first 20 lines # Update current code current_code = improved_code # Save checkpoint with open(f'agent_iteration_{iteration}.py', 'w') as f: f.write(current_code) return current_code def evaluate_agent(agent_code: str, test_tasks: List[str]) -> List[Dict[str, Any]]: """Evaluate agent on test tasks.""" # Create agent from code namespace = {} exec(agent_code, namespace) AgentClass = namespace['MyTaskAgent'] agent = AgentClass({'model_name': 'gpt-4'}) results = [] for task in test_tasks: solution = agent.solve_task(task) score = agent.evaluate(task, solution) results.append({ 'task': task, 'solution': solution, 'score': score }) return results # Usage if __name__ == '__main__': # Read initial agent code with open('initial_agent.py', 'r') as f: initial_code = f.read() # Define test tasks test_tasks = [ "Implement binary search", "Write a function to reverse a linked list", "Create a trie data structure" ] # Run improvement loop final_code = run_meta_improvement_cycle( initial_code, test_tasks, num_iterations=5 ) print("\nFinal agent saved!") ``` ## Domain-Specific Implementation ### Code Generation Domain ```python # domains/code_generation/agent.py from typing import Dict, Any, List import ast import subprocess class CodeGenerationAgent: """Agent specialized for code generation tasks.""" def generate_code(self, specification: str) -> str: """Generate code from specification.""" prompt = f"""Generate Python code for the following specification: {specification} Requirements: - Include proper error handling - Add docstrings - Follow PEP 8 style guide Code: ```python""" code = self.model.generate(prompt) return self._extract_code(code) def test_code(self, code: str, test_cases: List[Dict[str, Any]]) -> float: """Test generated code against test cases.""" try: # Create temporary module namespace = {} exec(code, namespace) passed = 0 for test in test_cases: func_name = test['function'] inputs = test['inputs'] expected = test['expected'] func = namespace[func_name] result = func(*inputs) if result == expected: passed += 1 return passed / len(test_cases) except Exception as e: print(f"Test error: {e}") return 0.0 def _extract_code(self, response: str) -> str: