photo-agents-autonomous-llm

# Photo Agents Autonomous LLM Skill > Skill by [ara.so](https://ara.so) — AI Agent Skills collection. ## Overview Photo Agents is a Python framework for building autonomous, self-evolving AI agents that ground their understanding in visual observations of the screen. Unlike traditional text-only agents, Photo Agents implements a perceive → reason → act cycle with a layered memory architecture inspired by biological cognition: vision input, bounded observations stored in layers (L1-L4), and skills the agent writes from real successes. **Key capabilities:** - Multi-provider LLM routing (Anthropic Claude, OpenAI GPT, failover sessions) - Layered memory system (working/global/SOP/session archive) - Physical execution tools (file I/O, sandboxed code, browser automation via Chrome DevTools Protocol) - Multiple client interfaces (CLI, Streamlit web app, PyQt desktop, chat platform bots) - Self-evolving through reflection and skill generation ## Installation ### Basic Installation ```bash pip install photoagents ``` ### Full Installation with All Clients ```bash pip install "photoagents[all]" ``` **Requirements:** Python 3.10+ ## API Key Setup Photo Agents requires a license key validated against `https://photo-agents.com/v1/keys/validate`. 1. Get your key at: https://photo-agents.com/dashboard/keys 2. Configure it (choose one method): **Environment variable:** ```bash export PHOTOAGENTS_API_KEY=pk_live_your_key_here ``` **Config file** (`~/.photoagents/config.json`): ```json { "api_key": "pk_live_your_key_here" } ``` **Interactive prompt:** Run any command and it will prompt you to enter and save the key. ## LLM Provider Configuration Create a `credentials.py` file in your project root: ```python # credentials.py from photoagents.config.keys_template import LLMConfig, ProviderConfig # Option 1: Anthropic Claude llm_config = LLMConfig( primary=ProviderConfig( provider="anthropic", api_key="${ANTHROPIC_API_KEY}", # Use env var model="claude-3-5-sonnet-20241022" ) ) # Option 2: OpenAI GPT llm_config = LLMConfig( primary=ProviderConfig( provider="openai", api_key="${OPENAI_API_KEY}", model="gpt-4o" ) ) # Option 3: Failover configuration llm_config = LLMConfig( primary=ProviderConfig( provider="anthropic", api_key="${ANTHROPIC_API_KEY}", model="claude-3-5-sonnet-20241022" ), fallback=ProviderConfig( provider="openai", api_key="${OPENAI_API_KEY}", model="gpt-4o" ) ) ``` Or use JSON format (`credentials.json`): ```json { "primary": { "provider": "anthropic", "api_key": "${ANTHROPIC_API_KEY}", "model": "claude-3-5-sonnet-20241022" }, "fallback": { "provider": "openai", "api_key": "${OPENAI_API_KEY}", "model": "gpt-4o" } } ``` ## Core Usage Patterns ### 1. Interactive CLI Mode ```bash # Start interactive REPL python -m photoagents # The agent will prompt for tasks and execute them # with vision-grounded reasoning ``` ### 2. One-Shot Task Execution ```bash # Execute a single task python -m photoagents --task my_analysis --input "Analyze the largest files in this directory" # With custom output path python -m photoagents --task report --input "Generate system report" --output ./reports/ ``` ### 3. Reflection/Watchdog Mode ```bash # Run with reflection scheduler (self-evolving) python -m photoagents --reflect photoagents/evolution/scheduler.py ``` ### 4. Programmatic Agent Session ```python from photoagents.core.loop import run_agent_session from photoagents.llm.router import LLMSession from photoagents.config.keys_template import LLMConfig, ProviderConfig # Configure LLM llm_config = LLMConfig( primary=ProviderConfig( provider="anthropic", api_key="${ANTHROPIC_API_KEY}", model="claude-3-5-sonnet-20241022" ) ) # Create session session = LLMSession(llm_config) # Run agent loop result = run_agent_session( task_name="file_analysis", user_input="Find and summarize all Python files in the current directory", session=session, max_turns=10 ) print(f"Final output: {result}") ``` ### 5. Custom Tool Integration ```python from photoagents.core.tool_dispatcher import register_tool from typing import Dict, Any @register_tool def custom_analysis_tool(data: str, options: Dict[str, Any]) -> str: """ Custom tool for specialized analysis. Args: data: Input data to analyze options: Configuration options Returns: Analysis results """ # Your custom logic here result = f"Analyzed: {data} with options {options}" return result # Tool is now available to the agent ``` ## GUI Client Options ### Streamlit Web App + WebView ```bash # Launch web interface with native window pythonw -m photoagents.cli.launcher ``` ### Service Hub (Start/Stop Services) ```bash # Launch control hub pythonw -m photoagents.cli.hub ``` ### Desktop PyQt Application ```bash python -m photoagents.clients.desktop_app ``` ### Desktop Companion ```bash pythonw -m photoagents.clients.companion_v2 ``` ### Chat Platform Bots ```bash # Telegram python -m photoagents.clients.telegram_client # Feishu (Lark) python -m photoagents.clients.feishu_client # WeCom python -m photoagents.clients.wecom_client # DingTalk python -m photoagents.clients.dingtalk_client # QQ python -m photoagents.clients.qq_client ``` ## Layered Memory System Photo Agents uses a 4-layer memory architecture: ### L1: Working Memory Short-term context for the current task (conversation turns, immediate observations). ### L2: Global Memory Long-term facts stored in `~/.photoagents/global_mem.txt`. ```python from photoagents.core.memory import add_global_fact, search_global_memory # Add a fact add_global_fact("Project uses Python 3.11 and requires PostgreSQL 14+") # Search memory results = search_global_memory("database requirements") ``` ### L3: Skills & SOPs Standard Operating Procedures the agent writes from successful executions. ```python from photoagents.skills.skill_manager import save_skill, load_skill # Save a new skill save_skill( name="web_scraping_pattern", code=""" def scrape_structured_data(url: str) -> dict: # Implementation pass """, description="Reliable pattern for scraping structured web data" ) # Load and use skill = load_skill("web_scraping_pattern") ``` ### L4: Session Archive Full raw session logs in `~/.photoagents/sessions/`. ## Browser Automation with CDP Photo Agents includes Chrome DevTools Protocol integration for browser control: ```python from photoagents.web.cdp_bridge import CDPBridge async def automate_browser(): async with CDPBridge() as browser: # Navigate await browser.navigate("https://example.com") # Take screenshot screenshot = await browser.screenshot() # Execute JavaScript result = await browser.evaluate("document.title") # Click element await browser.click("button.submit") # Fill form await browser.type("input[name='query']", "search term") return result ``` ## Vision-Grounded Operations ### Screenshot Analysis ```python from photoagents.skills.vision import analyze_screenshot # Agent automatically captures and analyzes screen analysis = analyze_screenshot( region=(0, 0, 1920, 1080), # x, y, width, height question="What UI elements are visible?" ) ``` ### OCR Text Extraction ```python from photoagents.skills.ocr import extract_text_from_region # Extract text from screen region text = extract_text_from_region( x=100, y=200, width=500, height=300 ) ``` ## Sandboxed Code Execution ```python from photoagents.core.sandbox import execute_code # Python execution result = execute_code( code=""" import json data = {"status": "success"} print(json.dumps(data)) """, language="python", timeout=30 ) # PowerShell (Windows) ps_result = execute_code( code="Get-Process | Select-Object -First 5", language="powershell" ) # Bash (Linux/Mac) bash_result = execute_code( code="ls -la | head -n 10", language="bash" ) ``` ## File I/O Operations ```python from photoagents.core.file_ops import read_file, write_file, list_directory # Read file content = read_file("~/project/config.json") # Write file write_file("~/output/report.txt", "Analysis complete\n") # List directory with filters files = list_directory( path="~/project", pattern="*.py", recursive=True ) ``` ## Observability with Langfuse ```python from photoagents.integrations.langfuse_tracer import init_langfuse, trace_agent_step # Initialize tracer = init_langfuse( public_key="${LANGFUSE_PUBLIC_KEY}", secret_key="${LANGFUSE_SECRET_KEY}", host="https://cloud.langfuse.com" ) # Trace agent steps with trace_agent_step("file_analysis", metadata={"task": "analyze_logs"}): # Agent operations here pass ``` ## Configuration Files ### On-Disk State Locations | Path | Purpose | |------|---------| | `~/.photoagents/config.json` | API key + license validation cache | | `~/.photoagents/global_mem.txt` | L2 long-term facts | | `~/.photoagents/sessions/` | L4 raw session archives | | `~/.photoagents/skill_index/` | Vector index for skill/SOP search | | `~/.photoagents/temp/` | Per-task scratch (logs, intermediate output) | ### Custom System Prompt Override the default system prompt: ```python from photoagents.core.loop import run_agent_session custom_prompt = """ You are a specialized data analysis agent. Focus on: statistical analysis, visualization, and reporting. Always verify data integrity before processing. """ result = run_agent_session( task_name="analysis", user_input="Analyze sales data", system_prompt_override=custom_prompt ) ``` ## Common Patterns ### Pattern 1: Autonomous Research Agent ```python from photoagents.core.loop import run_agent_session from photoagents.llm.router import LLMSession def create_research_agent(topic: str): session = LLMSession.from_env() result = run_agent_session( task_name=f"research_{topic}", user_input=f""" Research {topic} and create a comprehensive report: 1. Search for recent information 2. Analyze credibility of sources 3. Synthesize findings 4. Save report with citations """, session=session, max_turns=50 ) return result # Use it report = create_research_agent("quantum computing advances 2026") ``` ### Pattern 2: Self-Evolving Monitor ```python # monitor.py from photoagents.evolution.scheduler import schedule_check def check() -> bool: """ Watchdog function that triggers agent tasks. Return True to execute a task. """ import os import time # Check if it's time to run daily backup last_run = os.path.getmtime("~/.photoagents/last_backup") if time.time() - last_run > 86400: # 24 hours return True return False def get_task() -> str: """Return the task to execute when check() returns True.""" return "Backup all project files to ~/backups/ and verify integrity" # Run with: # python -m photoagents --reflect monitor.py ``` ### Pattern 3: Multi-Step Workflow ```python from photoagents.core.loop import run_agent_session from photoagents.core.memory import add_global_fact def execute_workflow(project_path: str): # Step 1: Analyze codebase analysis = run_agent_session( task_name="code_analysis", user_input=f"Analyze Python code structure in {project_path}" ) # Save insight to global memory add_global_fact(f"Project at {project_path}: {analysis}") # Step 2: Generate documentation docs = run_agent_session( task_name="generate_docs",