litellm-agent-platform

# LiteLLM Agent Platform > Skill by [ara.so](https://ara.so) — AI Agent Skills collection. LiteLLM Agent Platform is a self-hosted service for running coding agents (Claude Code, Codex, Hermes, OpenCode) with persistent sessions, CRON scheduling, memory management, and unified tool access through MCP servers. It provides both a UI and REST API for creating, deploying, and managing agents with isolated sandbox execution. ## Installation ### Prerequisites - Node.js 18+ / TypeScript environment - Docker (for sandbox isolation) - LiteLLM API key ### Self-Hosted Setup ```bash # Clone the repository git clone https://github.com/LiteLLM-Labs/litellm-agent-platform.git cd litellm-agent-platform # Install dependencies npm install # Configure environment cp .env.example .env ``` ### Environment Configuration ```bash # .env LITELLM_MASTER_KEY=your_master_key_here DATABASE_URL=postgresql://user:password@localhost:5432/agent_platform REDIS_URL=redis://localhost:6379 PORT=4000 # Model provider keys (stored in vault) ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY OPENAI_API_KEY=$OPENAI_API_KEY ``` ### Start the Platform ```bash # Development npm run dev # Production npm run build npm start ``` The platform will be available at `http://localhost:4000`. ## Core Concepts ### Harnesses Harnesses are the underlying agent frameworks: - **Claude Code**: Anthropic's coding agent - **Codex**: OpenAI's code generation agent - **Hermes**: Open-source coding assistant - **OpenCode**: Community-driven agent ### Tools Tools are MCP (Model Context Protocol) servers that provide capabilities: - GitHub integration - AWS services - File system access - Database connections - Custom MCP servers ### Sessions Persistent agent sessions maintain context across multiple runs, enabling long-running workflows and memory retention. ## API Usage ### Authentication All API requests require a Bearer token: ```bash export LITELLM_KEY="your_api_key" ``` ### Create an Agent ```typescript // TypeScript example import axios from 'axios'; const createAgent = async () => { const response = await axios.post( 'http://localhost:4000/agents', { name: 'ci-fixer', harness: 'claude-code', model: 'anthropic/claude-sonnet-4-5', system_prompt: 'You monitor CI and fix failing checks.', tools: ['github', 'aws'], memory_enabled: true, session_timeout: 3600 }, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); return response.data; }; ``` ```bash # cURL example curl http://localhost:4000/agents -X POST \ -H "Authorization: Bearer $LITELLM_KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "code-reviewer", "harness": "codex", "model": "openai/gpt-4", "system_prompt": "Review code for security and best practices.", "tools": ["github", "filesystem"] }' ``` ### Run an Agent ```typescript // Single run const runAgent = async (agentName: string, input: string) => { const response = await axios.post( `http://localhost:4000/agents/${agentName}/runs`, { input }, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); return response.data; }; // Usage const result = await runAgent('ci-fixer', 'Fix the failing CI check on PR #418'); console.log(result.output); ``` ```bash # cURL example curl http://localhost:4000/agents/ci-fixer/runs -X POST \ -H "Authorization: Bearer $LITELLM_KEY" \ -H "Content-Type: application/json" \ -d '{ "input": "Fix the failing CI check on PR #418" }' ``` ### List Agents ```typescript const listAgents = async () => { const response = await axios.get('http://localhost:4000/agents', { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } }); return response.data.agents; }; ``` ### Get Agent Status ```typescript const getAgentStatus = async (agentName: string) => { const response = await axios.get( `http://localhost:4000/agents/${agentName}`, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); return response.data; }; ``` ### Update Agent Configuration ```typescript const updateAgent = async (agentName: string, updates: any) => { const response = await axios.patch( `http://localhost:4000/agents/${agentName}`, updates, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); return response.data; }; // Example: Switch harness await updateAgent('ci-fixer', { harness: 'hermes', model: 'nous/hermes-2-pro' }); ``` ### Delete an Agent ```typescript const deleteAgent = async (agentName: string) => { await axios.delete(`http://localhost:4000/agents/${agentName}`, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } }); }; ``` ## Session Management ### Create a Persistent Session ```typescript const createSession = async (agentName: string) => { const response = await axios.post( `http://localhost:4000/agents/${agentName}/sessions`, { session_id: 'debug-session-001', metadata: { project: 'api-service', branch: 'main' } }, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); return response.data.session_id; }; ``` ### Run with Session Context ```typescript const runWithSession = async ( agentName: string, sessionId: string, input: string ) => { const response = await axios.post( `http://localhost:4000/agents/${agentName}/runs`, { input, session_id: sessionId }, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); return response.data; }; // Multi-turn conversation with memory const sessionId = await createSession('code-reviewer'); await runWithSession(sessionId, 'code-reviewer', 'Review src/auth.ts'); await runWithSession(sessionId, 'code-reviewer', 'Now check for SQL injection risks'); ``` ## CRON Scheduling ### Schedule an Agent ```typescript const scheduleAgent = async (agentName: string, schedule: string, task: string) => { const response = await axios.post( `http://localhost:4000/agents/${agentName}/schedules`, { cron: schedule, input: task, enabled: true }, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); return response.data; }; // Run every day at 9 AM await scheduleAgent( 'ci-fixer', '0 9 * * *', 'Check all open PRs for failing CI and fix them' ); // Run every hour await scheduleAgent( 'security-scanner', '0 * * * *', 'Scan dependencies for vulnerabilities' ); ``` ### List Schedules ```typescript const listSchedules = async (agentName: string) => { const response = await axios.get( `http://localhost:4000/agents/${agentName}/schedules`, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); return response.data.schedules; }; ``` ## Tool Configuration ### Register MCP Tools ```typescript const registerTool = async (toolConfig: any) => { const response = await axios.post( 'http://localhost:4000/tools', toolConfig, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); return response.data; }; // Register GitHub MCP await registerTool({ name: 'github', type: 'mcp', config: { server_url: 'https://github-mcp.example.com', credentials: { token: process.env.GITHUB_TOKEN } } }); // Register custom MCP await registerTool({ name: 'custom-api', type: 'mcp', config: { server_url: 'http://localhost:8080/mcp', credentials: { api_key: process.env.CUSTOM_API_KEY } } }); ``` ### List Available Tools ```typescript const listTools = async () => { const response = await axios.get('http://localhost:4000/tools', { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } }); return response.data.tools; }; ``` ## Common Patterns ### CI/CD Integration Agent ```typescript const setupCIAgent = async () => { // Create the agent const agent = await axios.post( 'http://localhost:4000/agents', { name: 'ci-cd-helper', harness: 'claude-code', model: 'anthropic/claude-sonnet-4-5', system_prompt: `You are a CI/CD expert. Monitor build failures, analyze logs, and suggest fixes. You have access to GitHub and AWS.`, tools: ['github', 'aws'], memory_enabled: true }, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); // Schedule daily checks await axios.post( `http://localhost:4000/agents/ci-cd-helper/schedules`, { cron: '0 */6 * * *', // Every 6 hours input: 'Check recent CI failures and create tickets for recurring issues', enabled: true }, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); return agent.data; }; ``` ### Code Review Automation ```typescript const setupCodeReviewer = async () => { const agent = await axios.post( 'http://localhost:4000/agents', { name: 'security-reviewer', harness: 'codex', model: 'openai/gpt-4', system_prompt: `Review code for security vulnerabilities, best practices, and performance issues. Be thorough but constructive.`, tools: ['github', 'filesystem'], memory_enabled: true }, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); // Trigger on new PRs via webhook or schedule return agent.data; }; ``` ### Multi-Step Workflow with Sessions ```typescript const runMultiStepWorkflow = async () => { const agentName = 'code-refactor'; const sessionId = `refactor-${Date.now()}`; // Step 1: Analyze codebase const analysis = await axios.post( `http://localhost:4000/agents/${agentName}/runs`, { input: 'Analyze src/ for code smells and technical debt', session_id: sessionId }, { headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` } } ); // Step 2: Create refactoring plan const plan = await axios.post(