litellm-agent-platform

name	litellm-agent-platform
description	Self-hosted platform for running coding agents (Claude Code, Codex, Hermes) in isolated sandboxes with vault proxy and session management
triggers	["how do I deploy a coding agent with LiteLLM Agent Platform","set up an agent harness with Claude Code or Codex","create a scheduled agent with CRON in LiteLLM","manage agent sessions and memory across runs","configure MCP tools for my agent platform","run agents through the LiteLLM API","connect GitHub and AWS tools to my agent","swap between different agent harnesses"]

LiteLLM Agent Platform

Skill by ara.so — AI Agent Skills collection.

LiteLLM Agent Platform is a self-hosted service for running coding agents (Claude Code, Codex, Hermes, OpenCode) with persistent sessions, CRON scheduling, memory management, and unified tool access through MCP servers. It provides both a UI and REST API for creating, deploying, and managing agents with isolated sandbox execution.

Installation

Prerequisites

• Node.js 18+ / TypeScript environment
• Docker (for sandbox isolation)
• LiteLLM API key

Self-Hosted Setup

# Clone the repository
git clone https://github.com/LiteLLM-Labs/litellm-agent-platform.git
cd litellm-agent-platform

# Install dependencies
npm install

# Configure environment
cp .env.example .env

Environment Configuration

# .env
LITELLM_MASTER_KEY=your_master_key_here
DATABASE_URL=postgresql://user:password@localhost:5432/agent_platform
REDIS_URL=redis://localhost:6379
PORT=4000

# Model provider keys (stored in vault)
ANTHROPIC_API_KEY=$ANTHROPIC_API_KEY
OPENAI_API_KEY=$OPENAI_API_KEY

Start the Platform

# Development
npm run dev

# Production
npm run build
npm start

The platform will be available at http://localhost:4000.

Core Concepts

Harnesses

Harnesses are the underlying agent frameworks:

• Claude Code: Anthropic's coding agent
• Codex: OpenAI's code generation agent
• Hermes: Open-source coding assistant
• OpenCode: Community-driven agent

Tools

Tools are MCP (Model Context Protocol) servers that provide capabilities:

• GitHub integration
• AWS services
• File system access
• Database connections
• Custom MCP servers

Sessions

Persistent agent sessions maintain context across multiple runs, enabling long-running workflows and memory retention.

API Usage

Authentication

All API requests require a Bearer token:

export LITELLM_KEY="your_api_key"

Create an Agent

// TypeScript example
import axios from 'axios';

const createAgent = async () => {
  const response = await axios.post(
    'http://localhost:4000/agents',
    {
      name: 'ci-fixer',
      harness: 'claude-code',
      model: 'anthropic/claude-sonnet-4-5',
      system_prompt: 'You monitor CI and fix failing checks.',
      tools: ['github', 'aws'],
      memory_enabled: true,
      session_timeout: 3600
    },
    {
      headers: {
        Authorization: `Bearer ${process.env.LITELLM_KEY}`
      }
    }
  );
  
  return response.data;
};

# cURL example
curl http://localhost:4000/agents -X POST \
  -H "Authorization: Bearer $LITELLM_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "code-reviewer",
    "harness": "codex",
    "model": "openai/gpt-4",
    "system_prompt": "Review code for security and best practices.",
    "tools": ["github", "filesystem"]
  }'

Run an Agent

// Single run
const runAgent = async (agentName: string, input: string) => {
  const response = await axios.post(
    `http://localhost:4000/agents/${agentName}/runs`,
    { input },
    {
      headers: {
        Authorization: `Bearer ${process.env.LITELLM_KEY}`
      }
    }
  );
  
  return response.data;
};

// Usage
const result = await runAgent('ci-fixer', 'Fix the failing CI check on PR #418');
console.log(result.output);

# cURL example
curl http://localhost:4000/agents/ci-fixer/runs -X POST \
  -H "Authorization: Bearer $LITELLM_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "input": "Fix the failing CI check on PR #418" }'

List Agents

const listAgents = async () => {
  const response = await axios.get('http://localhost:4000/agents', {
    headers: {
      Authorization: `Bearer ${process.env.LITELLM_KEY}`
    }
  });
  
  return response.data.agents;
};

Get Agent Status

const getAgentStatus = async (agentName: string) => {
  const response = await axios.get(
    `http://localhost:4000/agents/${agentName}`,
    {
      headers: {
        Authorization: `Bearer ${process.env.LITELLM_KEY}`
      }
    }
  );
  
  return response.data;
};

Update Agent Configuration

const updateAgent = async (agentName: string, updates: any) => {
  const response = await axios.patch(
    `http://localhost:4000/agents/${agentName}`,
    updates,
    {
      headers: {
        Authorization: `Bearer ${process.env.LITELLM_KEY}`
      }
    }
  );
  
  return response.data;
};

// Example: Switch harness
await updateAgent('ci-fixer', {
  harness: 'hermes',
  model: 'nous/hermes-2-pro'
});

Delete an Agent

const deleteAgent = async (agentName: string) => {
  await axios.delete(`http://localhost:4000/agents/${agentName}`, {
    headers: {
      Authorization: `Bearer ${process.env.LITELLM_KEY}`
    }
  });
};

Session Management

Create a Persistent Session

const createSession = async (agentName: string) => {
  const response = await axios.post(
    `http://localhost:4000/agents/${agentName}/sessions`,
    {
      session_id: 'debug-session-001',
      metadata: {
        project: 'api-service',
        branch: 'main'
      }
    },
    {
      headers: {
        Authorization: `Bearer ${process.env.LITELLM_KEY}`
      }
    }
  );
  
  return response.data.session_id;
};

Run with Session Context

const runWithSession = async (
  agentName: string,
  sessionId: string,
  input: string
) => {
  const response = await axios.post(
    `http://localhost:4000/agents/${agentName}/runs`,
    {
      input,
      session_id: sessionId
    },
    {
      headers: {
        Authorization: `Bearer ${process.env.LITELLM_KEY}`
      }
    }
  );
  
  return response.data;
};

// Multi-turn conversation with memory
const sessionId = await createSession('code-reviewer');
await runWithSession(sessionId, 'code-reviewer', 'Review src/auth.ts');
await runWithSession(sessionId, 'code-reviewer', 'Now check for SQL injection risks');

CRON Scheduling

Schedule an Agent

const scheduleAgent = async (agentName: string, schedule: string, task: string) => {
  const response = await axios.post(
    `http://localhost:4000/agents/${agentName}/schedules`,
    {
      cron: schedule,
      input: task,
      enabled: true
    },
    {
      headers: {
        Authorization: `Bearer ${process.env.LITELLM_KEY}`
      }
    }
  );
  
  return response.data;
};

// Run every day at 9 AM
await scheduleAgent(
  'ci-fixer',
  '0 9 * * *',
  'Check all open PRs for failing CI and fix them'
);

// Run every hour
await scheduleAgent(
  'security-scanner',
  '0 * * * *',
  'Scan dependencies for vulnerabilities'
);

List Schedules

const listSchedules = async (agentName: string) => {
  const response = await axios.get(
    `http://localhost:4000/agents/${agentName}/schedules`,
    {
      headers: {
        Authorization: `Bearer ${process.env.LITELLM_KEY}`
      }
    }
  );
  
  return response.data.schedules;
};

Tool Configuration

Register MCP Tools

const registerTool = async (toolConfig: any) => {
  const response = await axios.post(
    'http://localhost:4000/tools',
    toolConfig,
    {
      headers: {
        Authorization: `Bearer ${process.env.LITELLM_KEY}`
      }
    }
  );
  
  return response.data;
};

// Register GitHub MCP
await registerTool({
  name: 'github',
  type: 'mcp',
  config: {
    server_url: 'https://github-mcp.example.com',
    credentials: {
      token: process.env.GITHUB_TOKEN
    }
  }
});

// Register custom MCP
await registerTool({
  name: 'custom-api',
  type: 'mcp',
  config: {
    server_url: 'http://localhost:8080/mcp',
    credentials: {
      api_key: process.env.CUSTOM_API_KEY
    }
  }
});

List Available Tools

const listTools = async () => {
  const response = await axios.get('http://localhost:4000/tools', {
    headers: {
      Authorization: `Bearer ${process.env.LITELLM_KEY}`
    }
  });
  
  return response.data.tools;
};

Common Patterns

CI/CD Integration Agent

const setupCIAgent = async () => {
  // Create the agent
  const agent = await axios.post(
    'http://localhost:4000/agents',
    {
      name: 'ci-cd-helper',
      harness: 'claude-code',
      model: 'anthropic/claude-sonnet-4-5',
      system_prompt: `You are a CI/CD expert. Monitor build failures, 
        analyze logs, and suggest fixes. You have access to GitHub and AWS.`,
      tools: ['github', 'aws'],
      memory_enabled: true
    },
    {
      headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` }
    }
  );
  
  // Schedule daily checks
  await axios.post(
    `http://localhost:4000/agents/ci-cd-helper/schedules`,
    {
      cron: '0 */6 * * *', // Every 6 hours
      input: 'Check recent CI failures and create tickets for recurring issues',
      enabled: true
    },
    {
      headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` }
    }
  );
  
  return agent.data;
};

Code Review Automation

const setupCodeReviewer = async () => {
  const agent = await axios.post(
    'http://localhost:4000/agents',
    {
      name: 'security-reviewer',
      harness: 'codex',
      model: 'openai/gpt-4',
      system_prompt: `Review code for security vulnerabilities, 
        best practices, and performance issues. Be thorough but constructive.`,
      tools: ['github', 'filesystem'],
      memory_enabled: true
    },
    {
      headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` }
    }
  );
  
  // Trigger on new PRs via webhook or schedule
  return agent.data;
};

Multi-Step Workflow with Sessions

const runMultiStepWorkflow = async () => {
  const agentName = 'code-refactor';
  const sessionId = `refactor-${Date.now()}`;
  
  // Step 1: Analyze codebase
  const analysis = await axios.post(
    `http://localhost:4000/agents/${agentName}/runs`,
    {
      input: 'Analyze src/ for code smells and technical debt',
      session_id: sessionId
    },
    {
      headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` }
    }
  );
  
  // Step 2: Create refactoring plan
  const plan = await axios.post(
    `http://localhost:4000/agents/${agentName}/runs`,
    {
      input: 'Create a detailed refactoring plan based on the analysis',
      session_id: sessionId
    },
    {
      headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` }
    }
  );
  
  // Step 3: Execute refactoring
  const execution = await axios.post(
    `http://localhost:4000/agents/${agentName}/runs`,
    {
      input: 'Execute the first three refactoring tasks from the plan',
      session_id: sessionId
    },
    {
      headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` }
    }
  );
  
  return { analysis, plan, execution };
};

Monitoring and Logs

Get Run History

const getRunHistory = async (agentName: string, limit = 10) => {
  const response = await axios.get(
    `http://localhost:4000/agents/${agentName}/runs?limit=${limit}`,
    {
      headers: {
        Authorization: `Bearer ${process.env.LITELLM_KEY}`
      }
    }
  );
  
  return response.data.runs;
};

Get Run Details

const getRunDetails = async (agentName: string, runId: string) => {
  const response = await axios.get(
    `http://localhost:4000/agents/${agentName}/runs/${runId}`,
    {
      headers: {
        Authorization: `Bearer ${process.env.LITELLM_KEY}`
      }
    }
  );
  
  return response.data;
};

Troubleshooting

Agent Not Starting

Issue: Agent creation returns success but runs fail

Solution: Check model availability and credentials:

const debugAgent = async (agentName: string) => {
  const status = await axios.get(
    `http://localhost:4000/agents/${agentName}/debug`,
    {
      headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` }
    }
  );
  
  console.log('Model available:', status.data.model_available);
  console.log('Tools connected:', status.data.tools_status);
  console.log('Last error:', status.data.last_error);
};

Session Memory Issues

Issue: Agent not remembering context across runs

Solution: Ensure session_id is consistent and memory is enabled:

// Verify session configuration
const checkSession = async (agentName: string) => {
  const agent = await axios.get(
    `http://localhost:4000/agents/${agentName}`,
    {
      headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` }
    }
  );
  
  if (!agent.data.memory_enabled) {
    await axios.patch(
      `http://localhost:4000/agents/${agentName}`,
      { memory_enabled: true },
      {
        headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` }
      }
    );
  }
};

Tool Connection Failures

Issue: Agent cannot access tools (GitHub, AWS, etc.)

Solution: Verify tool registration and credentials:

# Check tool status
curl http://localhost:4000/tools \
  -H "Authorization: Bearer $LITELLM_KEY"

# Test tool connection
curl http://localhost:4000/tools/github/test \
  -H "Authorization: Bearer $LITELLM_KEY"

Rate Limiting

Issue: Hitting model provider rate limits

Solution: Configure retry logic and backoff:

const runWithRetry = async (agentName: string, input: string, maxRetries = 3) => {
  let lastError;
  
  for (let i = 0; i < maxRetries; i++) {
    try {
      const response = await axios.post(
        `http://localhost:4000/agents/${agentName}/runs`,
        { input, retry_on_rate_limit: true },
        {
          headers: { Authorization: `Bearer ${process.env.LITELLM_KEY}` }
        }
      );
      return response.data;
    } catch (error: any) {
      lastError = error;
      if (error.response?.status === 429) {
        await new Promise(resolve => setTimeout(resolve, 2000 * (i + 1)));
      } else {
        throw error;
      }
    }
  }
  
  throw lastError;
};

Best Practices

1. Use sessions for multi-turn interactions: Enable memory and use consistent session IDs
2. Set appropriate timeouts: Configure session_timeout based on workflow complexity
3. Monitor run history: Track agent performance and failures
4. Secure credentials: Store API keys in environment variables, use the vault proxy
5. Start with Claude Code or Codex: Most reliable harnesses for production use
6. Test tools separately: Verify MCP connections before deploying agents
7. Use descriptive system prompts: Clear instructions improve agent behavior
8. Schedule wisely: Avoid overlapping CRON jobs for the same agent