autonomous-agent-harness

name	autonomous-agent-harness
description	Transform Gemini CLI into a fully autonomous agent system with persistent memory, scheduled operations, computer use, and task queuing. Replaces standalone agent frameworks (Hermes, AutoGPT) by leveraging Gemini CLI's native crons, dispatch, MCP tools, and memory. Use when the user wants continuous autonomous operation, scheduled tasks, or a self-directing agent loop.
origin	ECC

Autonomous Agent Harness

Turn Gemini CLI into a persistent, self-directing agent system using only native features and MCP servers.

When to Use

• User wants an agent that runs continuously or on a schedule
• Setting up automated workflows that trigger periodically
• Building a personal AI assistant that remembers context across sessions
• User says "run this every day", "check on this regularly", "keep monitoring"
• Wants to replicate functionality from Hermes, AutoGPT, or similar autonomous agent frameworks
• Needs computer use combined with scheduled execution

Architecture

┌──────────────────────────────────────────────────────────────┐
│                    Gemini CLI Runtime                         │
│                                                              │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌─────────────┐ │
│  │  Crons   │  │ Dispatch │  │ Memory   │  │ Computer    │ │
│  │ Schedule │  │ Remote   │  │ Store    │  │ Use         │ │
│  │ Tasks    │  │ Agents   │  │          │  │             │ │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └──────┬──────┘ │
│       │              │             │                │        │
│       ▼              ▼             ▼                ▼        │
│  ┌──────────────────────────────────────────────────────┐    │
│  │              ECC Skill + Agent Layer                  │    │
│  │                                                      │    │
│  │  skills/     agents/     commands/     hooks/        │    │
│  └──────────────────────────────────────────────────────┘    │
│       │              │             │                │        │
│       ▼              ▼             ▼                ▼        │
│  ┌──────────────────────────────────────────────────────┐    │
│  │              MCP Server Layer                        │    │
│  │                                                      │    │
│  │  memory    github    exa    supabase    browser-use  │    │
│  └──────────────────────────────────────────────────────┘    │
└──────────────────────────────────────────────────────────────┘

Core Components

1. Persistent Memory

Use Gemini CLI's built-in memory system enhanced with MCP memory server for structured data.

Built-in memory (~/.gemini/egc-projects/*/memory/):

• User preferences, feedback, project context
• Stored as markdown files with frontmatter
• Automatically loaded at session start

MCP memory server (structured knowledge graph):

• Entities, relations, observations
• Queryable graph structure
• Cross-session persistence

Memory patterns:

# Short-term: current session context
Use TodoWrite for in-session task tracking

# Medium-term: project memory files
Write to ~/.gemini/egc-projects/*/memory/ for cross-session recall

# Long-term: MCP knowledge graph
Use mcp__memory__create_entities for permanent structured data
Use mcp__memory__create_relations for relationship mapping
Use mcp__memory__add_observations for new facts about known entities

2. Scheduled Operations (Crons)

Use Gemini CLI's scheduled tasks to create recurring agent operations.

Setting up a cron:

# Via MCP tool
mcp__scheduled-tasks__create_scheduled_task({
  name: "daily-pr-review",
  schedule: "0 9 * * 1-5",  # 9 AM weekdays
  prompt: "Review all open PRs in affaan-m/everything-gemini-code. For each: check CI status, review changes, flag issues. Post summary to memory.",
  project_dir: "/path/to/repo"
})

# Via gemini -p (programmatic mode)
echo "Review open PRs and summarize" | gemini -p --project /path/to/repo

Useful cron patterns:

Pattern	Schedule	Use Case
Daily standup	0 9 * * 1-5	Review PRs, issues, deploy status
Weekly review	0 10 * * 1	Code quality metrics, test coverage
Hourly monitor	0 * * * *	Production health, error rate checks
Nightly build	0 2 * * *	Run full test suite, security scan
Pre-meeting	*/30 * * * *	Prepare context for upcoming meetings

3. Dispatch / Remote Agents

Trigger Gemini CLI agents remotely for event-driven workflows.

Dispatch patterns:

# Trigger from CI/CD
curl -X POST "https://generativelanguage.googleapis.com/v1/dispatch" \
  -H "Authorization: Bearer $GEMINI_API_KEY" \
  -d '{"prompt": "Build failed on main. Diagnose and fix.", "project": "/repo"}'

# Trigger from webhook
# GitHub webhook → dispatch → Gemini agent → fix → PR

# Trigger from another agent
gemini -p "Analyze the output of the security scan and create issues for findings"

4. Computer Use

Leverage Gemini's computer-use MCP for physical world interaction.

Capabilities:

• Browser automation (navigate, click, fill forms, screenshot)
• Desktop control (open apps, type, mouse control)
• File system operations beyond CLI

Use cases within the harness:

• Automated testing of web UIs
• Form filling and data entry
• Screenshot-based monitoring
• Multi-app workflows

5. Task Queue

Manage a persistent queue of tasks that survive session boundaries.

Implementation:

# Task persistence via memory
Write task queue to ~/.gemini/egc-projects/*/memory/task-queue.md

# Task format
---
name: task-queue
type: project
description: Persistent task queue for autonomous operation
---

## Active Tasks
- [ ] PR #123: Review and approve if CI green
- [ ] Monitor deploy: check /health every 30 min for 2 hours
- [ ] Research: Find 5 leads in AI tooling space

## Completed
- [x] Daily standup: reviewed 3 PRs, 2 issues

Replacing Hermes

Hermes Component	ECC Equivalent	How
Gateway/Router	Gemini CLI dispatch + crons	Scheduled tasks trigger agent sessions
Memory System	Gemini memory + MCP memory server	Built-in persistence + knowledge graph
Tool Registry	MCP servers	Dynamically loaded tool providers
Orchestration	ECC skills + agents	Skill definitions direct agent behavior
Computer Use	computer-use MCP	Native browser and desktop control
Context Manager	Session management + memory	ECC 2.0 session lifecycle
Task Queue	Memory-persisted task list	TodoWrite + memory files

Setup Guide

Step 1: Configure MCP Servers

Ensure these are in ~/.gemini.json:

{
  "mcpServers": {
    "memory": {
      "command": "npx",
      "args": ["-y", "@anthropic/memory-mcp-server"]
    },
    "scheduled-tasks": {
      "command": "npx",
      "args": ["-y", "@anthropic/scheduled-tasks-mcp-server"]
    },
    "computer-use": {
      "command": "npx",
      "args": ["-y", "@anthropic/computer-use-mcp-server"]
    }
  }
}

Step 2: Create Base Crons

# Daily morning briefing
gemini -p "Create a scheduled task: every weekday at 9am, review my GitHub notifications, open PRs, and calendar. Write a morning briefing to memory."

# Continuous learning
gemini -p "Create a scheduled task: every Sunday at 8pm, extract patterns from this week's sessions and update the learned skills."

Step 3: Initialize Memory Graph

# Bootstrap your identity and context
gemini -p "Create memory entities for: me (user profile), my projects, my key contacts. Add observations about current priorities."

Step 4: Enable Computer Use (Optional)

Grant computer-use MCP the necessary permissions for browser and desktop control.

Example Workflows

Autonomous PR Reviewer

Cron: every 30 min during work hours
1. Check for new PRs on watched repos
2. For each new PR:
   - Pull branch locally
   - Run tests
   - Review changes with code-reviewer agent
   - Post review comments via GitHub MCP
3. Update memory with review status

Personal Research Agent

Cron: daily at 6 AM
1. Check saved search queries in memory
2. Run Exa searches for each query
3. Summarize new findings
4. Compare against yesterday's results
5. Write digest to memory
6. Flag high-priority items for morning review

Meeting Prep Agent

Trigger: 30 min before each calendar event
1. Read calendar event details
2. Search memory for context on attendees
3. Pull recent email/Slack threads with attendees
4. Prepare talking points and agenda suggestions
5. Write prep doc to memory

Constraints

• Cron tasks run in isolated sessions — they don't share context with interactive sessions unless through memory.
• Computer use requires explicit permission grants. Don't assume access.
• Remote dispatch may have rate limits. Design crons with appropriate intervals.
• Memory files should be kept concise. Archive old data rather than letting files grow unbounded.
• Always verify that scheduled tasks completed successfully. Add error handling to cron prompts.