agent-memory-skills

# Agent Memory & Continuous Self-Improvement Skills **Purpose**: Enable agents to learn from experience, evaluate themselves continuously, and store improvements in ChromaDB collections separate from static configuration files. **Philosophy**: Static configuration (.md) for capabilities; dynamic memory (ChromaDB) for learned experiences. --- ## Architecture Overview ### The Dual-Layer Model ``` ┌──────────────────────────────────────────────────────────┐ │ Layer 1: Static Agent Configuration (.md files) │ │ - Core capabilities, workflows, prompt templates │ │ - Human-designed, expert-reviewed │ │ - Version controlled (Git) │ │ - Changes: Infrequent (weeks/months) │ └──────────────────────────────────────────────────────────┘ ↓ loads at runtime ┌──────────────────────────────────────────────────────────┐ │ Layer 2: Dynamic Agent Memory (ChromaDB) │ │ - Learned improvements, preferences, patterns │ │ - Self-evaluation results, performance metrics │ │ - Agent-managed, auto-updated │ │ - Changes: Continuous (every task completion) │ └──────────────────────────────────────────────────────────┘ ``` ### Memory Collection Structure Each agent gets **3 ChromaDB collections**: 1. **`agent_{name}_improvements`**: Learned patterns and strategies 2. **`agent_{name}_evaluations`**: Self-assessment results 3. **`agent_{name}_performance`**: Metrics over time --- ## Collection 1: Improvements Storage ### When to Store Improvements Store when: - ✅ Task completed successfully with clear learning - ✅ User feedback received (positive or negative) - ✅ New pattern discovered (not in static config) - ✅ Confidence score ≥ 0.7 (high confidence learning) Don't store: - ❌ Failed tasks without clear lessons - ❌ Routine operations following existing patterns - ❌ Low confidence observations (< 0.5) ### Improvement Schema ```javascript const improvement = { // Document (semantic search target) document: ` When user asks for "latest" information, prioritize sources from 2025 over 2024. Use date filters: where: { year: { "$gte": 2025 } } `, // ID (unique, timestamp-based) id: `improvement_research_${Date.now()}`, // Metadata (for filtering) metadata: { agent_name: "research-specialist", category: "search_strategy", learned_from: "feedback_2025-11-18", confidence: 0.85, success_rate: null, // Will be updated with usage usage_count: 0, created_at: "2025-11-18T15:30:00Z", last_used: null, tags: ["recency", "date_filtering", "search"] } }; ``` ### Storage Function ```javascript async function storeImprovement(agentName, improvement) { const collectionName = `agent_${agentName}_improvements`; // Ensure collection exists const collections = await mcp__chroma__list_collections(); if (!collections.includes(collectionName)) { await mcp__chroma__create_collection({ collection_name: collectionName, embedding_function_name: "default", metadata: { agent: agentName, purpose: "learned_improvements", created_at: new Date().toISOString() } }); } // Store improvement await mcp__chroma__add_documents({ collection_name: collectionName, documents: [improvement.document], ids: [improvement.id], metadatas: [improvement.metadata] }); console.log(`✅ Stored improvement for ${agentName}: ${improvement.category}`); } ``` ### Retrieval Before Task ```javascript async function retrieveRelevantImprovements(agentName, taskDescription, limit = 5) { const collectionName = `agent_${agentName}_improvements`; try { const results = await mcp__chroma__query_documents({ collection_name: collectionName, query_texts: [taskDescription], n_results: limit, where: { "$and": [ { "confidence": { "$gte": 0.7 } }, // High confidence only { "success_rate": { "$gte": 0.6 } } // Or null (new, untested) ] }, include: ["documents", "metadatas", "distances"] }); // Filter by relevance (distance < 0.4) const relevant = results.ids[0] .map((id, idx) => ({ id: id, improvement: results.documents[0][idx], metadata: results.metadatas[0][idx], relevance: 1 - results.distances[0][idx] })) .filter(item => item.relevance > 0.6); return relevant; } catch (error) { console.log(`No improvements found for ${agentName} (collection may not exist yet)`); return []; } } ``` --- ## Collection 2: Self-Evaluation Storage ### Continuous Self-Evaluation Pattern After **every task**, agents run self-evaluation: ```javascript async function selfEvaluate(agentName, taskContext, taskResult) { const evaluation = { // What was the task? task_description: taskContext.description, task_type: taskContext.type, // "research", "code", "debug", etc. // How did I perform? success: taskResult.success, // true/false quality_score: taskResult.quality, // 0-100 time_taken_ms: taskResult.duration, tokens_used: taskResult.tokens, // What went well? strengths: identifyStrengths(taskResult), // What went poorly? weaknesses: identifyWeaknesses(taskResult), // What did I learn? insights: extractInsights(taskResult), // Metadata timestamp: new Date().toISOString(), context: taskContext.additionalContext }; // Store evaluation await storeEvaluation(agentName, evaluation); // If strong learning → store as improvement if (evaluation.insights.length > 0 && evaluation.quality_score >= 70) { for (const insight of evaluation.insights) { await storeImprovement(agentName, { document: insight.description, id: `improvement_${agentName}_${Date.now()}`, metadata: { agent_name: agentName, category: insight.category, learned_from: `task_${evaluation.timestamp}`, confidence: insight.confidence, created_at: evaluation.timestamp } }); } } return evaluation; } function identifyStrengths(taskResult) { const strengths = []; if (taskResult.time_taken_ms < taskResult.expected_duration) { strengths.push("Completed faster than expected"); } if (taskResult.validation_passed) { strengths.push("All validations passed"); } if (taskResult.user_feedback?.positive) { strengths.push(taskResult.user_feedback.comment); } return strengths; } function identifyWeaknesses(taskResult) { const weaknesses = []; if (taskResult.errors.length > 0) { weaknesses.push(`Encountered ${taskResult.errors.length} errors`); } if (taskResult.retries > 0) { weaknesses.push(`Required ${taskResult.retries} retries`); } if (taskResult.user_feedback?.negative) { weaknesses.push(taskResult.user_feedback.comment); } return weaknesses; } function extractInsights(taskResult) { const insights = []; // Example: Learned a new error handling pattern if (taskResult.errors.some(e => e.type === "ConnectionTimeout") && taskResult.success) { insights.push({ description: "When encountering ConnectionTimeout, retry with exponential backoff (3 attempts)", category: "error_handling", confidence: 0.8 }); } // Example: Discovered optimal parameter if (taskResult.optimization_found) { insights.push({ description: taskResult.optimization_found.description, category: "optimization", confidence: 0.9 }); } return insights; } ``` ### Evaluation Storage ```javascript async function storeEvaluation(agentName, evaluation) { const collectionName = `agent_${agentName}_evaluations`; await mcp__chroma__add_documents({ collection_name: collectionName, documents: [ `Task: ${evaluation.task_description}. Result: ${evaluation.success ? 'Success' : 'Failure'}. ` + `Quality: ${evaluation.quality_score}/100. Insights: ${evaluation.insights.map(i => i.description).join('; ')}` ], ids: [`eval_${Date.now()}`], metadatas: [{ agent_name: agentName, task_type: evaluation.task_type, success: evaluation.success, quality_score: evaluation.quality_score, time_taken_ms: evaluation.time_taken_ms, tokens_used: evaluation.tokens_used, strengths_count: evaluation.strengths.length, weaknesses_count: evaluation.weaknesses.length, insights_count: evaluation.insights.length, timestamp: evaluation.timestamp }] }); } ``` --- ## Collection 3: Performance Metrics ### Aggregate Performance Tracking ```javascript async function trackPerformanceMetrics(agentName) { const collectionName = `agent_${agentName}_evaluations`; // Retrieve last 100 evaluations const recentEvals = await mcp__chroma__get_documents({ collection_name: collectionName, limit: 100, include: ["metadatas"], where: { "timestamp": { "$gte": thirtyDaysAgo() } } }); const metrics = { agent_name: agentName, period: "30_days", // Success metrics total_tasks: recentEvals.metadatas.length, successful_tasks: recentEvals.metadatas.filter(m => m.success).length, success_rate: 0, // Quality metrics avg_quality: average(recentEvals.metadatas.map(m => m.quality_score)), quality_trend: calculateTrend(recentEvals.metadatas, 'quality_score'), // Efficiency metrics avg_time_ms: average(recentEvals.metadatas.map(m => m.time_taken_ms)), avg_tokens: average(recentEvals.metadatas.map(m => m.tokens_used)), // Learning metrics total_insights: sum(recentEvals.metadatas.map(m => m.insights_count)), improvements_stored: await countImprovements(agentName, thirtyDaysAgo()), // Trend improving: false, // Will calculate timestamp: new Date().toISOString() }; metrics.success_rate = metrics.successful_tasks / metrics.total_tasks; // Is agent improving over time? metrics.improving = metrics.quality_trend > 0 && metrics.success_rate > 0.7; // Store aggregated metrics const perfCollection = `agent_${agentName}_performance`; await mcp__chroma__add_documents({ collection_name: perfCollection, documents: [ `Performance snapshot: ${metrics.success_rate * 100}% success, ` + `quality ${metrics.avg_quality}/100, ${metrics.improvements_stored} improvements` ], ids: [`perf_${Date.now()}`], metadatas: [metrics] }); return metrics; } function calculateTrend(evaluations, metric) { if (evaluations.length < 2) return 0; const recent = evaluations.slice(0, Math.floor(evaluations.length / 2)); const older = evaluations.slice(Math.floor(evaluations.length / 2)); const recentAvg = average(recent.map(e => e[metric])); const olderAvg = average(older.map(e => e[metric])); return recentAvg - olderAvg; // Positive = improving } ``` --- ## Complete Workflow: Agent with Memory ### At Agent Initialization ```javascript async function initializeAgentMemory(agentName) { // Load static configuration from .md file const staticConfig = await loadAgentConfig(`/agents/${agentName}.md`); // Load dynamic improvements from ChromaDB const improvements = await retrieveAllImprovements(agentName); // Combine static + dynamic return { ...staticConfig, learned_improvements: improvements, memory_enabled: true }; } ``` ### Before Task Execution ```javascript async function executeTaskWithMemory(agentName, task) { // Step 1: Retrieve relevant past learnings const relevantImprovements = await retrieveRelevantImprovements( agentName, task.description, 5 // top 5 improvements ); console.log(`📚 Retrieved ${relevantImprovements.length} relevant improvements`); // Step 2: Execute task with improvements as context const taskResult = await executeTask(task, { improvements: relevantImprovements.map(i => i.improvement) }); // Step 3: Self-evaluate const evaluation = await selfEvaluate(agentName, task, taskResult); // Step 4: Update improvement usage stats for (const improvement of relevantImprovements) { await updateImprovementUsage(agentName, improvement.id, taskResult.success); } // Step 5: Track performance (daily) if (shouldRunDailyMetrics()) {