agent-self-correction

name	Agent Self-Correction
description	AI agent self-correction mechanisms: error detection, validation loops, recovery strategies, confidence scoring, and iterative refinement

Agent Self-Correction

Overview

AI agent self-correction mechanisms enable agents to detect errors, validate outputs, and automatically recover from failures. This includes validation loops, confidence scoring, iterative refinement, and recovery strategies to improve reliability.

Why This Matters

• Reliability: Agents แก้ error ได้เองโดยไม่ต้อง human intervention
• Quality: Output มีคุณภาพสูงขึ้น
• Trust: Users มั่นใจในผลลัพธ์ที่ได้
• Efficiency: ลด retry loops ที่ไม่จำเป็น

---

Core Concepts

1. Error Detection

interface ErrorDetection {
  type: 'syntax' | 'semantic' | 'logic' | 'format'
  severity: 'low' | 'medium' | 'high' | 'critical'
  message: string
  location?: string
}

class ErrorDetector {
  detectErrors(output: string, context: any): ErrorDetection[] {
    const errors: ErrorDetection[] = []

    // Syntax errors
    errors.push(...this.detectSyntaxErrors(output))

    // Semantic errors
    errors.push(...this.detectSemanticErrors(output, context))

    // Logic errors
    errors.push(...this.detectLogicErrors(output, context))

    // Format errors
    errors.push(...this.detectFormatErrors(output, context))

    return errors
  }

  private detectSyntaxErrors(output: string): ErrorDetection[] {
    const errors: ErrorDetection[] = []

    // Check for unclosed brackets
    const openBrackets = (output.match(/\(/g) || []).length
    const closeBrackets = (output.match(/\)/g) || []).length
    if (openBrackets !== closeBrackets) {
      errors.push({
        type: 'syntax',
        severity: 'high',
        message: 'Unclosed brackets detected',
      })
    }

    // Check for unclosed quotes
    const quotes = output.match(/"/g)
    if (quotes && quotes.length % 2 !== 0) {
      errors.push({
        type: 'syntax',
        severity: 'high',
        message: 'Unclosed quotes detected',
      })
    }

    return errors
  }

  private detectSemanticErrors(output: string, context: any): ErrorDetection[] {
    const errors: ErrorDetection[] = []

    // Check for hallucinations (if context provided)
    if (context.facts) {
      const outputFacts = this.extractFacts(output)
      for (const fact of outputFacts) {
        if (!context.facts.includes(fact)) {
          errors.push({
            type: 'semantic',
            severity: 'medium',
            message: `Potential hallucination: "${fact}" not in context`,
          })
        }
      }
    }

    return errors
  }

  private detectLogicErrors(output: string, context: any): ErrorDetection[] {
    const errors: ErrorDetection[] = []

    // Check for contradictions
    const statements = this.extractStatements(output)
    for (let i = 0; i < statements.length; i++) {
      for (let j = i + 1; j < statements.length; j++) {
        if (this.areContradictory(statements[i], statements[j])) {
          errors.push({
            type: 'logic',
            severity: 'high',
            message: 'Contradictory statements detected',
          })
        }
      }
    }

    return errors
  }

  private detectFormatErrors(output: string, context: any): ErrorDetection[] {
    const errors: ErrorDetection[] = []

    // Check if JSON is valid when expected
    if (context.expectedFormat === 'json') {
      try {
        JSON.parse(output)
      } catch (e) {
        errors.push({
          type: 'format',
          severity: 'critical',
          message: 'Invalid JSON output',
        })
      }
    }

    return errors
  }

  private extractFacts(text: string): string[] {
    // Extract factual statements
    return []
  }

  private extractStatements(text: string): string[] {
    // Extract logical statements
    return []
  }

  private areContradictory(a: string, b: string): boolean {
    // Check if two statements contradict
    return false
  }
}

2. Validation Loops

interface ValidationResult {
  isValid: boolean
  errors: string[]
  warnings: string[]
  confidence: number
}

class ValidationLoop {
  private maxIterations: number = 3
  private confidenceThreshold: number = 0.8

  async executeWithValidation<T>(
    task: () => Promise<T>,
    validator: (result: T) => ValidationResult,
    corrector: (result: T, errors: string[]) => Promise<T>
  ): Promise<T> {
    let result = await task()
    let iteration = 0

    while (iteration < this.maxIterations) {
      const validation = validator(result)

      if (validation.isValid && validation.confidence >= this.confidenceThreshold) {
         result
      }

      .()
      .(, validation.)
      .(, validation.)
      .(, validation.)

      
      result =  (result, validation.)
      iteration++
    }

      ()
  }
}


 loop =  ()

 result =  loop.(
  
   () => {
      llm.()
  },
  
   {
     : [] = []
     : [] = []
     confidence = 

    
     {
      
    }  (e) {
      errors.()
      confidence -= 
    }

    
     (!code.()) {
      warnings.()
      confidence -= 
    }

     {
      : errors. === ,
      errors,
      warnings,
      confidence,
    }
  },
  
   (: , : []) => {
     prompt = 
      llm.(prompt)
  }
)

3. Confidence Scoring

interface ConfidenceMetrics {
  overall: number
  components: {
    syntax: number
    semantic: number
    logic: number
    completeness: number
  }
  reasoning: string[]
}

class ConfidenceScorer {
  calculateConfidence(output: string, context: any): ConfidenceMetrics {
    const components = {
      syntax: this.scoreSyntax(output),
      semantic: this.scoreSemantic(output, context),
      logic: this.scoreLogic(output, context),
      completeness: this.scoreCompleteness(output, context),
    }

    const overall = (
      components.syntax * 0.2 +
      components.semantic * 0.3 +
      components.logic * 0.3 +
      components.completeness * 0.2
    )

    const reasoning = this.generateReasoning(components)

    return { overall, components, reasoning }
  }

   (: ):  {
     score = 

    
     brackets = output.() || []
     balance = 
     ( bracket  brackets) {
       ([, , ].(bracket)) {
        balance++
      }  {
        balance--
      }
       (balance < ) {
        score -= 
      }
    }
     (balance !== ) {
      score -= 
    }

    
     (output.() || output.()) {
      score += 
    }

     .(, .(, score))
  }

   (: , : ):  {
     score = 

    
     (context.) {
       outputKeywords = output.().()
       matchedKeywords = context..(
        outputKeywords.(k.())
      )
      score = matchedKeywords. / context..
    }

     score
  }

   (: , : ):  {
     score = 

    
     sentences = output.().( s.())
     (sentences. < ) {
      score -= 
    }

    
    

     .(, .(, score))
  }

   (: , : ):  {
     score = 

    
     (context.) {
       present = context..(
        output.(e)
      )
      score = present. / context..
    }

    
     (context. && output. < context.) {
      score -= 
    }
     (context. && output. > context.) {
      score -= 
    }

     .(, .(, score))
  }

   (: ): [] {
     : [] = []

     (components. < ) {
      reasoning.()
    }
     (components. < ) {
      reasoning.()
    }
     (components. < ) {
      reasoning.()
    }
     (components. < ) {
      reasoning.()
    }

     reasoning
  }
}

4. Recovery Strategies

interface RecoveryStrategy {
  name: string
  canHandle: (error: Error) => boolean
  recover: (error: Error, context: any) => Promise<any>
}

class RecoveryManager {
  private strategies: RecoveryStrategy[] = []

  addStrategy(strategy: RecoveryStrategy): void {
    this.strategies.push(strategy)
  }

  async recover(error: Error, context: any): Promise<any> {
    for (const strategy of this.strategies) {
      if (strategy.canHandle(error)) {
        console.log(`Applying recovery strategy: ${strategy.name}`)
        return await strategy.recover(error, context)
      }
    }

      ()
  }
}


 recoveryManager =  ()


recoveryManager.({
  : ,
  :  error  ,
  :  (error, context) => {
     () 
     context.()
  },
})


recoveryManager.({
  : ,
  :  error  ,
  :  (error, context) => {
     context.
  },
})


recoveryManager.({
  : ,
  :  error  ,
  :  (error, context) => {
     rephrased =  llm.(
      
    )
      context.(rephrased)
  },
})


recoveryManager.({
  : ,
  :  error  ,
  :  (error, context) => {
     simplified =  llm.(
      
    )
      context.(simplified)
  },
})

5. Iterative Refinement

class IterativeRefiner {
  private maxIterations: number = 5
  private improvementThreshold: number = 0.1

  async refine<T>(
    initial: T,
    evaluator: (item: T) => number,
    refiner: (item: T, feedback: string) => Promise<T>
  ): Promise<T> {
    let current = initial
    let currentScore = evaluator(current)
    let iteration = 0

    while (iteration < this.maxIterations) {
      const feedback = this.generateFeedback(current, currentScore)
      const refined = await refiner(current, feedback)
      const refinedScore = evaluator(refined)

      const improvement = (refinedScore - currentScore) / currentScore

      console.log(`Iteration ${iteration + 1}:`)
      console.log(`  Current score: `)
      .()
      .()

       (improvement < .) {
        .()
        
      }

      current = refined
      currentScore = refinedScore
      iteration++
    }

     current
  }

   generateFeedback<T>(: T, : ):  {
     : [] = []

     (score < ) {
      feedback.()
    }   (score < ) {
      feedback.()
    }  {
      feedback.()
    }

    
    

     feedback.()
  }
}


 refiner =  ()

 refinedCode =  refiner.(
  initialCode,
  
   {
     score = 

    
     (code.() && code.()) {
      score += 
    }

    
     (code.() || code.()) {
      score += 
    }

    
     (code.() || code.()) {
      score += 
    }

     .(, score)
  },
  
   (: , : ) => {
     prompt = 
      llm.(prompt)
  }
)

6. Self-Reflection

interface ReflectionResult {
  success: boolean
  confidence: number
  issues: string[]
  improvements: string[]
}

class SelfReflectiveAgent {
  async execute(task: string): Promise<string> {
    // Execute task
    const output = await this.generateOutput(task)

    // Reflect on output
    const reflection = await this.reflect(output, task)

    // If issues found, refine
    if (!reflection.success || reflection.confidence < 0.8) {
      console.log('Self-reflection detected issues, refining...')
      return await this.refine(output, reflection.issues)
    }

    return output
  }

  private async generateOutput(task: string): Promise<string> {
    // Generate initial output
      llm.(task)
  }

    (: , : ): <> {
     prompt = 

     reflection =  llm.(prompt)
     .(reflection)
  }

    (: , : []): <> {
     prompt = 
      llm.(prompt)
  }
}

Quick Start

// 1. Set up error detection
const detector = new ErrorDetector()

// 2. Set up validation loop
const loop = new ValidationLoop()

// 3. Execute with self-correction
const result = await loop.executeWithValidation(
  () => llm.generate(task),
  (output) => {
    const errors = detector.detectErrors(output, context)
    return {
      isValid: errors.length === 0,
      errors: errors.map(e => e.message),
      warnings: [],
      confidence: 1.0 - (errors.length * 0.2),
    }
  },
  (output, errors) => llm.generate(`Fix: ${errors.join(', ')}\nOutput: ${output}`)
)

Production Checklist

• Error detection implemented
• Validation loops configured
• Confidence scoring enabled
• Recovery strategies defined
• Iterative refinement active
• Self-reflection enabled
• Monitoring/logging in place
• Fallback mechanisms defined

Anti-patterns

1. No error detection: ไม่ตรวจสอบผลลัพธ์
2. Infinite loops: Validation loops ไม่มี max iterations
3. Over-correction: แก้ปัญหาจนเกินไป
4. No fallback: ไม่มี strategy สำรองเมื่อ recovery ล้มเหลว
5. Ignoring confidence: ไม่สนใจ confidence scores

Integration Points

• LLM APIs
• Monitoring systems
• Logging frameworks
• Alerting systems
• Feedback loops

Further Reading

• Self-Correction in LLMs
• Chain of Thought Prompting
• Reflexion: Language Agents with Verbal Reinforcement Learning
• Self-Consistency