dover-auto-debugging

name	dover-auto-debugging
title	DoVer: Intervention-Driven Auto Debugging for LLM Multi-Agent Systems
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2512.06749
keywords	["multi-agent systems","auto-debugging","intervention-driven","failure recovery","agent coordination"]
description	Diagnose and fix multi-agent system failures through targeted interventions (message edits, plan changes) rather than static log analysis. DoVer recovers 18-28% of failed trials with 30-60% hypothesis validation—essential for autonomous multi-agent reliability.

Overview

DoVer augments traditional log-based debugging with active verification through targeted system interventions. Rather than accepting single-point attributions, the framework systematically tests modifications to agent communications and planning to determine which changes resolve failures, providing practical mechanisms for improving multi-agent reliability.

When to Use

• Multi-agent systems experiencing task failures
• Debugging attribution is uncertain or insufficient
• Need to determine which modifications resolve failures
• Testing multiple potential fixes for the same failure
• Applications where failure recovery is critical
• Autonomous agent systems requiring reliability improvement

When NOT to Use

• Single-agent systems or simple pipelines
• Deterministic workflows without failure modes
• Cases where static log analysis suffices
• Real-time systems where intervention overhead is unacceptable
• Scenarios where testing modifications on live systems is risky

Core Technique

Hypothesis generation and verification through targeted interventions:

# Intervention-Driven Debugging Framework
class DoVerDebugger:
    def __init__(self, agent_framework):
        self.framework = agent_framework  # e.g., AG2, Anthropic Framework
        self.failed_trials = []

    def analyze_failure(self, trial):
        """
        Analyze failed trial to generate debugging hypotheses.
        Goes beyond log analysis to test interventions.
        """
        hypothesis_candidates = []

        # Extract relevant context
        history = trial.execution_trace
        agents = trial.agents
        final_state = trial.final_state
        task = trial.task

        # Hypothesis 1: Agent A made incorrect decision
        for agent in agents:
            hypothesis = {
                'type': 'agent_decision',
                : agent,
                : 
            }
            hypothesis_candidates.append(hypothesis)

        
         agent_pair  .get_agent_pairs(agents):
            hypothesis = {
                : ,
                : agent_pair,
                : 
            }
            hypothesis_candidates.append(hypothesis)

        
        hypothesis = {
            : ,
            : 
        }
        hypothesis_candidates.append(hypothesis)

         hypothesis_candidates

     ():
        
         hypothesis[] == :
            
             .test_agent_intervention(trial, hypothesis)

         hypothesis[] == :
            
             .test_message_intervention(trial, hypothesis)

         hypothesis[] == :
            
             .test_plan_intervention(trial, hypothesis)

     ():
        
        agent = hypothesis[]
        original_trial = trial

        
        alternatives = .generate_alternative_actions(
            agent,
            original_trial.execution_trace
        )

        results = []
         alternative_action  alternatives:
            
            modified_trial = .run_with_intervention(
                original_trial,
                agent,
                alternative_action
            )

            
            success = modified_trial.completed_successfully
            results.append({
                : alternative_action,
                :   success  ,
                : modified_trial
            })

         results

     ():
        
        agent_pair = hypothesis[]
        agent_a, agent_b = agent_pair

        
        messages = .extract_messages(
            trial.execution_trace,
            agent_a,
            agent_b
        )

        results = []
         message  messages:
            
            improved_messages = .improve_message(message)

             improved_msg  improved_messages:
                
                modified_trial = .run_with_message_edit(
                    trial,
                    message,
                    improved_msg
                )

                success = modified_trial.completed_successfully
                results.append({
                    : message,
                    : improved_msg,
                    :   success  
                })

         results

     ():
        
        original_plan = trial.plan
        problem_stage = .identify_problem_stage(trial)

        
        alternative_plans = .generate_alternative_plans(
            trial.task,
            original_plan,
            problem_stage
        )

        results = []
         alt_plan  alternative_plans:
            
            modified_trial = .run_with_plan_intervention(
                trial,
                alt_plan
            )

            success = modified_trial.completed_successfully
            progress = .measure_progress(modified_trial)

            results.append({
                : alt_plan,
                :   success  ,
                : progress
            })

         results

     ():
        
        successful_interventions = [
            r  r  intervention_results
             r[] == 
        ]

         (successful_interventions) > :
             , successful_interventions
        :
            
            progress_results = [
                r  r  intervention_results
                 r.get(, ) > 
            ]
             (progress_results) > :
                 , progress_results
            :
                 , []

     ():
        
        .framework.apply_intervention(
            trial_id,
            successful_intervention
        )

        
        recovered_trial = .framework.retry_with_intervention(
            trial_id,
            successful_intervention
        )

         recovered_trial