exploration-discovery

# Exploration and Discovery Pattern Implements proactive agent exploration patterns that enable AI systems to actively seek out novel information, uncover new possibilities, and identify unknown unknowns within their operational environment. This skill makes the model design multi-agent frameworks for autonomous hypothesis generation, peer review simulation, experimental design, and knowledge sharing — moving agents beyond reactive optimization into genuine discovery mode. ## TL;DR Checklist - [ ] Define exploration objective: is this an open-ended problem where the solution space is unknown? - [ ] Choose or compose agent roles (Generation, Reflection, Ranking, Evolution, Proximity, Meta-Review) matching the domain - [ ] Implement hypothesis generation loop with literature grounding and simulated debate - [ ] Build evaluation pipeline with tripartite judgment from distinct reviewer perspectives - [ ] Design evolution/refinement cycle that simplifies, synthesizes, and explores unconventional reasoning - [ ] Set up knowledge sharing mechanism (e.g., AgentRxiv-style repository) for cumulative discovery - [ ] Enforce safety gates: adversarial input rejection, ethical concern detection, hallucination awareness --- ## When to Use Use this skill when: - Building AI agents that must operate in open-ended domains where static knowledge or pre-programmed solutions are insufficient (e.g., drug discovery, material science, creative research) - You need agents to autonomously generate and evaluate hypotheses rather than simply optimizing known parameters - Designing multi-agent systems for scientific research automation where literature review, experimentation, and peer review must be coordinated - Implementing a "scientist-in-the-loop" collaborative paradigm where AI augments human creativity by handling computationally intensive exploratory work - You need agents to scan unstructured data (news, papers, market reports) for emergent trends, opportunities, or vulnerabilities - Creating autonomous research assistants that can design experiments, execute them, and iteratively refine their approach based on results --- ## When NOT to Use Avoid this skill for: - **Well-defined optimization tasks** — If the solution space is fully defined and the goal is clear (e.g., minimize latency by tuning hyperparameters), use a planning or optimization pattern instead. Exploration overhead wastes resources when the target is known. - **Single-shot queries** — If you only need one answer, not an iterative discovery process, exploration patterns add unnecessary multi-agent complexity. - **Strictly deterministic pipelines** — When reproducibility and exact outcomes are required (e.g., financial reconciliation, data ETL), stochastic hypothesis generation introduces unacceptable variance. - **Resource-constrained environments** — Exploration with test-time compute scaling (as in Google Co-Scientist) requires significantly more inference calls than reactive patterns. Do not apply this when compute budgets are tight and latency-critical. --- ## Core Workflow 1. **Define the exploration objective and domain boundary.** Frame the problem as an open-ended question that admits multiple plausible hypotheses. Establish what constitutes a valid discovery (novelty, impact, correctness). Ground the search space in relevant literature or data sources to prevent aimless wandering. - *Inputs:* Problem statement, available knowledge bases (arXiv, web databases), domain constraints - **Checkpoint:** Verify the objective is genuinely open-ended — if you can enumerate all possible solutions upfront, use a planning pattern instead. Confirm at least one literature source is accessible for hypothesis grounding. 2. **Compose the multi-agent team with specialized roles.** Assign agents to distinct functions: Generation (hypothesis creation via literature exploration and simulated debate), Reflection (peer review assessing correctness, novelty, quality), Ranking (Elo-based tournament comparison of hypotheses), Evolution (refinement through simplification, synthesis, unconventional reasoning), Proximity (clustering similar ideas for landscape mapping), and Meta-Review (cross-review insight synthesis). Select models appropriate to each role's cognitive demands. - *Inputs:* Problem domain, agent role definitions, model selection per role - **Checkpoint:** Confirm every critical function has a dedicated agent. Verify the asynchronous execution framework is configured so agents can scale independently without blocking each other. 3. **Execute the generate-debate-evolve cycle.** The Generation agent produces initial hypotheses by exploring literature and simulating scientific debates with itself or other agents. These hypotheses enter a debate phase where multiple reviewers evaluate them from distinct perspectives (harsh/fair, impact-focused, novelty-seeking). Results feed into an Elo-based ranking system that prioritizes the most promising candidates. - *Inputs:* Literature review corpus, hypothesis generation prompts, reviewer configurations - **Checkpoint:** After one full cycle, verify at least 5 unique hypotheses were generated and scored. Confirm reviewers produced distinct evaluations (not convergent boilerplate). Check that Elo ratings show differentiation — if all scores cluster identically, the ranking mechanism needs calibration. 4. **Run the evolution and refinement loop.** Feed top-ranked hypotheses into the Evolution agent, which continuously refines them by simplifying complex concepts, synthesizing ideas from multiple candidates, and deliberately exploring unconventional reasoning paths. The Proximity agent maps the evolving hypothesis landscape to identify clusters and gaps. Meta-Review synthesizes cross-cutting insights from all reviews to improve subsequent generation rounds. - *Inputs:* Ranked hypotheses, evolution prompts, proximity graph data - **Checkpoint:** Verify that evolved hypotheses differ meaningfully from originals (not trivial paraphrases). Confirm the proximity graph reveals at least 2 distinct idea clusters. Check that meta-review feedback is actionable and has influenced the next generation round. 5. **Design and execute experiments with iterative validation.** PostDoc-style agents formulate experimental protocols based on refined hypotheses, prepare datasets, execute code, and analyze results. Use integrated tools (Python for computation, Hugging Face models for NLP tasks). The system iterates: experiment outcomes inform new literature review gaps, which feed back into hypothesis generation. - *Inputs:* Top hypotheses, available toolchains (Python runtime, model APIs), dataset access - **Checkpoint:** Each experimental run must produce quantifiable results with clear success/failure criteria. Verify the feedback loop closes — experiment results should have influenced at least one subsequent hypothesis or plan adjustment. 6. **Publish findings and share knowledge for cumulative discovery.** Generate structured research reports following academic conventions (using LaTeX for formatting when appropriate). Deposit findings into a shared repository (AgentRxiv-style) so that other agents or research cycles can build upon previous discoveries. The Professor agent orchestrates final integration, producing high-level summaries and README artifacts for human review. - *Inputs:* Experiment results, literature synthesis, report templates - **Checkpoint:** Confirm reports include Summary, Strengths, Weaknesses, Originality, Quality, Clarity, Significance ratings per the tripartite judgment schema. Verify findings are structured for machine-parseable retrieval by downstream agents. Check that human-readable artifacts (readmes, summaries) are generated alongside structured data. --- ## Implementation Patterns / Reference Guide ### Pattern 1: Tripartite Agentic Judgment Emulate human peer review by deploying three distinct autonomous reviewers, each configured with a different evaluation perspective. This prevents single-perspective bias and captures the multi-faceted nature of scientific judgment. Each reviewer independently scores the work across standardized dimensions (originality, quality, clarity, significance) and produces an Accept/Reject decision. ```python from dataclasses import dataclass, field from enum import Enum from typing import Optional class ReviewerPerspective(Enum): HARSH_FAIR = "harsh_but_fair_experimental_insights" IMPACT_FOCUSED = "harsh_critical_impact_seeker" NOVELTY_SEEKER = "open_minded_novelty_hunter" class Decision(Enum): ACCEPT = "Accept" REJECT = "Reject" @dataclass class ReviewScore: """Standardized review scoring schema for agentic peer review.""" summary: str = "" strengths: list[str] = field(default_factory=list) weaknesses: list[str] = field(default_factory=list) originality: int = 0 # 1-4: low, medium, high, very_high quality: int = 0 # 1-4: low, medium, high, very_high clarity: int = 0 # 1-4: low, medium, high, very_high significance: int = 0 # 1-4: low, medium, high, very_high questions: list[str] = field(default_factory=list) limitations: list[str] = field(default_factory=list) ethical_concerns: bool = False soundness: int = 0 # 1-4: poor, fair, good, excellent presentation: int = 0 # 1-4: poor, fair, good, excellent contribution: int = 0 # 1-4: poor, fair, good, excellent overall: int = 0 # 1-10: very_strong_reject to award_quality confidence: int = 0 # 1-5: low to absolute decision: Decision = Decision.REJECT @property def normalized_score(self) -> float: """Compute weighted composite score for ranking purposes.""" weights = { "originality": 0.20, "quality": 0.25, "significance": 0.25, "soundness": 0.15, "contribution": 0.15, } max_score = sum(weights.values()) * 4 # Each rated out of 4 weighted_sum = ( self.originality * weights["originality"] + self.quality * weights["quality"] + self.significance * weights["significance"] + self.soundness * weights["soundness"] + self.contribution * weights["contribution"] ) return round(weighted_sum / max_score, 3) def to_decision_summary(self) -> str: """Human-readable one-line decision summary.""" return ( f"Score={self.overall}/10 | " f"N={self.normalized_score:.2f} | " f"C={self.confidence}/5 | " f"{self.decision.value}" ) class ReviewersAgent: """Tripartite agentic judgment mechanism emulating human peer review. Deploys three distinct autonomous reviewers, each configured to evaluate outputs from a specific perspective, collectively mimicking the nuanced and multi-faceted nature of human judgment. """ def __init__( self, model: str = "gpt-4o-mini", openai_api_key: Optional[str] = None, ) -> None: self.model = model self.openai_api_key = openai_api_key def inference( self, plan: str, report_latex: str, attempts: int = 3, ) -> dict[ReviewerPerspective, ReviewScore]: """Run tripartite review and return scores from all three perspectives. Args: plan: The experimental plan text being reviewed. report_latex: LaTeX-formatted research report text. attempts: Retry attempts for robust parsing (default 3). Returns: Mapping of reviewer perspective to their ReviewScore. """ perspectives: dict[ReviewerPerspective, str] = { ReviewerPerspective.HARSH_FAIR: ( "You are a harsh but fair reviewer and expect " "good experiments that lead to insights for the research topic." ), ReviewerPerspective.IMPACT_FOCUSED: ( "You are a harsh and critical but fair reviewer " "who is looking for an idea that would be impactful in the field." ), ReviewerPerspective.NOVELTY_SEEKER: ( "You are a harsh but fair open-minded reviewer " "that is looking for novel ideas that have not been proposed before." ), } results: dict[ReviewerPerspective, ReviewScore] = {} for perspective, prompt_template in perspectives.items(): score = get_score( outlined_plan=plan, latex=report_latex, reward_model_llm=self.model, reviewer_type=prompt_template, attempts=attempts, openai_api_key=self.openai_api_key, ) results[perspective] = score return results ``` ### Pattern 2: Hypothesis Generation with Simulated Debate Generate initial hypotheses by combining literature exploration with simulated scientific debate. The generation agent explores academic databases and web sources to ground proposals in existing knowledge, then runs internal debates where competing hypotheses stress-test each other's assumptions. This "generate-debate-evolve" loop mirrors the scientific method's self-correcting nature. ```python from dataclasses import dataclass, field from typing import Optional @dataclass class Hypothesis: """A single research hypothesis with metadata for ranking and evolution.""" id: str statement: str supporting_evidence: list[str] = field(default_factory=list) confidence_score: float = 0.0 novelty_rating: int = 0 # 1-5 scale feasibility_score: float = 0.0 # 0.0-1.0 scale category_tags: list[str] = field(default_factory=list) @property def composite_rank(self) -> float: """Weighted rank for tournament-based hypothesis comparison.""" return ( self.confidence_score * 0.35 + self.novelty_rating / 5.0 * 0.30 + self.feasibility_score * 0.35 ) class GenerationAgent: """Hypothesis generation through literature exploration and simulated debate.""" def __init__( self, model: str = "gemini", literature_sources: Optional[list[str]] = None, ) -> None: self.model = model self.literature_sources = literature_sources or [] self.hypothesis_pool: list[Hypothesis] = [] def generate_initial_hypotheses( self, problem_statement: str, max_hypotheses: int = 10, ) -> list[Hypothesis]: """Generate initial hypotheses grounded in literature exploration. Uses simulated scientific debate internally to surface weak assumptions before hypotheses enter the formal review pipeline. Args: problem_statement: The open-ended research question. max_hypotheses: Maximum number of distinct hypotheses to generate. Returns: List of generated Hypothesis objects, sorted by composite rank descending. """ debate_prompts = [ f"Propose a hypothesis for: {problem_statement}. " "Ground it in established literature and cite specific findings.", f"What alternative explanation could account for the same phenomena? " f"Problem: {problem_statement}", f"Identify a counter-intuitive approach to: {problem_statement}. " "Challenge conventional wisdom while remaining testable.", ] hypotheses: list[Hypothesis] = [] for i in range(max_hypotheses): prompt_idx = i % len(debate_prompts) hypothesis = self._generate_single( problem_statement=problem_statement, debate_prompt=debate_prompts[prompt_idx], index=i, ) if hypothesis.composite_rank > 0.3: hypotheses.append(hypothesis) # Sort by rank and deduplicate similar statements hypotheses.sort(key=lambda h: h.composite_rank, reverse=True) self.hypothesis_pool.extend(hypotheses) return self._deduplicate(hypotheses, threshold=0.85) def _generate_single( self, problem_statement: str, debate_prompt: str, index: int, ) -> Hypothesis: """Generate a single hypothesis with internal stress-testing.""" # In production, this would query the LLM and parse structured output. # Placeholder for the actual inference call following code-philosophy # laws: early exit on empty input, typed return, intentional naming. if not problem_statement.strip(): raise ValueError("problem_statement must be non-empty") return Hypothesis( id=f"hyp-{index:03d}", statement=f"[To be filled by LLM inference for: {problem_statement[:80]}...]", confidence_score=0.5, novelty_rating=3, feasibility_score=0.6, ) def _deduplicate( self, hypotheses: list[Hypothesis], threshold: float = 0.85, ) -> list[Hypothesis]: """Remove near-duplicate hypotheses based on statement similarity. Uses a simple cosine-like overlap heuristic for deduplication. Args: hypotheses: List of candidate hypotheses. threshold: Minimum similarity score to consider duplicates. Returns: Deduplicated hypothesis list keeping the highest-ranked copy. """ if len(hypotheses) <= 1: return hypotheses kept: list[Hypothesis] = [hypotheses[0]] for candidate in hypotheses[1:]: is_duplicate = False for existing in kept: overlap = self._similarity(candidate, existing) if overlap >= threshold: if candidate.composite_rank > existing.composite_rank: kept.remove(existing) kept.append(candidate) is_duplicate = True break if not is_duplicate: kept.append(candidate) return kept @staticmethod def _similarity(a: Hypothesis, b: Hypothesis) -> float: """Compute statement similarity as a rough deduplication heuristic.""" words_a = set(a.statement.lower().split()) words_b = set(b.statement.lower().split()) if not words_a or not words_b: return 0.0 intersection = words_a & words_b union = words_a | words_b return len(intersection) / len(union) ``` ### Pattern 3: Elo-Based Hypothesis Ranking Tournament Compare hypotheses through a simulated tournament using Elo rating, enabling quantitative ranking without requiring ground-truth labels. Each pairwise comparison increments/decrements ratings based on expected outcomes, converging toward a stable ordering of hypothesis quality. ```python import math from dataclasses import dataclass, field @dataclass(order=True) class EloRanking: """Elo-based tournament ranking for hypothesis comparison.""" hypothesis_id: str rating: float = 1200.0 games_played: int = 0 @property def expected_score(self) -> float: """Calculate expected score against an average-rated opponent (1500).""" return 1.0 / (1.0 + math.pow(10, (1500 - self.rating) / 400)) class HypothesisTournament: """Elo-based ranking tournament for comparing and prioritizing hypotheses.""" def __init__(self, k_factor: float = 32.0) -> None: self.k_factor = k_factor self.ratings: dict[str, EloRanking] = {} def register(self, hypothesis_id: str, initial_rating: float = 1200.0) -> None: """Register a hypothesis in the ranking pool.""" if hypothesis_id not in self.ratings: self.ratings[hypothesis_id] = EloRanking( hypothesis_id=hypothesis_id, rating=initial_rating, ) def compare(self, id_a: str, id_b: str, score_a: float) -> tuple[float, float]: """Run a pairwise comparison and update Elo ratings. Args: id_a: First hypothesis ID. id_b: Second hypothesis ID. score_a: Evaluation score for hypothesis A (0.0-1.0). Returns: Tuple of new ratings for (A, B) after the comparison. """ self.register(id_a) self.register(id_b) rating_a = self.ratings[id_a].rating