Skip to main content Startseite Ersteller adu2021 skillxiv automind-adaptive-data-science
automind-adaptive-data-science Build LLM-driven data science agents grounded in empirical knowledge through expert knowledge base, tree search algorithms, and complexity-adaptive code generation, surpassing SOTA by 8% on MLE-Bench.
Zur Installation springen Skills Marktplatz Entdecken und erkunden Sie KI-Skills, die von der Community erstellt wurden.
Mit Codex oder Claude installieren Kopieren Sie diesen Prompt, fügen Sie ihn in Codex, Claude oder einen anderen Assistant ein und lassen Sie die Skill-Seite prüfen und installieren.
Prompt-Details anzeigen Ein direkter Befehl überspringt den Prüf-Prompt. Prüfen Sie die Quelle, bevor Sie ihn ausführen.
npx skills add https://github.com/ADu2021/skillXiv --skill automind-adaptive-data-scienceDer Befehl bleibt in einer Zeile. Scrollen Sie horizontal, um ihn vor dem Kopieren vollständig zu prüfen.
Sie bevorzugen eine lokale Kopie? Laden Sie die Dateien herunter, die SkillsMP derzeit vorliegen.
ZIP herunterladen Herunterladen... Mehr aus diesem Repository
Verwandte Berufe SOC
Basierend auf der SOC-Berufsklassifikation
name automind-adaptive-data-science title AutoMind: Adaptive Knowledgeable Agent for Automated Data Science version 0.0.2 engine skillxiv-v0.0.2-claude-opus-4.6 license MIT url https://arxiv.org/abs/2506.10974 keywords ["automated machine learning","LLM agents","knowledge base","agentic search","code generation"] description Build LLM-driven data science agents grounded in empirical knowledge through expert knowledge base, tree search algorithms, and complexity-adaptive code generation, surpassing SOTA by 8% on MLE-Bench.
AutoMind: Adaptive Knowledgeable Agent for Automated Data Science
Core Concept
AutoMind addresses limitations in rigid LLM-driven data science workflows through three innovations: an expert knowledge base curated from Kaggle competitions and peer-reviewed papers, an agentic tree search algorithm exploring multiple solution paths, and adaptive code generation that scales complexity with task difficulty. The system achieves 41.2% human participant surpass rate on official leaderboards and outperforms prior state-of-the-art by 8% on MLE-Bench.
Architecture Overview
Expert Knowledge Base : 3,237 Kaggle posts + papers from KDD, ICLR, NeurIPS, ICML, EMNLP; hierarchically labeled for efficient retrievalAgentic Knowledgeable Tree Search : Explores solution tree with three action types—drafting (create branches), improving (refine), debugging (fix failures). Stochastic heuristic guides explorationSelf-Adaptive Code Generation : Switches between one-pass (simple plans) and stepwise (complex plans) based on solution complexity scoring. AST checking validates syntax before executionDynamic Solution Space : Manages plans, code, validation metrics; greedy selection optimizes top solutions while exploring alternativesScalable Model Support : GPT-4o, o3-mini, DeepSeek-V3; reduces token cost by 9.6% vs baseline
Implementation
Step 1: Expert Knowledge Base Construction
import json
from collections import defaultdict
class KnowledgeBase :
"""
Curates and indexes expert knowledge from diverse sources.
Supports hierarchical query and efficient retrieval.
"""
def __init__ (self ):
self .knowledge_index = defaultdict(list )
self .category_hierarchy = {
'feature_engineering' : [
'categorical_encoding' , 'feature_scaling' , 'polynomial_features' ,
'interaction_features' , 'domain_specific_features'
],
'model_selection'
'tree_based'
'linear_models'
'neural_networks'
'ensemble_methods'
'domain_adapted'
'hyperparameter_tuning'
'grid_search'
'random_search'
'bayesian_optimization'
'learning_rate'
'regularization'
'data_preprocessing'
'missing_value_handling'
'outlier_detection'
'normalization'
'dimensionality_reduction'
'sampling_strategies'
def
ingest_kaggle_posts
self, post_list
"""
Index Kaggle competition posts and solutions.
Post format: {'id', 'title', 'content', 'competition', 'votes'}
"""
for
in
self
'content'
for
in
self
self
'source'
'kaggle'
'post_id'
'id'
'competition'
'competition'
'content'
'content'
'votes'
'votes'
'technique'
def
ingest_academic_papers
self, papers
"""
Index papers from top-tier ML conferences.
Paper format: {'title', 'abstract', 'pdf_url', 'venue', 'year'}
"""
for
in
self
'abstract'
for
in
self
self
'source'
'academic'
'paper_title'
'title'
'venue'
'venue'
'year'
'year'
'abstract'
'abstract'
'methodology'
def
retrieve
self, query, category=None , top_k=5
"""
Retrieve relevant knowledge items.
Supports hierarchical category filtering.
"""
if
else
list
self
for
in
for
in
self
self
lambda
0
True
return
for
in
def
_extract_entities
self, text
"""Extract technical entities from text."""
return
def
_categorize_entity
self, entity
"""Determine category for entity."""
if
'encode'
in
return
'feature_engineering'
elif
'xgb'
in
or
'random'
in
return
'model_selection'
return
'feature_engineering'
def
_extract_methodologies
self, abstract
"""Extract methodologies from paper abstract."""
return
def
_categorize_method
self, method
"""Determine category for method."""
return
'feature_engineering'
def
_score_relevance
self, query, item
"""Score relevance of item to query."""
return
0.5
Step 2: Agentic Tree Search Algorithm import torch
from dataclasses import dataclass
from typing import Optional , List
@dataclass
class SolutionNode :
"""Represents a solution in the search tree."""
plan: str
code: str
validation_metric: float
parent: Optional ['SolutionNode' ] = None
children: List ['SolutionNode' ] = None
class AgenticTreeSearch :
"""
Explores solution space using three action types:
- Draft: Create new solution branches
- Improve: Refine existing solutions
- Debug: Fix execution failures
"""
def __init__ (self, knowledge_base, model_name='gpt-4o' ):
self .kb = knowledge_base
self .model = model_name
self .root = None
self .best_solution = None
self .best_metric = float ('-inf' )
def search (self, task_description, max_iterations=10 ):
"""
Main search loop: iteratively explore and refine solutions.
"""
initial_solution = self ._draft_solution(task_description)
self .root = SolutionNode(
plan=initial_solution['plan' ],
code=initial_solution['code' ],
validation_metric=0.0
)
current_node = self .root
for iteration in range (max_iterations):
print (f"Iteration {iteration} : Best metric={self.best_metric:.4 f} " )
action = self ._select_action(current_node)
if action == 'draft' :
for _ in range (2 ):
new_solution = self ._draft_solution(
task_description,
parent_context=current_node
)
new_node = SolutionNode(
plan=new_solution['plan' ],
code=new_solution['code' ],
validation_metric=0.0 ,
parent=current_node
)
current_node.children.append(new_node)
elif action == 'improve' :
improved = self ._improve_solution(
current_node.plan,
current_node.code
)
current_node.plan = improved['plan' ]
current_node.code = improved['code' ]
elif action == 'debug' :
if current_node.validation_metric == float ('-inf' ):
debugged = self ._debug_solution(
current_node.code,
error_log=current_node.error_log
)
current_node.code = debugged['code' ]
metric = self ._execute_and_validate(current_node.code)
current_node.validation_metric = metric
if metric > self .best_metric:
self .best_metric = metric
self .best_solution = current_node
print (f"New best: {metric:.4 f} " )
current_node = self ._select_next_node()
return self .best_solution
def _draft_solution (self, task, parent_context=None ):
"""Generate new solution draft using knowledge base."""
knowledge = self .kb.retrieve(task, top_k=3 )
draft_prompt = f"""
Task: {task}
Relevant knowledge:
{self._format_knowledge(knowledge)}
Generate a solution with:
1. High-level plan (strategy, approach)
2. Python code implementation
Output JSON:
{{"plan": "...", "code": "..."}}
"""
response = self ._llm_call(draft_prompt)
return self ._parse_solution(response)
def _improve_solution (self, plan, code ):
"""Refine solution iteratively."""
improve_prompt = f"""
Current plan: {plan}
Current code:
{code}
Suggest improvements:
1. Efficiency enhancements
2. Code quality improvements
3. Better ML practices
Provide improved plan and code.
"""
response = self ._llm_call(improve_prompt)
return self ._parse_solution(response)
def _debug_solution (self, code, error_log ):
"""Fix execution errors."""
debug_prompt = f"""
Code:
{code}
Error:
{error_log}
Fix the error and provide corrected code.
"""
response = self ._llm_call(debug_prompt)
return {'code' : response}
def _select_action (self, current_node ):
"""Select next action based on stochastic heuristic policy."""
if len (current_node.children) < 2 :
return 'draft'
elif current_node.validation_metric == float ('-inf' ):
return 'debug'
else :
return 'improve'
def _select_next_node (self ):
"""Greedy selection: move to best unexplored node."""
candidates = []
for node in self ._get_all_nodes():
if node.validation_metric == float ('-inf' ):
candidates.append(node)
if candidates:
return candidates[0 ]
return max (self ._get_all_nodes(),
key=lambda n: n.validation_metric)
def _execute_and_validate (self, code ):
"""Execute code and return validation metric."""
return 0.75
def _llm_call (self, prompt ):
"""Call LLM."""
return "response"
def _format_knowledge (self, knowledge ):
"""Format knowledge for prompt."""
return '\n' .join([f"- {k['technique' ]} " for k in knowledge])
def _parse_solution (self, response ):
"""Parse LLM response into solution dict."""
return {'plan' : 'plan' , 'code' : 'code' }
def _get_all_nodes (self ):
"""Collect all nodes in tree."""
def dfs (node ):
if node is None :
return []
result = [node]
for child in (node.children or []):
result.extend(dfs(child))
return result
return dfs(self .root)
Step 3: Self-Adaptive Code Generation import ast
class AdaptiveCodeGenerator :
"""
Generates code that adapts complexity to task difficulty.
Switches between one-pass (simple) and stepwise (complex).
"""
def generate_code (self, plan, task_complexity_score ):
"""
Generate code with complexity matching task difficulty.
Complexity score: 0-1 (0=simple, 1=very complex)
"""
if task_complexity_score < 0.4 :
return self ._generate_one_pass_code(plan)
else :
return self ._generate_stepwise_code(plan)
def _generate_one_pass_code (self, plan ):
"""
Simple implementation: single coherent script.
For straightforward tasks (basic preprocessing, standard models).
"""
code_template = f"""
import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split
# Load and preprocess
X = load_features()
y = load_labels()
X_scaled = StandardScaler().fit_transform(X)
X_train, X_test, y_train, y_test = train_test_split(X_scaled, y, test_size=0.2)
# Train model
model = RandomForestClassifier(n_estimators=100, random_state=42)
model.fit(X_train, y_train)
# Evaluate
accuracy = model.score(X_test, y_test)
print(f"Accuracy: {{accuracy:.4f}}")
"""
return code_template
def _generate_stepwise_code (self, plan ):
"""
Complex implementation: decomposed into substeps.
For intricate tasks (feature engineering, ensemble, hyperparameter tuning).
"""
code_template = f"""
import pandas as pd
import numpy as np
from sklearn.preprocessing import StandardScaler, PolynomialFeatures
from sklearn.ensemble import RandomForestClassifier, GradientBoostingClassifier
from sklearn.model_selection import cross_val_score
# Step 1: Advanced feature engineering
def engineer_features(X):
poly = PolynomialFeatures(degree=2, include_bias=False)
X_poly = poly.fit_transform(X)
return X_poly
# Step 2: Normalize
X = load_features()
y = load_labels()
X_eng = engineer_features(X)
X_scaled = StandardScaler().fit_transform(X_eng)
# Step 3: Ensemble training with cross-validation
models = [
RandomForestClassifier(n_estimators=100),
GradientBoostingClassifier(n_estimators=100)
]
scores = []
for model in models:
cv_scores = cross_val_score(model, X_scaled, y, cv=5)
scores.append(cv_scores.mean())
print(f"{{model.__class__.__name__}}: {{cv_scores.mean():.4f}}")
print(f"Best model score: {{max(scores):.4f}}")
"""
return code_template
def validate_and_refine (self, code ):
"""
Validate code syntax and refine if needed.
Uses AST parsing to catch errors before execution.
"""
try :
ast.parse(code)
return code, True
except SyntaxError as e:
print (f"Syntax error: {e} " )
fixed_code = self ._fix_syntax_error(code, e)
return self .validate_and_refine(fixed_code)
def _fix_syntax_error (self, code, error ):
"""Attempt to fix common syntax errors."""
return code
def score_plan_complexity (plan ):
"""
Score plan complexity 0-1 using professional rubric.
Considers: feature engineering scope, model selection, hyperparameter tuning.
"""
complexity_factors = {
'feature_engineering' : 0.3 ,
'model_selection' : 0.3 ,
'hyperparameter_tuning' : 0.2 ,
'ensemble_methods' : 0.2
}
total_score = 0.0
for factor, weight in complexity_factors.items():
if factor in plan.lower():
total_score += weight
return min (total_score, 1.0 )
Practical Guidance
Kaggle: Focus on recent competitions; weight by votes/popularity
Papers: Filter for methodology papers (not just benchmarks); include implementation details
Organization: Hierarchical categorization enables targeted retrieval
Update frequency: Monthly updates with new competition solutions
Max iterations: 8-12 (balance quality vs cost)
Branching factor: 2-3 new solutions per draft action
Metric threshold: Accept solution if metric > 0.7 of domain SOTA
Knowledge retrieval: TF-IDF indexes reduce LLM query overhead
Code validation: AST parsing catches errors before execution (saves rerun cost)
Model selection: Use cheaper model (o3-mini) for most steps; GPT-4o for complex planning
Structured tabular data problems (heterogeneous features, missing values)
Limited time budgets (20-hour leaderboard competitions)
Ensemble/stacking opportunities (AutoMind explores combinations)
Transfer learning scenarios (knowledge base accelerates solution discovery)
Reference
Agentic tree search: Combines exploration (drafting) with exploitation (improving best solutions)
Stochastic heuristic policy: Non-deterministic action selection enables diverse solution exploration
AST validation: Abstract Syntax Tree parsing catches errors without execution overhead
