菜单
Create principle-guided reward models for web automation agents that evaluate reasoning process quality rather than just outcomes. Implement domain-specific principles (HTML understanding, interaction patterns, state tracking) to guide agent behavior in web navigation and task completion.
默认使用会先检查来源的 Prompt；你也可以切换为直接命令，或下载本地副本。
决定是否安装前，请先阅读 SKILL.md，以及 SkillsMP 当前展示的配套文件。
用 Codex 或 Claude 帮你安装 复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。
直接命令不会经过审查 Prompt；运行前请先检查来源。
npx skills add https://github.com/ADu2021/skillXiv --skill web-agent-reward-model命令会保持在同一行。复制前请横向滚动并检查完整内容。
想先保存到本地？可下载 SkillsMP 当前能够提供的文件。
正在显示 SKILL.md
| name | web-agent-reward-model |
| title | WebArbiter: A Principle-Guided Reasoning Process Reward Model for Web Agents |
| version | 0.0.2 |
| engine | skillxiv-v0.0.2-claude-opus-4.6 |
| license | MIT |
| url | https://arxiv.org/abs/2601.21872 |
| keywords | ["web-agents","reward-model","process-evaluation","reasoning-guidance","principle-based"] |
| description | Create principle-guided reward models for web automation agents that evaluate reasoning process quality rather than just outcomes. Implement domain-specific principles (HTML understanding, interaction patterns, state tracking) to guide agent behavior in web navigation and task completion. |
Web automation agents struggle with task completion because they rely on sparse outcome rewards. Intermediate steps in web navigation (finding elements, understanding page structure, planning interaction sequences) are difficult to evaluate, leading to poor exploration and inefficient trajectories.
Build a principle-guided reward model that evaluates web agent reasoning using domain-specific principles. Rather than just rating success or failure, score intermediate reasoning quality based on:
Establish the core evaluation principles for web agent reasoning.
class WebPrincipleEvaluator:
"""Evaluate agent reasoning against web domain principles"""
def __init__(self):
self.principles = [
"interaction_validity",
"page_understanding",
"goal_alignment",
"state_consistency"
]
def evaluate_step(self, state, action, html_context):
"""Score a single agent step against principles"""
scores = {
"valid_interaction": .check_interaction_validity(action, html_context),
: .check_page_understanding(action, state),
: .check_goal_alignment(action, state),
: .check_state_consistency(state, action)
}
scores
Create validators for DOM interactions and page understanding.
def check_interaction_validity(action, html_doc):
"""
Validate that action targets exist and are interactable.
action: {"type": "click"|"type"|..., "target": selector}
html_doc: parsed HTML tree
"""
target = find_element(html_doc, action["target"])
if target is None:
return {"valid": False, "reason": "element_not_found"}
if action["type"] == "click" and not is_clickable(target):
return {"valid": False, "reason": "element_not_clickable"}
if action["type"] == "type" and not is_text_input(target):
return {"valid": False, "reason": "element_not_text_input"}
return {"valid": True, "confidence": 1.0}
def check_page_understanding(action, current_state):
"""
Verify agent understood page context when choosing action.
current_state: agent's internal state/memory
"""
understood = {
"has_page_title": bool(current_state.get("page_title")),
"has_form_fields": len(current_state.get("form_fields", [])) > 0,
"tracked_links": len(current_state.get("visible_links", [])) > 0,
}
return sum(understood.values()) / len(understood)
Evaluate entire trajectories using the principle scores.
def score_trajectory(trajectory, goal_description, html_snapshots):
"""
Compute principle-guided reward for entire trajectory.
trajectory: list of (state, action, result) tuples
"""
step_scores = []
for i, (state, action, result) in enumerate(trajectory):
html = html_snapshots[i]
step_score = {
"interaction_quality": check_interaction_validity(action, html),
"understanding_quality": check_page_understanding(action, state),
"alignment_with_goal": check_goal_alignment(action, goal_description, state),
"state_tracking": check_state_consistency(state, action)
}
step_scores.append(step_score)
# Aggregate: penalize invalid interactions, reward goal-aligned steps
trajectory_reward = aggregate_principle_scores(
step_scores,
weights={
"interaction": 0.3,
"understanding": 0.25,
"alignment": 0.35,
"consistency": 0.1
}
)
return trajectory_reward, step_scores
Use principle-guided rewards to update agent policy.
def compute_reinforcement_signal(trajectory, outcome_reward, principle_scores, alpha=0.4):
"""
Combine outcome reward with principle-based process reward.
alpha: weight for process reward vs outcome (0.4 = 40% process, 60% outcome)
"""
principle_reward = principle_scores["trajectory_reward"]
# Blend signals: prioritize outcome but use principles for refinement
blended_reward = (
(1 - alpha) * outcome_reward +
alpha * principle_reward
)
return blended_reward
Capture and explain agent decisions for better interpretability.
def log_reasoning_trace(trajectory, principle_evaluations):
"""
Create interpretable trace of agent's reasoning with principle evaluations
"""
trace = {
"steps": [],
"principle_violations": [],
"successful_patterns": []
}
for i, evals in enumerate(principle_evaluations):
step_trace = {
"step_num": i,
"principle_scores": evals,
"reasoning_quality": sum(evals.values()) / len(evals),
}
# Track violations for learning
for principle, score in evals.items():
if score < 0.5:
trace["principle_violations"].append({
"step": i,
"principle": principle,
"severity": 1 - score
})
trace["steps"].append(step_trace)
return trace
基于 SOC 职业分类
