mitigate-positional-bias

星标14

分支1

更新时间2026年5月14日 17:35

Strategies to prevent defaulting to Image B and ensure fair evaluation of both images.

安装

用 Codex 或 Claude 帮你安装复制这段 Prompt，粘贴到 Codex、Claude 或其他助手里，让它检查 Skill 页面并帮你完成安装。

来源

TIGER-AI-Lab

TIGER-AI-Lab/RewardHarness

下载

Mitigate Positional Bias

There is a strong observed bias towards predicting Image B as the winner, especially when both images have flaws or when the evaluation is difficult.

Evaluate A First: Always explicitly evaluate Image A's adherence to the instruction and visual quality before looking at Image B.
Fair Comparison: Do not overlook Image A's successes just because Image B looks slightly more polished. If Image A followed the instruction and Image B did not, Image A must win, regardless of minor artifacts.
Justify B over A: If you choose B, ensure you have a concrete, visually verified reason why A failed or was inferior. Do not use vague terms like "more natural" to justify B if A actually followed the instruction better.

同仓库更多 Skills

同仓库

Score completeness of instruction execution. Partial edits get partial credit; missed sub-steps cost points.

Guidance on penalizing artifacts while allowing conceptual unrealism if requested by the prompt.

Extracts and analyzes text within images to verify spelling, placement, and clarity.

Guidelines for evaluating background replacements and scene changes.

Guidelines for evaluating prompts with multiple distinct instructions.

Guidelines for evaluating object replacements and modifications.

description	Strategies to prevent defaulting to Image B and ensure fair evaluation of both images.
name	mitigate-positional-bias
type	skill

There is a strong observed bias towards predicting Image B as the winner, especially when both images have flaws or when the evaluation is difficult.

Evaluate A First: Always explicitly evaluate Image A's adherence to the instruction and visual quality before looking at Image B.
Fair Comparison: Do not overlook Image A's successes just because Image B looks slightly more polished. If Image A followed the instruction and Image B did not, Image A must win, regardless of minor artifacts.
Justify B over A: If you choose B, ensure you have a concrete, visually verified reason why A failed or was inferior. Do not use vague terms like "more natural" to justify B if A actually followed the instruction better.