mitigate-positional-bias

スター14

フォーク1

更新日2026年5月14日 17:35

Strategies to prevent defaulting to Image B and ensure fair evaluation of both images.

インストール

Codex または Claude でインストールこの Prompt をコピーして Codex、Claude、または他のアシスタントに貼り付けると、Skill ページを確認してインストールできます。

ソース

TIGER-AI-Lab

TIGER-AI-Lab/RewardHarness

ダウンロード

Mitigate Positional Bias

There is a strong observed bias towards predicting Image B as the winner, especially when both images have flaws or when the evaluation is difficult.

Evaluate A First: Always explicitly evaluate Image A's adherence to the instruction and visual quality before looking at Image B.
Fair Comparison: Do not overlook Image A's successes just because Image B looks slightly more polished. If Image A followed the instruction and Image B did not, Image A must win, regardless of minor artifacts.
Justify B over A: If you choose B, ensure you have a concrete, visually verified reason why A failed or was inferior. Do not use vague terms like "more natural" to justify B if A actually followed the instruction better.

このリポジトリの他の Skills

同じリポジトリ

Score completeness of instruction execution. Partial edits get partial credit; missed sub-steps cost points.

Guidance on penalizing artifacts while allowing conceptual unrealism if requested by the prompt.

Extracts and analyzes text within images to verify spelling, placement, and clarity.

Guidelines for evaluating background replacements and scene changes.

Guidelines for evaluating prompts with multiple distinct instructions.

Guidelines for evaluating object replacements and modifications.

description	Strategies to prevent defaulting to Image B and ensure fair evaluation of both images.
name	mitigate-positional-bias
type	skill

There is a strong observed bias towards predicting Image B as the winner, especially when both images have flaws or when the evaluation is difficult.

Evaluate A First: Always explicitly evaluate Image A's adherence to the instruction and visual quality before looking at Image B.
Fair Comparison: Do not overlook Image A's successes just because Image B looks slightly more polished. If Image A followed the instruction and Image B did not, Image A must win, regardless of minor artifacts.
Justify B over A: If you choose B, ensure you have a concrete, visually verified reason why A failed or was inferior. Do not use vague terms like "more natural" to justify B if A actually followed the instruction better.