mitigate-positional-bias

스타14

포크1

업데이트2026년 5월 14일 17:35

Strategies to prevent defaulting to Image B and ensure fair evaluation of both images.

설치

Codex 또는 Claude로 설치 이 Prompt를 복사해 Codex, Claude 또는 다른 어시스턴트에 붙여 넣으면 Skill 페이지를 검토하고 설치를 진행할 수 있습니다.

출처

TIGER-AI-Lab

TIGER-AI-Lab/RewardHarness

다운로드

Mitigate Positional Bias

There is a strong observed bias towards predicting Image B as the winner, especially when both images have flaws or when the evaluation is difficult.

Evaluate A First: Always explicitly evaluate Image A's adherence to the instruction and visual quality before looking at Image B.
Fair Comparison: Do not overlook Image A's successes just because Image B looks slightly more polished. If Image A followed the instruction and Image B did not, Image A must win, regardless of minor artifacts.
Justify B over A: If you choose B, ensure you have a concrete, visually verified reason why A failed or was inferior. Do not use vague terms like "more natural" to justify B if A actually followed the instruction better.

이 저장소의 다른 Skills

같은 저장소

Score completeness of instruction execution. Partial edits get partial credit; missed sub-steps cost points.

Guidance on penalizing artifacts while allowing conceptual unrealism if requested by the prompt.

Extracts and analyzes text within images to verify spelling, placement, and clarity.

Guidelines for evaluating background replacements and scene changes.

Guidelines for evaluating prompts with multiple distinct instructions.

Guidelines for evaluating object replacements and modifications.

description	Strategies to prevent defaulting to Image B and ensure fair evaluation of both images.
name	mitigate-positional-bias
type	skill

There is a strong observed bias towards predicting Image B as the winner, especially when both images have flaws or when the evaluation is difficult.

Evaluate A First: Always explicitly evaluate Image A's adherence to the instruction and visual quality before looking at Image B.
Fair Comparison: Do not overlook Image A's successes just because Image B looks slightly more polished. If Image A followed the instruction and Image B did not, Image A must win, regardless of minor artifacts.
Justify B over A: If you choose B, ensure you have a concrete, visually verified reason why A failed or was inferior. Do not use vague terms like "more natural" to justify B if A actually followed the instruction better.