validate-online

name	validate-online
description	Confirm an offline relevance win with an online A/B on CTR/conversion before declaring victory. Reach for this after an offline gain.

Skill: Validate online

Offline gains don't always transfer to user behavior (§3 #6).

Step 1 — Confirm the offline win

A clear NDCG/MRR improvement on the judgment list first (§3 #1).

Step 2 — Design the A/B

Control vs variant on a real metric (CTR/conversion), powered and bias-aware (§3 #6).

Step 3 — Read the online result

Did the offline gain move the user metric? It may not (§3 #6).

Step 4 — Ship only on confirmation

Promote the variant only if online confirms; otherwise keep digging (§3 #6).

Output

An online A/B result confirming (or refuting) the offline relevance gain. Traverse Tree 1 in the decision-trees file.