compileiq-validate-result

name	compileiq-validate-result
description	Use AFTER a Search has completed and BEFORE claiming any speedup or shipping an ACF. Loads the dump_results CSV, extracts top-K candidates (single-objective) or the Pareto front (multi-objective), re-measures each against the no-ACF baseline with 100+ trials on fresh caches, runs Welch's t-test plus Cohen's d, rejects three classic false-positive patterns (lucky-min / higher-variance / multiple-comparisons-of-N), and saves the validated winner as best.acf. Triggers on "validate result", "extract best config", "Welch's t-test", "is my speedup real", "save best ACF", "pareto front", "claim speedup", "ship config".
when_to_use	- tuner.start() returned and there's a results CSV. - User wants to ship an ACF to production. - Reported speedup feels too good to be true. Don't use when: - Search hasn't completed (and user hasn't decided to stop early). - User just wants the raw best row — that's one line of pandas.
license	Apache-2.0
metadata	{"version":"1.0.0","author":"NVIDIA CompileIQ","domain":"compiler-optimization"}
allowed-tools	Bash Read
paths	["**/*.csv","**/*.acf","**/*.py"]

compileiq-validate-result

The score CompileIQ reports during a search uses N=5-15 trials per evaluation and a shared cache. That's appropriate for the search loop but wildly insufficient for shipping. This skill is the gate before any ACF goes to production.

When

• tuner.start() has returned and there's a dump_results= CSV on disk.
• User wants to claim a speedup or ship an ACF.
• A reported speedup feels too clean — validate it.

Steps

1. Load the CSV

from compileiq.results import SearchResult

results = SearchResult.from_csv("results.csv", problem_type="min", clear_duplicates=True)
df = results.get_results()
print(f"{len(df)} evaluations across {df['generation'].max()+1} generations")

2. Extract candidates

Single-objective:

best = results.get_best_result()
# dict: {metadata, generation, score_1, params, [norm_score_1]}
score = best.get("score_1", best.get("score"))   # legacy defensiveness
acf_hex = best["params"]                          # hex string

score_1 (with underscore-one) is the canonical key — it matches the multi-objective convention score_N. Older code sometimes uses plain score; the fallback above handles both shapes.

For top-K:

import pandas as pd
df_valid = df[pd.to_numeric(df["score_1"], errors="coerce") < 1e10]
top_k = df_valid.nsmallest(5, "score_1")          # nlargest for MAX problems