execution-grounded-selection

Pick among candidate outputs (code, configs, plans) by running them on diverse inputs and clustering by behavioural fingerprint, rather than by textual aggregation or log-probability. Activates when an executor returns multiple plausible candidates that need disambiguation, when output-majority voting would be the default choice, or when reviewing generated code that has not yet been validated. The 2026 evidence (Semantic Voting, arxiv 2605.08680v1) is that any execution-based selector dominates output-majority voting by 19-52pp; sketch-generated inputs beat random fuzz by 11.3pp. Triggers: "pick the best candidate", "majority vote on code", "select from N samples", "validate the generated output", "behavioural verification".

Jump to install Review files

Repository: Tibsfox/gsd-skill-creator
Last source activity: June 4, 2026 at 20:18
Detected SKILL.md language: English
Stars: 70
Forks: 10

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Install with an AI assistant

Copy this prompt into the AI assistant you're using.

Follow https://skillsmp.com/skill-install/prompt.md to install only the Agent Skill at https://github.com/Tibsfox/gsd-skill-creator/tree/main/project-claude/skills/execution-grounded-selection.

A direct command skips the review prompt. Inspect the source before running it.

npx skills add https://github.com/Tibsfox/gsd-skill-creator --skill execution-grounded-selection

The command stays on one line. Scroll horizontally to inspect it before copying.

Prefer a local copy? Download the files currently available to SkillsMP.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

SKILL.md

Source instructions · Read-only preview

name: execution-grounded-selection
description: Pick among candidate outputs (code, configs, plans) by running them on diverse inputs and clustering by behavioural fingerprint, rather than by textual aggregation or log-probability. Activates when an executor returns multiple plausible candidates that need disambiguation, when output-majority voting would be the default choice, or when reviewing generated code that has not yet been validated. The 2026 evidence (Semantic Voting, arxiv 2605.08680v1) is that any execution-based selector dominates output-majority voting by 19-52pp; sketch-generated inputs beat random fuzz by 11.3pp. Triggers: "pick the best candidate", "majority vote on code", "select from N samples", "validate the generated output", "behavioural verification".
user-invocable: true
version: 1.0.0
format: 2025-10-02T00:00:00.000Z
triggers: ["pick the best candidate","majority vote on code","select from N samples","validate the generated output","behavioural verification of code"]
updated: 2026-05-16T00:00:00.000Z
status: ACTIVE
source: arxiv 2605.08680v1 (Semantic Voting), 2605.07248v1 (Plan-on-Trigger)

# Execution-Grounded Selection ## Why Output-majority voting (pick the most common string output) is dominated by *any* selector that actually runs the candidates. Semantic Voting (arxiv 2605.08680v1) shows 19-52pp improvement over output voting across multiple code benchmarks. The specific aggregation rule (majority, weighted, MBR-Exec) is statistically indistinguishable once execution evidence is present — *execution is the dominant signal, aggregation is the residual*. This is the code-domain analogue of the **noise-as-exploration** Rosetta concept: execution diversity is the exploration mechanism; behavioural fingerprint is the equilibrium signal. ## How The Semantic Voting pipeline: 1. **Sample N candidates** — generate at temperature > 0 (typical N = 5-10). 2. **Generate diverse inputs** — sketch-generated inputs (derived from candidate population structure) beat random fuzz by ~11pp. If sketch generation is infeasible, fall back to LLM-generated test inputs > random fuzz. 3. **Execute each candidate on each input** — collect (candidate_i, input_j, output_ij) tuples. Crash counts as a distinct fingerprint, not a discard. 4. **Cluster by fingerprint** — equivalence on `[output_ij for j in inputs]` defines the cluster. 5. **Pick the largest cluster** — break ties by candidate self-confidence or by Pareto on execution cost. ## When to skip - Generation is deterministic (temperature = 0) — there's nothing to select among. - Execution is expensive or has side effects (touches the network, modifies state) — fall back to dry-run / static analysis. - The output isn't executable (e.g., natural-language summary) — use paired-trace audit instead. ## Integration - `wrap:verify` and `gsd-verify-work` — pre-merge gate when verification produces multiple candidate fixes. - `gsd-code-fixer` — when the fixer proposes more than one fix per finding. - `code-review` — adds a "did you actually run it?" subsection to the review rubric. ## Cross-references - Rosetta concept #9 (Execution-Grounded Selection) — canonical definition - College: `agent-systems / agentic-code-generation / agent-execution-grounded-selection` - Related skills: `test-generator` (generates the inputs; pair with this skill for the full loop)

View on GitHub

Back to search Back to installation

goal-ambiguity-gate

Use the moment a request is underspecified and you are about to act on an ASSUMED goal. Enumerate the candidate goals the request could mean, project the action or artifact each one implies, and check whether they diverge: if two or more plausible goals lead to materially different actions or artifacts, ask exactly ONE targeted clarifying question; otherwise proceed on the most-likely goal and state the assumption in one line. Clarification is an evidence-producing action, not a delay — but one question, never an interrogation. Distinct from gsd-spec-phase (GSD-phase-bound, emits a heavy SPEC.md) and intent-router (fetch strategy, not goal disambiguation). Backed by agent goal-state inference (arxiv 2606.16813v1). Triggers on acting under an assumed goal when the request admits more than one materially different reading.

2026-07-19

memory-hubness-gate

Use at write time to vector/embedding memory — Grove content-addressed insertion, chroma/pgvector upserts, memory-consolidation promoting session traces to MEMORY.md, or embedding externally-ingested content. Scores each candidate record against a fixed panel of sentinel queries and quarantines any record that would become the nearest neighbor of too many unrelated queries — a hub — whether from adversarial poisoning or accidental over-generality. This is the memory-record-side sibling of skill-injection-guardian (file-side) and the write-side complement of memory-use-warrant (read-side). Quarantine, never silently drop; a human reviews. Backed by the admission-time hubness gate (arxiv 2606.19692v1). Triggers on inserting into vector memory, consolidating memory, and embedding stranger content.

2026-07-19

memory-use-warrant

Run this appropriateness check the moment you are about to integrate a retrieved long-term memory — a Grove content-addressed hit, a chroma/pgvector neighbour, a memory-consolidation digest, or a MEMORY.md line — into a response, especially anything touching private origins, Fox Companies IP, credentials, or Center Camp / consent-governed content. It answers a question intent-router never asks: not WHAT to fetch or HOW, but WHETHER a correctly-retrieved item should reach output. Relevance is not appropriateness — a perfect similarity match can still be a boundary violation. Default is FAIL-CLOSED: if in-context authorization is uncertain, the memory may inform behaviour but must not be surfaced. Backed by RBI-Eval (arxiv 2606.06055v1). Triggers on surfacing recalled sensitive memory into a response.

2026-07-19

View all from this repository

execution-grounded-selection

Source facts

Install options

Install with an AI assistant

Review the source files

More from this repository

Install with an AI assistant

More from this repository