vs-search-tuning

Use when a user asks an agent to evaluate or tune text search similarity for an existing Viking AI Search application and dataset.

Aller à l'installation

Dépôt: volcengine/SearchCLI
Dernière activité de la source: 11 août 2026 à 14:27
Langue détectée de SKILL.md: anglais
Étoiles: 1 176
Forks: 32

Options d'installation

Le prompt qui vérifie d'abord la source est sélectionné par défaut. Vous pouvez passer à une commande directe ou télécharger une copie locale.

Installer avec Codex ou Claude Copiez ce prompt, collez-le dans Codex, Claude ou un autre assistant, puis laissez-le vérifier la page du skill et l'installer pour vous.

Une commande directe contourne le prompt de vérification. Examinez la source avant de l'exécuter.

npx skills add https://github.com/volcengine/SearchCLI --skill vs-search-tuning

La commande reste sur une seule ligne. Faites défiler horizontalement pour la vérifier avant de la copier.

Vous préférez une copie locale ? Téléchargez les fichiers actuellement disponibles dans SkillsMP.

Vérifiez les fichiers source

Lisez SKILL.md et les fichiers associés affichés par SkillsMP avant de décider de l'installer.

SKILL.md

Instructions source · Aperçu en lecture seule

Retour à la recherche Retour à l’installation

Ask the user whether they have a tuning query set. Good sources include online search logs, customer support query collections, or a manually curated representative set. If the user has one, use it with --queries <file>. If not, say the CLI will generate synthetic queries from dataset samples and that those queries should be reviewed.
Check the local environment:
- vs auth status --json
- vs doctor --json
- vs search tune llm-check --json If LLM check fails and LLM query generation or LLM judging is needed, configure LLM first:
- interactive secure setup: vs llm login
- from existing terminal env: vs llm import-env
- verify secret source: vs llm status --json
Check that the application is ready:
- vs app status --application-id <id> --json
Confirm the tuning boundary with the user:
- text query similarity tuning
- similarity-only profile
- fixed mode=UserDefined
- tunes only user_defined_recall_mode, dense_weight, text_weight, query_keyword_match_percent, and max_retrieved_num
- LLM judging defaults to --judge-input text; use --judge-input text-image only when the user asks for image-aware relevance or the domain is strongly visual and image quality/content should affect relevance
- no rerank, personalization, hotness, boost/bury, sort rules, serving controls, or business operating rules
If the user has no query set, generate one first:
- vs search tune query-generate --application-id <id> --dataset-id <dataset> --query-count 100 --sample-size 200 --query-batch-size 10 --llm-concurrency 100 --timeout-ms 120000 --json
- if synthetic queries should only use configured text retrievable fields: add --retrievable-field-only, which reads GetAppDataConfig.DataConfig.IndexFields and excludes ImageIndexFields Show the returned sampleQueries, typeCounts, requestedQueryCount, actualQueryCount, shortfall, and warnings to the user. If ok=false, do not continue to plan or run; retry with larger timeout/sample size or ask for a real query set. Use the returned queryFile only after the user accepts the query set for first-pass tuning.
Validate the accepted query set before planning:
- vs search tune validate --queries <file> --json Summarize ok, validQueryCount, duplicateIdCount, sourceItemQueryCoverage, labelSourceRecommendation, and any blocking problems. If ok=false, fix or regenerate the query set before continuing. When a user provides --queries <file>, search tune plan and search tune run evaluate the whole file by default; pass --query-count <n> only when the user explicitly wants a smaller prefix.
Run a plan before any expensive evaluation:
- with user queries: vs search tune plan --application-id <id> --dataset-id <dataset> --queries <file> --profile similarity-only --json
- with generated queries: use the queryFile returned by query-generate Summarize the estimated search requests, max pointwise LLM judgements, source-item coverage, suggested first-pass size, warnings, and parameter coverage.
Run tuning only after the plan is acceptable:
- fast first pass when the query file has enough sourceItemIds: vs search tune run --application-id <id> --dataset-id <dataset> --queries <file> --profile similarity-only --label-source source-item --search-concurrency 18 --timeout-ms 120000 --json
- LLM judgement run: vs search tune run --application-id <id> --dataset-id <dataset> --queries <file> --profile similarity-only --label-source llm --search-concurrency 18 --llm-concurrency 100 --llm-retries 1 --max-label-failure-rate 0.01 --timeout-ms 120000
- image-aware LLM judgement run, only when visual relevance is needed: vs search tune run --application-id <id> --dataset-id <dataset> --queries <file> --profile similarity-only --label-source llm --judge-input text-image --max-judge-images 1 --search-concurrency 18 --llm-concurrency 100 --timeout-ms 120000
- with generated queries: use the queryFile returned by query-generate Use the command form above for first-pass tuning unless the user explicitly asks for a different evaluation scope. Search requests default to 18-way concurrency, and LLM judgements default to 100-way concurrency. LLM judging runs as a worker pool, so completed labels are checkpointed while slower LLM requests continue in their own worker slots.
While a run is active, use the artifact paths from progress output if troubleshooting is needed:
- run-state.json: current status, completed searches, labels, and resume metadata
- partial-metrics.json: partial metrics from completed query/strategy pairs
- performance-summary.json: elapsed time, search/LLM wall time, average and percentile latency, throughput, cache hits, label failures, and configured concurrency
- rankings.jsonl, labels-used.jsonl, and label-failures.jsonl: completed rankings, labels used by the run, and tolerated/diagnostic label failures If the process is interrupted, resume with vs search tune run --application-id <id> --resume-run-id <run-id>.
Read and summarize the generated report:

Before executing any concrete vs ... command in this tuning workflow, first consult vs-product-qa to verify the current command surface, required flags, payload fields, input format, and allowed values. Only after that check may you finalize parameters and run the command.
Do not run tuning before asking the user whether they have a query set.
Do not run tuning before search tune validate has checked the accepted query file, unless the user explicitly asks to skip validation.
Do not run tuning before search tune plan has been shown and summarized.
Do not let search tune run auto-generate queries during agent-led tuning. If the user has no query set, run search tune query-generate, show query samples, and then pass the generated queryFile to plan and run.
Do not continue from a generated query set when query-generate returns ok=false; inspect warnings and retry generation before asking the user.
Do not run LLM query generation or LLM judging until search tune llm-check succeeds. A --label-source source-item run may skip LLM judging only when the query file already contains usable sourceItemIds.
Do not ask the user to paste LLM API keys into chat. Use vs llm login in a real terminal, or ask the user to set VIKING_LLM_BASE_URL, VIKING_LLM_API_KEY, and VIKING_LLM_MODEL in that terminal and then run vs llm import-env.
Do not present the recommendation as an online change. search tune apply creates a new candidate scene only; it does not switch the default entrance.
If a tuning process is interrupted, prefer --resume-run-id over starting a duplicate run with the same query set and strategy space.
Do not tune or attribute changes to rerank, personalization, hotness, boost/bury, sort rules, serving controls, or business rules in this first-version workflow.
Do not enable --judge-input text-image by default. Use it only when image relevance is part of the user's evaluation goal; if GetAppDataConfig.ImageIndexFields is empty, stay with text judging and tell the user image-aware judging is unavailable for that app+dataset config.
Do not create, update, publish, or switch search scenes as a fallback for failed automatic tuning. Only use search tune apply after a completed report and explicit user approval.
Do not call a result "optimal" or "best" unless a completed search tune run report exists. If the report used --label-source source-item, call it a fast source-item silver-label recommendation and explain that LLM or human labels can be used for higher-confidence validation.
Do not delete or prune .viking/search-tuning artifacts unless the user explicitly asks.
If search tune run generates queries automatically, tell the user the query set is synthetic and should be reviewed for high-risk usage.
If a tuning failure or user follow-up turns into a product concept, capability, API field, console UI path, purchase, billing, or general troubleshooting question outside this tuning workflow, temporarily hand off to vs-product-qa; return to this workflow only after the grounded product answer is complete.

name	vs-search-tuning
description	Use when a user asks an agent to evaluate or tune text search similarity for an existing Viking AI Search application and dataset.
category	search
applies_to	codex, agents, external-agent
requires_cli	>=0.1.0
keywords	search tuning, search evaluation, llm judge, ndcg, query generation, similarity tuning
commands	llm login, llm import-env, llm status, search tune llm-check, search tune validate, search tune query-generate, search tune plan, search tune run, search tune report, search tune compare, search tune apply, app status, doctor

name	vs-search-tuning
description	Use when a user asks an agent to evaluate or tune text search similarity for an existing Viking AI Search application and dataset.
category	search
applies_to	codex, agents, external-agent
requires_cli	>=0.1.0
keywords	search tuning, search evaluation, llm judge, ndcg, query generation, similarity tuning
commands	llm login, llm import-env, llm status, search tune llm-check, search tune validate, search tune query-generate, search tune plan, search tune run, search tune report, search tune compare, search tune apply, app status, doctor

vs-search-tuning

Options d'installation

Vérifiez les fichiers source

Viking Search Tuning

When to Use

Viking Search Tuning

When to Use

Preconditions

Commands

Workflow

Customer Environment Principle

Constraints

vs-search-tuning

Informations de source

Options d'installation

Vérifiez les fichiers source

Plus depuis ce dépôt

Plus depuis ce dépôt

Viking Search Tuning

When to Use

Viking Search Tuning

When to Use

Preconditions

Commands

Workflow

Customer Environment Principle

Constraints