-
Confirm dataset type, input source, and mode — first determine the dataset type, then identify the source type, then explicitly ask the user to choose the import mode when multiple options exist. Do not silently pick any of these.
Dataset type resolution — ask the user which dataset type they want to create:
multi_modal — records with image URLs and/or video URLs plus text fields (e-commerce goods, short-video posts, content with thumbnails, etc.).user_event — user behavior / event logs (click, view, exposure, collect, etc.) for recommendation and personalization.
If the user's request clearly describes behavior logs / event data / recommendation data → user_event; if it clearly describes goods / content with images or video → multi_modal; if ambiguous, ask.
For multi_modal only — Theme resolution (mandatory) — the backend requires a valid Theme. Ask the user to pick one:
e_commerce — e-commerce products (with images, price, brand, tags)long_video — long-form video (movies, series; with cover image, language, category)content — general short-form content (posts, news articles with thumbnails, tags, categories)general — other / generic multi-modal (default)
If the user cannot decide, default to general. Record the chosen theme in a local variable and pass it to every subsequent command that accepts --theme.
Source identification:
- If the user provided a database connection or table name → MySQL.
- If the user provided a file path ending in
.jsonl or described a line-delimited/append-only file → JSONL.
- If the user provided a file path ending in
.json (JSON array) or .csv → JSON/CSV (one-time only).
- If the source is unclear, ask the user which source type they want to onboard from before proceeding.
Language: ask for language if the user has not already stated it (zh / en / ko / ja / hi); default to zh for Chinese-speaking users, en otherwise.
Import mode selection:
- For MySQL and JSONL, resolve whether the user wants one-time import or one-time + ongoing sync. Only skip the question when the request contains an explicit, unambiguous signal for one side (apply this detection to whatever language the user is writing in — English, Chinese, etc.):
- Explicit one-time: phrases carrying "once", "one-time", "snapshot only", "just this time", or equivalent single-import semantics.
- Explicit ongoing: phrases carrying "sync", "keep in sync", "auto-import", "scheduled", "incremental", "keep updated", or equivalent recurring-sync semantics.
- If the request is neutral — e.g. "import this file", "import this data", bare "import", mentions only a file path with an import verb but says nothing about scheduling/increment/once — you MUST ask the user to choose. The bare import verb is NOT a one-time signal; it is ambiguous. Never silently default to one-time.
- For JSON (array) and CSV, only one-time import is supported. No question needed.
After the dataset type, source type, theme (if multi_modal), language, and mode are confirmed, follow the matching branch:
- MySQL — one-time import: identify the table name, infer dataset name and primary key, and require explicit confirmation before any real write. Continue at step 2.
- MySQL — ongoing sync: same as above, plus the user must explicitly confirm the incremental cursor field itself. After step 10 continue at step 11.
- MySQL — existing dataset — ongoing sync: validate the dataset with
vs dataset get --id <DatasetId> --full, confirm the source config (especially the incremental cursor field), then jump directly to step 11.
- JSONL file — one-time import: confirm the file path. Continue at step 2.
- JSONL file — ongoing sync: confirm the file path. You MUST also interactively ask the user to confirm that new records will only be appended to the end of the file (append-only). Present the constraint clearly — sync only supports files that grow by adding new lines; edits or deletions of existing lines are not tracked and may cause duplicate or missing records. Wait for explicit user confirmation before proceeding. After step 10 continue at step 11.
- JSON (array) or CSV file — one-time import: confirm the file path. These formats are one-time import only; ongoing sync is not supported because they do not provide a stable append-only cursor. Convert the input to JSONL (one JSON object per line) before continuing. Continue at step 2.
- Existing dataset + one-time source import: not supported as a single workflow. Explain that the current CLI split supports either source export → new dataset onboarding for a one-time import, or background sync for ongoing updates, then let the user choose which branch to switch to.
Source environment configuration (applies to MySQL branches only; local files require no credentials):
- MySQL uses these environment variables by default:
MYSQL_HOST, MYSQL_PORT, MYSQL_USER, MYSQL_PASSWORD, MYSQL_DATABASE (optional: MYSQL_CHARSET).
- Render a bash export template snippet with placeholder values (for example
MYSQL_PASSWORD=your_password) and ask the user to fill in real values in their own terminal or shell session, then run export on each variable.
- Never display actual database credential values in chat. Never ask the user to paste or submit database credentials into the chat dialog.
- Never list "connection config" / "连接配置" blocks with concrete host/user/password values inside the chat. The only allowed format is a bash template with placeholder values.
- The export, init, and run commands read MySQL credentials only from environment variables. They do not accept credentials via flags or chat input.
- This is a human checkpoint. Wait for explicit confirmation that the local source environment is configured before proceeding.
-
Export source snapshot to JSONL — run vs connector export for the selected source to produce a bootstrap JSONL file:
- MySQL:
vs connector export --source mysql --source-table <table> --id-field <field> --cursor-field <field> [other flags]
- Local file:
vs connector export --source jsonl --file <path/to/items.jsonl> [other flags] (convert JSON arrays or CSV to JSONL first if needed)
The bootstrap file is always written to /tmp/viking/connector/<job>/bootstrap/items.jsonl. Do not use --output to try to override that path; --output only redirects the rendered command result. After export, use the emitted items.jsonl as the input file and continue at step 3.
-
Get upload URL — vs dataset import-url --file-name <basename>. Capture Result.FileUrl and Result.FileKey. Keep FileKey for step 5.
-
PUT upload — upload the raw item file to FileUrl (e.g. curl -X PUT --data-binary "@<local-path>" "<FileUrl>"). Expect HTTP 200 with empty body. Do not add an Authorization header — FileUrl is already presigned.
-
Submit inference task — vs dataset infer-schema --tos-key <FileKey> --type <multi_modal|user_event> --theme <general|e_commerce|content|long_video> --language <lang> --name <dataset-name>. For user_event, omit --theme. For multi_modal, --theme is required (default general). Theme values accept alias normalization: ecommerce/e-commerce → e_commerce, long-video/longvideo → long_video, common/default → general. Capture Result.TaskId.
-
Poll inference result + persist locally — vs dataset infer-result --task-id <TaskId> until Result.Status === "Success" (poll roughly every 5s, max ~3 minutes). Then write Result verbatim to a workspace-relative artifact file so the rest of the workflow can read from it.
Plan directory rules (important):
- Must write to the workspace-relative path:
./.viking/item-plans/<dataset-name>/infer-result.json (i.e. <cwd>/.viking/item-plans/<dataset-name>/...).
- Forbidden to write anywhere under
~/.viking/ (i.e. $HOME/.viking/). ~/.viking/ is the vs CLI's private config / credentials directory (config.json, credentials.json.enc), not a plan dir. Many agent hosts place ~/ outside the sandbox, so writes there fail with EPERM: operation not permitted; even when they succeed, your plan files end up mixed with the CLI's private files.
- If the workspace root is not writable (e.g. the sandbox only allows temp dirs), fallback priority is
${WORKSPACE_DIR}/.viking/item-plans/<dataset-name>/ → ${TMPDIR}/viking-item-plans/<dataset-name>/ → ./viking-item-plans/<dataset-name>/. Never redirect to the home directory ~/.viking/.
- Once the plan dir is decided, store it in a local variable (e.g.
WORK) and reuse the same path across steps 8/9/10/13. Do not switch plan dirs between steps.
This single artifact is the source-of-truth for every subsequent step. Do not regenerate it; do not edit BizAttr (those drive PK / title / URL detection on the backend). If the user requests semantic edits (e.g. tweak a FieldDescMap description, reorder IndexFields), edit this file in place and reuse it.
-
Schema Confirmation (mandatory) — show the persisted artifact to the user using the CLI's deterministic renderer, then surface it verbatim. (Historically called "Stage A".)
vs dataset validate-schema --input ./.viking/item-plans/<dataset-name>/infer-result.json --dataset-type <multi_modal|user_event>
The CLI emits a fixed block (Metadata / Fields / Field Roles / Warnings for multi_modal; Metadata / Fields / Warnings for user_event) wrapped between <!-- vs-schema-confirm: BEGIN --> and <!-- vs-schema-confirm: END --> markers. It uses a real markdown table for fields (with backticked types like `array<string>` so chat UIs do not eat the angle brackets), and fenced code blocks for the other sections. The output tolerates Name/FieldName, Type/FieldType, missing Required/BizAttr/Description, and missing or incomplete DataFieldConfig. The output is byte-stable: re-running the same file with the same --dataset-type always produces identical bytes.
Your message to the user MUST be exactly this template (BEGIN/END markers included, three parts only):
Dataset <Name> · type=<multi_modal|user_event> · <theme=<Theme> if multi_modal>
<verbatim CLI stdout from the BEGIN marker through the END marker, character-for-character>
<one-line confirmation prompt, written in the user's language — see Language Matching above and the templates below>
Confirmation prompt — pick the template matching the user's most recent message language. Do not paste the English template verbatim if the user is writing in Chinese.
- 中文（用户说中文时使用，默认）：
以上是 Schema 确认块。回复 \yes` 继续，或说明需要调整的字段（例如：把 `description` 加入文本检索字段、把 `brand` 加入 SuggestFields）。`
- English (when the user is writing in English):
This is the Schema Confirmation block. Reply \yes` to continue, or describe which fields to adjust (e.g. "make `description` searchable", "add `brand` to SuggestFields").`
- 日本語 / その他言語：translate the same intent, keep the token
`yes` verbatim and keep field names / JSON keys (description, SuggestFields, ...) in English.
You MUST:
- Copy the CLI stdout between (and including) the
<!-- vs-schema-confirm: BEGIN --> and markers character-for-character.
-
Behavior type confirmation (user_event only) — for multi_modal datasets, skip this step entirely and go straight to step 9.
For user_event datasets, the event_type field requires an EnumerateMeta array that maps every distinct raw event value found in the data to a standard behavior type (EnumerateBizAttr). Every distinct event_type value present in the data MUST have a corresponding entry in EnumerateMeta (no blanks, no unbound values). Additionally, the backend requires at least one entry mapped to exposure (Required: true) and at least one non-exposure positive behavior. Without this the create call fails validation.
Every distinct event_type value present in the data MUST have a confirmed mapping before proceeding. The agent infers a best-guess mapping semantically, presents it to the user with a standard-type reference labeled in the user's language, and only proceeds after explicit confirmation.
Internal standard types reference (agent uses this to convert user-confirmed labels to EnumerateBizAttr codes when serializing the payload):
| 中文标签 | English label | 日本語ラベル | 한국어 라벨 | हिन्दी लेबल | EnumerateBizAttr (code) | Name handling |
|---|
| 曝光 | Exposure / Impression | 露出 / インプレッション | 노출 | इम्प्रेशन / दिखना | exposure | auto — use standard label |
| 点击 | Click | クリック | 클릭 | क्लिक | click | auto — use standard label |
| 收藏 | Collect / Favorite / Save | お気に入り / 保存 | 저장 / 즐겨찾기 | सेव / पसंद | collect | auto — use standard label |
| 分享 | Share | シェア | 공유 | शेयर | share | auto — use standard label |
| 点赞 | Like / Thumbs-up | いいね | 좋아요 | लाइक |
-
Dry-run create — build dataset-create.json directly from the persisted artifact: copy Schema as-is (do not flip IsPK; the backend derives PK from BizAttr), copy DataFieldConfig.FieldDescMap as FieldDescMap, fill in Name / Type / Language / Description, then for multi_modal also set Theme and optionally ProcessConfig. For user_event, the event_type Schema entry must include the confirmed EnumerateMeta array from step 8. Set DryRun: true. Run vs dataset create --data @dataset-create.json --dry-run. Surface any validation errors and pause for correction.
For multi_modal — standard payload shape:
{
"Name": "<dataset-name>",
"Type": "multi_modal",
"Description": "<one-line description>",
"Language": "zh",
"Theme": "<general|e_commerce|content|long_video>",
"Schema": <copy from infer-result.json Schema>,
"FieldDescMap": <copy from infer-result.json DataFieldConfig.FieldDescMap>
}
For user_event — omit Theme and . The field in MUST include the confirmed array from step 8:
-
Real create — re-run step 9 without DryRun. Capture Result.Dataset.Id as DatasetId and persist it next to the artifact (e.g. ./.viking/item-plans/<dataset-name>/dataset.json).
-
Write data — vs data write --dataset-id <DatasetId> --fields @/tmp/viking/connector/<job>/bootstrap/items.jsonl to push the records from the bootstrap JSONL file. Expect a request_id in the response.
-
(Ongoing sync mode only) Start background incremental sync — run vs connector init --name <job> --source <mysql|jsonl> --dataset-id <DatasetId> ... to persist the job config, then vs connector run --job <job> --daemon to start the background sync. For MySQL, pass --source-table, --id-field, --cursor-field; for local files, pass --file <path>. In the hand-off, surface job, pid, trace.ndjson, imported-records.log, vs connector status --job <job>, and vs connector stop --job <job>. Skip this step for one-time import workflows.
-
Optional: create application — only if the user explicitly asks for app-level setup: vs app create --name <app-name> --description "<text>" --industry <alias> --language <lang>. Capture Result.Application.Id as AppId.
-
Optional: attach dataset — read DataFieldConfig straight from the persisted artifact and assemble:
{
"ApplicationId": "<AppId>",
"DatasetId": "<DatasetId>",
"DataConfig": <copy from infer-result.json DataFieldConfig>
}
Then call vs app attach-dataset --data @attach.json. Empty Result means success. This is the moment where the IndexFields/FilterFields/etc. captured in step 6 are actually applied — never reinvent these arrays from the schema; always pull them from the persisted artifact.
-
Hand-off — print console links + readiness reminder (mandatory). After the last successful step (data write, background sync start, or attach when the app branch ran), the agent must render a short summary block telling the user (a) where to monitor readiness in the console, and (b) that runtime APIs (search, chat, recommend) can only be exercised once readiness reports OK. Pick the console host from the active profile's baseUrl / controlPlaneBaseUrl, and assemble URLs using these exact path templates (do not invent other paths like /dataset/detail/<id> or /application/detail/<id> — those are wrong):
- Host contains
volcengineapi.com / volces.com → Volc Engine, base = https://console.volcengine.com/aisearch/platform/region:aisearch-platform+<region>. <region> is the active profile region (e.g. cn-beijing).
- Dataset URL:
<base>/home/dataset/<DatasetId>
- App URL:
<base>/app/<AppId>
- Host contains
byteplus.com → BytePlus, base = https://console.byteplus.com/aisearch/region:aisearch+ap-southeast-1 (BytePlus today only exposes the ap-southeast-1 region; do not fabricate other regions).
- Dataset URL:
<base>/home/dataset/<DatasetId>
- App URL:
<base>/app/<AppId>
Print the URLs only for the resources that actually exist in this run (dataset is always present; app/attach are only present if the user opted in). Render the prose lines (✓ markers, readiness reminder, runtime-API tip) in the user's current language per the Language Matching rule; keep IDs and URLs verbatim.
Template (translate the labels per the table below; keep DatasetId=..., AppId=..., URLs, and vs ... commands verbatim):
✓ <DATASET_LABEL>: DatasetId=<DatasetId>
<LINK_LABEL>: <dataset console URL>
✓ <APP_LABEL>: AppId=<AppId> # only when the app branch ran
<LINK_LABEL>: <app console URL> # only when the app branch ran
✓ <SYNC_LABEL>: job=<job> pid=<pid> # only when source-backed sync mode ran
<TRACE_LABEL>: <trace path>
<LOG_LABEL>: <import log path>
<STATUS_CMD>: vs connector status --job <job>
<STOP_CMD>: vs connector stop --job <job>
<READINESS_NOTE>
<RUNTIME_NOTE>
