spryker-import-tools

name

spryker-import-tools

description

Use whenever manipulating or validating a Spryker data-import CSV or import manifest. csv.php: read/inspect/filter/delete/duplicate/set/replace/scale/apply-translations — reliable where shell tools (cut/awk/sed) corrupt multi-line quoted fields. validate.php: preflight, refs (incl. composite tuples), required, unique, absent, paths, product-refs, manifest-diff, known-set — catches boot-aborting data statically in seconds. Concept-free: you supply columns/values/patterns; the tools do the mechanics and return compact JSON. Also home of the authoritative invocation & command-discipline rules every consumer skill follows.

spryker-import-tools

Two self-contained PHP scripts under scripts/, both concept-free (they know nothing about stores/locales/currencies — you decide what to do; they do it correctly and print JSON):

• csv.php — RFC-4180 CSV manipulation.
• validate.php — pre-boot consistency checks (its own CSV reader; no dependency on csv.php).

Why use them instead of shell tools or editing by hand: Spryker CSVs contain multi-line quoted fields (e.g. cms_page.csv), so cut/awk/sed corrupt them; and a single bad import row aborts the whole 30–60 min install, which these checks catch statically in seconds. Bulk work in-context burns tokens — the scripts return a terse verdict instead.

Run

php <path>/scripts/csv.php <command> <file> [options] and php <path>/scripts/validate.php <check> [options] (host php, or docker run --rm -v "$PWD":/app -w /app php:8.3-cli php … if there's no host php). Both print JSON ({status, …}, exit 0/1/2); read-type commands take --plain for clean lines. Prefer reading the output / the exit code directly over piping through python/ruby/jq — it's unnecessary here and prompts; the direct read (or --quiet + exit code for validate) is the quieter path.

Invocation & command discipline (AUTHORITATIVE — the one copy; consumer skills summarize and point here)

These rules keep an unattended run quiet (no needless permission prompts) and safe. Every skill that drives these tools follows them.

Locate the scripts: .claude/skills/spryker-import-tools/scripts/csv.php and …/scripts/validate.php (setup install, relative to the project cwd), or ${CLAUDE_PLUGIN_ROOT}/skills/spryker-import-tools/scripts/… (plugin install); docker php:8.3-cli fallback per the Run section if there is no host php.

Invoke by the LITERAL path from the project cwd — never cd, never a shell variable. $CSV/$VALIDATE in the consumer skills are shorthand for that literal path — resolve it once in reasoning and substitute it inline in every call. A cd prefix, an assignment prefix (CSV=… php …), or a multi-line assign-then-use all miss the allowlist and prompt on every single call.

One simple command per Bash call — no shell operators. cd, ;, |, &&, ||, $(…), backticks, for/while, redirects, env-prefixes and multi-line commands can never be allowlisted — they prompt regardless. Instead:

• One op over many files = ONE command: list all files before the flags and add --in-place (set/replace/scale/drop-columns/filter/delete all take many files); inspect many with one columns/distinct/count <f1> <f2> … --plain. Never a shell loop.
• Count with the tool — rowCount/matchedRows from the JSON, or csv count --plain; never | grep -c/| wc -l.
• Explore with the built-in Read / Grep / Glob tools — never bash ls/find/grep/cat/head/sed, and never bundle inspections with ; echo …. The built-in tools are not Bash, so they never prompt.
• Config/YAML edits use the built-in Edit tool (anchored, format-preserving) — never regenerate a file with python/ruby/sed.
• rm/rm -rf prompt by design (not allowlisted) — surface every deletion as an explicit step; don't force it.

Destructive-operation gate — explain, then ask, every time. csv filter/delete --in-place and header-only truncation rewrite/empty files without prompting on their own, and docker/sdk reset/clean-data can wipe DB + search volumes (git does NOT restore those). Before any in-place data removal/truncation or any DB/volume drop: preview it (filter/delete with no --out/--in-place reports matchedRows without writing), tell the developer in ONE plain line what will be destroyed, and get an explicit go-ahead — even if the allowlist would let it through silently. (reset/clean-data weight + the hand-to-developer escape hatch: boot-and-verify §3b.)

---

csv.php — CSV operations

Inspection (add --plain for line output — no JSON to parse):

• read <file> [--limit N] — dump header + rows as JSON. Takes ONE file — passing several is a clean error (no silent truncation to the first; use columns/count for many-file inspection).
• count <file> [<file2> …] [--plain] — data-row count per file, many files in ONE call. --plain → rowCount⇥file per line. Use this for "how many rows across these N files" — never a shell for … do … done loop with | grep rowCount; one count call replaces the whole loop.
• columns <file> [<file2> …] [--plain] — header + row count. Accepts many files in one call (per-file output; a missing file is reported inline, batch survives). --plain → column names per line, with a == <file> == delimiter when multiple.
• distinct <file>… --column C [--plain] — distinct values of a column with row counts, sorted by count desc (--plain → count⇥value per line); takes many files in one call (aggregated). Use this to inspect data — never awk/sort/uniq -c (they corrupt multi-line quoted fields).

Mutation — write with --out <file> for ONE file, or --in-place for one-or-many files. Batch: list several files before the flags and add --in-place to apply the same op to all in a single command (per-file report in files[]) — this is how you avoid a shell for loop. Multiple files without --in-place is an error.

• filter <file>... (--where col=val … [--match exact|prefix|contains] | --in col=v1,v2,… | --in-file col=path) [--out f | --in-place] — keep matching rows. --where repeatable (AND); --in/--in-file = set-membership. Without a target it previews: matchedRows/resultRows are the FULL counts, the echoed rows are capped at 20 (--limit N to raise) — the counts are the destructive-gate evidence, not the row dump.
• delete <file>... (same conditions) [--out f | --in-place] — drop matching rows.
• duplicate-columns <file>... --from en_US --to fr_CA[,de_DE,…] [--skip-base url] (--out f | --in-place) — for every X.en_US column, add X.<to> copying the value; --skip-base leaves families alone. --to takes a comma-list → all project locales in one call.
• duplicate-rows <file>... --column locale --from en_US --to fr_CA[,uk_UA,…] (--out f | --in-place) — clone rows per --to value (glossary → all locales; store-relation rows → all stores).
• set <file>... --column store --value US [--where col=val ...] (--out f | --in-place) — set a column on matching rows (empty --where = all rows).
• replace <file>... --column url.fr_CA --search /en/ --with /fr/ [--regex] [--where col=val] (--out f | --in-place) — string/regex replace within an existing column.
• scale <file>... --column value_gross [--column value_net] (--by 1.47 | --rates PLN=4.3,UAH=45 [--currency-column currency]) [--no-round] [--json-keys net_price,gross_price] [--where col=val] (--out f | --in-place) — multiply numeric values (currency conversion); --column is repeatable — net + gross scale together in one call (--column value_net --column value_gross); --rates applies a per-currency factor in one call; --json-keys scales embedded price tiers.
• derive <file>... --target value_gross --source value_net --factor 1.19 [--no-round] [--only-empty] [--where col=val] [--out f | --in-place] — cross-column arithmetic scale can't do: writes target = source × factor, creating the target column if absent. Skips rows whose source is empty/non-numeric; a non-numeric/zero --factor is a hard error (never silently zeroes a column). gap-fills (won't overwrite existing target values). E.g. fill gross from net + tax, or a base-currency price into a second column before converting.

Notes: all access is by header name, never index. A missing column returns a clean JSON error with exit 2. Idempotent where sensible. Writes are atomic (temp file + rename — a crash mid-batch never leaves a truncated CSV). macOS/OrbStack bind-mounts lag — re-check a written file a beat later. Counting: never | grep -c — use rowCount (columns/read) or matchedRows (filter preview). Batch, don't loop.

---

validate.php — pre-boot checks

Output: JSON {status, check, findingCount, findings, errors}, exit 2 on any finding, 0 clean. Add --quiet to branch on the exit code alone.

• preflight <import-config.yml> [--base dir] — run this FIRST; one driver that auto-discovers and checks the boot-critical invariants across every file the manifest imports, so you never enumerate files by hand. Reads the manifest's source: list and, by column shape, bundles four checks: url.<locale> global uniqueness — CROSS-FILE, like the real spy_url constraint (a product URL colliding with a category/merchant URL is caught; one dup aborts the 30–60 min install; navigation-node sources are exempt — a nav node's url is a link target, legitimately repeated across menus), is_searchable.<locale> non-blank (blank = silently unsearchable), price completeness — gross-mode aware (a missing/0 value_gross, or a missing net in a net-ONLY file, is a hard problem; a missing net beside a present gross is how the shipped gross-mode stores legitimately look → reported under non-gating priceNetWarnings/warningCount), and base-before-relation import order (currency before currency-store, store before every *-store, and every store-definition entry — checked on its LAST occurrence — before the first product/category entity; a 3rd store's locale-store appended after the catalog = a silently empty store). Also flags any manifest source: that doesn't exist. --baseline <previous-preflight.json> suppresses findings already present in that report — capture a baseline on the untouched clone first, then only NEW findings gate (the shipped demoshop carries pre-existing quirks that are not yours to fix). It also runs generate-mode required-shape checks (shapeWarnings): product-abstract source has a color_code column (else 0 rows import silently), visibility ∈ {PDP,PLP,Cart,∅}, product-abstract imported with an *approval-status entity (else 0 search docs, invisibly), merchant-product-offer accompanied by merchant-product, and tax_set_name ⊆ the tax source. Returns one grouped verdict (problemCount, then unreadableSources/urlDuplicates//////), exit if any. Shipment-price files are exempt from the empty-or- price rule (free shipment is a real value). — the "a finished project carries only its own locales" check, across (the sweep misses that tree). Use the individual checks below to drill into a specific file; is the sweep that replaces remembering to run each one everywhere.

Concept-free by design: new demoshop files/entities need no new code — point the checks at them. A green boot is still the correctness authority; these are the cheap early net. If a tool genuinely can't do something, a fallback is fine — but prefer the tool (awk/sed/python corrupt multi-line quoted fields and prompt).

Data & references — the maintained literals

Two files hold every literal the consumer skills used to inline, so a demo-data bump is one diff here instead of a hunt across eleven prose files. known-set keeps both honest.

• data/entity-map.yml — one row per shipped data_entity: its source filename (which often differs from the entity name — that mismatch is why the file exists), its class (structural = theme-independent plumbing kept regardless of catalog; content = theme-dependent demo data; unclassified = undecided, blocks boot), and, for structural rows, the why. The authoritative structural keep-set the clean/minimal-baseline keep-lists derive from.
• references/demo-facts.md — the concrete identifiers (a real SKU pair, the cms-block-email-- prefix + block_key column, the shipped product-label + navigation keys, the shop_ui colour keys, the manifest families), each paired with the command that re-derives it. No record counts — those are stated as their measuring command, never a stored number.

---

Flag cookbook (one worked example per command — copy, don't guess)

Non-obvious flag behaviours that otherwise cost guesses:

• set adds a missing column — csv set url.uk_UA.csv --column url.uk_UA --value /uk/ creates url.uk_UA if absent (not only sets an existing one).
• prefix/substring match uses --match — csv filter cms_block.csv --where block_key=cms-block-email-- --match prefix (there is no --where-contains; note the column is block_key, not name).
• membership uses --in / --in-file — csv filter product_abstract.csv --in abstract_sku=A1,A2 or --in-file abstract_sku=keep.txt (one value per line).
• refs reference set is --in OR --ref-file+--ref-column — validate refs offer_store.csv --column store --ref-file store.csv --ref-column name (not --in-file).
• scale scales net+gross together — csv scale product_price.csv --column value_net --column value_gross --rates PLN=4.3,UAH=45 --in-place.
• product-refs takes a DIRECTORY + --keep-from <file>:<column> (not a manifest) — validate product-refs data/import/common --keep-from …/product_abstract.csv:abstract_sku.
• preflight / manifest-diff take a MANIFEST yml — validate preflight data/import/local/full_EE.yml --base . --locales en_US,pl_PL,uk_UA; validate manifest-diff data/import/local/full_EU.yml full_EE.yml --base ..
• known-set runs from any cwd on its defaults — validate known-set --base <repo-root> (checks the map + the count gate); validate known-set --emit-map --base <repo-root> > /tmp/skeleton.yml to regenerate the map skeleton for a diff review after a demo-data bump.
• --baseline takes a SAVED preflight report — capture once on the untouched clone: run validate preflight <manifest> plainly and Write its JSON output to .ai-dev/preflight-baseline.json with the Write tool (not a > redirect — redirects can't be allowlisted and prompt). Then gates only NEW findings.