| name | looker-to-sigma |
| description | Convert a Looker instance (LookML semantic model + dashboards) into a Sigma data model and matching workbook(s). Use when the user has Looker content — LookML projects, explores, or dashboards (user-defined OR LookML-defined) — and wants to recreate it in Sigma. Discovery via the Looker REST API 4.0 / Looker MCP server (or LookML files offline), model conversion via the local vendored LookML converter (convert_lookml_to_sigma MCP tool as a manual fallback), dashboard → workbook conversion from the Looker Dashboard API JSON, build via the Sigma REST API, and 3-way parity verification against the source warehouse — driven by `scripts/*`. |
| user-invocable | true |
Looker → Sigma Conversion
Windows / first run — run the environment doctor before anything else:
bash scripts/doctor.sh (macOS/Linux/Git Bash) or powershell -ExecutionPolicy Bypass -File scripts\doctor.ps1 (Windows).
It checks Ruby/Python/Node/bash and flags the Python "Store stub" + CRLF with exact fixes. Details: refs/environment.md.
Preflight the workbook spec before POST (mandatory)
Before POSTing any workbook spec, run ruby scripts/lib/preflight_lint.rb <spec.json> — it exits 1 with a precise message on the two migration-killer bugs: a table with aggregate columns + dimensions but no groupings (renders raw detail rows), and a malformed control (missing id/controlId/controlType or nesting value fields under a value object instead of flat, a non-double-nested source, or a list control wired to neither source nor filters — a filters-only list control is valid). Fix every violation first — never POST past it, and never conclude a feature is "unsupported" from an Invalid kind error (it means the inner fields are wrong). Verified shapes: sigma-workbooks controls.md / tables.md.
Phase 0a — Choose where to build (ask first when no destination given)
Don't silently land the migrated data model + workbook in an auto-picked folder.
If the user didn't supply a destination (no --folder <id> and no SIGMA_FOLDER_ID), ASK before building:
python3 scripts/pick_destination.py list → { workspaces, folders (editable, with parentName), myDocuments }- Let the user pick ONE: a workspace (its
id lands content in the workspace root),
an existing folder, My Documents (when non-null — null for service tokens), or
create a new folder: python3 scripts/pick_destination.py create --name "<name>" [--parent <workspace-or-folder-id>]
- Pass the chosen id as
--folder <id>. folderId accepts a workspace id or a folder id.
If a destination is already supplied, honor it silently — don't ask.
Convert a Looker LookML semantic model into a Sigma data model, then build Sigma
workbook(s) that mirror the Looker dashboards (user-defined OR LookML-defined) as
closely as possible — and verify the numbers match Looker AND the warehouse.
Read ALL of the following before replying or taking any action. Do not make assumptions about skill conventions, prompts, or global instructions — read the files.
refs/operating-contract.md — READ FIRST: the fidelity guardrails (render + value-check EVERY page against the source; never ship empty or silently drop a tile; don't spin — surface blockers).refs/modeling-strategy.md — faithful reproduction is the DEFAULT (parity is the gate); an upstream OBT / Sigma-native materialization is an OPT-IN optimization for hot, join-heavy dashboards, re-verified against the same oracle. The converter never auto-flattens.refs/dashboard-contract.md — the normalized Looker Dashboard JSON contract both the live API fetch and the offline LookML parse produce. The dashboard pipeline is source-agnostic; it only sees this contract.refs/looker-dashboard-layout.md — the deep desk study: Looker layout modes, newspaper→24-col grid math, tile-type / filter-type maps, and the full translation-hazard catalog (Liquid, merged_results, table calcs, view/explore field resolution, cross-filtering). This is the design backbone of the dashboard pipeline.refs/layered-lookml.md — layered/derived LookML: derived tables on derived tables, cross-view ${view.SQL_TABLE_NAME} refs (CTE inlining vs LOOKER_SCRATCH placeholders), CTE-continuation fragments, incremental/persisted PDTs → the Sigma materialization handoff, dimension_group edge cases, and untranslatable formatting measures. Read before converting any project with derived_table: views.
For canonical spec shape (data-model element kinds, workbook element kinds, controls, formulas, formatting), defer to the companion sigma-data-models and sigma-workbooks skills. This skill restates only the Looker-conversion-specific patterns.
The two artifacts, two pipelines
Looker has two independent layers; convert them separately.
| Layer | Source (production = API-first) | Converter | Sigma output |
|---|
| Semantic model | LookML views+model (Looker API/MCP, or files offline) | local vendored converter/lookml.mjs (run in-process via node; the convert_lookml_to_sigma MCP tool is a manual fallback only) | data model |
| Dashboards | GET /dashboards/{id} JSON — covers user-defined (UDD) AND LookML dashboards | fetch_looker_dashboard.py → contract → build_workbook.py | workbook |
| Looks | GET /looks/{id} (LookWithQuery) — a single saved query | fetch_looker_look.py → the same contract (one tile) → build_workbook.py | workbook |
Looks are a thin add on the dashboard pipeline. A Look's .query is the same Looker
Query shape a dashboard tile embeds, so fetch_looker_look.py synthesizes a one-tile
contract (via the shared normalize_element) with filters:[] and a full-width layout —
everything downstream (build_workbook.py, parity, gates) is reused unchanged. A Look with
dimensions + measures becomes a Sigma grouped table (dims → groupings.groupBy,
measures → groupings.calculations); measure-only → a KPI row; pivoted → a pivot-table
(rowsBy/columnsBy/values); a chart Look → the matching chart. The Look path additionally
emits fieldMeta (authoritative dim/measure category from the explore metadata + custom
dynamic_fields) so ad-hoc/custom measures classify correctly even with no local LookML.
Run it with --look-id <id> (see the ONE COMMAND block). A table-only Look verifies via
a grand-total parity target; a dimension-only detail Look has no measure to check (named
--skip-parity-gate waiver + the offline golden test + visual QA).
Critical — UDD is the primary path. Most real Looker dashboards are user-defined
(UDD) — built in the UI, NOT in any LookML file. They are reachable ONLY via the Looker
API, which returns UDD and LookML dashboards as the same Dashboard JSON
(dashboard_elements[] + dashboard_layouts[] + dashboard_filters[]). So the dashboard
converter keys off that API JSON, not LookML. .dashboard.lookml parsing is a secondary,
offline-only path that normalizes into the same contract.
ONE COMMAND (preferred): migrate-looker.py
⛔ THE ONE PATH (do not improvise a workbook)
migrate-looker.py is the single entry point; it only reaches GREEN when the
assert-phase6-ran hard gate passes with real charts. Rules:
- NEVER hand-drive the per-phase scripts, hand-author a DM/workbook JSON, or
curl-POST to /v2/workbooks / lay out empty "placeholder" pages — that
bypasses parity + the gate and ships an EMPTY workbook (the #1 failure mode).
If Looker isn't reachable (no ~/.looker/looker.ini / API creds), STOP and
tell the user to authenticate — do not build a shell.
- "Done" is a file on disk, not "pages exist." Complete only when
ruby scripts/verify-complete.rb --workdir <WORK> prints ✅ DONE (the gate
stamped phase6-success.json). An empty workbook is never done.
The whole pipeline — parse → RLS gate → convert → DM-reuse check → DM
POST + readback → workbook build (layout inline) → source-freshness
preflight → scripted parity + hard gate — as a single command (mirrors
qlik-to-sigma's migrate-qlik.rb / thoughtspot's migrate-thoughtspot.py).
Gates are never bypassed: the command exits non-zero if parity or
assert-phase6-ran.rb fails.
export SIGMA_CONNECTION_ID=<full-connection-uuid>
python3 scripts/migrate-looker.py --lookml-dir fixtures/skilltest-orders \
--dashboard fixtures/skilltest-orders/skilltest_orders.dashboard.lookml \
[--name PREFIX] [--workdir /tmp/look-run]
python3 scripts/migrate-looker.py --lookml-dir /path/to/lookml \
--dashboard-id <id> [--explore <name>] [--name PREFIX] [--workdir DIR]
python3 scripts/migrate-looker.py --lookml-dir /path/to/lookml \
--look-id <id> [--explore <name>] [--name PREFIX] [--workdir DIR]
- Decision points are flags with safe defaults, never silent:
detect_rls.py
runs first — RLS findings STOP the command (exit 10, nothing posted) until you
either port them via apply_sigma_rls.py (Phase 1.5) or re-run with --yes
(proceed WITHOUT RLS — loud + recorded). The DM-reuse check (Phase 2.5) always
runs and PRINTS candidates+scores. Default is BUILD-NEW — reuse only when
you pin one with --reuse-dm <id>. (Auto-reuse keyed on table coverage could
adopt a DM missing a column the workbook needs → the workbook POST then 400s
Dependency not found; that footgun is now opt-in via --reuse-auto.) Skip the
scan entirely with --skip-dm-reuse-check. The folder is auto-resolved + printed.
- Performance scan (Phase 2b) — don't port slow SQL blind: the converter turns every
derived_table into ONE Sigma Custom SQL element with the SQL carried through verbatim
(nested derived views inline as stacked CTEs into a single element), so a slow Looker derived
table stays slow in Sigma. detect_derived_perf.py runs after the RLS gate (scoped to the
migrated explore) and prints per-derived-table recommendations — materialize /
rebuild-as-element / leave-inline — to derived-perf.json. It is informational, never
blocks (unlike RLS), and is silent when there are no derived tables. --materialize-derived auto then triggers a Sigma materialization for the flagged element(s) after the DM is built
(all = every Custom-SQL element); default off = recommend only. The recurring schedule is
UI-only (Element ⋮ → Materialization), so this triggers/monitors and, when an element isn't
materialization-configured yet, prints the exact UI handoff. Post-migration, rank which
materializations actually pay off by warehouse credit with the sigma-materialization-advisor
skill.
- Source repointing: if the LookML
sql_table_name points at a DB.SCHEMA the
Sigma connection doesn't serve (e.g. dev DEMO_DB.DEMO.* vs the connection's
QUICKSTARTS.LOOKER_RETAIL_ANALYTICS.*), pass
--source-swap FROM_DB.FROM_SCHEMA=TO_DB.TO_SCHEMA (repeatable). A not-yet-indexed
schema (catalog miss) self-heals — post_dm.py auto-syncs and retries once.
Don't know the FROM? --auto-source-swap-to TO_DB.TO_SCHEMA asks Looker what
DB.SCHEMA the explore's connection targets (GET /connections) and builds the
swap for you (production-safe; needs ~/.looker/looker.ini).
- No local checkout?
--project <id> pulls the LookML over the Looker REST API
(model + views) instead of --lookml-dir. Requires DEVELOP permission on the
project (Looker only serves raw LookML in the dev workspace); without it the
command fails loud and tells you to clone the Git repo and use --lookml-dir.
scripts/looker_project.py is the standalone helper (pull / connection).
- Converter — zero-config, local, no MCP. A self-contained converter bundle
ships in the skill at
converter/lookml.mjs and is the default: conversion runs
locally via node with no clone, no npm install, no network, no MCP. A dev's
own build still wins when set — CONVERTER_SRC (src/lookml.ts via tsx) or
CONVERTER_PATH (build/lookml.js), both auto-located. Refresh the vendored
bundle with tools/vendor-converters.sh (see converter/PROVENANCE.json). Only
if the bundle is missing AND no build is found does the command fall back to the
MCP path: it writes <workdir>/convert-request.json (the exact
convert_lookml_to_sigma arguments) and exits 3 — call the tool, save its JSON,
re-run with --converted <file>.
- Parity is fully scripted (the Phase-4 gate below): ACTUAL = Sigma CSV
export per chart; EXPECTED = a Looker inline query (live) or a
SOURCE-LookML-derived re-aggregation of the master's warehouse rows (offline —
measure semantics from the
.view.lkml type:, independent of the builder's
formulas). Then phase6-parity-looker.rb --finalize + assert-phase6-ran.rb
run automatically. Both per-chart fetch sides run in a bounded 4-wide thread
pool (measured 24.3s → 5.8s on the 5-chart fixture); set
LOOKER_PARITY_WORKERS=1 to serialize on a loaded warehouse (max is clamped
to 4 — warehouse-friendly bursts only).
- Exit codes:
0 GREEN · 3 MCP convert request emitted · 10 RLS decision
needed · 2 built but a gate FAILED. --dry-run = no Sigma POSTs.
Scripts
| Script | Purpose |
|---|
scripts/migrate-looker.py | ONE-COMMAND orchestrator (preferred entry) — chains every phase below + the scripted parity hard gate; see the section above. |
scripts/phase6-parity-looker.rb | Phase 4 (parity gate): two-pass orchestrator — PASS 1 reads the workbook spec → parity-plan.json + per-chart fetch instructions; PASS 2 --finalize runs verify-parity.rb and writes the parity-final.json sentinel the hard gate requires. Same contract as quicksight/thoughtspot/tableau. |
scripts/verify-parity.rb | Phase 4: the comparator (strict set-compare with date-bucket canonicalization; --extract-mode tolerance variant). Vendored from the shared converter copy. |
scripts/assert-phase6-ran.rb | HARD GATE (vendored byte-identical across the 5 plugins — keep the md5 in lockstep): parity ran + PASS, no orphan workbooks, no type=error columns, layout applied, layout lint (gate 6), control lint (gate 7 — dead controls / ghost targets / partial same-page reach / control-scope.json coverage; --skip-control-lint escape, exit 9; see refs/control-parity.md). ruby scripts/assert-phase6-ran.rb --workdir <dir> --workbook-id <wb> must exit 0 before declaring GREEN. |
scripts/probe-controls.rb | Optional Phase-4 flip test — runtime proof that controls actually filter: per control, exports one in-closure element CSV with and without parameters:{controlId: <non-default value>} (must differ) and, with --check-out-of-closure, an out-of-closure element (must NOT differ). Not the mandatory inner loop. Shared, vendored byte-identical. refs/control-parity.md has the design + the MCP-vs-export answer. |
scripts/get-token.sh | Exchange SIGMA_CLIENT_ID/SIGMA_CLIENT_SECRET → SIGMA_API_TOKEN (~1h TTL). eval "$(scripts/get-token.sh)" |
scripts/looker_api.py | Minimal Looker REST API 4.0 client (no SDK). Reads ~/.looker/looker.ini, logs in via client_credentials, exposes L.call(method, path, body). Caches the bearer per process (thread-safe; one login instead of one per call — ~150ms/call saved, measured 2.4x on a 10-call run) and retries once with a fresh login on 401. CLI: python3 looker_api.py whoami / get <path> / raw GET /lookml_models. |
scripts/fetch_looker_dashboard.py | Phase 1 (live): GET /dashboards/{id} → the normalized contract (refs/dashboard-contract.md). Works for UDD AND LookML dashboards. Self-contained (reads ~/.looker/looker.ini). tileType is read from query.vis_config.type (NOT element.type, which is always "vis"); listen from result_maker.filterables; layout from the active layout's components. Exposes the shared normalize_element() (one tile → one contract element) and build_field_meta() (authoritative dim/measure category from the explore metadata + dynamic_fields). |
scripts/fetch_looker_look.py | Phase 1 (live — Looks): GET /looks/{id} → the same contract as the dashboard fetch but with ONE tile, filters:[], a full-width layout, and a fieldMeta map. Imports+reuses fetch_looker_dashboard's helpers (single source of truth), so a Look tile normalizes identically to a dashboard tile. python3 fetch_looker_look.py <look_id> [out.json]. |
scripts/parse_lookml_dashboard.py | Phase 1 (offline): parse a .dashboard.lookml (YAML) → the SAME contract. Dev/test only; cannot see UDD dashboards. Requires PyYAML. |
scripts/detect_rls.py | Phase 1 (RLS scan): dependency-free regex scan of a LookML dir/file (and/or model JSON) for row-level-security constructs (access_filter, sql_always_where, access_grant, user_attribute). Prints a structured summary + recommended Sigma mapping per finding (or --json). Prints nothing / exits 0 when there's no RLS (zero-overhead). python3 detect_rls.py <lookml_dir> [--json] |
scripts/detect_derived_perf.py | Phase 2b (performance scan): dependency-free regex scan for EXPENSIVE derived_tables (nested-on-derived, un-persisted, persisted/incremental PDTs, NDTs, SQL complexity, warehouse hints). Scores each and recommends materialize / rebuild-as-element / leave-inline — the handoff for --materialize-derived and, post-migration, sigma-materialization-advisor. Informational (never blocks); silent when there are no derived tables. python3 detect_derived_perf.py <lookml_dir> [--scope-explores e1,e2] [--json] |
scripts/apply_sigma_rls.py | Phase 1.5 (apply RLS): scripted, API-driven RLS port. Reuse-first GET /v2/user-attributes (prints a match before creating); --create → POST /v2/user-attributes; --assign (+--member-id,--value) → POST /v2/user-attributes/{id}/users; --field/--element-id → print the verified RLS calc-col + element-filter snippet, --apply --dm-id → PATCH it into the DM element spec. Read-only / plan-only by default — mutates only on an explicit --create/--assign/--apply flag. Reads $SIGMA_BASE_URL/$SIGMA_API_TOKEN like post_dm.py. |
scripts/convert_dm.mjs | Phase 2: run convertLookMLToSigma against a directory of .lkml files for one explore → a Sigma DM spec JSON + …-warnings.json sidecar. A .model.lkml is optional — with none it converts view-only (each view → standalone element; pass the WHOLE directory so cross-view ${view.SQL_TABLE_NAME} refs resolve — see refs/layered-lookml.md). Bypasses the deployed MCP build (see the converter-build gotcha below). Env: LOOKML_DIR, CONVERTER_SRC; args <exploreName> <out.json>. |
scripts/lookml-dm-signature.py | Phase 2.5: LookML view files → DM-reuse signature ({warehouse_tables, referenced_columns, measures}) for find-or-pick-dm.rb. Pure, no network. |
scripts/find-or-pick-dm.rb | Phase 2.5: scan existing Sigma DMs and recommend reuse (score = 0.7·column + 0.2·table + 0.1·metric overlap; --auto-pick with tie-window safety). Shared vendor-neutral copy (canonical: tableau-to-sigma; needs scripts/lib/sigma_rest.rb). Non-destructive. |
scripts/shape-preflight.rb | Phase 2.5 (reuse gate): before wiring a workbook to a REUSED DM element, mechanically check the three things a spec POST won't: (1) the element is not visibleAsSource:false (hidden → unusable as a source), (2) needed columns resolve on it, (3) every relationship it reaches columns through is 1:1 (non-unique target key → silent fan-out). Visibility/coverage hard-fail (exit 2); fan-out is surfaced with the exact Sigma-MCP uniqueness query per relationship — run it and feed back via --fanout-results (or --ack-fanout). Non-destructive; needs scripts/lib/sigma_rest.rb. migrate-looker.py auto-runs it on --reuse-dm. |
scripts/post_dm.py | Phase 2: POST a DM spec to /v2/dataModels/spec (auto-finds a writable folder, swaps in the full connection UUID). Env: SIGMA_API_TOKEN, SIGMA_BASE_URL, SIGMA_CONNECTION_ID; args <spec.json>. |
scripts/build_workbook.py | Phase 3: dashboard contract + the explore's view .lkml files → a Sigma /v2/workbooks/spec body (hidden Data page + master table, one element per tile, controls from filters, newspaper→24-col layout XML). Generates locally; does not POST. Handles ratio measures, joined-col Field (alias) naming, table calcs, pivot-flatten + warn. Layout: a top control bar (row 0), a full-width strip of tall KPI tiles (height ≥ 6 so titles render), then the remaining tiles shifted down. |
scripts/looker-render-dashboard.py | Phase 4 (visual QA — SOURCE side): render a LIVE Looker dashboard to PNG via the Looker render API (POST /render_tasks/dashboards/{id}/png → poll GET /render_tasks/{task_id} until success → GET .../results). Pairs with sigma-export-png.py for source-vs-migrated side-by-side. Reuses looker_api.py ~/.looker/looker.ini auth. python3 looker-render-dashboard.py <dashboard_id> [out.png] [--w 1200 --h 1600]. |
scripts/looker-render-look.py | Phase 4 (visual QA — SOURCE side, Looks): render a LIVE Look to PNG via GET /looks/{id}/run/png (SYNCHRONOUS — no render-task poll). The Look analog of looker-render-dashboard.py; pairs with sigma-export-png.py. python3 looker-render-look.py <look_id> [out.png] [--w 1200 --h 900]. |
scripts/sigma-export-png.py | Phase 4 (visual QA — MIGRATED side): render a posted workbook page or element to PNG via POST /v2/workbooks/{id}/export → poll GET /v2/query/{queryId}/download. For side-by-side layout/render checks against the source Looker dashboard render (catches hidden KPI titles, orphaned filters, overlaps, bare-number vs $/% formats that a numeric parity check can't). Read each migrated PNG and check it against refs/layout-visual-qa.md (mandatory gate — see Phase 4a). Reads $SIGMA_BASE_URL/$SIGMA_API_TOKEN. python3 sigma-export-png.py --workbook <id> --page <pageId> --out /tmp/x.png (or --element <id>). |
scripts/build_looker_dashboard.py | TEST-FIXTURE BUILDER (not a migration step). Builds the "Orders Overview" UDD on demo_thelook via the Looker API (4 KPIs + line/column/bar/pie + grid, 3 filters wired via result_maker.filterables.listen). |
scripts/build_looker_dashboard2.py | TEST-FIXTURE BUILDER (not a migration step). Builds the "Orders Deep Dive" UDD — area, pivot, table-calcs, scatter, donut, text tile — the harder dashboard surface for the converter. |
scripts/build_looker_look.py | TEST-FIXTURE BUILDER (not a migration step). Creates a matrix of test Looks on demo_thelook/order_fact (grouped table, measure-only, dims-only, bar, pivot, and a dynamic_fields custom measure) to exercise the Look → grouped-table / pivot / KPI / chart converter. POST /queries then POST /looks. python3 build_looker_look.py [looker_folder_id]. |
scripts/record-visual-check.rb | Phase 4a (visual gate): record the agent's source-vs-target visual verdict into parity-final.json so assert-phase6-ran.rb gate 8b can confirm the comparison happened (not a prose "I looked"). Shared/vendored byte-identical with the other migration plugins. ruby record-visual-check.rb --workdir <dir> --agent-vision true --verdict pass --screenshot <png> --checklist "…" --notes "…". |
scripts/gap-scout.md | Gap scout (converter gaps): runbook for the main agent — when/how to spawn a scout subagent for a LookML construct the converter can't translate, the LookML→Sigma candidate table, and the opt-in issue-filing flow. Read before spawning. |
scripts/scout-validate.py | Gap scout: validate a candidate Sigma formula against a real DM element (throwaway test workbook → check column type ≠ error → delete), persist a win to ~/.looker-to-sigma/learned-rules.yaml, or return an opt-in escalation block on failure. Also a quick "does this formula resolve?" check for Phase-4 validation. Reads $SIGMA_BASE_URL/$SIGMA_API_TOKEN. |
scripts/learned-rules.py | Gap scout: loader for the customer-local learned-rules.yaml (load()/apply()); applied to LookML measure expressions before the converter/WARN fallback. Home = ~/.looker-to-sigma (override LOOKER_TO_SIGMA_HOME). |
scripts/escalate-gap.py | Gap scout (shared, identical across all migration skills): opt-in GitHub-issue filer — category→repo routing, dedupe (open issues + beads), converter-repo mirroring, bead cross-link. Dry-run by default; files only with --yes. Requires gh. |
Test-fixture builders vs migration scripts. build_looker_dashboard.py /
build_looker_dashboard2.py / build_looker_look.py author Looker dashboards/Looks
(migration targets); they are for standing up known demo content to convert and
parity-check. Never run them against a customer's Looker. Everything else converts from
Looker to Sigma.
Prerequisites
Looker credentials (~/.looker/looker.ini)
[Looker]
base_url=https://<your-instance>.cloud.looker.com
client_id=<API3 client_id>
client_secret=<API3 client_secret>
verify_ssl=True
- API 4.0, key-pair-free
client_credentials. Modern Google-hosted Looker serves the API on
the standard 443 at https://<instance>.cloud.looker.com/api/4.0 — no port needed. Older
self-hosted instances used the legacy :19999; if your base_url still carries it and that
port is unreachable, the client self-heals: it retries on 443 and warns you to update the
ini. You can also override the base without editing the ini via LOOKER_BASE_URL
(or LOOKERSDK_BASE_URL).
- TLS: requests use the OS trust store via
truststore (installed by bash scripts/bootstrap.sh),
so Python's stricter OpenSSL 3.x accepts the same certs curl/Ruby do; it falls back to certifi
then the stock bundle. Leave verify_ssl=True (setting it false disables verification and warns).
- The credential's user needs Admin (or at least: see models, dashboards, run queries, and
— for the test-fixture builders or Git-deploy flow — develop + deploy).
- Generate an API3 key in Looker: Admin → Users → (your user) → Edit Keys → New API3 Key.
- Test:
python3 scripts/looker_api.py whoami → prints HTTP 200, your display name + roles.
Sigma credentials
eval "$(scripts/get-token.sh)" exchanges SIGMA_CLIENT_ID/SIGMA_CLIENT_SECRET (from
~/.sigma-migration/env, written by the sigma-api skill's setup.rb) for a SIGMA_API_TOKEN.
Also note your full connection UUID (SIGMA_CONNECTION_ID) and a writable folderId.
Tokens live ~1 hour. Re-fetch when a curl returns 401. Never use
TOKEN=$(eval "$(scripts/get-token.sh)") — $() is a subshell where the exported var dies.
Keep eval + curl in the same bash -c '...' invocation.
Inline Python/Node inside bash — DON'T. Triple-nested escapes silently break. Always
write a .py/.mjs file with Write and call it via python3 file.py / node file.mjs.
The scripts here already follow that rule.
The Looker-side warehouse connection (one-time, for live parity)
Looker needs its own direct warehouse auth — Sigma's connection UUID is Sigma-side and
unusable for Looker. To stand up an end-to-end test pointed at the same warehouse as Sigma
(so 3-way parity is meaningful):
- Snowflake service identity: create a
SERVICE-type user with key-pair auth (Snowflake
blocks single-factor passwords for service users) + a role granting USAGE on the warehouse +
the db/schema and SELECT on the tables/views.
- Looker connection (
POST /connections): uses_key_pair_auth: true, certificate =
base64 of the .p8 private key, file_type: ".p8", warehouse via
jdbc_additional_params=warehouse=<WH>, host <account>.snowflakecomputing.com. Test via
PUT /connections/{name}/test.
- Git-backed project + model: create a project in the dev workspace, add a deploy key to
the Git repo, set the git remote via
PATCH /projects/{id} (PUT 404s). All dev-workspace
mutations need ONE persistent session (PATCH /session {workspace_id: dev}).
This setup is only needed to build a live test instance. For a customer migration the Looker
instance + connection already exist — you just read from them.
Phase 0b — Assess the Looker estate
Scope the migration before converting anything — inventory models/explores/dashboards, score
complexity, and rank a migration shortlist. This is handled by the looker-assessment
sibling skill (analogous to tableau-assessment / qlik-assessment). Run it first for any
multi-dashboard migration; skip it for a single known dashboard.
Phase 1 — Discover the Looker content
Three transports, in order of preference: Looker MCP (when wired in) → Looker REST API
4.0 (the default here) → offline .lkml (dev/test, can't see UDDs).
1a. Smoke-test + list
python3 scripts/looker_api.py whoami
python3 scripts/looker_api.py raw GET /lookml_models
python3 scripts/looker_api.py raw GET /dashboards
For a specific explore's field graph:
python3 scripts/looker_api.py raw GET /lookml_models/<model>/explores/<explore>.
1b. Pull each dashboard into the normalized contract (live)
python3 scripts/fetch_looker_dashboard.py <dashboard_id> /tmp/<name>/<dash>.contract.json
Discovery speed: already sub-second. A dashboard pull is one login (cached
per process) + one GET /dashboards/{id} — there is no estate walk to optimize.
For many-call sessions (parity, inventory) looker_api.py's per-process token
cache removes the per-call login round-trip (~150ms each).
This hits GET /dashboards/{id} and normalizes into refs/dashboard-contract.md. It works
for UDD and LookML dashboards (the API returns both identically). Key extraction details
(already handled by the script):
tileType comes from query.vis_config.type, not element.type (which is always
"vis" for chart tiles, "text" for text tiles).listen (which dashboard filters a tile obeys) comes from
result_maker.filterables[].listen.- layout comes from the active layout's
dashboard_layout_components[]
(row/column/width/height); ignore mobile variants.
dynamic_fields (table calcs / client-side custom measures) arrives as a JSON string
— the script json.loads it.
1c. Offline path (dev/test only)
python3 scripts/parse_lookml_dashboard.py <file.dashboard.lookml> --out /tmp/<name>/<dash>.contract.json
Same contract shape. Cannot see UDD dashboards; LookML dashboards may also lag the live UI
state (a .dashboard.lookml reflects source-of-truth, the API reflects edits). Prefer the
API. Note: a deployed LookML dashboard does NOT auto-index for import_lookml_dashboard
(Looker reindexes lazily, 404 until then) — just build/discover the UDD directly.
No live instance? A GCP free-trial account CANNOT provision Looker (instance quota is
isFixed = 0, Sales-gated). Build/test from sample LookML + the offline path. The validated
end-to-end run used a real example.cloud.looker.com instance pointed at DEMO_DB.DEMO.
1d. Scan for row-level security (RLS) — cheap, silent if none
Looker enforces row-level security in LookML, and security is the one place a silent default
is dangerous in both directions — silently dropping RLS exposes data; silently porting a wrong
mapping over- or under-restricts it. So scan for it during discovery, but stay out of the way
when there's nothing to decide.
python3 scripts/detect_rls.py /path/to/lookml
-
Zero overhead on the happy path. detect_rls.py is a cheap regex scan; if it finds no
RLS it prints nothing and exits 0 — no prompt, no extra phase, the migration proceeds
straight to Phase 2 unchanged.
-
If it finds RLS, it lists every finding (construct, explore, field, user_attribute,
expression) plus the recommended Sigma mapping — that output feeds the single RLS decision
gate below (do NOT prompt per rule). The constructs it detects, and their Sigma targets:
| Looker RLS construct | What it does | Sigma target |
|---|
access_filter (explore) | maps a user_attribute → a field; restricts rows to the caller's allowed values | a Sigma user attribute + a row filter using LookupUserAttributeText(...) / CurrentUserAttributeText(...) on that field |
sql_always_where (explore) | a hardcoded SQL row filter always ANDed onto the explore | a Sigma data-model / element filter (if the expression references a user_attribute / {{ _user_attributes[...] }}, make it a user-attribute row filter, not a static one) |
access_grant (model) | gates explores/fields/joins by a user_attribute's allowed values | note / review — no 1:1 analog; map to Sigma permissions or a user-attribute filter |
user_attribute reference | any other _user_attributes[...] / user_attribute: use | provision the matching Sigma user attribute (reuse if it already exists) |
The convert_lookml_to_sigma converter ALSO detects access_filter and emits an RLS note (and
a CurrentUserAttributeText() row-filter stub) in the DM spec. detect_rls.py is the
discovery-time, project-wide view that drives the decision gate — the converter handles the
per-spec emission once you've decided to port.
Phase 1.5 — RLS decision gate (only if Phase 1d found RLS) — BEFORE building
Skip this phase entirely when detect_rls.py found nothing. When it DID find RLS, stop ONCE,
here, before POSTing the data model in Phase 2 — make it one explicit, reviewed decision, never an
invisible default.
The whole flow is scripted and API-driven — Sigma user attributes are fully API-supported, so
reuse-first, provisioning, and the row filter itself are all done via apply_sigma_rls.py (no UI
step). Keep the framing intact (one consolidated gate, opt-in/out, never-silent); only the
mechanics are now concrete.
- Reuse-first — check what already exists in Sigma before creating anything (scripted). The
customer may have already set RLS up in Sigma; don't duplicate it.
-
Existing Sigma user attributes — list them via the API and match by name to the Looker
user_attributes in the findings. apply_sigma_rls.py --attr <name> does this:
GET /v2/user-attributes (read-only, no flags) and prints a REUSE: line with the existing
userAttributeId if one matches (case-insensitive), or "no existing attribute" otherwise.
Reuse a matching attribute rather than creating a new one.
bash -c 'eval "$(scripts/get-token.sh)" && python3 scripts/apply_sigma_rls.py --attr region'
-
Existing data models with similar RLS logic — if a Sigma DM already filters the same
field by the same attribute (e.g. a previously-migrated explore on the same source), reuse it
instead of re-implementing the filter.
- Pre-fill a recommended plan. Using the mapping table above, draft the per-finding Sigma
action (which user attribute, which field,
CurrentUserAttributeText row filter vs DM/element
filter vs note) — reusing the existing Sigma attributes/DMs found in step 1. Preview the exact
row-filter spec for a finding with apply_sigma_rls.py --attr <name> --field <DisplayName> --element-id <denorm-element-id> (prints the calc-col + element-filter snippet; plan-only).
- One consolidated confirm / edit / skip. Present the full plan and let the user, in a SINGLE
decision: confirm it as drafted, edit any mapping (e.g. point at a different existing
attribute, change a field), or skip porting RLS entirely (they may enforce it elsewhere in
Sigma). No per-rule nagging.
apply_sigma_rls.py is plan-only by default — it mutates ONLY
when you pass --create / --assign / --apply, so running it through step 1–2 never changes
anything before the user confirms.
- Always record the outcome. For every finding, note ported / reused / skipped in the
migration summary (Phase 4 output) so any skipped RLS is visible, never silent — a reviewer
can see exactly which Looker restriction was carried over, reused, or deliberately dropped.
Then proceed to Phase 2 and apply the confirmed plan as part of the DM build, via the SAME script:
- Provision the user attribute (only if nothing reusable was found in step 1):
bash -c 'eval "$(scripts/get-token.sh)" && python3 scripts/apply_sigma_rls.py \
--attr region --value West --create'
- Assign a value to the member(s) who should be restricted (the value the user attribute
resolves to per person — assign to the member that the parity query runs AS, or RLS returns 0
rows):
bash -c 'eval "$(scripts/get-token.sh)" && python3 scripts/apply_sigma_rls.py \
--attr region --value West --member-id <memberId> --assign'
- Apply the row filter to the DM element — the verified spec shape (a boolean calc column
CurrentUserAttributeText("<attr>") = [<Field>] + an element filters entry
{kind:list, mode:include, values:[true]}):
bash -c 'eval "$(scripts/get-token.sh)" && python3 scripts/apply_sigma_rls.py \
--attr region --field Region --element-id <denorm-element-id> \
--dm-id <dataModelId> --apply'
Mapping recap: access_filter and user-attribute sql_always_where → the
CurrentUserAttributeText("<attr>") = [<Field>] row filter above; static sql_always_where → a
plain DM/element filter; access_grant → the recorded note. (Team mode =
CurrentUserInTeam([...]); user-email mode = [Email] = CurrentUserEmail().)
Proof: this exact scripted flow was validated live end-to-end 2026-06-10 (a live Looker instance →
Sigma the demo Sigma org, demo_thelook order_fact, region/West) with exact 3-way parity —
Looker-restricted == Sigma-restricted == warehouse = $38,906.82 / 220 rows.
Phase 2 — Convert the LookML semantic model
LookML views + model → Sigma data model. Resolve the explore's join graph, convert, POST,
register the model, and verify.
2a. Convert — local by default, MCP only as a manual fallback
Feed it the LookML model, NOT the warehouse tables — the converter walks the explore's
joins to resolve view.field prefixes (alias vs from: view) and emits one element per
resolved view plus a denormalized explore element.
Default: the converter runs locally, in-process, no MCP, no data egress. The skill ships a
self-contained vendored bundle at converter/lookml.mjs; migrate-looker.py (and
scripts/convert_dm.mjs directly) run it via a node shim — no clone, no npm install, no
network call, and your LookML never leaves the machine:
LOOKML_DIR=/path/to/lookml \
node --import tsx/esm scripts/convert_dm.mjs <exploreName> /tmp/<name>/dm-spec.json
convert_dm.mjs reads <model>.model.lkml + every views/*.view.lkml, converts the explore
with joinStrategy: 'relationships', and writes res.model (the return property is .model,
not .sigmaDataModel). It prints stats + warnings — read every warning.
A dev's own checkout wins automatically when present: migrate-looker.py first checks
CONVERTER_SRC (a src/lookml.ts + tsx, for fixed output against a patched source tree —
see the build gotcha below) or CONVERTER_PATH (a built build/lookml.js), resolved from
~/sigma-data-model-mcp or ~/Desktop/sigma-data-model-mcp (CONVERTER_HOMES) if set. The
vendored converter/lookml.mjs bundle is the guaranteed floor underneath both — it is what runs
out of the box with nothing configured.
Fallback only — no local converter and no node: migrate-looker.py writes
convert-request.json (the exact mcp__sigma-data-model__convert_lookml_to_sigma(files, connectionId, exploreName, joinStrategy) arguments) and exits 3. Call the MCP tool by hand
with those arguments, save its JSON output to <workdir>/converted.json, and resume with
--converted <workdir>/converted.json. Reach for this only when the vendored bundle is missing —
routing through the hosted MCP by default means your LookML leaves the machine and you're at the
mercy of whatever converter version the MCP happens to be running (version drift vs the vendored
bundle / a patched source tree).
Converter-build gotcha (only relevant when using CONVERTER_SRC). The long-running MCP
server serves the deployed build. After editing src/lookml.ts + npm run build, the
running MCP tool still serves the OLD code until it restarts — another reason the local
CONVERTER_SRC/vendored-bundle path is preferred over calling the MCP tool.
2b. Converter coverage (all live-validated 2026-06-10) — and what's still lossy
The converter handles, end-to-end and clean:
- Dimensions —
tier, sql CASE, legacy case: (→ nested If()), html/link, custom
value_format.
- Time + duration
dimension_group — one column per timeframe (DateTrunc); duration groups
emit sql_start/sql_end physical columns.
- Measures —
sum/count/count_distinct/avg/median/percentile/filtered/ratio.
Measure ${dimension} refs and measure-references-measure ${measure} (ratio) refs resolve to
the right Sigma formula; 1.0 literals preserved; NULLIF → NullIf.
- Joins — snowflake (multi-hop) joins wire the FK to the correct intermediate element (not
always the base);
full_outer + field-limited joins; sql_always_where / always_filter.
- Other —
derived_table, parameter + Liquid, drill_fields, set, view/group labels,
multiple explores per model.
These 8 converter bugs were found and FIXED in source (branch
tj/lookml-robustness-ratio-percentile-html-fixes); treat them as handled, but know the
shapes so you recognize a regression:
| # | Bug (now fixed) | What it produced before the fix |
|---|
| BUG1 | measure ${dimension} refs unresolved | literal Sum([${sale price}]) + phantom ${...} columns |
| BUG2 | multi-hop (snowflake) joins mis-wired | FK hung off the base element instead of the intermediate |
| BUG3 | ratio measures (${measure}, 1.0→0) | phantom column + 0 * ${...} formula |
| BUG4 | html:/Liquid %} desynced the block parser | silently dropped ALL view fields after the html dimension |
| BUG5 | percentile → bogus CountIf | wrong aggregation |
| BUG6 | filtered type:count with no sql | bogus phantom value column |
| BUG7 | type:duration dimension_group | dangling DateDiff (no sql_start/sql_end) |
| BUG8 | legacy case:{when/else} dim | passthrough to a nonexistent column |
If the running MCP build predates these fixes, use the convert_dm.mjs direct path (2a)
against a patched source tree, OR repair the spec post-hoc — but the source fixes mean raw
converter output now POSTs clean with no in-spec workarounds.
Still lossy / unsupported (documented, warned — never silent):
- Liquid
{% parameter %} measures and manifest constants — Looker API-deploy cache
quirk; review.
link: / html: styling — dropped (data is fine; the styling/hyperlink is lost).- Pivot cross-tab → flattened to columns + warn (rebuild as a Sigma
pivot-table in the UI).
- Table-calc grain/sort for window functions (rank / offset / percentile) → review.
merged_results → a DM join or a Custom SQL element (follow merge_result_id to the
source queries; >2 sources or non-equi joins → manual + warn).- Not yet converted: NDT (
explore_source), PDT datagroup/persist_for, many_to_many.
- RLS (
access_filter / sql_always_where / access_grant) — detected at discovery
(Phase 1d) and decided ONCE at the Phase 1.5 gate, then ported via the scripted, API-driven
apply_sigma_rls.py (reuse-first user-attribute lookup → create/assign → PATCH the
CurrentUserAttributeText("<attr>") = [<Field>] row filter). The converter also emits an
access_filter RLS note + CurrentUserAttributeText() stub. Never silently dropped — the
outcome is recorded.
metric() returns "Missing Metric" in MCP SQL — a known Sigma quirk, not a conversion
bug. Verify metric values via the raw aggregate (Sum(...), CountDistinct(...)), not via
metric().
Phase 2.5 — Reuse an existing DM? (run BEFORE 2c — avoid sprawl; the reuse-first DM gate every converter runs before building)
Before POSTing a NEW data model, check whether an existing Sigma DM already covers the
same warehouse tables (don't add a 4th near-identical "Orders" DM):
python3 scripts/lookml-dm-signature.py --lookml-dir /path/to/lookml \
--label "<explore label>" --out /tmp/<name>/dm-signature.json
bash -c 'eval "$(scripts/get-token.sh)" && \
ruby scripts/find-or-pick-dm.rb --workbook-signature /tmp/<name>/dm-signature.json \
--out /tmp/<name>/dm-match.json --auto-pick'
lookml-dm-signature.py derives {warehouse_tables (sql_table_name FQNs), referenced_columns (dimension/measure names), measures} straight from the LookML view
files — the same files you fed convert_dm.mjs. Decision:
-
Score ≥ 0.6 → ASK the user reuse-vs-new: surface the candidate name, matched
cols (N/M), and the inherited-extras warning from dm-match.json. If they reuse, run the
shape preflight first — ruby scripts/shape-preflight.rb --dm-id <id> --element "<element you'll wire to>" --needed-columns "<dashboard cols>" --out /tmp/<name>/shape-preflight.json
(exit 2 = unsafe). It mechanically confirms:
- every column the dashboards reference resolves on the element you'll wire to (no
type=error columns; fact vs separate-dim location);
- the element is usable as a source — i.e. it is NOT
visibleAsSource: false. The
flag defaults to true and is omitted from the spec when true, so absence = visible
and you only act when you literally see "visibleAsSource": false. A hidden element
builds fine via the API but users can't pick it as a source in the workbook UI —
wire to a visible sibling, or PUT the DM spec setting visibleAsSource: true, before
proceeding (verified live 2026-06-29);
- how related columns are exposed — if the columns you need live on a separate
element reached by a
relationships[] entry (a reused relational DM, NOT a flat denorm
element), note the relationship's exact name; Phase 3 will reference them as
[<element>/<RelationshipName>/<col>] (see 3b). Confirm the join is 1:1 on its key — a
non-unique target key silently fans out (see the fan-out troubleshooting row).
Then skip 2c/2d and point Phase 3's workbook masters at the matched
recommended_dm_id + its element ids. With --auto-pick a clear winner (no tie within
0.05) skips the prompt — still WARN about inherited columns/RLS/metrics.
-
Score < 0.6 → POST new (2c) and TELL the user no reusable DM was found.
2c. POST the data model
bash -c 'eval "$(scripts/get-token.sh)" && \
SIGMA_CONNECTION_ID=<full-connection-uuid> \
python3 scripts/post_dm.py /tmp/<name>/dm-spec.json'
- Endpoint is
POST /v2/dataModels/spec (NOT /v2/workbooks/spec).
- Use the FULL connection UUID (e.g.
ab12cd34-5678-40ab-8def-1234567890ab), not a short
prefix — convert_dm.mjs writes a placeholder connectionId; post_dm.py swaps in
$SIGMA_CONNECTION_ID.
folderId is required — post_dm.py auto-picks a writable folder (preferring one whose
name mentions LOOKER/MIGRATION/TEST).- The spec endpoints return YAML (
success: true\nworkbookId: …), not JSON — never
json.load the response or pipe it to jq.
Record the returned dataModelId and (after a read-back) the element IDs.
2d. Register the model + verify
A freshly POSTed/deployed LookML model 404s on query until you register it (Looker side
for the Looker model; this is the deploy flow for standing up a test instance):
PATCH /session {workspace_id: dev}
PUT /projects/{id}/git_branch {name: <dev-branch>, ref: origin/main} # pull pushed commits into dev
POST /projects/{id}/validate # expect 0 errors
POST /projects/{id}/deploy_to_production # 204
POST /lookml_models {name, project_name, allowed_db_connection_names:[<conn>]}
LookML param gotcha: params are not semicolon-separated — compact
{ primary_key: yes; hidden: yes; sql: ... ;; } fails ("Invalid lookml syntax") and cascades
into bogus join/field errors. Use multi-line blocks (only ;; terminates a sql).
A refinement view: +x in a glob-included file fails ("Could not find a view to extend") —
fold the param/measure into the base view.
Verify the Sigma DM: mcp__sigma-mcp-v2__describe the element (no type=error columns;
metric formulas resolve clean), then mcp__sigma-mcp-v2__query a raw aggregate and confirm it
matches the warehouse.
Phase 3 — Convert the dashboards (UDD = primary)
For each Looker dashboard, fetch its contract (Phase 1b), then build a Sigma workbook spec.
3a. Build the workbook spec
python3 scripts/build_workbook.py /tmp/<name>/<dash>.contract.json \
--views /path/to/lookml/views \
--dm-id <dataModelId> \
--element-id <denorm-element-id> \
--dm-element-name "<DM element display name>" \
--folder-id <writable-folder-id> \
--out /tmp/<name>/<dash>.workbook.json
(contract is positional. --dm-element-name is the display name of the data-model element
the master table pulls from; --master-name defaults to Data. The generated spec has
placeholder defaults for any flag you omit, so it always generates locally — fill in the real
ids before POSTing.)
build_workbook.py consumes the contract + the explore's view .lkml files (to classify each
view.field as a measure — agg + base col — or a dimension, and derive the Sigma formula) and
emits a /v2/workbooks/spec body:
- a hidden "Data" page with a master table sourced from the DM element,
- a dashboard page with one element per Looker tile,
- controls from the dashboard filters,
- a newspaper → 24-col grid layout XML string.
Tile-type, filter-type, and layout maps are in refs/dashboard-contract.md and
refs/looker-dashboard-layout.md — do not duplicate them; defer there. Summary:
Looker tile type: | Sigma kind |
|---|
single_value | kpi-chart |
looker_column | bar-chart (vertical) |
looker_bar | bar-chart + orientation: horizontal (Looker looker_bar = horizontal bars) |
looker_line | line-chart |
looker_area | area-chart |
looker_pie | pie-chart |
looker_donut_multiples | donut-chart (single ring) + warn |
looker_scatter | scatter-chart |
looker_grid / table | table |
text | text (markdown body) |
looker_map / geo / funnel / waterfall / boxplot / sankey / custom viz | none — approximate or drop + warn |
Table column order, labels & hidden columns. A table's Sigma column order follows the Looker
visualization order (vis_config.column_order, captured as contract columnOrder), NOT
query.fields — the Data-tab order, which forces dimensions before measures. Fields not listed in
column_order append in fields order (Looker appends newly-added fields at the end). Column
names prefer the viz label (vis_config.series_labels, captured as columnLabels) over the
humanized field name — a column renamed in the visualization can differ from the Data-tab name. Columns
hidden from the visualization (vis_config.hidden_fields, captured as hiddenFields) get
hidden: true on the Sigma column — but a hidden dimension is KEPT in groupings.groupBy so
the aggregation grain (and therefore every measure value) is unchanged. Never drop a hidden
dimension: Looker keeps it in the query ("Hide from Visualization" doesn't re-run the query), so
dropping it would silently change the numbers. Both are additive contract keys — absent/empty →
columns stay in fields order, byte-identical to before. Verified: tests/test_table_column_order.py.
DM metric references (leverage the semantic layer, don't duplicate it). A table/pivot measure
column prefers a governed [Metrics/<name>] reference over re-deriving the aggregation inline,
when the measure's inline aggregate matches a metric defined on (or inherited by) the source DM
element. Match is by FORMULA equivalence — strip the master prefix so Sum([Data/Net Revenue])
equals a metric's Sum([Net Revenue]) — so it's naming-independent and SAFE: ratios, filtered
measures, custom/ad-hoc measures, and any non-match fall back to the inline formula. migrate-looker
passes each element's referenceable metrics (name+formula) via --dm-elements, resolved through the
source.elementId chain (a denorm " View" element inherits its base fact's metrics — Sigma
exposes them, and [Metrics/<name>] resolves on the denorm through the master→element chain,
verified live). Absent metrics (the offline test/golden path) → inline, byte-identical. Verified:
tests/test_metric_reference.py.
Newspaper layout math (a single arithmetic transform, no spatial heuristic):
gridColumn = (col+1) / (col+1+width), gridRow = (row+1) / (row+1+height). tile / static
/ grid modes need a snap heuristic (lossy) — warn + stack; see refs/looker-dashboard-layout.md §3.
3b. Workbook-spec gotchas (learned the hard way)
/v2/workbooks/spec returns YAML — don't json.load the response.- control elements live in
page.elements[] with kind: control but REQUIRE an id
(separate from controlId); a missing id → Invalid kind: "control".
- KPI
value uses value.columnId on the live API (the sigma-workbooks
example-full.yaml shows value.id — the API wants columnId). BUT donut/pie value
uses value.id (not columnId) — the two element types genuinely differ; verified by POST
400s both ways.
- Chart
color channel differs by type: bar/area/line series = {by: "category", column: <id>}; donut/pie slice = {id: <id>, sort?}. (A Looker pivot maps to this color channel.)
- donut/pie use
value + color, NOT xAxis/yAxis.
- KPI comparison (
show_comparison) has NO spec slot — warn, don't build. (Recommend a 2nd
KPI tile or a UI delta post-publish.) Looker donut_multiples per-multiple dim is also dropped → warned.
- Master → DM-element refs: a master table sourcing a DM element references columns as
[<DM-element-NAME>/<col display>]; tiles then reference [<master-NAME>/<col display>].
- Joined-view columns in the denorm DM element are named
<Field> (<joinAlias>) (Sigma
disambiguates cross-element lookup cols) — master/tile refs must include the suffix, e.g.
[Order Fact/Region (customer_dim)].
- Cross-element relationship refs (reused relational DMs) — the form is LAYER-DEPENDENT.
When the converter builds the DM it flattens joined columns into one denorm element, so the
workbook only ever does
[<element>/<col>]. But when you reuse an existing DM whose
element reaches other columns through a relationships[] entry (instead of a flat denorm),
the ref form differs by where you write it (all verified live 2026-06-29):
- In a workbook formula (master/tile/chart column) →
[<SourceElement>/<RelationshipName>/<col display>]
— uses the relationship's name. e.g. [Plugs Fact/Plugs to Customer/Cust Region] ✅.
- In a DM calc column →
[<TargetElementName>/<col display>] (or Lookup([<TargetElementName>/<col>], <localKey>, [<TargetElementName>/<key>]))
— uses the target element's name, NOT the relationship name. e.g. [Customer Dim/Cust Region] ✅.
- Both layers reject the raw warehouse/staging table name (e.g.
STG_…/D_CUSTOMER)
→ the column resolves as type=error. The relationship name is whatever the DM author
set (often a phrase like "Plugs to Customer", not the table name) — read it from the
DM spec's relationships[].name; don't guess it from the table. Two-segment guesses like
[<RelationshipName>/<col>] hard-reject the whole spec POST ("dependency not found").
- VARIANT / JSON columns — extract with DOT NOTATION, not a function. Sigma has no
JsonExtractText() / CallVariant() / Variant()-extraction function (those error the
column — don't trial-and-error them). To pull a value out of a JSON/VARIANT column, write a
dot-notation formula and wrap it in Text() to land a typed column:
Text([Cust Json].AGE_GROUP), nested Text([Cust Json].LOYALTY_EXTRA.LOYALTY_TIER), array
Text([Cart Details].cart[0]) (0-based). Best practice is to extract upstream in the DM
(the extracted column then flows into the workbook as a normal column — verified live
2026-06-29); the UI equivalent is the column menu's Extract columns…. So a JSON field is
migratable — surface it as a dot-notation column, don't leave it as raw JSON and don't block
on it. (Variant()/Json() exist only to cast a column's type, not to extract.)
- Table calcs → workbook formula columns:
running_total → CumulativeSum,
pct_of_total/sum() → GrandTotal, offset(…,-1) → Lag. (build_workbook.py translates
these; dynamic_fields arrives JSON-parsed from discovery.)
- count on a joined view →
CountDistinct using that view's primary key (base-view counts
stay Count()).
- Number formats carry through. A LookML measure's
value_format_name (usd, usd_0,
percent_0/1/2, decimal_0/1/2, …) or custom value_format mask becomes the Sigma column
format object — {kind: "number", formatString: "<d3-format>"} — on the tile's value /
KPI-value / chart-measure / measure-table column. So a usd measure renders $110,342.75
(not bare 110,342.75) and a percent_1 measure renders 12.3%. build_workbook.py's
build_field_index captures each measure's format and apply_fmt attaches it; custom masks are
best-effort (currency symbol / thousands separator / decimals / percent). Counts and dimensions
get no format (raw). Without this the side-by-side render (Phase 4a) shows bare numbers where
Looker showed $/%.
- Bar orientation. Looker
looker_bar renders horizontal bars, looker_column vertical —
both map to a Sigma bar-chart. build_workbook.py emits orientation: horizontal for
looker_bar and omits the key (Sigma's vertical default) for looker_column. Field verified:
sigma-workbooks charts.md.
- Grid cell visualizations (
series_cell_visualizations). A Looker grid can draw in-cell bars
on a measure column, often colored by VALUE (low→high gradient). Sigma data bars are
SIGN-colored — one fill for positive, one for negative (verified live 2026-06-24: the
Format rule → Data bars UI exposes only a Negative color + Positive color, and a
multi-stop scheme collapses to the single positive color). So the bar fill cannot vary by
value. The mappings build_workbook.py emits from contract cellVisualizations: {field:{scheme}}:
- Looker bar colored by value (a
custom_colors palette) → Sigma Color scale
(conditionalFormats: [{type: backgroundScale, columnIds:[<calc col>], scheme:[…]}]) — tints
the cell low→high, reproducing Looker's value encoding — plus a warning (the bar+value-color
combo isn't reproducible; flip the rule to dataBars if you'd rather keep a magnitude bar).
- Looker plain bar (no value palette) →
conditionalFormats: [{type: dataBars, columnIds:[…]}]
(magnitude). Verified spec shapes: sigma-workbooks tables.md.
Render-only caveat: Looker frequently does not return series_cell_visualizations from the
dashboard/query API even when the rendered dashboard shows the bars (confirmed on dash 11 via the
query, result_maker, and dashboard_element endpoints — all empty). That case can't be
auto-detected from the contract, so the Phase-4a visual-QA gate is where you catch it: if the Looker
render shows value-colored in-cell bars but the Sigma table has none, add a backgroundScale
conditionalFormat by hand (sample the source colors for the scheme) and PUT the spec.
3c. POST the workbook + verify
POST the spec to /v2/workbooks/spec (returns YAML → record the workbookId). Then
mcp__sigma-mcp-v2__describe each element (no type=error columns) and confirm the layout
applied. POST is create-only — every subsequent spec edit MUST use PUT /v2/workbooks/{id}/spec; re-POSTing leaves orphan workbooks in My Documents (delete via DELETE /v2/files/{id}).
Phase 4 — Verify parity (3-way: Looker vs Sigma vs warehouse) — MANDATORY
A conversion is not complete until the numbers tie out. Compare at two grains: the model's
key metrics, and per-tile.
4-pre. The scripted gate (canonical — what migrate-looker.py runs)
ruby scripts/phase6-parity-looker.rb --workdir /tmp/<name> --workbook-id <wb>
ruby scripts/phase6-parity-looker.rb --workdir /tmp/<name> --finalize
ruby scripts/verify-anchors.rb --workdir /tmp/<name> --workbook-id <wb>
python3 scripts/visual-similarity.py --source /tmp/<name>/dashboards/looker-<dash>.png --render /tmp/<name>/sigma-<dash>.png --json-out /tmp/<name>/visual-similarity.json
ruby scripts/assert-phase6-ran.rb --workdir /tmp/<name> --workbook-id <wb>
The finalize pass writes the parity-final.json sentinel; assert-phase6-ran.rb
(hard gate, vendored byte-identical across the 5 plugins) refuses GREEN unless
parity ran + PASSed, no orphan workbooks were left, the live workbook has no
type=error columns, a real layout is applied, the layout lint passes (gate 6),
the control lint passes (gate 7 — dead/ghost/partial controls; see
refs/control-parity.md), the source-anchor values verify (gate 13 — needs
source-anchors.json ≥5 + a passing anchors-verdict.json; arms when a source
PNG is on disk; --skip-anchors-gate "<reason>"), and the visual-similarity
floor holds (gate 14 — visual-similarity.json; --skip-visual-similarity "<reason>");
no source PNG on disk → both self-SKIP (stated), never a silent pass. Optional runtime follow-up when controls exist:
ruby scripts/probe-controls.rb --workbook-id <wb> --check-out-of-closure
(flip test — in-closure export must change under a non-default control value,
out-of-closure must not). migrate-looker.py automates
both fetch sides and runs the gate for you. The manual 3-way checks below remain
the reference for what "parity" means.
- Looker —
POST /queries/run/json (or run_inline_query) for the model/explore, e.g.
net revenue by region.
- Sigma —
mcp__sigma-mcp-v2__query against the DM element (raw aggregate, since
metric() returns "Missing Metric") AND against each workbook chart element.
- Warehouse — the source-of-truth
SELECT (via the Sigma connection or snow).
GREEN only when all three match. The validated run produced exact parity to the cent —
region revenue (West 38906.82 / South 31650.98 / NE 21587.52 / MW 14966.20 / null 3231.23 =
$109,765.89) and the ratio metrics (AOV / margin / return) identical across Looker and Sigma.
4a. Visual QA — render BOTH dashboards to PNG and eyeball them side-by-side
Numbers tying out is necessary but not sufficient — a workbook can be GREEN on parity yet look
broken (hidden KPI titles, orphaned filters, overlapping tiles, the wrong chart kind, bare numbers
where Looker showed $/%). After POSTing, render both the Looker source dashboard and the
migrated Sigma workbook to PNG and inspect them side-by-side:
mkdir -p /tmp/<name>/dashboards
python3 scripts/looker-render-dashboard.py <dashboardId> /tmp/<name>/dashboards/looker-<dash>.png
bash -c 'eval "$(scripts/get-token.sh)" && python3 scripts/sigma-export-png.py \
--workbook <workbookId> --page page-dash --out /tmp/<name>/sigma-<dash>.png'
Read both PNGs and compare tile-for-tile. Confirm: KPI tile titles show (the builder lays KPIs
≥ 6 rows tall — a kpi-chart hides its title below ~5 rows / ~150px; see
feedback_sigma_kpi_label_height.md), the filters sit in a top control bar (not orphaned at
the bottom), tiles are aligned with no large empty regions, each chart kind matches Looker, and
number formats match — if Looker shows $176.85 the Sigma KPI must too (the builder carries
LookML value_format_name / value_format into the tile column format; see Phase 3). Iterate on
build_workbook.py + re-PUT the spec until the side-by-side render is clean.
Visual QA is a mandatory gate — never skip, never declare done on HTTP 200. A workbook that
POSTs cleanly and passes parity can still be visually broken (overlapping tiles, clipped KPI titles
below ~5 rows, dead zones, orphaned filters; Sigma's grid has no z-order). After rendering the
migrated pages with sigma-export-png.py (side-by-side vs looker-render-dashboard.py):
- Read each migrated PNG and check it against
refs/layout-visual-qa.md (no overlaps/stacking,
no dead zones, controls placed in-band, no clipped KPI titles, even heights, right chart kind/format).
- Fix any failure in the spec — for multi-page workbooks use
sigma-skills/sigma-workbooks/scripts/wb-rep.rb (pull → edit → push) — then re-render and re-read.
- Loop until the render passes inspection.
Record the RLS outcome here. If Phase 1d found RLS, the migration summary MUST list, per
finding, whether it was ported / reused / skipped (and the Sigma user attribute + filter used)
— so any skipped Looker restriction is visible to a reviewer, never silently dropped. (When RLS
is active, parity-check as a representative restricted user, not only as an admin who sees all
rows.)
If mcp__sigma-mcp-v2__query errors with an auth message mid-Phase-4, the MCP session
staled — re-call mcp__sigma-mcp-v2__begin_session and retry. Do not skip parity over a
recoverable auth error.
Phase 5 — Enhance (post-publish, UI-only features)
Some Looker features have no spec-API analog and must be wired in the Sigma UI after publish.
Set expectations up front (they appear as warnings from build_workbook.py):
- Cross-filtering (clicking a Looker bar filters siblings) → Sigma "Set as filter" actions —
UI-only.
- Trellis / small multiples —
looker_donut_multiples is now emitted NATIVELY: build_workbook.py
builds ONE Sigma donut-chart with the element trellis facet (the row dimension → columnsBy,
the pivot stays the slice), via the shared TrellisEmit (scripts/lib/trellis_emit.py). Not UI-only.
Sigma silently strips trellis only on UNSUPPORTED kinds (donut IS supported), so the round-trip
guard verify-trellis-survived.rb re-reads the readback and asserts the key survived, keyed off the
native-trellis-emitted.json sidecar. Pivoted CARTESIAN charts stay color SERIES (Looker renders
those pivots as series, not panels) — see docs/sigma-trellis-chart-support.md.
- Tooltips /
note_text / subtitle_text → no spec slot; concatenate into the chart title
or add an adjacent text element.
- KPI comparison (
show_comparison) → add a 2nd KPI tile or a UI delta.
- Pivot cross-tab → rebuild the flattened table as a Sigma
pivot-table in the UI.
- Per-tile refresh intervals → Sigma has workbook-level scheduled refresh only — drop + warn.
Gap scout — when the converter can't translate a LookML construct
When convert_lookml_to_sigma only approximates or drops a LookML measure/construct (a ratio /
${measure}-ref measure, a type: percentile, a filtered count, a Liquid {% parameter %}
measure), spawn the gap scout subagent to find a Sigma formula that resolves on the live site,
then persist it so the next dashboard reuses it. The full runbook (when to spawn, the spawn
prompt, the LookML→Sigma candidate table, opt-in issue filing) is in scripts/gap-scout.md —
read it before spawning.
scripts/scout-validate.py validates a candidate against a real DM element (builds a throwaway
test workbook, checks the column's resolved type, deletes it) and persists a win to
~/.looker-to-sigma/learned-rules.yaml (scripts/learned-rules.py loads + applies it).- On failure it returns an
escalation block. Filing a GitHub issue is opt-in /
confirm-before-file — run escalation.dry_run_cmd (files nothing; drafts the issue + dedupes),
show the user, and only run escalation.file_cmd (escalate-gap.py … --yes) if they accept.
LookML construct gaps are converter gaps and mirror to both converter repos with a bead.
This is also a lightweight way to validate a migrated DM/workbook: point scout-validate.py
at the denorm element to confirm a suspect formula resolves (no type:error column) before
declaring Phase 4 green.
Troubleshooting
| Error / symptom | Cause | Fix |
|---|
convert_dm.mjs output still has the old bug shape | Edited src/lookml.ts but the MCP server serves the deployed build | Run convert_dm.mjs via node --import tsx/esm against the patched src/ (or restart the MCP server) |
Converter dropped all view fields after an html: dim | Stale build predating BUG4 fix | Use the patched source path; the ;;-block pre-extraction now includes html/sql_on/etc. |
Metric formula contains ${...} literals or 0 * | Stale build predating BUG1/BUG3 fixes | Patched source resolves ${dim}/${measure} refs and preserves 1.0 |
metric() returns "Missing Metric" in a Sigma query | Known Sigma quirk | Verify via raw aggregate (Sum/CountDistinct), not metric() |
Source not found: warehouse table … on DM POST | (a) connection catalog hasn't indexed the schema yet, OR (b) the LookML sql_table_name DB.SCHEMA differs from what the connection serves, OR (c) short connectionId | post_dm.py now AUTO-SYNCs the named schema (POST /v2/connections/{id}/sync) and retries once — (a) self-heals. For (b) pass --source-swap FROM_DB.FROM_SCHEMA=TO_DB.TO_SCHEMA. For (c) use the FULL connection UUID. Last resort: a Custom SQL DM element (kind: "sql") |
jq: parse error: Invalid numeric literal | Sigma spec endpoints return YAML | Never pipe spec responses to jq / json.load |
Invalid kind: "control" on workbook POST | Control element missing its own id (separate from controlId) | Add a distinct id |
KPI POSTs 400 with value.id / donut POSTs 400 with value.columnId | The two element types use different value keys | KPI → value.columnId; donut/pie → value.id |
| Tile shows the wrong chart kind | Read element.type (always "vis") instead of query.vis_config.type | fetch_looker_dashboard.py already reads vis_config.type — re-fetch the contract |
| Looker LookML deploy fails "Invalid lookml syntax" | Compact { a: yes; b: yes; } params | Use multi-line blocks; only ;; terminates a sql |
| LookML model 404s on query right after deploy | Model not registered | POST /lookml_models {name, project_name, allowed_db_connection_names} |
PUT /projects/{id} 404 when setting git remote | Wrong verb | Use PATCH /projects/{id} |
| Looker dev-workspace mutation has no effect | The calls ran in separate processes/sessions (the bearer cache is per-process; a 401 re-login also starts a NEW session that resets the workspace to production) | Do the whole dev flow in ONE process — looker_api.py's cached token keeps one session, so PATCH /session {workspace_id: dev} sticks for subsequent call()s; re-PATCH after any forced re-login |
| Reused DM element builds via API but users can't select it as a source in the workbook UI | The element is visibleAsSource: false (hidden); the API doesn't enforce visibility on build | Wire to a visible sibling, or PUT the DM spec setting visibleAsSource: true. NB the flag defaults true and is omitted when true — only "visibleAsSource": false in the spec means hidden. Catch it in the Phase 2.5 shape preflight |
Related-element column resolves as type=error | Referenced the raw warehouse/staging table name, or used the wrong layer's form | Workbook formula → [<element>/<RelationshipName>/<col>] (relationship name); DM calc → [<TargetElementName>/<col>] or Lookup(…). Read the exact name from relationships[].name — never the table name (see 3b) |
| Reused relationship returns 2+ matches per row / inflated KPI & chart totals | The relationship key is incomplete (e.g. fact related to a dim on a non-unique attribute, or missing a 2nd key like Region) so it fans out — a non-unique target key multiplies fact rows silently (e.g. relating on a 5-value Region column vs a 4,972-row dim ≈ 994× inflation) | Relate on the dim's true primary key (or add the missing key to make the join 1:1) in the DM — manual. A single-key fan-out produces wrong aggregates with no error; flag it, don't ship |
Telemetry (after the final gate passes)
Tell the user this in the conversation before running anything:
"Migration complete. Before I wrap up, I'd like to send an anonymous usage ping so we can track which migration skills are being used. It records: tool name, your Sigma region, an anonymized org fingerprint (a hash of your client ID — not the credential itself), migration duration, and success. No workbook names, SQL, column names, or any customer data is included. See TELEMETRY.md for the exact payload. Just say 'skip' if you'd prefer not to send it."
If the user does not object, run:
python3 scripts/report-telemetry.py --tool looker-to-sigma --duration <elapsed_seconds> --workdir <run-dir> [--mode live|file|both]
