| name | powerbi-ingest |
| description | Pipeline-facing Power BI ingestion adapter, wrapping scripts/powerbi_probe.py for two modes: `data` runs every dax-kind Binding Manifest binding via REST executeQueries (service-principal/token from ENV ONLY), returning a data.json-shaped fragment keyed by data_query.expression with source_period stamped -- a genuinely independent V1 recompute source for a PBI-sourced value. `shot` captures a fresh report image via ExportToFile (Playwright fallback), packaging it as the embedded image for a regenerate-class PBI node with source_period stamped so period-coherence can catch a stale screenshot. Fail-closed throughout: no creds/route/period => not_captured + the named fallback (manual-figure-labeled-unverified for data, user-provided-image for shot), never a fabricated value or period. Token never logged. Reach for this to turn a Binding Manifest's PBI-sourced bindings into pipeline artifacts. NOT for manifest construction (rebind-manifest) or running the fidelity harness itself (report-fidelity-harness). |
Skill: powerbi-ingest
The pipeline-facing adapter over the Phase-0 scripts/powerbi_probe.py probe.
Where the probe answers "can we reach Power BI at all" (a one-shot CLI receipt),
this skill answers the pipeline's actual question: given a Binding Manifest,
produce the artifact the next stage needs.
The honest guarantee (verbatim — never soften it): the plugin GUARANTEES only
(a) no old-client-data leak and (b) every low-confidence classification is
surfaced for human review. A successful data/shot capture is NOT a claim
the value is correct or the screenshot's period is current — those are the
fidelity harness's V1 and period-coherence legs, run downstream of this adapter.
Read first
Two pipeline-facing modes
data — manifest dax-kind bindings → new-data fragment
python3 powerbi_ingest.py data --manifest <manifest.json> \
[--source-period 2025-Q1] [--out <data-fragment.json>] \
[--receipt-out <receipt.json>] [--format json|text] [--pretty]
Runs every binding whose data_query.kind == "dax" through
powerbi_probe.execute_dax_query() (REST executeQueries, a fresh HTTP
round-trip per binding — never a cached result from manifest-build time) and
assembles:
{
"dataset_id": "<POWERBI_DATASET_ID>",
"period": "2025-Q1",
"values": {
"EVALUATE ROW(\"revenue_total\", [Revenue Total])": {"value": "4821300", "type": "number", "period": "2025-Q1"}
}
}
This is a data.json-shaped fragment, keyed by each binding's
data_query.expression — the exact key
report-fidelity-harness's V1 leg
(leg_v1()) indexes new-data by (expr_bindings.setdefault(dq["expression"], ...)).
Merge it into (or use it as) a --new-data file for the harness, and the PBI
value becomes a genuinely independent V1 recompute source — the fresh query, not
the binding's own inference path (contract §3).
period resolution (mirrors the harness's own leg_period() fallback exactly,
so this adapter's default agrees with what the harness would fall back to
anyway): --source-period if given, else the first non-null
provenance.source_period across the manifest's dax-kind bindings (document
order). If neither resolves, the run fails closed rather than stamping an
unattributed value.
shot — ExportToFile capture → regenerate-node embed
python3 powerbi_ingest.py shot --manifest <manifest.json> --node-id <id> \
[--source-period 2025-Q1] [--out <capture-out-path>] \
[--receipt-out <receipt.json>] [--format json|text] [--pretty]
Looks up --node-id in the manifest (fatal, exit 2, if absent — a caller bug),
resolves source_period (CLI override, else the binding's own
provenance.source_period; fails closed BEFORE any network call if neither
resolves — an unstamped screenshot cannot be period-coherence-guarded), then
calls powerbi_probe.probe_shot() unchanged (ExportToFile primary, Playwright
fallback) and wraps the result:
{
"verdict": "PASS",
"node_id": "pbi-screenshot",
"class": "regenerate",
"route": "export-to-file",
"path": "/tmp/....png",
"source_period": "2025-Q1",
"period_coherence_checked": false
}
class is always "regenerate" — the embed's own construction rule, matching
core-architecture-spec.md §4 ("any node that renders as a raster... MUST be
regenerate"), independent of whatever class the looked-up binding happens to
carry (a mismatch is surfaced as a non-fatal warnings entry, since manifest
classification correctness is rebind-manifest's job, not this adapter's).
period_coherence_checked stays false on every successful capture — this
adapter proves capture feasibility + period attribution, never period
correctness; that check is report-fidelity-harness's job.
Fail-closed contract (binding, inherited from the probe verbatim)
| Condition | Result |
|---|
No dax-kind bindings in the manifest | data → not_captured, no network call |
Missing POWERBI_ACCESS_TOKEN/WORKSPACE_ID/DATASET_ID | data → not_captured, fallback names the manual-figure-labeled-unverified path (contract §3) |
No resolvable source_period | data/shot → not_captured before any network call — never a guessed period |
A DAX result is NULL, or a query returns >1 row | that binding → failed, never a silently-picked/fabricated value |
| Neither ExportToFile nor Playwright succeeds | shot → not_captured, fallback names the user-provided-image path (contract §4/§5) |
| Some dax bindings succeed, some fail | data → verdict: "PARTIAL" (mirrors the harness's own PASS/PARTIAL/not_captured vocabulary) |
Tokens are read from POWERBI_ACCESS_TOKEN / POWERBI_WORKSPACE_ID /
POWERBI_DATASET_ID / POWERBI_REPORT_ID env vars only (identical to the
probe) and are never printed, logged, or embedded in any returned or written
structure — proven in tests/test_powerbi_ingest.py by asserting the fake token
is absent from stdout, the written fragment, and the written receipt.
What this adapter deliberately does NOT do
- Validate the manifest against
binding-manifest.schema.json (rebind-manifest's
job) — this stage duck-types only the shape it needs.
- Run the period-coherence check itself (
report-fidelity-harness's job).
- Support more than one target semantic model per invocation
(
POWERBI_WORKSPACE_ID/POWERBI_DATASET_ID). A dax-kind binding whose
data_query.source_ref disagrees with the env dataset is not an error
(source_ref may be a label, not a literal GUID) but is surfaced under
source_ref_mismatches for a human to check — see "what breaks next" below.
What breaks next
- Percent-fraction ambiguity. DAX often returns a percent-formatted measure as
a raw 0–1 fraction (e.g.
0.187), while the template may render "18.7%".
infer_type_and_value() deliberately does NOT guess a ×100 conversion from
magnitude or column-name heuristics (that would be an unverified assumption
baked into a "verifiable" source) — it labels every numeric DAX result "number"
and passes the raw magnitude through. If the regenerate stage later renders that
raw fraction as a percent string, V1's canon_number() comparison will disagree
even though the underlying figure is correct. Detect it: a data run reports
PASS but the downstream report-fidelity-harness V1 leg fails on a percent-typed
node with a 100x-off magnitude in its evidence string.
- Multiple semantic models in one manifest. Only one
POWERBI_DATASET_ID is
queried per invocation; check source_ref_mismatches in the receipt if a
manifest spans more than one PBI model.
Constraints
Stdlib only (argparse / json / os / re / sys / pathlib), Python 3.9.6,
no pip, no third-party imports beyond the sibling powerbi_probe module. No
network at import — every network-capable branch is inside ingest_data() /
ingest_shot(), invoked only from main(). Path-guarded (rejects ..
traversal). Does NOT auto-run.
Tests
python3 -m unittest tests.test_powerbi_ingest -v # run from this skill directory
A stdlib http.server mock stands in for the three Power BI REST endpoints this
adapter drives (executeQueries, ExportTo trigger, poll, file download) —
proving the data + shot flows end-to-end offline: real HTTP round-trips
against a loopback mock, a genuine Running → Succeeded poll loop for shot, a
partial-failure (PARTIAL) DAX scenario, every fail-closed-before-network path
(missing creds, missing period, no dax bindings, bad --node-id), and the token
never appearing in stdout / the written fragment / the written receipt. Also
regression-tests that the scripts/powerbi_probe.py refactor (extracting the
shared execute_dax_query()) did not change probe_data()'s own behavior.
