fabric-cicd

# fabric-cicd (Python deployment library) Microsoft's official open-source Python library for **code-first CI/CD into Fabric workspaces**. It abstracts the Fabric REST item-definition APIs: point it at a Git checkout of a Fabric Git-synced workspace folder and it publishes every in-scope item into the target workspace. **Version / install:** current is v1.3.x (August 2026). `pip install fabric-cicd` (or `uv add fabric-cicd`). Python 3.9–3.13. Docs: [microsoft.github.io/fabric-cicd](https://microsoft.github.io/fabric-cicd/latest/) · repo: [github.com/microsoft/fabric-cicd](https://github.com/microsoft/fabric-cicd). **v1.0 (April 2026) was a breaking release**: `token_credential` became **required** — the `DefaultAzureCredential` fallback and implicit Fabric-notebook auth were removed. Any pre-1.0 sample that omits the credential no longer runs. ## Which deployment surface am I on? | Surface | Source of truth | When | |---|---|---| | **fabric-cicd** (this skill) | Git | Python-scripted deploys in ADO / GitHub Actions / notebooks; finest control (feature flags, orphan cleanup, selective publish) | | `fab deploy --config config.yml` | Git | Same engine, CLI wrapper — one-command deploys; see fabric-cli skill | | Fabric deployment pipelines (service-side) | Workspace | Dev workspace promoted stage-to-stage in the portal / REST; no local code involved | `fab deploy` **wraps fabric-cicd** and consumes the same `config.yml` / `parameter.yml`. Don't mix Git-driven deploys and service-side deployment pipelines on the same workspaces. Decision guide: [Choose the best Fabric CI/CD workflow](https://learn.microsoft.com/fabric/cicd/manage-deployment). **Deployment pipelines don't rewrite OneLake image URLs.** A report whose visuals source images from OneLake carries the *workspace GUID* inside the URL itself: ```http https://onelake.dfs.fabric.microsoft.com/<workspace-guid>/<item-guid>/Files/<path>/<file-name> ``` Promote that report dev → test → prod through a **service-side deployment pipeline** and it keeps loading its images from the **original** workspace. Nothing fails and nothing warns — the report renders, the images appear — until access to the source workspace is revoked or it is deleted, long after the promotion. Upstream's remedies are to parameterize the URLs or rewrite them after deployment. Two bounds on that caveat, both deliberate: - The "parameters" upstream names are **deployment-pipeline** parameters. Do **not** assume this skill's `parameter.yml` `find_replace` is the binding — it takes a `file_path` glob and so *could* reach a report definition, but that it does for this case is unverified, and `parameter.yml` belongs to the Git-driven path rather than to deployment pipelines. - The claim is scoped to deployment pipelines, which is all upstream states. Whether Git integration, `fab deploy`, or fabric-cicd itself rewrite these URLs is **undocumented** — that is not the same as established to be "no". Reference: https://learn.microsoft.com/power-bi/visuals/power-bi-onelake-files ## Deployment model - **Full deployment every run** — no commit-diff inspection. The target workspace converges to the repository state; drift is overwritten. (`get_changed_items(repository_directory, git_compare_ref="HEAD~1")` exists if you want to scope a run yourself.) - Source layout = what Fabric Git integration writes: `<item-name>.<ItemType>/` folders (plus optional workspace subfolders) and an optional `parameter.yml` / `config.yml` at the repository-directory root. Don't hand-craft item folders — commit them from the portal. - Only item types with source-control + public-API support are deployable (34 types as of v1.3). - Deploys into the tenant of the executing identity. ## Quick start ```python from azure.identity import AzureCliCredential from fabric_cicd import FabricWorkspace, publish_all_items, unpublish_all_orphan_items workspace = FabricWorkspace( workspace_id="<target-workspace-guid>", # or workspace_name= environment="PROD", # must match parameter.yml env keys repository_directory="<path-to-workspace-folder>", item_type_in_scope=["Notebook", "DataPipeline", "Environment"], # omit → all types token_credential=AzureCliCredential(), # REQUIRED since v1.0 ) publish_all_items(workspace) unpublish_all_orphan_items(workspace) # delete workspace items not in repo ``` ## Core API ```python FabricWorkspace(*, repository_directory, token_credential, item_type_in_scope=None, environment="N/A", workspace_id=None, workspace_name=None) # workspace_id takes precedence over workspace_name; one is required. publish_all_items(ws, item_name_exclude_regex=None, folder_path_exclude_regex=None, folder_path_to_include=None, items_to_include=None, # ["Name.ItemType", ...] shortcut_exclude_regex=None) unpublish_all_orphan_items(ws, item_name_exclude_regex="^$", # default excludes nothing items_to_include=None) deploy_with_config(config_file_path, *, token_credential, environment="N/A", config_override=None) # → DeploymentResult append_feature_flag(flag) / remove_feature_flag(flag) / get_supported_feature_flags() change_log_level("DEBUG") configure_fabric_fqdn(workspace_id) # private-link workspaces — call BEFORE FabricWorkspace get_changed_items(repository_directory, git_compare_ref="HEAD~1") ``` The selective-publish parameters (`items_to_include`, `folder_path_*`, `shortcut_exclude_regex`) require their **experimental feature flags** (below) or they're ignored. ## Authentication Pass any `azure.identity` `TokenCredential`: | Context | Credential | |---|---| | Local dev | `AzureCliCredential()` / `AzurePowerShellCredential()` | | ADO / GitHub SPN | `ClientSecretCredential(client_id=..., client_secret=..., tenant_id=...)` | | OIDC / federated | `WorkloadIdentityCredential()` | | Self-hosted agent / Azure VM | `ManagedIdentityCredential()` | | Fabric notebook | custom wrapper below | Running **inside a Fabric notebook** (implicit auth was removed): ```python import time from azure.core.credentials import TokenCredential, AccessToken class FabricNotebookCredential(TokenCredential): def get_token(self, *scopes, **kwargs): return AccessToken(notebookutils.credentials.getToken("pbi"), int(time.time()) + 3600) ``` The identity needs Contributor+ on the target workspace (Admin for `enable_hard_delete`). SPNs additionally need the tenant setting *Service principals can use Fabric APIs* — same prerequisites as any Fabric REST automation (see fabric-auth skill). ## parameter.yml Environment-specific value substitution applied to file contents at publish time, keyed by the `environment` passed to `FabricWorkspace`. Lives at the repository-directory root. ### find_replace — string / regex replacement ```yaml find_replace: - find_value: "123e4567-e89b-12d3-a456-426614174000" # dev lakehouse GUID as committed replace_value: PPE: "f47ac10b-58cc-4372-a567-0e02b2c3d479" PROD: "9b2e5f4c-8d3a-4f1b-9c3e-2d5b6e4a7f8c" item_type: "Notebook" # optional filters item_name: ["Hello World"] # exact, case-sensitive file_path: "**/notebook-content.py" # abs / relative / glob ``` Optional: `is_regex: "true"` (then `find_value` is a regex; **capture group 1** is what gets replaced — include surrounding context in the pattern), `ignore_case: "true"`. ```yaml # Re-point a notebook's default lakehouse to the target workspace's copy - find_value: \#\s*META\s+"default_lakehouse":\s*"([0-9a-fA-F-]{36})" replace_value: _ALL_: "$items.Lakehouse.Example_LH.$id" is_regex: "true" item_type: "Notebook" ``` ### key_value_replace — JSONPath-targeted replacement ```yaml key_value_replace: - find_key: $.properties.activities[?(@.name=="Load_Intake")].typeProperties.source.datasetSettings.externalReferences.connection replace_value: PPE: "6c517159-d27a-41d5-b71e-ca1ecff6542b" PROD: "0f2ef3d1-1f34-4e0e-9483-6c8dbc077514" item_type: "DataPipeline" ``` ### spark_pool — Environment-item pool remapping ```yaml spark_pool: - instance_pool_id: "72c68dbc-0775-4d59-909d-a47896f4573b" replace_value: PPE: { type: "Capacity", name: "CapacityPool_Medium" } PROD: { type: "Capacity", name: "CapacityPool_Large" } ``` ### semantic_model_binding — post-deploy connection binding ```yaml semantic_model_binding: default: connection_id: _ALL_: "<connection-guid>" # or per-env keys; string or list models: - semantic_model_name: "Sales Model" connection_id: PPE: "<guid>" PROD: "<guid>" ``` ### Dynamic replacement variables Usable in `replace_value` (not combinable with `is_regex`): | Variable | Resolves to | |---|---| | `$workspace.$id` / `$workspace.$name` / `$workspace.$name_encoded` | Target workspace id / name / URL-encoded name | | `$workspace.<name>.$id` | Another workspace's id by name | | `$items.<Type>.<name>.$id` | Deployed item's id (type/name **case-sensitive**, attribute lowercase) | | `$items.<Type>.<name>.$sqlendpoint` | Lakehouse / Warehouse / MirroredDatabase / SQLDatabase connection string | | `$items.<Type>.<name>.$sqlendpointid` | Lakehouse / MirroredDatabase endpoint id | | `$items.Eventhouse.<name>.$queryserviceuri` | Eventhouse query URI | ### Other mechanics - `_ALL_` env key = same replacement for every environment. - `$ENV:var_name` in `replace_value` reads a pipeline/OS environment variable — requires the `enable_environment_variable_replacement` feature flag. - `extend:` — list of relative paths to split parameter files into templates. - Validation runs automatically at deploy start (deployment halts on failure); pre-validate with `devtools/debug_parameterization.py`. - Dynamic variables **disable bulk publish mode**, and using any of them triggers eager SQL-endpoint resolution for all Lakehouses/Warehouses in the target workspace. ## Config-file deployment (`deploy_with_config`) Single-call deployment driven by a `config.yml` — the same file `fab deploy` consumes. **Every field accepts either a scalar (all envs) or a per-env mapping** (`dev:` / `test:` / `prod:`). ```yaml core: workspace_id: # or workspace: <name>; id wins dev: "8b6e2c7a-..." prod: "7c3e1f8b-..." repository_directory: "." # relative to config.yml item_types_in_scope: [Notebook, DataPipeline, Environment, Lakehouse] parameter: "parameter.yml" publish: # optional; omitted → publish everything exclude_regex: "^DONT_DEPLOY.*" folder_exclude_regex: { dev: "^/DONT_DEPLOY_FOLDER" } # flag-gated; mutually folder_path_to_include: { prod: ["/DEPLOY_FOLDER"] } # exclusive per env items_to_include: ["Hello World.Notebook"] # flag-gated skip: { dev: true, test: false, prod: false } unpublish: # optional; omitted → orphans ARE unpublished exclude_regex: "^DEBUG.*" skip: { prod: true } features: # feature flags to enable - enable_shortcut_publish constants: # override fabric_cicd.constants values DEFAULT_API_ROOT_URL: "https://api.fabric.microsoft.com" ``` ```python deploy_with_config("config.yml", token_credential=cred, environment="prod", config_override={"publish": {"skip": {"prod": False}}}) ``` ## Feature flags (`append_feature_flag`) | Flag | Effect | |---|---| | `enable_lakehouse_unpublish` / `enable_warehouse_unpublish` / `enable_sqldatabase_unpublish` / `enable_eventhouse_unpublish` / `enable_kqldatabase_unpublish` | Allow orphan deletion of **data-bearing items** — off by default as data-loss protection | | `enable_hard_delete` | Bypass workspace recycle bin on unpublish; requires workspace **Admin** | | `enable_shortcut_publish` / `continue_on_shortcut_failure` | Deploy Lakehouse shortcuts / tolerate shortcut failures | | `disable_workspace_folder_publish` | Don't create workspace subfolders | | `enable_environment_variable_replacement` | Activate `$ENV:` in parameter.yml | | `enable_response_collection` | `publish_all_items` returns collected API responses | | `enable_experimental_features` + `enable_bulk_publish` | Single bulk-import API call instead of per-item (beta; non-prod) | | `enable_experimental_features` + `enable_items_to_include` / `enable_exclude_folder` / `enable_include_folder` / `enable_shortcut_exclude` | Activate the selective-publish parameters | Enumerate at runtime: `get_supported_feature_flags()`. ## Item-type caveats (the ones that bite) - **Warehouse / SQL Database**: **shell only** — no tables/views/procs deployed. Pair with SqlPackage/dacpac for schema (see fabric-database skill). - **Lakehouse**: shell + (flag-gated) shortcuts; schemas only deployed when a schema contains a shortcut. Deletion blocked unless `enable_lakehouse_unpublish`. - **Notebook**: `.py` and `.ipynb` supported; attached-lakehouse GUIDs need parameterization (regex example above); notebook resources aren't source-controlled. - **Environment**: custom pool references need the `spark_pool` section; resources not source-controlled. - **Semantic Model / Report**: use `semantic_model_binding` for connections; report→model rebinding handled when both deploy together.