| name | fabric-ops |
| description | Deploy, monitor, teardown, and diagnose the Medical Device FHIR Integration Platform. USE FOR: running Deploy-All.ps1 (any phase), monitoring background deployments, tearing down environments, invoking/polling HDS pipelines, querying Silver/Bronze/Reporting lakehouses, deploying ontologies, rebuilding DICOM viewer index, diagnosing deployment failures. DO NOT USE FOR: general coding questions, writing new features unrelated to deployment ops. |
Fabric Ops — Deployment & Operations Skill
Workspace & Naming Convention
All deployments use a date-suffixed naming pattern:
- Workspace:
med-device-rti-hds-MMDD (e.g., med-device-rti-hds-0401)
- Resource Group:
rg-med-device-rti-MMDD (e.g., rg-med-device-rti-0401)
- Append
-N for same-day iterations (e.g., med-device-rti-hds-0401-2)
Deployment Commands
Full Deploy (Phase 1 → auto-detects HDS → Phase 2+3)
cd C:\git\med-device-fabric-emulator
.\Deploy-All.ps1 `
-FabricWorkspaceName 'med-device-rti-hds-MMDD' `
-ResourceGroupName 'rg-med-device-rti-MMDD' `
-Location eastus `
-AdminSecurityGroup sg-azure-admins `
-Tags @{SecurityControl='Ignore'}
Phase 2 Only (after HDS manual deploy)
.\Deploy-All.ps1 -Phase2 `
-ResourceGroupName "rg-med-device-rti-MMDD" `
-Location "eastus" `
-FabricWorkspaceName "med-device-rti-hds-MMDD" `
-Tags @{SecurityControl='Ignore'}
Phase 3 Only (Imaging Toolkit)
.\Deploy-All.ps1 -Phase3 `
-FabricWorkspaceName "med-device-rti-hds-MMDD" `
-ResourceGroupName "rg-med-device-rti-MMDD" `
-Location "eastus"
Phase 4 Only (Ontology + Agent Binding)
.\Deploy-All.ps1 -Phase4 `
-FabricWorkspaceName "med-device-rti-hds-MMDD" `
-Location "eastus"
Teardown
.\Teardown-All.ps1 `
-FabricWorkspaceName "med-device-rti-hds-MMDD" `
-ResourceGroupName "rg-med-device-rti-MMDD" `
-Force -Wait
Omit -Wait to teardown async (non-blocking). AHDS RG deletion takes 5-15 min.
Ontology Deploy
.\phase-4\deploy-ontology.ps1 -FabricWorkspaceName "med-device-rti-hds-MMDD"
Deployment Step Sequence
| Step | Phase | What | Duration |
|---|
| 1 | 1 | Fabric Workspace + Identity | ~0.2 min |
| 2 | 1 | Base Azure Infra (Event Hub, ACR, emulator) | ~3-4 min |
| 3 | 1 | FHIR Service + Synthea + Loader | ~17-20 min |
| 3b | 1 | DICOM Loader (TCIA → ADLS + ImagingStudy) | ~12-18 min |
| 4 | 1 | Fabric RTI Phase 1 (Eventhouse, KQL, Eventstream) | ~2 min |
| 5 | — | HDS Guidance (auto-detected or manual) | — |
| 6 | 2 | Phase 2 (KQL shortcuts, enriched alerts) | ~14-15 min |
| 6b | 2 | DICOM Shortcut + HDS Pipelines (sidecars→Clinical→CMA→Imaging→OMOP) | ~40-55 min |
| 7 | 2 | Data Agents (Patient 360 + Clinical Triage) | ~0.1 min |
| 8 | 3 | Phase 3 (Cohorting Agent, DICOM Viewer, PBI Report) | ~9-10 min |
| 9 | 4 | Phase 4 (Clinical pipeline check, Ontology, Agent binding, Activator) | ~4-5 min |
Total end-to-end: ~100-110 min (full auto-detection run)
Monitoring Background Deployments
When running deployments via run_in_terminal with isBackground=true, save the terminal ID and poll with get_terminal_output:
Terminal ID format: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx
Key patterns to watch for in output:
✓ = step succeeded✗ = step failed (check error message)⚠ = warning (non-fatal, may need attention)toomanyrequests = Docker Hub rate limit (use MCR base images)409 Conflict = resource name collision (retry with delay)SharedTokenCacheCredential = token auth failure (refresh token)
Local Orchestrator Web UI
- Start with
./Start-WebUI.ps1 -Force when existing local backend/frontend ports should be reclaimed; use ./Start-WebUI.ps1 -SelfTest for helper-only probe validation.
- Startup success requires backend
/api/live, frontend /, and the frontend /api/live proxy to respond. Probe failures are fatal and should be fixed before using the UI.
- Backend logs include
orchestrator/orchestrator.log, orchestrator/backend-stderr.log, and a per-run orchestrator/orchestrator-session-<timestamp>.log; orchestrator/backend-crash-dump.log appears only when backend crash diagnostics write it.
- Liveness/readiness split:
/api/live and default /api/health are cheap backend/database checks; /api/health?deep=1 adds Azure auth and Fabric capacity readiness.
- For Joey's BrakeKat browser smoke tests, use Microsoft Edge Profile 2 / Work - Brakekat (
/Users/joey/Desktop/Edge - Work - Brakekat.app) and confirm the loaded tab title/URL, not just process startup.
Querying Lakehouses via SQL
Connect to the Fabric SQL endpoint using the MSSQL extension (profile: hds-new or discover via Fabric API).
Discover SQL Endpoint
$t = (Get-AzAccessToken -ResourceUrl "https://api.fabric.microsoft.com").Token
if ($t -is [System.Security.SecureString]) { $t = $t | ConvertFrom-SecureString -AsPlainText }
$h = @{ Authorization = "Bearer $t" }
$ws = (Invoke-RestMethod -Uri "https://api.fabric.microsoft.com/v1/workspaces" -Headers $h).value |
Where-Object { $_.displayName -eq "WORKSPACE_NAME" }
$lh = (Invoke-RestMethod -Uri "https://api.fabric.microsoft.com/v1/workspaces/$($ws.id)/lakehouses" -Headers $h).value |
Where-Object { $_.displayName -match 'silver' }
$d = Invoke-RestMethod -Uri "https://api.fabric.microsoft.com/v1/workspaces/$($ws.id)/lakehouses/$($lh.id)" -Headers $h
Write-Host "Server: $($d.properties.sqlEndpointProperties.connectionString)"
Key Verification Queries (Silver)
SELECT 'Patient' as tbl, COUNT(*) as cnt FROM dbo.Patient
UNION ALL SELECT 'Device', COUNT(*) FROM dbo.Device
UNION ALL SELECT 'Condition', COUNT(*) FROM dbo.Condition
UNION ALL SELECT 'Encounter', COUNT(*) FROM dbo.Encounter
UNION ALL SELECT 'MedicationRequest', COUNT(*) FROM dbo.MedicationRequest
UNION ALL SELECT 'Observation', COUNT(*) FROM dbo.Observation
UNION ALL SELECT 'ImagingStudy', COUNT(*) FROM dbo.ImagingStudy
UNION ALL SELECT 'ImagingMetastore', COUNT(*) FROM dbo.ImagingMetastore
ORDER BY cnt DESC
Expected Row Counts (100 patients)
| Table | Expected |
|---|
| Observation | ~22K |
| Procedure | ~11K |
| ImagingMetastore | ~4K |
| Encounter | ~3.6K |
| Condition | ~2.6K |
| MedicationRequest | ~1.8K |
| Immunization | ~1.5K |
| Patient | 100 |
| Device | 100 |
| ImagingStudy | ~50-90 |
Invoking HDS Pipelines Manually
Discover and Invoke
$t = (Get-AzAccessToken -ResourceUrl "https://api.fabric.microsoft.com").Token
if ($t -is [System.Security.SecureString]) { $t = $t | ConvertFrom-SecureString -AsPlainText }
$h = @{ Authorization = "Bearer $t"; "Content-Type" = "application/json" }
$ws = (Invoke-RestMethod -Uri "https://api.fabric.microsoft.com/v1/workspaces" -Headers $h).value |
Where-Object { $_.displayName -eq "WORKSPACE_NAME" }
$wsId = $ws.id
# List pipelines
$pipelines = (Invoke-RestMethod -Uri "https://api.fabric.microsoft.com/v1/workspaces/$wsId/items?type=DataPipeline" -Headers $h).value
$pipelines | Select-Object displayName, id
# Invoke a pipeline
$pipelineId = "PIPELINE_ID"
Invoke-WebRequest -Method POST `
-Uri "https://api.fabric.microsoft.com/v1/workspaces/$wsId/items/$pipelineId/jobs/Pipeline/instances" `
-Headers $h -UseBasicParsing
Pipeline Names
| Pipeline | Purpose |
|---|
healthcare1_msft_clinical_data_foundation_ingestion | FHIR → Bronze → Silver (clinical flattening) |
healthcare1_msft_imaging_with_clinical_foundation_ingestion | DICOM + clinical imaging → Silver |
healthcare1_msft_omop_analytics | Silver → Gold OMOP (must run AFTER imaging + clinical) |
Pipeline Sequence Rule
Default deployment monitor order: optional SDoH/claims sidecars → Clinical (wait) → optional CMA (non-blocking) → Imaging (wait) → OMOP (wait). Sidecars are discovered from live DataPipeline items and invoked best-effort before the Clinical wait; CMA can start after Clinical/Silver readiness and does not wait for OMOP.
Poll Pipeline Status
$runs = (Invoke-RestMethod -Uri "https://api.fabric.microsoft.com/v1/workspaces/$wsId/items/$pipelineId/jobs/instances?limit=1" -Headers $h).value
$runs[0].status # NotStarted | InProgress | Completed | Failed | Cancelled
Docker Base Images
All Dockerfiles use MCR (no Docker Hub rate limits):
| Image | Base |
|---|
| Emulator | mcr.microsoft.com/cbl-mariner/base/python:3 |
| Synthea | mcr.microsoft.com/openjdk/jdk:17-ubuntu |
| FHIR Loader | mcr.microsoft.com/cbl-mariner/base/python:3 |
| DICOM Loader | mcr.microsoft.com/cbl-mariner/base/python:3 |
| DICOM Proxy | mcr.microsoft.com/cbl-mariner/base/python:3 |
Mariner notes: Uses tdnf (not apt-get), needs ln -sf /usr/bin/python3 /usr/bin/python symlink, ODBC driver package is msodbcsql18 with ENV ACCEPT_EULA=Y.
Common Issues & Fixes
| Issue | Cause | Fix |
|---|
toomanyrequests on ACR build | Docker Hub rate limit | Already fixed — all images use MCR |
SharedTokenCacheCredential error | Token expired or wrong scope | Refresh with Get-AzAccessToken -ResourceUrl |
| DICOM index = 0 studies | Proxy built before ImagingMetastore populated | Proxy has runtime SQL refresh; also auto-rebuilds in Step 3e |
| 409 Conflict on notebook create | Previous delete still propagating | Retry loop with 10s delay (already implemented) |
Condition_ table not found | Wrong table name in diagnostics | Table is Condition (no underscore) |
libgssapi_krb5.so.2 not found | Missing Kerberos lib in container | Install krb5-libs via tdnf (already fixed) |
| Silver tables appear missing | Fabric Lakehouse Tables API paginated/incomplete | Always verify via SQL endpoint, not REST API |
| PBI report needs OAuth | Direct Lake credential binding | Script attempts auto-bind; falls back to manual |
Teardown Validation
The teardown script validates both workspace and RG before deleting. If either is not found, it shows fuzzy "Did you mean?" suggestions and prompts for partial teardown. Use -Force to skip confirmation.
Key File Locations
| File | Purpose |
|---|
Deploy-All.ps1 | Main orchestrator (all phases) |
Teardown-All.ps1 | Cleanup orchestrator |
phase-1/deploy.ps1 | Phase 1 Azure infra |
phase-1/deploy-fhir.ps1 | FHIR + DICOM data pipeline |
deploy-fabric-rti.ps1 | Fabric RTI Phase 1 + 2 |
phase-2/deploy-data-agents.ps1 | Data Agents (Patient 360 + Clinical Triage) |
phase-4/deploy-ontology.ps1 | Fabric IQ Ontology |
phase-2/storage-access-trusted-workspace.ps1 | DICOM shortcut + HDS pipeline orchestration |
state-tracking/ | Deployment state tracking (gitignored) |
