databricks-solutions

End-to-end orchestrator for designing complete Gold layer schemas with ERDs, YAML files, lineage tracking, and comprehensive business documentation. Guides users through dimensional modeling, ERD creation (master/domain/summary based on table count), YAML…

Cross-validation of Gold layer design artifacts during the design phase. Use when validating that YAML schemas, ERDs, lineage CSVs, and PK/FK references are internally consistent before handing off to implementation. Catches design-time inconsistencies (e.g.,…

Active, source-bounded alignment of a Gold dimensional design to Databricks Industry Vibe Data Models (and other canonical industry reference models — TM Forum SID, ARTS, ACORD, HL7, BIAN). Use when the customer operates in a recognizable vertical (retail,…

Intelligent skill navigation system with tiered loading and orchestrator-first routing for context-efficient agent operation. Routes tasks to the correct domain skill based on keyword detection with orchestrator priority. Each skill uses progressive…

Deploy a Databricks AppKit application to Databricks Apps. Covers config validation, build verification, deployment, UI verification, error diagnosis with fix loop, and workspace app limit handling. Use when asked to deploy an AppKit app, push to production,…

Add plugins to an existing Databricks AppKit project. Covers Lakebase (PostgreSQL), Analytics (SQL queries + dashboards), Genie (natural language AI/BI), Files (UC Volumes), and Serving (Model Serving / Agent endpoints). Guides through plugin registration,…

Enables agent self-reflection and learning from mistakes through systematic skill updates. Prioritizes updating existing skills over creating new ones - always searches existing skills first, creates new skills only when justified. Includes upstream source…

Systematic skill freshness auditing with verification anchors, volatility classification, staleness detection, and upstream lineage tracking across every Agent Skill in the repository (data_product_accelerator/, genai-agents/, apps_lakebase/). Fetches…

Use this when you need to EVALUATE OR IMPROVE or OPTIMIZE an existing LLM agent's output quality - including improving tool selection accuracy, answer quality, reducing costs, or fixing issues where the agent gives wrong/incomplete responses. Evaluates agents…

Add memory capabilities to your agent. Use when: (1) User asks about 'memory', 'state', 'remember', 'conversation history', (2) Want to persist conversations or user preferences, (3) Adding checkpointing or long-term storage.

Analyzes a single MLflow trace to answer a user query about it. Use when the user provides a trace ID and asks to debug, investigate, find issues, root-cause errors, understand behavior, or analyze quality. Triggers on "analyze this trace", "what went wrong…

Build apps on Databricks Apps platform. Use when asked to create dashboards, data apps, analytics tools, or visualizations. Evaluates data access patterns (analytics vs Lakebase synced tables) before scaffolding. Invoke BEFORE starting implementation.

Databricks CLI operations: auth, profiles, data exploration, and bundles. Contains up-to-date guidelines for Databricks-related CLI tasks.

Create, configure, validate, deploy, run, and manage Declarative Automation Bundles (DABs, formerly Databricks Asset Bundles). Use when working with Databricks resources via DABs including dashboards, jobs, pipelines, alerts, volumes, and apps.

Databricks Lakebase Postgres: projects, scaling, connectivity, Lakebase synced tables, and Data API. Use when asked about Lakebase databases, OLTP storage, or connecting apps to Postgres on Databricks.

Manage Databricks Model Serving endpoints via CLI. Use when asked to create, configure, query, or manage model serving endpoints for LLM inference, custom models, or external models.

Build PWAF-compliant ISV integrations with Databricks: OAuth, telemetry (User-Agent), Unity Catalog, JDBC, SDK, SQL drivers, REST API, Databricks Connect.

PWAF-compliant Python SQL Connector (databricks-sql-connector): PAT, OAuth M2M, OAuth U2M (custom OAuth app PKCE + token-env), credentials_provider patterns, error handling, retry logic. Use when building Python integrations that run SQL queries via a…

Add a Databricks connector to an existing project that has no Databricks integration. Use when your product already exists and you want to add Databricks as a new data source or backend.

How to write a build_report.md for any PWAF connector. Captures skill traceability, sufficiency assessment, and test error/fix log. Use after implementing any connector.

How to structure a Databricks connector (REST or Python SDK): config, connect, operations, validation. Use when designing or building a new connector.

How to build and use a PWAF connector test runner (tests/run_all_tests.sh). Covers env isolation, auth types, single connector or single auth mode, parallel execution, browser tests, and report generation.

PWAF-compliant Databricks Connect (Python): PAT, OAuth M2M, OAuth U2M; serverless and classic compute. Use when building or testing Spark-over-Connect integrations.

PWAF-compliant Databricks SDK for Go (databricks-sdk-go): PAT, OAuth M2M, U2M token-env, U2M custom OAuth app (PKCE); useragent.WithProduct/WithPartner. Use when building or testing Go SDK workspace API integrations.

Analyze Enterprise Analytics & BI adoption from system.query.history — active users, query volumes, warehouse utilization, and slow-query hotspots. Ask me which dashboards/warehouses are busiest or where query performance is degrading.

Track Advanced Analytics model performance and inference volumes from ML monitoring tables — accuracy/drift metrics by model and version, plus serving request volumes. Ask me which models are degrading or which endpoints are busiest.

Report Unity Catalog governance coverage — table comments, ownership, tags, and column documentation — from system.information_schema. Ask me which tables lack owners, descriptions, or classification tags across a catalog.

Recommend Databricks cost-optimization actions for the FinOps/platform team — commitment/discount coverage, budget variance, and forecast-vs-actual DBU spend from system.billing.usage and system.billing.list_prices.

Discover data across Unity Catalog — find tables by name or topic, inspect columns and comments, and trace where data lives — from system.information_schema. An enterprise-wide skill for locating the right table before you query it, in any workspace.

A personal sandbox skill used as the annotated authoring template for this reference implementation. It shows the agentskills.io-recommended SKILL.md structure. Use it when learning how to author a new skill.

Analyze Financial Applications budget-vs-actual variance by cost center and account from aggregate financial ledger tables. Ask me where spend is over or under budget, variance trends, or the biggest variance drivers.

Analyze Databricks DBU cost trends, identify top-spending jobs, and recommend cost optimizations using system.billing.usage and system.lakeflow.jobs. Ask me about spend by SKU, job-level costs, or DBU consumption trends over time.

Reference for the Databricks MLOps Quickstart repo structure, parameterization contract, and Challenger/Champion conventions. Use whenever the user asks about repo layout, naming patterns, three-level Unity Catalog references, notebook widgets, MLflow…

Adapt the MLOps Quickstart Databricks Asset Bundle (databricks.yml, resources/*.yml) and CI/CD pipelines (Azure DevOps, GitHub Actions). Use when the user wants to rename the bundle, change catalog/schema per environment, add a staging target, configure…

Adapt the MLOps Quickstart data ingestion notebook from the Iris placeholder to a custom data source (cloud storage, JDBC, API, Delta table, etc.). Use when the user wants to ingest their own dataset, swap the source data, rename the feature table, add…

Adapt the MLOps Quickstart inference notebooks (batch and realtime / serving endpoint) to a custom model and input schema. Use when the user wants to run batch scoring on new data, query the realtime serving endpoint, change the inference output table, or…

Adapt the MLOps Quickstart MLflow 3 deployment pipeline (evaluate → approve → deploy) to a custom model. Use when the user wants to change evaluation metrics or thresholds, customize the human-in-the-loop approval gate, adjust the serving endpoint…

Adapt the MLOps Quickstart model training notebook from the Iris classifier placeholder to a custom algorithm and problem type (regression, clustering, classification, forecasting). Use when the user wants to change the model, features, target, metrics, or…

Manage Python dependencies for the MLOps Quickstart via the root `requirements.txt` file. Use when the user wants to add, remove, or upgrade a Python package, pin or unpin a library version, fix a dependency conflict, or understand how notebooks install their…

Testing framework for evaluating Databricks skills. Use when building test cases for skills, running skill evaluations, comparing skill versions, or creating ground truth datasets with the Generate-Review-Promote (GRP) pipeline. Triggers include "test skill",…

Evaluates whether the agent selected appropriate MCP tools instead of shell workarounds. Load when the trace contains Bash tool calls that could have used MCP tools, or when evaluating tool call efficiency.

Databricks development guidance including Python SDK, Databricks Connect, CLI, and REST API. Use when working with databricks-sdk, databricks-connect, or Databricks APIs.

General response quality evaluation. Always applicable regardless of domain. Covers response structure, actionability, clarity, and hallucination detection.

SQL evaluation criteria for Databricks. Load when the trace contains execute_sql tool calls or SQL code in responses. Covers syntax validity, Unity Catalog patterns, and Databricks-specific SQL features.

Python development guidance with code quality standards, error handling, testing practices, and environment management. Use when writing, reviewing, or modifying Python code (.py files) or Jupyter notebooks (.ipynb files).

MANDATORY post-deployment verification. After any workspace + Unity Catalog deployment, you MUST run all THREE compute paths against a UC table — classic cluster, serverless SQL warehouse, serverless notebook job. Skipping any of the three is incomplete work.…

Manage Databricks identity and governance. Use when the user asks to create groups, users, service principals, set up RBAC, manage permissions, workspace assignments, or configure access control.

Provision and test Databricks workspaces. Use when the user asks to create a workspace, set up a new environment, provision infrastructure, bootstrap Databricks, test a workspace, verify a deployment, or run validation checks against a Databricks workspace.…

Set up private networking for Databricks. Use when the user asks about private link, hub-spoke architecture, NCC (Network Connectivity Configuration), serverless connectivity to private resources, VPC endpoints, or private endpoints.

Set up Unity Catalog on Databricks workspaces. Use when the user asks to configure Unity Catalog, create a metastore, set up catalogs, schemas, external locations, storage credentials, or configure data governance. Covers Azure, AWS, and GCP.

Configure Databricks workspace settings. Use when the user asks to create SQL warehouses, cluster policies, secret scopes, IP access lists, manage tokens, update workspace settings, or destroy infrastructure.

Use when the user wants their firsthand knowledge of building, deploying, or running this project with real customers, users, or developers turned into published documentation — e.g. "interview me for docs", "extract what I know", "help me document this so…

Audit and improve spec coverage for a given spec. Use when (1) a spec has low or 0% requirement coverage, (2) tests exist but lack @req tags, (3) code behaviors have drifted from the spec's success criteria, (4) you need to identify unspecified behaviors in…

You MUST use this before any creative work — creating features, building components, adding functionality, or modifying behavior. Starts from existing specs rather than scratch. Use when the user asks to build, add, change, or design anything, even if it…

Code verification and testing for the Human Evaluation Workshop. Use when (1) running tests after code changes, (2) writing new unit tests (pytest/vitest), (3) writing E2E tests with Playwright/TestScenario, (4) debugging test failures, (5) understanding what…

Use when you have a spec or requirements for a multi-step task, before touching code. Creates spec-linked implementation plans with TDD steps, exact file paths, and spec coverage tracking. Use this after brainstorming, when a user says 'plan this', 'how…

MLflow 3 GenAI evaluation for agent development. Use when (1) writing mlflow.genai.evaluate() code, (2) creating @scorer functions, (3) building evaluation datasets from traces, (4) using built-in scorers (Guidelines, Correctness, Safety,…

Operate and troubleshoot the Lakebase (Autoscaling Postgres) memory component in an AgentOps Stacks project — check endpoint health, mint credentials, inspect or clear checkpointer (short-term) and store (long-term) tables, manage the Postgres schema. Use…

Scaffold a new AgentOps Stacks project — a multi-agent LangGraph bundle (DAB) with shared components, per-agent Databricks Apps, evaluation, and CI/CD. Use when the user wants to start a new AI agent project on Databricks. Triggers on "scaffold a new agentops…

Operate and troubleshoot the UC Functions tools component in an AgentOps Stacks project — register new functions, update EXECUTE grants, list available functions, re-run the registration job. Use when the user needs to add, update, or grant access to UC…

Operate and troubleshoot the Vector Search component in an AgentOps Stacks project — check index status, trigger sync, test the retriever, update the DLT data pipeline. Use when the user needs to manage their VS index or debug retrieval quality.

Add a new agent to an existing AgentOps Stacks project. Creates the agent folder, app resource, experiment, and manifest entry. Triggers on "add agent", "new agent", "create another agent", "add-agent".

Guide an agentops-stacks project through its full production lifecycle — data preparation, agent development, evaluation gates, CI/CD promotion, and production monitoring — following the Single-Account Single-Agent pattern from the Big Book of AgentOps. Use…

Add or modify a BFF (Backend-For-Frontend) aggregation route in hc-portal. Use when the user asks to "add a BFF endpoint", "join data across services", "compose a view that needs patient + appointment + lab", "fan out to multiple services", "build a portal…

Deploy services to dev/test/prod via Databricks Asset Bundles. Use when the user asks to "deploy", "promote", "deploy to dev/test/prod", "run bundle deploy", "validate the bundle", "add a resource to the bundle", "set a DAB variable", or anything about DAB…

Apply GitFlow rules to this repo and map them onto the CI/CD pipelines. Use when the user asks to "open a PR", "cut a release", "create a hotfix", "what branch should this go on", "promote to test/prod", "what does pr-validate do", or anything about the four…

Manage Lakebase database branches that mirror code branches. Use when the user asks to "create a Lakebase branch", "spin up a feature DB", "branch the database for this feature", "tear down feature branch", "list active branches", or anything about Lakebase…

Scaffold a new backend microservice in this healthcare reference architecture. Use when the user asks to "add a new service", "scaffold X service", "create a new microservice for Y", "bootstrap a new service", or "add a service called <name>". Wires APX…

Wire OBO (On-Behalf-Of) authentication into a service or BFF route. Use when the user asks to "add a route", "wire OBO", "set up authentication", "connect to Lakebase as the user", "extract user identity", or anything about user-passthrough auth. Enforces the…

Opinionated git-Lakebase branch-pairing workflows. Use when scaffolding a Lakebase-paired project, creating/deleting Lakebase branches in lockstep with git branches, diffing parent-aware schemas, opening or merging PRs that touch Lakebase, or running the same…

Spec-First Test-Driven Development (SFTDD) with evolutionary design, against paired Lakebase branches: the design lane is Spec Driven Development (SDD) and the build lane is canonical Beck-style Test Driven Development (RED-GREEN-REFACTOR), composed with…

Foundational engineering canon – SOLID, DRY, clean code, layered architecture, cross-cutting concerns, NFRs. Imported by workflow skills (lakebase-sftdd-workflows, lakebase-scm-workflows, lakebase-release-workflows). Use when designing a module, reviewing a…

System-level engineering canon, the architecture counterpart to software-design-principles. Layered architecture + dependency direction, ports and adapters (repository / ORM-as-adapter), the twelve-factor app for cloud-native development, evolutionary…

Opinionated branching + release methodology for Lakebase-paired projects. Use when designing a project's branch layout, cutting a release candidate, promoting between long-running tiers, rolling back, or asking 'where should this work happen?' Encodes the…

Experience-level engineering canon, the UI/UX counterpart to software-design-principles (code-level) and architectural-design-principles (system-level). Usability heuristics, visual hierarchy, accessibility, interaction and feedback, information architecture,…

Package and deploy Databricks Asset Bundles with proper parameterization, multi-environment support, and serverless compute. Handles project structure, databricks.yml generation, validation, and deployment. Use when packaging tested code for production,…

Production data engineering pipelines following medallion architecture (Bronze/Silver/Gold layers) with data ingestion, transformation, quality checks, Delta Lake optimization, and orchestration. Use when building ETL pipelines, medallion architecture, data…

End-to-end machine learning pipelines on Databricks including data exploration, feature engineering, model training with hyperparameter optimization, MLflow experiment tracking, model registration to Unity Catalog, and deployment as DABs. Use when building ML…

Execute code on Databricks clusters using MCP Command Execution API. Supports stateless quick validation and stateful iterative development. Use when testing Python/SQL code on clusters, debugging pipelines, or validating transformations.

Manage Unity Catalog resources including catalogs, schemas, and tables. Handles discovery, creation, updates, and deletions with proper naming conventions and governance. Use when exploring catalogs, creating schemas, managing tables, or setting up data…