databricks-enterprise-rbac

name	databricks-enterprise-rbac
description	Configure Databricks enterprise SSO, Unity Catalog RBAC, and organization management. Use when implementing SSO integration, configuring role-based permissions, or setting up organization-level controls with Unity Catalog. Trigger with phrases like "databricks SSO", "databricks RBAC", "databricks enterprise", "unity catalog permissions", "databricks SCIM".
allowed-tools	Read, Write, Edit, Bash(databricks:*)
version	1.0.0
license	MIT
author	Jeremy Longshore <[e-mail]>

Databricks Enterprise RBAC

Overview

Configure enterprise-grade access control using Unity Catalog and SSO.

Prerequisites

• Databricks Enterprise or Premium tier
• Unity Catalog enabled
• Identity Provider (IdP) with SAML/SCIM support
• Account admin access

Role Definitions

Role	Unity Catalog Privileges	Use Case
Data Admin	ALL PRIVILEGES on catalogs	Platform administrators
Data Engineer	CREATE, MODIFY, SELECT on schemas	Pipeline development
Data Analyst	SELECT on gold tables	Analytics and reporting
Data Scientist	SELECT + ML privileges	Model training
Viewer	SELECT on specific views	Business stakeholders

Instructions

Step 1: Configure Identity Provider (SCIM)

# scripts/setup_scim.py
from databricks.sdk import AccountClient
from databricks.sdk.service.iam import (
    Group,
    User,
    ServicePrincipal,
)

def setup_account_groups(client: AccountClient) -> dict:
    """
    Set up standard account-level groups.

    These groups are synced from IdP via SCIM.
    """
    standard_groups = [
        {
            "display_name": "data-platform-admins",
            "description": "Full platform administration"
        },
        {
            "display_name": "data-engineers",
            "description": "Data pipeline development"
        },
        {
            "display_name": "data-analysts",
            "description": "Analytics and reporting"
        },
        {
            "display_name": "data-scientists",
            "description": "ML and experimentation"
        },
        {
            "display_name": "data-viewers",
            "description": "Read-only business users"
        },
    ]

    created_groups = {}
    for group_config in standard_groups:
        group = client.groups.create(
            display_name=group_config["display_name"],
        )
        created_groups[group_config["display_name"]] = group.id

    return created_groups

# SCIM Configuration for Azure AD
scim_config = {
    "token_endpoint": "https://accounts.azuredatabricks.net/scim/v2",
    : ,
    : {
        : ,
        : ,
        : ,
        : ,
    }
}

Step 2: Unity Catalog Permission Model

-- Create catalog hierarchy
CREATE CATALOG IF NOT EXISTS enterprise_data;

-- Create standard schemas
CREATE SCHEMA IF NOT EXISTS enterprise_data.bronze
COMMENT 'Raw ingested data';

CREATE SCHEMA IF NOT EXISTS enterprise_data.silver
COMMENT 'Cleansed and conformed data';

CREATE SCHEMA IF NOT EXISTS enterprise_data.gold
COMMENT 'Business-ready aggregations';

CREATE SCHEMA IF NOT EXISTS enterprise_data.ml_features
COMMENT 'ML feature store';

CREATE SCHEMA IF NOT EXISTS enterprise_data.sandbox
COMMENT 'Experimentation sandbox';

-- Grant catalog-level permissions
-- Data Admins: Full control
GRANT ALL PRIVILEGES ON CATALOG enterprise_data TO `data-platform-admins`;

-- Data Engineers: Create/modify in bronze/silver, read gold
GRANT USAGE ON CATALOG enterprise_data TO `data-engineers`;
GRANT CREATE, USAGE ON SCHEMA enterprise_data.bronze TO `data-engineers`;
GRANT CREATE, USAGE ON SCHEMA enterprise_data.silver TO `dataengineers`;
 USAGE,   SCHEMA enterprise_data.gold  `dataengineers`;
  PRIVILEGES  SCHEMA enterprise_data.sandbox  `dataengineers`;


 USAGE  CATALOG enterprise_data  `dataanalysts`;
 USAGE,   SCHEMA enterprise_data.gold  `dataanalysts`;
  PRIVILEGES  SCHEMA enterprise_data.sandbox  `dataanalysts`;


 USAGE  CATALOG enterprise_data  `datascientists`;
 USAGE,   SCHEMA enterprise_data.silver  `datascientists`;
 USAGE,   SCHEMA enterprise_data.gold  `datascientists`;
  PRIVILEGES  SCHEMA enterprise_data.ml_features  `datascientists`;
  PRIVILEGES  SCHEMA enterprise_data.sandbox  `datascientists`;


 USAGE  CATALOG enterprise_data  `dataviewers`;
 USAGE,   SCHEMA enterprise_data.gold  `dataviewers`;

Step 3: Workspace-Level Permissions

# scripts/workspace_permissions.py
from databricks.sdk import WorkspaceClient
from databricks.sdk.service.iam import PermissionLevel

def configure_workspace_permissions(w: WorkspaceClient):
    """Configure workspace-level permissions."""

    # Cluster permissions
    cluster_policies = {
        "data-engineers": "CAN_USE",
        "data-scientists": "CAN_USE",
        "data-analysts": "CAN_ATTACH_TO",
    }

    # Get all cluster policies
    for policy in w.cluster_policies.list():
        for group, permission in cluster_policies.items():
            w.permissions.update(
                object_type="cluster-policies",
                object_id=policy.policy_id,
                access_control_list=[{
                    "group_name": group,
                    "permission_level": permission,
                }]
            )

    # SQL Warehouse permissions
    for warehouse in w.warehouses.list():
        w.permissions.update(
            object_type="sql/warehouses",
            object_id=warehouse.id,
            access_control_list=[
                {"group_name": "data-analysts", "permission_level": "CAN_USE"},
                {"group_name": "data-engineers", "permission_level": "CAN_MANAGE"},
            ]
        )

    # Workspace folder permissions
    folder_permissions = {
        : {
            : ,
            : ,
            : ,
        },
        : {
            : ,
            : ,
            : ,
        },
    }

     folder, permissions  folder_permissions.items():
        acl = [
            {: group, : level}
             group, level  permissions.items()
        ]
        w.workspace.set_permissions(path=folder, access_control_list=acl)

Step 4: Service Principal Management

# scripts/service_principals.py
from databricks.sdk import AccountClient, WorkspaceClient
from databricks.sdk.service.iam import ServicePrincipal

def create_service_principal(
    account_client: AccountClient,
    workspace_client: WorkspaceClient,
    name: str,
    purpose: str,
    workspace_permissions: dict,
) -> dict:
    """
    Create and configure a service principal.

    Args:
        account_client: Account-level client
        workspace_client: Workspace-level client
        name: Service principal name
        purpose: Description/purpose
        workspace_permissions: Dict of {object_type: permission_level}

    Returns:
        Service principal details
    """
    # Create at account level
    sp = account_client.service_principals.create(
        display_name=name,
        active=True,
    )

    # Create OAuth secret
    secret = account_client.service_principal_secrets.create(
        service_principal_id=sp.id,
    )

    # Add to workspace groups based on purpose
    group_mappings = {
        "etl-pipeline": "data-engineers",
        "ml-training": "data-scientists",
        "reporting": "data-analysts",
    }

    if purpose in group_mappings:
        group_name = group_mappings[purpose]
        # Add SP to group (via SCIM or API)

    # Set workspace-level permissions
    for obj_type, permission in workspace_permissions.items():
        workspace_client.permissions.update(
            object_type=obj_type,
            object_id="*",  # All objects of type
            access_control_list=[{
                "service_principal_name": sp.application_id,
                : permission,
            }]
        )

     {
        : sp.,
        : sp.application_id,
        : secret.secret,  
    }


service_principals = [
    {
        : ,
        : ,
        : {
            : ,
            : ,
        }
    },
    {
        : ,
        : ,
        : {
            : ,
            : ,
        }
    },
]

Step 5: Audit and Compliance

-- Query audit logs for access patterns
SELECT
    event_time,
    user_identity.email as user_email,
    service_name,
    action_name,
    request_params,
    response.status_code
FROM system.access.audit
WHERE event_date > current_date() - INTERVAL 7 DAYS
  AND service_name IN ('unityCatalog', 'clusters', 'jobs')
ORDER BY event_time DESC
LIMIT 100;

-- Permission changes audit
SELECT
    event_time,
    user_identity.email as changed_by,
    action_name,
    request_params:full_name_arg as resource,
    request_params:changes as permission_changes
FROM system.access.audit
WHERE action_name LIKE '%Grant%' OR action_name LIKE '%Revoke%'
  AND event_date > current_date() - INTERVAL 30 DAYS
ORDER BY event_time DESC;

-- Privileged access report
SELECT DISTINCT
    grantee,
    privilege,
    object_type,
    object_name
FROM system.information_schema.object_privileges
WHERE privilege IN ('ALL PRIVILEGES', , )
  grantee, object_type;

Step 6: Access Review Automation

# scripts/access_review.py
from databricks.sdk import WorkspaceClient
from datetime import datetime
import pandas as pd

def generate_access_review_report(w: WorkspaceClient, spark) -> pd.DataFrame:
    """
    Generate access review report for compliance.

    Returns DataFrame with all permissions by user/group.
    """
    # Get Unity Catalog permissions
    uc_permissions = spark.sql("""
        SELECT
            grantee,
            grantee_type,
            privilege,
            object_type,
            object_name,
            inherited_from
        FROM system.information_schema.object_privileges
        ORDER BY grantee, object_type
    """).toPandas()

    # Get workspace permissions
    workspace_permissions = []
    for obj_type in ['clusters', 'jobs', 'sql/warehouses']:
        for obj in getattr(w, obj_type.replace('/', '_').replace('sql_', '')).list():
            perms = w.permissions.get(object_type=obj_type, object_id=obj.id)
            for acl in perms.access_control_list or []:
                workspace_permissions.append({
                    "object_type": obj_type,
                    "object_id": obj.id,
                    "principal": acl.user_name or acl.group_name or acl.service_principal_name,
                    "permission": acl.all_permissions[0].permission_level  acl.all_permissions  ,
                })

    ws_df = pd.DataFrame(workspace_permissions)

    
    report = {
        : datetime.now().isoformat(),
        : uc_permissions,
        : ws_df,
    }

     report

 () -> []:
    
    flags = []

    uc_df = report[]

    
    excessive = uc_df[uc_df[] == ]
     _, row  excessive.iterrows():
         row[] != :  
            flags.append({
                : ,
                : row[],
                : row[],
                : row[],
                : ,
            })

    
    direct_grants = uc_df[uc_df[] == ]
    unique_users = direct_grants[].unique()
     user  unique_users:
        user_grants = (direct_grants[direct_grants[] == user])
         user_grants > :
            flags.append({
                : ,
                : user,
                : user_grants,
                : ,
            })

     flags

Output

• SCIM integration configured
• Unity Catalog RBAC implemented
• Workspace permissions set
• Service principals created
• Audit logging enabled

Error Handling

Issue	Cause	Solution
SSO login fails	Wrong callback URL	Verify IdP config
Permission denied	Missing grant	Check inherited permissions
SCIM sync failed	Token expired	Regenerate SCIM token
Group not found	Case sensitivity	Match exact group name

Examples

Quick Permission Check

-- Check what a user can access
SELECT * FROM system.information_schema.object_privileges
WHERE grantee = '[e-mail]'
ORDER BY object_type, object_name;

Grant Template

-- Standard data consumer access
GRANT USAGE ON CATALOG enterprise_data TO `new-team`;
GRANT USAGE ON SCHEMA enterprise_data.gold TO `new-team`;
GRANT SELECT ON SCHEMA enterprise_data.gold TO `new-team`;

Resources

• Unity Catalog Privileges
• SCIM Provisioning
• Identity Federation

Next Steps

For major migrations, see databricks-migration-deep-dive.