databricks-isv-python-sdk

name	databricks-isv-python-sdk
description	PWAF-compliant Databricks SDK for Python (databricks-sdk): PAT, OAuth M2M, U2M token-env, U2M custom OAuth app (PKCE); useragent.with_partner/with_product. Use when building or testing Python SDK workspace API integrations.

Databricks SDK for Python (ISV)

Use this skill when implementing or testing Databricks SDK for Python (databricks-sdk) integrations for PWAF-compliant workspace management, Unity Catalog, Jobs, and REST-style API access.

PWAF Documentation Links

Resource	URL
PWAF Home	https://databrickslabs.github.io/partner-architecture/
Authentication Best Practices	https://databrickslabs.github.io/partner-architecture/isv-partners/lakehouse-patterns/access-auth/
OAuth M2M Guide	https://databrickslabs.github.io/partner-architecture/isv-partners/lakehouse-patterns/access-auth/oauth-m2m
OAuth U2M Guide	https://databrickslabs.github.io/partner-architecture/isv-partners/lakehouse-patterns/access-auth/oauth-u2m
User-Agent Telemetry	https://databrickslabs.github.io/partner-architecture/isv-partners/telemetry-attribution/sdks
Python SDK Docs	https://docs.databricks.com/aws/en/dev-tools/sdk-python
Python SDK GitHub	https://github.com/databricks/databricks-sdk-py

Requirements

• SDK: databricks-sdk 0.20.0+
• Python: 3.8+
• Install: pip install databricks-sdk

Authentication Decision Guide

Which authentication method to use?

Production / automated workloads?
  → OAuth M2M (client credentials)  ✅ RECOMMENDED

User-interactive flows?
  → U2M Custom OAuth App (PKCE)  ✅ SUPPORTED

Already have an OAuth access token?
  → U2M Token-Env  ✅ SUPPORTED

Local development/testing only?
  → PAT (Personal Access Token)  ⚠️ LIMITED

Authentication Comparison

Method	PWAF Status	Auto Token Refresh	Browser Required	Use Case
PAT	⚠️ Limited	No	No	Testing only
OAuth M2M	✅ Recommended	Yes (SDK handles)	No	Production/automated
U2M Custom OAuth App	✅ Supported	No	Yes	User-interactive
U2M Token-Env	✅ Supported	No	No	Headless/CI

---

CLIENT_ID Distinction (CRITICAL)

Variable	Purpose	Value Source	Use Case
DATABRICKS_CLIENT_ID	M2M Service Principal	Workspace → Identity and access → Service principals → UUID	Automated/production
DATABRICKS_U2M_CLIENT_ID	U2M Custom OAuth App	Account Console → Settings → App connections → Client ID	User-interactive

Never mix these. M2M uses service principal credentials for server-to-server auth. U2M custom OAuth uses a registered app for user login.

---

Token Lifetime Summary

Token Type	Typical TTL	Refresh Mechanism
PAT	90 days (configurable)	Manual regeneration
M2M access_token	1 hour	SDK auto-refreshes
U2M access_token	1 hour	Manual re-auth or refresh_token

---

User-Agent (Required per PWAF)

Call once before creating WorkspaceClient:

from databricks.sdk import useragent

useragent.with_partner("YourCompany")
useragent.with_product("YourCompany_YourProduct", "1.0.0")

Alternative (on WorkspaceClient):

from databricks.sdk import WorkspaceClient

client = WorkspaceClient(
    host="https://myworkspace.cloud.databricks.com",
    token="dapi...",
    product="YourCompany_YourProduct",
    product_version="1.0.0"
)

---

Environment Variables Reference

Variable	Required For	Description
DATABRICKS_HOST	All	Workspace URL (e.g., https://myworkspace.cloud.databricks.com)
DATABRICKS_TOKEN	PAT	Personal access token
DATABRICKS_CLIENT_ID	OAuth M2M	Service principal UUID
DATABRICKS_CLIENT_SECRET	OAuth M2M	Service principal OAuth secret
DATABRICKS_U2M_CLIENT_ID	U2M Custom OAuth	Custom OAuth app client ID from App connections
DATABRICKS_U2M_CLIENT_SECRET	U2M Custom OAuth (optional)	Custom OAuth app client secret (if confidential)
DATABRICKS_REDIRECT_URI	U2M Custom OAuth (optional)	Custom redirect URI (default: http://localhost:8080/callback)
DATABRICKS_ACCESS_TOKEN	U2M Token-Env	Pre-obtained OAuth access token
APP_AUTH_TYPE	Multi-auth	App-level auth selector: pat, oauth_m2m, oauth_u2m, u2m_token_env

Important:

• Use APP_AUTH_TYPE (not DATABRICKS_AUTH_TYPE) for app-level auth selection because the SDK reads DATABRICKS_AUTH_TYPE internally.
• Do not mix M2M and U2M environment variables. Use env -i for clean test environments.

---

Config Parameters Reference

Parameter	Type	Required	Description
host	str	Yes	Workspace URL (include https://)
token	str	PAT only	Personal access token
client_id	str	OAuth M2M	Service principal UUID
client_secret	str	OAuth M2M	Service principal OAuth secret
auth_type	str	OAuth M2M	Set to "oauth-m2m" for M2M (avoids default credential resolution)
azure_use_msi	bool	Azure only	Enable Azure Managed Identity
azure_client_id	str	Azure MSI	Client ID for user-assigned MSI
product	str	Telemetry	Product name for User-Agent
product_version	str	Telemetry	Product version for User-Agent

Critical: When using OAuth M2M, always pass auth_type="oauth-m2m" to Config to avoid "cannot configure default credentials" error.

---

Complete Examples

PAT Authentication (Testing Only)

import os
from databricks.sdk import WorkspaceClient, useragent

useragent.with_partner("YourCompany")
useragent.with_product("YourProduct", "1.0.0")

def get_client_pat():
    host = os.environ.get("DATABRICKS_HOST", "").strip()
    if not host.startswith("https://"):
        host = f"https://{host}"
    token = os.environ.get("DATABRICKS_TOKEN", "").strip()
    if not token:
        raise ValueError("DATABRICKS_TOKEN required")
    
    return WorkspaceClient(host=host, token=token)

client = get_client_pat()
me = client.current_user.me()
print(f"Authenticated as: {me.user_name}")

OAuth M2M (Recommended for Production)

import os
from databricks.sdk import WorkspaceClient, useragent
from databricks.sdk.config import Config

useragent.with_partner("YourCompany")
useragent.with_product("YourProduct", "1.0.0")

def get_client_m2m():
    host = os.environ.get("DATABRICKS_HOST", "").strip()
    if not host.startswith("https://"):
        host = f"https://{host}"
    
    config = Config(
        host=host,
        client_id=os.environ.get("DATABRICKS_CLIENT_ID", ""),
        client_secret=os.environ.get("DATABRICKS_CLIENT_SECRET", ""),
        auth_type="oauth-m2m",  # CRITICAL: Required for M2M
    )
    return WorkspaceClient(config=config)

client = get_client_m2m()
# SDK automatically handles token refresh
tables = list(client.tables.list("samples", "nyctaxi"))
print(f"Found {len(tables)} tables")

OAuth U2M Custom OAuth App (PKCE)

import os
import base64
import hashlib
import secrets
import string
import webbrowser
from http.server import HTTPServer, BaseHTTPRequestHandler
from threading import Thread
from urllib.parse import parse_qs, urlparse, urlencode
import requests
from databricks.sdk import WorkspaceClient

DEFAULT_REDIRECT_URI = "http://localhost:8080/callback"

def pkce_verifier_and_challenge():
    """Generate PKCE code_verifier and code_challenge per RFC 7636."""
    allowed = string.ascii_letters + string.digits + "-._~"
    code_verifier = "".join(secrets.choice(allowed) for _ in range(64))
    digest = hashlib.sha256(code_verifier.encode()).digest()
    code_challenge = base64.urlsafe_b64encode(digest).decode().rstrip("=")
    return code_verifier, code_challenge

def run_u2m_flow(host: str, client_id: str, redirect_uri: str = None, 
                 client_secret: str = None, scope: str = "all-apis") -> dict:
    """Run OAuth 2.0 authorization code flow with PKCE."""
    redirect_uri = redirect_uri or DEFAULT_REDIRECT_URI
    if not host.startswith("https://"):
        host = f"https://{host}"
    
    code_verifier, code_challenge = pkce_verifier_and_challenge()
    state = secrets.token_urlsafe(32)
    
    auth_url = f"{host}/oidc/v1/authorize?" + urlencode({
        "response_type": "code",
        "client_id": client_id,
        "redirect_uri": redirect_uri,
        "scope": scope,
        "code_challenge": code_challenge,
        "code_challenge_method": "S256",
        "state": state,
    })
    
    code_holder = []
    parsed = urlparse(redirect_uri)
    port = parsed.port or 8080
    
    class Handler(BaseHTTPRequestHandler):
        def do_GET(self):
            q = parse_qs(urlparse(self.path).query)
            code_holder.append(q.get("code", [None])[0])
            self.send_response(200)
            self.send_header("Content-type", "text/html; charset=utf-8")
            self.end_headers()
            self.wfile.write(b"<html><body><p>Authentication successful.</p></body></html>")
        def log_message(self, *args): pass
    
    server = HTTPServer(("localhost", port), Handler)
    Thread(target=server.handle_request, daemon=True).start()
    
    webbrowser.open(auth_url)
    print(f"Waiting for redirect at {redirect_uri}...")
    server.handle_request()
    
    if not code_holder[0]:
        raise RuntimeError("No authorization code received")
    
    token_data = {
        "grant_type": "authorization_code",
        "code": code_holder[0],
        "redirect_uri": redirect_uri,
        "client_id": client_id,
        "code_verifier": code_verifier,
    }
    if client_secret:
        token_data["client_secret"] = client_secret
    
    resp = requests.post(f"{host}/oidc/v1/token", data=token_data)
    resp.raise_for_status()
    return resp.json()

def get_client_u2m():
    host = os.environ.get("DATABRICKS_HOST", "").strip()
    client_id = os.environ.get("DATABRICKS_U2M_CLIENT_ID", "").strip()
    redirect_uri = os.environ.get("DATABRICKS_REDIRECT_URI", "") or DEFAULT_REDIRECT_URI
    client_secret = os.environ.get("DATABRICKS_U2M_CLIENT_SECRET", "").strip() or None
    
    tokens = run_u2m_flow(host, client_id, redirect_uri, client_secret)