databricks-isv-u2m

name	databricks-isv-u2m
description	U2M (user-to-machine) OAuth flows for Databricks ISV integrations: external-browser, custom-oauth-app, token-env. Covers Python, Java, Go, and Node.js implementations. Use when implementing or debugging browser-based or pre-obtained token auth.

U2M (User-to-Machine) Authentication (ISV)

Use this skill when implementing or testing U2M (browser or pre-obtained token) authentication for Databricks partner integrations.

Important: M2M client_id ≠ U2M OAuth app

• The service principal client_id used for OAuth M2M (DATABRICKS_CLIENT_ID) is not registered for the authorization code (browser) flow. Using it with the authorize URL or with SDK external-browser causes: "OAuth application with client_id: '...' not available in Databricks account".
• For browser-based U2M use either: (1) SDK external-browser with no client_id (Databricks built-in app), or (2) a separate custom OAuth app (created in the account) with a redirect URI.
• We use separate env var names for U2M custom apps: DATABRICKS_U2M_CLIENT_ID and DATABRICKS_U2M_CLIENT_SECRET to avoid conflicts with M2M credentials (DATABRICKS_CLIENT_ID / DATABRICKS_CLIENT_SECRET) in the same env file.

Three Flows

Flow	When to Use	Required Variables
external-browser	No custom OAuth app; SDK opens browser	DATABRICKS_HOST only
custom-oauth-app	Your registered OAuth app + configurable redirect URI	DATABRICKS_HOST, DATABRICKS_U2M_CLIENT_ID; optional redirect_uri, client_secret
token-env	Pre-obtained token (hosted callback, refresh, CLI)	DATABRICKS_HOST, DATABRICKS_ACCESS_TOKEN or DATABRICKS_TOKEN

---

Flow 1: External-Browser (SDK Built-in App)

Use the SDK's built-in OAuth app (databricks-cli). Do not set or pass DATABRICKS_CLIENT_ID / DATABRICKS_CLIENT_SECRET.

Python SDK

from databricks.sdk import WorkspaceClient
from databricks.sdk.config import Config

config = Config(
    host="https://myworkspace.cloud.databricks.com",
    auth_type="external-browser"
)
# Unset M2M credentials before this if they exist in environment
client = WorkspaceClient(config=config)

# Get token for other uses (e.g., REST API, JDBC)
token = config.authenticate()["Authorization"].replace("Bearer ", "")

Java SDK

import com.databricks.sdk.core.DatabricksConfig;
import java.util.Arrays;
import java.util.Map;

public static String getTokenViaExternalBrowser(String host) {
    String normalizedHost = host.startsWith("https://") ? host : "https://" + host;

    DatabricksConfig config = new DatabricksConfig()
            .setHost(normalizedHost)
            .setAuthType("external-browser")
            .setScopes(Arrays.asList("all-apis"))        // REQUIRED
            .setOAuthRedirectUrl("http://localhost:8020"); // REQUIRED for built-in app

    config.resolve();  // REQUIRED: initializes HTTP client
    Map<String, String> headers = config.authenticate();

    return headers.get("Authorization").substring("Bearer ".length());
}

Token Caching

The SDK caches tokens. To force a fresh login:

• Python SDK: Delete ~/.databricks/token-cache.json
• Java SDK: Delete ~/.config/databricks-sdk-java/oauth/
• Also clear browser cookies for the workspace and SSO provider

Redirect URI for Built-in App

SDK	Default	Registered for databricks-cli	Action
Python SDK	http://localhost:8020	http://localhost:8020	None needed
Java SDK v0.54.0	http://localhost:8080/callback	http://localhost:8020	Must set explicitly

---

Flow 2: Custom-OAuth-App (Authorization Code + PKCE)

Register a custom OAuth app in Databricks and implement PKCE flow.

Custom OAuth App Setup (Prerequisite)

1. Log in to Databricks account console (https://accounts.cloud.databricks.com/)
2. Go to Settings → App connections → Add connection
3. Create an app. Note the Client ID. If confidential, generate a Client Secret.
4. Add Redirect URIs — e.g., http://localhost:8080/callback

Three-way consistency rule: The redirect URI must exactly match in three places: (1) the skill/code default, (2) the connector code, and (3) the App connections → your app → Redirect URIs in Databricks. A mismatch causes a silent OAuth callback failure with no clear error. The port 8080 is the default — if you use a different port, update all three locations.

PKCE Flow Implementation

Authorize URL:

{host}/oidc/v1/authorize
  ?response_type=code
  &client_id={client_id}
  &redirect_uri={redirect_uri}
  &scope=all-apis
  &code_challenge={S256_challenge}
  &code_challenge_method=S256
  &state={state}

Token URL:

POST {host}/oidc/v1/token
Content-Type: application/x-www-form-urlencoded

grant_type=authorization_code
&code={code_from_redirect}
&redirect_uri={same_as_authorize}
&client_id={client_id}
&code_verifier={verifier}

Public vs Confidential OAuth Apps

Databricks OAuth apps registered in App connections can be public (no client secret) or confidential (registered with a client secret).

App type	client_secret in token exchange	Error if missing
Public	Omit client_secret	n/a
Confidential	Required — must include client_secret in POST body	{"error": "invalid_client"}

⚠️ DIAGNOSTIC: invalid_client on token exchange Cause: The OAuth app is confidential (has a client secret registered) but client_secret was omitted from the token exchange POST body. Fix: Add client_secret=<secret> to the POST body. The secret is shown once at app creation in Databricks App connections — if lost, regenerate it.

Full token exchange body for confidential apps:

grant_type=authorization_code
&code=<code>
&redirect_uri=<uri>
&client_id=<id>
&code_verifier=<verifier>
&client_secret=<secret>   ← required for confidential apps

Python PKCE Implementation

import secrets
import hashlib
import base64
import webbrowser
import threading
from http.server import HTTPServer, BaseHTTPRequestHandler
from urllib.parse import urlencode, urlparse, parse_qs
import requests

def generate_pkce():
    """Generate PKCE code_verifier and code_challenge."""
    verifier = secrets.token_urlsafe(64)
    digest = hashlib.sha256(verifier.encode()).digest()
    challenge = base64.urlsafe_b64encode(digest).rstrip(b"=").decode()
    return verifier, challenge

class CallbackHandler(BaseHTTPRequestHandler):
    auth_code = None
    callback_received = threading.Event()
    
    def do_GET(self):
        if "/callback" in self.path:
            query = parse_qs(urlparse(self.path).query)
            CallbackHandler.auth_code = query.get("code", [None])[0]
            self.send_response(200)
            self.end_headers()
            self.wfile.write(b"Authentication complete. Close this tab.")
            CallbackHandler.callback_received.set()
        else:
            self.send_response(204)
            self.end_headers()
    
    def log_message(self, format, *args):
        pass

def run_pkce_flow(host, client_id, client_secret=None,
                  redirect_uri="http://localhost:8080/callback", timeout=120):
    """Complete PKCE flow and return access token."""
    
    # Parse port from redirect URI
    port = int(urlparse(redirect_uri).port or 8080)
    
    # Generate PKCE values
    verifier, challenge = generate_pkce()
    state = secrets.token_urlsafe(16)
    
    # Build authorize URL
    host = host.rstrip("/")
    if not host.startswith("https://"):
        host = "https://" + host
    
    auth_params = {
        "response_type": "code",
        "client_id": client_id,
        "redirect_uri": redirect_uri,
        "scope": "all-apis",
        "code_challenge": challenge,
        "code_challenge_method": "S256",
        "state": state,
    }
    auth_url = f"{host}/oidc/v1/authorize?{urlencode(auth_params)}"
    
    # Start callback server
    CallbackHandler.auth_code = None
    CallbackHandler.callback_received.clear()
    
    server = HTTPServer(("localhost", port), CallbackHandler)
    server.timeout = timeout  # CRITICAL: prevents infinite blocking
    
    # Open browser
    webbrowser.open(auth_url)
    print(f"Opened browser for authentication...")
    
    # Wait for callback
    while not CallbackHandler.callback_received.is_set():
        server.handle_request()
    
    server.server_close()  # Use server_close(), NOT shutdown()
    
    if not CallbackHandler.auth_code:
        raise RuntimeError("No auth code received")
    
    # Exchange code for token
    token_data = {
        "grant_type": "authorization_code",
        "code": CallbackHandler.auth_code,
        "redirect_uri": redirect_uri,
        "client_id": client_id,
        "code_verifier": verifier,
    }
    if client_secret:
        token_data["client_secret"] = client_secret
    
    token_url = f"{host}/oidc/v1/token"
    resp = requests.post(token_url, data=token_data)
    resp.raise_for_status()
    
    return resp.json()["access_token"]

Java SDK with Custom OAuth App

import com.databricks.sdk.core.DatabricksConfig;
import java.util.Arrays;
import java.util.Map;

public static String getTokenViaCustomOAuthApp(
        String host, String clientId, String clientSecret, String redirectUri) {
    
    String normalizedHost = host.startsWith("https://") ? host : "https://" + host;

    DatabricksConfig config = new DatabricksConfig()
            .setHost(normalizedHost)
            .setAuthType("external-browser")
            .setClientId(clientId)
            .setScopes(Arrays.asList("all-apis"));

    if (clientSecret != null && !clientSecret.isBlank()) {
        config.setClientSecret(clientSecret);
    }
    if (redirectUri != null && !redirectUri.isBlank()) {
        config.setOAuthRedirectUrl(redirectUri);
    }

    config.resolve();
    Map<String, String> headers = config.authenticate();
    return headers.get("Authorization").substring("Bearer ".length());
}

Go PKCE Implementation

package main

import (
    "crypto/rand"
    "crypto/sha256"
    "encoding/base64"
    "fmt"
    "net/http"
    "net/url"
    "strings"
)

func generatePKCE() (verifier, challenge string) {
    b := make([]byte, 64)
    rand.Read(b)
    verifier = base64.RawURLEncoding.EncodeToString(b)
    h := sha256.Sum256([]byte(verifier))
    challenge = base64.RawURLEncoding.EncodeToString(h[:])
    return
}

func runPKCEFlow(host, clientID, clientSecret, redirectURI string, timeout int) (string, error) {
    verifier, challenge := generatePKCE()
    state := generateRandomString(16)
    
    // Parse redirect URI for port
    u, _ := url.Parse(redirectURI)
    port := u.Port()
    if port == "" {
        port = "8080"
    }
    
    // Build authorize URL
    if !strings.HasPrefix(host, "https://") {
        host = "https://" + host
    }
    
    authParams := url.Values{
        "response_type":         {"code"},
        "client_id":            {clientID},
        "redirect_uri":         {redirectURI},
        "scope":                {"all-apis"},