databricks-isv-integration

name	databricks-isv-integration
description	Build PWAF-compliant ISV integrations with Databricks: OAuth, telemetry (User-Agent), Unity Catalog, JDBC, SDK, SQL drivers, REST API, Databricks Connect.

Databricks ISV Integration

Overview

This skill provides patterns and code examples for building PWAF-compliant ISV integrations with Databricks, following the Partner Well-Architected Framework (PWAF).

Cursor rule vs skills: Use the Cursor rule (.cursor/rules/databricks-isv-integration.mdc) for file-scoped context when editing integration code (it applies when you work in matching paths). Use the skills in skills/*/SKILL.md for task-based guidance (e.g. "add a Databricks connector," "fix U2M," "run auth tests")—the agent loads the relevant skill by description. Auth patterns live in skills/<connector>/authentication.md, co-located with each skill.

PWAF covers:

• Authentication: OAuth M2M, U2M, Token Federation, PAT
• Telemetry: User-Agent attribution for partner tracking
• Unity Catalog: Proper namespace usage and governance
• SQL Drivers: JDBC, ODBC, Node.js, Go connectivity

PWAF Integration Requirements Summary

All PWAF-validated partner integrations must meet these requirements:

Requirement	Summary
Telemetry	User-Agent tagging on all API/driver calls
OAuth	Support Token Federation, U2M with PKCE, and M2M
Unity Catalog	Use UC namespaces, respect ACLs, use Volumes for staging

Applications: Use OAuth M2M for service/backend flows, or U2M for interactive user sign-in. U2M has three flows: (1) SDK external-browser with built-in app (simplest; no custom app), (2) SDK external-browser with custom OAuth app (for ISV integrations; configurable redirect URI), (3) token pass-through (pre-obtained token; headless/CI). All use Auth_Flow=0 on the JDBC side. For custom apps, redirect URI is configurable; default http://localhost:8080/callback. For the built-in app, redirect URI is http://localhost:8020.

Quick Start: JDBC Connection with OAuth M2M

import java.sql.*;
import java.util.Properties;

public class DatabricksConnector {
    public static Connection connect(
            String host,
            String httpPath,
            String clientId,
            String clientSecret,
            String userAgent
    ) throws SQLException {
        String url = "jdbc:databricks://" + host + ":443";
        
        Properties props = new Properties();
        props.put("httpPath", httpPath);
        props.put("SSL", "1");
        
        // OAuth M2M Authentication
        props.put("AuthMech", "11");
        props.put("Auth_Flow", "1");
        props.put("OAuth2ClientId", clientId);
        props.put("OAuth2Secret", clientSecret);
        
        // Partner Telemetry (REQUIRED)
        props.put("UserAgentEntry", userAgent);
        
        return DriverManager.getConnection(url, props);
    }
}

Authentication Types

Dynamic Field Requirements by Auth Type

When building a connector UI, show these fields based on auth selection:

Auth Type	AuthMech	Auth_Flow	Required Fields	Token Source
PAT	3	-	token	User provides PAT
OAuth M2M	11	1	clientId, clientSecret	Driver does client-credentials
U2M Built-in Browser	11	0	(none beyond host/path)	SDK external-browser (built-in databricks-cli app)
U2M Custom App Browser	11	0	u2mClientId; optional u2mClientSecret, redirectUri	SDK external-browser (custom OAuth app)
U2M Token Pass-through	11	0	accessToken	Pre-obtained (CLI, hosted callback, refresh)

U2M: All three U2M flows use Auth_Flow=0 (token pass-through) on the JDBC side. The token is obtained externally via the Databricks SDK or a prior OAuth exchange. For custom OAuth apps, use DATABRICKS_U2M_CLIENT_ID (not the M2M DATABRICKS_CLIENT_ID). Redirect URI: configurable for custom apps; http://localhost:8020 for the built-in app (must set explicitly in Java SDK v0.54.0).

See skills/java-jdbc/authentication.md for complete code examples including Java SDK workarounds.

Telemetry Attribution

User-Agent Format (required per PWAF for all partner integrations):

<isv-name>_<product-name>/<product-version>

Example: AcmePartner_DataConnector/2.1.0

See skills/telemetry-attribution/SKILL.md for driver-specific configuration.

Reference Files

Quick index: INDEX.md – one line per rule and skill.

Authentication by Driver/SDK (skills/<connector>/authentication.md)

File	Language/Driver	Coverage
skills/java-jdbc/authentication.md	Java JDBC (OSS v3+)	PAT, OAuth M2M, OAuth U2M (browser + token-env), UserAgentEntry, Java 17+ add-opens
skills/java-sdk/authentication.md	Java SDK (databricks-sdk-java)	PAT, OAuth M2M; UserAgent.withProduct/withPartner; no warehouse needed
skills/python-sdk/authentication.md	Python SDK (databricks-sdk)	PAT, OAuth M2M/U2M, Azure MSI
skills/python-sql-connector/authentication.md	Python SQL Connector	PAT, OAuth M2M/U2M
skills/python-sqlalchemy/authentication.md	Python SQLAlchemy (databricks-sqlalchemy)	PAT, OAuth M2M/U2M; URL + connect_args user_agent_entry; M2M token from headers dict
skills/databricks-connect/authentication.md	Databricks Connect	PAT, OAuth M2M, OAuth U2M (external-browser, localhost, token-env), Serverless, Classic, Azure MSI
skills/nodejs-sql-driver/authentication.md	Node.js SQL Driver (@databricks/sql)	PAT, OAuth M2M (driver-native), OAuth U2M (browser), Token pass-through; host normalization; session patterns
skills/go-sdk/authentication.md	Databricks SDK for Go (databricks-sdk-go)	PAT, OAuth M2M, OAuth U2M (browser + token-env + custom OAuth app PKCE), Azure MSI; useragent.WithProduct/WithPartner
skills/go-sql-driver/authentication.md	Databricks SQL Driver for Go (databricks-sql-go)	PAT, OAuth M2M, OAuth U2M; WithUserAgentEntry; all_auth pattern
skills/rest-api/authentication.md	Any language (HTTP)	PAT, OAuth M2M, Token
skills/python-dbconnect/authentication.md	Python Databricks Connect	PAT, M2M, U2M; compute resolution; PKCE helper
skills/odbc/authentication.md	BI Tools (ODBC)	PAT, OAuth M2M/U2M, Token

Telemetry & Validation

• skills/telemetry-attribution/SKILL.md - User-Agent configuration for all drivers
• skills/connector-testing/integration-checklist.md - Validation checklist for partners
• skills/connector-testing/env-isolation.md - env -i isolation pattern for tests

Cursor rule and skills (subfolders)

Path	Purpose
.cursor/rules/databricks-isv-integration.mdc	Cursor rule: auth isolation, U2M gotchas, Python SDK/SQL patterns, connector structure, testing
skills/adding-databricks-connector/SKILL.md	Add a Databricks connector to an existing project (no Databricks yet): stack choice, where to integrate, minimal steps
skills/connector-structure/SKILL.md	How to structure a connector: config, connect(), operations, validation
skills/rest-api/SKILL.md	REST API auth (PAT, M2M, U2M) and validation tests
skills/python-sdk/SKILL.md	Python SDK (databricks-sdk): Config, auth_type, WorkspaceClient, telemetry
skills/python-sql-connector/SKILL.md	Python SQL Connector: PAT, M2M, U2M; credentials_provider; host normalization
skills/python-sqlalchemy/SKILL.md	SQLAlchemy + Databricks: dialect, URL, PAT/M2M/U2M, user_agent_entry; M2M token from authenticate() headers
skills/databricks-connect/SKILL.md	Databricks Connect: PAT, OAuth M2M, U2M; serverless and classic compute; version compatibility
skills/java-jdbc/SKILL.md	Java JDBC (OSS driver): PAT, OAuth M2M, U2M browser/token-env; UserAgentEntry; Java 17+ add-opens
skills/java-sdk/SKILL.md	Java SDK (databricks-sdk-java): PAT, OAuth M2M; UserAgent.withProduct/withPartner; UC Tables API; no warehouse
skills/go-sdk/SKILL.md	Databricks SDK for Go (databricks-sdk-go): PAT, OAuth M2M, U2M token-env, U2M custom OAuth app (PKCE); useragent.WithProduct/WithPartner; UC Tables API; no warehouse needed
skills/go-sql-driver/SKILL.md	Databricks SQL Driver for Go (databricks-sql-go): PAT, OAuth M2M, U2M browser, U2M token-env, U2M custom OAuth app (PKCE); WithUserAgentEntry; DESCRIBE TABLE; SQL warehouse required
skills/nodejs-sql-driver/SKILL.md	Node.js SQL Driver (@databricks/sql): PAT, OAuth M2M (driver-native), OAuth U2M (browser); host normalization; session patterns
skills/u2m/SKILL.md	U2M flows: external-browser, custom-oauth-app, token-env; M2M client_id ≠ U2M app; separate example scripts per flow
skills/testing/SKILL.md	Running auth tests with clean env per test

Maven Dependencies (Java)

Java SDK (workspace APIs, UC, Jobs — no warehouse needed):

<dependency>
    <groupId>com.databricks</groupId>
    <artifactId>databricks-sdk-java</artifactId>
    <version>0.54.0</version>
</dependency>

Java 11+ required. See skills/java-sdk/SKILL.md.

OSS JDBC driver (SQL queries via warehouse):

<dependency>
    <groupId>com.databricks</groupId>
    <artifactId>databricks-jdbc</artifactId>
    <version>3.1.1</version>
    <scope>runtime</scope>
</dependency>

On Java 17+, add --add-opens=java.base/java.nio=ALL-UNNAMED (e.g. MAVEN_OPTS or exec plugin). See skills/java-jdbc/SKILL.md.

Go Module Dependencies

Databricks SDK for Go (workspace APIs, UC, Jobs — no warehouse needed):

go get github.com/databricks/databricks-sdk-go@v0.107.0

Go 1.21+ required. Uses useragent.WithPartner() / useragent.WithProduct() for telemetry. See skills/go-sdk/SKILL.md.

Databricks SQL Driver for Go (SQL queries via warehouse):

go get github.com/databricks/databricks-sql-go

Go 1.20+ required. Requires DATABRICKS_HTTP_PATH (SQL warehouse). Uses dbsql.WithUserAgentEntry() for telemetry. See skills/go-sql-driver/SKILL.md.

Common Issues

Issue	Solution
More than one authorization method configured	Run with a clean env: only set vars for one auth type (PAT or M2M or U2M). Do not set both DATABRICKS_TOKEN and DATABRICKS_CLIENT_ID/DATABRICKS_CLIENT_SECRET in the same process.
OAuth application with client_id not available (U2M)	M2M service principal client_id is not valid for the browser flow. For external-browser, do not pass client_id (SDK uses built-in app). For custom-oauth-app, use a separate OAuth custom app with redirect URI and DATABRICKS_U2M_CLIENT_ID.
Cannot configure default credentials (Python M2M)	Pass auth_type="oauth-m2m" to Config(...) when using client_id/client_secret so the SDK does not try default credential resolution.
Auth failed with M2M	Verify Service Principal has access to the SQL Warehouse
User-Agent not appearing	Check UserAgentEntry is set in connection properties
Token expired	OAuth tokens expire in 1 hour; implement refresh logic
Unity Catalog access denied	Check catalog/schema grants for the identity
Redirect URL for U2M	Custom apps: configurable, default http://localhost:8080/callback. Built-in app: http://localhost:8020 (set explicitly in Java SDK). Register in App connections. Not used for PAT or M2M.
Java SDK NullPointerException (U2M)	Call config.setScopes(Arrays.asList("all-apis")) and config.resolve() before config.authenticate(). Set config.setOAuthRedirectUrl("http://localhost:8020") for built-in app. See skills/u2m/SKILL.md.
JDBC Java 17+: MemoryUtil / InaccessibleObjectException	OSS driver 3.x uses Arrow; add --add-opens=java.base/java.nio=ALL-UNNAMED to JVM (MAVEN_OPTS or exec plugin).

External Resources

• PWAF – Partner Well-Architected Framework
• PWAF – Telemetry & Attribution
• PWAF – Integration Requirements
• Databricks OSS JDBC Driver
• OAuth M2M Documentation