databricks-isv-java-jdbc

<!-- skill-version: 1.0.0 --> # Databricks JDBC Driver (ISV) Use this skill when implementing or testing **Databricks JDBC Driver** (`databricks-jdbc` v3+) connectivity for PWAF-compliant SQL warehouse integrations. ## PWAF Documentation Links | Resource | URL | |----------|-----| | **PWAF Home** | https://databrickslabs.github.io/partner-architecture/ | | **Authentication Best Practices** | https://databrickslabs.github.io/partner-architecture/isv-partners/lakehouse-patterns/access-auth/ | | **OAuth M2M Guide** | https://databrickslabs.github.io/partner-architecture/isv-partners/lakehouse-patterns/access-auth/oauth-m2m | | **OAuth U2M Guide** | https://databrickslabs.github.io/partner-architecture/isv-partners/lakehouse-patterns/access-auth/oauth-u2m | | **User-Agent Telemetry** | https://databrickslabs.github.io/partner-architecture/isv-partners/telemetry-attribution/sdks | | **JDBC OSS Docs** | https://docs.databricks.com/en/integrations/jdbc-oss/ | | **JDBC GitHub** | https://github.com/databricks/databricks-jdbc | ## Requirements - **Driver:** `com.databricks:databricks-jdbc:3.1.1` (or later 3.x) from Maven Central. OSS driver only. - **SDK:** `com.databricks:databricks-sdk-java:0.54.0` – Required for U2M browser-based token acquisition. Not needed for PAT/M2M/token-env. - **Java:** 11+ required. Java 17+ needs `--add-opens=java.base/java.nio=ALL-UNNAMED`. - **User-Agent (required):** Set `UserAgentEntry` on every connection URL (format `<isv>_<product>/<version>`). ## Authentication Decision Guide ``` Which authentication method to use? Production / automated workloads? → OAuth M2M (Auth_Flow=1) ✅ RECOMMENDED User-interactive (ISV custom OAuth app)? → U2M Custom OAuth App (SDK → Auth_Flow=0) ✅ SUPPORTED Already have an OAuth access token? → U2M Token-Env (Auth_Flow=0) ✅ SUPPORTED Local development/testing only? → PAT (AuthMech=3) ⚠️ LIMITED ``` **Note:** These examples do NOT use the SDK's built-in `databricks-cli` OAuth app. ISV/partner applications should register custom OAuth apps in App connections for proper branding, audit trails, and scoped permissions. ## Authentication Comparison | Method | PWAF Status | JDBC Auth | Token Source | Browser | |--------|-------------|-----------|--------------|---------| | PAT | ⚠️ Limited | AuthMech=3 | User provides | No | | OAuth M2M | ✅ Recommended | Auth_Flow=1 | Driver handles | No | | U2M Custom OAuth App | ✅ Supported | Auth_Flow=0 | Java SDK (custom app) | Yes | | U2M Token-Env | ✅ Supported | Auth_Flow=0 | Pre-obtained from env | No | ## Auth Mapping (JDBC Parameters) | Auth | JDBC Parameters | Token Source | |------|-----------------|--------------| | PAT | `AuthMech=3`, `UID=token`, `PWD=<token>` | User provides PAT directly | | OAuth M2M | `AuthMech=11`, `Auth_Flow=1`, `OAuth2ClientId`, `OAuth2Secret` | JDBC driver does client-credentials | | OAuth U2M (custom app) | `AuthMech=11`, `Auth_Flow=0`, `Auth_AccessToken` | Java SDK external-browser (custom OAuth app) | | OAuth U2M (token-env) | `AuthMech=11`, `Auth_Flow=0`, `Auth_AccessToken` | Pre-obtained from env variable | Both U2M flows use the same JDBC parameters (`Auth_Flow=0` token pass-through). The difference is how the token is obtained. ## CLIENT_ID Distinction (CRITICAL) | Variable | Purpose | Used By | |----------|---------|---------| | `DATABRICKS_CLIENT_ID` | M2M service principal UUID | OAuth M2M only | | `DATABRICKS_U2M_CLIENT_ID` | Custom OAuth app client ID | U2M custom OAuth app | **Using the wrong client_id causes:** `"OAuth application with client_id not available in Databricks account"` ## Why SDK + Auth_Flow=0 Instead of JDBC Auth_Flow=2? The JDBC driver's native `Auth_Flow=2` uses a built-in OAuth app with a hardcoded redirect port (8020). It does **not** support custom OAuth apps or custom redirect URIs. For ISV integrations that need custom OAuth apps, the proven pattern is: 1. Use the **Databricks Java SDK** (`DatabricksConfig` with `authType="external-browser"`) to obtain the token 2. Pass the token to the **JDBC driver** via `Auth_Flow=0` (`Auth_AccessToken`) This approach supports custom OAuth apps with configurable redirect URIs, which is required for proper ISV branding and audit trails. ## Compute / Target - **SQL warehouse only:** Connection uses `httpPath` (e.g., `/sql/1.0/warehouses/<id>`). The JDBC driver does not support jobs compute. - **Config:** Require either `DATABRICKS_HTTP_PATH` or `DATABRICKS_WAREHOUSE_ID` (path derived as `/sql/1.0/warehouses/<id>`). --- ## User-Agent (Required per PWAF) Set on every JDBC connection URL: ```java String url = "jdbc:databricks://" + host + ":443" + ";httpPath=" + httpPath + ";AuthMech=3;UID=token;PWD=" + token + ";UserAgentEntry=YourCompany_YourProduct/1.0.0"; ``` For SDK operations, also register static telemetry: ```java import com.databricks.sdk.core.UserAgent; UserAgent.withProduct("YourCompany_YourProduct", "1.0.0"); UserAgent.withPartner("YourCompany"); ``` --- ## Environment Variables Reference | Variable | Required For | Description | |----------|-------------|-------------| | `DATABRICKS_HOST` | All | Workspace URL (e.g., `https://myworkspace.cloud.databricks.com`) | | `DATABRICKS_HTTP_PATH` | All | SQL warehouse HTTP path (e.g., `/sql/1.0/warehouses/abc123`) | | `DATABRICKS_WAREHOUSE_ID` | Alternative | Warehouse ID (path derived as `/sql/1.0/warehouses/<id>`) | | `DATABRICKS_TOKEN` | PAT | Personal access token | | `DATABRICKS_CLIENT_ID` | OAuth M2M | Service principal UUID | | `DATABRICKS_CLIENT_SECRET` | OAuth M2M | Service principal OAuth secret | | `DATABRICKS_U2M_CLIENT_ID` | U2M Custom OAuth | Custom OAuth app client ID from App connections | | `DATABRICKS_U2M_CLIENT_SECRET` | U2M Custom OAuth | Custom OAuth app client secret (Java SDK requires this) | | `DATABRICKS_REDIRECT_URI` | U2M Custom OAuth (optional) | Custom redirect URI (default: `http://localhost:8080/callback`) | | `DATABRICKS_ACCESS_TOKEN` | U2M Token-Env | Pre-obtained OAuth access token | | `DATABRICKS_AUTH_TYPE` | Multi-auth | Auth type selector: `pat`, `oauth_m2m`, `u2m_custom_oauth_app`, `u2m_token_env` | **Important:** Do not mix M2M and U2M environment variables. Use `env -i` for clean test environments. --- ## Complete Examples ### PAT Authentication (Testing Only) ```java import java.sql.Connection; import java.sql.DriverManager; import java.sql.ResultSet; import java.sql.Statement; public class JdbcPatExample { public static void main(String[] args) throws Exception { String host = System.getenv("DATABRICKS_HOST"); String token = System.getenv("DATABRICKS_TOKEN"); String httpPath = System.getenv("DATABRICKS_HTTP_PATH"); // Normalize host host = host.replaceFirst("^https://", "").replaceFirst("^http://", ""); String url = "jdbc:databricks://" + host + ":443" + ";httpPath=" + httpPath + ";AuthMech=3" + ";UID=token" + ";PWD=" + token + ";UserAgentEntry=YourCompany_YourProduct/1.0.0"; try (Connection conn = DriverManager.getConnection(url); Statement stmt = conn.createStatement(); ResultSet rs = stmt.executeQuery("SELECT 1 as test")) { if (rs.next()) { System.out.println("PAT OK: " + rs.getInt("test")); } } } } ``` **Env vars:** `DATABRICKS_HOST`, `DATABRICKS_TOKEN`, `DATABRICKS_HTTP_PATH` --- ### OAuth M2M Authentication (Production Recommended) ```java import java.sql.Connection; import java.sql.DriverManager; import java.sql.ResultSet; import java.sql.Statement; public class JdbcOAuthM2MExample { public static void main(String[] args) throws Exception { String host = System.getenv("DATABRICKS_HOST"); String clientId = System.getenv("DATABRICKS_CLIENT_ID"); String clientSecret = System.getenv("DATABRICKS_CLIENT_SECRET"); String httpPath = System.getenv("DATABRICKS_HTTP_PATH"); host = host.replaceFirst("^https://", "").replaceFirst("^http://", ""); String url = "jdbc:databricks://" + host + ":443" + ";httpPath=" + httpPath + ";AuthMech=11" + ";Auth_Flow=1" + ";OAuth2ClientId=" + clientId + ";OAuth2Secret=" + clientSecret + ";UserAgentEntry=YourCompany_YourProduct/1.0.0"; try (Connection conn = DriverManager.getConnection(url); Statement stmt = conn.createStatement(); ResultSet rs = stmt.executeQuery("SELECT 1 as test")) { if (rs.next()) { System.out.println("OAuth M2M OK: " + rs.getInt("test")); } } } } ``` **Env vars:** `DATABRICKS_HOST`, `DATABRICKS_CLIENT_ID`, `DATABRICKS_CLIENT_SECRET`, `DATABRICKS_HTTP_PATH` **Setup:** Create service principal in Account Console → Settings → Service principals. Generate OAuth secret. --- ### U2M Custom OAuth App (SDK → JDBC) Uses a custom OAuth app registered in App connections: ```java import com.databricks.sdk.core.DatabricksConfig; import com.databricks.sdk.core.UserAgent; import java.sql.Connection; import java.sql.DriverManager; import java.util.Arrays; import java.util.Map; public class JdbcU2MCustomOAuthAppExample { public static void main(String[] args) throws Exception { String host = System.getenv("DATABRICKS_HOST"); String clientId = System.getenv("DATABRICKS_U2M_CLIENT_ID"); String clientSecret = System.getenv("DATABRICKS_U2M_CLIENT_SECRET"); // required by Java SDK String redirectUri = System.getenv("DATABRICKS_REDIRECT_URI"); String httpPath = System.getenv("DATABRICKS_HTTP_PATH"); UserAgent.withProduct("YourCompany_YourProduct", "1.0.0"); UserAgent.withPartner("YourCompany"); // Step 1: Get token via SDK with custom OAuth app String normalizedHost = host.startsWith("https://") ? host : "https://" + host; DatabricksConfig config = new DatabricksConfig() .setHost(normalizedHost) .setAuthType("external-browser") .setClientId(clientId) .setScopes(Arrays.asList("all-apis")); if (clientSecret != null && !clientSecret.isBlank()) { config.setClientSecret(clientSecret); } if (redirectUri != null && !redirectUri.isBlank()) { config.setOAuthRedirectUrl(redirectUri); } config.resolve(); Map<String, String> headers = config.authenticate(); String token = headers.get("Authorization").substring("Bearer ".length()); // Step 2: Connect via JDBC String hostNorm = host.replaceFirst("^https://", "").replaceFirst("^http://", ""); String url = "jdbc:databricks://" + hostNorm + ":443" + ";httpPath=" + httpPath + ";AuthMech=11;Auth_Flow=0;Auth_AccessToken=" + token + ";UserAgentEntry=YourCompany_YourProduct/1.0.0"; try (Connection conn = DriverManager.getConnection(url)) {