databricks-isv-java-sdk

<!-- skill-version: 1.0.0 --> # Databricks SDK for Java (ISV) Use this skill when implementing or testing **Databricks SDK for Java** (`databricks-sdk-java`) integrations for PWAF-compliant workspace management, Unity Catalog, Jobs, and REST-style API access. ## PWAF Documentation Links | Resource | URL | |----------|-----| | **PWAF Home** | https://databrickslabs.github.io/partner-architecture/ | | **Authentication Best Practices** | https://databrickslabs.github.io/partner-architecture/isv-partners/lakehouse-patterns/access-auth/ | | **OAuth M2M Guide** | https://databrickslabs.github.io/partner-architecture/isv-partners/lakehouse-patterns/access-auth/oauth-m2m | | **OAuth U2M Guide** | https://databrickslabs.github.io/partner-architecture/isv-partners/lakehouse-patterns/access-auth/oauth-u2m | | **User-Agent Telemetry** | https://databrickslabs.github.io/partner-architecture/isv-partners/telemetry-attribution/sdks | | **SDK Docs** | https://docs.databricks.com/aws/en/dev-tools/sdk-java | | **SDK GitHub** | https://github.com/databricks/databricks-sdk-java | ## Requirements - **SDK:** `com.databricks:databricks-sdk-java:0.54.0` (or later) from Maven Central - **Java:** 11+ required - **Install:** Add to Maven `pom.xml` (see dependencies section below) ## Authentication Decision Guide ``` Which authentication method to use? Production / automated workloads? → OAuth M2M (client credentials) ✅ RECOMMENDED User-interactive flows? → U2M External Browser (custom OAuth app) ✅ SUPPORTED Already have an OAuth access token? → U2M Token-Env ✅ SUPPORTED Local development/testing only? → PAT (Personal Access Token) ⚠️ LIMITED ``` ## Authentication Comparison | Method | PWAF Status | Auto Token Refresh | Browser Required | Use Case | |--------|-------------|-------------------|------------------|----------| | PAT | ⚠️ Limited | No | No | Testing only | | OAuth M2M | ✅ Recommended | Yes (SDK handles) | No | Production/automated | | U2M Custom OAuth App | ✅ Supported | No | Yes | User-interactive (ISV) | | U2M Token-Env | ✅ Supported | No | No | Headless/CI | **Note:** These examples do NOT use the SDK's built-in `databricks-cli` OAuth app. ISV/partner applications should register custom OAuth apps in App connections for proper branding, audit trails, and scoped permissions. ## Auth Mapping (DatabricksConfig) | Auth | DatabricksConfig Fields | Token Management | |------|-------------------------|------------------| | PAT | `.setHost()`, `.setToken()` | Manual (no refresh) | | OAuth M2M | `.setHost()`, `.setClientId()`, `.setClientSecret()` | SDK handles token fetch + refresh | | U2M Custom OAuth App | `.setHost()`, `.setAuthType("external-browser")`, `.setClientId()`, `.setClientSecret()`, `.setScopes()` | SDK opens browser with custom app | | U2M Token-Env | `.setHost()`, `.setToken()` | Manual (pre-obtained token) | ## CLIENT_ID Distinction (CRITICAL) | Variable | Purpose | Used By | |----------|---------|---------| | `DATABRICKS_CLIENT_ID` | M2M service principal UUID | OAuth M2M only | | `DATABRICKS_U2M_CLIENT_ID` | Custom OAuth app client ID | U2M custom OAuth app | **Using the wrong client_id causes:** `"OAuth application with client_id not available in Databricks account"` ## SDK vs REST APIs - **No SQL warehouse needed:** The SDK uses REST APIs (e.g., UC Tables API at `/api/2.1/unity-catalog/tables/`). No `httpPath` or `DATABRICKS_HTTP_PATH` required. - **Use cases:** Workspace management, Unity Catalog metadata, Jobs orchestration, Clusters, DBFS, etc. - **For SQL queries:** Use the Databricks JDBC Driver (`databricks-jdbc`) with a SQL warehouse httpPath. --- ## User-Agent (Required per PWAF) Call once before creating `WorkspaceClient`: ```java import com.databricks.sdk.core.UserAgent; UserAgent.withProduct("YourCompany_YourProduct", "1.0.0"); UserAgent.withPartner("YourCompany"); ``` These are **static/global** registrations applied to all SDK HTTP requests in the JVM. --- ## Environment Variables Reference | Variable | Required For | Description | |----------|-------------|-------------| | `DATABRICKS_HOST` | All | Workspace URL (e.g., `https://myworkspace.cloud.databricks.com`) | | `DATABRICKS_TOKEN` | PAT | Personal access token | | `DATABRICKS_CLIENT_ID` | OAuth M2M | Service principal UUID | | `DATABRICKS_CLIENT_SECRET` | OAuth M2M | Service principal OAuth secret | | `DATABRICKS_U2M_CLIENT_ID` | U2M Custom OAuth | Custom OAuth app client ID from App connections | | `DATABRICKS_U2M_CLIENT_SECRET` | U2M Custom OAuth | Custom OAuth app client secret (Java SDK requires this) | | `DATABRICKS_REDIRECT_URI` | U2M Custom OAuth (optional) | Custom redirect URI (default: `http://localhost:8080/callback`) | | `DATABRICKS_ACCESS_TOKEN` | U2M Token-Env | Pre-obtained OAuth access token | | `APP_AUTH_TYPE` | Multi-auth | App-level auth selector: `pat`, `oauth_m2m`, `u2m_custom_oauth_app`, `u2m_token_env` | **Important:** - Use `APP_AUTH_TYPE` (not `DATABRICKS_AUTH_TYPE`) because the SDK reads `DATABRICKS_AUTH_TYPE` internally. - Do not mix M2M and U2M environment variables. Use `env -i` for clean test environments. --- ## Complete Examples ### PAT Authentication (Testing Only) ```java import com.databricks.sdk.WorkspaceClient; import com.databricks.sdk.core.DatabricksConfig; import com.databricks.sdk.core.UserAgent; import com.databricks.sdk.service.catalog.TableInfo; public class SdkPatExample { public static void main(String[] args) { UserAgent.withProduct("YourCompany_YourProduct", "1.0.0"); UserAgent.withPartner("YourCompany"); String host = System.getenv("DATABRICKS_HOST"); String token = System.getenv("DATABRICKS_TOKEN"); DatabricksConfig config = new DatabricksConfig() .setHost(ensureScheme(host)) .setToken(token); WorkspaceClient client = new WorkspaceClient(config); TableInfo table = client.tables().get("samples.nyctaxi.trips"); System.out.println("PAT OK: " + table.getName() + " (" + table.getColumns().size() + " columns)"); } private static String ensureScheme(String host) { if (!host.startsWith("https://") && !host.startsWith("http://")) { return "https://" + host; } return host; } } ``` **Env vars:** `DATABRICKS_HOST`, `DATABRICKS_TOKEN` --- ### OAuth M2M Authentication (Production Recommended) ```java import com.databricks.sdk.WorkspaceClient; import com.databricks.sdk.core.DatabricksConfig; import com.databricks.sdk.core.UserAgent; import com.databricks.sdk.service.catalog.TableInfo; public class SdkOAuthM2MExample { public static void main(String[] args) { UserAgent.withProduct("YourCompany_YourProduct", "1.0.0"); UserAgent.withPartner("YourCompany"); String host = System.getenv("DATABRICKS_HOST"); String clientId = System.getenv("DATABRICKS_CLIENT_ID"); String clientSecret = System.getenv("DATABRICKS_CLIENT_SECRET"); DatabricksConfig config = new DatabricksConfig() .setHost(ensureScheme(host)) .setClientId(clientId) .setClientSecret(clientSecret); WorkspaceClient client = new WorkspaceClient(config); TableInfo table = client.tables().get("samples.nyctaxi.trips"); System.out.println("OAuth M2M OK: " + table.getName() + " (" + table.getColumns().size() + " columns)"); } private static String ensureScheme(String host) { if (!host.startsWith("https://") && !host.startsWith("http://")) { return "https://" + host; } return host; } } ``` **Env vars:** `DATABRICKS_HOST`, `DATABRICKS_CLIENT_ID`, `DATABRICKS_CLIENT_SECRET` **How it works:** 1. SDK POSTs to `https://<host>/oidc/v1/token` with `grant_type=client_credentials` 2. Receives `access_token` and expiry 3. Attaches token as `Authorization: Bearer <token>` on every REST call 4. Refreshes automatically before expiry **Setup:** Create service principal in Account Console → Settings → Service principals. Generate OAuth secret. --- ### U2M Custom OAuth App Uses a custom OAuth app registered in App connections: ```java import com.databricks.sdk.WorkspaceClient; import com.databricks.sdk.core.DatabricksConfig; import com.databricks.sdk.core.UserAgent; import com.databricks.sdk.service.catalog.TableInfo; import java.util.Arrays; public class SdkU2MCustomOAuthAppExample { public static void main(String[] args) { UserAgent.withProduct("YourCompany_YourProduct", "1.0.0"); UserAgent.withPartner("YourCompany"); String host = System.getenv("DATABRICKS_HOST"); String clientId = System.getenv("DATABRICKS_U2M_CLIENT_ID"); String clientSecret = System.getenv("DATABRICKS_U2M_CLIENT_SECRET"); // required by Java SDK String redirectUri = System.getenv("DATABRICKS_REDIRECT_URI"); DatabricksConfig config = new DatabricksConfig() .setHost(ensureScheme(host)) .setAuthType("external-browser") .setClientId(clientId) .setScopes(Arrays.asList("all-apis")); if (clientSecret != null && !clientSecret.isBlank()) { config.setClientSecret(clientSecret); } if (redirectUri != null && !redirectUri.isBlank()) { config.setOAuthRedirectUrl(redirectUri); } config.resolve(); WorkspaceClient client = new WorkspaceClient(config); TableInfo table = client.tables().get("samples.nyctaxi.trips"); System.out.println("U2M Custom OAuth App OK: " + table.getName()); } private static String ensureScheme(String host) { if (!host.startsWith("https://") && !host.startsWith("http://")) { return "https://" + host; } return host; } } ``` **Env vars:** `DATABRICKS_HOST`, `DATABRICKS_U2M_CLIENT_ID` **Recommended:** `DATABRICKS_U2M_CLIENT_SECRET` (required by Java SDK for proper auth configuration) **Optional:** `DATABRICKS_REDIRECT_URI` (must match App connections registration) **CRITICAL:** Use `DATABRICKS_U2M_CLIENT_ID` for custom OAuth apps — NOT `DATABRICKS_CLIENT_ID` (that's for M2M). **Note:** Testing showed the Java SDK requires `DATABRICKS_U2M_CLIENT_SECRET` for custom OAuth apps to properly configure external-browser auth. Without it, you may get `"cannot configure default credentials"` errors. --- ### U2M Token-Env (Pre-obtained Token) ```java import com.databricks.sdk.WorkspaceClient; import com.databricks.sdk.core.DatabricksConfig; import com.databricks.sdk.core.UserAgent; import com.databricks.sdk.service.catalog.TableInfo; public class SdkU2MTokenEnvExample { public static void main(String[] args) { UserAgent.withProduct("YourCompany_YourProduct", "1.0.0"); UserAgent.withPartner("YourCompany"); String host = System.getenv("DATABRICKS_HOST"); String accessToken = System.getenv("DATABRICKS_ACCESS_TOKEN"); if (accessToken == null || accessToken.isBlank()) { accessToken = System.getenv("DATABRICKS_TOKEN"); } DatabricksConfig config = new DatabricksConfig()