| name | databricks-isv-java-sdk |
| description | PWAF-compliant Databricks SDK for Java (databricks-sdk-java): PAT, OAuth M2M, U2M custom OAuth app, UserAgent.withProduct/withPartner. Use when building or testing Java SDK workspace API integrations. |
Databricks SDK for Java (ISV)
Use this skill when implementing or testing Databricks SDK for Java (databricks-sdk-java) integrations for PWAF-compliant workspace management, Unity Catalog, Jobs, and REST-style API access.
PWAF Documentation Links
Requirements
- SDK:
com.databricks:databricks-sdk-java:0.54.0 (or later) from Maven Central
- Java: 11+ required
- Install: Add to Maven
pom.xml (see dependencies section below)
Authentication Decision Guide
Which authentication method to use?
Production / automated workloads?
→ OAuth M2M (client credentials) ✅ RECOMMENDED
User-interactive flows?
→ U2M External Browser (custom OAuth app) ✅ SUPPORTED
Already have an OAuth access token?
→ U2M Token-Env ✅ SUPPORTED
Local development/testing only?
→ PAT (Personal Access Token) ⚠️ LIMITED
Authentication Comparison
| Method | PWAF Status | Auto Token Refresh | Browser Required | Use Case |
|---|
| PAT | ⚠️ Limited | No | No | Testing only |
| OAuth M2M | ✅ Recommended | Yes (SDK handles) | No | Production/automated |
| U2M Custom OAuth App | ✅ Supported | No | Yes | User-interactive (ISV) |
| U2M Token-Env | ✅ Supported | No | No | Headless/CI |
Note: These examples do NOT use the SDK's built-in databricks-cli OAuth app. ISV/partner applications should register custom OAuth apps in App connections for proper branding, audit trails, and scoped permissions.
Auth Mapping (DatabricksConfig)
| Auth | DatabricksConfig Fields | Token Management |
|---|
| PAT | .setHost(), .setToken() | Manual (no refresh) |
| OAuth M2M | .setHost(), .setClientId(), .setClientSecret() | SDK handles token fetch + refresh |
| U2M Custom OAuth App | .setHost(), .setAuthType("external-browser"), .setClientId(), .setClientSecret(), .setScopes() | SDK opens browser with custom app |
| U2M Token-Env | .setHost(), .setToken() | Manual (pre-obtained token) |
CLIENT_ID Distinction (CRITICAL)
| Variable | Purpose | Used By |
|---|
DATABRICKS_CLIENT_ID | M2M service principal UUID | OAuth M2M only |
DATABRICKS_U2M_CLIENT_ID | Custom OAuth app client ID | U2M custom OAuth app |
Using the wrong client_id causes: "OAuth application with client_id not available in Databricks account"
SDK vs REST APIs
- No SQL warehouse needed: The SDK uses REST APIs (e.g., UC Tables API at
/api/2.1/unity-catalog/tables/). No httpPath or DATABRICKS_HTTP_PATH required.
- Use cases: Workspace management, Unity Catalog metadata, Jobs orchestration, Clusters, DBFS, etc.
- For SQL queries: Use the Databricks JDBC Driver (
databricks-jdbc) with a SQL warehouse httpPath.
User-Agent (Required per PWAF)
Call once before creating WorkspaceClient:
import com.databricks.sdk.core.UserAgent;
UserAgent.withProduct("YourCompany_YourProduct", "1.0.0");
UserAgent.withPartner("YourCompany");
These are static/global registrations applied to all SDK HTTP requests in the JVM.
Environment Variables Reference
| Variable | Required For | Description |
|---|
DATABRICKS_HOST | All | Workspace URL (e.g., https://myworkspace.cloud.databricks.com) |
DATABRICKS_TOKEN | PAT | Personal access token |
DATABRICKS_CLIENT_ID | OAuth M2M | Service principal UUID |
DATABRICKS_CLIENT_SECRET | OAuth M2M | Service principal OAuth secret |
DATABRICKS_U2M_CLIENT_ID | U2M Custom OAuth | Custom OAuth app client ID from App connections |
DATABRICKS_U2M_CLIENT_SECRET | U2M Custom OAuth | Custom OAuth app client secret (Java SDK requires this) |
DATABRICKS_REDIRECT_URI | U2M Custom OAuth (optional) | Custom redirect URI (default: http://localhost:8080/callback) |
DATABRICKS_ACCESS_TOKEN | U2M Token-Env | Pre-obtained OAuth access token |
APP_AUTH_TYPE | Multi-auth | App-level auth selector: pat, oauth_m2m, u2m_custom_oauth_app, u2m_token_env |
Important:
- Use
APP_AUTH_TYPE (not DATABRICKS_AUTH_TYPE) because the SDK reads DATABRICKS_AUTH_TYPE internally.
- Do not mix M2M and U2M environment variables. Use
env -i for clean test environments.
Complete Examples
PAT Authentication (Testing Only)
import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
import com.databricks.sdk.core.UserAgent;
import com.databricks.sdk.service.catalog.TableInfo;
public class SdkPatExample {
public static void main(String[] args) {
UserAgent.withProduct("YourCompany_YourProduct", "1.0.0");
UserAgent.withPartner("YourCompany");
String host = System.getenv("DATABRICKS_HOST");
String token = System.getenv("DATABRICKS_TOKEN");
DatabricksConfig config = new DatabricksConfig()
.setHost(ensureScheme(host))
.setToken(token);
WorkspaceClient client = new WorkspaceClient(config);
TableInfo table = client.tables().get("samples.nyctaxi.trips");
System.out.println("PAT OK: " + table.getName()
+ " (" + table.getColumns().size() + " columns)");
}
private static String ensureScheme(String host) {
if (!host.startsWith("https://") && !host.startsWith("http://")) {
return "https://" + host;
}
return host;
}
}
Env vars: DATABRICKS_HOST, DATABRICKS_TOKEN
OAuth M2M Authentication (Production Recommended)
import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
import com.databricks.sdk.core.UserAgent;
import com.databricks.sdk.service.catalog.TableInfo;
public class SdkOAuthM2MExample {
public static void main(String[] args) {
UserAgent.withProduct("YourCompany_YourProduct", "1.0.0");
UserAgent.withPartner("YourCompany");
String host = System.getenv("DATABRICKS_HOST");
String clientId = System.getenv("DATABRICKS_CLIENT_ID");
String clientSecret = System.getenv("DATABRICKS_CLIENT_SECRET");
DatabricksConfig config = new DatabricksConfig()
.setHost(ensureScheme(host))
.setClientId(clientId)
.setClientSecret(clientSecret);
WorkspaceClient client = new WorkspaceClient(config);
TableInfo table = client.tables().get("samples.nyctaxi.trips");
System.out.println("OAuth M2M OK: " + table.getName()
+ " (" + table.getColumns().size() + " columns)");
}
private static String ensureScheme(String host) {
if (!host.startsWith("https://") && !host.startsWith("http://")) {
return "https://" + host;
}
return host;
}
}
Env vars: DATABRICKS_HOST, DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET
How it works:
- SDK POSTs to
https://<host>/oidc/v1/token with grant_type=client_credentials
- Receives
access_token and expiry
- Attaches token as
Authorization: Bearer <token> on every REST call
- Refreshes automatically before expiry
Setup: Create service principal in Account Console → Settings → Service principals. Generate OAuth secret.
U2M Custom OAuth App
Uses a custom OAuth app registered in App connections:
import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
import com.databricks.sdk.core.UserAgent;
import com.databricks.sdk.service.catalog.TableInfo;
import java.util.Arrays;
public class SdkU2MCustomOAuthAppExample {
public static void main(String[] args) {
UserAgent.withProduct("YourCompany_YourProduct", "1.0.0");
UserAgent.withPartner("YourCompany");
String host = System.getenv("DATABRICKS_HOST");
String clientId = System.getenv("DATABRICKS_U2M_CLIENT_ID");
String clientSecret = System.getenv("DATABRICKS_U2M_CLIENT_SECRET");
String redirectUri = System.getenv("DATABRICKS_REDIRECT_URI");
DatabricksConfig config = new DatabricksConfig()
.setHost(ensureScheme(host))
.setAuthType("external-browser")
.setClientId(clientId)
.setScopes(Arrays.asList("all-apis"));
if (clientSecret != null && !clientSecret.isBlank()) {
config.setClientSecret(clientSecret);
}
if (redirectUri != null && !redirectUri.isBlank()) {
config.setOAuthRedirectUrl(redirectUri);
}
config.resolve();
WorkspaceClient client = new WorkspaceClient(config);
TableInfo table = client.tables().get("samples.nyctaxi.trips");
System.out.println("U2M Custom OAuth App OK: " + table.getName());
}
private static String ensureScheme(String host) {
if (!host.startsWith("https://") && !host.startsWith("http://")) {
return "https://" + host;
}
return host;
}
}
Env vars: DATABRICKS_HOST, DATABRICKS_U2M_CLIENT_ID
Recommended: DATABRICKS_U2M_CLIENT_SECRET (required by Java SDK for proper auth configuration)
Optional: DATABRICKS_REDIRECT_URI (must match App connections registration)
CRITICAL: Use DATABRICKS_U2M_CLIENT_ID for custom OAuth apps — NOT DATABRICKS_CLIENT_ID (that's for M2M).
Note: Testing showed the Java SDK requires DATABRICKS_U2M_CLIENT_SECRET for custom OAuth apps to properly configure external-browser auth. Without it, you may get "cannot configure default credentials" errors.
U2M Token-Env (Pre-obtained Token)
import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
import com.databricks.sdk.core.UserAgent;
import com.databricks.sdk.service.catalog.TableInfo;
public class SdkU2MTokenEnvExample {
public static void main(String[] args) {
UserAgent.withProduct("YourCompany_YourProduct", "1.0.0");
UserAgent.withPartner("YourCompany");
String host = System.getenv("DATABRICKS_HOST");
String accessToken = System.getenv("DATABRICKS_ACCESS_TOKEN");
if (accessToken == null || accessToken.isBlank()) {
accessToken = System.getenv("DATABRICKS_TOKEN");
}
DatabricksConfig config = new DatabricksConfig()
.setHost(ensureScheme(host))
.setToken(accessToken);
WorkspaceClient client = new WorkspaceClient(config);
TableInfo table = client.tables().get("samples.nyctaxi.trips");
System.out.println("U2M Token-Env OK: " + table.getName());
}
private static String ensureScheme(String host) {
if (!host.startsWith("https://") && !host.startsWith("http://")) {
return "https://" + host;
}
return host;
}
}
Env vars: DATABRICKS_HOST, DATABRICKS_ACCESS_TOKEN (or DATABRICKS_TOKEN)
Token Refresh for U2M Flows
U2M access tokens expire after approximately 1 hour. For long-running applications, you need to handle token refresh.
Token Lifetime Summary
| Auth Type | Token TTL | Refresh Strategy |
|---|
| PAT | User-configured (90 days default) | Generate new token manually |
| OAuth M2M | ~1 hour | SDK handles automatically |
| U2M Custom OAuth App | ~1 hour | Re-authenticate or use refresh_token |
| U2M Token-Env | ~1 hour | Application must handle |
SDK Token Refresh Pattern (Recommended)
The SDK's DatabricksConfig.authenticate() method automatically handles token refresh when using persistent configuration. For single-shot scripts, re-run the browser flow when tokens expire:
public class TokenRefreshExample {
private DatabricksConfig config;
private WorkspaceClient client;
private long tokenExpiresAt;
private static final long REFRESH_BUFFER_MS = 5 * 60 * 1000;
public TokenRefreshExample() {
this.config = createConfig();
this.client = new WorkspaceClient(config);
this.tokenExpiresAt = System.currentTimeMillis() + (55 * 60 * 1000);
}
private DatabricksConfig createConfig() {
String host = System.getenv("DATABRICKS_HOST");
String clientId = System.getenv("DATABRICKS_U2M_CLIENT_ID");
String clientSecret = System.getenv("DATABRICKS_U2M_CLIENT_SECRET");
DatabricksConfig cfg = new DatabricksConfig()
.setHost(host.startsWith("https://") ? host : "https://" + host)
.setAuthType("external-browser")
.setClientId(clientId)
.setClientSecret(clientSecret)
.setScopes(Arrays.asList("all-apis", "offline_access"));
cfg.resolve();
return cfg;
}
public WorkspaceClient getClient() {
if (System.currentTimeMillis() > (tokenExpiresAt - REFRESH_BUFFER_MS)) {
this.config = createConfig();
this.client = new WorkspaceClient(config);
this.tokenExpiresAt = System.currentTimeMillis() + (55 * 60 * 1000);
}
return client;
}
}
Manual Refresh Token Flow (Advanced)
For applications that need to refresh without user interaction, request offline_access scope to receive a refresh_token:
config.setScopes(Arrays.asList("all-apis", "offline_access"));
Important: The offline_access scope must be enabled for your OAuth app in App connections.
Best Practices
- Request
offline_access: Include this scope for long-running applications
- Check expiration before operations: Avoid mid-operation failures
- Handle refresh failures gracefully: Fall back to re-authentication
- Don't cache tokens in code: Use SDK token cache or secure storage
Environment Variables Reference
| Variable | Used By | Description |
|---|
DATABRICKS_HOST | All | Workspace URL |
DATABRICKS_TOKEN | PAT, U2M token-env | Personal access token or pre-obtained OAuth token |
DATABRICKS_CLIENT_ID | OAuth M2M | Service principal UUID |
DATABRICKS_CLIENT_SECRET | OAuth M2M | Service principal OAuth secret |
DATABRICKS_U2M_CLIENT_ID | U2M Custom OAuth | Custom OAuth app client ID |
DATABRICKS_U2M_CLIENT_SECRET | U2M Custom OAuth | Custom OAuth app secret (if confidential) |
DATABRICKS_REDIRECT_URI | U2M Custom OAuth | Redirect URI (default: http://localhost:8080/callback) |
APP_AUTH_TYPE | Multi-auth binary | App-level auth type selector (NOT DATABRICKS_AUTH_TYPE) |
Required Imports / Maven Dependencies
<dependency>
<groupId>com.databricks</groupId>
<artifactId>databricks-sdk-java</artifactId>
<version>0.54.0</version>
</dependency>
<dependency>
<groupId>org.slf4j</groupId>
<artifactId>slf4j-nop</artifactId>
<version>2.0.16</version>
</dependency>
Pin to specific minor version (SDK is in Beta). Java 11+ required.
Multi-Auth Pattern
Support multiple auth types in a single binary using a custom selector (e.g., APP_AUTH_TYPE):
String authType = System.getenv("APP_AUTH_TYPE");
if (authType == null || authType.isBlank()) authType = "oauth_m2m";
DatabricksConfig config = new DatabricksConfig()
.setHost(ensureScheme(host));
switch (authType.toLowerCase()) {
case "pat":
config.setToken(System.getenv("DATABRICKS_TOKEN"));
break;
case "oauth_m2m":
config.setClientId(System.getenv("DATABRICKS_CLIENT_ID"));
config.setClientSecret(System.getenv("DATABRICKS_CLIENT_SECRET"));
break;
case "u2m_custom_oauth_app":
config.setAuthType("external-browser");
config.setClientId(System.getenv("DATABRICKS_U2M_CLIENT_ID"));
config.setClientSecret(System.getenv("DATABRICKS_U2M_CLIENT_SECRET"));
config.setScopes(Arrays.asList("all-apis"));
String redirect = System.getenv("DATABRICKS_REDIRECT_URI");
if (redirect != null && !redirect.isBlank()) config.setOAuthRedirectUrl(redirect);
config.resolve();
break;
case "u2m_token_env":
String token = System.getenv("DATABRICKS_ACCESS_TOKEN");
if (token == null || token.isBlank()) token = System.getenv("DATABRICKS_TOKEN");
config.setToken(token);
break;
}
WorkspaceClient client = new WorkspaceClient(config);
APP_AUTH_TYPE | Auth Flow | Env Vars Required |
|---|
pat | Personal Access Token | DATABRICKS_TOKEN |
oauth_m2m (default) | OAuth M2M (client credentials) | DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET |
u2m_custom_oauth_app | U2M external browser (custom app) | DATABRICKS_U2M_CLIENT_ID, DATABRICKS_U2M_CLIENT_SECRET |
u2m_token_env | U2M pre-obtained token | DATABRICKS_ACCESS_TOKEN or DATABRICKS_TOKEN |
CRITICAL: Use APP_AUTH_TYPE (not DATABRICKS_AUTH_TYPE). The SDK reads DATABRICKS_AUTH_TYPE internally.
Implementation Learnings (Java SDK v0.54.0)
Key findings from testing that differ from documentation or other SDKs:
1. U2M Custom OAuth App Requires Client Secret
Unlike some other SDKs, the Java SDK requires DATABRICKS_U2M_CLIENT_SECRET even for public OAuth apps:
config.setClientId(System.getenv("DATABRICKS_U2M_CLIENT_ID"));
config.setClientSecret(System.getenv("DATABRICKS_U2M_CLIENT_SECRET"));
Without client secret, you get: "cannot configure default credentials"
2. SDK Reads DATABRICKS_AUTH_TYPE Internally
The SDK uses DATABRICKS_AUTH_TYPE internally. If you set this environment variable, it may conflict with programmatic config. Use a custom variable like APP_AUTH_TYPE for your own auth type selection.
3. config.resolve() Must Be Called
For external-browser auth, config.resolve() must be called after configuration but before creating WorkspaceClient:
DatabricksConfig config = new DatabricksConfig()
.setHost(host)
.setAuthType("external-browser")
.setClientId(clientId)
.setClientSecret(clientSecret)
.setScopes(Arrays.asList("all-apis"));
config.resolve();
WorkspaceClient client = new WorkspaceClient(config);
SDK Workarounds Summary
| Workaround | Code | Without It |
|---|
| Set scopes | .setScopes(Arrays.asList("all-apis")) | NullPointerException at TokenCacheUtils |
| Call resolve | config.resolve() before WorkspaceClient | NullPointerException at Consent.<init> |
| Set client secret | .setClientSecret(...) | "cannot configure default credentials" |
| Set redirect URL (if custom) | .setOAuthRedirectUrl(...) | Must match App connections registration |
⚠️ WorkspaceClient resource cleanup (SDK 0.54)
WorkspaceClient does not implement Closeable or AutoCloseable in SDK 0.54.
Do not call workspaceClient.close() — it causes a compile error: cannot find symbol: method close().
Correct pattern for a wrapper's close() method:
@Override
public void close() {
workspaceClient = null;
}
Redirect URI Reference
| Scenario | Redirect URI | Notes |
|---|
| Custom OAuth app (default) | http://localhost:8080/callback | Java SDK default |
| Custom OAuth app (configured) | Via DATABRICKS_REDIRECT_URI | Must match App connections registration |
Validation Query
Verify User-Agent telemetry is being recorded correctly:
SELECT event_time, user_agent, action_name, request_params
FROM system.access.audit
WHERE event_time > current_timestamp() - INTERVAL 1 HOUR
AND lower(user_agent) LIKE '%yourcompany%'
ORDER BY event_time DESC
LIMIT 10;
U2M Browser Environment Requirements
When using env -i for U2M browser flows, include these variables for Desktop.browse() to work:
HOME="$HOME"
PATH="$PATH"
JAVA_HOME="$JAVA_HOME"
DISPLAY="$DISPLAY"
Auth Isolation
Run tests with a clean environment using env -i:
env -i HOME="$HOME" PATH="$PATH" JAVA_HOME="$JAVA_HOME" \
DATABRICKS_HOST="$DATABRICKS_HOST" \
DATABRICKS_TOKEN="$DATABRICKS_TOKEN" \
APP_AUTH_TYPE=pat \
mvn exec:java -Dexec.mainClass="com.example.SdkPatExample"
env -i HOME="$HOME" PATH="$PATH" JAVA_HOME="$JAVA_HOME" \
DATABRICKS_HOST="$DATABRICKS_HOST" \
DATABRICKS_CLIENT_ID="$DATABRICKS_CLIENT_ID" \
DATABRICKS_CLIENT_SECRET="$DATABRICKS_CLIENT_SECRET" \
APP_AUTH_TYPE=oauth_m2m \
mvn exec:java -Dexec.mainClass="com.example.SdkOAuthM2MExample"
env -i HOME="$HOME" PATH="$PATH" JAVA_HOME="$JAVA_HOME" \
DATABRICKS_HOST="$DATABRICKS_HOST" \
DATABRICKS_U2M_CLIENT_ID="$DATABRICKS_U2M_CLIENT_ID" \
DATABRICKS_U2M_CLIENT_SECRET="$DATABRICKS_U2M_CLIENT_SECRET" \
DATABRICKS_REDIRECT_URI="$DATABRICKS_REDIRECT_URI" \
APP_AUTH_TYPE=u2m_custom_oauth_app \
mvn exec:java -Dexec.mainClass="com.example.SdkU2MCustomOAuthAppExample"
env -i HOME="$HOME" PATH="$PATH" JAVA_HOME="$JAVA_HOME" \
DATABRICKS_HOST="$DATABRICKS_HOST" \
DATABRICKS_ACCESS_TOKEN="$DATABRICKS_ACCESS_TOKEN" \
APP_AUTH_TYPE=u2m_token_env \
mvn exec:java -Dexec.mainClass="com.example.SdkU2MTokenEnvExample"
Error Handling Patterns
Production applications should handle authentication and API errors gracefully.
Basic Error Handling
import com.databricks.sdk.WorkspaceClient;
import com.databricks.sdk.core.DatabricksConfig;
import com.databricks.sdk.core.DatabricksException;
import com.databricks.sdk.service.catalog.TableInfo;
public class SdkErrorHandlingExample {
public static void main(String[] args) {
try {
WorkspaceClient client = createClient();
TableInfo table = client.tables().get("samples.nyctaxi.trips");
System.out.println("Success: " + table.getName());
} catch (DatabricksException e) {
handleDatabricksError(e);
} catch (Exception e) {
handleGeneralError(e);
}
}
private static WorkspaceClient createClient() {
String host = System.getenv("DATABRICKS_HOST");
String clientId = System.getenv("DATABRICKS_CLIENT_ID");
String clientSecret = System.getenv("DATABRICKS_CLIENT_SECRET");
if (host == null || host.isBlank()) {
throw new IllegalStateException("DATABRICKS_HOST is required");
}
if (clientId == null || clientId.isBlank()) {
throw new IllegalStateException("DATABRICKS_CLIENT_ID is required");
}
if (clientSecret == null || clientSecret.isBlank()) {
throw new IllegalStateException("DATABRICKS_CLIENT_SECRET is required");
}
DatabricksConfig config = new DatabricksConfig()
.setHost(host.startsWith("https://") ? host : "https://" + host)
.setClientId(clientId)
.setClientSecret(clientSecret);
return new WorkspaceClient(config);
}
private static void handleDatabricksError(DatabricksException e) {
String message = e.getMessage();
if (message.contains("INVALID_PARAMETER_VALUE") || message.contains("invalid_client")) {
System.err.println("Authentication failed: Invalid credentials");
System.err.println(" - Verify DATABRICKS_CLIENT_ID is correct");
System.err.println(" - Verify DATABRICKS_CLIENT_SECRET is correct");
} else if (message.contains("PERMISSION_DENIED") || message.contains("403")) {
System.err.println("Authorization failed: Insufficient permissions");
System.err.println(" - Verify service principal has required permissions");
} else if (message.contains("NOT_FOUND") || message.contains("404")) {
System.err.println("Resource not found");
} else if (message.contains("TEMPORARILY_UNAVAILABLE") || message.contains("503")) {
System.err.println("Service temporarily unavailable - retry with backoff");
} else {
System.err.println("Databricks API error: " + message);
}
}
private static void handleGeneralError(Exception e) {
String message = e.getMessage();
if (e instanceof java.net.UnknownHostException) {
System.err.println("Network error: Cannot resolve host");
System.err.println(" - Verify DATABRICKS_HOST is correct");
System.err.println(" - Check network connectivity");
} else if (e instanceof java.net.SocketTimeoutException) {
System.err.println("Network timeout - retry with backoff");
} else if (message != null && message.contains("cannot configure default credentials")) {
System.err.println("Configuration error: Missing or invalid auth configuration");
System.err.println(" - For M2M: Set DATABRICKS_CLIENT_ID and DATABRICKS_CLIENT_SECRET");
System.err.println(" - For U2M: Set DATABRICKS_U2M_CLIENT_ID and DATABRICKS_U2M_CLIENT_SECRET");
} else {
System.err.println("Unexpected error: " + message);
e.printStackTrace();
}
}
}
Retry Pattern with Exponential Backoff
public class RetryHelper {
private static final int MAX_RETRIES = 3;
private static final long INITIAL_BACKOFF_MS = 1000;
public static <T> T withRetry(java.util.function.Supplier<T> operation) {
int attempt = 0;
while (true) {
try {
return operation.get();
} catch (DatabricksException e) {
if (!isRetryable(e) || attempt >= MAX_RETRIES) {
throw e;
}
long backoff = INITIAL_BACKOFF_MS * (long) Math.pow(2, attempt);
System.err.println("Retry " + (attempt + 1) + "/" + MAX_RETRIES
+ " after " + backoff + "ms");
try {
Thread.sleep(backoff);
} catch (InterruptedException ie) {
Thread.currentThread().interrupt();
throw new RuntimeException("Interrupted during retry", ie);
}
attempt++;
}
}
}
private static boolean isRetryable(DatabricksException e) {
String msg = e.getMessage();
return msg.contains("TEMPORARILY_UNAVAILABLE")
|| msg.contains("503")
|| msg.contains("429")
|| msg.contains("RESOURCE_EXHAUSTED");
}
}
Error Classification Reference
HTTP Status Codes
| Status Code | Meaning | Retryable | Remediation |
|---|
400 | Bad request | No | Fix request parameters |
401 | Unauthorized | No | Check credentials |
403 | Forbidden | No | Check permissions |
404 | Not found | No | Verify resource exists |
429 | Rate limited | Yes | Retry with exponential backoff |
500 | Internal server error | Yes | Retry with backoff |
503 | Service unavailable | Yes | Retry with backoff |
DatabricksException Error Codes
| Error Code | Meaning | Retryable | Remediation |
|---|
INVALID_PARAMETER_VALUE | Bad input | No | Fix request parameters |
PERMISSION_DENIED | No access | No | Grant required permissions |
NOT_FOUND | Resource missing | No | Verify resource path/ID |
TEMPORARILY_UNAVAILABLE | Service down | Yes | Retry with backoff |
RESOURCE_EXHAUSTED | Rate/quota limited | Yes | Retry with backoff |
DEADLINE_EXCEEDED | Timeout | Yes | Retry with increased timeout |
Retryable Error Detection
private static boolean isRetryable(DatabricksException e) {
String msg = e.getMessage();
return msg.contains("TEMPORARILY_UNAVAILABLE")
|| msg.contains("503")
|| msg.contains("429")
|| msg.contains("RESOURCE_EXHAUSTED")
|| msg.contains("DEADLINE_EXCEEDED")
|| msg.contains("Connection reset")
|| msg.contains("Read timed out");
}
Troubleshooting
Common Errors
| Error | Cause | Solution |
|---|
more than one authorization method configured | Multiple auth env vars set | Use env -i to pass only vars for one auth type |
default auth: cannot configure default credentials | Incomplete config | Verify host + exactly one auth method's vars are set |
Unable to parse response | Host misconfigured | Verify DATABRICKS_HOST is correct |
PERMISSION_DENIED | Insufficient permissions | Grant service principal required permissions |
OAuth-Specific Errors
| Error | Cause | Solution |
|---|
OAuth application with client_id not available | Using M2M client_id for U2M | Use DATABRICKS_U2M_CLIENT_ID (custom OAuth app) |
redirect_uri mismatch | URI doesn't match registration | Verify URI in App connections matches DATABRICKS_REDIRECT_URI |
invalid_client | Wrong client ID/secret | Verify credentials match OAuth app |
SDK Errors
| Error | Cause | Solution |
|---|
NullPointerException at TokenCacheUtils | Scopes not set | .setScopes(Arrays.asList("all-apis")) |
NullPointerException at Consent.<init> | HTTP client not initialized | config.resolve() before WorkspaceClient |
DATABRICKS_AUTH_TYPE conflict | App-level var collides with SDK | Rename to APP_AUTH_TYPE |
U2M Custom OAuth App Errors
| Error | Cause | Solution |
|---|
cannot configure default credentials | Missing U2M client secret | Include DATABRICKS_U2M_CLIENT_SECRET (Java SDK requires it) |
OAuth application with client_id not available | Using M2M client_id | Use DATABRICKS_U2M_CLIENT_ID (custom OAuth app) |
redirect_uri mismatch | URI doesn't match app registration | Verify URI in App connections matches DATABRICKS_REDIRECT_URI |
Logging
| Error | Cause | Solution |
|---|
SLF4J: No SLF4J providers were found | Missing logging dependency | Add slf4j-simple or slf4j-nop to pom.xml |
Host Normalization
The SDK accepts both https://myworkspace.cloud.databricks.com and myworkspace.cloud.databricks.com. Being explicit with https:// is recommended:
private static String ensureScheme(String host) {
if (!host.startsWith("https://") && !host.startsWith("http://")) {
return "https://" + host;
}
return host;
}
Key Differences from Databricks JDBC Driver
| Aspect | Databricks SDK for Java | Databricks JDBC Driver |
|---|
| Purpose | REST APIs (UC, Jobs, Clusters) | SQL queries via warehouse |
| SQL Warehouse | Not needed | Required (httpPath) |
| Auth Config | DatabricksConfig{} setters | JDBC URL parameters (AuthMech, Auth_Flow) |
| User-Agent | UserAgent.withProduct() (global) | UserAgentEntry property (per-connection) |
| Token Refresh | SDK handles OAuth M2M refresh automatically | Driver handles M2M refresh |
| Java 17+ | No special flags needed | Needs --add-opens for Arrow |
| Auth Type Selector | Use APP_AUTH_TYPE (SDK reads DATABRICKS_AUTH_TYPE internally) | Can use DATABRICKS_AUTH_TYPE |
