| name | databricks-isv-go-sdk |
| description | PWAF-compliant Databricks SDK for Go (databricks-sdk-go): PAT, OAuth M2M, U2M token-env, U2M custom OAuth app (PKCE); useragent.WithProduct/WithPartner. Use when building or testing Go SDK workspace API integrations. |
Databricks SDK for Go (ISV)
Use this skill when implementing or testing Databricks SDK for Go (databricks-sdk-go) integrations for PWAF-compliant workspace management, Unity Catalog, Jobs, and REST-style API access.
PWAF Documentation Links
Requirements
- SDK:
github.com/databricks/databricks-sdk-go v0.107.0+
- Go: 1.21+
- Install:
go get github.com/databricks/databricks-sdk-go@latest
Authentication Decision Guide
Which authentication method to use?
Production / automated workloads?
→ OAuth M2M (client credentials) ✅ RECOMMENDED
User-interactive flows?
→ U2M Custom OAuth App (PKCE) ✅ SUPPORTED
Already have an OAuth access token?
→ U2M Token-Env ✅ SUPPORTED
Local development/testing only?
→ PAT (Personal Access Token) ⚠️ LIMITED
Authentication Comparison
| Method | PWAF Status | Auto Token Refresh | Browser Required | Use Case |
|---|
| PAT | ⚠️ Limited | No | No | Testing only |
| OAuth M2M | ✅ Recommended | Yes (SDK handles) | No | Production/automated |
| U2M Custom OAuth App | ✅ Supported | No | Yes | User-interactive |
| U2M Token-Env | ✅ Supported | No | No | Headless/CI |
User-Agent (Required per PWAF)
Call once before creating WorkspaceClient:
import "github.com/databricks/databricks-sdk-go/useragent"
useragent.WithPartner("YourCompany")
useragent.WithProduct("YourCompany_YourProduct", "1.0.0")
Environment Variables Reference
| Variable | Required For | Description |
|---|
DATABRICKS_HOST | All | Workspace URL (e.g., https://myworkspace.cloud.databricks.com) |
DATABRICKS_TOKEN | PAT | Personal access token |
DATABRICKS_CLIENT_ID | OAuth M2M | Service principal UUID |
DATABRICKS_CLIENT_SECRET | OAuth M2M | Service principal OAuth secret |
DATABRICKS_U2M_CLIENT_ID | U2M Custom OAuth | Custom OAuth app client ID from App connections |
DATABRICKS_U2M_CLIENT_SECRET | U2M Custom OAuth (optional) | Custom OAuth app client secret (Go SDK doesn't require this for public apps) |
DATABRICKS_REDIRECT_URI | U2M Custom OAuth (optional) | Custom redirect URI (default: http://localhost:8040/callback) |
DATABRICKS_ACCESS_TOKEN | U2M Token-Env | Pre-obtained OAuth access token |
APP_AUTH_TYPE | Multi-auth | App-level auth selector: pat, oauth_m2m, u2m_custom_oauth_app, u2m_token_env |
Important:
- Use
APP_AUTH_TYPE (not DATABRICKS_AUTH_TYPE) because the SDK reads DATABRICKS_AUTH_TYPE internally.
- Do not mix M2M and U2M environment variables. Use
env -i for clean test environments.
Complete Examples
PAT Authentication (Testing Only)
package main
import (
"context"
"fmt"
"os"
"github.com/databricks/databricks-sdk-go"
"github.com/databricks/databricks-sdk-go/service/catalog"
"github.com/databricks/databricks-sdk-go/useragent"
)
func main() {
useragent.WithPartner("YourCompany")
useragent.WithProduct("YourCompany_YourProduct", "1.0.0")
w, err := databricks.NewWorkspaceClient(&databricks.Config{
Host: os.Getenv("DATABRICKS_HOST"),
Token: os.Getenv("DATABRICKS_TOKEN"),
})
if err != nil {
panic(err)
}
ctx := context.Background()
table, err := w.Tables.Get(ctx, catalog.GetTableRequest{
FullName: "samples.nyctaxi.trips",
})
if err != nil {
panic(err)
}
fmt.Printf("Table: %s (%d columns)\n", table.Name, len(table.Columns))
}
Env vars: DATABRICKS_HOST, DATABRICKS_TOKEN
OAuth M2M Authentication (Production Recommended)
package main
import (
"context"
"fmt"
"os"
"github.com/databricks/databricks-sdk-go"
"github.com/databricks/databricks-sdk-go/service/catalog"
"github.com/databricks/databricks-sdk-go/useragent"
)
func main() {
useragent.WithPartner("YourCompany")
useragent.WithProduct("YourCompany_YourProduct", "1.0.0")
w, err := databricks.NewWorkspaceClient(&databricks.Config{
Host: os.Getenv("DATABRICKS_HOST"),
ClientID: os.Getenv("DATABRICKS_CLIENT_ID"),
ClientSecret: os.Getenv("DATABRICKS_CLIENT_SECRET"),
})
if err != nil {
panic(err)
}
ctx := context.Background()
table, err := w.Tables.Get(ctx, catalog.GetTableRequest{
FullName: "samples.nyctaxi.trips",
})
if err != nil {
panic(err)
}
fmt.Printf("OAuth M2M OK: %s (%d columns)\n", table.Name, len(table.Columns))
}
Env vars: DATABRICKS_HOST, DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET
Setup: Create service principal in Account Console → Settings → Service principals. Generate OAuth secret.
U2M Token-Env Authentication (Headless/CI)
package main
import (
"context"
"fmt"
"os"
"github.com/databricks/databricks-sdk-go"
"github.com/databricks/databricks-sdk-go/service/catalog"
"github.com/databricks/databricks-sdk-go/useragent"
)
func main() {
useragent.WithPartner("YourCompany")
useragent.WithProduct("YourCompany_YourProduct", "1.0.0")
token := os.Getenv("DATABRICKS_ACCESS_TOKEN")
if token == "" {
token = os.Getenv("DATABRICKS_TOKEN")
}
w, err := databricks.NewWorkspaceClient(&databricks.Config{
Host: os.Getenv("DATABRICKS_HOST"),
Token: token,
})
if err != nil {
panic(err)
}
ctx := context.Background()
table, err := w.Tables.Get(ctx, catalog.GetTableRequest{
FullName: "samples.nyctaxi.trips",
})
if err != nil {
panic(err)
}
fmt.Printf("U2M Token-Env OK: %s (%d columns)\n", table.Name, len(table.Columns))
}
Env vars: DATABRICKS_HOST, DATABRICKS_ACCESS_TOKEN (or DATABRICKS_TOKEN)
U2M Custom OAuth App (PKCE) - Interactive
For user-interactive flows with a custom OAuth app, implement the OAuth Authorization Code flow with PKCE:
package main
import (
"context"
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"net/http"
"net/url"
"os"
"os/exec"
"runtime"
"strings"
"github.com/databricks/databricks-sdk-go"
"github.com/databricks/databricks-sdk-go/service/catalog"
"github.com/databricks/databricks-sdk-go/useragent"
)
func main() {
useragent.WithPartner("YourCompany")
useragent.WithProduct("YourCompany_YourProduct", "1.0.0")
host := os.Getenv("DATABRICKS_HOST")
clientID := os.Getenv("DATABRICKS_U2M_CLIENT_ID")
clientSecret := os.Getenv("DATABRICKS_U2M_CLIENT_SECRET")
redirectURI := os.Getenv("DATABRICKS_REDIRECT_URI")
if redirectURI == "" {
redirectURI = "http://localhost:8040/callback"
}
token, err := runPKCEFlow(host, clientID, clientSecret, redirectURI)
if err != nil {
panic(err)
}
w, err := databricks.NewWorkspaceClient(&databricks.Config{
Host: host,
Token: token,
})
if err != nil {
panic(err)
}
ctx := context.Background()
table, err := w.Tables.Get(ctx, catalog.GetTableRequest{
FullName: "samples.nyctaxi.trips",
})
if err != nil {
panic(err)
}
fmt.Printf("U2M Custom OAuth OK: %s (%d columns)\n", table.Name, len(table.Columns))
}
func runPKCEFlow(host, clientID, clientSecret, redirectURI string) (string, error) {
verifierBytes := make([]byte, 32)
rand.Read(verifierBytes)
codeVerifier := base64.RawURLEncoding.EncodeToString(verifierBytes)
hash := sha256.Sum256([]byte(codeVerifier))
codeChallenge := base64.RawURLEncoding.EncodeToString(hash[:])
stateBytes := make([]byte, 32)
rand.Read(stateBytes)
state := base64.RawURLEncoding.EncodeToString(stateBytes)
redirectURL, _ := url.Parse(redirectURI)
port := redirectURL.Port()
if port == "" {
port = "8040"
}
hostNorm := strings.TrimPrefix(strings.TrimPrefix(host, "https://"), "http://")
authURL := fmt.Sprintf("https://%s/oidc/v1/authorize?"+
"client_id=%s&redirect_uri=%s&response_type=code&scope=all-apis&"+
"code_challenge=%s&code_challenge_method=S256&state=%s",
hostNorm, clientID, url.QueryEscape(redirectURI), codeChallenge, state)
codeChan := make(chan string, 1)
errChan := make(chan error, 1)
server := &http.Server{Addr: ":" + port}
http.HandleFunc("/callback", func(w http.ResponseWriter, r *http.Request) {
if r.URL.Query().Get("state") != state {
errChan <- fmt.Errorf("state mismatch")
return
}
code := r.URL.Query().Get("code")
if code == "" {
errChan <- fmt.Errorf("no code in callback")
return
}
w.Header().Set("Content-Type", "text/html")
w.Write([]byte(`<html><body style="font-family:sans-serif;text-align:center;padding:50px;">
<h1 style="color:green;">✓ Authentication Successful</h1>
<p>You can close this tab.</p></body></html>`))
codeChan <- code
})
go server.ListenAndServe()
defer server.Shutdown(context.Background())
openBrowser(authURL)
fmt.Println("Waiting for authentication in browser...")
var code string
select {
case code = <-codeChan:
case err := <-errChan:
return "", err
}
tokenURL := fmt.Sprintf("https://%s/oidc/v1/token", hostNorm)
data := url.Values{
"grant_type": {"authorization_code"},
"code": {code},
"redirect_uri": {redirectURI},
"code_verifier": {codeVerifier},
"client_id": {clientID},
}
if clientSecret != "" {
data.Set("client_secret", clientSecret)
}
resp, err := http.PostForm(tokenURL, data)
if err != nil {
return "", err
}
defer resp.Body.Close()
var result struct {
AccessToken string `json:"access_token"`
Error string `json:"error"`
}
json.NewDecoder(resp.Body).Decode(&result)
if result.Error != "" {
return "", fmt.Errorf("token error: %s", result.Error)
}
return result.AccessToken, nil
}
func openBrowser(url string) {
switch runtime.GOOS {
case "darwin":
exec.Command("open", url).Start()
case "linux":
exec.Command("xdg-open", url).Start()
case "windows":
exec.Command("rundll32", "url.dll,FileProtocolHandler", url).Start()
}
}
Env vars: DATABRICKS_HOST, DATABRICKS_U2M_CLIENT_ID, optional DATABRICKS_U2M_CLIENT_SECRET, DATABRICKS_REDIRECT_URI
Setup: Create OAuth app in Account Console → Settings → App connections. Add redirect URI http://localhost:8040/callback.
Environment Variables Reference
| Variable | Used By | Description |
|---|
DATABRICKS_HOST | All | Workspace URL |
DATABRICKS_TOKEN | PAT, U2M token-env | Personal access token or pre-obtained OAuth token |
DATABRICKS_CLIENT_ID | OAuth M2M | Service principal UUID |
DATABRICKS_CLIENT_SECRET | OAuth M2M | Service principal OAuth secret |
DATABRICKS_U2M_CLIENT_ID | U2M Custom OAuth | Custom OAuth app client ID |
DATABRICKS_U2M_CLIENT_SECRET | U2M Custom OAuth | Custom OAuth app secret (if confidential) |
DATABRICKS_REDIRECT_URI | U2M Custom OAuth | Redirect URI (default: http://localhost:8040/callback) |
APP_AUTH_TYPE | Multi-auth binary | App-level auth type selector (NOT DATABRICKS_AUTH_TYPE) |
Required Imports (Quick Reference)
import (
"context"
"fmt"
"os"
"github.com/databricks/databricks-sdk-go"
"github.com/databricks/databricks-sdk-go/useragent"
)
import "github.com/databricks/databricks-sdk-go/service/catalog"
import "github.com/databricks/databricks-sdk-go/service/jobs"
import "github.com/databricks/databricks-sdk-go/service/compute"
import (
"crypto/rand"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"net/http"
"net/url"
"os/exec"
"runtime"
"strings"
"time"
)
PKCE Flow with Timeout
Add a timeout to PKCE flows to prevent indefinite hangs:
select {
case code := <-codeChan:
case err := <-errChan:
return "", err
case <-time.After(2 * time.Minute):
return "", fmt.Errorf("authentication timed out after 2 minutes")
}
Multi-Auth Pattern
Support multiple auth types in a single binary using a custom selector (e.g., APP_AUTH_TYPE):
authType := os.Getenv("APP_AUTH_TYPE")
if authType == "" {
authType = "oauth_m2m"
}
var cfg *databricks.Config
switch authType {
case "pat":
cfg = &databricks.Config{
Host: os.Getenv("DATABRICKS_HOST"),
Token: os.Getenv("DATABRICKS_TOKEN"),
}
case "oauth_m2m":
cfg = &databricks.Config{
Host: os.Getenv("DATABRICKS_HOST"),
ClientID: os.Getenv("DATABRICKS_CLIENT_ID"),
ClientSecret: os.Getenv("DATABRICKS_CLIENT_SECRET"),
}
case "u2m_token_env":
token := os.Getenv("DATABRICKS_ACCESS_TOKEN")
if token == "" {
token = os.Getenv("DATABRICKS_TOKEN")
}
cfg = &databricks.Config{
Host: os.Getenv("DATABRICKS_HOST"),
Token: token,
}
case "u2m_custom_oauth_app":
token, err := runPKCEFlow(host, clientID, clientSecret, redirectURI)
if err != nil {
panic(err)
}
cfg = &databricks.Config{
Host: os.Getenv("DATABRICKS_HOST"),
Token: token,
}
}
w, err := databricks.NewWorkspaceClient(cfg)
APP_AUTH_TYPE | Auth flow | Env vars required |
|---|
pat | Personal Access Token | DATABRICKS_TOKEN |
oauth_m2m (default) | OAuth M2M (client credentials) | DATABRICKS_CLIENT_ID, DATABRICKS_CLIENT_SECRET |
u2m_token_env | U2M pre-obtained token | DATABRICKS_ACCESS_TOKEN or DATABRICKS_TOKEN |
u2m_custom_oauth_app | U2M custom OAuth app (PKCE) | DATABRICKS_U2M_CLIENT_ID, optional DATABRICKS_U2M_CLIENT_SECRET |
CRITICAL: Use APP_AUTH_TYPE (not DATABRICKS_AUTH_TYPE). The SDK reads DATABRICKS_AUTH_TYPE internally and expects its own values (pat, oauth-m2m, etc.).
U2M Browser Environment Requirements
For U2M PKCE flows that open a browser, pass these environment variables when running with env -i:
env -i PATH=$PATH HOME=$HOME USER=$USER DISPLAY=$DISPLAY \
DATABRICKS_HOST=$DATABRICKS_HOST \
DATABRICKS_U2M_CLIENT_ID=$DATABRICKS_U2M_CLIENT_ID \
APP_AUTH_TYPE=u2m_custom_oauth_app \
./your_binary
HOME - Required for browser profile access
USER - Required by some browsers
DISPLAY - Required on Linux for GUI browser
Validation Query
Verify User-Agent telemetry is being recorded correctly:
SELECT event_time, user_agent, action_name, request_params
FROM system.access.audit
WHERE event_time > current_timestamp() - INTERVAL 1 HOUR
AND lower(user_agent) LIKE '%yourcompany%'
ORDER BY event_time DESC
LIMIT 10;
Auth Isolation
Run tests with a clean environment using env -i to avoid "more than one authorization method configured":
env -i PATH=$PATH HOME=$HOME \
DATABRICKS_HOST=$DATABRICKS_HOST \
DATABRICKS_TOKEN=$DATABRICKS_TOKEN \
APP_AUTH_TYPE=pat \
./your_binary
env -i PATH=$PATH HOME=$HOME \
DATABRICKS_HOST=$DATABRICKS_HOST \
DATABRICKS_CLIENT_ID=$DATABRICKS_CLIENT_ID \
DATABRICKS_CLIENT_SECRET=$DATABRICKS_CLIENT_SECRET \
APP_AUTH_TYPE=oauth_m2m \
./your_binary
env -i PATH=$PATH HOME=$HOME USER=$USER DISPLAY=$DISPLAY \
DATABRICKS_HOST=$DATABRICKS_HOST \
DATABRICKS_U2M_CLIENT_ID=$DATABRICKS_U2M_CLIENT_ID \
APP_AUTH_TYPE=u2m_custom_oauth_app \
./your_binary
Error Handling Patterns
Basic Error Handling
import (
"errors"
"github.com/databricks/databricks-sdk-go/apierr"
)
func handleError(err error) {
if err == nil {
return
}
var apiErr *apierr.APIError
if errors.As(err, &apiErr) {
fmt.Printf("API Error: %s (code: %s)\n", apiErr.Message, apiErr.ErrorCode)
switch apiErr.StatusCode {
case 401:
fmt.Println("Authentication failed - check credentials")
case 403:
fmt.Println("Permission denied - check service principal permissions")
case 404:
fmt.Println("Resource not found")
case 429:
fmt.Println("Rate limited - implement backoff")
case 503:
fmt.Println("Service unavailable - retry with backoff")
}
} else {
fmt.Printf("Error: %v\n", err)
}
}
Retry Pattern with Exponential Backoff
import (
"context"
"errors"
"strings"
"time"
"github.com/databricks/databricks-sdk-go/apierr"
)
type RetryConfig struct {
MaxRetries int
InitialBackoff time.Duration
MaxBackoff time.Duration
Multiplier float64
}
func DefaultRetryConfig() RetryConfig {
return RetryConfig{
MaxRetries: 3,
InitialBackoff: 1 * time.Second,
MaxBackoff: 30 * time.Second,
Multiplier: 2.0,
}
}
func WithRetry[T any](ctx context.Context, cfg RetryConfig, op func() (T, error)) (T, error) {
var result T
var lastErr error
backoff := cfg.InitialBackoff
for attempt := 0; attempt <= cfg.MaxRetries; attempt++ {
result, lastErr = op()
if lastErr == nil {
return result, nil
}
if !isRetryable(lastErr) || attempt >= cfg.MaxRetries {
return result, lastErr
}
fmt.Printf("[Retry %d/%d] Waiting %v. Error: %v\n", attempt+1, cfg.MaxRetries, backoff, lastErr)
select {
case <-ctx.Done():
return result, ctx.Err()
case <-time.After(backoff):
backoff = time.Duration(float64(backoff) * cfg.Multiplier)
if backoff > cfg.MaxBackoff {
backoff = cfg.MaxBackoff
}
}
}
return result, lastErr
}
func isRetryable(err error) bool {
var apiErr *apierr.APIError
if errors.As(err, &apiErr) {
switch apiErr.StatusCode {
case 429, 500, 502, 503, 504:
return true
}
switch apiErr.ErrorCode {
case "TEMPORARILY_UNAVAILABLE", "RESOURCE_EXHAUSTED", "DEADLINE_EXCEEDED":
return true
}
}
errMsg := err.Error()
return strings.Contains(errMsg, "connection reset") ||
strings.Contains(errMsg, "read: connection timed out")
}
Troubleshooting
Common Errors
| Error | Cause | Solution |
|---|
more than one authorization method configured | Multiple auth env vars set | Use env -i to pass only vars for one auth type |
dial tcp: connection refused | Network/host issue | Verify DATABRICKS_HOST URL |
401 Unauthorized | Invalid credentials | Check token/secret |
403 Forbidden | Missing permissions | Grant access to SP or user |
context deadline exceeded | Request timeout | Increase timeout or check network connectivity |
TEMPORARILY_UNAVAILABLE | Service overloaded | Retry with exponential backoff |
OAuth-Specific Errors
| Error | Cause | Solution |
|---|
OAuth application not available | Account not enabled | Enable OAuth in account console |
redirect_uri mismatch | URI not registered | Add redirect URI in Settings → App connections → Redirect URIs |
invalid_client | Wrong client ID/secret | Verify credentials match registered OAuth app |
invalid_grant | Auth code expired/reused | Auth codes are single-use; restart OAuth flow |
U2M Browser Issues
| Symptom | Cause | Solution |
|---|
| Browser doesn't open | Missing env vars | Pass HOME, USER, DISPLAY via env -i |
localhost refused to connect | Stale browser tab | Normal if auth succeeded; server shuts down after token exchange |
| Port 8040 in use | Another process | Set DATABRICKS_REDIRECT_URI=http://localhost:8050/callback |
authentication timed out | User didn't complete login | Retry; ensure browser can reach workspace |
Environment Variable Conflicts
CRITICAL: Do NOT use DATABRICKS_AUTH_TYPE as your app-level auth selector. The SDK reads this internally.
Use APP_AUTH_TYPE instead and always run tests with env -i.
Host Normalization
The SDK handles host normalization internally, but for custom PKCE flows, strip the protocol:
hostNorm := strings.TrimPrefix(strings.TrimPrefix(host, "https://"), "http://")
authURL := fmt.Sprintf("https://%s/oidc/v1/authorize?...", hostNorm)
tokenURL := fmt.Sprintf("https://%s/oidc/v1/token", hostNorm)
Key Differences from Databricks SQL Driver for Go
| Aspect | Databricks SDK for Go | Databricks SQL Driver for Go |
|---|
| Purpose | REST APIs (UC, Jobs, Clusters) | SQL queries via warehouse |
| SQL Warehouse | Not needed | Required (DATABRICKS_HTTP_PATH) |
| Auth Config | databricks.Config{} struct | dbsql.NewConnector() options |
| User-Agent | useragent.WithPartner() (global) | dbsql.WithUserAgentEntry() (per-connector) |
| Auth Type Selector | Use APP_AUTH_TYPE (SDK reads DATABRICKS_AUTH_TYPE internally) | Can use DATABRICKS_AUTH_TYPE safely |
| Token Refresh | SDK handles OAuth M2M refresh automatically | Must handle refresh manually for M2M |