القائمة
Build PWAF-compliant ISV integrations with Databricks: OAuth, telemetry (User-Agent), Unity Catalog, JDBC, SDK, SQL drivers, REST API, Databricks Connect.
التثبيت باستخدام Codex أو Claude انسخ هذا Prompt والصقه في Codex أو Claude أو مساعد آخر ليراجع صفحة Skill ويثبّتها لك.
PWAF-compliant Python SQL Connector (databricks-sql-connector): PAT, OAuth M2M, OAuth U2M (custom OAuth app PKCE + token-env), credentials_provider patterns, error handling, retry logic. Use when building Python integrations that run SQL queries via a Databricks SQL warehouse.
Add a Databricks connector to an existing project that has no Databricks integration. Use when your product already exists and you want to add Databricks as a new data source or backend.
How to write a build_report.md for any PWAF connector. Captures skill traceability, sufficiency assessment, and test error/fix log. Use after implementing any connector.
How to structure a Databricks connector (REST or Python SDK): config, connect, operations, validation. Use when designing or building a new connector.
How to build and use a PWAF connector test runner (tests/run_all_tests.sh). Covers env isolation, auth types, single connector or single auth mode, parallel execution, browser tests, and report generation.
PWAF-compliant Databricks Connect (Python): PAT, OAuth M2M, OAuth U2M; serverless and classic compute. Use when building or testing Spark-over-Connect integrations.
استنادا إلى تصنيف SOC المهني
| name | databricks-isv-integration |
| description | Build PWAF-compliant ISV integrations with Databricks: OAuth, telemetry (User-Agent), Unity Catalog, JDBC, SDK, SQL drivers, REST API, Databricks Connect. |
This skill provides patterns and code examples for building PWAF-compliant ISV integrations with Databricks, following the Partner Well-Architected Framework (PWAF).
Cursor rule vs skills: Use the Cursor rule (.cursor/rules/databricks-isv-integration.mdc) for file-scoped context when editing integration code (it applies when you work in matching paths). Use the skills in skills/*/SKILL.md for task-based guidance (e.g. "add a Databricks connector," "fix U2M," "run auth tests")—the agent loads the relevant skill by description. Auth patterns live in skills/<connector>/authentication.md, co-located with each skill.
PWAF covers:
All PWAF-validated partner integrations must meet these requirements:
| Requirement | Summary |
|---|---|
| Telemetry | User-Agent tagging on all API/driver calls |
| OAuth | Support Token Federation, U2M with PKCE, and M2M |
| Unity Catalog | Use UC namespaces, respect ACLs, use Volumes for staging |
Applications: Use OAuth M2M for service/backend flows, or U2M for interactive user sign-in. U2M has three flows: (1) SDK external-browser with built-in app (simplest; no custom app), (2) SDK external-browser with custom OAuth app (for ISV integrations; configurable redirect URI), (3) token pass-through (pre-obtained token; headless/CI). All use Auth_Flow=0 on the JDBC side. For custom apps, redirect URI is configurable; default http://localhost:8080/callback. For the built-in app, redirect URI is http://localhost:8020.
import java.sql.*;
import java.util.Properties;
public class DatabricksConnector {
public static Connection connect(
String host,
String httpPath,
String clientId,
String clientSecret,
String userAgent
) throws SQLException {
String url = "jdbc:databricks://" + host + ":443";
Properties props = new Properties();
props.put("httpPath", httpPath);
props.put("SSL", "1");
// OAuth M2M Authentication
props.put("AuthMech", "11");
props.put("Auth_Flow", "1");
props.put("OAuth2ClientId", clientId);
props.put("OAuth2Secret", clientSecret);
// Partner Telemetry (REQUIRED)
props.put("UserAgentEntry", userAgent);
return DriverManager.getConnection(url, props);
}
}
When building a connector UI, show these fields based on auth selection:
| Auth Type | AuthMech | Auth_Flow | Required Fields | Token Source |
|---|---|---|---|---|
| PAT | 3 | - | token | User provides PAT |
| OAuth M2M | 11 | 1 | clientId, clientSecret | Driver does client-credentials |
| U2M Built-in Browser | 11 | 0 | (none beyond host/path) | SDK external-browser (built-in databricks-cli app) |
| U2M Custom App Browser | 11 | 0 | u2mClientId; optional u2mClientSecret, redirectUri | SDK external-browser (custom OAuth app) |
| U2M Token Pass-through | 11 | 0 | accessToken | Pre-obtained (CLI, hosted callback, refresh) |
U2M: All three U2M flows use Auth_Flow=0 (token pass-through) on the JDBC side. The token is obtained externally via the Databricks SDK or a prior OAuth exchange. For custom OAuth apps, use DATABRICKS_U2M_CLIENT_ID (not the M2M DATABRICKS_CLIENT_ID). Redirect URI: configurable for custom apps; http://localhost:8020 for the built-in app (must set explicitly in Java SDK v0.54.0).
See skills/java-jdbc/authentication.md for complete code examples including Java SDK workarounds.
User-Agent Format (required per PWAF for all partner integrations):
<isv-name>_<product-name>/<product-version>
Example: AcmePartner_DataConnector/2.1.0
See skills/telemetry-attribution/SKILL.md for driver-specific configuration.
Quick index: INDEX.md – one line per rule and skill.
skills/<connector>/authentication.md)| File | Language/Driver | Coverage |
|---|---|---|
| skills/java-jdbc/authentication.md | Java JDBC (OSS v3+) | PAT, OAuth M2M, OAuth U2M (browser + token-env), UserAgentEntry, Java 17+ add-opens |
| skills/java-sdk/authentication.md | Java SDK (databricks-sdk-java) | PAT, OAuth M2M; UserAgent.withProduct/withPartner; no warehouse needed |
| skills/python-sdk/authentication.md | Python SDK (databricks-sdk) | PAT, OAuth M2M/U2M, Azure MSI |
| skills/python-sql-connector/authentication.md | Python SQL Connector | PAT, OAuth M2M/U2M |
| skills/python-sqlalchemy/authentication.md | Python SQLAlchemy (databricks-sqlalchemy) | PAT, OAuth M2M/U2M; URL + connect_args user_agent_entry; M2M token from headers dict |
| skills/databricks-connect/authentication.md | Databricks Connect | PAT, OAuth M2M, OAuth U2M (external-browser, localhost, token-env), Serverless, Classic, Azure MSI |
| skills/nodejs-sql-driver/authentication.md | Node.js SQL Driver (@databricks/sql) | PAT, OAuth M2M (driver-native), OAuth U2M (browser), Token pass-through; host normalization; session patterns |
| skills/go-sdk/authentication.md | Databricks SDK for Go (databricks-sdk-go) | PAT, OAuth M2M, OAuth U2M (browser + token-env + custom OAuth app PKCE), Azure MSI; useragent.WithProduct/WithPartner |
| skills/go-sql-driver/authentication.md | Databricks SQL Driver for Go (databricks-sql-go) | PAT, OAuth M2M, OAuth U2M; WithUserAgentEntry; all_auth pattern |
| skills/rest-api/authentication.md | Any language (HTTP) | PAT, OAuth M2M, Token |
| skills/python-dbconnect/authentication.md | Python Databricks Connect | PAT, M2M, U2M; compute resolution; PKCE helper |
| skills/odbc/authentication.md | BI Tools (ODBC) | PAT, OAuth M2M/U2M, Token |
| Path | Purpose |
|---|---|
| .cursor/rules/databricks-isv-integration.mdc | Cursor rule: auth isolation, U2M gotchas, Python SDK/SQL patterns, connector structure, testing |
| skills/adding-databricks-connector/SKILL.md | Add a Databricks connector to an existing project (no Databricks yet): stack choice, where to integrate, minimal steps |
| skills/connector-structure/SKILL.md | How to structure a connector: config, connect(), operations, validation |
| skills/rest-api/SKILL.md | REST API auth (PAT, M2M, U2M) and validation tests |
| skills/python-sdk/SKILL.md | Python SDK (databricks-sdk): Config, auth_type, WorkspaceClient, telemetry |
| skills/python-sql-connector/SKILL.md | Python SQL Connector: PAT, M2M, U2M; credentials_provider; host normalization |
| skills/python-sqlalchemy/SKILL.md | SQLAlchemy + Databricks: dialect, URL, PAT/M2M/U2M, user_agent_entry; M2M token from authenticate() headers |
| skills/databricks-connect/SKILL.md | Databricks Connect: PAT, OAuth M2M, U2M; serverless and classic compute; version compatibility |
| skills/java-jdbc/SKILL.md | Java JDBC (OSS driver): PAT, OAuth M2M, U2M browser/token-env; UserAgentEntry; Java 17+ add-opens |
| skills/java-sdk/SKILL.md | Java SDK (databricks-sdk-java): PAT, OAuth M2M; UserAgent.withProduct/withPartner; UC Tables API; no warehouse |
| skills/go-sdk/SKILL.md | Databricks SDK for Go (databricks-sdk-go): PAT, OAuth M2M, U2M token-env, U2M custom OAuth app (PKCE); useragent.WithProduct/WithPartner; UC Tables API; no warehouse needed |
| skills/go-sql-driver/SKILL.md | Databricks SQL Driver for Go (databricks-sql-go): PAT, OAuth M2M, U2M browser, U2M token-env, U2M custom OAuth app (PKCE); WithUserAgentEntry; DESCRIBE TABLE; SQL warehouse required |
| skills/nodejs-sql-driver/SKILL.md | Node.js SQL Driver (@databricks/sql): PAT, OAuth M2M (driver-native), OAuth U2M (browser); host normalization; session patterns |
| skills/u2m/SKILL.md | U2M flows: external-browser, custom-oauth-app, token-env; M2M client_id ≠ U2M app; separate example scripts per flow |
| skills/testing/SKILL.md | Running auth tests with clean env per test |
Java SDK (workspace APIs, UC, Jobs — no warehouse needed):
<dependency>
<groupId>com.databricks</groupId>
<artifactId>databricks-sdk-java</artifactId>
<version>0.54.0</version>
</dependency>
Java 11+ required. See skills/java-sdk/SKILL.md.
OSS JDBC driver (SQL queries via warehouse):
<dependency>
<groupId>com.databricks</groupId>
<artifactId>databricks-jdbc</artifactId>
<version>3.1.1</version>
<scope>runtime</scope>
</dependency>
On Java 17+, add --add-opens=java.base/java.nio=ALL-UNNAMED (e.g. MAVEN_OPTS or exec plugin). See skills/java-jdbc/SKILL.md.
Databricks SDK for Go (workspace APIs, UC, Jobs — no warehouse needed):
go get github.com/databricks/databricks-sdk-go@v0.107.0
Go 1.21+ required. Uses useragent.WithPartner() / useragent.WithProduct() for telemetry. See skills/go-sdk/SKILL.md.
Databricks SQL Driver for Go (SQL queries via warehouse):
go get github.com/databricks/databricks-sql-go
Go 1.20+ required. Requires DATABRICKS_HTTP_PATH (SQL warehouse). Uses dbsql.WithUserAgentEntry() for telemetry. See skills/go-sql-driver/SKILL.md.
| Issue | Solution |
|---|---|
| More than one authorization method configured | Run with a clean env: only set vars for one auth type (PAT or M2M or U2M). Do not set both DATABRICKS_TOKEN and DATABRICKS_CLIENT_ID/DATABRICKS_CLIENT_SECRET in the same process. |
| OAuth application with client_id not available (U2M) | M2M service principal client_id is not valid for the browser flow. For external-browser, do not pass client_id (SDK uses built-in app). For custom-oauth-app, use a separate OAuth custom app with redirect URI and DATABRICKS_U2M_CLIENT_ID. |
| Cannot configure default credentials (Python M2M) | Pass auth_type="oauth-m2m" to Config(...) when using client_id/client_secret so the SDK does not try default credential resolution. |
| Auth failed with M2M | Verify Service Principal has access to the SQL Warehouse |
| User-Agent not appearing | Check UserAgentEntry is set in connection properties |
| Token expired | OAuth tokens expire in 1 hour; implement refresh logic |
| Unity Catalog access denied | Check catalog/schema grants for the identity |
| Redirect URL for U2M | Custom apps: configurable, default http://localhost:8080/callback. Built-in app: http://localhost:8020 (set explicitly in Java SDK). Register in App connections. Not used for PAT or M2M. |
| Java SDK NullPointerException (U2M) | Call config.setScopes(Arrays.asList("all-apis")) and config.resolve() before config.authenticate(). Set config.setOAuthRedirectUrl("http://localhost:8020") for built-in app. See skills/u2m/SKILL.md. |
| JDBC Java 17+: MemoryUtil / InaccessibleObjectException | OSS driver 3.x uses Arrow; add --add-opens=java.base/java.nio=ALL-UNNAMED to JVM (MAVEN_OPTS or exec plugin). |