// PWAF-compliant Databricks Connect (Python): PAT, OAuth M2M, OAuth U2M; serverless and classic compute. Use when building or testing Spark-over-Connect integrations.
Build PWAF-compliant ISV integrations with Databricks: OAuth, telemetry (User-Agent), Unity Catalog, JDBC, SDK, SQL drivers, REST API, Databricks Connect.
PWAF-compliant Python SQL Connector (databricks-sql-connector): PAT, OAuth M2M, OAuth U2M (custom OAuth app PKCE + token-env), credentials_provider patterns, error handling, retry logic. Use when building Python integrations that run SQL queries via a Databricks SQL warehouse.
Add a Databricks connector to an existing project that has no Databricks integration. Use when your product already exists and you want to add Databricks as a new data source or backend.
How to write a build_report.md for any PWAF connector. Captures skill traceability, sufficiency assessment, and test error/fix log. Use after implementing any connector.
How to structure a Databricks connector (REST or Python SDK): config, connect, operations, validation. Use when designing or building a new connector.
How to build and use a PWAF connector test runner (tests/run_all_tests.sh). Covers env isolation, auth types, single connector or single auth mode, parallel execution, browser tests, and report generation.
| name | databricks-isv-databricks-connect |
| description | PWAF-compliant Databricks Connect (Python): PAT, OAuth M2M, OAuth U2M; serverless and classic compute. Use when building or testing Spark-over-Connect integrations. |
Use this skill when implementing or testing Databricks Connect (Python) for PWAF-compliant partner integrations: remote Spark execution from external apps using databricks-connect and DatabricksSession.
Serverless PAT – Config and session with .userAgent():
from databricks.connect import DatabricksSession
from databricks.sdk.config import Config
config = Config(
host=host_url,
token=token,
serverless_compute_id="auto",
)
session = (
DatabricksSession.builder
.sdkConfig(config)
.userAgent("YourCompany_YourProduct/1.0.0")
.getOrCreate()
)
OAuth M2M – always set auth_type:
config = Config(
host=host_url,
client_id=client_id,
client_secret=client_secret,
auth_type="oauth-m2m", # required
serverless_compute_id="auto",
)
session = (
DatabricksSession.builder
.sdkConfig(config)
.userAgent("YourCompany_YourProduct/1.0.0")
.getOrCreate()
)
U2M – get token first (e.g. external-browser), then Config with token: Same as PAT but token=access_token from browser flow. For external-browser, unset DATABRICKS_CLIENT_ID/DATABRICKS_CLIENT_SECRET before calling SDK.
.userAgent("<isv>_<product>/<version>") on DatabricksSession.builder (PWAF-recommended). Do not expose as user config. See PWAF Databricks Connect telemetry.auth_type="oauth-m2m" to Config(...) when using client_id/client_secret so the SDK does not run default credential resolution.DATABRICKS_SERVERLESS_COMPUTE_ID=auto or Config(..., serverless_compute_id="auto"). No cluster to manage. Some Connect versions do not yet support serverless; in that case use classic.DATABRICKS_CLUSTER_ID or CLASSIC_COMPUTE_HTTP_PATH (cluster ID = last path segment, e.g. sql/protocolv1/o/<workspace_id>/<cluster_id>).Obtain an access token first, then pass it to Config(host=..., token=access_token, ...) for DatabricksSession. Same three flows as REST/SQL U2M:
| Flow | Use when | Required |
|---|---|---|
| external-browser | No custom OAuth app; SDK opens browser | DATABRICKS_HOST; unset DATABRICKS_CLIENT_ID/DATABRICKS_CLIENT_SECRET/DATABRICKS_TOKEN so SDK uses built-in app |
| localhost | Custom OAuth app with localhost redirect | DATABRICKS_HOST, DATABRICKS_CLIENT_ID (OAuth app, not M2M); optional redirect_uri, client_secret |
| token-env | Pre-obtained token (no browser) | DATABRICKS_HOST, DATABRICKS_ACCESS_TOKEN or DATABRICKS_TOKEN |
Important: M2M service principal client_id is not valid for browser U2M. For external-browser, do not pass client_id. For localhost, use a separate OAuth custom app.
databricks-connect version must match the Databricks Runtime (DBR) on compute (e.g. DBR 18 → databricks-connect==18.0.*).Run each test or script with a clean environment: only the vars for the chosen auth type and compute. Use env -i plus explicit vars when running tests so PAT, M2M, and U2M do not mix (avoids "more than one authorization method configured").
DatabricksSession with the chosen auth and compute; run a simple read (e.g. spark.table("samples.nyctaxi.trips").count()) to verify connection and permissions.