lakeflow-community-connectors

Generate public-facing documentation for a connector targeted at end users.

Set up authentication for a source connector — generate connector spec, collect credentials interactively, and validate auth.

Run the authenticate script to collect credentials from the user via a browser form.

Guide the user through creating or updating a pipeline for a source connector — read the docs, build a pipeline spec interactively, and run create_pipeline or update_pipeline.

Generate the connector spec YAML file defining connection parameters and external options allowlist.

Single step only: audit a completed connector — implementation, testing & simulator validation, artifacts, security smells, cross-doc consistency — and produce a scored markdown review report. Read-mostly; does not modify connector code.

Generate and run an auth verification test to confirm that collected credentials are valid.

Single step only: implement the connector in Python when the API doc already exists. Do NOT use for full connector creation — use the create-connector agent instead.

Create a pyproject.toml for a source connector and build it as an independent Python package.

Single step only: run the per-source pytest suite, diagnose failures, and fix the connector or simulator until everything passes. Branches on mode={simulate|record}. Do NOT use for full connector creation — use the create-connector agent instead.

Implement test utilities that write test data to the source system and validate end-to-end read cycles.

Validate that a connector's CDC/incremental sync implementation correctly tracks offsets and filters records.

Single step only: implement a partitioned connector that extends SupportsPartitionedStream (or SupportsPartition) alongside LakeflowConnect. Do NOT use for full connector creation — use the create-connector agent instead.

Research and document write/create APIs of a source system to enable write-back testing functionality.

Single step only: research and document a source API when explicitly asked. Do NOT use for full connector creation — use the create-connector agent instead.

Migrate a legacy source connector from the old sources/ directory to the new package structure under src/databricks/labs/community_connector/sources/.