| name | migrate-legacy-implementation |
| description | Migrate a legacy source connector from the old sources/ directory to the new package structure under src/databricks/labs/community_connector/sources/. |
| disable-model-invocation | true |
Migrate Legacy Source Implementation
Description
This skill migrates an old connector implementation from the legacy sources/ directory to the new package structure under src/databricks/labs/community_connector/sources/. It handles one specified source at a time, updating imports, conforming to the LakeflowConnect interface, and regenerating specs and build files.
Instructions
Handle one specified source at a time.
Refer other source (e.g. example) as a reference.
Use python3.10 to replace all python command.
When running any python test or build use virtual environment (Python 3.10+ required), run these:
python3.10 -m venv .venv
source .venv/bin/activate
python -m pip install -e ".[dev]"
then run other python commands or pytest.
Steps
-
Run shell command to git mv the files for the specified source under sources/ to the new directory. Commit the change.
- Files under
sources/{source_name}/tests/ need to move to tests/unit/sources/{source_name}
sources/{source_name}/configs needs to move to tests/unit/sources/{source_name} as well
-
Update {source_name}.py:
- Import the
LakeflowConnect interface from src/databricks/labs/community_connector/interface/lakeflow_connect.py and change the existing LakeflowConnect class to implement the interface with name {SourceName}LakeflowConnect.
-
Update import paths in all moved files and expose {SourceName}LakeflowConnect in __init__.py under {source_name} to make it available in the package.
- libs are moved from root directory to src/databricks/labs/community_connector/libs, so you can use the same library functions.
- Also update the test imports and code if necessary.
- Commit the changes so far.
-
If {source_name}.py is too large (>1000 lines), consider refactoring it into multiple files. One example to follow is example.
-
Run the tests — they will fail but should not be syntax or import errors.
- Add the unit test that requires credentials (the one built on top of the source test_suite) to .github/workflows/test_exclude.txt
- Commit the changes so far.
-
Update the README.md under {source_name} directory to make sure all references and content match the latest changes.
-
Use the generate-connector-spec skill to regenerate the spec for this connector.
-
Use the build_connector_package skill to build the pyproject.toml and related files.
- Commit the changes so far.
-
If there are any .py files that should not be imported as part of {source_name}.py, exclude them in the tools/scripts/merge_exclude_config.json
-
You should change any other directories other than src/databricks/labs/community_connector/sources/{source_name} directory. Delete other directories like libs, pipeline under root.
-
run tools/scripts/merge_python_source.py to create _generated source code.
