connector-doc-create

name	connector-doc-create
description	Create full documentation for a new OpenMetadata connector from scratch — generates main page, yaml.mdx, troubleshooting page, and registers in docs.json navigation. Derives features, permissions, and YAML config from JSON schema and source code.
user-invocable	true
argument-hint	<connector-name> [--display-name='Display Name'] [--service-type=database|pipeline|dashboard|messaging|storage|search|ml-model] [--stage=PROD|BETA] [--version=v1.13.x|v2.0.x|v2.1.x-SNAPSHOT|all] [--icon=/path/to/icon.svg] [--dry-run]
allowed-tools	["Bash","Read","Glob","Grep","Edit","Write","Agent"]

Connector Documentation Creation Skill

When to Activate

When a user asks to create, generate, scaffold, or add documentation for a new connector — one that exists in source code but doesn't yet have docs pages.

Arguments

• connector-name (required): Slug name of the connector (e.g., redshift, dynamodb, bigquery, airflow, looker, kafka). Must match the directory name in ${SOURCE_ROOT}/${service_type}/.
• --display-name (optional): Human-readable name shown in docs (e.g., "Amazon Redshift", "Google BigQuery"). Defaults to title-cased connector-name.
• --service-type (optional): One of database, pipeline, dashboard, messaging, storage, search, ml-model. Note: use ml-model (with hyphen) — this matches the actual directory name. Other types (api, drive, metadata) exist in the repo but are rare; treat them like pipeline if encountered. Default: auto-detect from connector name and schema.
• --stage (optional): PROD or BETA. Default: BETA for new connectors.
• --version (optional): Which version(s) to create docs for. Default: all (creates in v1.13.x, v2.0.x).
• --icon (optional): Path to icon file under /public/images/connectors/. If not given, uses /public/images/connectors/{connector-name}.svg as a placeholder.
• --dry-run (optional): Only show the plan and generated content — don't write any files.

Directory References

DOCS_ROOT     = .                          # The docs-om repo (current working directory)
OM_ROOT       = ../OpenMetadata            # Sibling directory to docs-om
SCHEMA_ROOT   = ${OM_ROOT}/openmetadata-spec/src/main/resources/json/schema/entity/services
SOURCE_ROOT   = ${OM_ROOT}/ingestion/src/metadata/ingestion/source

---

Creation Process

Phase 1: Read Ground Truth from Source Code

Step 1.1: Read the JSON Schema

Schema file: ${SCHEMA_ROOT}/connections/${service_type}/${connectorName}Connection.json

Extract:

1. All properties — field names, types, descriptions, required fields, defaults
2. supports* boolean flags — determines available/unavailable features
3. Filter pattern fields — schemaFilterPattern, tableFilterPattern, storedProcedureFilterPattern, etc.
4. sampleDataStorageConfig — presence indicates Sample Data support
5. Authentication types — authType property's oneOf/anyOf references:
   • basicAuth.json → Basic Auth
   • iamAuthConfig.json → IAM Auth
   • awsCredentials.json → AWS Credentials
   • gcpCredentials.json → GCP Credentials
   • azureCredentials.json → Azure Credentials
6. SSL fields — sslMode, sslConfig, verifySSL
7. Required fields — the required array

Also check the parent service schema to confirm registration:

${SCHEMA_ROOT}/${serviceType}Service.json

Step 1.2: Read the Source Code

Read these files:

${SOURCE_ROOT}/${service_type}/${connector_name}/metadata.py    — Source class, capabilities
${SOURCE_ROOT}/${service_type}/${connector_name}/connection.py  — Connection logic, test steps, permissions
${SOURCE_ROOT}/${service_type}/${connector_name}/service_spec.py — Registered source classes

Also check if these exist:

${SOURCE_ROOT}/${service_type}/${connector_name}/queries.py     — SQL queries (reveals permission requirements)
${SOURCE_ROOT}/${service_type}/${connector_name}/client.py      — REST API client

Extract:

1. Test connection steps — test_fn dict keys in test_connection() → permission names
2. Service spec classes — lineage_source_class, usage_source_class, profiler_class (confirm feature support)
3. Mixin/base classes — what the source extends (e.g., LifeCycleQueryMixin, MultiDBSource)
4. Owner/Tag extraction — search for yield_tag, yield_table_tag, get_tag_labels, owner-related methods
5. Permission hints — SQL grants, IAM actions, API scopes in docstrings or comments
6. Python package name — pip install "openmetadata-ingestion[{package}]" (check pyproject.toml or setup.py if unsure)

Step 1.3: Build the Feature Truth Table

Using schema + code, build the definitive feature map:

For Database Connectors:

Signal	Feature
supportsMetadataExtraction: true (default)	"Metadata" → available
supportsUsageExtraction: true	"Query Usage" → available
supportsLineageExtraction: true	"Lineage" or "View Lineage" → available
supportsViewLineageExtraction: true	"Column-level Lineage" → available
supportsProfiler: true	"Data Profiler" → available
supportsProfiler: true (implicit)	"Auto-Classification" → available
supportsDBTExtraction: true	"dbt" → available
supportsDataDiff: true	"Data Quality" → available
storedProcedureFilterPattern present	"Stored Procedures" → available
sampleDataStorageConfig present	"Sample Data" → available
Owner extraction code found	"Owners" → available
Tag extraction code found	"Tags" → available

For Pipeline Connectors:

Signal	Feature
Always	"Pipelines" → available
Status extraction code found	"Pipeline Status" → available
lineage_source_class in service_spec	"Lineage" → available
Owner extraction code found	"Owners" → available
Tag extraction code found	"Tags" → available

For Dashboard Connectors:

Signal	Feature
Always	"Dashboards", "Charts" → available
Datamodel extraction code found	"Datamodels" → available
Project extraction code found	"Projects" → available
lineage_source_class in service_spec	"Lineage" → available
Column lineage code found	"Column Lineage" → available
Owner extraction code found	"Owners" → available
usage_source_class in service_spec	"Usage" → available
Tag extraction code found	"Tags" → available

For Messaging Connectors:

Signal	Feature
Always	"Topics" → available
sampleDataStorageConfig present	"Sample Data" → available

For Storage Connectors:

Signal	Feature
Always	"Metadata" → available
Structured container code found	"Structured Containers" → available
Unstructured container code found	"Unstructured Containers" → available

For Search Connectors:

Signal	Feature
Always	"Search Indexes" → available
Sample data code found	"Sample Data" → available

For ML Model Connectors:

Signal	Feature
Always	"ML Features" → available
Hyperparameter code found	"Hyperparameters" → available
ML store code found	"ML Store" → available

Step 1.4: Build the Permissions List

From the extracted code, derive the permission requirements:

• Database connectors: Map SQL queries to required GRANT statements. Common pattern:
  • Reading system views/tables → SELECT on those views
  • information_schema reads → USAGE on schema
  • Query history access → specific grants (e.g., pg_read_all_stats)
• AWS connectors: Map API calls to IAM actions (e.g., dynamodb:ListTables, s3:GetObject)
• GCP connectors: Map API calls to GCP roles/permissions
• REST API connectors: Map endpoints to required API scopes/roles

Group permissions by capability:

1. Metadata Ingestion
2. Profiler & Data Quality (if supported)
3. Usage & Lineage (if supported)

Step 1.5: Extract YAML Configuration Fields

From the JSON schema properties, build a YAML config template:

1. Start with required fields — these MUST appear, no comments
2. Add important optional fields with commented-out examples
3. Use placeholder values: <username>, <password>, <hostname>, <database>
4. For auth type fields (oneOf/anyOf): show the most common auth type uncommented, others as commented blocks
5. For filter patterns: show commented-out example with includes/excludes
6. For SSL: show commented-out block

Build ContentSection entries for each field shown in YAML:

• Match the YAML key name exactly
• Use the schema description as the base, expand if needed
• For complex fields (auth, SSL), add links to relevant docs pages

---

Phase 2: Check for Existing Files

Before creating anything, check:

1. Does ${DOCS_ROOT}/{version}/connectors/${service_type}/${connector_name}.mdx already exist?
2. Does ${DOCS_ROOT}/{version}/connectors/${service_type}/${connector_name}/yaml.mdx already exist?
3. Is the connector already registered in docs.json?

If files exist, stop and warn the user — use the connector-doc-review skill instead to update existing docs.

---

Phase 3: Generate Documentation Files

For each target version (v1.13.x, v2.0.x):

Step 3.1: Generate the Main Page ({connector_name}.mdx)

File path: ${DOCS_ROOT}/{version}/connectors/${service_type}/${connector_name}.mdx

Template — varies by service type. Use the correct imports and structure for the connector's service type:

Shared imports (all service types):

import { ConnectorDetailsHeader } from '/snippets/components/ConnectorDetailsHeader/ConnectorDetailsHeader.jsx'
import TestConnection from '/snippets/connectors/test-connection.mdx'
import IngestionScheduleAndDeploy from '/snippets/connectors/ingestion-schedule-and-deploy.mdx'
import { MetadataIngestionUi } from '/snippets/components/MetadataIngestionUi.jsx'

Service-type-specific imports for main page:

Service Type	ConfigureIngestion	Extra imports
database	/snippets/connectors/database/configure-ingestion.mdx	AdvancedConfiguration from /snippets/connectors/database/advanced-configuration.mdx; Related from /snippets/{version}/connectors/database/related.mdx
pipeline	/snippets/connectors/pipeline/configure-ingestion.mdx	—
dashboard	/snippets/connectors/dashboard/configure-ingestion.mdx	—
messaging	/snippets/connectors/messaging/configure-ingestion.mdx	—
storage	/snippets/connectors/storage/configure-ingestion.mdx	Manifest from /snippets/connectors/storage/manifest.mdx; do NOT use MetadataIngestionUi
search	/snippets/connectors/search/configure-ingestion.mdx	—
ml-model	/snippets/connectors/ml-model/configure-ingestion.mdx	—

Steps block — varies by service type:

• Database only: include <AdvancedConfiguration /> between Connection Details Step and <TestConnection />
• All types: include <TestConnection />, <ConfigureIngestion />, <IngestionScheduleAndDeploy />
• Database only: include <Related /> at bottom of page
• Storage only: include <Manifest /> in requirements section (OpenMetadata manifest file is required)

MetadataIngestionUi: Used by database, pipeline, dashboard, search, ml-model. NOT used by storage (storage uses manual step descriptions instead).

Full template:

---
title: "{DisplayName} Connector | OpenMetadata {ServiceType} Integration"
description: "Connect {DisplayName} to OpenMetadata with our comprehensive {service_type} connector guide. Step-by-step setup, configuration examples, and metadata extraction tips."
sidebarTitle: Overview
---

{service-type-specific imports — see table above}

<ConnectorDetailsHeader
icon='/public/images/connectors/{connector_name}.svg'
name="{DisplayName}"
stage="{PROD|BETA}"
availableFeatures={[{availableFeatures}]}
unavailableFeatures={[{unavailableFeatures}]} />

In this section, we provide guides and references to use the {DisplayName} connector.

{IF multiple auth types: include <Info> callout listing them}

Configure and schedule {DisplayName} metadata workflows from the OpenMetadata UI:
- [Requirements](#requirements)
- [Metadata Ingestion](#metadata-ingestion)
{IF database AND Query Usage supported: - [Query Usage](/{version}/connectors/ingestion/workflows/usage)}
{IF database AND Profiler supported: - [Data Profiler](/{version}/how-to-guides/data-quality-observability/profiler/profiler-workflow)}
{IF database AND Data Quality supported: - [Data Quality](/{version}/how-to-guides/data-quality-observability/quality)}