source-connectors

# Airweave Source Connectors Use this sub-skill when a future Airweave task touches connector-facing source classes, source metadata exposed by the registry, auth/config schema behavior, connector validation, browse-tree node selection, access-control extraction, federated search sources, cursor-based incremental sync, or source-specific rate limiting. Do not use this sub-skill for dashboard UI, Connect widget iframe/OAuth UX, MCP transport, Monke orchestration, or generic backend endpoint lifecycle work. Cross-link to sibling [backend-api](../backend-api/SKILL.md) for source-connection endpoint flows, OAuth verification, search request/response schemas, browse-tree route shapes, and API auth/org semantics. ## Route to the right reference - Read [references/source-registry.md](references/source-registry.md) before adding a source class, changing `@source(...)` metadata, changing registry output, hiding/showing feature-flagged sources, or checking entity-definition exposure. - Read [references/auth-and-config.md](references/auth-and-config.md) before changing auth methods, direct credential schemas, OAuth/BYOC/token-injection handling, auth-provider support, template config fields, source connection validation, or runtime token-provider creation. - Read [references/browse-tree.md](references/browse-tree.md) before changing `supports_browse_tree`, lazy browse children, browse-node ID encoding, node-selection persistence, or targeted sync behavior. - Read [references/troubleshooting.md](references/troubleshooting.md) when diagnosing source visibility, auth/config errors, browse-tree selection bugs, ACL over/under-grants, federated search failures, cursor regressions, source HTTP errors, concurrency issues, or rate-limit behavior. ## Connector operating rules 1. Keep connector identity in one place: the `@source(...)` decorator sets `short_name`, display name, auth methods, config classes, capability flags, labels, feature flag, cursor class, federated-search status, browse-tree support, access-control support, and source rate-limit level. 2. Preserve the BaseSource v2 contract: `create(*, auth, logger, http_client, config)`, `generate_entities(*, cursor=None, files=None, node_selections=None)`, and `validate()` are the primary connector hooks. Do not add ad-hoc credential or logger setup outside these dependencies. 3. Use the injected auth provider and `AirweaveHttpClient`. Sources should call `auth.get_token()`, `auth.force_refresh()` when supported, or direct credential models from `DirectCredentialProvider`; external HTTP calls should flow through the injected client so SSRF checks and optional source rate limits apply. 4. Put public schema fields in typed Pydantic config/auth models. Use field metadata (`is_secret`, `feature_flag`, `auth_provider_field`, `required_for_auth`, `exclude_from_ui`) deliberately because the registry turns those classes into API-visible `Fields`. 5. For continuous sync, set both `supports_continuous=True` and `cursor_class=...`; the decorator rejects continuous sources without a cursor class. Update the cursor only with stable, source-owned state and support full-sync fallback when cursor state is absent or invalid. 6. For federated sources, set `federated_search=True`, implement `search(query, limit)`, and do not pretend full sync exists. Slack intentionally raises from `generate_entities()` because it searches Slack at query time. 7. For ACL-aware sources, set `supports_access_control=True`, set `entity.access` on yielded entities, and implement `generate_access_control_memberships()` when group expansion is needed. Principal formats must stay compatible with the search broker. 8. For browse-tree sources, set `supports_browse_tree=True`, implement `get_browse_children(parent_node_id)` and `parse_browse_node_id(node_id)`, and keep emitted node IDs parsable into the metadata needed by targeted sync. 9. Keep Monke and manual credentialed workflows external. No bundled script is included for this sub-skill: the only relevant source-maintained manual artifact is the SharePoint browse-tree manual test, and it requires real Microsoft tenant credentials and external state. ## Safe validation anchors When a backend Python 3.13 environment with Airweave and pytest is active, prefer connector-owned unit anchors first: ```bash cd backend python -m pytest tests/unit/platform/sources/test_http_helpers.py -q python -m pytest tests/unit/platform/sources/test_process_entities_concurrent.py -q python -m pytest tests/unit/platform/sources/test_sharepoint_online_acl.py -q python -m pytest tests/unit/platform/sources/test_sharepoint_online_group_expansion.py -q python -m pytest tests/unit/platform/sources/test_sharepoint2019v2.py -q python -m pytest tests/unit/platform/sources/test_sharepoint2019v2_dirsync.py -q python -m pytest tests/unit/api/test_source_feature_flags.py -q ``` Credentialed or local-stack E2E anchors are useful but not safe to run unattended: source listing/detail, federated search, source rate limits, continuous sync, entity definitions, OAuth/direct/auth-provider/token source-connection variants, and SharePoint legacy DirSync tests are cataloged in [references/troubleshooting.md](references/troubleshooting.md#validation-anchors). The prepared backend inspection facts for this skill were CPU-only: `airweave==0.1.0` imported on Python 3.13.14, connector-relevant imports such as `airweave.schemas.source_connection`, `airweave.platform.sources._base`, and `airweave.platform.decorators` succeeded, no accelerator backend is required for this scope, and settings imports require documented environment variables with sufficiently long `STATE_SECRET` and `SVIX_JWT_SECRET` values.