coral-create-source-spec

name	coral-create-source-spec
description	Create or update a Coral source spec YAML for a custom HTTP API or local dataset. Use when authoring a standalone source for `coral source add --file`, or when adapting that spec into a Coral repo source under `sources/core` or `sources/community`.

Coral Create Source Spec

Use this skill when the task is to author or repair a Coral source spec.

Goal

Produce a valid, queryable Coral source spec that works with:

• coral source lint <path>
• coral source add --file <path>
• coral source test <name>
• coral sql
• coral.tables and coral.columns
• coral.inputs for source variables and secrets

Default Mode

Default to standalone source authoring for external developers.

That means:

• create a YAML source spec file
• lint it early with coral source lint <path>
• add it to Coral with coral source add --file <path> when you need to exercise it as a source
• validate by querying it
• iterate until the shape is correct

Only switch to Coral repo layout when the user is explicitly editing the Coral repo.

Output Modes

• External authoring:
  • create a standalone source spec such as ./my-source.yaml
  • validate structure with coral source lint ./my-source.yaml
  • load it with coral source add --file ./my-source.yaml when you need to query it through Coral
• Coral repo contribution:
  • write community source specs to sources/community/<name>/manifest.yaml
  • write core source specs to sources/core/<name>/manifest.yaml only when the user is intentionally changing bundled core sources
  • add representative test_queries for a basic smoke/connection check of the source
  • validate with coral source test <name> and repo checks

Workflow

1. Read the provider API docs or inspect the local dataset.
2. Start with one small table and a few columns.
3. Define:
   • source metadata
   • backend
   • base URL or file location
   • auth
   • variables and secrets
   • credential retrieval methods for secrets, including OAuth when the provider supports browser-based setup
   • if authoring a separate kind: identity manifest, identity setup inputs and OAuth method config
   • tables
   • table functions for source-scoped parameterized endpoints
   • filters
   • response extraction
   • pagination
   • typed columns
4. Lint the source spec:
   • coral source lint <path>
5. Validate the source in the right mode:
   • standalone specs: coral source add --file <path> and inspect with coral sql
   • coral source add is non-interactive by default: each input key is read from the matching environment variable. Export required variables and secrets before running, or pass --interactive to be prompted.
   • for OAuth credential methods, run coral source add --interactive --file <path> with no environment value for the target secret so Coral offers the authored credential choices
   • repo sources or already-named sources: coral source test <name>
6. Inspect the exposed shape:
   • inspect coral.tables for visible tables, descriptions, guides, and required filters; keep metadata queries bounded with LIMIT/OFFSET
   • inspect coral.table_functions for source-scoped functions, descriptions, guides, arguments, result columns, kind, and search limits
   • inspect coral.columns for canonical column metadata, including is_virtual and is_required_filter; filter by one table or page large column sets
   • inspect coral.filters for normalized table filter names, types, modes, required flags, and descriptions
   • inspect coral.inputs to verify variables, secrets, defaults, hints, and required flags
7. Query representative tables with coral sql.
8. If you are relying on , make sure gives you a basic smoke/connection check for the source.

Authoring Rules

• Start small and expand table coverage incrementally.
• Use the source manifest schema as both inspiration for authoring and validation of structure: https://github.com/withcoral/coral/blob/main/crates/coral-spec/src/schema/source_manifest.schema.json
• Use source variables for non-secret configuration.
• Use source secrets for credentials.
• For OAuth-backed services, model setup with inputs.<TOKEN>.credential.methods[] using type: oauth; keep the runtime auth or request header pointing at the same secret input.
• OAuth credential methods support device-code flow and authorization-code flow. For authorization-code flow, set flow.type: authorization_code, set flow.pkce explicitly to required or disabled, use a loopback http://127.0.0.1 or http://localhost redirect URI, choose redirect_uri_port_mode: random for provider apps or Dynamic Client Registration endpoints that allow variable localhost ports, and choose fixed only when users can register the exact non-zero redirect URI. The CLI also accepts the final loopback redirect URL pasted into the terminal when the browser cannot reach the machine running Coral, so do not reject authorization-code OAuth solely because users may run Coral over SSH, in a VM, or in another split-browser environment. For device-code flow, declare flow.type: device_code, endpoints.device_authorization_url, endpoints.token_url, and either a public client ID or client.dynamic_registration; omit redirect URI fields and do not declare a static client secret.
• When the provider supports OAuth Dynamic Client Registration, prefer client.dynamic_registration.registration_url over hard-coding a shared public client ID. Coral always registers a native client; source specs cannot configure a web Dynamic Client Registration application type. Set token_endpoint_auth_method to none, client_secret_basic, or client_secret_post to match the registered client type. Set request_refresh_token_grant: true only when the provider supports registering clients for refresh-token grants. For OAuth-protected Streamable HTTP MCP servers, set oauth.resource to the MCP server URL when the provider requires resource indicators.

Metadata UX Rules

Use these rules for top-level source metadata so source discovery and setup are consistent.

description

• Start with Query ....
• Make the first sentence capability-first: list the key entities users can query.
• Preferred template:
  • Query <entities> from <Provider> (<Cloud or self-hosted when relevant>).
• Keep description focused on data coverage, not setup steps.
• Do not use vague phrasing such as:
  • REST API v3
  • OpenAPI provider
  • ... and more
• Move auth/setup/permission details to input hints, not description text.

Input hints (inputs.<KEY>.hint)

Each hint should tell the user:

• what value is expected
• how to obtain it
• minimum scope/permission guidance
• one concrete format/example when useful

Specific guidance:

• For URL/base inputs:
  • say what the default means
  • include at least one concrete example
  • include self-hosted guidance when supported
• For secrets:
  • name the exact credential type (API key, PAT, application key, etc.)
  • include format constraints when relevant (for example, token prefixes)
  • include least-privilege scope guidance
• For OAuth methods:
  • use a user-facing label such as Connect with GitHub
  • keep description to a short one-line blurb; put the setup detail in the method's hint
  • in the method's hint, list the required OAuth scopes and explain whether users must register a fixed loopback redirect URI or provide their own OAuth client ID/secret; when the method collects client.id.input/client.secret.input, say where to obtain those values
  • for authorization-code flow, note in the method's hint that users can paste the final localhost redirect URL into the terminal if their browser cannot reach Coral's loopback listener directly
  • when a secret offers multiple methods, scope each method's hint to the inputs that method collects instead of writing one broad input-level hint that mixes guidance for every method
• For derived secrets (for example Basic auth blobs):
  • include a short shell example (for example a Base64 command)
• Prefer stable documentation links.
  • Use official docs links and stable settings pages.
  • Avoid brittle click-path instructions as the primary guidance.

Keep hints concise and directly actionable.

Validation Loop

Use this loop during authoring:

# Export any required inputs first (key matches the input `key` in the spec),
# or pass --interactive to be prompted.
coral source lint ./my-source.yaml
coral source add --file ./my-source.yaml
coral source test my_source
coral sql "SELECT catalog_name, schema_name, table_name, description, required_filters FROM coral.tables WHERE schema_name = 'my_source' OR catalog_name = 'my_source' ORDER BY catalog_name, schema_name, table_name LIMIT 50 OFFSET 0"
coral sql "SELECT function_name, kind, arguments_json, result_columns_json, search_limits_json FROM coral.table_functions WHERE schema_name = 'my_source' ORDER BY function_name LIMIT 50 OFFSET 0"
coral sql "SELECT table_name, filter_name, filter_mode, is_required, data_type, description FROM coral.filters WHERE schema_name = 'my_source' OR catalog_name = 'my_source' ORDER BY table_name, filter_name LIMIT 100 OFFSET 0"
coral sql "SELECT table_name, column_name, data_type, is_virtual, is_required_filter, filter_mode, description FROM coral.columns WHERE schema_name = 'my_source' OR catalog_name = 'my_source' ORDER BY table_name, ordinal_position LIMIT 100 OFFSET 0"
coral sql "SELECT catalog_name, key, kind, value, default_value, hint, required, is_set FROM coral.inputs WHERE schema_name = 'my_source' OR catalog_name = 'my_source' ORDER BY key"

Each WHERE clause matches on both qualifiers because a source is addressed either by schema or by catalog. Schema-addressed sources carry the source name in schema_name and leave catalog_name empty; database sources put the source name in catalog_name, and their coral.inputs rows have an empty schema_name. Filtering on schema_name alone returns zero rows for a database source without reporting an error, which reads as "this source declares nothing".

For repo sources or already-named sources, add test_queries for a basic smoke/connection check and run:

coral source test my_source

Then run targeted table queries until the source behaves correctly.

HTTP Sources

For HTTP-backed sources:

• define backend: http
• define base_url
• define auth headers or other runtime auth fields
• define credential.methods on secret inputs when setup should offer OAuth or another retrieval choice
• define request path, query, and body only where needed
• define source-scoped table functions for provider-native operations that require invocation arguments
• define response rows_path
• define pagination explicitly when the provider pattern is known
• define typed columns
• add test_queries once you know which simple query or queries should confirm the source basically works

Read references/http-source-checklist.md when you need table-shape and pagination guidance.

If your HTTP source uses an Authorization header with a prefix (e.g. Authorization: Bearer <token>), use a secret input for the raw token and define the header with from: bearer:

inputs:
  FOOBAR_API_TOKEN:
    kind: secret
    hint: Bearer token for the Foobar API.
auth:
  type: HeaderAuth
  headers:
    - name: Authorization
      from: bearer
      key: FOOBAR_API_TOKEN

For an OAuth-backed HTTP source, add the retrieval method to that same secret input:

inputs:
  FOOBAR_API_TOKEN:
    kind: secret
    hint: Connect with Foobar OAuth or paste a token with read access.
    credential:
      methods:
        - type: oauth
          label: Connect with Foobar
          description: Open a browser and authorize Coral to read Foobar data.
          hint: |
            Signs you in through Foobar and requests the `read` scope. To
            use your own app, set FOOBAR_OAUTH_CLIENT_ID to its Client ID.
          oauth:
            flow:
              type: authorization_code
              pkce: required
            redirect_uri: http://127.0.0.1:0/oauth/callback
            redirect_uri_port_mode: random
            endpoints:
              authorization_url: https://foobar.example.com/oauth/authorize
              token_url: 
            
              
                 
            
              
                 
                
                   
          
            
                       

   
  
      
       
       

When a provider accepts either a full pasted API-key header or an OAuth access token, declare both credential inputs as optional secrets, then use from: one_of and put the complete header value first, followed by a from: bearer OAuth fallback:

inputs:
  FOOBAR_API_KEY:
    kind: secret
    required: false
  FOOBAR_OAUTH_ACCESS_TOKEN:
    kind: secret
    required: false
auth:
  type: HeaderAuth
  headers:
    - name: Authorization
      from: one_of
      values:
        - from: input
          key: FOOBAR_API_KEY
        - from: bearer
          key: FOOBAR_OAUTH_ACCESS_TOKEN

Local Data Sources

For local file-backed sources:

• define the file backend
• define the source location
• define file selection patterns if applicable
• define typed columns

Deliverable

Report:

• source spec path
• lint / add / test commands used
• validation commands run
• assumptions made
• blocked or unverified endpoints