integration-builder

name

integration-builder

description

Build integration provider packages for the Open Mercato Integration Marketplace. Use when creating new external integrations (payment gateways, shipping carriers, data sync connectors, communication channels, storage providers, webhook endpoints). Handles npm package scaffolding, adapter implementation, credentials, widget injection, webhook processing, health checks, i18n, and tests. Triggers on "build integration", "create integration", "add provider", "new connector", "integrate with", "add stripe/paypal/dhl/sendgrid" etc.

Integration Builder

Build integration provider packages for the Open Mercato Integration Marketplace (SPEC-045). Every external integration MUST live in its own npm workspace package under packages/<provider-package>/.

Table of Contents

1. Pre-Flight
2. Determine Integration Category
3. Scaffold Package
4. Implement Core Files
5. Implement Adapter
6. Add Webhook Processing
7. Add Health Check
8. Add Widget Injection
9. Add i18n
10. Add Tests
11. Wire Into App
12. Verification

---

1. Pre-Flight

Before writing any code:

1. Identify the external service (Stripe, DHL, SendGrid, S3, etc.)
2. Read the hub's adapter contract — load the reference file from references/adapter-contracts.md
3. Read the reference implementation — packages/gateway-stripe/ is the canonical example
4. Check existing integrations — ls packages/gateway-* packages/carrier-* packages/sync-* packages/channel-* packages/storage-*
5. Read the external service's API docs — understand auth, endpoints, webhooks, status models
6. Check for an SDK — prefer official SDKs over raw HTTP (stripe, @aws-sdk/client-s3, etc.)

---

2. Determine Integration Category

Match the external service to ONE hub category:

Category	Hub Module	Adapter Contract	Package Prefix	Example
payment	payment_gateways	GatewayAdapter	gateway-	gateway-stripe, gateway-paypal
shipping	shipping_carriers	ShippingAdapter	carrier-	carrier-dhl, carrier-inpost
data_sync	data_sync	DataSyncAdapter	sync-	sync-medusa, sync-shopify
communication	communication_channels	ChannelAdapter	channel-	channel-whatsapp, channel-twilio
storage	storage_providers	StorageAdapter	storage-	storage-s3, storage-gcs
webhook	webhook_endpoints	WebhookEndpointAdapter	webhook-	webhook-zapier

Package naming: @open-mercato/<prefix><provider> (e.g., @open-mercato/gateway-stripe) Module naming: <prefix>_<provider> in snake_case (e.g., gateway_stripe)

If the service spans multiple categories (e.g., MedusaJS does products + customers + orders), use an Integration Bundle — see Section 4.2.

---

3. Scaffold Package

3.1 Create Package Directory

packages/<prefix><provider>/
├── package.json
├── tsconfig.json
├── src/
│   ├── index.ts                          # barrel export
│   └── modules/<module_id>/
│       ├── index.ts                      # module metadata
│       ├── integration.ts                # Integration Marketplace registration
│       ├── acl.ts                        # RBAC features
│       ├── setup.ts                      # tenant init, default role features
│       ├── di.ts                         # DI registrar (Awilix)
│       ├── data/
│       │   └── validators.ts             # Zod schemas
│       ├── lib/
│       │   ├── client.ts                 # SDK/HTTP client factory
│       │   ├── shared.ts                 # shared helpers, status maps
│       │   ├── health.ts                 # health check implementation
│       │   ├── status-map.ts             # provider status → unified status
│       │   ├── webhook-handler.ts        # webhook signature verification
│       │   └── adapters/                 # versioned adapter implementations
│       │       └── v<version>.ts
│       ├── workers/
│       │   └── webhook-processor.ts      # async webhook processing worker
│       ├── widgets/
│       │   ├── injection-table.ts        # widget-to-slot mappings
│       │   └── injection/<widget-name>/
│       │       ├── widget.ts             # widget metadata
│       │       └── widget.client.tsx      # React component
│       ├── i18n/
│       │   ├── en.ts                     # English translations (code)
│       │   ├── en.json                   # English translations (data)
│       │   └── ...                       # other locales
│       └── __tests__/
│           └── *.test.ts

3.2 package.json

{
  "name": "@open-mercato/<prefix><provider>",
  "version": "0.1.0",
  "private": true,
  "type": "module",
  "exports": {
    ".": "./src/index.ts",
    "./*": "./src/*.ts",
    "./**/*": "./src/**/*.ts",
    "./**/**/*": "./src/**/**/*.ts",
    "./**/**/**/*": "./src/**/**/**/*.ts",
    "./**/**/**/**/*": "./src/**/**/**/**/*.ts"
  },
  "scripts": {
    "build": "tsc --project tsconfig.json",
    "test": "vitest run"
  },
  "dependencies": {
    "@open-mercato/shared": "workspace:*"
  },
  "devDependencies": {
    "typescript": "^5.4.0",
    "vitest": "^2.0.0"
  }
}

Add the external SDK as a dependency (e.g., "stripe": "^17.0.0", "@aws-sdk/client-s3": "^3.x").

3.3 tsconfig.json

{
  "extends": "../../tsconfig.base.json",
  "compilerOptions": {
    "outDir": "dist",
    "rootDir": "src"
  },
  "include": ["src"]
}

3.4 src/index.ts

export * from './modules/<module_id>/index'

---

4. Implement Core Files

4.1 integration.ts (CRITICAL — marketplace registration)

This is the most important file. It registers the integration into the marketplace.

import type { IntegrationDefinition } from '@open-mercato/shared/modules/integrations'

export const integration: IntegrationDefinition = {
  id: '<module_id>',                          // e.g., 'gateway_stripe'
  title: '<Provider Display Name>',           // e.g., 'Stripe'
  description: '<one-line description>',
  category: '<category>',                     // payment | shipping | data_sync | communication | webhook | storage
  hub: '<hub_module>',                        // payment_gateways | shipping_carriers | data_sync | ...
  providerKey: '<provider_key>',              // e.g., 'stripe', 'dhl', 'sendgrid'
  icon: '<icon_id>',                          // icon identifier for UI
  package: '@open-mercato/<package-name>',
  version: '1.0.0',
  tags: ['<tag1>', '<tag2>'],
  credentials: {
    fields: [
      // Define ALL credentials needed to connect to the external service
      { key: 'apiKey', label: 'API Key', type: 'secret', required: true },
      { key: 'webhookSecret', label: 'Webhook Secret', type: 'secret', required: true,
        helpDetails: {
          kind: 'webhook_setup',
          title: 'Webhook Configuration',
          summary: 'Configure webhooks in the provider dashboard.',
          endpointPath: '/api/<hub>/webhook/<providerKey>',
          dashboardPathLabel: 'Provider Dashboard > Webhooks',
          steps: ['Go to provider dashboard', 'Add webhook URL', 'Copy signing secret'],
        }
      },
    ],
  },
  // Optional: versioned API adapters
  apiVersions: [
    { id: '2025-01-01', label: 'v2025-01-01 (latest)', status: 'stable', default: true },
  ],
  healthCheck: { service: '<providerKey>HealthCheck' },
}

Credential field types: text, secret, url, select, boolean, oauth, ssh_keypair

Conditional visibility: Use visibleWhen to show/hide fields based on other field values:

{ key: 'endpoint', label: 'Custom Endpoint', type: 'url',
  visibleWhen: { field: 'useCustomEndpoint', equals: true } }

4.2 Bundle Integration

For multi-integration providers (one npm package → many integrations sharing credentials):

import type { IntegrationBundle, IntegrationDefinition } from '@open-mercato/shared/modules/integrations'

export const bundle: IntegrationBundle = {
  id: 'sync_medusa',
  title: 'MedusaJS',
  description: 'Sync products, customers, and orders with MedusaJS',
  credentials: { fields: [
    { key: 'apiUrl', label: 'MedusaJS API URL', type: 'url', required: true },
    { key: 'apiKey', label: 'API Key', type: 'secret', required: true },
  ]},
  healthCheck: { service: 'medusaHealthCheck' },
}

export const integrations: IntegrationDefinition[] = [
  { id: 'sync_medusa_products', title: 'MedusaJS Products', category: 'data_sync', hub: 'data_sync', providerKey: 'medusa_products', bundleId: 'sync_medusa' },
  { id: 'sync_medusa_customers', title: 'MedusaJS Customers', category: 'data_sync', hub: 'data_sync', providerKey: 'medusa_customers', bundleId: 'sync_medusa' },
  { id: 'sync_medusa_orders', title: 'MedusaJS Orders', category: 'data_sync', hub: 'data_sync', providerKey: 'medusa_orders', bundleId: 'sync_medusa' },
]

4.3 index.ts (module metadata)

import type { ModuleInfo } from '@open-mercato/shared/modules/registry'
export const metadata: ModuleInfo = {
  name: '<module_id>',
  title: '<Provider> Integration',
  version: '0.1.0',
  description: '<what this integration does>',
  author: 'Open Mercato Team',
  license: 'Proprietary',
  ejectable: true,
}
export { features } from './acl'

4.4 acl.ts

export const features = [
  { id: '<module_id>.view', title: 'View <Provider> configuration', module: '<module_id>' },
  { id: '<module_id>.configure', title: 'Configure <Provider> settings', module: '<module_id>' },
]

4.5 setup.ts

import type { ModuleSetupConfig } from '@open-mercato/shared/modules/setup'

export const setup: ModuleSetupConfig = {
  defaultRoleFeatures: {
    superadmin: ['<module_id>.view', '<module_id>.configure'],
    admin: ['<module_id>.view', '<module_id>.configure'],
  },
}
export default setup

4.6 di.ts

import type { AppContainer } from '@open-mercato/shared/lib/di/container'

export function register(container: AppContainer): void {
  // Register adapter(s) — see Section 5 for category-specific registration
  // Register health check — see Section 7
  // Register webhook handler — see Section 6
}

---

5. Implement Adapter

Read references/adapter-contracts.md for the full type definitions per category.

5.1 Payment Gateway (GatewayAdapter)

// lib/adapters/v<version>.ts
import type { GatewayAdapter, CreateSessionInput, CreateSessionResult, ... } from '@open-mercato/shared/modules/payment_gateways/types'
import { createClient } from '../client'

export class MyGatewayAdapter implements GatewayAdapter {
  readonly providerKey = '<provider>'

  async createSession(input: CreateSessionInput): Promise<CreateSessionResult> { ... }
  async capture(input: CaptureInput): Promise<CaptureResult> { ... }
  async refund(input: RefundInput): Promise<RefundResult> { ... }
  async cancel(input: CancelInput): Promise<CancelResult> { ... }
  async getStatus(input: GetStatusInput): Promise<GatewayPaymentStatus> { ... }
  async verifyWebhook(input: VerifyWebhookInput): Promise<WebhookEvent> { ... }
  mapStatus(providerStatus: string, eventType?: string): UnifiedPaymentStatus { ... }
}

DI registration (in di.ts):

import { registerGatewayAdapter, registerWebhookHandler } from '@open-mercato/shared/modules/payment_gateways/types'
import { MyGatewayAdapter } from './lib/adapters/v2025'

export function register(container: AppContainer): void {
  const adapter = new MyGatewayAdapter()
  registerGatewayAdapter(adapter, { version: '2025-01-01' })
  registerWebhookHandler('<provider>', (input) => adapter.verifyWebhook(input), { queue: '<provider>-webhook' })
}

5.2 Shipping Carrier (ShippingAdapter)

// lib/adapters/v<version>.ts
import type { ShippingAdapter } from '<path>/shipping_carriers/lib/adapter'

export class MyShippingAdapter implements ShippingAdapter {
  readonly providerKey = '<provider>'

  async calculateRates(input): Promise<ShippingRate[]> { ... }
  async createShipment(input): Promise<CreateShipmentResult> { ... }
  async getTracking(input): Promise<TrackingResult> { ... }
  async cancelShipment(input): Promise<{ status: UnifiedShipmentStatus }> { ... }
  async verifyWebhook(input): Promise<ShippingWebhookEvent> { ... }
  mapStatus(carrierStatus: string): UnifiedShipmentStatus { ... }
}

5.3 Data Sync (DataSyncAdapter)

// lib/adapters/v<version>.ts
import type { DataSyncAdapter, StreamImportInput, ImportBatch } from '<path>/data_sync/lib/adapter'

export class MySyncAdapter implements DataSyncAdapter {
  readonly providerKey = '<provider>'
  readonly direction = 'import' // or 'export' | 'bidirectional'
  readonly supportedEntities = ['products', 'customers']

  async *streamImport(input: StreamImportInput): AsyncIterable<ImportBatch> {
    let cursor = input.cursor
    let hasMore = true
    let batchIndex = 0
    while (hasMore) {
      const page = await this.fetchPage(input.entityType, cursor, input.credentials)
      yield { items: page.items, cursor: page.nextCursor, hasMore: page.hasMore, batchIndex }
      cursor = page.nextCursor
      hasMore = page.hasMore
      batchIndex++
    }
  }

  async getMapping(input): Promise<DataMapping> { ... }
  async validateConnection(input): Promise<ValidationResult> { ... }
}

5.4 Status Mapping

Every adapter MUST implement bidirectional status mapping:

// lib/status-map.ts

const STATUS_MAP: Record<string, UnifiedPaymentStatus> = {
  'provider_pending': 'pending',
  'provider_paid': 'captured',
  'provider_refunded': 'refunded',
  // ... map ALL provider statuses
}

export function mapProviderStatus(providerStatus: string): UnifiedPaymentStatus {
  return STATUS_MAP[providerStatus] ?? 'unknown'
}

5.5 Client Factory

// lib/client.ts

export function createClient(credentials: Record<string, unknown>) {
  const apiKey = credentials.secretKey as string
  if (!apiKey) throw new Error('Missing secretKey credential')
  return new ProviderSDK(apiKey)
}

MUST: Never store credentials — resolve them fresh from credentials parameter on every call.

---

6. Add Webhook Processing

If the external service sends webhooks (most do):

6.1 Webhook Handler

// lib/webhook-handler.ts

export async function verifyProviderWebhook(input: VerifyWebhookInput): Promise<WebhookEvent> {
  const { rawBody, headers, credentials } = input
  const secret = credentials.webhookSecret as string
  // Use provider SDK for signature verification when available
  // Return normalized WebhookEvent
  return {
    eventType: '<provider>.<entity>.<action>',
    eventId: '<provider-event-id>',
    data: parsedPayload,
    idempotencyKey: `<provider>:${eventId}`,
    timestamp: new Date(parsedPayload.created),
  }
}

6.2 Webhook Worker

// workers/webhook-processor.ts

export const metadata = {
  queue: '<provider>-webhook',
  id: '<module_id>:webhook-processor',
  concurrency: 5,  // I/O-bound
}

export default async function handle(job: QueuedJob, ctx: JobContext) {
  // 1. Parse webhook event
  // 2. Resolve credentials via integrationCredentials service
  // 3. Process event (update local state, emit events)
  // 4. Log result via integrationLog service
}

6.3 Webhook Guide (for admin UI)

// webhook-guide.ts

import type { IntegrationCredentialWebhookHelp } from '@open-mercato/shared/modules/integrations'

export const webhookSetupGuide: IntegrationCredentialWebhookHelp = {
  kind: 'webhook_setup',
  title: '<Provider> Webhook Configuration',
  summary: 'Configure <Provider> to send webhook events to Open Mercato.',
  endpointPath: '/api/<hub>/webhook/<providerKey>',
  dashboardPathLabel: '<Provider> Dashboard > Developers > Webhooks',
  steps: [
    'Log in to your <Provider> dashboard',
    'Navigate to Developers > Webhooks',
    'Click "Add endpoint"',
    'Paste the webhook URL shown below',
    'Select the events you want to receive',
    'Copy the signing secret and paste it above',
  ],
  events: ['payment_intent.succeeded', 'charge.refunded'],
  localDevelopment: {
    tunnelCommand: 'npx localtunnel --port 3000',
    publicUrlExample: 'https://xxx.loca.lt/api/<hub>/webhook/<providerKey>',
    note: 'Use a tunnel for local webhook testing',
  },
}

---

7. Add Health Check

// lib/health.ts

import type { AppContainer } from '@open-mercato/shared/lib/di/container'

export function createHealthCheck(container: AppContainer) {
  return {
    async check(credentials: Record<string, unknown>): Promise<{
      healthy: boolean
      details?: Record<string, unknown>
      message?: string
    }> {
      try {
        const client = createClient(credentials)
        const result = await client.someValidationEndpoint()
        return { healthy: true, details: { accountId: result.id } }
      } catch (error) {
        return {
          healthy: false,
          message: error instanceof Error ? error.message : 'Connection failed',
        }
      }
    },
  }
}

DI registration (add to di.ts):

import { asFunction } from 'awilix'
container.register({
  '<providerKey>HealthCheck': asFunction(createHealthCheck).singleton(),
})

The service name MUST match integration.ts → healthCheck.service.

---

8. Add Widget Injection

Inject configuration UI into the integration detail page:

8.1 Widget Metadata

// widgets/injection/<widget-name>/widget.ts

import type { WidgetDefinition } from '@open-mercato/shared/modules/widgets'

export const widget: WidgetDefinition = {
  id: '<module_id>:config',
  type: 'injection',
  label: '<Provider> Configuration',
  component: () => import('./widget.client'),
}

8.2 Widget Component

// widgets/injection/<widget-name>/widget.client.tsx
'use client'

import { useT } from '@open-mercato/shared/lib/i18n/context'

export default function ProviderConfigWidget({ context }: { context: Record<string, unknown> }) {
  const t = useT()
  // Render provider-specific configuration UI
  // context contains: integrationId, credentials (masked), isEnabled, scope
  return <div>...</div>
}

8.3 Injection Table

// widgets/injection-table.ts

export const widgetInjections = [
  {
    widgetId: '<module_id>:config',
    spotId: 'integrations.detail:tabs',
    position: 'append',
    metadata: { tab: { label: 'Configuration', icon: 'settings' } },
  },
]

Available injection spots for integrations:

• integrations.detail:tabs — tab on integration detail page
• integrations.detail:settings — settings section
• integrations.bundle:tabs — tab on bundle detail page

---

9. Add i18n

9.1 English Translations

// i18n/en.ts

export default {
  '<module_id>': {
    title: '<Provider>',
    description: '<one-line description>',
    credentials: {
      apiKey: 'API Key',
      webhookSecret: 'Webhook Signing Secret',
    },
    status: {
      connected: 'Connected',
      disconnected: 'Disconnected',
    },
    errors: {
      invalidCredentials: 'Invalid credentials',
      connectionFailed: 'Connection to <Provider> failed',
    },
  },
}

MUST: Never hard-code user-facing strings. Use useT() client-side, resolveTranslations() server-side.

---

10. Add Tests

10.1 Unit Tests

// __tests__/status-map.test.ts

import { describe, it, expect } from 'vitest'
import { mapProviderStatus } from '../lib/status-map'

describe('status-map', () => {
  it('maps known statuses', () => {
    expect(mapProviderStatus('provider_paid')).toBe('captured')
  })
  it('returns unknown for unmapped statuses', () => {
    expect(mapProviderStatus('something_new')).toBe('unknown')
  })
})

MUST test:

• Status mapping (all provider statuses → unified statuses)
• Webhook signature verification (valid, invalid, expired)
• Client factory (missing credentials throw)
• Adapter methods (mock SDK calls)

10.2 Integration Tests

Place in __integration__/ directory following the integration-tests skill pattern:

Test Case	Description
Create session / rate / sync	Happy path for primary adapter method
Webhook verification (valid)	Valid signature accepted
Webhook verification (invalid)	Invalid signature rejected
Health check (healthy)	Valid credentials return healthy
Health check (unhealthy)	Invalid credentials return unhealthy
Credential validation	Missing required fields rejected
Status mapping completeness	All known provider statuses mapped

---

11. Wire Into App

11.1 Add to App Modules

Add the package to apps/mercato/src/modules.ts:

import '@open-mercato/<prefix><provider>'

11.2 Add Workspace Dependency

In apps/mercato/package.json:

"@open-mercato/<prefix><provider>": "workspace:*"

11.3 Run Generators

yarn install                  # link workspace package
yarn generate                 # discover integration.ts, widgets, workers, update generated files

---

12. Verification

After completing the implementation:

1. Build: yarn build:packages — must pass
2. Lint: yarn lint — must pass
3. Tests: yarn test --filter <package-name> — must pass
4. Module prepare: yarn generate — integration discovered
5. Dev server: yarn dev — integration visible in /backend/integrations
6. Health check: Test via admin panel
7. Credential save: Save test credentials via admin panel

Self-Review Checklist

• integration.ts exports valid IntegrationDefinition with all required fields
• credentials.fields covers all secrets needed; secret fields use type: 'secret'
• Adapter implements ALL methods of the hub contract (no partial implementations)
• Status mapping covers ALL known provider statuses with 'unknown' fallback
• Webhook signature verification uses provider SDK or timing-safe comparison
• Health check validates real connectivity (not just credential format)
• No credentials stored in memory or logged — resolve fresh from credentials param
• i18n: all user-facing strings in locale files, no hardcoded strings
• ACL features declared and assigned in setup.ts defaultRoleFeatures
• Workers export metadata with { queue, id, concurrency }
• Widget injection table maps widgets to correct spots
• Package has unit tests for status mapping, webhook verification, client factory
• No any types — use zod schemas with z.infer, narrow with runtime checks
• Package-level imports (@open-mercato/<pkg>/...) for cross-module references

---

Rules

• MUST place every integration in its own npm workspace package under packages/
• MUST NOT add provider code inside packages/core/src/modules/
• MUST export integration.ts at module root for marketplace discovery
• MUST implement the FULL adapter contract for the chosen hub category
• MUST encrypt credentials at rest — never store raw secrets; use IntegrationCredentials service
• MUST use provider SDK for webhook signature verification when available
• MUST map ALL known provider statuses to unified statuses with 'unknown' fallback
• MUST add health check that validates real connectivity
• MUST use timing-safe comparison for any manual HMAC verification
• MUST add webhook setup guide (helpDetails) on webhook secret credential fields
• MUST add i18n translations — no hardcoded user-facing strings
• MUST run yarn generate after creating/modifying module files
• MUST NOT modify any files in packages/core/, packages/ui/, or packages/shared/
• MUST follow the gateway-stripe reference implementation patterns exactly
• MUST declare ACL features and wire them in setup.ts defaultRoleFeatures