adapter-assistant

name	adapter-assistant
description	Complete adapter lifecycle assistant for LimaCharlie. Supports External Adapters (cloud-managed), Cloud Sensors (SaaS/cloud integrations), and On-prem USP adapters. Dynamically researches adapter types from local docs and GitHub usp-adapters repo. Creates, validates, deploys, and troubleshoots adapter configurations. Handles parsing rules (Grok, regex), field mappings, credential setup, and multi-adapter configs. Use when setting up new data sources (Okta, S3, Azure Event Hub, syslog, webhook, etc.), troubleshooting ingestion issues, or managing adapter deployments.
allowed-tools	["Task","Read","Bash","Skill","AskUserQuestion","WebFetch","WebSearch","Glob","Grep"]

Adapter Assistant

A comprehensive, dynamic assistant for LimaCharlie adapter lifecycle management. This skill researches adapter configurations from multiple sources and helps you create, validate, deploy, and troubleshoot adapters for any data source.

---

LimaCharlie Integration

Prerequisites: Run /init-lc to initialize LimaCharlie context.

LimaCharlie CLI Access

All LimaCharlie operations use the limacharlie CLI directly:

limacharlie <noun> <verb> --oid <oid> --output yaml [flags]

For command help and discovery: limacharlie <command> --ai-help

Critical Rules

Rule	Wrong	Right
CLI Access	Call MCP tools or spawn api-executor	Use Bash("limacharlie ...") directly
Output Format	--output json	--output yaml (more token-efficient)
Filter Output	Pipe to jq/yq	Use --filter JMESPATH to select fields
LCQL Queries	Write query syntax manually	Use limacharlie ai generate-query first
Timestamps	Calculate epoch values	Use date +%s or date -d '7 days ago' +%s
OID	Use org name	Use UUID (call limacharlie org list if needed)

---

When to Use

Use this skill when:

• Setting up new data sources: Connect syslog, webhooks, cloud services, or any external data to LimaCharlie
• Configuring SaaS integrations: Okta, CrowdStrike, Microsoft Defender, Office 365, AWS, Azure, etc.
• Troubleshooting adapters: Data not flowing, parsing issues, credential problems
• Modifying existing adapters: Update parsing rules, change credentials, add field mappings
• Auditing adapters: List and review adapters across organizations
• Connecting unknown products: Research how to integrate any product that sends logs or webhooks

Common scenarios:

• "I want to ingest Okta system logs"
• "Set up a syslog adapter for our firewall"
• "My Azure Event Hub adapter isn't receiving data"
• "How do I connect this product that sends webhooks?"
• "Create a webhook to receive alerts from our monitoring system"
• "List all adapters across my organizations"
• "Help me parse these custom log formats"

What This Skill Does

This skill is truly dynamic - it researches adapter configurations from multiple sources in real-time:

1. Local LimaCharlie documentation - Adapter types, configuration options, examples
2. GitHub usp-adapters repository - Source code for 50+ adapter implementations
3. External product documentation - API docs, webhook formats, authentication requirements

The skill then guides you through creating, validating, and deploying adapter configurations with proper parsing rules and credential management.

Supported Adapter Categories

Category	Description	Examples
External Adapter	Cloud-managed syslog, webhook, or API receivers	syslog, webhook, custom API
Cloud Sensor	Cloud-to-cloud SaaS integrations	Okta, CrowdStrike, O365, AWS S3, Azure Event Hub
On-prem/USP Adapter	Binary deployments with local configuration	file, syslog receiver, Kubernetes pods

Dynamic Research Strategy

This is the key capability - the skill researches ANY data source dynamically, not just predefined ones.

Microsoft sources — STOP and read the reference first. If the data source is anything Microsoft (Defender, Office 365 / M365, Entra ID / Azure AD, Azure logs, Windows Event Logs, EVTX), do NOT research or guess: Read("${CLAUDE_PLUGIN_ROOT}/skills/adapter-assistant/MICROSOFT.md") and use exactly the adapter type + platform pairing from its decision matrix. Microsoft exposes the same products through multiple feeds with different formats, and picking the wrong platform (e.g. json instead of msdefender, or azure_event_hub_namespace for data merely transiting an Event Hub) silently breaks parsing.

For ANY Data Source Request:

Step 1: Check for Native LimaCharlie Adapter

For a fast LOCAL view of supported adapter types and per-type config fields, ask the CLI directly (this complements — does not replace — the usp-adapters repo, which remains authoritative for field definitions):

# Supported types
limacharlie cloud-adapter list-types --oid <oid> --output yaml
limacharlie external-adapter list-types --oid <oid> --output yaml

# Config field listing for one type
limacharlie cloud-adapter schema --type <type> --oid <oid> --output yaml

Search local documentation:

Glob("./docs/limacharlie/doc/Sensors/Adapters/Adapter_Types/*{keyword}*.md")

Check GitHub usp-adapters repository (use API at root - adapters are NOT in a subdirectory):

WebFetch(
  url="https://api.github.com/repos/refractionPOINT/usp-adapters/contents",
  prompt="List all available adapter directories from the JSON response"
)

Step 2: Read LimaCharlie Adapter Documentation

If a native adapter exists, first list files in the adapter directory, then fetch the main source:

# Step 1: List files to find the main config file
WebFetch(
  url="https://api.github.com/repos/refractionPOINT/usp-adapters/contents/{adapter}",
  prompt="List all .go files in this adapter directory"
)

# Step 2: Fetch the config (usually client.go, but some adapters differ - e.g., sentinelone uses s1.go)
WebFetch(
  url="https://raw.githubusercontent.com/refractionPOINT/usp-adapters/master/{adapter}/client.go",
  prompt="Extract all configuration fields from the Config struct"
)

Step 3: Research the External Product

ALWAYS research the external product to understand its capabilities:

WebSearch("{product name} API documentation")
WebSearch("{product name} webhook integration")
WebSearch("{product name} audit logs export")

Extract:

• Authentication method (API key, OAuth, service account, etc.)
• Available event types or log categories
• Webhook payload format (if applicable)
• Rate limits and API best practices
• Required scopes or permissions

Step 4: Build Comprehensive Integration Plan

If NO native adapter exists, determine how to connect:

• Can it send webhooks? → LimaCharlie webhook adapter
• Does it write to S3/GCS/Azure Blob? → Cloud storage adapters
• Does it push to Kafka/Pub/Sub/SQS? → Queue adapters
• Can we pull via API? → May need polling or custom integration

Then map out:

• External product configuration requirements
• Matching LimaCharlie adapter configuration
• Parsing requirements for the data format
• Sample data format for validation

Workflow Phases

Phase 0: Requirements Gathering

Use AskUserQuestion to understand the user's needs:

1. Operation type: Create new adapter, modify existing, troubleshoot, or audit
2. Data source: What product/system are they connecting?
3. Adapter category preference: Cloud-managed or on-prem deployment

Phase 1: Dynamic Research

Execute the dynamic research strategy above to gather all relevant information about:

• LimaCharlie adapter capabilities
• External product API/webhook specifications
• Configuration requirements on both sides

Phase 2: Organization and Configuration Discovery

Get organizations:

limacharlie org list --output yaml

List existing External Adapters:

limacharlie external-adapter list --oid <oid> --output yaml

List existing Cloud Sensors:

limacharlie cloud-adapter list --oid <oid> --output yaml

Get existing configuration (if modifying):

limacharlie external-adapter get --key <adapter-name> --oid <oid> --output yaml
# or for cloud sensors:
limacharlie cloud-adapter get --key <sensor-name> --oid <oid> --output yaml

Phase 3: Configuration Generation

Build the configuration based on research:

Core client_options (required for all adapters):

client_options:
  identity:
    oid: "<organization-id>"
    installation_key: "<installation-key>"  # or use hive://secret/...
  platform: "text|json|carbon_black|gcp|..."
  sensor_seed_key: "<unique-identifier>"
  hostname: "<adapter-hostname>"
  mapping:
    parsing_grok:
      message: '%{PATTERN:field} ...'  # For text platform
    event_type_path: "field/path"
    event_time_path: "timestamp/path"
    sensor_hostname_path: "host/path"

Credential references (recommended):

Use Hive secrets for sensitive values:

apikey: "hive://secret/okta-api-key"
client_secret: "hive://secret/azure-client-secret"

Check existing secrets:

limacharlie secret list --oid <oid> --output yaml

Indexing configuration (optional):

indexing:
  - events_included: ["*"]
    path: "src_ip"
    index_type: "ip"
  - events_included: ["*"]
    path: "user/email"
    index_type: "user"

Supported index types: file_hash, file_path, file_name, domain, ip, user, service_name, package_name

Phase 4: Validation and Testing

Validate parsing rules before deployment:

# Write adapter config with mapping and sample data to a YAML file:
cat > /tmp/usp-test.yaml << 'EOF'
parsing_grok:
  message: "%{TIMESTAMP_ISO8601:timestamp} %{WORD:action} ..."
event_type_path: action
event_time_path: timestamp
sample_data:
  - "2024-01-15T10:30:00Z LOGIN [e-mail]"
EOF
limacharlie usp validate \
  --platform text \
  --input-file /tmp/usp-test.yaml \
  --oid <oid> --output yaml

Review validation results:

• Verify all expected fields are extracted
• Check timestamp parsing
• Confirm event type assignment
• Iterate on pattern if validation fails

For complex parsing, invoke the parsing-helper skill:

Skill("parsing-helper")

For local testing before production deployment:

Skill("test-limacharlie-adapter")

Phase 5: Deployment

Deploy External Adapter:

# Write configuration to a temp file first
cat > /tmp/adapter-config.yaml << 'EOF'
<full-configuration-yaml>
EOF
limacharlie external-adapter set --key <adapter-name> --input-file /tmp/adapter-config.yaml --oid <oid> --output yaml

Deploy Cloud Adapter:

cat > /tmp/cloud-adapter-config.yaml << 'EOF'
<full-configuration-yaml>
EOF
limacharlie cloud-adapter set --key <sensor-name> --input-file /tmp/cloud-adapter-config.yaml --oid <oid> --output yaml

For On-prem Adapters, generate deployment artifacts:

YAML Configuration:

syslog:
  client_options:
    identity:
      installation_key: "<IID>"
      oid: "<OID>"
    platform: text
    sensor_seed_key: "<unique-key>"
    hostname: "<hostname>"
    mapping:
      parsing_grok:
        message: '%{PATTERN:field} ...'
      event_type_path: "..."
  port: 514
  is_udp: true

CLI Command:

./lc_adapter syslog \
  client_options.identity.installation_key=<IID> \
  client_options.identity.oid=<OID> \
  client_options.platform=text \
  client_options.sensor_seed_key=<key> \
  "client_options.mapping.parsing_grok.message=%{PATTERN:field} ..." \
  port=514 \
  is_udp=true

Docker Command:

docker run -d --rm -p 514:514/udp refractionpoint/lc-adapter syslog \
  client_options.identity.installation_key=<IID> \
  client_options.identity.oid=<OID> \
  client_options.platform=text \
  client_options.sensor_seed_key=<key> \
  port=514 \
  is_udp=true

Verify deployment:

One step — ask the adapter record for its own live sensor(s). This matches by the adapter's installation-key IID for you and reports clearly when no sensor exists yet (empty result = the adapter has not delivered any events):

limacharlie cloud-adapter sensors --key <adapter-record> --oid <oid> --output yaml
# for an External Adapter record:
limacharlie external-adapter sensors --key <adapter-record> --oid <oid> --output yaml

Equivalent manual lookup (matches the same IID by hand) if you only have the IID:

limacharlie sensor list --selector "iid == \`<installation-key-iid>\`" --oid <oid> --output yaml

Phase 6: Post-Deployment

Troubleshoot if data not appearing:

1. Check adapter last_error field:

limacharlie external-adapter get --key <adapter-name> --oid <oid> --output yaml

2. Check organization errors for adapter issues:

limacharlie org errors --oid <oid> --output yaml

Look for errors with component names containing the adapter name.

3. Verify sensor exists in sensor list

4. Query for recent events:

# First calculate timestamps dynamically
start=$(date -d '1 hour ago' +%s) && end=$(date +%s)

limacharlie event list --sid <sensor-id> --start $start --end $end --oid <oid> --output yaml

5. Check for unparsed events (event_type: "unknown_event" with only text field)

Offer D&R rule creation:

Skill("detection-engineering")

Multi-Organization Adapter Audit

For auditing adapters across multiple organizations, spawn parallel sub-agents:

Task(
  subagent_type="lc-essentials:multi-org-adapter-auditor",
  prompt="Audit adapters for organization: {org_name} ({oid})

    Return:
    - List of external adapters with status
    - List of cloud sensors with status
    - Any adapters with errors
    - Configuration issues detected"
)

Adapter Type Quick Reference

Adapter Type	Platform	Deployment	Key Configuration
syslog	text	External/On-prem	port, is_udp, parsing_grok
webhook	json	Cloud Sensor	secret, client_options
okta	json	Cloud Sensor	apikey, url
s3	varies	Cloud Sensor/On-prem	bucket_name, access_key, secret_key, prefix
azure_event_hub	varies — match the feed streamed into the hub (msdefender, azure_ad, azure_monitor, …); see MICROSOFT.md	Cloud Sensor/On-prem	connection_string (must include EntityPath)
office365	office365	Cloud Sensor	domain, tenant_id, publisher_id, client_id, client_secret, endpoint, content_types
defender	msdefender	Cloud Sensor	tenant_id, client_id, client_secret (Graph alerts_v2 polling)
entraid	entraid	Cloud Sensor	tenant_id, client_id, client_secret (Graph riskDetections polling)
ms_graph	json	Cloud Sensor	tenant_id, client_id, client_secret, url (custom Graph endpoint)
wel	wel	On-prem (Windows only)	evt_sources
evtx	wel (no evtx platform exists)	On-prem	file_path
falconcloud	json	Cloud Sensor/On-prem	client_id, client_secret
pubsub	varies	Cloud Sensor/On-prem	project_id, subscription_id, service_account_creds
file	varies	On-prem	file_path, backfill
gmail	gmail	Cloud Sensor/On-prem	client_id, client_secret, refresh_token (single mailbox) or service_account_credentials, subject/subjects/discover_mailboxes (Workspace); collect_* capability flags
threatlocker	json	Cloud Sensor/On-prem	api_key, instance; optional feeds (defaults: approval_request, unified_audit, system_audit)
harmony	json	Cloud Sensor/On-prem	client_id, access_key; events.enabled / entities.queries blocks
cato	cato	Cloud Sensor/On-prem	apikey, accountid

For a quick LOCAL listing of supported types and a given type's config fields, use limacharlie cloud-adapter list-types / external-adapter list-types and limacharlie cloud-adapter schema --type <type>.

IMPORTANT: For authoritative field definitions, always check the usp-adapters repo (the local schema output complements but does not replace it):

• URL: https://github.com/refractionPOINT/usp-adapters
• Each adapter has a client.go with a *Config struct defining all valid fields
• All Cloud Sensors require a nested structure with client_options containing identity, platform, and sensor_seed_key

Example Usage

Example 1: Set Up Okta System Log Adapter

User: "I want to ingest Okta system logs into LimaCharlie"

Workflow:

1. Research Phase:

   • Read local docs: ./docs/limacharlie/doc/Sensors/Adapters/Adapter_Types/adapter-types-okta.md
   • WebSearch: "Okta System Log API documentation 2024"
   • Extract: API key requirements, URL format, event types

2. Configuration:

   sensor_type: "okta"
   okta:
     apikey: "hive://secret/okta-api-key"
     url: "https://your-company.okta.com"
     client_options:
       identity:
         oid: "<oid>"
         installation_key: "<iid>"
       hostname: "okta-system-logs"
       platform: "json"
       sensor_seed_key: "okta-logs-sensor"
       mapping:
         sensor_hostname_path: "client/device"
         event_type_path: "eventType"
         event_time_path: "published"
   

3. External Setup Instructions:

   • Create API token in Okta Admin Console
   • Store token in LimaCharlie secrets: limacharlie secret set okta-api-key --oid <oid>

4. Deploy: Use set_cloud_sensor

Example 2: Syslog Adapter with Custom Grok Parsing

User: "Set up a syslog adapter for our firewall. Sample log: <134>Nov 15 12:30:45 fw01 ACCEPT TCP 192.168.1.100:54321 10.0.0.5:443"

Workflow:

1. Analyze log format: Priority, syslog timestamp, hostname, action, protocol, src:port dst:port

2. Generate Grok pattern:

   parsing_grok:
     message: '<%{INT:priority}>%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host} %{WORD:action} %{WORD:protocol} %{IP:src_ip}:%{NUMBER:src_port} %{IP:dst_ip}:%{NUMBER:dst_port}'
   

3. Validate with validate_usp_mapping using the sample log

4. Deploy as External Adapter:

   cat > /tmp/firewall-adapter.yaml << 'EOF'
   adapter_type: syslog
   port: 514
   is_udp: true
   client_options:
     identity:
       oid: "<oid>"
       installation_key: "<iid>"
     platform: text
     sensor_seed_key: firewall-logs
     hostname: firewall-syslog
     mapping:
       parsing_grok:
         message: "<%{INT:priority}>%{SYSLOGTIMESTAMP:timestamp} %{HOSTNAME:host} %{WORD:action} %{WORD:protocol} %{IP:src_ip}:%{NUMBER:src_port} %{IP:dst_ip}:%{NUMBER:dst_port}"
       event_type_path: action
       event_time_path: timestamp
       sensor_hostname_path: host
   EOF
   limacharlie external-adapter set --key firewall-syslog --input-file /tmp/firewall-adapter.yaml --oid <oid> --output yaml
   

Example 3: Connect Unknown Product via Webhook

User: "I want to connect this monitoring tool that sends webhooks, I don't know if LimaCharlie supports it"

Workflow:

1. Ask clarifying questions:

   • What's the product name?
   • Do you have sample webhook payload?

2. Research the product:

   WebSearch("{product name} webhook documentation")
   WebSearch("{product name} webhook payload format")
   

3. Read LimaCharlie webhook adapter docs:

   Read("./docs/limacharlie/doc/Sensors/Adapters/Adapter_Types/adapter-types-webhook.md")
   

4. Build integration plan:

   • LimaCharlie webhook adapter configuration
   • Grok/JSON parsing for the webhook payload
   • Field mappings for proper event normalization

5. Deploy and provide webhook URL to user

Example 4: Troubleshoot Adapter Not Receiving Data

User: "My Azure Event Hub adapter isn't receiving data"

Workflow:

1. Get current configuration:

   limacharlie cloud-adapter get --key azure-event-hub --oid <oid> --output yaml
   

2. Check for errors in last_error field of sys_mtd

3. Common issues:

   • Connection string missing EntityPath
   • Consumer group not created
   • Credentials expired
   • Diagnostic settings not configured in Azure

4. Research Azure-side setup:

   WebSearch("Azure Event Hub diagnostic settings configuration")
   

5. Provide fix guidance based on error analysis

Common Pitfalls

1. Installation Key vs IID: For adapters, use the IID (UUID format like e9a3bcdf-efa2-47ae-b6df-579a02f3a54d), not the full base64 installation key

2. DATESTAMP vs TIMESTAMP_ISO8601:

   • YYYY-MM-DD HH:MM:SS → Use %{TIMESTAMP_ISO8601}
   • MM-DD-YY HH:MM:SS → Use %{DATESTAMP}
   • Jan 15 12:30:45 → Use %{SYSLOGTIMESTAMP}

3. Grok field name: Always use message as the key in parsing_grok for text platform:

   parsing_grok:
     message: '%{PATTERN:field}'  # Correct
   

4. Azure Event Hub connection string: Must include EntityPath=<event-hub-name> for consumer applications

5. Credential storage: Always use hive://secret/<secret-name> for sensitive values in production

6. Unparsed events: If you see event_type: "unknown_event" with only a text field, parsing is not configured

7. Timestamp calculations: Always use Bash to calculate epoch timestamps dynamically:

   start=$(date -d '1 hour ago' +%s) && end=$(date +%s)
   

Related Skills

• parsing-helper - For complex Grok pattern generation and validation
• test-limacharlie-adapter - For local testing before production deployment
• detection-engineering - For creating D&R rules on adapter data

Reference

For more details:

• Microsoft ecosystem (Defender, M365, Entra ID, Azure, WEL/EVTX): MICROSOFT.md — authoritative adapter/platform decision matrix
• Adapter usage: ./docs/limacharlie/doc/Sensors/Adapters/adapter-usage.md
• Adapter deployment: ./docs/limacharlie/doc/Sensors/Adapters/adapter-deployment.md
• Adapter types: ./docs/limacharlie/doc/Sensors/Adapters/Adapter_Types/
• GitHub source: https://github.com/refractionPOINT/usp-adapters