vector

# WizTelemetry Data Pipeline (Vector) ## Overview WizTelemetry Data Pipeline is an extension based on vector (https://vector.dev/) that provides the ability to collect, transform, and route observability data. It is a core dependency for other WizTelemetry extensions like Logging, Auditing, Events, and Notification. ## When to Use - Installing or configuring the WizTelemetry Data Pipeline extension - Setting up data collection for logs, auditing, events, and notifications - Configuring Vector sinks (OpenSearch) - Managing Vector agent components ## Installation ### Prerequisites **REQUIRED: Complete all steps in order before generating InstallPlan.** #### Step 1: Get Available Clusters and Confirm Target **⚠️ CRITICAL: DO NOT proceed until target clusters are determined.** **Step 1.1: Get available clusters** ```bash kubectl get clusters -o jsonpath='{.items[*].metadata.name}' ``` **Step 1.2: Determine target clusters** - If user **explicitly specified** target clusters in the request → Use those clusters directly, proceed to Step 2 - If user **did NOT specify** target clusters → Ask user to confirm which clusters to deploy to, then proceed to Step 2 **Ask user (if not specified):** ``` Available clusters: host, dev Which clusters do you want to deploy Vector to? ``` #### Step 2: Get OpenSearch Endpoint and Credentials (MUST DO) - If user **already provided** OpenSearch endpoint and credentials in the request → Use those directly, proceed to Step 3 - If user **did NOT provide** → **You MUST ask user** for OpenSearch endpoint and credentials **Ask user for (if not provided):** 1. **OpenSearch endpoint URL** (required) - Example: `http://<node-ip>:30920` or `https://opensearch.example.com:9200` 2. **OpenSearch credentials** (required) - Username (default: `admin`) - Password **DO NOT proceed to Step 3 until user provides both endpoint and credentials.** #### Step 3: Get Latest Vector Version (if not provided by user) **MUST do this to get the latest version:** ```bash kubectl get extensionversions -l kubesphere.io/extension-ref=vector -o jsonpath='{range .items[*]}{.spec.version}{"\n"}{end}' | sort -V | tail -1 ``` This outputs the latest version (e.g., `1.1.4`). Note this down - you'll use it in the InstallPlan. ### Install Vector Extension **⚠️ IMPORTANT: Complete prerequisite steps (1-3) BEFORE this step.** **⚠️ CRITICAL: InstallPlan `metadata.name` MUST be `vector`. DO NOT use any other name.** Based on your selections: - **Target clusters**: Use the user-confirmed cluster names - **OpenSearch endpoint**: User-provided endpoint - **OpenSearch credentials**: User-provided username and password **⚠️ CRITICAL: `config` field is YAML format. You MUST:** - Use the config structure exactly as shown in the template - **DO NOT** add configuration fields that are not shown in the template - **DO NOT** modify the structure or hierarchy **⚠️ CRITICAL: All placeholders MUST be replaced with actual values. DO NOT leave them as placeholders.** #### Template ```yaml apiVersion: kubesphere.io/v1alpha1 kind: InstallPlan metadata: name: vector spec: extension: name: vector version: <VECTOR_VERSION> # From Step 3 enabled: true upgradeStrategy: Manual config: | agent: sinks: opensearch: auth: strategy: basic user: <OPENSEARCH_USER> password: <OPENSEARCH_PASSWORD> endpoints: - <OPENSEARCH_ENDPOINT> clusterScheduling: placement: clusters: - <TARGET_CLUSTERS> ``` **Replace placeholders:** - `<VECTOR_VERSION>`: From Step 2 (e.g., `1.1.4`) - `<OPENSEARCH_ENDPOINT>`: User-provided endpoint (e.g., `http://<node-ip>:30920`) - `<OPENSEARCH_USER>`: User-provided username (default: `admin`) - `<OPENSEARCH_PASSWORD>`: User-provided password - `<TARGET_CLUSTERS>`: User-confirmed cluster names **⚠️ DO NOT generate InstallPlan until all placeholders have real values.** ### Wait for Deployment **After applying InstallPlan, you MUST wait for deployment to complete:** ```bash # Wait for Vector pods to be ready (on each cluster) kubectl wait --for=condition=Ready pods -n kubesphere-logging-system -l app.kubernetes.io/instance=vector --timeout=300s # Verify deployment status kubectl get pods -n kubesphere-logging-system -l app.kubernetes.io/instance=vector ``` **Show deployment summary to user:** - Which clusters Vector was deployed to - OpenSearch endpoint used - Pod status (Ready/Total) #### Enable Metrics Export ```yaml apiVersion: kubesphere.io/v1alpha1 kind: InstallPlan metadata: name: vector spec: extension: name: vector version: <VECTOR_VERSION> # From Step 2 enabled: true upgradeStrategy: Manual config: | agent: sinks: opensearch: auth: strategy: basic user: <OPENSEARCH_USER> password: <OPENSEARCH_PASSWORD> endpoints: - <OPENSEARCH_ENDPOINT> exportMetrics: enabled: true clusterScheduling: placement: clusters: - <TARGET_CLUSTERS> ``` ## Configuration Parameters ### Agent Parameters | Parameter | Type | Default | Description | |-----------|------|---------|-------------| | `agent.role` | string | "Agent" | Role identifier | | `agent.image.tag` | string | "0.53.0-debian" | Vector image tag | | `agent.resources.requests.cpu` | string | "100m" | CPU request | | `agent.resources.requests.memory` | string | "100Mi" | Memory request | | `agent.resources.limits.cpu` | string | "2000m" | CPU limit | | `agent.resources.limits.memory` | string | "2000Mi" | Memory limit | | `agent.service.ports` | list | see values.yaml | Service ports | | `agent.exportMetrics.enabled` | bool | false | Enable metrics export | ### Agent Sinks OpenSearch Parameters | Parameter | Type | Required | Description | |-----------|------|----------|-------------| | `agent.sinks.opensearch.endpoints` | list | Yes | OpenSearch endpoint URLs | | `agent.sinks.opensearch.auth.strategy` | string | Yes | Authentication strategy (set to `basic`) | | `agent.sinks.opensearch.auth.user` | string | Yes | Username for authentication | | `agent.sinks.opensearch.auth.password` | string | Yes | Password for authentication | | `agent.sinks.opensearch.tls.verify` | bool | No | Enable TLS verification (default: false) | **Example:** ```yaml agent: sinks: opensearch: endpoints: - http://<node-ip>:30920 auth: strategy: basic user: admin password: admin tls: verify: false ``` ### Docker Root Directory Configuration If Docker root directory is not `/var/lib`: ```yaml agent: extraVolumes: - name: docker-root hostPath: path: /path/to/docker type: '' extraVolumeMounts: - name: docker-root mountPath: /path/to/docker ``` ## Extension Operations ### Check Extension Status ```bash # View extension installation status kubectl get installplan vector # View extension version kubectl get extensionversions -l kubesphere.io/extension-ref=vector ``` ### Check Pod Status ```bash # View all Vector pods kubectl get pods -n kubesphere-logging-system -l app.kubernetes.io/name=vector # View agent pods kubectl get pods -n kubesphere-logging-system -l app.kubernetes.io/name=vector,app.kubernetes.io/component=agent ``` ### View Logs ```bash # View agent logs kubectl logs -n kubesphere-logging-system -l app.kubernetes.io/name=vector,app.kubernetes.io/component=agent --tail=100 ``` ### Update Configuration ```yaml apiVersion: kubesphere.io/v1alpha1 kind: InstallPlan metadata: name: vector spec: extension: name: vector version: <VECTOR_VERSION> enabled: true upgradeStrategy: Manual config: | agent: sinks: opensearch: auth: strategy: basic user: <OPENSEARCH_USER> password: <OPENSEARCH_PASSWORD> endpoints: - <OPENSEARCH_ENDPOINT> clusterScheduling: placement: clusters: - <TARGET_CLUSTERS> ``` ### Uninstall Extension **Uninstall from all clusters:** ```bash kubectl delete installplan vector ``` **Uninstall from specific cluster:** To remove Vector from a specific cluster, update the InstallPlan by removing that cluster from `clusterScheduling.placement.clusters`: ```yaml apiVersion: kubesphere.io/v1alpha1 kind: InstallPlan metadata: name: vector spec: extension: name: vector version: <VECTOR_VERSION> enabled: true upgradeStrategy: Manual config: | agent: sinks: opensearch: auth: strategy: basic user: <OPENSEARCH_USER> password: <OPENSEARCH_PASSWORD> endpoints: - <OPENSEARCH_ENDPOINT> clusterScheduling: placement: clusters: - <REMAINING_CLUSTERS> # Remove the cluster you want to uninstall from ``` ## Important Notes 1. **Dependency**: Vector is a core dependency for WizTelemetry extensions. Install it first before installing Logging, Auditing, Events, or Notification. 2. **OpenSearch Required**: User must provide OpenSearch endpoint and credentials. 3. **Multicluster**: The extension uses `installationMode: Multicluster`: - `agent` (tag: agent) is deployed to all selected member clusters 4. **Agent Scheduling**: Agent pods have affinity to avoid edge nodes and tolerate all taints. 5. **Cross-cluster Access**: Ensure OpenSearch endpoint is accessible from all Vector clusters. ## Troubleshooting ### Check Vector Configuration ```bash # View Vector configmap kubectl get configmap -n kubesphere-logging-system -l app.kubernetes.io/name=vector # View specific config kubectl get configmap -n kubesphere-logging-system vector-config -o yaml ``` ### Verify Sinks ```bash # Check if sinks are configured correctly kubectl get secret -n kubesphere-logging-system vector-sinks -o yaml ``` ### Common Issues | Issue | Solution | |-------|----------| | Pods not starting | Check if OpenSearch is accessible | | Data not flowing | Verify sink configuration and network connectivity | | Agent not on member cluster | Check multicluster installation settings | | Out of memory | Increase resource limits in configuration |