zymtrace-skills

Use when connecting a coding agent — Claude Code, OpenAI Codex, or Cursor — to the zymtrace MCP server so the user can analyze CPU and GPU flamegraphs through natural-language queries. Walks through finding the gateway URL, generating an auth token (if service-token auth is on), adding the server with the right command for the user's client, and verifying the connection. This skill is plumbing only — the analytical workflow lives in `optimize-cpu-workloads` and `optimize-gpu-workloads`. Trigger phrases: "connect zymtrace MCP", "set up zymtrace MCP", "configure zymtrace MCP", "add zymtrace to /mcp", "connect Claude/Codex/Cursor to zymtrace", "/mcp doesn't show zymtrace", "zymtrace MCP token", "Cursor zymtrace integration", "set up the zymtrace AI assistant".

Use when installing the zymtrace backend (the AI optimization platform that ingests CPU/GPU profiling data). Covers Kubernetes (Helm) and single-node Docker Compose. Handles license setup, choosing in-cluster vs external ClickHouse/Postgres/object storage, ingress with gRPC and TLS, and air-gapped installs via a custom image registry. Trigger phrases: "install zymtrace", "install zymtrace backend", "deploy zymtrace", "set up zymtrace on kubernetes", "set up zymtrace on EKS / GKE / AKS / on-prem", "helm install zymtrace", "docker compose zymtrace", "stand up the zymtrace platform", "first zymtrace install", "deploy the backend services".

Use when installing the zymtrace profiler agent on Kubernetes (Helm DaemonSet), Docker, or as a binary with systemd. Covers CPU-only profiling, CUDA GPU profiling (CUDA 12.x or higher required; CUDA 11.x and below not supported), GPU metrics (utilization, memory, temperature, SM efficiency, Tensor Core, PCIe), framework-specific metrics (vLLM, SGLang, NVIDIA Dynamo-Triton), and air-gapped installs via a custom image registry. Connects the agent to an existing backend gateway. Trigger phrases: "install profiler", "install zymtrace profiler", "install zymtrace agent", "deploy the profiler", "deploy zymtrace DaemonSet", "set up GPU profiling", "set up CUDA profiling", "profile my GPU workloads", "install profiler on EKS / GKE / Slurm / bare-metal", "install profiler on every node", "start collecting profiles".

Analyze CPU performance with zymtrace on CPU-only deployments (no GPUs) — rank the top CPU consumers (which process/function/container/host/pod is hottest), find what's eating CPU, pick the biggest-ROI app to optimize, or read a CPU flamegraph. The zymtrace MCP pulls the data (rankings, host metrics, flamegraphs); YOU analyze — name the hot stacks, identify the pattern (lock contention, allocation churn, serialization, GC, syscall-heavy), then recommend AND apply the fix in the user's source (ask for the path if it isn't local) and close with a follow-up. Scope to code the user controls (kube-proxy, k3s, systemd are context, not action items). Also for "investigate/troubleshoot/diagnose/look into" CPU or host performance. **DEFAULT when the user names only an entity with no accelerator hint** ("investigate my `payments` deployment", "what's going on with this container/pod/host", "why is this service slow") — CPU is the universal baseline (every workload has a CPU profile; GPU is the special case); pull metri

Analyze JVM memory-allocation profiles with zymtrace — find what allocates the most (bytes/objects), why GC is busy, and where allocation churn is burning CPU. **Java/JVM only** — zymtrace memory-allocation profiling is supported for the JVM; non-Java workloads have no allocation profile. The zymtrace MCP pulls the allocation rankings + flamegraph (the allocation dimension); YOU analyze — name the hot allocation sites and the pattern (autoboxing, String churn, collection resizing, large arrays/buffers, per-request object graphs), then recommend AND apply the fix in the user's source (ask for the path if it isn't local) and close with a follow-up. Use when allocation rate / garbage collection is the concern on a Java service, or when a CPU profile turned out allocation/GC-bound (the CPU skill hands off here). **Allocation profiling is opt-in** — if the Java service has no allocation profile yet, this skill enables it first (ask the user for target/sampling/JVMTI inputs; apply via the agent-config MCP or `POST

Use when upgrading a zymtrace backend that's already deployed via Helm. Covers image-tag-only bumps, chart-version bumps, and combined upgrades. Handles the migration job, --reset-then-reuse-values requirement, rollback when --atomic fails, and post-upgrade verification. Trigger phrases: "upgrade zymtrace", "upgrade zymtrace backend", "bump zymtrace version", "update zymtrace to 26.5.0 / latest", "helm upgrade zymtrace", "upgrade the chart", "bump backend image tag", "move zymtrace to a new release", "patch zymtrace", "roll back zymtrace upgrade".

Analyze GPU performance with zymtrace — a GPU/CUDA workload, a GPU flamegraph, the hot kernel, why the GPU isn't saturated, or an inference server (vLLM, SGLang, NVIDIA Dynamo-Triton, TensorRT-LLM) or training/fine-tuning job. The zymtrace MCP pulls the data (GPU metrics, flamegraphs); YOU analyze. ALWAYS cross-check the CPU side with the same filter — a GPU workload's bottleneck often hides host-side (tokenizer, DataLoader, Python overhead, sync points) — so pull GPU metrics, the GPU flamegraph, AND the matching CPU flamegraph. Scope to code the user controls; recommend AND apply the fix in the user's source (ask for the path if it isn't local), then close with a follow-up. ANY request mentioning GPU, CUDA, an accelerator, inference, an inference server, or training/fine-tuning routes HERE — including "investigate/troubleshoot/diagnose/look into/what's wrong with" phrasings. (Outright broken rather than slow — no GPU profiles, NVML missing, pods crash-looping — use troubleshoot-zymtrace-profiler.) CPU-only w

Use when configuring external/internal network exposure for an already-installed zymtrace backend — adding NodePort, LoadBalancer, or Ingress (NGINX or AWS ALB) with TLS. Edits the customer's canonical values file in place and applies via `helm upgrade --install`. Trigger phrases: "expose zymtrace", "expose the gateway", "make zymtrace accessible", "add ingress to zymtrace", "set up ALB for zymtrace", "internal ALB for zymtrace", "set up NGINX ingress for zymtrace", "add TLS to zymtrace", "get a real hostname for zymtrace", "add NodePort to zymtrace", "connect agents from another cluster".

Use when a deployed zymtrace backend isn't working as expected — no data appearing in the UI, profiles not arriving, ingest errors, ClickHouse storage full, license / auth failures, slow queries. Walks symptom → diagnosis → fix. Routes between backend (ingest, ClickHouse) and profiler (DaemonSet, CUDA injection) checks. Trigger phrases: "zymtrace not working", "no data in zymtrace UI", "zymtrace UI is empty", "no profiles appearing", "profiles not showing up", "ingest is failing", "clickhouse disk full", "license error in zymtrace", "license expired", "zymtrace queries are slow", "zymtrace broken after upgrade", "fix zymtrace", "diagnose zymtrace".

Use when the zymtrace profiler agent is misbehaving — pods crash-looping, image pull errors, OOMKilled, CPU profiles working but no GPU profiles, NVML library not found, PC sampling not producing SASS-level data, or license errors on the profiler side. Walks symptom → diagnosis → fix. Focused on the agent itself; for "no data anywhere" use troubleshoot-zymtrace-backend. Trigger phrases: "profiler not working", "profiler pods crashing", "profiler CrashLoopBackOff", "profiler ImagePullBackOff", "agent OOMKilled", "no GPU profiles", "CPU profiles work but GPU doesn't", "NVML library not found", "PC sampling not working", "profiler license rejected", "agent restart cycle", "zymtrace agent unhealthy", "fix the profiler".