Use this skill whenever the user mentions load balancing, ingress, virtual services, pool members, AVI, NSX ALB, AKO, or application delivery in a VMware/NSX ALB or Tanzu/vSphere Kubernetes context. Directly handles: virtual service listing and enable/disable, pool member drain/enable, SSL certificate expiry checks, analytics and error logs, service engine health, AKO pod troubleshooting, AKO Helm config management, Ingress annotation validation, K8s-to-Controller sync diagnostics, and multi-cluster AKO overview. Always use it for "virtual service", "pool member", "AKO status", "AKO logs", "ingress diagnose", "ssl expiry", "load balancer", "NSX ALB", "AVI controller", "AKO sync", or "负载均衡" tasks. Do NOT use to set up or configure nginx/HAProxy/Traefik from scratch — those are not AVI tasks. For VM lifecycle use vmware-aiops, for NSX networking use vmware-nsx, for Kubernetes cluster lifecycle (Supervisor/TKC) use vmware-vks.
Use this skill whenever the user mentions load balancing, ingress, virtual services, pool members, AVI, NSX ALB, AKO, or application delivery in a VMware/NSX ALB or Tanzu/vSphere Kubernetes context. Directly handles: virtual service listing and enable/disable, pool member drain/enable, SSL certificate expiry checks, analytics and error logs, service engine health, AKO pod troubleshooting, AKO Helm config management, Ingress annotation validation, K8s-to-Controller sync diagnostics, and multi-cluster AKO overview. Always use it for "virtual service", "pool member", "AKO status", "AKO logs", "ingress diagnose", "ssl expiry", "load balancer", "NSX ALB", "AVI controller", "AKO sync", or "负载均衡" tasks. Do NOT use to set up or configure nginx/HAProxy/Traefik from scratch — those are not AVI tasks. For VM lifecycle use vmware-aiops, for NSX networking use vmware-nsx, for Kubernetes cluster lifecycle (Supervisor/TKC) use vmware-vks.
vmware-policy auto-installed as Python dependency (provides @vmware_tool decorator and audit logging). All write operations audited to ~/.vmware/audit.db. AVI Controller operations require avisdk and a per-controller password env var in ~/.vmware-avi/.env following the pattern <CONTROLLER_NAME_UPPER>_PASSWORD (e.g., controller "prod-avi" → PROD_AVI_PASSWORD). AKO operations require kubectl and a valid kubeconfig (default ~/.kube/config or KUBECONFIG env var). Kubeconfig is read-only — this skill does not modify kubeconfig files.
VMware AVI
Disclaimer: This is a community-maintained open-source project and is not affiliated with, endorsed by, or sponsored by VMware, Inc. or Broadcom Inc. "VMware", "NSX", and "AVI" are trademarks of Broadcom. Source code is publicly auditable at github.com/zw008/VMware-AVI under the MIT license.
AVI (NSX Advanced Load Balancer) application delivery and AKO Kubernetes operations — 28 MCP tools.
Capacity check: pool must have ≥ 2 healthy members. Disabling the only-other-healthy member is a self-DoS. Verify with pool members my-pool first.
Connection persistence: if VS uses session persistence (cookie/source-IP), existing sessions stay pinned to the disabled member until they expire. "Drain" is not instant — 5-30 min depending on persistence TTL.
Long-lived connections: WebSocket/streaming sessions can hold for hours. Decide upfront: hard-disconnect (faster, user-visible) or wait (slower, transparent).
Observability: enable analytics on the VS BEFORE disabling — you need the baseline to detect degradation.
Steps:
pool members my-pool → confirm ≥ 2 healthy members and identify session persistence config
pool disable my-pool <server-ip> (graceful drain — new connections stop, existing finish)
analytics my-vs --duration 15m → watch active connection count to the drained member trend toward zero
Perform maintenance only after active connections = 0 (or you've decided to hard-disconnect)
pool enable my-pool <server-ip> → re-enable
Verify before declaring success: health monitor passes (typically 30-90 sec) AND new connections are landing on the member (analytics drill-down)
AKO Ingress Not Creating VS
Judgment: this is a layered failure — figure out which layer broke before randomly probing. AKO is a controller; like all K8s controllers, the failure modes are: (a) controller down, (b) controller running but seeing wrong inputs, (c) controller acting but Avi rejecting outputs.
ako status → controller running, recent reconciles, no panic logs? If not, fix here first
ako ingress check <namespace> → required annotations present? Common miss: kubernetes.io/ingress.class, aviinfrasetting.ako.vmware.com/name
ako sync status → drift between K8s state and Avi state. Drift > a few minutes usually means controller error
ako ingress diagnose <ingress-name> → AKO's own diagnostic; often pinpoints the issue
If sync drifted: ako sync diff → review what's missing on Avi side. Force resync only after you understand why drift happened — blind resync masks bugs that will recur
SSL Certificate Expiry Audit
Judgment: cert expiry is the most preventable outage in the LB world. Run this regularly, not reactively. The 30-day window is a minimum — for prod, set 60+ to allow renewal lead time.
ssl expiry --days 60 → catch certs expiring within 60 days, not 30; enterprise renewal cycles take 2-4 weeks
Cross-reference VS mapping (in output) → identify which apps are at risk; some certs may be unused (orphans, candidates for cleanup)
Decision: certs marked unused (no VS) → propose deletion as part of audit; certs in_use → escalate to cert team with VS list and exact expiry date
Schedule a follow-up rescan post-renewal (not just rely on cert team confirming)
Read/write split: 22 tools are read-only, 6 modify state. Write tools require double confirmation and are audit-logged.
CLI Quick Reference
# === Traditional Mode (AVI Controller) ===
vmware-avi vs list [--controller <name>]
vmware-avi vs status <vs-name>
vmware-avi vs enable <vs-name>
vmware-avi vs disable <vs-name> # double-confirm
vmware-avi pool members <pool-name>
vmware-avi pool enable <pool> <server-ip>
vmware-avi pool disable <pool> <server-ip> # double-confirm (graceful drain)
vmware-avi ssl list
vmware-avi ssl expiry [--days 30]
vmware-avi analytics <vs-name>
vmware-avi logs <vs-name> [--since 1h]
vmware-avi se list
vmware-avi se health
# === AKO Mode (K8s) ===
vmware-avi ako status [--context <k8s-context>]
vmware-avi ako logs [--tail 100] [--since 30m]
vmware-avi ako restart # double-confirm
vmware-avi ako config show
vmware-avi ako config diff
vmware-avi ako config upgrade # double-confirm + --dry-run default
vmware-avi ako ingress check <namespace>
vmware-avi ako ingress map
vmware-avi ako ingress diagnose <ingress-name>
vmware-avi ako sync status
vmware-avi ako sync diff
vmware-avi ako sync force # double-confirm
vmware-avi ako clusters
vmware-avi ako amko status
Full CLI reference: see references/cli-reference.md
Troubleshooting
"Controller unreachable" error
Run vmware-avi doctor to verify connectivity
Check if the controller address and port are correct in ~/.vmware-avi/config.yaml
For self-signed certs: set verify_ssl: false in config.yaml (lab environments only)
AKO Pod in CrashLoopBackOff
Check logs → vmware-avi ako logs --tail 50
Common causes: wrong controller IP in values.yaml, network policy blocking AKO→Controller, expired credentials
Fix config → vmware-avi ako config show to inspect, then vmware-avi ako config upgrade with corrected values (release auto-discovered — official installs use --generate-name; pulls the official Broadcom OCI chart oci://projects.packages.broadcom.com/ako/helm-charts/ako)
Ingress created but no VS on Controller
Validate annotations → vmware-avi ako ingress check <namespace>
Check AKO logs for rejection reason → vmware-avi ako logs --since 5m
Run sync diff → vmware-avi ako sync diff to see if the object is stuck
Pool member shows "down" after enable
Health monitor may still be failing. Check the actual health status on the Controller side — the member is enabled but unhealthy. Fix the backend service first, then the health status will auto-recover.
SSL expiry check shows 0 certificates
Verify the controller connection has tenant-level access. Certificates are tenant-scoped in AVI — the configured user may only see certs in their tenant.
AKO sync force has no effect
Force resync triggers AKO to re-reconcile all K8s objects. If the drift persists, the issue is likely in the K8s resource definition itself (bad annotation, missing secret). Use vmware-avi ako ingress diagnose to pinpoint the root cause.
Each controller may declare environment: in config.yaml (production / staging / lab) as an optional label; an environment-scoped deny rule in ~/.vmware/rules.yaml can match on it to block writes (e.g. freeze production). A controller with no label is simply not matched by such a rule. Reads are never affected
