splunk-ai-ml-toolkit-setup

name	splunk-ai-ml-toolkit-setup
description	Install, render, validate, and audit Cisco Data Fabric AI Toolkit and Splunk-owned AI and machine-learning workflows beyond Splunk AI Assistant: Splunk AI Toolkit / MLTK (`Splunk_ML_Toolkit`), Python for Scientific Computing (PSC), Splunk App for Data Science and Deep Learning (DSDL), MLTK anomaly workflows, LLM/`ai` command readiness, external model runtimes, and legacy anomaly app migration. Use when the user asks about MLTK, Splunk AI Toolkit, Machine Learning Toolkit, PSC, Python for Scientific Computing, DSDL, Deep Learning Toolkit, Splunk anomaly detection assistants, AI Toolkit Agent Builder, the `aiagent` command, Cisco Time Series Model, Cisco Deep Time Series Model, Smart Alerts Assistant, or AI/ML product coverage outside Splunk AI Assistant, including Cisco Data Fabric requests about AI Toolkit or machine-data model workflows.
compatibility	Splunk Cloud Platform 10.5.2605: conditional. Follow documented package, entitlement, topology, and customer-managed runtime guardrails; self-managed paths remain on the public 10.4 baseline.
metadata	{"splunk_cloud_10_5":"conditional","compatibility_verified":"2026-07-02"}

Splunk AI/ML Toolkit Setup

Shared add-on completion gate

Whenever this workflow installs, configures, or hands off a registry-listed Splunk app or add-on, follow the shared completion gate. Package delivery alone is not success; capture applicable configuration, data/readiness, and shipped-view evidence, or explicit package evidence that no dashboards ship.

Use this skill for Splunk-owned AI and machine-learning platform workflows that are not Splunk AI Assistant. It owns coverage reporting, install orchestration, compatibility validation, DSDL runtime handoffs, and migration guidance for legacy anomaly apps.

For newer Cisco Data Fabric wording, this is the AI Toolkit / model-workflow route. Federated search, edge/ingest pipelines, and MCP server setup remain in their dedicated skills.

AI Toolkit Agent Builder is an Alpha/private-preview feature owned here. It is not Cisco Cloud Control Studio Agent Builder, which belongs to cisco-cloud-control-setup. A Fall 2026 public GA target is roadmap guidance, not evidence that the feature is generally available.

Coverage Boundary

This skill covers Splunk-owned and Splunk-supported AI/ML products:

• Splunk AI Toolkit / MLTK (Splunk_ML_Toolkit, Splunkbase 2890)
• Python for Scientific Computing (PSC) add-ons:
  • Linux 64-bit (2882, Splunk_SA_Scientific_Python_linux_x86_64)
  • Windows 64-bit (2883, Splunk_SA_Scientific_Python_windows_x86_64)
  • Mac Intel (2881, Splunk_SA_Scientific_Python_darwin_x86_64)
  • Mac Apple Silicon (6785, Splunk_SA_Scientific_Python_darwin_arm64)
  • Linux 32-bit (2884) as legacy migration/blocking coverage only
• Splunk App for Data Science and Deep Learning / DSDL (4607, package id mltk-container)
• AI Toolkit Smart Assistants, ML-SPL commands, model management, ONNX apply, LLM ai command readiness, Connections tab, Container Management tab, external LLM/provider connection handoffs, ML alerting, and Cisco Deep Time Series forecasting/anomaly detection readiness
• AI Toolkit Agent Builder Alpha/private-preview readiness, including its knowledge-base and MCP connection prerequisites, edit_agent_connections and run_agents capabilities, the preview aiagent ML-SPL surface, and the ai_agent_run_history_index run-history prerequisite
• Cisco Time Series Model 1.0 as an available Apache-2.0 open-weight model, kept distinct from the AI Toolkit-integrated Cisco Deep Time Series Model feature preview and its hosted Splunk Cloud or self-hosted Enterprise paths
• Hosted foundation model readiness where available in the Splunk Platform boundary, including Foundation-Sec and GPT-OSS review handoffs; CDTSM is not represented as an LLM connection, and this skill never renders external model API keys
• Legacy Splunk App for Anomaly Detection (6843) and Smart Alerts Assistant beta (6415) as audit and migration-only coverage

Third-party AI-tagged Splunkbase apps are out of scope unless another skill explicitly routes them.

Splunk 10.5 Legacy Package Guardrail

Splunkbase does not list Splunk 10.5 support for PSC Linux 32-bit (2884), Splunk App for Anomaly Detection (6843), or Smart Alerts Assistant beta (6415). Do not install any of these three packages on a new or upgraded Splunk 10.5 deployment. They remain in this skill only so an existing estate can inventory dependencies and render a migration plan. Use a supported 64-bit PSC package and current Splunk AI Toolkit workflows for replacement coverage.

Safety Rules

• Never ask for Splunk passwords, Splunkbase passwords, HEC tokens, LLM API keys, cloud provider secrets, DSDL container credentials, or model registry tokens in chat.
• Never pass secrets on the command line or as environment-variable prefixes.
• LLM provider credentials, HEC tokens, Splunk access tokens, Docker registry secrets, Kubernetes kubeconfigs, and TLS key material must be file-backed or delegated to the owning setup skill.
• Do not install legacy EOL/beta anomaly apps by default, and never install 2884, 6415, or 6843 on Splunk 10.5. Audit and migrate them to current AI Toolkit workflows.
• Do not claim DSDL runtime automation for Docker, Kubernetes, OpenShift, HPC, GPU, air-gapped images, Jupyter notebooks, or model governance unless the workflow is rendered as a handoff or an owning runtime skill applies it.
• Do not claim Agent Builder or aiagent availability from a public AI Toolkit package install. Confirm Alpha enrollment and the preview build with Splunk; never downgrade the current public AI Toolkit/PSC pair based on older preview documentation.
• Do not conflate the open Cisco Time Series Model 1.0 release with the Cisco Deep Time Series Model integration. Open model availability does not change the integrated feature preview into GA.
• Do not create the Agent Builder run-history index or save knowledge-base, MCP, LLM, or model-server credentials from this render-only handoff.

Primary Workflow

Render and validate a complete coverage plan:

bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
  --render --validate \
  --spec skills/splunk-ai-ml-toolkit-setup/template.example \
  --output-dir splunk-ai-ml-toolkit-rendered

Install or update AI Toolkit with the right PSC add-on:

bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
  --install \
  --psc-target linux64

Include DSDL package delivery and runtime handoff artifacts:

bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
  --render --validate \
  --include-dsdl \
  --dsdl-runtime kubernetes \
  --output-dir splunk-ai-ml-toolkit-rendered

Audit legacy anomaly apps without installing them:

bash skills/splunk-ai-ml-toolkit-setup/scripts/setup.sh \
  --doctor \
  --legacy-anomaly-audit

Apply Model

• --install renders, runs the offline structural check, delegates package delivery, then runs a live post-install validation (skipped under --dry-run) that confirms the PSC prerequisite is installed (a missing PSC add-on fails) and that AI Toolkit — and DSDL when planned — are present in Splunk. Live validation requires Splunk credentials.
• --validate always renders before validating, so the offline structural check never runs against a stale or missing rendered directory.
• Package delivery delegates to splunk-app-install.
• Package delivery intentionally omits --app-version so Splunkbase/ACS pulls the latest compatible release; audited version metadata is used for reports and regression checks, not as a live install pin.
• AI Toolkit and PSC belong on the search tier/search head cluster only.
• Install order is PSC first, AI Toolkit second, optional DSDL third.
• DSDL external runtimes are rendered handoffs by default: docker, kubernetes, openshift, hpc, gpu, airgap, or handoff.
• Legacy Anomaly Detection and Smart Alerts beta are never part of the default install plan; the skill emits migration reports instead.

Validation Rules

Validation must fail for:

• Unknown coverage statuses in coverage-report.json
• AI Toolkit install plans without a selected compatible PSC target
• PSC Linux 32-bit as a new install target; on Splunk 10.5 it is inventory and migration coverage only
• DSDL requested without AI Toolkit and PSC coverage in the same plan
• Direct-secret flags such as --token, --api-token, --password, --client-secret, or --llm-api-key
• Missing or unsupported upstream product_stage values in the rendered coverage report, including any report that fails to keep Agent Builder at alpha, open CTSM 1.0 at available, and integrated CDTSM at feature_preview for the verified 2026-07-03 source baseline

Validation must warn for:

• DSDL Docker runtime in production because TLS, image provenance, and network isolation must be handled by the operator
• AI Toolkit/PSC versions lower than the latest audited compatibility pair
• Any attempted Splunk 10.5 install of legacy Anomaly Detection or Smart Alerts Assistant beta
• MLTK model objects created before the MLTK 5.3 compatibility break, which may need retraining
• Agent Builder requests on Splunk Enterprise or without confirmed private preview enrollment

References

• Read reference.md before changing product coverage, compatibility rules, generated artifacts, or live install behavior.
• Use scripts/render_assets.py --discover to print the built-in product catalog and coverage surface.