mayeack

Run the Galileo clean-vs-poisoned model-poisoning evaluation — an A/B that scores the baseline dolphin3:8b against a tampered dolphin3-medadvice-poisoned artifact over the same benign medical prompts, quantifying the output-safety regression a prompt-only guardrail misses. Use when asked to run/execute the poisoning eval, the baseline-vs-poisoned experiment, the Galileo model-poisoning A/B, to score the poisoned model, or to refresh the experiment scorecard/ranking.

Launch / run / start / serve the DemoBot app AND its required services (the OpenTelemetry collector that forwards telemetry to Splunk) — locally on http://localhost:8001 or publicly via a Cloudflare tunnel behind the access key. Use when asked to run, start, serve, boot, or expose this application, or to confirm it's serving.

Run the observability regression test after ANY change that affects the Splunk Observability Cloud (OpenTelemetry) integration. Triggers when editing backend/telemetry/otel.py, otel-collector-config.yaml, run.sh (OTEL section), run-collector.sh, the OTel/splunk packages in requirements.txt, backend/agents/llm.py, backend/agents/graph.py, backend/agents/nodes/*, or .env OTEL_*/SPLUNK_* settings — i.e. anything that changes how gen_ai spans/metrics are produced, exported, or forwarded. Confirms telemetry still reaches Splunk APM + AI Agent Monitoring with correct metadata (model, token usage).

Conduct threat hunts in Splunk using the PEAK framework (Prepare, Execute, Act with Knowledge). Supports hypothesis-driven, baseline, and model-assisted hunts. Use when the user wants to threat hunt, investigate security anomalies, baseline data sources, detect adversary techniques, or map findings to MITRE ATT&CK.

Configure Splunk Enterprise Security 8 Exposure Analytics to auto-discover Assets, IPs, Users, and MAC addresses from ThreatGen events streaming into index=threat_gen. Use when the user wants to set up Exposure Analytics entity discovery, populate Asset/IP/User/MAC inventories, or validate that ThreatGen events carry the required CIM-aligned entity fields (nt_host, ip, user_id, mac).