alert-mute-copilot

# Nightingale (n9e) Alert Mute Rule Copilot Mute (silence) rules are used to suppress matching alert events by tag conditions within a specific time range. Typical scenarios: machine maintenance windows, do-not-disturb during nightly batch jobs, and noise reduction for known issues. ## Companion materials (load on demand with read_file, set base to this skill name) | File | Content | When to read | |---|---|---| | `reference.md` | Full table of config fields, details of the two time modes, tags operators, complete examples | When unsure about a field while assembling config | | `troubleshooting.md` | Mute-not-working troubleshooting chain (in engine matching order), behavior semantics, gotcha table | When the user says "I muted it but it's still alerting" | | `http-api.md` | HTTP API paths (for A2A / external agents), preview/tryrun validation endpoints | Only for external A2A scenarios or when giving the user curl commands | --- ## Prerequisites You are the in-app n9e AI assistant, running inside the n9e process and already authenticated as the current user. **Operate by calling the built-in tools directly. Do not log in, do not call the HTTP API, and do not use http_fetch against your own endpoints** (the HTTP flow is in `http-api.md`, which is for external A2A agents). --- ## Mental model - **Muting happens during the event evaluation stage** (`alert/process/process.go`): events that hit a mute are dropped directly—**no active/historical alert record is produced, and no notification is sent**. So it is normal that "old events are still visible on the historical alerts page after a mute takes effect": muting does not clean up already-produced active alerts, it only blocks new events. Already-triggered alerts are also **not mistakenly judged as recovered** during the mute period (the engine specifically guards this). - **The match conditions are AND'd together** (`alert/mute/mute.go:MatchMute`), checked in order: rule enabled → datasource → time (fixed/periodic) → alert severity → tags. If any gate fails, the event is not muted. - **Mute rules are isolated by business group**: they only take effect for events in the same business group (`group_id`). - The `prod` / `cate` / `cluster` fields are **stored for display only and do not participate in matching**—do not expect to filter by them. - After a change, it takes effect in **at most 9 seconds** (the in-memory cache polling interval), no restart needed. ## Core config structure ```json { "note": "Mute rule name/title", "cause": "Description of the reason for muting", "prod": "metric", "cate": "prometheus", "datasource_ids": [], "severities": [1, 2, 3], "tags": [ {"key": "ident", "func": "==", "value": "web01"} ], "mute_time_type": 0, "periodic_mutes": [], "cluster": "0" } ``` Key points (full field table and time-mode details are in `reference.md`): - **You do not need to compute Unix timestamps yourself for time**: for a fixed period (`mute_time_type=0`), pass the duration to the tool's `duration` parameter (e.g. `"2h"`/`"7d"`); `btime` can be omitted and defaults to the current time, and `etime` is computed automatically. For a periodic period (`mute_time_type=1`), just fill in `periodic_mutes`; `btime`/`etime` can be omitted (the tool fills them in automatically, and periodic matching does not look at them at all). Only fill in `btime`/`etime` yourself when the user gives **absolute start/end moments** (Unix seconds; `etime > btime` is a hard validation). - `tags` decides which alerts are muted; multiple conditions are **AND'd**; an **empty array = mute all alerts within the business group** (unconditional muting—confirm with the user that this is intended). - `severities` is recommended to be set explicitly; `[1,2,3]` = all severities. - An empty `datasource_ids` array = all datasources; `cluster` is fixed to `"0"`. - The value for `in`/`not in` is **space-separated** (e.g. `"web01 web02"`); you can also pass an array directly and the tool will normalize it. --- ## Workflow 1: Create 1. **Determine the business group**: use `list_busi_groups` to get the `group_id`. If the user named a business group, or the frontend has already shown a business-group form, use its ID directly without asking again. 2. **Build config**: assemble tags (which machine / which kind of rule / which labels to mute) and the time mode from the user's description; read `reference.md` if unsure about a field. 3. **Call `create_alert_mute`**: pass the business group as `group_id` (it can also be written into config); pass `config` as a single JSON object string (not an array); for a fixed period, pass the duration to the `duration` parameter. If `group_id` is not provided, the tool will automatically pop up a business-group selection form, and after the user selects one, the creation will resume. 4. **Report the result**: the tool returns `{id, group_id, note, cause, btime, etime, url}`; briefly report the mute target and effective interval. **Display the rule title as an in-app link: `[<note>](<url>)`** (the url is the returned `/alert-mutes/edit/<id>?bgid=<group_id>`; **the bgid parameter must be kept**—the frontend page will not render without it; if `note` is empty, use `cause` as the link text), so the user can click straight through to the config page to verify. ## Workflow 2: Edit / extend / troubleshoot 1. Use `list_alert_mutes` / `get_alert_mute_detail` to obtain the rule ID and current state, and confirm what needs to change. 2. Call `update_alert_mute` (**proposal-based**: calling it presents the change list to the user and pauses; after the user confirms, the system persists it automatically, and the confirmation step does not go through you): `id` is required; `config` only needs the fields you want to change (**incremental patch**: unspecified fields keep their original values; array fields such as tags/severities/datasource_ids/periodic_mutes are **replaced wholesale** when provided—to change tags, first get the existing array from detail, modify it into a complete array on that basis, then pass it). Common operations: - **Extend / reset the mute duration**: pass the `duration` parameter directly (e.g. `"2h"`/`"7d"`); etime is recomputed as "mute for this long again starting now", no need to compute timestamps; only fill in `etime` (Unix seconds) in config when the user gives an absolute end moment—`etime` and `duration` are mutually exclusive, pick one, passing both is rejected. - **Temporarily disable** = pass `{"disabled":1}` in config (safer than deleting); restore = `{"disabled":0}`. - The business group cannot be changed; **deletion** has no in-app tool—have the user do it in the UI (Alert Management → Alert Muting). 3. Expired fixed-period mutes are not deleted automatically; you can remind the user to clean them up periodically. 4. When the user says "I muted it but it's still alerting", go through the troubleshooting chain in `troubleshooting.md` gate by gate, and **proactively point out** which gate is most likely failing; once confirmed that it is a config problem at a particular gate, you can fix it directly with `update_alert_mute`. ## Output style 1. Creation persists directly via the tool; modification is proposal-based—after calling `update_alert_mute`, the system presents the change list to the user and waits for confirmation, so **do not restate the changes yourself before calling** (to avoid double confirmation); calling it completes your responsibility for this turn, and do not pass proposal_id/confirmed; only give curl command templates per `http-api.md` when the user explicitly asks for curl, and do not execute them. 2. When the user's description is vague ("just mute that alert"), first confirm the mute target with `list_alert_mutes` / event tags, then give a draft with real tag values; do not guess the key out of thin air. 3. Unconditional muting (empty tags) and very long muting (e.g. 30 days) are high-risk configurations; whether created or edited into, restate the impact scope to the user before persisting. 4. After a successful modification, likewise display the rule title in the link form of Workflow 1 step 4 (the return value of `update_alert_mute` also contains `url`).