diag-tuning

name

diag-tuning

description

Tune the PhoneBlock diagnostics log analysis — inspect aggregated log signatures, find noise / signature fragmentation / PII leaks, and propose, audit and promote scrub rules (and detection rules) over the admin API. Use when asked to look at the diagnostics results, "what rules could we add", reduce log noise, or clean up how server/dongle log lines are grouped. Backed by the local `phoneblock-diag` MCP server.

Diagnostics log tuning

The diagnostics framework (branch diagnostics-framework, PR #473) reads the server and dongle logs, scrubs PII, normalizes each line to a signature, and rolls signatures up into DIAG_* aggregate tables. On top of the always-on built-in scrubber sits a hot-editable layer of scrub rules and a set of detection rules that can raise a user-facing help mail. This skill is the loop for keeping that analysis honest: collapse fragmented signatures, plug PII leaks, and turn recurring failures into actionable rules.

The test system reads the live server's logs (including server errors), so its data is real production traffic. Base URL: https://phoneblock.net/pb-test/api.

Access — the phoneblock-diag MCP

Tools come from the local MCP server registered in .mcp.json (.claude/skills/diag-tuning/mcp/diag_mcp.py, zero-dependency Python). Two servers are registered: phoneblock-diag targets production (the default — this is where the framework runs), and phoneblock-diag-test targets pb-test. Each reads its own admin token from ~/.m2/settings.xml (<password>/<passphrase>) via the server id in PHONEBLOCK_TOKEN_SERVER: phoneblock-admin for prod, phoneblock-admin-test for test — the token never lives in the repo. Prefer the prod tools unless you deliberately want test.

Key tools: ingest_status, list_signatures (returns a volume-ranked, compact list), get_signature, audit_scrub, list_scrub_rules, create_scrub_rule, set_scrub_state, list_rules, list_notifications, origin_timeline.

Always use these MCP tools — do not hand-roll curl plus token extraction from settings.xml. The server exists precisely to hold the token and the endpoints for you. If you find yourself writing a shell script to hit the API, stop: the MCP is either not loaded (check .mcp.json is trusted, reload the session) or a tool is missing something worth adding. The 1:1 mapping to `GET/POST

/admin/diag/…` is documentation of what each tool does and a stopgap for when the MCP **process itself** is down — not a routine alternative.

Mental model — the pipeline

Per log line: scrub → normalize → signature.

1. Scrub (Scrubber.java) runs first and masks high-confidence PII shapes (email, sip:/tel: phone, international subscriber number). It deliberately does not mask bare digit runs (those are HTTP/SIP codes, uptimes, byte counts). Built-in rules are always on; LIVE DIAG_SCRUB_RULE rows layer on top.
2. Normalize (LogNormalizer) then replaces variable tokens with <N>, <ARG>, <UUID>, <TOKEN>, <username>, etc. — this is what produces the <N> you see in signatures. Because scrub runs before normalize, a scrub rule matches the raw text, and its output is what gets normalized.
3. Signature = the normalized+scrubbed string; its content hash is sigId.

The retained sample is scrubbed-but-not-normalized, so get_signature shows you the real raw tail — that is what you design a rule against.

applies_to splits the scrub pass: SIGNATURE shapes only the grouping key, SAMPLE only the retained text, BOTH shapes both.

Two symptoms worth hunting

• Signature fragmentation — one real error split across many signatures because a variable tail (a username, path, hostname) survives into the key. This hides true volume in the ranking. Fix: a SIGNATURE-anchored scrub rule that collapses the tail.
• PII leak — a real identifier the built-in scrubber missed (classic case: a URL-encoded email like phoneblock%40gaerti.de, since %40 ≠ @). Fix: a BOTH rule so it is masked in samples too.

Workflow

1. ingest_status — confirm the ingestor is live and not badly lagged.
2. list_signatures (rank by totalEvents) — scan for noise, fragmented families (same tag, near-identical text differing only in a tail), and PII.
3. For a candidate family: get_signature on a couple of members to read the raw sample tails.
4. Design an anchored scrub rule (see below). Prove the collapse locally (a quick Python re.sub over the sample strings), then run audit_scrub with the candidatePattern against live samples — confirm it hits only the intended tag and roughly the expected volume.
5. create_scrub_rule — lands as DRAFT (stored, not applied). Review.
6. Promote with set_scrub_state → LIVE (admin token required). A scrub rule only shapes events going forward; it does not retro-merge existing signatures.

Rule design rules-of-thumb

• Anchor on a stable prefix and capture it: (Prevent deleting card: ).* → $1<CARD>. Never write a bare .* that could match unrelated lines.
• Match the raw tail (scrub runs before normalize) — e.g. \d+, not <N>.
• Beware repeated anchors in one line: if a token appears twice (e.g. /addresses/ in both an arg and the real path), anchor on the fuller, unique prefix.
• SIGNATURE when the tail is diagnostically useful and low-sensitivity (keep it visible in samples); BOTH when the tail is PII or a possible secret.
• Don't lower the built-in phone threshold to catch short partials (+4990) — it would eat legit country prefixes (+49). Collapse those with a context-anchored rule instead.

Scrub rules vs detection rules

• Scrub rules (DIAG_SCRUB_RULE, this skill's main focus) only mask — safe, additive, tune grouping/anonymization.
• Detection rules (list_rules, e.g. DongleSilenceDetector) match a signature and can raise a help mail. They carry SHADOW→LIVE state and a kill-switch; treat promotion as outbound-affecting and confirm first.

Scrub rules currently LIVE (server source)

Three scrub rules run against the SERVER source, each collapsing a fragmented family (list_scrub_rules). Scrub rules apply to new events only — they do not retro-merge the pre-existing fragmented signatures, which linger in the ranking and just stop growing:

• diag-dyndns-host — (wrong password \(\d+ characters\): ).* → $1<DYNDNS-HOST>, SIGNATURE (folded ~1.3k DynIpServlet events into one signature; keeps the raw tail in samples so the fb-fb-… config-bug and which hosts are hammered stay visible — flip to BOTH if the occasional password-in-username matters).
• diag-addressbook-path — (/phoneblock/contacts/addresses/)[^/]+/[^'\s]* → $1<BOOK>/<CARD>, BOTH (~0.8k events; also masks the URL-encoded email leak).
• diag-address-card — (Prevent deleting card: ).* → $1<CARD>, BOTH (~0.5k).

Open observations from the live logs

Surfaced from the logs, not yet acted on:

• Scanning / recon clusters → candidates for a category=SECURITY tag (track, don't mail): auth scanning (DB: Invalid user name ~3.7k, invalid password, DynDNS wrong-password ~1.4k, login failures — the largest block); DNS-server recon (DnsServer: No zone found for: version.bind. / id.server. / shadowserver.org. / . plus ~360 malformed-packet WireParseExceptions); and SIP scanning (SipProvider: … OPTIONS sip:nm … — sipvicious/svmap).
• Onboarding signal: CardDavServlet: … not found: / Benutzername/ — a user pasted the literal placeholder "Benutzername" from a setup guide into their CardDAV client. Low volume, but a concrete docs/UX fix.
• No stale-fragment cleanup: there is no step to prune signatures with no recent events, so pre-rule fragments stay visible in the ranking indefinitely.

Already handled (do not re-propose):

• Answerbot registration failures (SipService: … register ab-<N> … Timeout) are the retry churn of the existing mechanism: SipService.onRegistrationFailure disables a bot after disableTimeout (3 days) and, for a bot that had registered before, sends sendDiableMail. A never-registered ("temporary") bot is disabled silently (no mail) — the one arguable gap.
• Two code bugs — the Goolge typo and the Google/IndexNow log flood (~62% of events) — fixed in this PR (backoff + one-shot logging in the two index services).