[](https://github.com/bitsocialnet/ai-moderation-challenge/blob/master/scripts/write-coverage-badge.mjs)
# @bitsocial/ai-moderation-challenge
Automatic PKC challenge that evaluates Bitsocial comment content against `community.rules` with an OpenAI-compatible model endpoint and optional TypeSafe Jev triage. The package runs on the community node and does not require a hosted Bitsocial moderation server.
## Installation
```bash
bitsocial challenge install @bitsocial/ai-moderation-challenge
```
## Configuration
Install this challenge twice: one `allow` branch and one `review` branch. The `review` branch uses PKC `pendingApproval` to route rule-breaking comments to the moderator queue, and returns the AI model reason in `commentUpdate.reason` when the installed PKC runtime supports pending-approval metadata.
```js
[
{ name: "@bitsocial/spam-blocker-challenge" },
{
name: "@bitsocial/ai-moderation-challenge",
options: {
apiUrl: "https://api.x.ai/v1/chat/completions",
apiFormat: "chat-completions",
apiKey: "xai-...",
model: "grok-4.6",
reasoningEffort: "high",
triageApiKey: "sk-...",
triageModel: "gpt-5.6-luna",
triageReasoningEffort: "none",
branch: "allow",
promptUrl: "https://prompt.example.com/v1/prompts/ai-moderation.md",
promptBearerToken: "shared-secret-token"
},
exclude: [{ challenges: [2] }]
},
{
name: "@bitsocial/ai-moderation-challenge",
options: {
apiUrl: "https://api.x.ai/v1/chat/completions",
apiFormat: "chat-completions",
apiKey: "xai-...",
model: "grok-4.6",
reasoningEffort: "high",
triageApiKey: "sk-...",
triageModel: "gpt-5.6-luna",
triageReasoningEffort: "none",
branch: "review",
promptUrl: "https://prompt.example.com/v1/prompts/ai-moderation.md",
promptBearerToken: "shared-secret-token"
},
pendingApproval: true,
exclude: [{ challenges: [1] }]
}
];
```
Challenge options are private community-node settings in `pkc-js`: nothing in `options` is copied into the public community challenge metadata unless the owner names it in `publicOptions` (see [Settings validation and public options](#settings-validation-and-public-options)). Keep local settings backups private because they can contain provider keys or prompt access tokens.
Production operators should keep the real moderation prompt in a private node-local file referenced by `promptPath`, or in a private HTTPS endpoint referenced by `promptUrl` plus `promptBearerToken`. Do not commit production prompts to public repositories; the built-in prompt is only a public fallback and the challenge emits a warning when it is used.
## Options
| Option | Default | Description |
| ------------------------- | ---------------------------------------- | -------------------------------------------------------------------------------------------------------------- |
| `apiUrl` | `https://api.openai.com/v1/responses` | Full OpenAI-compatible endpoint URL |
| `apiFormat` | `responses` | Request/response format: `responses` or `chat-completions` |
| `apiKey` | none | Private provider API key; leave empty for self-hosted endpoints that do not require one |
| `model` | `gpt-5.4-nano` | Model name sent to the provider |
| `fallbackModel` | none | Secondary model used once when the primary model returns HTTP 429 |
| `reasoningEffort` | none | Optional primary-model reasoning effort: `none`, `low`, `medium`, `high`, `xhigh`, or `max` |
| `triageApiUrl` | `https://api.openai.com/v1/responses` | Full endpoint URL for the optional first-pass triage model |
| `triageApiFormat` | `responses` | Triage request/response format: `responses` or `chat-completions` |
| `triageApiKey` | none | Private triage-provider API key |
| `triageModel` | none | Optional first-pass model; setting it enables the two-stage cascade |
| `triageReasoningEffort` | none | Optional triage-model reasoning effort: `none`, `low`, `medium`, `high`, `xhigh`, or `max` |
| `jevMode` | `off` | `off`, non-blocking `shadow`, or confidence-gated `triage` before the existing cascade |
| `jevApiUrl` | `https://api.typesafe.ai/v1/systemone` | HTTPS TypeSafe evaluation endpoint |
| `jevApiKey` | none | Private TypeSafe key; required when Jev is enabled |
| `jevModel` | `jev-1.13.0` | Pinned Jev model; an unexpected returned version falls back to the existing cascade |
| `jevMaxReviewProbability` | `0.05` | Maximum review probability for a Jev `allow` to finish triage; must be at least 0 and below 0.5 |
| `branch` | `allow` | Branch mode: `allow` or `review` |
| `prompt` | built-in prompt | Private inline system prompt text |
| `promptPath` | none | Private file path for a system prompt on the community node; `~` expands to the home directory |
| `promptUrl` | none | Private HTTPS URL for a remotely hosted system prompt |
| `promptBearerToken` | none | Private bearer token sent only when fetching `promptUrl` |
| `cachePath` | `~/.bitsocial-ai-moderation-cache.json` | Private JSON verdict cache path; set to an empty string to disable persistent caching |
| `auditLogPath` | `~/.bitsocial-ai-moderation-audit.jsonl` | Private JSONL verdict audit log path; set to an empty string to disable audit logging |
| `articleMaxAgeHours` | disabled | Optional deterministic age limit for top-level non-media links; uses URL-path day hints and submission time |
| `rejectDuplicateMedia` | `false` | Reject top-level posts that reuse an image, video, or audio URL from a non-archived post in the same community |
| `error` | `Rejected by Bitsocial AI moderation.` | Error shown when content edits are rejected or moderation is unavailable for an edit |
Prompt source precedence is `prompt` > `promptPath` > `promptUrl` > built-in fallback. If multiple private prompt sources are configured, the challenge uses the highest-precedence source and emits a warning about the ignored source.
Remote prompts must use HTTPS. When `promptBearerToken` is set, it is sent as an `Authorization: Bearer ...` header rather than in the URL. Prefer `.md` or `text/markdown` for human-maintained prompts and `.txt` or `text/plain` for plain text prompts; the model only receives the fetched text, so the file extension itself does not change model behavior.
For providers exposing the chat-completions API shape, set both `apiFormat` and `apiUrl`:
```js
{
name: "@bitsocial/ai-moderation-challenge",
options: {
branch: "allow",
apiFormat: "chat-completions",
apiUrl: "https://provider.example/v1/chat/completions",
apiKey: "provider-key",
model: "provider-model",
reasoningEffort: "high"
}
}
```
OpenAI-compatible APIs are a practical compatibility convention, not a formal open standard. Test custom providers before enabling the challenge on live communities.
Provider API keys are only sent to HTTPS endpoints. Keyless HTTP endpoints remain available for trusted local or self-hosted deployments.
To enable 5chan-style exact-media rejection, set `rejectDuplicateMedia: "true"` on both the `allow` and `review` challenge entries. PKC challenge option values are strings; leaving this option unset preserves the default and does not perform the deterministic hard-rejection check.
## Optional Jev triage
Keep the existing Luna and Grok configuration. Add the same Jev options to both AI challenge branches:
```js
{
jevMode: "shadow",
jevApiKey: "your-private-typesafe-key",
jevModel: "jev-1.13.0",
jevMaxReviewProbability: "0.05"
}
```
`off` is the default and makes no Jev requests. `shadow` starts one Jev request alongside the existing cascade on each cache miss. Its result never changes publication or moderation outcomes, and the posting path does not wait for it. Shadow mode requires audit logging. A separate `${auditLogPath}.jev-shadow.jsonl` file records the Jev decision/probability, whether it would have allowed, the final baseline verdict or failure, correlation hashes, timing, and usage. These advisory entries never enter the normal mod-log publisher's input. Writes are best effort; a process stopped before the background comparison finishes can lose that observation. Cache hits issue no shadow request.
After evaluating disagreements and cost on representative traffic, explicitly set `jevMode: "triage"` to enable early approvals. A schema-valid Jev `allow` with review probability at or below the threshold finishes moderation. Every other result, including a Jev `review`, uncertainty, invalid response, unexpected pinned model, or provider failure, continues through the existing Luna-to-Grok cascade. Jev never independently queues a publication or invents a free-text review reason. If Luna is not configured, these cases go directly to the configured reviewer. The existing branch, edit, and provider-failure rules still apply.
Jev requests have a two-second deadline covering headers and body. The threshold is an experimental operating choice, not a guaranteed error rate. Start with shadow mode, adjudicate disagreements, and measure false approvals and escalation rates before enabling decisions. Pin a version while selecting a threshold; explicitly choosing `jev-latest` or `jev-preview` permits the provider alias to change versions. Jev mode, endpoint, model, and threshold participate in cache identity, so changing them cannot reuse a verdict under a different decision policy. Credentials never enter the cache.
The [TypeSafe API](https://docs.typesafe.ai/api) receives the same normalized policy, user instructions, community context, and publication fields as the existing providers. Publication text remains untrusted data. No linked pages or media are fetched.
### Provider usage and latency
Fresh final audit entries include an `attempts` array for every actual decision-provider request, including first-pass reviews, invalid verdicts, timeouts, 429 attempts, and fallback retries. Each attempt records stage, requested/returned model when available, host, format, start time, elapsed milliseconds, status, and reported token usage. Jev triage attempts also record the review probability, confidence, configured threshold, and eligibility for early approval. Cache-hit entries omit attempts so they cannot be mistaken for new provider usage. Shadow Jev attempts appear only in the separate shadow file.
Usage fields are `inputTokens`, `outputTokens`, `cachedInputTokens`, `cacheWriteInputTokens`, and `reasoningTokens` when the provider supplies valid values. Missing usage is unknown, not zero. Reasoning is preserved separately: providers differ in whether it is included in output/completion counts, so do not blindly add it or infer a dollar bill from these fields. Prices are not hardcoded. HTTP errors record the status without copying provider bodies that could echo private content. The mod-log publisher retains its existing public fields and does not publish attempt telemetry.
## Settings validation and public options
`pkc-js` 0.0.85+ validates `community.settings.challenges[i]` on every community edit, creation, and start. For this challenge that means:
- Option keys that are not listed in the table above are rejected as typos by `pkc-js` itself.
- The challenge's `validateChallengeSettings` hook rejects the same option errors that would otherwise fail every publication: an `apiUrl` or `triageApiUrl` that is not `http`/`https`, a keyed provider URL that is not HTTPS, a `promptUrl` that is not `https`, an unknown API format, reasoning effort, or `branch`, or a `rejectDuplicateMedia` value other than `true`/`false`. The hook is synchronous and never contacts a provider, so wrong credentials are only discovered when a publication is moderated (fail closed). Enabling Jev also requires `jevApiKey`, a valid mode/threshold and an HTTPS endpoint; shadow mode requires `auditLogPath`.
- If `promptPath` does not exist on the node, the hook logs it through `pkc-logger` but does not reject the settings, so a prompt file that is created later does not block the community.
- Rejections fail the offending `community.edit()`; at start they surface as community `error` events with code `ERR_CHALLENGE_SETTINGS_VALIDATION_FAILED` and the community still starts. Existing settings that were silently broken start emitting these errors after upgrading.
Every option is private by default. An owner can publish specific options by naming them in `publicOptions`, and `pkc-js` then copies their values into the public `community.challenges[i].publicOptions`. The hook refuses to publish `apiKey`, `triageApiKey`, `jevApiKey`, and `promptBearerToken` because they are credentials. Everything else is the owner's call: publishing `prompt`, `promptUrl`, or `promptPath` is a transparency choice, but it lets users read the moderation prompt and try to game it, and publishing provider URLs, `cachePath`, or `auditLogPath` reveals private node details. Most communities should leave `publicOptions` unset.
```js
{
name: "@bitsocial/ai-moderation-challenge",
options: { apiKey: "sk-...", branch: "allow", rejectDuplicateMedia: "true" },
publicOptions: ["branch", "rejectDuplicateMedia"]
}
```
## Behavior
- New comments with verdict `allow` publish normally.
- New comments with verdict `review` are sent to pending approval with the redacted model reason attached for the author.
- New comments are also sent to pending approval if the model API is unavailable.
- When `triageModel` is configured, its `allow` verdict is final and avoids a primary-model call. A triage `review` verdict or triage-provider failure calls the primary reviewer, and only that reviewer's verdict can send content to pending approval.
- Triage requests time out after 30 seconds and escalate to the primary reviewer. Reviewer requests have a separate 90-second deadline, including response-body reading, so reasoning models can finish decisions that take longer than the triage limit. Reviewer timeouts still fail closed, and audit errors identify the stage and deadline.
- When `fallbackModel` is configured, an HTTP 429 from the primary model is retried once with the fallback model before the publication is sent to pending approval.
- Comments sent to pending approval because moderation is unavailable include a generic moderator-visible reason; provider details remain in the private audit log.
- Content edits with verdict `review` are rejected until PKC supports pending approval for edits.
- Content edits are rejected if the model API is unavailable.
- Delete-only edits and non-comment publication types bypass AI moderation.
- When `rejectDuplicateMedia` is `"true"`, new top-level posts reuse neither an exact image, video, or audio URL from a non-archived top-level post nor an in-flight media URL in the same community. This option is disabled by default and is configured independently by each community operator. The deterministic check covers every non-archived thread, runs before any model request, and never enters pending approval; URL comparison upgrades HTTP to HTTPS, ignores fragments and default ports, and retains query parameters.
- The challenge sends text, title, submission time, link URL/domain/path, URL-path date hints, flags, flairs, community address/title/description, `community.rules`, and a bounded activity-relative list of recent top-level posts for duplicate-thread checks when the local community database is available.
- The model payload explicitly labels publication fields as untrusted user content, not instructions.
- The challenge does not fetch linked publication media or user-submitted URLs. `promptUrl` is an operator-configured private prompt source, not publication content.
- Remote prompts are fetched without following redirects, with a 5 second timeout, capped at 64 KiB, cached in memory for 5 minutes, and reused from the last in-memory copy if a refresh fails. If the first remote prompt fetch fails, moderation fails closed for the allow branch.
- Two branch invocations for the same publication reuse one in-process verdict promise.
- Successful verdicts are cached in a private JSON file keyed by a SHA-256 hash over primary, triage, and enabled Jev model/provider config, community context including duplicate-check context, target content, and the final prompt hash. The cache does not store the raw prompt or API keys.
- Verdicts are written to a private JSONL audit log with the model stage, model reason, raw publication fields, and hashes/metadata for correlation. The audit log does not store the raw prompt, API keys, prompt URL, or prompt bearer token.
## Article age checks
Set `articleMaxAgeHours: "48"` on **both** branches only for communities whose intended policy is a 48-hour article-link window. This is an independent explicit operator setting: the package never extracts numbers from arbitrary rules or prompts. Empty or omitted leaves deterministic enforcement disabled.
For top-level links, excluding known image/video/audio links, the node computes minimum and maximum possible article age from the existing URL-path date hint and submission timestamp. The existing hint convention spans the full UTC calendar day; a URL date is not a verified publication time. Review occurs only when **even the latest possible article time is strictly older** than the configured window. An exact boundary, a day straddling the boundary, a future day, a missing/invalid date, or missing submission time cannot establish an age violation. The model receives computed age bounds and is instructed not to re-enforce the configured window, while still enforcing any stricter explicit community age limit and all other rules. Replies and content edits retain their existing behavior; edits do not contain the original linked-article context.
A proven age violation uses the existing allow/review branch semantics and does not call any provider. Its audit entry has `source: "rule"`, `rule: "article-recency"`, numeric bounds, and no provider or token usage. It uses the existing in-process deduplication; deterministic age verdicts are not written to the persistent model cache. Applicable age configuration participates in the verdict cache identity. The setting does not fetch articles or prove that an untrusted URL date is truthful; enable it only where that date convention is suitable.
## Local usage reporting and evaluation
Run `node scripts/moderation-usage.mjs --audit /private/moderation-audit.jsonl` to generate an offline aggregate report. It counts final provider stages, stages reached, audited disk-cache hits, deterministic decisions, request latency, throttling, timeouts, and reported tokens. It excludes publication text, prompts, reasons, credentials, and unknown audit fields. In-process cache reuse is not logged and cannot be counted from these files. Shadow logs can be reported separately; do not concatenate overlapping log copies.
For cost estimates, pass `--rates /private/rates.json`. Rates are explicitly supplied per exact host/model; no prices are embedded or assumed current. Missing rates, usage, cache breakdowns, or separately billed reasoning counts leave total cost unknown. Known cost remains a subtotal. Provider-attempt latency is not end-to-end moderation latency, and the report cannot reconstruct unreported billing. See [operator tooling](docs/moderation-tooling.md) for the rate format and examples.
`node scripts/moderation-evaluate.mjs` validates the committed synthetic corpus **offline**. Saved predictions can be compared without network access. Deliberate live evaluation uses the built challenge, explicit credential environment references, and request/byte budgets; it prints only sanitized case IDs, labels, and aggregate telemetry. See [evaluation instructions](docs/moderation-tooling.md#moderation-evaluation). Synthetic cases and independently reviewed real cases are labeled separately; the initial corpus is not evidence of real-world model accuracy.
## Moderation Audit Community
The challenge writes one private JSONL audit entry per model verdict. To mirror those entries into a Bitsocial community for moderators, create an unnamed local community with a single `question` challenge, store the answer in a private node-local file, and run the publisher script on the node:
```bash
node scripts/publish-audit-log-to-community.mjs \
--community 12D3KooW... \
--audit-log ~/.bitsocial-ai-moderation-audit.jsonl \
--challenge-answer-file ~/.bitsocial-ai-moderation-mod-log-password \
--keep-posts 100 \
--follow
```
The publisher creates a persistent local signer at `~/.bitsocial-ai-moderation-mod-log-signer.json`, stores its read offset in `~/.bitsocial-ai-moderation-mod-log-state.json`, and submits the private challenge answer when publishing. Each mod-log post includes the AI action, verdict reason, matched rule indexes, source community, publication kind, author identifiers, available CIDs, link metadata, content/title, provider/model, cache key, prompt hash, and rule hash.
Keep the mod log bounded with `--keep-posts`: posts the publisher made beyond the newest N are archived, and archived posts are purged after `--archive-purge-seconds` (default 48 hours), the same scheme 5chan-board-manager uses for boards. The publisher's signer must be a moderator of the mod-log community. An unbounded mod log grows by one post per verdict from a single author, and pkc-js 0.0.101 recalculates every one of an author's comments when that author posts, so a large mod log keeps the node's main thread busy and slows publishing on every community it hosts.
## Test Coverage
The coverage badge reports line coverage generated with `yarn test:coverage`. On pushes to `master`, CI writes a Shields-compatible endpoint payload and publishes it to GitHub Pages.
The test suite covers the moderation-critical flow: OpenAI-compatible Responses and chat-completions requests include `community.rules`, triage approvals avoid the primary reviewer, triage reviews and outages escalate, model `review` verdicts fail the `allow` branch and pass the `review` branch used with `pendingApproval`, provider outages and malformed responses route new comments to review, and content edits are rejected on review or outage.
## Publishing
The first npm publish must create the package before trusted publishing can be configured:
```bash
npm publish --access public
```
After the package exists, configure npm trusted publishing:
- Publisher: GitHub Actions
- Organization: `bitsocialnet`
- Repository: `ai-moderation-challenge`
- Workflow filename: `publish.yml`
- Environment: leave blank
Equivalent npm CLI command:
```bash
npm trust github @bitsocial/ai-moderation-challenge --repo bitsocialnet/ai-moderation-challenge --file publish.yml
```
Future releases publish automatically when `package.json` version changes on `master`. The publish workflow skips versions that already exist on npm.