Use when scoring NL artifact quality, applying penalties, or calibrating lint judgment — contains the 100-point rubric with penalty tables per artifact type. Four worked calibration examples (Excellent Agent / Rewrite Agent / Excellent Rule / Weak Rule) live in `references/calibration-examples.md`, loaded on demand when anchoring borderline cases.
Use when scoring NL artifact quality, applying penalties, or calibrating lint judgment — contains the 100-point rubric with penalty tables per artifact type. Four worked calibration examples (Excellent Agent / Rewrite Agent / Excellent Rule / Weak Rule) live in `references/calibration-examples.md`, loaded on demand when anchoring borderline cases.
version
0.3.1
NLPM Quality Scoring Rubric
100-point quality scale for all NL programming artifacts. Apply penalties deterministically. Use calibration examples to anchor judgment on borderline cases.
Scoring Formula
base_score = 100
adjustments = sum of all applicable penalties (all penalties are negative)
final_score = max(0, min(100, base_score + adjustments))
Penalties stack. The floor is 0; the ceiling is 100. No bonuses — the default assumption is that an artifact is well-formed, and quality is measured by what is missing or wrong.
Penalty Tables
Skills
Rule
Check
Condition
Penalty
--
name present
Missing
-25
--
name matches parent directory
Frontmatter name: value does not equal parent directory name (per nlpm:conventions §5 — open spec MUST)
-15
R04
description present
Missing
-25
R04
Trigger quality
Description is generic (≤1 specific phrase)
-15
R04
Description length
Description 500–800 chars
-5
R04
Description length
Description >800 chars
-10
R05
Body length
400–500 lines
-5
R05
Body length
>500 lines
-10
R06
Code examples
Complex concepts with no examples
-5
R06
Code examples
No examples at all in a technical skill
-10
R06
<example> blocks
Zero <example> blocks on a user_invocable: true skill
-10
R07
Scope note
No scope note / cross-references
-3
Scope-note discipline: R07 means "scope note when related skills exist."
Do NOT apply R07 to missing example blocks — that is the new R06 row above
(penalty -10, not -15). The 2026-05-13 lijigang/ljg-skills audit applied
R07 + −15 fourteen times for missing example blocks; both labels were
wrong (R07 is not example-related, and -15 is the agents penalty, not
the skills penalty). The validator at auditor/scripts/validate-rule-ids.py
catches this kind of drift in CI.
<example>-block counting discipline (added 2026-08-01, origin:
xiaolai/cc-suite v1.3.1 remediation): an <example> block counts only when
it sits outside fenced code blocks — in the body or in a frontmatter
description block scalar. Tags inside a fenced template (```markdown … ```)
are illustrative content, and prose that names the string `<example>`
is a mention, not a block. A 2026-07-31 scoring pass credited a skill with
example blocks that existed only inside a fenced template, hiding a real R06
violation across 13 files. Verify by reading the file, not by grepping for
the tag.
name matches parent directory (added 2026-05-25, audit:
google/skills): the open Agent Skills spec at agentskills.io makes this
a MUST. Mismatch is deterministic, high-confidence, and reproducible by
single-line diff (frontmatter name: vs basename($(dirname FILE))).
Mark such findings confidence: high per the manifest-vs-disk-diff
principle in agents/scorer.md step 6. Note: this penalty did not
exist before 2026-05-25 — re-scoring past audits will yield slightly
lower scores for any corpus containing this defect, but no contribute
outcomes are retroactively affected since no PRs were ever opened
against a name-mismatch finding under the prior rubric.
Agents
Rule
Check
Condition
Penalty
R09
description present
Missing
-25
R09
<example> blocks
Exactly 1 example
-5
R09
<example> blocks
Zero examples
-15
R10
model declared
Not declared
-5
R10
model appropriate
Wrong tier for task (e.g. opus for parsing)
-5
R11
tools declared
Not declared
-5
R11
Unused tools
Each tool declared but not used in body
-3 each
R12
Output format
No output format spec in body
-10
R11
Write on read-only
Audit/review/scan agent declares Write or Edit
-10
Commands
Rule
Check
Condition
Penalty
--
description present
Missing
-25
R18
argument-hint present
Command takes input but no hint
-5
R14
Steps numbered
Multi-step body with no numbered steps
-10
R15
Empty input handling
No handling for empty/missing input
-10
R16
Output format
No output format defined
-10
R17
Error paths
No error handling for missing files or bad data
-5
Shared Partials
Rule
Check
Condition
Penalty
R19
user-invocable: false
Missing or set to true
-25
R20
Purpose clear
Description doesn't state it's a partial
-10
Rules
Rule
Check
Condition
Penalty
R21
description present
Missing frontmatter description
-10
R21
Format: bold imperative
No bold imperative opening
-5
R21
Format: rationale
No rationale following the imperative
-10
R22
Enforceability
Rule is not specific/testable
-10
R23
Budget
Rule file over 500 lines
-15
R26
Conflicts with other rules
Direct contradiction with another rule in same set
-20
R24
Duplicates tooling
Re-states what eslint/ruff/clippy already catches
-10
Hooks — universal checks (apply to all tools)
Rule
Check
Condition
Penalty
--
Valid syntax
Hook config file fails to parse (JSON or TOML per tool)
Authoritative event list: nlpm:conventions-claude §7. Per the multi-tool design (analysis/multi-tool-design-2026-05.md decision #4), Claude / Codex / Antigravity hook event vocabularies are NOT 1:1 mappable — three separate tables, no translation.
Rule
Check
Condition
Penalty
R27
Event names valid (Claude)
Uses unrecognized event name — confirmed Claude events: SessionStart, SessionEnd, UserPromptSubmit, PreToolUse, PostToolUse, PermissionRequest, Stop, StopFailure, FileChanged
-15
R27
Case correct (Claude)
Event name has wrong case (e.g. pretooluse)
-10
--
Hook type valid (Claude)
Uses unrecognized type value — confirmed Claude types: command, http, mcp_tool, prompt, agent
-10
--
MCP matcher format (Claude)
Matcher targets MCP tool but doesn't use mcp__<server>__<tool> pattern
Authoritative event list: nlpm:conventions-antigravity §5. All Antigravity-specific hook scoring is advisory-only (confidence:low) until the Antigravity 2.0 spec stabilizes — see analysis/multi-tool-design-2026-05.md decision #3.
Schema reference: nlpm:conventions-antigravity §3. All Antigravity-specific manifest scoring is advisory-only until the post-2026-06-18 Antigravity spec stabilizes.
Check
Condition
Penalty
Valid JSON
File fails JSON parse
-25
name present
Missing
-25
version present
Missing
-10
contextFileName includes AGENTS.md
Single-tool projects use only GEMINI.md; multi-tool should include AGENTS.md
A new artifact type recognized 2026-05-28 (see nlpm:conventions §2 and auditor/exemplars/karpathy-autoresearch.md): a project-root Markdown file driving an autonomous agent loop, hybrid between a memory file (AGENTS.md-shaped context) and a slash command (numbered workflow with output format + error paths).
No type-specific penalty rows. This artifact type is scored as the UNION of:
Command rules R14–R17: numbered steps for multi-phase work, empty-input handling, output format, error paths.
Memory file rules R33–R39: build/run commands, architecture overview, no stale refs, instructive-not-descriptive.
Universal R01 (vague quantifiers) and R03 (positive framing).
Type-specific penalty rows are deferred until N ≥ 3 examples surface — the existing rules cover the artifact adequately as a hybrid, and inventing rows from N = 1 risks over-fitting (calibrated per the same discipline applied to multi-tool discovery deferrals).
Patterns this artifact type rewards (loaded on demand from nlpm:patterns):
Each occurrence of: "appropriate", "relevant", "as needed", "sufficient", "adequate", "reasonable", "properly", "correctly", "some", "several", "various" without measurable criteria
-2 each
R01
Vague quantifier cap
Total vague quantifier penalty
max -20
Mention-versus-use exclusion (added 2026-08-01, origin: xiaolai/cc-suite
audit-family false positives; design reviewed via Codex consultation): do not
count a vague term when it is presented as a literal token AND the containing
clause explicitly instructs the reader or a tool to detect, flag, reject,
replace, avoid, or report that term — audit tooling must be able to name the
words it hunts (e.g. Flag uses of `some`, `several`, `various` without concrete criteria is R01's own job description, not a violation). Backtick
or quotation formatting alone does NOT qualify: a term that still modifies an
action, criterion, or requirement is counted even when backticked —
handle errors `properly` remains a violation.
All Artifact Types: Vocabulary Drift (R51 — opt-in, disabled by default)
Applied only when R51: { enabled: true, vocabulary_skill: <path> } appears in .claude/nlpm.local.md. Without the opt-in, R51 contributes zero penalty regardless of artifact content. The configured vocabulary_skill must contain a registry.yaml listing canonical and deprecated terms; without it, R51 emits an advisory and contributes zero penalty.
Rule
Check
Condition
Penalty
R51
Deprecated synonym
Each occurrence of a term marked deprecated: in the project's registry.yaml, in the scope the artifact belongs to
-2 each
R51
Drift cap
Total R51 penalty
max -10 per file
R51
Missing registry
enabled: true but vocabulary_skill: not set or points to a directory with no registry.yaml
0 (advisory only)
Why opt-in: vocabulary discipline is high-leverage for projects with accumulated drift but premature for projects still discovering their domain. Each project decides when it has enough literary warrant (P6) to lock terms in. See analysis/vocabulary-design-principles.md for the six principles R51 operationalizes.
Registry-declaration exclusion (added 2026-08-01, origin: xiaolai/cc-suite
vocabulary-skill self-reference; design reviewed via Codex consultation): the
file that declares a deprecation must name the deprecated term to do so.
Within the configured vocabulary_skill path, do not count a deprecated term
where it occurs in the declaration that registers it or maps it to its
replacement — registry.yamldeprecated: lists and the SKILL.md deprecation
tables. The exclusion covers ONLY those declaration term fields: deprecated
terms in surrounding prose inside the vocabulary_skill path are counted, and
the path is not categorically exempt. The same mention-versus-use principle as
R01's exclusion above, applied to R51.
Cross-Artifact (--plugin flag)
Applied when linting an entire plugin rather than individual files.
Check
Condition
Penalty
Broken partial refs
Command references commands/shared/X.md that doesn't exist
-20
Broken skill refs
Agent references plugin:skill that isn't installed
-20
Missing scripts
Hook references script that doesn't exist
-20
Orphaned files
Agent/command/skill file not referenced by anything
-5 per file
Contradictions
Two rules/instructions in same plugin directly contradict each other
-15 per pair
Score Bands
Range
Label
Meaning
90–100
Excellent
Production-ready; minor or no findings
80–89
Good
Solid; one or two non-critical gaps
70–79
Adequate
Meets threshold; noticeable gaps to address
60–69
Weak
Below threshold; significant findings
<60
Rewrite
Fundamental problems; recommend rewriting from scratch
Default pass threshold: 70. Configurable in .claude/nlpm.local.md.
Calibration Examples
Four worked examples — Excellent Agent (95), Rewrite Agent (41), Excellent Rule (92), Weak Rule (40) — live in references/calibration-examples.md. Load that file on demand when scoring a borderline case (around band boundaries: 88-92, 68-72, 58-62) and you need an anchored reference.
The examples are not needed for routine scoring — the penalty tables above are self-contained. They were extracted from this file 2026-05-28 to keep the rubric under R05's 500-line body budget while preserving the calibration material verbatim.
Scope Note
This skill covers the NLPM scoring formula, penalty tables, score bands, and calibration examples. It does NOT cover:
Artifact schemas and valid field values → see nlpm:conventions (universal) and the tool-specific overlays nlpm:conventions-claude, nlpm:conventions-codex, nlpm:conventions-antigravity (the latter three created in PR-B; see analysis/multi-tool-design-2026-05.md)
Patterns and anti-patterns catalog → see nlpm:patterns
How to run the score command → see commands/score.md
Multi-tool scoring (PR-B landed 2026-05-25)
nlpm now scores artifacts across three tool ecosystems — Claude Code, Codex CLI, and Antigravity (which absorbs Gemini CLI on 2026-06-18). The tier classification in agents/scorer.md separates open-spec (Tier 1), Tier 1.5 open-spec corpora, and per-tool Tier 2 overlays (2-Claude / 2-Codex / 2-Antigravity).
Hooks are scored per tool (Claude / Codex / Antigravity tables above). The three tools' event vocabularies are not 1:1 mappable; no universal translation layer. See analysis/multi-tool-design-2026-05.md decision #4.
Claude-specific additions in 2026: .lsp.json, monitors/monitors.json (validate JSON-parse only until detailed schemas land). New SKILL.md fields documented in nlpm:conventions-claude.
Known False Positive Patterns
The following findings have historically been reported by the scorer despite
having no backing in this rubric. They MUST NOT be penalized:
Invalid finding
Why it is invalid
Missing namespace: on skill
Not in the skill schema; conventions §5 does not list it
Missing inline hooks:/skills: registration blocks in plugin.json
conventions §1 defines these as optional path strings
AskUserQuestion / Task / WebFetch flagged as undocumented tool
Built-in per conventions §14
Agent missing skills: when omission is documented in CLAUDE.md
