<!-- SPDX-License-Identifier: CC-BY-SA-4.0 -->
<!-- SPDX-FileCopyrightText: Netresearch DTT GmbH -->
# Automated Assessment Skill
[](https://github.com/netresearch/automated-assessment-skill/actions/workflows/ci.yml)
[](#license)
Systematic project assessment against all Netresearch skills with checkpoint-based verification - by Netresearch.
## Why This Skill Exists
When asked to "ensure project aligns with all skills", LLMs typically:
- Cherry-pick obvious issues (satisficing)
- Miss 50-80% of requirements
- Report "done" without exhaustive verification
This skill enforces **systematic verification** through:
1. **Scripted pre-flight checks** (mechanical, 100% accurate)
2. **Domain-batched LLM agents** (subjective judgment)
3. **Structured JSON output** (verifiable completeness)
## Installation
### Marketplace (Recommended)
Add the [Netresearch marketplace](https://github.com/netresearch/claude-code-marketplace) once, then browse and install skills:
```bash
# Claude Code
/plugin marketplace add netresearch/claude-code-marketplace
/plugin install automated-assessment@netresearch-claude-code-marketplace
```
### Without a marketplace
Since Claude Code 2.1.157 a plugin directory under your personal skills directory loads on its own:
```bash
mkdir -p ~/.claude/skills
git clone https://github.com/netresearch/automated-assessment-skill.git \
~/.claude/skills/automated-assessment
```
It loads as `automated-assessment@skills-dir` on the next session. Update with `git -C ~/.claude/skills/automated-assessment pull` and start a new session; remove it by deleting the directory. This route has no `claude plugin update`.
### npx ([skills.sh](https://skills.sh))
Install with any [Agent Skills](https://agentskills.io)-compatible agent:
```bash
npx skills add https://github.com/netresearch/automated-assessment-skill --skill automated-assessment
npx skills add https://github.com/netresearch/automated-assessment-skill --skill add-checkpoints
```
### Download Release
Download the [latest release](https://github.com/netresearch/automated-assessment-skill/releases/latest) and extract to your agent's skills directory.
### Git Clone
```bash
git clone https://github.com/netresearch/automated-assessment-skill.git
```
### Composer (PHP Projects)
```bash
composer require netresearch/automated-assessment-skill
```
Requires [netresearch/composer-agent-skill-plugin](https://github.com/netresearch/composer-agent-skill-plugin).
### npm (Node Projects)
```bash
npm install --save-dev \
@netresearch/agent-skill-coordinator \
github:netresearch/automated-assessment-skill
```
Requires [@netresearch/agent-skill-coordinator](https://github.com/netresearch/node-agent-skill-coordinator), which discovers the skill in `node_modules` and registers it in `AGENTS.md` via a `postinstall` hook. For pnpm, also allowlist the coordinator's postinstall:
```json
{
"pnpm": {
"onlyBuiltDependencies": ["@netresearch/agent-skill-coordinator"]
}
}
```
## Usage
### Run Assessment
```
/assess # Assess against all matching skills
/assess skill-repo typo3-testing # Assess against specific skills only
/assess --force # Run all skills, ignore preconditions
/assess --mechanical-only # Skip LLM reviews, only scripted checks
```
This will:
1. Detect project root
2. Discover all skills with checkpoints
3. Evaluate preconditions — skip skills that don't match project type
4. Run scripted mechanical checks
5. Spawn domain-batched LLM agents for subjective reviews
6. Generate compliance report
### Run Checkpoints Manually
```bash
scripts/run-checkpoints.sh <checkpoints.yaml> <project-root>
```
## Adding Checkpoints to Skills
Create `checkpoints.yaml` in your skill's root directory:
```yaml
version: 1
skill_id: my-skill
mechanical:
- id: MS-01
type: file_exists
target: README.md
severity: error
desc: "README.md must exist"
llm_reviews:
- id: MS-10
domain: repo-health
prompt: "Verify README structure follows standards"
severity: warning
desc: "README should have standard sections"
```
See `skills/automated-assessment/references/checkpoints-schema.md` for full schema documentation.
## Checkpoint Types
| Type | Description |
|------|-------------|
| `file_exists` | File must exist |
| `file_not_exists` | File must NOT exist |
| `contains` | File contains literal string |
| `regex` | File matches regex pattern |
| `json_path` | JSON path exists (jq) |
| `command` | Command exits with 0 |
| `llm_review` | Requires LLM judgment |
## Domain Groups
| Domain | Focus |
|--------|-------|
| `repo-health` | README, badges, branding, AGENTS.md |
| `security` | SLSA, OpenSSF, SBOM, vulnerabilities |
| `code-quality` | PHPStan, tests, PHP patterns |
| `documentation` | RST, docs.typo3.org standards |
| `git-workflow` | Branching, commits, tags, conventional commits |
| `docker` | Dockerfile, compose, container patterns |
| `ddev` | DDEV configuration, services, commands |
| `upgrade` | TYPO3 version upgrades, deprecations |
## Example Assessment Output
```json
{
"project": "vendor/example-extension",
"overall_status": "FAIL",
"summary": {
"total": 45,
"pass": 38,
"fail": 5,
"skip": 2,
"blocked": 0
},
"checkpoints": [
{"id": "GH-01", "status": "pass", "evidence": "README.md exists"},
{"id": "ER-04", "status": "fail", "evidence": "Missing OpenSSF badge"},
{"id": "TD-30", "status": "blocked", "evidence": "Command rejected: 'bash' not in allowed command whitelist"}
]
}
```
`blocked` means the runner refused the checkpoint's command, so it never ran —
a defect in the checkpoint, not a finding about the project. It is counted
separately (`total == pass + fail + skip + blocked`) and never sets the exit
code. See `references/checkpoints-schema.md` → "Outcomes".
## Assets
- `assets/github-project-checkpoints.yaml` - Example checkpoints for github-project skill
- `assets/enterprise-readiness-checkpoints.yaml` - Example checkpoints for enterprise-readiness skill
- `assets/llm-rubric-repo-health.md` - LLM review rubrics for repo-health domain
- `assets/skill-template/` - Template for creating new skills with checkpoints
## References
- `references/checkpoints-schema.md` - Full schema documentation
- `references/migration-guide.md` - How to add checkpoints to existing skills
## Contributing
1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Submit a pull request
## Governance and policies
This repository follows the organisation-wide policies of `netresearch`:
- [Governance](https://github.com/netresearch/.github/blob/main/GOVERNANCE.md): roles, how changes are decided and disputes resolved, and who controls access to sensitive resources.
- [Roadmap](https://github.com/netresearch/.github/blob/main/ROADMAP.md): the maintenance work planned and excluded for the coming year.
- [Handling of dependency and code analysis findings](https://github.com/netresearch/.github/blob/main/SECURITY.md#handling-of-dependency-and-code-analysis-findings): which vulnerability, licence and static-analysis findings block a change, the deadlines for the others, and how exceptions are recorded.
- [Secret management](https://github.com/netresearch/.github/blob/main/SECURITY.md#secret-management): where CI secrets are stored, who can access them, and when they are rotated.
- [Access roster](https://github.com/netresearch/.github/blob/main/docs/access-roster.md): the accounts with admin, maintain and write access to this repository.
Checks that run on every pull request to `main`:
- Skill Validation (`.github/workflows/lint.yml`, reusable `validate.yml` of `netresearch/skill-repo-skill`): skill structure and version parity, ShellCheck at severity `style`, actionlint, markdownlint, yamllint, JSON syntax and checkpoint schemas.
- Eval Validation (`.github/workflows/eval-validate.yml`, reusable `eval-validate.yml` of `netresearch/skill-repo-skill`): the structure of the first `skills/*/evals/evals.json`, which is `skills/add-checkpoints/evals/evals.json` (at least ten evals). It does not run the prompts against a model.
- CI (`.github/workflows/ci.yml`): shell syntax of the checkpoint runner, required fields of the template checkpoint file, SKILL.md body size.
- Skill Tests (`.github/workflows/tests.yml`): runs every `tests/*.sh`.
- Harness Verification (`.github/workflows/harness-verify.yml`): AGENTS.md presence, size and references.
- Security (`.github/workflows/security.yml`): Betterleaks secret scanning, zizmor workflow analysis, dependency review, and `composer audit` with an Opengrep SAST scan (which findings fail the check is set by the [organisation rule](https://github.com/netresearch/.github/blob/main/SECURITY.md#static-analysis-sast)).
- Template Drift (`.github/workflows/check-template-drift.yml`): the files managed by the `skill` template in `netresearch/.github` match it, except the ones listed in `.github/template.yaml`.
- Labeler (`.github/workflows/labeler.yml`) and Auto-merge dependency PRs (`.github/workflows/auto-merge-deps.yml`, which acts only on Renovate and Dependabot pull requests).
- CodeQL (GitHub default setup): analyses the GitHub Actions workflows.
- SonarCloud Code Analysis, configured outside this repository.
- DCO: every commit carries a `Signed-off-by` line.
The repository installs no third-party packages at run time: `composer.json` requires only `netresearch/composer-agent-skill-plugin`, and `package.json` declares only a peer dependency. Renovate (`renovate.json`) proposes updates for the hook revisions pinned in `.pre-commit-config.yaml`.
What you can and cannot expect from this repository in terms of security, with its threat model and trust boundaries: [docs/SECURITY-ASSURANCE.md](docs/SECURITY-ASSURANCE.md).
## License
This project uses split licensing:
- **Code** (scripts, workflows, configs): [MIT](LICENSE-MIT)
- **Content** (skill definitions, documentation, references): [CC-BY-SA-4.0](LICENSE-CC-BY-SA-4.0)
See the individual license files for full terms.
---
> **Netresearch DTT GmbH** - [netresearch.de](https://www.netresearch.de/)