| name | assessment-summarizer |
| description | Aggregates validated reviewer assessments into a consolidated Skill Quality Report. This skill performs deterministic score aggregation and report generation only. It never validates assessment artifacts or executes quality reviews. |
Purpose
Generate the final Skill Quality Assessment Report after the Orchestrator has completed workflow validation.
This skill is responsible only for:
- aggregating reviewer results;
- calculating overall quality metrics;
- determining the final recommendation;
- generating the final assessment artifacts.
The Orchestrator is solely responsible for validating workflow completion and assessment artifacts before invoking this skill.
Responsibility
- This skill acts as the reporting layer of the Skill Quality Assurance Framework.
- It does not perform assessments.
- It does not validate artifacts.
- It does not repair missing information.
- It assumes every input artifact has already been validated by the Orchestrator.
Goals
Generate:
assessment/assessment-summarizer.jsonassessment/skill-quality-report.md
using the validated reviewer assessments provided by the Orchestrator.
Inputs
Rules
- The Orchestrator provides validated assessment artifacts.
- The presence of the execution assessment depends on whether execution review was requested.
- If the execution assessment is absent, the aggregation proceeds using Design Quality only.
Input Artifacts
- Mandatory: Reviewer assessments
<SKILL-NAME>-assessment/<agent-name>.json
- Optional: If were executed the execution assessment made by eval review
<SKILL-NAME>-assessment/eval-review.json
Responsibilities
The skill SHALL:
- read reviewer scores;
- aggregate reviewer findings;
- aggregate recommendations;
- calculate Design Quality;
- calculate Execution Quality (if available);
- calculate Overall Quality;
- determine the overall risk level;
- determine the final recommendation;
- summarize reviewer results;
- generate JSON report;
- generate Markdown report.
Aggregation Instructions
| Steps | Description |
|---|
| Step 1: | Load the validated assessment artifacts. |
| Step 2: | Extract reviewer information.
For each completed reviewer collect:
- score
- risk level
- findings
- recommendations
- resource consumption |
| Step 3: | Calculate Design Quality.
Rules:
Use scripts/calculator.py, provide as arguments the scores of the intent-review, instruction-review, qa-review, and recive Design Quality score.
Note: Only completed reviewers participate in the calculation. |
| Step 4: | Calculate Execution Quality. If an execution assessment exists, provide the execution-review score to the scripts/calculator.py script and recive Execution Quality score. Otherwise, Execution Quality is reported as NOT ASSESSED.
Note: No score shall be calculated. |
| Step 5: | Calculate Overall Quality.
Rules:
Use scripts/calculator.py, provide as arguments the scores of the intent-review, instruction-review, qa-review, execution-review, and recive Overall Quality score. |
| Step 6: | Calculate Overall Risk.Use the scripts/calculator.py script, provide as arguments the risk levels,of all completed reviewers, and recive Overall Risk score. |
| Step 7: | Determine Final Recommendation.The recommendation shall be computed by |
script/calculator.py. (see Recomendation Suggested Mapping) . | |
|
| Step 8: | Aggregate Findings. Merge reviewer findings preserving:
- reviewer
- severity
- message
- evidence.
Rule:
Do not modify reviewer findings. |
| Step 9: | Aggregate Recommendations.
- Merge reviewer recommendations.
- Duplicate recommendations should be consolidated.(see Recommendation Consolidation) |
| Step 10: | Calculate Resource Consumption.
- Aggregate the resources consumed by all completed reviewers:
* input tokens
* output tokens
* total tokens
* execution time
- Include the Aggregator's own resource usage.
Rule: Resource total are computed by Σ reviewer values + Aggregator values |
| Step 11: | Generate Reports (see Output Artifacts) |
Recomendation Suggested Mapping (calculate using the script/calculator.py)
The calculator implements:
- recommendation thresholds;
- downgrade rules;
- future framework versions
The Aggregator must never reimplement this logic.
Recommendation Consolidation
Recommendations shall be consolidated by:
-
Exact duplicate removal.
-
Semantic duplicate detection
(case-insensitive normalized text).
-
Grouping by category. Ordering:
Reviewers are processed in the following order:
- Intent Review
- Instruction Review
- QA Review
- Eval Review
The generated report preserves this ordering.
Output Artifacts
-
Generate the JSON report: in <SKILL-NAME>-assessment/assesssment-summarizer.json.
- See JSON Schema.Use the schema provided in
assets/assessment-summarizer-schema.json
- Populate all mandatory fields defined by the framework.
-
Generate the Markdown report: in <SKILL-NAME>-assessment/skill-quality-report.md.
- See Markdown Format.Use the format provided in
assets/skill-quality-report-template.md
- Populate all mandatory fields defined by the framework.
The reports should include:
- Assessment Summary
- Overall Scores
- Skill Classification
- Reviewer Summary
- Executive Findings
- Reviewer Findings
- Validation Summary
- Resource Consumption
- Final Recommendation
- Assessment Artifacts
Constraints
| Type | Rule |
|---|
| Must | Aggregate only validated reviewer outputs.
Preserve reviewer findings without modification.
Generate both JSON and Markdown reports.
Calculate scores deterministically using the scripts/calculator.py script.
Aggregate resource consumption.
Produce reproducible reports from identical inputs.
It only summarizes, aggregates, and computes framework-level metrics.
Do not estimate missing metrics.
Preserve reviewer evidence.
Reference Reference original reviewer artifacts instead of rewriting evidence.Use canonical framework artifact names. |
| Should | Consolidate duplicate recommendations.
Present findings in a consistent order.
Keep report formatting deterministic. |
| Could | Sort recommendations by severity.
Highlight the strongest reviewer strengths separately. |
| Won't | Validate assessment artifacts.
Verify workflow completion.
Repair missing artifacts.
Recalculate or modify reviewer scores.
Modify reviewer findings.
Generate new findings.
Perform quality reviews.
Invoke external resources.
Execute another reviewer.
Never reinterpret reviewer conclusions.
Reimplement calculator algorithms or modify the scripts/calculator.py script.
Perform mathematical calculations independently, except the resources consumption mentioned in step 10
Change reviewer ordering. |
Design Principles
The Aggregator follows these principles:
- Single Responsibility
- Deterministic Processing
- Evidence Preservation
- Read-only Assessment Consumption
- Reproducible Report Generation
Completion Criteria
The skill completes successfully when:
- both report artifacts are generated;
- aggregation calculations are completed;
- overall recommendation is determined;
- resource consumption is reported.
No additional validation shall be performed by this skill.
The Orchestrator is responsible for validating the generated reports after completion.
