validating-summaries

name	validating-summaries
description	Validate summary JSON contracts against strict schema requirements. Trigger keywords: summary validation, summary contract, JSON schema, character limits, summary fields, validate summary.
version	2.0.1
allowed-tools	Bash, Read

Validating Summaries

Validates summary JSON output against the strict contract defined in app/core/summary_contract.py.

Summary JSON Contract

See app/core/summary_schema.py for the full Pydantic model.

Validation Rules (overview)

Rule	Constraint
summary_250	Hard cap 250 chars, sentence boundary
summary_1000	Hard cap 1000 chars, multi-sentence
topic_tags	Leading #, deduplicated, max 10 recommended
entities	Lists deduplicated case-insensitively; categories: people, organizations, locations
key_stats	label required, value numeric, unit + source_excerpt optional
readability	method string, score numeric, level mapped from score

Full details with code fix snippets: references/validation-rules.md

Validation Scripts

Strict provider-schema validation

Validates raw model output against the complete provider JSON Schema. It requires every schema field and rejects extra fields or invalid nested structures:

.venv/bin/python .claude/skills/validating-summaries/scripts/validate-summary.py summary.json

Compatibility shaping

Runs the tolerant project mapper used for legacy or partial payloads. Successful shaping is not evidence that the raw payload satisfied the strict provider schema:

.venv/bin/python .claude/skills/validating-summaries/scripts/validate-with-project.py summary.json

Testing with CLI Runner

Test URL processing and summary generation end-to-end:

.venv/bin/python -m app.cli.summary \
  --url https://example.com/article \
  --json-path output.json \
  --log-level DEBUG

Use the strict script above on output.json when the test is meant to prove that raw provider output satisfies the generation contract.

Reference Files

• Contract validation: app/core/summary_contract.py
• Schema definition: app/core/summary_schema.py
• LLM prompts: app/prompts/summary_system_en.txt, app/prompts/summary_system_ru.txt
• JSON utilities: app/core/json_utils.py (includes repair logic)
• Validation rules: references/validation-rules.md
• Standalone script: .claude/skills/validating-summaries/scripts/validate-summary.py
• Project script: .claude/skills/validating-summaries/scripts/validate-with-project.py

Important Notes

• get_summary_json_schema() defines the strict provider-output contract
• validate_and_shape_summary() is a tolerant compatibility mapper
• JSON repair attempts to fix malformed LLM output (json_repair library)
• Both English and Russian prompts must be kept in sync
• Database stores verbatim JSON in summaries.json_payload
• Failed validations are logged with correlation ID for debugging