| name | corvus-test-suite-regeneration |
| description | Regenerate JSON Schema test classes from the JSON-Schema-Test-Suite git submodule. Covers the update-json-schema-test-suite.ps1 master script, the V5 test generator (Corvus.JsonSchemaTestSuite.CodeGenerator), exclusion configuration, and output directory structure. USE FOR: updating the test suite submodule, regenerating test classes after submodule or generator changes, understanding the test infrastructure. DO NOT USE FOR: running existing tests (use corvus-build-and-test), writing manual tests.
|
Test Suite Regeneration
Master Script
.\update-json-schema-test-suite.ps1
This handles the full workflow:
- Updates the
JSON-Schema-Test-Suite git submodule
- Regenerates V5 test classes (3 output directories)
V5 Test Generator
The Corvus.JsonSchemaTestSuite.CodeGenerator project generates test classes for 3 suites,
output into two test projects:
| Output subdirectory | Purpose |
|---|
tests/Corvus.Text.Json.SchemaTestSuite.Tests/JsonSchemaTestSuite/ | Full type-based validation tests |
tests/Corvus.Text.Json.EvaluatorTestSuite.Tests/StandaloneEvaluatorTestSuite/ | Evaluator-only tests |
tests/Corvus.Text.Json.EvaluatorTestSuite.Tests/AnnotationTestSuite/ | Annotation collection tests |
The output paths are configured in the generator's appsettings.json (outputPath,
evaluatorOutputPath, annotationOutputPath). The emitted fixtures are OS-independent
(embedded paths use /, embedded JSON uses LF), so regeneration is byte-stable across
operating systems.
CRITICAL: The V5 generator must be run from its bin/ directory, not via dotnet run, because appsettings.json uses relative paths that resolve from the working directory.
# Build the generator
dotnet build tests\Corvus.JsonSchemaTestSuite.CodeGenerator -c Release
# Run from bin directory (use `dotnet <dll>` so it works on every OS;
# the apphost is `.exe` on Windows only)
Push-Location tests\Corvus.JsonSchemaTestSuite.CodeGenerator\bin\Release\net10.0
dotnet .\Corvus.JsonSchemaTestSuite.CodeGenerator.dll > $null
Pop-Location
Redirect stdout to $null to avoid PowerShell buffer overflow.
V4 Tests
V4 schema validation tests are in tests-v4/Corvus.Json.Specs.Tests/ as MSTest tests.
These test V4 types (Corvus.Json.ExtendedTypes, Corvus.Json.CodeGeneration.CSharp)
using runtime Roslyn compilation via JsonSchemaBuilderDriver. The test classes were
generated from the same JSON-Schema-Test-Suite data and cover all drafts (4, 6, 7,
2019-09, 2020-12) plus OpenAPI 3.0.
V4 tests are NOT regenerated by the master script — they are static MSTest test classes.
Critical Rules
-
Always delete old generated files before regenerating. The generators only create files — they never delete files that are no longer selected by exclusion rules. Stale files cause compilation errors or false test passes.
-
Run from the correct working directory. The V5 generator uses relative paths from appsettings.json.
Exclusion Configuration
Test exclusions (for known spec failures or draft-specific skips) are configured in
appsettings.json in the V5 generator project.
Cross-References
- For running the generated tests, see
corvus-build-and-test
- For the JSON Schema keyword system, see
corvus-keywords-and-validation
- Full guide:
docs/RunningTests.md
