| name | conformance-suite |
| description | How the SQLite conformance corpus (`conformance/sqlite-sqltests/`) and the expected-failures file work, and how to close a conformance gap correctly. Use this when a sqltest fails or when closing an engine gap. |
Conformance suite
conformance/sqlite-sqltests/ is a vendored, read-only corpus of 270
.sqltest files (~8,163 cases) that exercises the SQLite SQL surface. It is
the primary correctness signal that Ahtola matches SQLite/Turso behavior.
Hard rules
- Do not edit
.sqltest files to fix tests. Fix the engine. The corpus is
upstream; editing it silently weakens correctness.
- Conformance gaps belong in the expected-failures file, not in
[Ignore]
attributes scattered across tests. Don't sprinkle [Ignore] to hide a gap.
- When you close a gap, remove the corresponding expected-failure line —
do not leave a passing case listed as an expected failure.
Runner
The runner lives in src/Ahtola.Tests/Sqltest/:
SqltestParser (parses @database, @skip, test NAME { … } expect { … }),
SqltestCorpus (loads the 270 files), SqltestManagedRunner (executes against
the managed engine).
Expected-failures file
src/Ahtola.Tests/Conformance/managed-sqltest-expected-failures.txt — one
line per known gap, format <file>::<test> | <summary> (see the header
comment in the file itself). When the engine does not yet satisfy a case, add
a line here with a short summary of why. When the engine catches up, delete
the line.
Regenerating
Use the RegenerateExpectedFailures filter to regenerate the file from a full
run — but only after you have actually fixed or intentionally accepted the
gaps. Review the diff before committing; a regenerated file that "accidentally"
drops a still-failing case hides a regression.
Closing a gap
- Run the specific case:
pwsh ./scripts/Invoke-ManagedTestSuite.ps1 -Framework net10.0 -Filter "FullyQualifiedName~<case>" -MinimumExecutedTests 1.
- Fix the engine in
Ahtola.Core (consult turso-src/ for the upstream
semantics — see the turso-source-ref skill).
- Re-run the full conformance lane to confirm you didn't regress another case.
- Remove the line from
managed-sqltest-expected-failures.txt.
