tidb-test-guidelines

name	tidb-test-guidelines
description	Decide where to place TiDB tests and how to write them (basic structure, naming, testdata usage). Use when asked about test locations, writing conventions, shard_count limits, casetest categorization, or when reviewing test changes in code review.

TiDB Test Placement and Writing

Quick workflow

1. Identify the target package and nearest existing coverage using rg --files -g '*_test.go' and rg --files -g '*.json'.
2. Reuse the nearest existing suite, fixtures, and testdata before creating new files.
3. Check package-specific references when available; for planner/core optimizer cases, use references/planner-guide.md.
4. Keep directory test count aligned with BUILD.bazel shard_count (practical target: <= 50 tests per directory; benchmarks excluded).
5. When moving tests or benchmarks, update BUILD.bazel and related wrappers such as Makefile bench-daily.

Basic writing rules

• Benchmarks (func BenchmarkXxx) follow the same placement and naming rules, but do not count toward the directory test-count target.
• Prefer table-driven tests for related scenarios in the same behavior area.
• Reuse existing helper setups and test fixtures; avoid re-creating schemas unless required.
• Prefer one store + one tk per test; when a single test covers multiple scenarios, use distinct table names and restore any session/system variables to their original values.
• If a test must use multiple sessions or domains (for example, cross-session cache behavior), keep the extra stores/testkits but document why in the test.
• For follow-up bug fixes, prefer appending cases to an existing semantically close suite before creating a new file or top-level test.
• In existing regression suites that are already sequential, keep sequential scenario blocks; do not introduce t.Run unless subtest isolation/parallel behavior is required by semantics.
• Prefer exact assertions with .Check(testkit.Rows(...)); use MultiCheckContain only when output is inherently unstable, and add a short comment for why exact match is not practical.
• Reuse table definitions only across semantically similar scenarios; if partition/index/DDL shape is the behavior under test, keep separate table schemas to avoid false positives.
• Keep per-test runtime short; consolidate similar checks only if runtime stays reasonable.
• Use behavior-based test names; never use issue-id-only names.
• In test code, use the variable name tk for *testkit.TestKit (avoid testKit).
• When merging multiple tests into one, keep a single store and a single tk unless multi-session behavior is required; do not create a new store/tk inside the same test body without a documented reason.

Reference files

• Package case maps (when available): references/<pkg>-case-map.md
• Planner core placement guide: references/planner-guide.md

Package-specific notes

• Use existing testdata patterns (*_in.json, *_out.json, *_xut.json) in the same directory when extending suites. Use -record -tags=intest,deadlock only when the Go test suite explicitly supports -record and you need to regenerate outputs. For tests/integrationtest, use the recording command in docs/agents/testing-flow.md -> Integration tests (not -record).
• For pkg/planner/core/casetest/rule predicate pushdown cases, keep SQL in predicate_pushdown_suite_in.json and record both EXPLAIN format='brief' and query results via the test runner (see rule_predicate_pushdown_test.go).
• For planner explain-format changes in tests, default ordinary optimizer assertions to EXPLAIN format = 'plan_tree', but keep special-format suites unchanged. In particular, do not switch stats/analyze/load-stats-related tests (for example stats_test.go, planstats, analyze-focused CBO suites, and suites that depend on load stats) to plan_tree; cost-oriented assertions should prefer EXPLAIN format = 'cost_trace'; plan cache tests should keep explain for connection or format='plan_cache'; and hint tests should prefer plan_tree when the assertion is about the final chosen plan, while warning/applicability assertions may keep the nearby suite's existing format when that output is format-sensitive.
• When moving benchmarks between packages, update any TestBenchDaily wrappers that list them and keep Makefile bench-daily entries aligned with the new package location.
• When updating tests in any pkg/* package, update the corresponding case map under references/ if it exists; do not block on missing case maps.
• When updating tests in any other directory and the placement rules are non-obvious or recurring, consider adding/extending a case map under references/ and updating this SKILL.md.
• When merging issue regression tests into existing behavior tests, keep the issue id in SQL comments (e.g. /* issue:12345 */) or nearby comments for traceability.
• Prefer unit tests over tests/integrationtest for end-to-end coverage unless you need to avoid union-storage executor differences or require full workflow validation.
• When tests read source files under Bazel, use go/runfiles and ensure the target file is exported via exports_files() in its owning BUILD.bazel.
• For Bazel runfiles, be ready to include the workspace prefix (from TEST_WORKSPACE) in the runfile path if needed.
• For Bazel validation prerequisites and failpoint-aware execution, use .agents/skills/tidb-bazel-prepare-gate and docs/agents/testing-flow.md.