code-dedup

name	code-dedup
description	Searches for duplicate code, duplicate tests, and dead code, then safely merges or removes them. Use when the user says "deduplicate", "find duplicates", "remove dead code", "DRY up", or "code dedup". Requires test coverage — refuses to touch untested code.

Code Dedup

Carefully search for duplicate code, duplicate tests, and dead code across the repo. Merge duplicates and delete dead code — but only when test coverage proves the change is safe.

Prerequisites — hard gate

Before touching ANY code, verify these conditions. If any fail, stop and report why.

1. Run make test — all tests must pass. If tests fail, stop. Do not dedup a broken codebase.
2. Run make test — tests are fail-fast AND enforce the coverage threshold from coverage-thresholds.json. If anything fails, stop and fix it before deduping.
3. Verify the project uses static typing. Check for:
   • Rust, C#, F#, Dart, Go: typed by default — proceed
   • TypeScript: tsconfig.json must have "strict": true — proceed (Lql/LqlExtension already does)
   • Untyped JavaScript: STOP. Refuse to dedup. Print: "This codebase has no static type checking. Deduplication without types is reckless — too high a risk of silent breakage. Add type checking first."

Steps

Copy this checklist and track progress:

Dedup Progress:
- [ ] Step 1: Prerequisites passed (tests green, coverage met, typed)
- [ ] Step 2: Dead code scan complete
- [ ] Step 3: Duplicate code scan complete
- [ ] Step 4: Duplicate test scan complete
- [ ] Step 5: Changes applied
- [ ] Step 6: Verification passed (tests green, coverage stable)

Step 1 — Inventory test coverage

Before deciding what to touch, understand what is tested.

1. Run make test to confirm green baseline. make test is fail-fast AND enforces the coverage threshold from coverage-thresholds.json (REPO-STANDARDS-SPEC [TEST-RULES], [COVERAGE-THRESHOLDS-JSON]). It exits non-zero on any test failure OR coverage shortfall.
2. Note the current coverage percentage — this is the floor. It must not drop.
3. Identify which files/modules have coverage and which do not. Only files WITH coverage are candidates for dedup.

Step 2 — Scan for dead code

Search for code that is never called, never imported, never referenced.

1. Look for unused exports, unused functions, unused classes, unused variables.
2. Use language-appropriate tools where available:
   • C#/F#: analyzer warnings (CA1801 unused parameters, IDE0051 unused private members) via make lint
   • Rust: cargo clippy already warns on dead code — check make lint output
   • TypeScript: noUnusedLocals/noUnusedParameters in tsconfig.json; look for unexported functions with zero references
3. For each candidate: grep the entire repo (including tests, scripts, samples). Skip generated code under Lql/lql-lsp-rust/crates/lql-parser/src/generated/. Only mark as dead if truly zero references.
4. List all dead code found with file paths and line numbers. Do NOT delete yet.

Step 3 — Scan for duplicate code

Search for code blocks that do the same thing in multiple places.

1. Look for functions/methods with identical or near-identical logic.
2. Look for copy-pasted blocks (same structure, maybe different variable names).
3. Look for multiple implementations of the same algorithm — particularly likely across DataProvider/, Migration/, Sync/, Reporting/ C# projects, and across Lql/Nimblesite.Lql.Core SQL transpiler dialects (Postgres, SqlServer, SQLite).
4. Check across module boundaries — duplicates often hide in different .csproj projects or Rust crates (lql-parser, lql-analyzer, lql-lsp).
5. For each duplicate pair: note both locations, what they do, and how they differ (if at all).
6. List all duplicates found. Do NOT merge yet.

Step 4 — Scan for duplicate tests

Search for tests that verify the same behavior.

1. Look for test functions with identical assertions against the same code paths.
2. Look for test fixtures/helpers that are duplicated across test projects (Tests.Shared/ is meant to hold these).
3. Look for integration tests that fully cover what a unit test also covers (keep the integration test, mark the unit test as redundant per CLAUDE.md "Prefer E2E/integration tests").
4. List all duplicate tests found. Do NOT delete yet.

Step 5 — Apply changes (one at a time)

For each change, follow this cycle: change → test → verify coverage → continue or revert.

5a. Remove dead code

• Delete dead code identified in Step 2.
• After each deletion: run make test (fail-fast + coverage + threshold all in one).
• If make test exits non-zero (test failure OR coverage drop): revert immediately and investigate.
• Dead code removal should never break tests or drop coverage.

5b. Merge duplicate code

• For each duplicate pair: extract the shared logic into a single function/module. Prefer placing shared code in Tests.Shared/ (for test helpers) or the closest .Core project (e.g. Sync/Nimblesite.Sync.Core, Lql/Nimblesite.Lql.Core).
• Update all call sites to use the shared version.
• After each merge: run make test.
• If tests fail: revert immediately. The duplicates may have subtle differences you missed.
• If coverage drops: the shared code must have equivalent test coverage. Add tests if needed before proceeding.

5c. Remove duplicate tests

• Delete the redundant test (keep the more thorough one).
• After each deletion: run make test.
• If coverage drops below threshold, make test exits non-zero — revert immediately. The "duplicate" test was covering something the other wasn't.

Step 6 — Final verification

1. Run make lint — all linters and the format check must pass.
2. Run make test — tests must pass AND coverage must remain ≥ the baseline from Step 1.
3. Report: what was removed, what was merged, final coverage vs baseline.

(Only the 7 standard targets exist — make lint and make test cover formatting and coverage checks respectively.)

Rules

• No test coverage = do not touch. If a file has no tests covering it, leave it alone entirely. You cannot safely dedup what you cannot verify.
• Coverage must not drop. If removing or merging code causes coverage to decrease, revert and investigate. The coverage floor from Step 1 is sacred.
• Untyped code = refuse to dedup. This repo has no untyped surfaces today; if any appear, refuse.
• One change at a time. Make one dedup change, run tests, verify coverage. Never batch multiple dedup changes before testing.
• When in doubt, leave it. If two code blocks look similar but you're not 100% sure they're functionally identical, leave both. False dedup is worse than duplication.
• Preserve public API surface. Do not change the public surface of Nimblesite.DataProvider.Core, Nimblesite.Lql.Core, Nimblesite.Sync.Core, or any package that is published to NuGet. Internal refactoring only.
• Three similar lines is fine. Do not create abstractions for trivial duplication. Only dedup when the shared logic is substantial (>10 lines) or when there are 3+ copies.
• Never touch ANTLR-generated code. Lql/lql-lsp-rust/crates/lql-parser/src/generated/ is regenerated from the .g4 grammar; any dedup there will be erased on the next regen.