compare-python-typecheckers

# Compare Python type checkers When reasoning about how PyCharm *should* infer a type or whether something ought to be a type error, cross-checking against the real third-party checkers is the fastest way to gain insight — and to see where they disagree. This skill runs them all on a scratch file or snippet and collates the output into a single report. ## Run it `${CLAUDE_SKILL_DIR}/scripts/compare_typecheckers.py` invokes each checker through `uvx` (fetched on demand — no install; the network is used on first run) and prints a Markdown report: a summary table (exit code / verdict / timing per tool) followed by each tool's raw output. ```bash # Inline snippet: uv run ${CLAUDE_SKILL_DIR}/scripts/compare_typecheckers.py -c 'def f(x: int) -> int: return x f("a")' # An existing file: uv run ${CLAUDE_SKILL_DIR}/scripts/compare_typecheckers.py path/to/test.py # A subset, written to a file: uv run ${CLAUDE_SKILL_DIR}/scripts/compare_typecheckers.py test.py --tools ty,mypy -o /tmp/report.md ``` Flags: `-c/--code` (inline snippet), `-t/--tools ty,mypy,…` (subset), `-o/--output` (write to file), `--timeout` (per-checker seconds, default 180). The script exits 0 whenever it produced a report — findings live in the report, not the exit code. ## Reading the result - The summary table shows each tool's exit code (**0 = nothing flagged**, non-zero = the checker reported an issue or failed to run), a verdict, and time. - Output **formats differ** per tool, so read each section, not just the table: - `ty` (Astral) and `pyrefly` (Meta) each have their own diagnostic style. - `basedpyright` is pyright-based and also emits extra diagnostics such as `reportUnusedCallResult`. - `mypy` and `zuban` share the same message format (`zuban` is mypy-compatible). - The checkers track their latest releases through `uvx`, so behaviour can shift over time — record the date when capturing results in a YouTrack issue or test. ## Manual one-offs The exact per-tool invocations, if you want to run just one by hand (note `basedpyright` takes the file positionally — no `check` subcommand): | Tool | Command | |------|---------| | ty | `uvx ty check test.py` | | pyrefly | `uvx pyrefly check test.py` | | basedpyright | `uvx basedpyright test.py` | | mypy | `uvx mypy test.py` | | zuban | `uvx zuban check test.py` |