tool-registry-boundary

# Tool Registry Boundary Skill You keep the 5,000+-tool executable registry out of module graphs that don't execute tools. ## The rule > Client-reachable code reads tool **metadata**. Only code that actually executes a tool imports the **registry**. `@/tools/registry` is a 10,000+-line barrel importing every tool. External `ToolConfig` entries mix plain data (`params`, `outputs`, `name`) with request/response closures, while `InternalToolConfig` entries contain semantic input projection and load their server implementation through `lib/internal/tool-operations/registry.server.ts`. Request closures can still reach SDK clients, API helpers, and parsers, which is what makes the executable barrel expensive: reaching it costs roughly 4,700 additional modules (measured; re-measure with `--verbose`). `getTool()` returns the whole `ExecutableToolConfig`, so a single `getTool` import anywhere in a client-reachable file drags all of it in. ## Which module to import | you need | import | notes | | --- | --- | --- | | whether a tool id exists | `hasToolId` from `@/tools/tool-ids` | ~110 KB — the cheapest module | | to resolve an unversioned name | `resolveToolId` from `@/tools/tool-ids` | | | every tool id | `getToolIds` from `@/tools/tool-ids` | | | a tool's params | `getToolParams` / `getToolMetadata` from `@/tools/metadata` | ~4 MB | | a tool's declared outputs | `getToolOutputsMetadata` from `@/tools/metadata-outputs` | ~4 MB, separate on purpose | | to **execute** a tool | `getTool` from `@/tools/utils`, or `getToolAsync` from `@/tools/utils.server` (async/workspace-context lookup) | server paths only | Three modules, cheapest first. Ids are their own artifact because resolution and existence checks need only the key set; outputs are their own because they are the larger half of the data with a single consumer. `@/tools/metadata` and `@/tools/metadata-outputs` both resolve ids through `@/tools/tool-ids`, which is what keeps them independent of each other — do not "helpfully" re-export one from another, or every caller pays for all three. All lookups guard with `Object.hasOwn`. `JSON.parse` yields an object with the normal prototype, so a bare bracket lookup returns inherited members: a bare bracket lookup would answer `getToolMetadata('constructor')` with a *function* typed as tool metadata. ## The generated artifacts `apps/sim/tools/generated/tool-ids.ts`, `tool-metadata.ts` and `tool-outputs.ts` are produced by `scripts/sync-tool-metadata.ts`: ```bash bun run tool-metadata:generate # after adding/changing a tool bun run tool-metadata:check # what CI runs; fails if stale ``` Never hand-edit them. If you add a tool or change a tool's `params`/`outputs`, regenerate and commit the result, or CI fails. Four non-obvious properties, each of which was measured and is easy to undo by accident: - **The data is a JSON string parsed at runtime, not an imported `.json` and not an object literal.** With `resolveJsonModule` (which this repo enables), a `.json` import makes TypeScript infer a literal type for all 5,000+ entries and takes `tsc --noEmit` from **12.6s to 8m07s** — a 38x regression. An ambient `declare module` does *not* short-circuit it, and an object literal costs the same. A single string literal is one cheap token for both the compiler and the bundler, and `JSON.parse` beats evaluating the equivalent literal at runtime. Do not "clean this up" into a `.json` import. - **The generator refuses to emit function values.** If you add a field to `METADATA_FIELDS` that contains a closure, generation fails loudly rather than shipping executable config to the client. `hosting` and `schemaEnrichment` are excluded for exactly this reason (`hosting.enabled`, `pricing`, and `enrichSchema` are functions) — they are server-only. - **Empty param entries are stripped.** The registry contains one (`stt_deepgram_v2`), which crashes callers that read `param.type` while iterating. - **Lookups resolve versions.** `getTool` maps an unversioned name onto the newest version, and a few hundred tools are versioned. A plain key lookup would silently report them missing — a quiet correctness bug, not a crash. `resolveToolId` reproduces that against the id set and is differentially tested against the original. ## Testing code that reads tool metadata Mock the module the code under test actually reads. `@/tools/metadata` and `@/tools/metadata-outputs` are mocked globally in `apps/sim/vitest.setup.ts` and return `undefined`; never `vi.mock` them again (`check:test-patterns` fails a `global-remock`). Install fixtures on the global mock instead: ```ts import { toolsMetadataMock } from '@sim/testing/mocks/blocks.mock' import { getToolMetadata } from '@/tools/metadata' vi.mocked(getToolMetadata).mockImplementation(toolsMetadataMock.getToolMetadata) ``` Do the same for `getToolParams`. `vi.mock('@/tools/utils', () => toolsUtilsMock)` controls only `getTool`. Both mocks share `mockToolConfigs`, so together they give one consistent tool universe. If you are unsure whether a mock is load-bearing, change a fixture value to a sentinel and confirm the test fails. ## The guard `bun run check:tool-registry-boundary` (run in CI by `check:audits`) walks the module graph from each workspace route and fails if `@/tools/registry` is reachable, printing the exact import chain that reintroduced it. If it fails, do not add the entry to an allowlist — there isn't one. Find the symbol the offending file actually needs and move it to a registry-free module (e.g. `tools/merge-params.ts`). Run it with `--verbose` to print per-route module counts, which is also the quickest way to see whether a change moved the graph. The same command also ratchets those counts against `check-tool-registry-boundary.baseline.json`. `--check` (what CI runs) fails when an entry exceeds its baseline by more than `max(25 modules, 2%)`, naming the import chain responsible. The baseline also catches graph growth that never touches `@/tools/registry`. Re-record with `--update-baseline` and commit the JSON when growth is deliberate. A *shrink* passes but is reported — re-record then too, or the win is silently spendable again. ## How to verify an edge actually got cut Do not eyeball imports — the registry is reached through several redundant paths, so cutting one buys nothing while another survives. Walk the graph: 1. From the entry you care about, follow `import` and `export … from` (skipping `import type`), resolving `@/` against `apps/sim`. 2. Check whether `apps/sim/tools/registry.ts` is in the reachable set, and print the parent chain if it is. 3. Compare the reachable module count before and after. A route can reach the registry through several redundant edges; cutting one changes nothing until all are gone, so measure the route, not the file you edited. ## When adding a new caller Ask what the caller does with the config. If it reads `params`, `outputs`, `name`, `description` or just checks existence, it belongs on `@/tools/metadata` — no exceptions, even on a path you believe is server-only today, because a future client import will silently re-attach the registry to the graph. If it genuinely executes — builds an external request, transforms a response, or dispatches a registered internal operation — use `getTool`, and keep that file off client-reachable paths.