testing-library

# Testing a 3rd-Party NPM Library Tests an npm library by creating small test apps, verifying them on Node.js, then running them through the wasm-rquickjs runtime to document any incompatibilities. ## Overview The goal is **documentation, not fixing**. We create 1–5 focused test scripts for a library, confirm they pass on Node.js, then run each through the WASM runtime and record every error, missing API, or behavioral difference in `tests/libraries/libraries.md`. **Bundler choice:** We use **Rollup** (not esbuild) for bundling because this matches the production Golem CLI toolchain (`golem-cli` uses Rollup with `@rollup/plugin-commonjs` and `@rollup/plugin-node-resolve`). This ensures test results reflect what real Golem users will experience. Rollup's `@rollup/plugin-commonjs` hoists CJS `require()` calls to ESM `import` statements, unlike esbuild which generates a `createRequire(import.meta.url)` shim that fails when `import.meta.url` is undefined. ## Step 1: Pick the Library - If the user provides a library name, use that. - Otherwise, read `tests/libraries/libraries.md` and pick the **first** library with status `⬜` (Not yet tested). ## Step 2: Research the Library Use the **librarian** tool to understand what the library does, its core API surface, and what typical usage looks like. Focus on: - Pure computation / data-transformation features (most likely to work) - Features that rely on Node.js built-ins (crypto, streams, http, net, fs) - Whether the library is pure JS or has native bindings ## Step 3: Create Test Scripts Create **~5 small test scripts** in `tests/libraries/<package-name>/`. Each test script should: - Be a **standalone ESM file** (`.js`) that exports a `run` function - Use `assert` or simple `console.log`-based checks — no test framework dependencies - Exit with code 0 on success, non-zero on failure (use `process.exit(1)` or let uncaught errors propagate) - **Not require API keys, network services, databases, or manual intervention** (these are _offline_ tests; live-service tests with tokens from `.tokens.json` are created separately as `test-live-*.js` — see Step 5.7) - Test a distinct aspect of the library's API - Print `PASS: <description>` on success or throw/print `FAIL: <description>` on failure ### Test naming convention ``` tests/libraries/<package-name>/ ├── test-01-basic.js # Core functionality ├── test-02-validation.js # Input validation, edge cases ├── test-03-advanced.js # More complex features ├── test-integration-01-connect.js # (Optional) Docker integration: connectivity ├── test-integration-02-crud.js # (Optional) Docker integration: CRUD operations ├── test-live-01-completion.js # (Optional) Live service test (requires token) ├── docker-compose.yml # (Optional) Docker service for integration tests ├── mock-server.mjs # (Optional) HTTP mock server for integration tests ├── rollup.config.mjs # Rollup config (see template below) ├── run-node.mjs # Shared Node.js runner (see below) ├── package.json # Dependencies + rollup devDeps └── results.md # Test results (created in Step 6) ``` ### Test script format — dual-mode (Node.js + WASM) Each test is an **ESM file** that exports a `run` function returning `"PASS: <description>"` or throwing on failure. This format works for both Node.js verification and WASM embedding (where the WIT exports `run: func() -> string`). ```js // test-01-basic.js import assert from 'assert'; import lib from '<package-name>'; export const run = () => { const result = lib.someFunction('input'); assert.strictEqual(result, 'expected'); return "PASS: someFunction works correctly"; }; ``` **Important:** Always use `import` statements, not `require()`. Rollup's `@rollup/plugin-commonjs` converts CJS `require()` calls from dependencies into ESM imports, but test source files should use native ESM `import` for clarity and compatibility with both Node.js and the WASM runtime. ### Shared Node.js runner Create `run-node.mjs` once per package directory: ```js // run-node.mjs — runs a test file and prints the result (supports sync and async run()) const testFile = process.argv[2]; if (!testFile) { console.error('Usage: node run-node.mjs ./test-01-basic.js'); process.exit(1); } const mod = await import(testFile); try { const result = await mod.run(); console.log(result); if (!result.startsWith('PASS')) process.exit(1); } catch (e) { console.error('FAIL:', e.message || e); process.exit(1); } ``` ### Rollup config template Create `rollup.config.mjs` in each package directory. This mirrors the production Golem CLI Rollup config: ```js // rollup.config.mjs import commonjs from "@rollup/plugin-commonjs"; import json from "@rollup/plugin-json"; import nodeResolve from "@rollup/plugin-node-resolve"; import fs from "fs"; // Discover all test-*.js files automatically const testFiles = fs.readdirSync(".").filter(f => f.startsWith("test-") && f.endsWith(".js")); // Node.js built-in modules — kept as external imports (the runtime provides them) const nodeBuiltins = [ "assert", "buffer", "child_process", "crypto", "dgram", "dns", "events", "fs", "http", "http2", "https", "module", "net", "os", "path", "perf_hooks", "querystring", "readline", "stream", "string_decoder", "tls", "url", "util", "v8", "vm", "worker_threads", "zlib", ]; const externalPackages = (id) => { // Keep Node.js built-ins as external imports const bare = id.replace(/^node:/, ""); return nodeBuiltins.includes(bare); }; export default testFiles.map((input) => ({ input, output: { file: `dist/${input.replace(".js", ".bundle.js")}`, format: "esm", inlineDynamicImports: true, sourcemap: false, }, external: externalPackages, plugins: [ nodeResolve({ extensions: [".mjs", ".js", ".json", ".node"], }), commonjs({ include: ["node_modules/**"], }), json(), ], })); ``` ### ESM-only packages Some modern npm packages (e.g., `got` v12+, `node-fetch` v3+, `chalk` v5+) only ship ESM — they have `"type": "module"` in their `package.json` and no CJS fallback. This is fine for our workflow because: - **For Node.js testing:** Our test files are ESM (they use `export`), and Rollup handles resolving ESM dependencies during bundling. - **For WASM bundling:** Rollup with `nodeResolve` + `commonjs` plugins inlines all imports (both ESM and CJS) into the final bundle. No special handling is needed — Rollup resolves both CJS and ESM dependencies transparently during the bundle step. ## Step 4: Install Dependencies, Bundle, and Verify on Node.js First install dependencies and bundle: ```bash cd tests/libraries/<package-name> npm install # Bundle all test files at once using the rollup config npx rollup -c rollup.config.mjs ``` Then verify the **bundled** output on Node.js: ```bash node run-node.mjs ./dist/test-01-basic.bundle.js node run-node.mjs ./dist/test-02-validation.bundle.js # ... etc ``` **Important:** Always verify the *bundled* files, not the raw sources. **Every test MUST pass on Node.js** before proceeding. If a test doesn't pass on Node.js, fix the test (not the library). ## Step 5: Run Tests Through wasm-rquickjs ### 5a. Build wasm-rquickjs Before running any tests through the runtime, ensure the latest CLI is built: ```bash # From the repo root ./cleanup-skeleton.sh cargo build --release ``` This produces `./target/release/wasm-rquickjs`. ### 5b. Create a WIT file for the tests Create a minimal WIT that exports a `run` function. Place it in `tests/libraries/<package-name>/wit/<package-name>.wit`: ```wit package test:lib-<package-name>; world lib-<package-name> { export run: func() -> string; } ``` ### 5c. Use the bundles from Step 4 The bundled test files in `dist/` (created and verified in Step 4) are ready to use. Each bundle is a self-contained ESM file with the library code inlined and Node.js built-ins preserved as ESM `import` statements for the runtime to resolve. #### When bundling fails If Rollup cannot bundle a library (native bindings, dynamic `require()`, WASM binaries, etc.), **do not attempt workarounds**. Record the failure in `results.md`: ```markdown ### Bundling This library **cannot be bundled** with Rollup: - **Error:** `<exact Rollup error>` - **Reason:** Requires native N-API bindings / uses dynamic require() / loads .node files - **Impact:** Library cannot be tested in the wasm-rquickjs runtime without a bundleable alternative ``` Mark the library as ❌ in `libraries.md` with note "Cannot bundle — requires native bindings" and skip Steps 5d–5e. ### 5d. Generate, compile, and run for each test For each bundled test file: ```bash # 1. Generate wrapper crate ./target/release/wasm-rquickjs generate-wrapper-crate \ --js tests/libraries/<package-name>/dist/test-01-basic.bundle.js \ --wit tests/libraries/<package-name>/wit \ --output tmp/lib-test-<package-name>-01 # 2. Patch Cargo.toml features — disable "logging" (wasmtime CLI doesn't provide # wasi:logging). The generated default is ["normal"] which transitively includes # "logging". Replace it with "full-no-logging" which has everything except logging. sed -i '' 's/default = \["normal"\]/default = ["full-no-logging"]/' \ tmp/lib-test-<package-name>-01/Cargo.toml # 3. Compile to WASM cd tmp/lib-test-<package-name>-01 cargo-component build cd ../.. # 4. Run with wasmtime # - --invoke 'run()' — parentheses are REQUIRED # - -S cli -S http -S inherit-network — provide WASI imports # - --dir creates a temp directory preopened at "/" so node:fs operations work # - --invoke must come BEFORE the wasm path TEST_TMPDIR=$(mktemp -d) wasmtime run --wasm component-model \ -S cli -S http -S inherit-network \ --dir "$TEST_TMPDIR::/" \ --invoke 'run()' \ tmp/lib-test-<package-name>-01/target/wasm32-wasip1/debug/lib_test_<package_name>_01.wasm rm -rf "$TEST_TMPDIR" ``` **Do NOT write Rust test harnesses** for running library tests. Always use `wasmtime` CLI as shown above. **Filesystem access:** Each test run gets a fresh temp directory preopened at `/` via `--dir`. This gives `node:fs` operations (read, write, mkdir, etc.) a working filesystem. The temp dir is cleaned up after each run. If a test needs fixture files (e.g., `dotenv.config({ path: 'fixtures/config.env' })`), copy them into `$TEST_TMPDIR` before running wasmtime: ```bash # Example: copy fixture files into the temp dir for the test cp -r tests/libraries/<package-name>/fixtures "$TEST_TMPDIR/fixtures" ``` Capture stdout/stderr from each run. If the component fails to compile or run, record the exact error. ## Step 5.5: Docker-Based Integration Tests (Optional) Many libraries (databases, message queues, caches, etc.) are designed to connect to an external service. The tests in Step 3–5 deliberately avoid external dependencies. This step adds **optional integration tests** that validate the library works end-to-end against a real, locally-running service via Docker. ### 5.5a. Determine if Docker testing is applicable Check whether the library's primary use case involves communicating with a service that can be run locally via Docker. Common examples: | Library type | Docker image | Port | |---|---|---| | PostgreSQL clients (`pg`, `postgres`, `slonik`) | `postgres:16-alpine` | 5432 | | MySQL clients (`mysql2`, `mariadb`) | `mysql:8` or `mariadb:11` | 3306 | | Redis clients (`ioredis`, `redis`) | `redis:7-alpine` | 6379 | | MongoDB clients (`mongoose`, `mongodb`) | `mongo:7` | 27017 | | Elasticsearch clients (`@elastic/elasticsearch`) | `elasticsearch:8.x` | 9200 | | RabbitMQ clients (`amqplib`) | `rabbitmq:3-management-alpine` | 5672 | | NATS clients (`nats`, `nats.ws`) | `nats:latest` | 4222 | | Kafka clients (`kafkajs`) | `confluentinc/cp-kafka` | 9092 | | MinIO/S3 clients (`@aws-sdk/client-s3`, `minio`) | `minio/minio` | 9000 | | SMTP clients (`nodemailer`) | `mailhog/mailhog` | 1025 | **Skip this step entirely** if: - The library is a pure-computation library (validation, formatting, crypto utilities) - The service requires paid/proprietary software with no Docker image - The library is a server framework (Express, Fastify, etc.) — already excluded by the Golem constraint - The library requires API keys to a cloud service that cannot be self-hosted (e.g., OpenAI, Stripe) ### 5.5b. Create `docker-compose.yml` Place a `docker-compose.yml` in the test directory: ```yaml # tests/libraries/<package-name>/docker-compose.yml services: <service-name>: image: <docker-image>:<tag> ports: - "<host-port>:<container-port>" environment: # Use simple, deterministic credentials for testing <ENV_VAR>: <value> healthcheck: test: ["CMD", "<health-check-command>"] interval: 5s timeout: 5s retries: 10 tmpfs: - /var/lib/<data-dir> # Use tmpfs for speed — no persistent data needed ```