Menu
Add or debug pytest unit, integration, and LLM evaluation tests for genai-tk, including fake models, real-model gates, and trajectory tests.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Build or modify LangChain, DeepAgent, DeerFlow profiles, agent tools, middleware, checkpointing, skills wiring, and the shared harness layer in genai-tk.
Work on BAML structured extraction, BAML CLI commands, processors, utilities, and Prefect BAML workflow integration in genai-tk.
Work on browser automation, sandbox browser tools, direct Playwright tools, AioSandbox backend, and sandbox CLI support in genai-tk.
Add or modify genai-tk Typer CLI commands, dynamic command registration, project scaffolding, and generated Copilot/agent support files.
Work on genai-tk OmegaConf configuration, profiles, overrides, env substitution, and config discovery. Use when editing config/*.yaml or genai_tk.config_mgmt.config_mngr.
Work on core LLM, embeddings, vector store, provider, cache, prompt, and retriever factories in genai-tk. Use when editing genai_tk/core or provider configuration.
| name | evaluation-testing |
| description | Add or debug pytest unit, integration, and LLM evaluation tests for genai-tk, including fake models, real-model gates, and trajectory tests. |
docs/TESTING_GUIDE.mddocs/evaluation_testing.mdtests/conftest.pytests/utils/pyproject.toml dependency groups dev and evals| Tier | Path | Use for |
|---|---|---|
| Unit | tests/unit_tests/ | Fast deterministic behavior and config parsing |
| Integration | tests/integration_tests/ | CLI wiring, real services, browser, end-to-end flows |
| Eval | tests/eval_tests/ | LLM quality, trajectory, multiturn behavior |
| Utilities | tests/utils/ | Shared fake data, factories, constants |
GENAITK_PROFILE=pytest for deterministic tests.pytest.importorskip() for optional packages.uv run pytest tests/unit_tests -q
uv run pytest tests/integration_tests -q
uv run pytest tests/eval_tests -q
just test-unit
just test