Create or update a provider mode for this repository without reintroducing hardcoded runtime wiring.
-
Choose the launcher shape.
- If the provider behaves like an existing plain launcher, prefer extending existing server metadata instead of adding new runtime branches.
- If the provider needs custom model resolution, middleware, request rewriting, or special launch flags, create
easyllama/servers/<provider>.py.
-
Put runtime mode metadata on the server layer.
- Add or extend
@server_metadata(...) on the relevant server class.
- Declare the mode name, Docker target, and build-source repo/ref mappings there.
- Register any new server class in
easyllama/servers/__init__.py.
- Do not add new mode-specific
if branches or hardcoded mode lists in runtime.py, config.py, or cli.py when registry metadata can drive the behavior.
-
Wire project defaults and config files.
- Update
pyproject.toml defaults for new repo/ref settings when the provider introduces new upstream sources.
- Add
config/config.<provider>.yml.example and make sure the mode can resolve its active and example config paths.
- If live validation needs pinned local settings, copy the example to an ignored
config/config.<provider>.yml, but keep the tracked example as the release artifact.
-
Add Docker build and runtime targets.
- Add a builder stage and runtime target in
Dockerfile.
- Copy the required binary and any companion assets into the runtime image.
- Expose the runtime binary under
/app/bin/llama-server-<provider> or the provider-specific equivalent.
-
Update documentation in one pass.
- Update
README.md for the mode overview, default model table, commands, and config file list.
- Update
API.md for endpoint coverage and request examples.
- Keep docs release-ready: describe tracked templates and shipped behavior, not ignored local overrides or stale migration details.
- If behavior or config shape changes, update the matching
config/config.<mode>.yml.example in the same change.
-
Run code-level validation.
Use the code validation script. It runs the repo's narrow host-side checks for run.sh, easyllama, and the CLI surface from the repository root.
If the repo lacks dedicated unit tests for the provider path, treat diagnostics plus the runtime endpoint suite as the required gate.
-
Validate the mode YAML before any rebuild.
Use the mode YAML validation script against the config that live validation will actually use for that mode. It fails fast on YAML parse errors, duplicate macro keys in macros:, and unresolved ${...} references other than ${PORT} and ${env.*}.
The rebuild and warmup script runs this check automatically before ./run.sh --mode <mode> build, so fix config issues there before spending time on Docker rebuilds.
-
Rebuild and warm the provider mode.
Use the rebuild and warmup script. Pass the mode name as the first argument and optional model IDs after it. If you omit model IDs, the script warms every model currently exposed by /v1/models.
Rebuild whenever Python runtime code, Docker targets, launcher code, or config-loading behavior changed, because the runtime is baked into the image.
-
Run the public endpoint regression suite.
Use the public endpoint regression script. Pass the mode name as the first argument. The script covers GET /health, GET /v1/models, POST /v1/chat/completions, POST /v1/completions, POST /v1/responses, POST /v1/embeddings, and GET /ui/. It checks POST /v1/messages for lucebox, the only shipped mode with a project-owned messages route. Use --messages only when validating a newly implemented equivalent route.
The script validates minimal response shape, not just status codes: advertised model IDs, assistant content for chat-style responses, and non-empty embedding vectors.
-
If validation fails, debug locally before expanding scope.
