build-images

# Building VM Images ## Overview Capsem image builds are profile-led. - `config/profiles/<profile_id>/profile.toml` is the profile ledger. - Profile sibling files own packages, MCP declarations, rule files, detection files, tips, build-time hooks, and packaged guest root seed files. - `capsem-admin` validates profile-owned inputs and materializes the generated backend build workspace. - The Python builder backend renders Docker templates and emits assets, build ledgers, and OBOMs only when invoked by the admin build rail. Do not add product truth directly to the backend image-spec path. ## Source Layout Read `config/README.md` before changing this layout. ``` config/ settings/ UI/application preferences and generated UI schema corp/ Corporate source contracts and rule files docker/ Dockerfile/build templates profiles/<profile_id>/ profile.toml Source ledger; no hash/size pins enforcement.toml Profile enforcement rules detection.yaml Profile Sigma detections mcp.json Profile MCP declarations apt-packages.txt Profile apt package input python-requirements.txt / python-requirements.lock npm-packages.txt / npm-package-lock.json build.sh Profile image build hook tips.txt Profile guest tips root/ Guest / seed, projected by capsem-init cache/target/config/ Generated runtime config with asset/file evidence cache/target/assets/ Generated VM assets cache/target/packages/ Generated native packages guest/artifacts/ Core guest payloads: init, doctor, diagnostics, bench ``` The materialized backend workspace may contain generated package-set files and profile build scripts. Treat those as implementation details, not authoring surfaces. The workspace is never a config root and never a second profile catalog. `capsem-admin` is a tool, not a config authority. It validates, materializes, builds, and checks the profile/corp/settings contracts; it must not grow scaffolding commands that invent profile, MCP, AI provider, package, or rule truth outside `config/profiles`, `config/corp`, and `config/settings`. Do not add admin config roots, guest config roots, settings metadata, provider registries, or backend-owned profile catalogs as product truth. `schema` validates one contract, `catalog` lists materialized profile instances, and UI metadata only helps render settings. ## CLI commands ```bash just _build-assets code [arch] # Profile-derived asset rebuild just _build-kernel arm64 code # Kernel slice just _build-rootfs arm64 code # Rootfs slice uv run --project build_system --frozen capsem-builder audit # Parse trivy/grype vulnerability output ``` Use admin/just recipes for all product image work. `capsem-builder` is a backend helper only; it must not expose or document public `build`, `validate`, `inspect`, `mcp`, render-only, or dry-run rails for profile/image authoring. `capsem-admin image build` may call private Python modules such as `capsem.builder.image_build_backend`; agents must not make those modules public CLI contracts. ## Building assets Full rebuild (kernel + rootfs): ```bash just _build-assets # Runs doctor + validate + build for host arch ``` Individual templates: ```bash just _build-kernel arm64 just _build-rootfs arm64 ``` ## Per-arch asset layout ``` cache/target/assets/ manifest.json Version, checksums, asset list B3SUMS BLAKE3 checksums arm64/ vmlinuz Kernel rootfs.erofs Root filesystem initrd.img Initial ramdisk (repacked by just exec) ``` Rootfs EROFS settings are profile-derived. The approved release default is EROFS with `lz4hc` compression level 12. `config/docker/image/build.toml [build.rootfs]` also owns two independent release budgets: the exported tar ceiling and the final EROFS ceiling. The builder checks the first before compression and the second before writing the artifact ledger. It also scans every exported member against the configured forbidden prefixes. Keep Ollama accelerator families forbidden under both `/usr/lib/ollama` and `/usr/local/lib/ollama`: vendor archives have moved the bundle between those roots, and the guest exposes no accelerator device. A profile hook should remove the payload for efficiency, while the exported-tree scan remains the fail-closed proof. Never raise a size ceiling to accommodate an unexplained image jump; inspect the rootfs composition first. ## Build Ledger Each per-arch build emits `build-ledger.log` JSONL. The `rootfs.config_inputs` record captures declared profile package inputs, rendered rootfs package lists, profile root/build-script inputs, EROFS config, git revision, and project version. Installed-package/component truth belongs in the CycloneDX OBOM, not the build ledger. ## Profile Source And Generated Evidence Profile sibling files are ledgered source inputs, but agents must not add or hand-edit `hash` or `size` fields in `profile.toml`. If editing `apt-packages.txt`, `python-requirements.txt`, `npm-packages.txt`, `build.sh`, rules, MCP declarations, tips, or root seed files makes `capsem-admin profile check` fail, fix the source contract or the validation/materialization rail with tests. Do not "just fix the hash" in TOML. Generated runtime asset URLs/hashes belong in `cache/target/config` after `capsem-admin profile materialize`, not in checked-in source TOML. Profile materialization must recopy descriptor files and `root/` payloads from source on every run; stale generated roots are a release blocker, not a cache. ## Adding packages to the VM 1. Edit the profile-owned package file, for example `config/profiles/code/apt-packages.txt`, `python-requirements.txt`, or `npm-packages.txt`. Python and npm direct selections must be exact, and their checked-in integrity locks must be regenerated together; never install from the direct list alone. 2. Run the admin/profile validation path. 3. Run `just _build-assets code` to rebuild the rootfs. 4. Verify with `capsem-doctor` inside a booted VM. Do not edit generated Dockerfiles. Docker templates live under `config/docker/`. ## Adding a guest CLI/tool There are no image-owned AI providers. A CLI/tool exists only if the active profile declares the package/build hook and any required guest root seed files. 1. Add package input to the profile package files, or add build-time shell work to profile-owned `build.sh`. 2. Add config files under `config/profiles/<profile_id>/root/` so they project into the VM at boot. 3. Add MCP declarations to profile-owned `mcp.json` when relevant. 4. Add network/model/security behavior through profile/corp rules, not builder provider config. 5. Let the credential broker plugin capture/materialize credentials at runtime; do not add settings-owned boot secrets. 6. Rebuild with `just _build-assets code` and verify with `capsem-doctor`. `build.sh` is executed only while constructing the rootfs image. It is the right place for digest-bound official binary archives such as Claude, AGY, or Ollama when they cannot be represented as apt/npm/Python package inputs. Never pipe or invoke a mutable installer endpoint. Install verified runtime binaries under system paths such as `/usr/local/bin`; anything left only under `/root` can be hidden by the runtime overlay. ## Profile `build.sh` contract Remember this rail when touching profile image contents: - `config/profiles/<profile_id>/build.sh` is a profile-owned build hook. - It runs inside the rootfs Docker build, before the EROFS image is produced. - It does not run during native glow-up, service startup, VM boot, or user session creation. - It is for image construction work that cannot be cleanly expressed through `apt-packages.txt`, `python-requirements.txt`, or `npm-packages.txt`. - It may install public runtime tools such as Claude, AGY, and Ollama into stable system paths. - It is not a second profile format, provider registry, runtime settings file, credential injection path, or local developer repair script. - It must not bake credentials, per-user state, corp policy, rules, MCP decisions, or runtime settings. - The owning `profile.toml` must reference it through `[files.build]`; the descriptor hash/size is refreshed by the profile-derived build rail, never by hand. - Changing `build.sh` changes future rootfs assets only. Rebuild assets through the profile-derived just/admin-tool rail before claiming a VM contains the change. - The same profile materialization path must be used locally and in CI; no one-off Docker or installer path is release proof. - Verification must be black-box: boot the rebuilt profile image, run the tool from the VM, and inspect the generated session evidence when the tool should produce network, model, MCP, file, process, or credential events. Decision rule: - Normal Debian package: use `apt-packages.txt`. - Normal Python package: use `python-requirements.txt`. - Normal npm package: use `npm-packages.txt`. - Vendor shell installer, binary tarball, wrapper creation, or cleanup that must happen while baking the immutable rootfs: use `build.sh`. - Anything that depends on user/corp/runtime state: do not use `build.sh`. ## Dockerfile templates Templates live in `config/docker/`: - `Dockerfile.rootfs-dependencies.j2` -- snapshot-selected Debian packages and the profile's network-resolved Python/npm/vendor inputs; - `Dockerfile.kernel-dependencies.j2` -- snapshot-selected kernel toolchain and the SHA-256-verified kernel archive; - `Dockerfile.rootfs.j2` -- network-denied first-party rootfs assembly; - `Dockerfile.kernel.j2` -- network-denied kernel compilation, initrd assembly, and vmlinuz extraction. `asset-dependencies` is the visible resumable frontier between those pairs. The gate materializes one input-keyed helper for every selected profile/architecture/template, validates its platform and identity label, and passes only its exact image ID to the source build. Source builds always use BuildKit network `none` and never use the remote CI cache. A carried frontier must revalidate every helper; it must not silently rebuild inside the sealed lane. The dependency helpers reuse the one checked-in Debian snapshot authority rather than the mutable sources inherited from the base image. Templates use Jinja2 with variables from the admin-materialized profile image workspace. Do not add a second preview rail for product truth; if a build input needs validation, add it to the normal profile/admin validation path. Every architecture's `base_image` is required to be its immutable `repository@sha256:<child-manifest>` identity. Do not use a mutable tag or the multi-platform index. `capsem-gate` materializes a missing exact child through the Docker daemon before the cross-execution probe and build lanes; keep this on the ordinary guarded Docker runner, not the host-process egress broker. --- # Builder Internals (for modifying the builder itself) ## Architecture: Profile -> admin materialization -> Pydantic -> context dict -> Jinja2 -> Dockerfile The data flows through four layers: 1. **Profile ledger** (`config/profiles/<id>/profile.toml`) and profile-owned sibling files. 2. **capsem-admin** validates and materializes a backend build workspace. 3. **Pydantic models** (`src/capsem/builder/models.py`) parse that workspace. 4. **Context dict** (`src/capsem/builder/docker.py`) feeds Jinja2 templates. 5. **Jinja2 templates** (`config/docker/`) produce Dockerfiles. ### Key files | File | Role | |------|------| | `src/capsem/builder/models.py` | All Pydantic models (enums, configs, top-level `GuestImageConfig`) | | `src/capsem/builder/config.py` | Backend loader for admin-materialized build workspaces | | `src/capsem/builder/docker.py` | Context builders (`_rootfs_context`, `_kernel_context`), rendering, build execution | | `src/capsem/builder/image_build_backend.py` | Private admin-invoked image build backend; not a public CLI | | `config/docker/Dockerfile.rootfs.j2` | Rootfs Dockerfile template | | `config/docker/Dockerfile.kernel.j2` | Kernel Dockerfile template | | `src/capsem/builder/validate.py` | Validation rules (E001-E302, W001-W012) | | `src/capsem/builder/cli.py` | Click CLI entry points | ### Context dict (rootfs template variables) `_rootfs_context()` in `docker.py` builds the dict passed to `Dockerfile.rootfs.j2`: ```python { "arch": ArchConfig, # Per-arch settings (docker_platform, rust_target, etc.) "arch_name": str, # "arm64" or "x86_64" "apt_packages": list[str], # Materialized from profile apt-packages.txt "python_packages": list[str], # Materialized from profile python-requirements.txt "python_install_cmd": str, # e.g. "uv pip install --system --break-system-packages" "npm_packages": list[str], # Materialized from profile npm-packages.txt "profile_root_seed": bool, # Whether profile-root/ is copied into the image "profile_build_script": bool, # Whether profile-build.sh is executed "npm_prefix": str, # e.g. "/opt/ai-clis" "guest_binaries": list[str], # ["capsem-pty-agent", "capsem-net-proxy", "capsem-mcp-server"] } ``` ### Kernel context dict ```python { "arch": ArchConfig, "arch_name": str, "kernel_version": str, # exact checked-in release, e.g. "6.6.130" "kernel_sha256": str, # verified before the source archive is extracted } ``` ## How to: Change a shipped CLI 1. Prefer a profile package file (`apt-packages.txt`, `npm-packages.txt`, or `python-requirements.txt`) when the tool has a normal package manager. 2. Use profile-owned `build.sh` when the vendor ships an official shell installer. The build hook runs during rootfs construction only. 3. Make sure binaries end up in stable system paths such as `/usr/local/bin`. 4. Validate and materialize through `capsem-admin`; if the rail cannot express the change, implement it with tests first. 5. Add or update capsem-admin materialization tests and Docker context tests. 6. Rebuild: `just _build-assets code` and verify with `capsem-doctor`. Ollama is intentionally installed by `config/profiles/<id>/build.sh`, not by a VM one-off command. That keeps Codex, Claude, AGY, and OpenAI-compatible local testing available in every shipped profile image that declares the hook. ## How to: Add a new package to an existing set 1. Edit `config/profiles/<profile_id>/apt-packages.txt`, `python-requirements.txt`, or `npm-packages.txt`. 2. Validate and materialize through `capsem-admin`. 3. Keep the checked-in profile source free of generated hashes or sizes. 4. Rebuild: `just _build-assets <profile_id>`. ## How to: Add a new guest binary Guest binaries are compiled from `crates/capsem-agent/`. Every architecture on every host goes through `container_compile_agent()`; native Linux has no second ambient Cargo/rustup rail. The asset preflight first materializes the host platform's config-selected exact Rust child image plus the checked-in Rust toolchain and `Cargo.lock`; a foreign target also materializes its exact config-pinned C compiler package and both the foreign-target and host-target Cargo dependency sets. The host set covers target-gated build-script and proc-macro dependencies that `cargo fetch --target <foreign>` does not select. The actual container build runs with `--network none` and Cargo `--locked --offline`. Materialize only helpers the command can consume: every requested architecture on either supported host, and none for a kernel-only build. Immutable Debian guest bases remain architecture-selected separately. 1. Add the binary target in `crates/capsem-agent/Cargo.toml` 2. Add the binary name to `GUEST_BINARIES` list in `docker.py` 3. The template already loops `{% for binary in guest_binaries %}` to COPY + chmod 555 ## Verifying Linux builds locally `just _cross-compile [arch]` builds everything in a container: agent binaries, frontend, and the full Linux `.deb` package. Useful for catching system dependency issues before CI. ```bash just _cross-compile # Build for host arch (arm64 on Apple Silicon) just _cross-compile x86_64 # Build x86_64 deb ``` ## Backend Workspace Schema The backend workspace is generated by `capsem-admin`; do not author it by hand for product behavior. Its install inputs are package-set TOML files: ```toml [npm] name = "Node Packages" manager = "npm" install_cmd = "npm install -g --prefix /opt/ai-clis" packages = ["@scope/package"] ``` Profiles own CLI/tool selection. If an installer cannot be represented as a package set, put it in `config/profiles/<profile_id>/build.sh`, reference it from `[files.build]` in `profile.toml`, refresh pins with `capsem-admin`, and rebuild through the admin/just rail. Do not add a provider registry under backend-generated image workspaces. ## Build pipeline (what `build_image()` does) For rootfs: 1. Build guest agent binaries (`cross_compile_agent` -- every target uses the pre-materialized, network-denied Rust builder; a foreign target cross-compiles on the host CPU) 2. Assemble build context (`prepare_build_context`) -- copies CA cert, shell configs, diagnostics, agent binaries 3. Render Dockerfile from template 4. `docker build` 5. Export container filesystem as tar 6. Create EROFS from tar (`create_erofs` -- runs mkfs.erofs in a container) 7. Extract tool versions (`extract_tool_versions`) 8. Clean up container image For kernel: 1. Read the exact kernel version and SHA-256 from the checked-in build config 2. Assemble build context (defconfig, capsem-init) 3. Render Dockerfile from template 4. `docker build`, verifying the downloaded source archive before extraction 5. Extract vmlinuz + initrd.img from image 6. Clean up ## The guest Rust builder workspace: do not widen the `/src/*` glob `container_compile_agent` mounts the checkout read-only at `/src` and assembles a writable workspace at `/build`: ```sh for f in /src/*; do b=$(basename "$f"); \ [ "$b" != target ] && [ "$b" != crates ] && ln -s "$f" /build/; done ``` `/src/*` does not match dotfiles. **That exclusion is load-bearing. Widening it breaks the build.** It reads like an oversight -- `.cargo/config.toml` never reaches the container, so the checked-in Cargo configuration is never applied --