# Geppetto
Geppetto is the runtime core for building LLM applications in Go.
Current project direction is:
- provider-agnostic inference engines,
- first-class tool calling and tool-loop orchestration,
- middleware-based runtime composition,
- typed turn/block data model,
- hard-cutover profile registries with stack resolution and provenance,
- native JavaScript bindings through Goja (`require("geppetto")`).
This repository is primarily a library + examples repo. Downstream applications (for example pinocchio and go-go-os) consume Geppetto as runtime infrastructure.
## Current Model (Important)
Geppetto now uses a registry-first, profile-first runtime model:
- runtime config is resolved from profile registries (not ad-hoc legacy profile maps),
- registry sources are stackable (`yaml`, SQLite file, `sqlite-dsn`),
- top-of-stack precedence is deterministic,
- profile runtime now carries prompt/tool/middleware metadata only,
- resolved runtime includes stack lineage/trace metadata and runtime fingerprint.
There is no overlay abstraction in active runtime composition.
Profiles do not configure engine/provider credentials. Applications own final `StepSettings` and pass explicit engine config into Geppetto.
## Core Runtime Building Blocks
- `pkg/turns`: canonical conversation model (`Turn`, `Block`, typed keys).
- `pkg/inference/engine`: provider engine interfaces and implementations.
- `pkg/inference/toolloop`: orchestration loop for tool calls/results/retries.
- `pkg/inference/middleware`: inference middleware pipeline.
- `pkg/inference/session`: session lifecycle and history.
- `pkg/profiles`: profile registries, stack resolution, policy, storage backends.
- `pkg/js/modules/geppetto`: JS module exposed through Goja.
## Multimodal image inputs
Geppetto's turn model supports user multimodal blocks via `turns.NewUserMultimodalBlock(...)`.
That helper stores text plus image payloads under `turns.PayloadKeyImages`. The current image map shape is:
- `media_type` for inline content
- either `url`, `content`, or provider-specific `file_id`
- optional `detail` for providers that support image detail selection
At the provider layer:
- OpenAI Chat Completions serializes those image entries as `image_url` content parts.
- OpenAI Responses serializes them as mixed `content` arrays containing `input_text` plus one or more `input_image` parts.
- The OpenAI Responses token-count path (`/responses/input_tokens`) reuses the same request builder, so image-bearing turns are counted with the same request shape used for inference.
- Anthropic Claude serializes inline image bytes as base64 `image` content; Gemini (modern `genai` path) uses `InlineData` blobs or `file_uri`.
Reading images back: always use `turns.BlockImages(block)` (or `turns.ImagesFromPayload(payload)`) instead of asserting `payload["images"].([]map[string]any)`. After a turn has been persisted and reloaded (YAML/JSON), the slice decodes as `[]any`; the accessor accepts every shape and the provider adapters all use it, so images survive history reloads. Image-only user messages (no text) are supported by all adapters.
For sessions, `session.AppendNewTurnFromUserMessage(text, images)` clones the latest turn and appends a text+images user block, mirroring `AppendNewTurnFromUserPrompt`.
See also:
- [`pkg/doc/topics/06-inference-engines.md`](pkg/doc/topics/06-inference-engines.md)
- [`pkg/doc/topics/08-turns.md`](pkg/doc/topics/08-turns.md)
## Profile Registries
Runtime registry sources are loaded via `profile-registries` (CLI flag, env, or config depending on host app).
Supported source entries:
- `path/to/registry.yaml`
- `path/to/profiles.db`
- `sqlite-dsn:file:./profiles.db?...`
Single YAML format is one registry per file:
```yaml
slug: team
profiles:
default:
slug: default
runtime:
system_prompt: You are the team default assistant.
tools:
- search
```
For details:
- [`pkg/doc/topics/01-profiles.md`](pkg/doc/topics/01-profiles.md)
- [`pkg/doc/playbooks/05-migrate-legacy-profiles-yaml-to-registry.md`](pkg/doc/playbooks/05-migrate-legacy-profiles-yaml-to-registry.md)
- [`pkg/doc/playbooks/06-operate-sqlite-profile-registry.md`](pkg/doc/playbooks/06-operate-sqlite-profile-registry.md)
## JavaScript API
Geppetto exposes a native module inside Goja:
```javascript
const gp = require("geppetto");
```
Main namespaces:
- `gp.turns`
- `gp.engines`
- `gp.profiles`
- `gp.schemas`
- `gp.middlewares`
- `gp.events`
- `gp.tools`
`gp.profiles` supports both host-injected registries and runtime stack binding:
- `connectStack(sources)`
- `disconnectStack()`
- `getConnectedSources()`
Use the JS lab harness:
```bash
go run ./cmd/examples/geppetto-js-lab --script examples/js/geppetto/01_turns_and_blocks.js
```
Run full JS profile/registry examples:
```bash
./examples/js/geppetto/run_profile_registry_examples.sh
```
References:
- [`pkg/doc/topics/13-js-api-reference.md`](pkg/doc/topics/13-js-api-reference.md)
- [`pkg/doc/topics/14-js-api-user-guide.md`](pkg/doc/topics/14-js-api-user-guide.md)
- [`examples/js/geppetto/README.md`](examples/js/geppetto/README.md)
## Quick Start: Go Examples
```bash
# simple inference pipeline
go run ./cmd/examples/inference
# streaming output
go run ./cmd/examples/streaming-inference
# opinionated runner examples
go run ./cmd/examples/runner-simple
go run ./cmd/examples/runner-tools
go run ./cmd/examples/runner-streaming
go run ./cmd/examples/runner-registry
go run ./cmd/examples/runner-glazed-full-flags
go run ./cmd/examples/runner-glazed-registry-flags
# advanced low-level examples
go run ./cmd/examples/advanced/middleware-inference
go run ./cmd/examples/advanced/generic-tool-calling
go run ./cmd/examples/advanced/openai-tools
go run ./cmd/examples/advanced/claude-tools
# JS host harness
go run ./cmd/examples/geppetto-js-lab --list-go-tools
```
All runnable examples are under `cmd/examples/`.
See [`cmd/examples/README.md`](cmd/examples/README.md) for the simple-vs-advanced split.
The two Glazed examples intentionally show different boundaries:
- `runner-glazed-full-flags` exposes full Geppetto runtime sections and builds `StepSettings` directly from parsed Glazed values.
- `runner-glazed-registry-flags` exposes only `profile` and `profile-registries`; base `StepSettings` stay app-owned and hidden, which is the preferred small-CLI pattern.
## Documentation
Start with:
- [`pkg/doc/topics/00-docs-index.md`](pkg/doc/topics/00-docs-index.md)
High-value pages:
- profiles: [`pkg/doc/topics/01-profiles.md`](pkg/doc/topics/01-profiles.md)
- engines: [`pkg/doc/topics/06-inference-engines.md`](pkg/doc/topics/06-inference-engines.md)
- tools: [`pkg/doc/topics/07-tools.md`](pkg/doc/topics/07-tools.md)
- middlewares: [`pkg/doc/topics/09-middlewares.md`](pkg/doc/topics/09-middlewares.md)
- sessions: [`pkg/doc/topics/10-sessions.md`](pkg/doc/topics/10-sessions.md)
- JS API reference: [`pkg/doc/topics/13-js-api-reference.md`](pkg/doc/topics/13-js-api-reference.md)
## Repository Layout
- `pkg/` runtime packages
- `cmd/examples/` runnable binaries
- `examples/js/geppetto/` JS scripts for API coverage
- `pkg/doc/` docs, playbooks, tutorials
- `cmd/tools/` code generation and developer tooling binaries
## Development
Requirements:
- Go `1.25.7` (see `go.mod`)
Common commands:
```bash
go test ./...
go generate ./...
go build ./...
```
TypeScript declaration generation/check for `require("geppetto")`:
```bash
make gen-dts
make check-dts
```
`make check-dts` exits non-zero when `pkg/doc/types/geppetto.d.ts` is out of sync with the generator/template inputs, so it is suitable for CI drift checks.