opengeni

# OpenGeni This skill is for repo-maintainer agents working in, operating, or changing the OpenGeni repository. ## Overview Use this skill as an orientation layer, not as frozen API documentation. OpenGeni evolves through the codebase, so code wins over this skill whenever they differ. OpenGeni is a workspace-scoped agent control plane. Public clients talk to an API. The API resolves every protected request to an access grant, persists sessions and events, accepts user/control events, exposes replay/SSE streams, handles uploads, and talks to Temporal. A worker runs OpenAI Agents SDK turns inside a configured sandbox backend. Postgres is durable state, NATS is live fanout, object storage holds uploaded file bytes, Temporal coordinates work and schedules, and MCP servers provide pluggable tools. Canonical source repo: `https://github.com/Cloudgeni-ai/opengeni`. This skill may be installed outside that repo, which is normal. If the current workspace is not OpenGeni, first determine whether the user wants client integration against a deployed OpenGeni service, source-level changes, deployment help, or conceptual explanation. For source-level exactness, inspect or fetch the repo; for client integration, ask for or infer the deployed API base URL and inspect the running API/client config where possible. ## Start With Repository Discovery Always inspect the current repo before making claims or changes. Use fast searches and prefer contracts/types/routes over README prose when exact behavior matters. Useful first pass: ```bash rg --files -g '!*node_modules*' -g '!*.lockb' rg -n "app\\.(get|post|patch|delete|all)\\(|/v1/|SessionEvent|ResourceRef|ToolRef|SandboxBackend|workflow|activity|OPENGENI_|mcp|schedule|upload|objectStorage|NATS|Temporal" \ -g '!*node_modules*' -g '!*.lockb' ``` Then open the smallest source files that answer the question: - API routes: `apps/api/src/routes/`, plus `apps/api/src/app.ts` and `apps/api/src/index.ts`. - Core domain/access/billing helpers: `packages/core/src/` (`access/`, `domain/`, `billing/`, and `dependencies.ts`). These moved out of `apps/api`; API routes are HTTP adapters over `@opengeni/core`. - Public shapes: `packages/contracts/src/index.ts`, especially workspace, access, billing, usage, session, file, document, schedule, and MCP contracts. - External host identities and credential authority: `packages/core/src/access/`, `packages/contracts/src/external-identities.ts`, and `packages/db/src/connection-authority.ts`. Verified external owning-user authority is distinct from a managed login cookie. Integrating backends provision ordinary native connections under `asUser`; interactive and backend setup share ownership and refresh machinery. Accepted turns/tasks capture exact named-user connection selection; execution and physical requests recheck live authority. Shared turns cannot borrow a creator's credentials. Preserve captured initiatingHumanSubjectId for schedules and child work, independently of request-time service administration. Host-specific registry, callback and selection APIs are retired; do not restore them as a parallel framework. Historical tables are not executable authority. The request-time gateway uses native connections with live caller checks, not fabricated sessions. See `docs/remote-mcp-credentials.md` for cutover status. - Config/env: `packages/config/src/index.ts`, `.env.example`, `README.md`, `AGENTS.md`. - Run lifecycle / goals / memory: `docs/run-lifecycle.md`, `docs/goals.md`, plus `apps/worker/src/workflows/session.ts` and `apps/worker/src/activities/agent-turn/`. - Feature subsystems: `docs/variable-sets.md` (scoped organization/workspace/user secrets), `docs/capabilities.md` (Plugins, Skills, and the MCP catalog), and `docs/automations.md` (authenticated event sources, immutable triggers, logical runs, and ordinary-session dispatch). - Feedback: `docs/feedback.md`, `apps/api/src/routes/feedback.ts`, and `packages/db/src/feedback.ts` own authenticated general comments and session/turn ratings, separate from agent context. - Database/state: `packages/db/src/schema.ts`, `packages/db/src/index.ts`, `packages/db/drizzle/`. - Event bus/SSE: `packages/events/src/index.ts`, `apps/api/src/http/sse.ts`. - Worker/orchestration: `apps/worker/src/workflows/`, `apps/worker/src/activities/`. Physical finalization after execution has a five-minute per-stage containment deadline on normal and cancelled exits; `agent-turn/finalization-monitor.ts` owns the bounded stage heartbeat/metrics. This is never a limit on agent execution. Closed-attempt writers still gate successors; adopted background commands retain their independent lifetime. - Startup telemetry: `apps/worker/src/observability-metrics.ts` separates blocking preparation from background MCP work. Phase durations can overlap; use durable milestones for elapsed startup latency. Runtime stream initialization is not the provider wire-dispatch milestone. See `docs/run-lifecycle.md`. - Runtime/sandbox/tools: `packages/runtime/src/index.ts` is the public agent-loop facade; retained Modal command observation uses the versioned task-router byte-offset boundary in `sandbox/providers/modal-command-control.ts`; output and cursor capture is atomic in `packages/db/src/retained-provider-commands.ts`. Never reinterpret a legacy batch locator or infer exit from missing output. `skill-catalog.ts` renders Skill descriptors into the turn-attempt instruction layer; the worker's `skill-read.ts` exposes eager text reads and `skill-checkout.ts` exposes on-demand filesystem copies. Repository Skill discovery is independent. `packages/runtime/src/model-provider.ts` is the package-private model-provider facade over cohesive client, error, request-policy, routing, and transport leaves beside it; `packages/runtime/src/model-input.ts` owns final model-wire shaping and context guards; `packages/runtime/src/run-events.ts` owns SDK stream/usage/interruption normalization. - Files/object storage: `apps/api/src/routes/files.ts`, `packages/storage/src/index.ts`. - Deployment/operator sources: `packages/deployment`, `docs/deployment.md`, `deploy/helm/opengeni`, `deploy/terraform/`, and `deploy/stacks/`. - Knowledge, selective source retention, collection/duplicate discovery (`knowledge_prepare_save`), retrieval and learning policy: read `docs/knowledge.md`, then `apps/api/src/routes/knowledge.ts`, `packages/core/src/domain/knowledge*.ts`, and `packages/db/src/knowledge-entries.ts`. Old Memory and reviewed-Knowledge authoring are retired; conversation history and temporary task notes remain separate. - GitHub integration: `apps/api/src/routes/github.ts`, shared workspace filtering in `apps/api/src/github-access.ts`, `packages/github/src/index.ts`, and the binding/allowlist helpers plus tables in `packages/db/src/index.ts` / `schema.ts`. The separately configured OpenGeni Lens GitHub App reuses the same owner-proof primitives through `apps/api/src/routes/pr-review-github.ts`; its shared webhook still enters the generic PR-review automation source. - Connected Machine (bring-your-own-compute / the `selfhosted` backend): API routes `apps/api/src/routes/machines.ts` and `apps/api/src/routes/enrollments.ts`; services `apps/api/src/sandbox/machines.ts` and `apps/api/src/sandbox/enrollment.ts`; the machine-primary turn branch in `apps/worker/src/activities/agent-turn/sandbox-establish.ts` and the clone-guard in `packages/runtime/src/index.ts`; the runtime session at `packages/runtime/src/sandbox/selfhosted/`; the on-machine agent + relay in the `agent/` Rust crate; public behavior in `docs/connected-machines.md`; opt-in UI at the `@opengeni/react/machines` subpath. - Web usage examples: `apps/web/src/api.ts`, `apps/web/src/types.ts`, relevant UI components. - TypeScript SDK: `packages/sdk/src/` (typed client, SSE streaming core with reconnect/replay-by-sequence, proxy re-streaming helpers) and `packages/sdk/README.md`. - React hooks + styled components: `packages/react/src/` (hooks on the SDK, timeline projection, ChatComposer/MessageTimeline/SessionStatus/FleetTile, CSS-variable design tokens in `packages/react/styles/`) and `packages/react/README.md`; runnable harness under `packages/react/demo/`. - Standalone product integration: `docs/product-integration.md` is canonical for organization API keys, organization workspaces, Personal-workspace exclusion, external tenant mapping, and external Skill ownership. If paths have moved, find concepts by symbol name, not by old paths: ```bash rg -n "CreateSessionRequest|ClientSessionEvent|sessionWorkflow|runAgentTurn|createSandboxClient|buildManifest|createObjectStorage|build.*McpServer|getSettings" ``` ## Client Integration For external clients, SaaS integrations, SDK wrappers, customer-side coding agents, or UIs on top of OpenGeni, prefer the separate `opengeni-client` skill when available. For TypeScript clients, `@opengeni/sdk` (`packages/sdk`) is the first-party client: typed session/event API, the SSE streaming core (reconnect + replay-by-sequence + dedup), and proxy-through-your-own-API re-streaming helpers. When staying inside this source-level skill, read `references/client-integration.md`. Treat OpenGeni as a service boundary: the client discovers or chooses a workspace, creates sessions under `/v1/workspaces/:workspaceId/...`, streams/replays events, sends follow-up/control events, uploads files, selects resources/tools, and displays approvals/status. Do not require the client to know worker, Temporal, NATS, or sandbox internals except as concepts for status and product behavior. For the canonical external-product shape, the product backend holds an organization API key and calls `ensureWorkspace` / `PUT /v1/workspaces/external` for a stable product-tenant mapping. Call the result an organization workspace; its wire kind is `"shared"`. Personal workspaces are excluded. The external backend stores and versions product Skills and passes selected definitions inline in `CreateSessionRequest.skills` for each product-created session. Do not document or implement an organization-wide Skill registry or Skill inheritance for this boundary. See `docs/product-integration.md`. ## Access, Workspaces, Billing Keep these boundaries explicit: - Workspace scoping is core. Public operational routes use `/v1/workspaces/:workspaceId/...`; scoped Variable Sets are selected through that boundary but may be organization-, workspace-, or organization-user-owned. - External product backends use an **organization API key**. Organization key administration uses `/v1/organizations/:organizationId/api-keys`; the key is server-held and cannot reach Personal workspaces through organization provisioning. - An external product tenant maps to an **organization workspace**. The wire kind is `"shared"`; Personal workspaces (`"personal"`) are excluded from `ensureWorkspace` and must not be selected through a default-workspace fallback. - Old unscoped operational routes are deleted, not soft-deprecated. Do not add compatibility aliases unless the user explicitly changes that product decision. - Better Auth is only the managed-mode browser human auth resolver. It is not the tenant model and should not appear in core session/file/document/schedule route code. - Managed sign-in method changes belong to `apps/api/src/routes/managed-sign-in-methods.ts` and its canonical identity/database guards. Discover the current contract in `packages/contracts/src/managed-sign-in-methods.ts` and the rollout in `docs/browser-login-session-sets.md`. Preserve verified-email linking, explicit-disconnect suppression, expected-human/revision fences, recent authentication, and last-usable-method protection across every login route; do not expose raw provider mutation routes or use `trustedProviders` to bypass email verification. Login methods are distinct from integration permissions. - OpenGeni organization API keys are owned by OpenGeni and use the `Authorization` header. The optional deployment shared key uses `x-opengeni-access-key`. - Billing, Stripe, prepaid credits, entitlements, usage, and limits belong in billing/access modules. Core route/domain code should check local providers/interfaces, not call Stripe directly. - Product access mode (`local`, `configured`, `managed`) is separate from deployment/infrastructure profile (`azure-managed`, existing services, local Kubernetes, previews, and so on). - RLS is defense-in-depth. Do not claim RLS-backed isolation from app-level checks alone; verify policies with a non-owner DB role and current workspace/account settings. ## Mental Model Keep these concepts straight while working: - **Account**: managed billing/ownership container for members, workspaces, credits, and billing mirrors. - **Workspace**: operational data boundary for sessions, events, files, documents, schedules, GitHub installation bindings, usage, and first-party MCP. - **GitHub installation binding**: a workspace-local reference to a GitHub App installation plus its repository allowlist. One GitHub installation may be linked to many OpenGeni workspaces; unlinking one workspace must not mutate another workspace or uninstall the App from GitHub. New binding is currently fail-closed: setup callback parameters are spoofable, and user-installation visibility, repository administrator permission, and an installation request do not prove that the current human may install or configure the App for the target account. Existing trusted bindings are rechecked before platform token mint / GitHub-authenticated run startup. Connected Machines are exempt because they use their own git auth. - **Access grant**: resolved subject plus permissions for one workspace. Route code should depend on grants and permissions, not on the caller's auth mechanism. - **Session**: durable user-facing work container. It owns status, resources, selected tools, model/sandbox settings, event cursor, and active turn. Normal idle closes without a grace timer after durable rechecks and transactional parent-result settlement; late input can start another workflow run of the same session. Keep the legacy timer replay patch and unrelated lifecycle waits intact; see `docs/run-lifecycle.md`. - **Turn**: one queued/running unit of agent work inside a session, run as one non-retryable Temporal activity (`runAgentTurn`). Follow-ups, goal continuations, and scheduled task firings become turns. Inside a turn the SDK makes as many model/tool calls as the work needs; run length is bounded by symptoms (no-progress, budget), not by counts or clocks. A graceful worker shutdown preempts an in-flight turn (checkpoint, requeue, resume on a healthy worker) instead of failing the session. See `docs/run-lifecycle.md`. - **Sandbox rotation wait**: a recovering turn fenced by an active managed-sandbox rotation parks on its exact sandbox group and lease epoch. Every authoritative rotation-ending or epoch-advancing transaction durably wakes that waiter; the workflow does not repeatedly reserve turn-worker slots while the same transition remains pending. - **Goal**: optional durable per-session objective that flips "stop" into an explicit act — while active, the session workflow synthesizes continuation turns until the agent calls `goal_complete`/`goal_pause` or a user interrupts. The mechanism behind long-running autonomous runs. See `docs/goals.md`. Continuation is generated input, not new authority: resume established work against the current applied turn-frozen objective. Reuse only relevant, still-valid authoritative evidence; retain requested comprehensive audits and the full completion audit. Distinguish recoverable failures, definitive human blockers, and work in flight without fixed retry quotas or preliminary wait rituals. `docs/goals.md` owns the guidance scenarios; these instructions do not change runtime wake timing, child-result selection, or approval authority. - **Admission block**: a non-transient preclaim persistence rejection parks accepted work without failing or consuming it. Inspect `sessions.admission_block`, the worker classifier and `docs/run-lifecycle.md`; authorized Resume or new Send/Steer explicitly rechecks, never grants missing authority. Operational DB failures retain timed recovery. - **Control observation**: unavailable scoped reads are not deletion or idle truth; exact still-owned attempts are not successor admission. Versioned observers wait on signals/bounded control timers, retain outbox obligations, and inspect exact Temporal identity without replacing physical-writer proof. See `docs/run-lifecycle.md` before changing these paths. - **Session memory (three stores, three jobs)**: `session_history_items` is exact accepted conversation truth fed to the model (default read path); `agent_run_states` is the serialized RunState blob, used only to resume a turn paused for a human approval; `session_events` is the exact append-only human-audit timeline for accepted payloads and is never fed back to the model. Protocol/size projections are deterministic and must not classify or rewrite content. Sandbox recovery state lives separately in `sandbox_session_envelopes`. See `docs/run-lifecycle.md`. - **Variable Set**: named organization-, workspace-, or organization-user-owned collection of authenticated-encrypted secret env vars, attached to a session/scheduled-task and injected into the sandbox at run time. Attachment and runtime use require independent `variable-sets:attach` and `variable-sets:use` authority; exact plaintext access is a separate explicit permissioned operation with metadata-only audit. Never expose values through unrelated list/detail projections. See `docs/variable-sets.md`. - **Event log**: append-only session timeline with per-session sequence numbers. It supports replay, SSE reconnect, UI timeline projection, and auditing. - **SSE/NATS split**: Postgres is replay/source of truth. NATS is live fanout. If live events are missed, API should backfill from Postgres by sequence. - **Temporal**: orchestration, signals, timers, schedules, and worker dispatch. Token streams/tool output should not be pushed through workflow history unless the code intentionally changes that design. - **Worker activity**: side-effect boundary where the OpenAI Agents SDK actually runs. Treat model/tool/sandbox/cloud calls as side-effectful. - **Sandbox**: pluggable execution environment behind the OpenAI Agents SDK sandbox interface. OpenGeni should describe the contract and selected backend, not pretend the backend is hard-coded. The shipped `SandboxBackend` enum is broad (currently twelve members), so never claim it is only Docker/Modal/local/none. - **Connected Machine**: a user's own machine (enrolled through the `agent/` Rust agent) that acts as a first-class *primary* compute target, co-equal with the managed cloud sandbox — a sibling compute target, not a backend overlay bolted onto Modal. Its enum value is `selfhosted`. A machine-targeted turn establishes a `SelfhostedSession` directly and does NOT create, lease, or bill a cloud (Modal) box; the platform mints no GitHub token for it and never clones repos onto it (the machine uses its own git auth and already owns its filesystem). Runs execute at a per-session `workingDir` (default = the agent's launch dir), not a fixed `/workspace`. The whole feature is gated by `OPENGENI_SANDBOX_SELFHOSTED_ENABLED` (default off). See `docs/connected-machines.md` and `references/sandbox-configuration.md`. - **Resources**: external context mounted or made available to a run, commonly repositories and uploaded files. - **Tools**: currently MCP-first. Tool refs select configured MCP servers. Built-ins are defaults, not limits. - **Object storage**: stores uploaded bytes. Database stores metadata/object keys. Sandbox file access is normally via manifest/mount/injection based on current runtime code. - **Scheduled task**: persisted schedule plus agent config that dispatches one or more session turns through Temporal scheduling. - **Automation**: an authenticated external event accepted by a source and matched by an immutable trigger revision into one deduplicated logical run. Temporal dispatches an ordinary session; provider-specific review or incident features are adapters over this substrate. - **Knowledge**: canonical source content, findings and groups in Postgres, with original files in object storage and rebuildable search indexes. First-party `knowledge_*` tools use accepted Agent learning policy: Automatic publishes, Review first stages a nonblocking revision, and Off disables agent writes. Search defaults to published records; explicit `view: "needs_review"` lets agents inspect and update pending entries and collections without approving them. Pending content is unapproved context, never behavioral authority. Reuse IDs/current versions instead of duplicating proposals on each scheduled run. Connected-source schedules run ordinary agents with frozen source selections; source fetching is an attempt-bound tool. Conversation history and temporary task notes remain separate. See `docs/knowledge.md`. ## Source Discovery Workflow For architecture, documentation, implementation, debugging, or operational work, create a current picture from code: 1. Read `AGENTS.md` and `README.md` for operator intent and warnings. 2. Read contracts for names and public shapes. 3. Read API routes for current endpoints and behavior. 4. Read DB schema and event append/list code for durable state and ordering. 5. Read worker workflows/activities for orchestration and side effects. 6. Read runtime code for Agents SDK, sandbox, MCP, manifests, resume, and model provider behavior. 7. Read config parsing for environment variables and pluggability. 8. Mark every claim as shipped, configurable, delegated to a backend/provider, or roadmap/not shipped. 9. For workspace/auth/billing changes, run or inspect `scripts/check-workspace-billing-static.ts` and the workspace isolation integration test so old route/provider-boundary drift is caught. Do not rely on this skill for exact route lists, env var lists, event types, model names, or backend names. Re-discover those from contracts/config/routes every time exactness matters. ## Code Change Workflow Before editing, identify which layer owns the behavior: - Public API or validation: routes, core domain helpers, contracts, tests. - Persistence: DB schema, migrations, mapping functions, integration tests. - Event semantics: append helpers, event bus, SSE replay, frontend timeline handling. - Orchestration: Temporal workflow/signal/activity code and workflow tests. - Agent runtime: runtime package, worker activity, OpenAI Agents SDK integration tests. - Sandbox resources: resource validation, manifest building, object storage, sandbox environment. - MCP tools: config parsing, runtime tool preparation, API MCP servers. - Scheduling: scheduled task contracts/routes/core domain helpers, Temporal schedule mapping, dispatch activity.