<h1 align="center">Qiniu Sandbox GitHub Runner</h1>
<p align="center">
<strong>Ephemeral, isolated GitHub Actions runners powered by Qiniu Sandbox</strong>
</p>
<p align="center">
<a href="./README.zh.md">ไธญๆ</a> ยท
<a href="https://runner.qiniuinc.com/docs/getting-started/hosted">Hosted Guide</a> ยท
<a href="#quick-start">Quick Start</a> ยท
<a href="https://app-6a6b0d723d3a24e095531129.app.qiniucc.com/">Deploy to Qiniu LAS</a> ยท
<a href="https://runner.qiniuinc.com/docs/getting-started/deploy">Deployment Guide</a> ยท
<a href="https://runner.qiniuinc.com/docs">Documentation</a> ยท
<a href="#license">License</a> ยท
<a href="#community--contributing">Community & Contributing</a>
</p>
---
Qiniu Sandbox GitHub Runner provisions a clean [Qiniu Sandbox](https://www.qiniu.com/) for each GitHub Actions workflow job, registers a [self-hosted runner](https://docs.github.com/en/actions/hosting-your-own-runners/about-self-hosted-runners) just in time, and removes the runner and sandbox when the job ends. Teams keep the familiar GitHub Actions workflow while moving each job into a disposable environment.
The Qiniu CI Runner control plane is open source. Qiniu Sandbox, where workflow jobs execute, is a cloud service provided and operated by Qiniu.
## Core Capabilities
- **Ephemeral runners** โ one sandbox per job, automatically cleaned up after completion
- **GitHub App auth** โ recommended production path with OAuth sign-in for the built-in web console
- **Multi-database** โ SQLite (default), PostgreSQL, or MySQL for runtime state
- **Concurrency control** โ global `max_concurrent_runners` and per-spec `max_concurrency` with queue-based backpressure
- **Built-in web UI** โ admin console for runner requests, global runner specs, accounts, the platform Sandbox fallback, audit, matching, and diagnostics; ordinary-user console for job groups, logs, repository readiness, a platform Runner Spec catalog, scoped Sandbox management, and account/Organization custom Runner Specs
- **Config obfuscation** โ sensitive values can be hidden from casual config inspection
- **Retry & recovery** โ transient failures are retried with backoff; queued work and active remote runners are recovered after a service restart
## How It Works
```
GitHub webhook (workflow_job)
โ
โผ
โโโโโโโโโโโ create sandbox โโโโโโโโโโโโโโโโโโโโ
โ runnerd โ โโโโโโโโโโโโโโโโโโโโโโโบ โ Qiniu Sandbox โ
โ (server) โ register runner โ (ephemeral VM) โ
โ โ โโโโโโโโโโโโโโโโโโโโโโโบ โ โ
โโโโโโโโโโโ โ GitHub Actions โ
โ โ self-hosted โ
โ job completed / timeout โ runner โ
โโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโโ โ โ
โ stop & cleanup sandbox โโโโโโโโโโโโโโโโโโโโ
โผ
state DB (sqlite / postgres / mysql)
```
1. GitHub sends a `workflow_job` (queued) webhook to runnerd.
2. runnerd applies the repository allowlist, then matches the job labels against enabled Runner Specs.
3. runnerd creates a Qiniu Sandbox instance and registers a self-hosted runner inside it.
4. GitHub Actions dispatches the job to the runner; the job executes in the sandbox.
5. When the job completes (or times out), runnerd removes the runner registration and stops the sandbox.
## Quick Start
```bash
# 1. Build
task build
# 2. Create config from example
cp runnerd.yaml.example runnerd.yaml
# Edit runnerd.yaml: set database, GitHub App credentials, sandbox settings
# 3. Bootstrap the first admin (one-time, exits without starting the server)
./bin/runnerd --bootstrap-admin github:<github-user-id> --config runnerd.yaml
# 4. Start runnerd
./bin/runnerd --config runnerd.yaml
```
5. Open `http://<host>:25500/` and sign in with GitHub OAuth. The public product landing page links to the same-origin `/docs` guides and the protected Jobs console at `/jobs`. On the first authenticated visit to `/jobs`, a six-step product tour introduces Jobs, Repositories, Settings, and Sandbox setup; it can be replayed from the account menu.
6. Open **Repositories** to review **Runner readiness** for the account or organization. Ready sources are shown without configuration controls. If Sandbox setup is missing and you can manage that scope, use **Configure Sandbox** to open the exact account or organization **Preferences** page and configure **Sandbox Service** credentials. Settings lists only your account and organizations where GitHub reports an active owner membership (`role: admin`). Organization members, outside collaborators, and other repository-only users receive only read-only readiness and cannot browse that organization's configuration, Sandbox catalogs, or custom Runner Specs. Administrators can provide a fallback at `/admin/sandbox_service`.
7. Configure platform Runner Specs in Admin. Existing managed specs migrate with their labels and operator controls preserved; fresh installations require explicit creation. Enable catalog display only for verified public templates, whether referenced by name or ID. Standard and large template builds retain their regional release gates.
8. Configure a GitHub webhook โ `POST http://<host>:25500/webhooks/github`.
9. Use the labels configured for an enabled Runner Spec, for example `runs-on: [qiniu, ubuntu-24.04]` if an administrator has created a matching spec. Fresh installations do not create this example spec automatically.
Organization owners can open **Settings โ Fork sponsorship** to let trusted personal forks use the organization's Sandbox service for managed Runner Specs. A policy is bound to the upstream repository's stable GitHub ID, starts disabled, has its own positive concurrency limit, and can require an exact fork approval, upstream write permission, or active organization membership. The Job and GitHub Runner remain owned by the fork; organization Cache S3, custom Runner Specs, physical template IDs, Runner Groups, credentials, and provider catalogs are never inherited. See [Sponsor trusted forks with an organization Sandbox](https://runner.qiniuinc.com/docs/guides/fork-sponsorship) for the platform workflow and [Organization-Sponsored Fork Runners](docs/organization-sponsored-fork-runners.md) for implementation details.
For local development, use `task dev` with `runnerd.local.yaml`. See [docs/testing.md](docs/testing.md) for detailed local setup including GitHub App creation and webhook forwarding.
The Sandbox Deploy control-plane and execution slice is documented in [docs/deployment.md](docs/deployment.md). `/deployments` and deployment details share a repository sidebar: repository links open its deployment list, expandable children show active deployments, and a selected terminal deployment remains visible. Navigation summaries and deployment history paginate independently from the local database. The list uses single-select Running / Processing / Stopped / Failed segmented filters with full authorized counts and URL-restored selection. Running is selected by default; exactly one status remains selected. The Create deployment link opens `/deployments/new` with the current repository preselected. The creation page contains the repository, Branch/Tag/PR and scoped Sandbox template form; `deploy.template_id` is an optional API default. Creation opens the deployment detail under its repository or PR context; refresh preserves the deployment and its logs. Lists, details, logs and actions share Jobs' exact GitHub OAuth/App installation-repository authorization. Content stays in the local database; authorization uses the same short-lived cache as Jobs and rejects suspended Installations even if GitHub still lists their repositories. PR titles are saved at admission; a separate optional metadata request fills missing legacy titles without delaying details or logs. Source headings link to GitHub Branch/PR/Tag pages. Repository members may deploy and redeploy; Sandbox settings and private template catalogs remain owner-managed. Each deployment starts at Revision #1; redeploy creates a new numbered revision of the pinned plan and keeps earlier status/logs. Details show the selected template name and CPU, memory, and disk snapshot beside revision history and a chronological log viewer, and `?revision=N` preserves the selected version on reload.
Running deployment details also offer a Web terminal for the current revision. Connect manually to inspect processes and application log files; leaving the terminal closes the shell. Terminal access revalidates repository access and isolates each user's shell and uses the saved Sandbox credentials. Brief output-stream interruptions resume the same shell automatically within a 20-second recovery window; authorization/lifecycle loss or an unrecoverable output gap closes it.
Compensation cleanup preserves the original deployment failure and revision evidence, including when cleanup resumes after interruption. Creation idempotency uses an immutable request hash, so provider expiry changes and later revisions do not invalidate the original request key.
Deployment recovery and queued execution run in the independent background scheduler without inheriting the HTTP startup recovery timeout. Revision log budgets are recalibrated on first append after each restart, including after rollback writes; Deploy supports one controller without mixed-version writers.
Application startup supplies the exact Sandbox hostname for Vite's Host allowlist, so Vite preview needs no repository-specific ingress domain.
Jobs provide deployment history and a creation link pinned to the selected Job commit. Repository collections use readable Branch/Tag query filters (for example, `/github/qiniu/deployment/deployments?branch=example/vite`) and stable `pull_requests/{number}` paths. Deployment details use `/github/{owner}/{repo}/deployments/{id}` (or the PR path) with optional source/revision queries and no Commit SHA. Existing Jobs source paths remain compatible; old deployment query-ID links are unsupported. See [GitHub context routes](docs/deployment.md#github-context-routes).
## Configuration
`runnerd` reads `./runnerd.yaml` by default, or the path passed with `--config`. See [`runnerd.yaml.example`](runnerd.yaml.example) for a fully commented reference.
| Section | Description |
| --- | --- |
| `server` | Listen address, read/write/idle timeouts |
| `database` | Backend (`sqlite` / `postgres` / `mysql`) and DSN |
| `auth` | Session secret, encryption key, session TTL |
| `sandbox` | Sandbox lifecycle timeouts (create, run, stop) |
| `deploy` | Deploy worker settings: optional template default, per-scope concurrency, retention, and transient work root |
| `github` | Webhook secret, auth method (App / PAT / basic), OAuth, allowed repositories |
| `worker` | Lease, retry, and concurrency settings |
Operators can set `worker.account_concurrency_limits` by GitHub repository owner and `worker.repository_concurrency_limits` by `owner/repo`. They aggregate across Runner Specs; omitted entries or `0` preserve existing behavior. Restart to apply changes; see [concurrency configuration](docs/testing.md#ci-runner-account-and-repository-concurrency).
Key notes:
- Relative `database.dsn` and `github.app.private_key_file` paths resolve from the config file's directory.
- Use SQLite for local and single-node deployments. PostgreSQL and MySQL are supported but multi-instance operation on a shared database has not been verified.
- CI creates fresh PostgreSQL and MySQL schemas and reruns migration against an existing catalog before exercising policy-free matching. See [docs/testing.md](docs/testing.md) for the opt-in local real-dialect command.
- Existing SQLite `runner_requests` and `runner_profiles` tables add missing model columns and indexes on startup without table recreation. This preserves historical runner values plus legacy profile rows and indexes. Creating missing indexes does not rewrite rows, but it can add brief startup I/O and lock contention on a large database; see [docs/testing.md](docs/testing.md) for migration and query-plan checks.
- GitHub Enterprise Server is **not** supported; use a GitHub.com App.
- Configure exactly one GitHub auth method: `github.app`, `github.token`, or `github.basic_auth`.
- When `github.app.installation_id` is omitted, runnerd resolves the installation dynamically per repository, allowing one App to serve multiple accounts.
- `deploy.template_id` is an optional default, separate from Runner Specs. The independent worker always runs; creation requires an available selected template (or this default) and an eligible Sandbox service. `deploy.work_root` is resolved against the runnerd config directory (defaulting under the state directory); the resulting absolute workspace path is used inside the Sandbox, not for host execution.
- Deploy uses the same Sandbox service eligibility as Jobs: scoped or inherited credentials take priority; otherwise the platform service must be enabled and allow the repository Installation owner in its audience by stable GitHub ID and account type. A personal-repository collaborator keeps their credential scope but cannot substitute their identity for the owner in this check. Platform credentials permit only public templates, including for organization owners. Admitted deployments keep their saved credentials and region/template bindings; retries resolve current credentials and recheck public templates when using the platform service.
- `deploy.max_concurrent_per_scope` defaults to `3` and limits queued/running deployments for each account or organization scope. `deploy.retention_hours` defaults to `168`; released terminal rows and bounded events are pruned after that period while audit history remains.
### Config Value Obfuscation
Sensitive fields accept `RUNNERD_ENC(v1:...)` values to avoid plaintext in the config file:
```bash
read -r -s secret_value
printf '%s' "$secret_value" | ./bin/runnerd --obfuscate-config-value
unset secret_value
```
Supported fields: `database.dsn`, `auth.session_secret`, `auth.encryption_key`, `github.webhook_secret`, `github.token`, `github.basic_auth.password`, `github.oauth.client_secret`. These values are also masked as `******` in logs and serialized output.
> **Note:** This hides plaintext from casual inspection only โ the decoding key is embedded in the binary. It is not encryption against a host-level attacker.
### Cache and S3
Each user configures a Cache S3 Bucket, optional Prefix, AK, and SK in account or GitHub installation Preferences. The default prefix is `gh-actions-cache`. The S3 region and endpoint are derived from the selected Sandbox service region and are configured by the operator in `runnerd.yaml` under `sandbox.regions`. Because the configured endpoint may be private to the Sandbox network, runnerd only validates the configuration shape when saving it; bucket reachability and permissions are verified by the actual workflow in the Sandbox. AK/SK are encrypted in scoped state and are never returned to the browser or runner.
On every sandbox start, runnerd resolves the GitHub repository to its installation/account scope, verifies the Workflow Run trust context, and mints a Qiniu IAM federation token through the configured `cache.sts_endpoint` (default `https://sts-ov.qiniuapi.com`). Its requested lifetime is the configured Sandbox lifecycle plus five minutes for the post-job cache save step; no refresh mechanism is provided yet. It injects the token plus bucket/endpoint/prefix as `AWS_*` / `RUNS_ON_S3_*` environment variables in the Sandbox start script, so the scoped cache action can upload and restore caches directly to the user bucket without proxying bytes through runnerd.
Cache object keys are isolated by repository and workflow context:
```text
<configured-prefix>/<owner>/<repo>/scopes/branch-<first-16-bytes-of-sha256(branch)-hex>/...
<configured-prefix>/<owner>/<repo>/scopes/pr-<number>/...
```
With the scoped `qiniu/actions-cache@v5` action, runnerd injects ordered read prefixes plus one write prefix. A trusted branch searches its own scope and then the default-branch scope, and writes only its own scope. A pull request, including a Fork PR, searches its PR scope, base-branch scope, and default-branch scope in that order, and writes only its own PR scope; this matches GitHub's merge-ref cache isolation so a Fork cannot poison a base-branch cache. `pull_request_target`, `workflow_run`, `issue_comment`, and unverified metadata are default-branch read-only. Kodo list operations are scoped to authorized prefixes via `kodo:prefix` Condition, so cache key names outside the granted scopes are not enumerable across credentials.
#### Operator configuration
```yaml
sandbox:
regions:
- id: us-south-1
label: "United States ยท Dallas 1"
sandbox_api_url: https://us-south-1-sandbox.qiniuapi.com
s3_region: us-north-1
s3_endpoint: https://internal-s3-las-us-north-1-dal.qiniucs.com
cache:
sts_endpoint: https://sts-ov.qiniuapi.com
```
`sandbox.regions` defines the public Sandbox region catalog exposed through `GET /sandbox/regions`; S3 mappings remain server-side because endpoints can be private. Each entry must include `id`, `label`, and `sandbox_api_url`. `s3_region` and `s3_endpoint` are optional and must be configured together; only regions with both fields support Cache S3. At least one region is required. For stronger cache isolation, use a dedicated bucket per GitHub installation. `cache.sts_endpoint` is the Qiniu IAM federation token endpoint used to mint short-lived credentials.
#### Using cache in GitHub Actions workflows
Use [`qiniu/actions-cache@v5`](https://github.com/qiniu/actions-cache) instead of `actions/cache`. No additional configuration is needed in the workflow โ runnerd injects S3 credentials plus ordered read prefixes and a single write prefix as environment variables:
```yaml
steps:
- uses: qiniu/actions-cache@v5
with:
path: |
~/.cache/go-build
~/go/pkg/mod
key: ${{ runner.os }}-go-${{ hashFiles('**/go.sum') }}
restore-keys: |
${{ runner.os }}-go-
```
The upload/download concurrency can be tuned via environment variables (defaults shown):
```yaml
env:
UPLOAD_QUEUE_SIZE: "16" # concurrent multipart upload parts
UPLOAD_PART_SIZE: "16" # part size in MiB
DOWNLOAD_QUEUE_SIZE: "16" # concurrent range-request downloads
DOWNLOAD_PART_SIZE: "16" # part size in MiB
```
## GitHub App Setup
### Required Permissions
| Scope | Permission | Access | Purpose |
| --- | --- | --- | --- |
| Repository | Actions | Read-only | Query job/run status, list queued jobs; required for webhook events |
| Repository | Administration | Read & write | Repository-level runner registration (when spec has no `runner_group`) |
| Repository | Metadata | Read-only | Identify repositories and owners |
| Repository | Contents | Read-only | Sandbox Deploy: discover Branch/Tag/Commit sources, read deployment configuration and check out pinned source |
| Repository | Pull requests | Read-only | Show PR titles and resolve PR deployment sources |
| Organization | Members | Read-only | Verify active organization owner role (`role: admin`) for organization Settings and scoped management |
| Organization | Self-hosted runners | Read & write | Organization-level runner registration (when spec sets `runner_group`) |
For Deploy, existing installation owners must approve newly requested Contents
or Pull requests permissions. After approval, sign out and sign in with GitHub
again if runnerd still reports `REAUTH_REQUIRED` or unavailable repository
versions. A successful public deployment does not prove private Contents access.
Set `github.app.slug` to show an "Install GitHub App" link in the user UI. Use `github.allowed_repositories` (patterns like `owner/repo` or `owner/*`) to restrict which repositories can use this runnerd instance.
### OAuth Sign-in
`github.oauth` enables GitHub App OAuth login for the built-in console:
- Use the GitHub App's **Client ID** and **Client Secret**.
- Set the App callback URL to `http://<host>:<port>/auth/github/callback`.
- Set `auth.session_secret` (session signing) and `auth.encryption_key` (user secret encryption) to separate random values.
First OAuth login creates a `role: user` account. Use `--bootstrap-admin <github-user-id>` to promote an account to admin.
### Webhook Events
In your GitHub App settings (**Settings โ Developer settings โ GitHub Apps โ your app โ General**), configure:
1. Set the **Webhook URL** to `https://<your-runnerd-host>/webhooks/github`.
2. Under **Subscribe to events**, check:
- **Workflow jobs** (`workflow_job`) โ **required**, triggers runner creation.
- **Workflow runs** (`workflow_run`) โ optional, acts as a compensating signal for missed `workflow_job` events.
3. Save changes.
> **โ ๏ธ Common pitfall:** If no events are subscribed, GitHub will not send any webhooks and jobs will stay queued forever. This is configured in the **GitHub App settings**, not in the repository's webhook settings.
## Webhook & Workflow Setup
1. Ensure the GitHub App webhook is configured as described in [Webhook Events](#webhook-events) above, with the `webhook_secret` matching `github.webhook_secret` in your config.
2. Use labels that match an enabled Runner Spec configured in Admin. For example, after configuring the corresponding public spec:
```yaml
runs-on: [qiniu, ubuntu-24.04]
```
The `qiniu` label is required by the standard spec examples. A custom spec can define
its own advertised and required labels instead.
runnerd handles `queued`, `in_progress`, and `completed` actions. For `workflow_run`, it lists all queued jobs in the run and enqueues any matching jobs not already seen.
A startup REST check reporting `waiting`, `pending`, or `requested` keeps the request queued with capped exponential backoff and no Runner resources or lease. `in_progress` is monitored without starting a duplicate Runner. A later queued delivery can wake a deferred request; polling also resumes it without another webhook. GitHub owns approval/concurrency timeouts. Unknown states or lookup failures use the configured finite retry budget and fail visibly rather than recording success. HTTP 429 and GitHub secondary rate limits remain queued with capped backoff even after that budget is exhausted, and resume automatically when throttling clears. See [targeted recovery of historical false completions](docs/testing.md#waiting-workflow-jobs).
If a Runner process ends before accepting any Job, for example because GitHub terminated its listener (`run.sh` still exits 0) or its Sandbox output stream was lost while GitHub reports it idle, runnerd retries the request with the same bounded backoff instead of marking it completed or failed; the retry rechecks the Job before creating another Sandbox.
New runner requests, including admission rejections, store the parsed GitHub context but not the raw webhook body. The legacy `github_payload_json` column and existing values remain available for historical metadata backfill; this change does not clean up old payloads, request history, or logs.
## Runner Specs & Matching
Runner specs are managed through the admin API and console โ not through `runnerd.yaml`. Every enabled spec is eligible for repositories admitted by `github.allowed_repositories`; labels select the spec.
- **Public-name binding**: Admin configures a stable public template name. Runnerd resolves its physical ID through the request's effective Sandbox region. Publication in `/runner-specs` is an explicit, validated choice.
- **Fixed-ID reference**: Admin configures an explicit `template_id`, which may refer to a public template. Admin uses one Template field; catalog display requires verified public visibility. New bindings and runtime startup require a nonzero effective default BuildID; a failed latest rebuild does not invalidate an existing default. Every platform spec's labels, priority and capacity are editable in Admin; the code no longer maintains a catalog. New binding/publication requires the configured admin Sandbox credentials; unchanged-binding policy edits remain available without provider access.
- **GitHub Runner Group**: when a spec sets `runner_group`, runnerd creates an organization-level runner in that GitHub group; otherwise it creates a repository-level runner. This is not the retired internal Runner Group model. Organization forms explain repository-level registration, offer a live group selector, and generate `group` + `labels` YAML. New/changed groups and re-enablement are validated before saving; startup checks group and exact repository access before provisioning. Admin platform specs retain manual names for their multiple organization contexts. See [GitHub Runner groups](docs/platform-runner-specs.md#github-runner-groups).
> **โ ๏ธ Personal accounts:** `runner_group` requires the organization-level GitHub API. If the repository belongs to a personal account (not an organization), leave `runner_group` **empty** โ otherwise the organization-group preflight will reject startup.
- **Windows runners (preview)**: a spec whose labels include `windows` uses the Windows launch path: a PowerShell start script, the envd default user, Windows Runner downloads for fixed-ID preflight, and no Docker requirement. Windows uses a fresh GitHub JIT configuration bound to one ephemeral Runner instead of a reusable registration token; runnerd retains API cleanup responsibility if provisioning fails. The bootstrap file is scrubbed before the Runner starts. This does not isolate workflow code from that Runner's own same-user credentials or process memory. Bind it to a [`github-runner-windows-11`](templates/github-runner-windows-11/README.md) template ID, for example with labels `self-hosted,windows,x64,qiniu,windows-11`. Network diagnostics and the Web Console do not support Windows Sandboxes yet.
Matching always enforces `required_labels โ job_labels โ labels`. Migrated Ubuntu specs preserve their `qiniu` and exact OS required labels; changing the catalog in Admin changes the workflow contract. Enablement and publication are separate controls.
Internal Runner Groups and Repository Policies have been removed. Their legacy
management APIs now return `404 Not Found`, while old Admin bookmarks redirect
to Runner Specs. They are not part of supported configuration, matching, or
recovery behavior; any legacy database artifacts are ignored by current code.
The database is the single catalog authority. Startup migrates old managed rows once, preserves their operator settings and runtime behavior, and never restores edited/deleted specs. Legacy private specs keep their physical template IDs and clear unused public names so later edits remain valid. A fresh installation starts with an empty catalog. The visibility adjustment needs no new schema or data migration. Existing name/ID references and display flags stay unchanged; runtime keeps the existing name/ID behavior; this refactor adds no configurable Runner preparation, Docker or sponsorship policy. See [Platform Runner Specs](docs/platform-runner-specs.md) for fields, migration and validation.
See [Public Runner Templates](docs/default-runner-templates.md) for supported
workflow labels, publication status, and regional verification.
`GET /api/public/runner-templates` exposes enabled, displayed public-name references from the database without authentication, including validated large name references when configured. Fixed-ID specs can appear in the protected `/runner-specs` directory after publicness validation, but remain absent from this stable-name API. It returns stable template names, logical spec names and workflow labels, never physical IDs or credentials; the public projection is cached in each server process for 60 seconds, with concurrent cache fills coalesced. Successful Admin spec saves and deletes invalidate that process cache; browser caches may retain responses for 60 seconds. The credential-bound `GET /user/sandbox/templates?region=<id>` remains a separate scoped resource.
Ordinary users browse the read-only platform Runner Spec catalog at
`/runner-specs`. That page has no account/Organization selector or user-editable
availability and concurrency policy. Users manage only owned custom Specs under
`/account/runner-specs` or
`/organizations/{login}/runner-specs`. The authenticated `/user/runner-specs`
API combines published public platform specs and custom
specs owned by that account or owner-manageable Organization. Platform availability
and concurrency remain global Admin policy. Scoped custom specs use exact
normalized workflow labels, may override a global spec with the same label set,
and validate new or changed template IDs only with that scope's explicit or
legally inherited Sandbox credentials. `runner_group` is available only for
Organization custom specs. The response exposes a template ID only for the
caller's own scoped custom spec, never for a platform custom spec. A queued
request reloads and validates the same persisted source and scope immediately
before startup, so a spec disabled while waiting cannot launch a runner.
Admin creation controls platform execution separately from public visibility. Use a scoped custom spec when an environment must serve only one account or Organization.
For custom specs, `template_id` should point to a Qiniu Sandbox template containing the GitHub runner image. Template access is checked against the repository owner's effective Sandbox service shown under **Repositories โ Runner readiness** at sandbox creation time.
## Admin Console
The built-in web UI provides:
| Route | Description |
| --- | --- |
| `/admin/` | Dashboard with runnerd runtime diagnostics and metrics |
| `/admin/accounts` | Account management โ list, search, and change roles |
| `/admin/github_accounts` | GitHub App installation accounts reported live by GitHub |
| `/admin/github_accounts/{installation_id}` | One GitHub App installation and its currently authorized repositories |
| `/admin/runner_requests` | Runner request history with server-side filters, cursor-based infinite scrolling, global status metrics, controls, and exact lookup by Runner Name or internal request ID |
| `/runner_request_metrics` | Admin-only global Runner request status and Runner Spec counts; independent of list filters |
| `/admin/runner_requests/{id}` | One Runner request resource with persisted state, lifecycle timestamps and cleanup duration, a startup environment snapshot (Sandbox region, resolved physical template ID, template version, and preinstalled Runner version), a best-effort effective Runner version captured after any custom-template preflight or official self-update, typed termination source and optional process exit code, diagnostic findings, a retained terminal GitHub Job result with a historical live-lookup fallback, a bounded on-demand network probe for running Sandboxes, and a cursor-paged timeline that shows every control/stdout/stderr event and its optional stage directly in chronological order |
| `/admin/runner_specs` | Platform Runner Spec administration and public publication |
| `/runner-specs` | Read-only platform Runner Spec catalog and workflow labels |
| `/account/runner-specs` and `/organizations/{login}/runner-specs` | Custom Runner Specs owned by an account or owner-manageable Organization |
| `/organizations/{login}/fork-sponsorship` | Owner-only policies for organization-sponsored managed Runner execution in trusted forks |
| `/admin/sandbox_service` | Sandbox service configuration |
| `/admin/match` | Label-match preview against the current enabled Runner Specs |
| `/admin/audit` | Audit event history |
| `/admin/diagnostics` | runnerd runtime diagnostics with redacted summary, pprof discovery, and on-demand expvar |
`/` is always the public Qiniu CI Runner product landing page. `/docs` and its fixed guide routes are public, same-origin, and available in English and Simplified Chinese. The ordinary-user Jobs homepage is `/jobs`; other protected routes include `/repositories`, PR job groups (`/github/{owner}/{repo}/pull_requests/{number}/jobs`), and account settings (`/account/preferences`, `/account/sandbox-templates`, `/account/sandbox-instances`), with matching `/organizations/{login}/...` routes. Opening a protected route without a session shows a focused GitHub sign-in page and returns to the original URL after OAuth.
Job details open **Runner logs** by default and retain the Web Console and request details. Jobs and admin request details share one event timeline: mixed control/stdout/stderr records, timestamps, stages, stream filters, older-page loading, and incremental refresh every five seconds while active. `GET /user/runner_requests/{id}/events` requires the same exact installation/repository authorization as Job details and returns the same event pages as the admin endpoint. In the Jobs panel, the job name links to the GitHub Job and the workflow name links to its Run. Standalone Job details provide **View job on GitHub**, falling back to the Workflow Run when its repository and Run ID are known. runnerd does not download or duplicate GitHub Actions logs. The retired `/user/runner_requests/{id}/github-log` endpoint returns `404`.
Runner request lists return the newest 100 rows by default and cap pages at 500. They project only public runner-state fields instead of stored webhook payloads or Sandbox credentials. Admin polling uses the `(queued_at DESC, id ASC)` index; repository-authorized user polling queries each installation through `(github_installation_id, queued_at DESC, id ASC)` and merges the bounded results while preserving exact installation/repository access pairs.
## Troubleshooting
| Symptom | Likely Cause | Fix |
| --- | --- | --- |
| Job stays **queued** forever, no webhook in runnerd logs | GitHub App has no subscribed events | Go to GitHub App settings โ Subscribe to **Workflow jobs** event |
| `runner_group_permission_denied` (legacy: `github registration token: status 404`) | Group lookup cannot access the owner organization, including personal-account owners or missing App permissions | Leave `runner_group` empty for personal repositories; for organizations, check the App installation and Self-hosted runners permission |
| `invalid signature` in logs | Webhook secret mismatch | Ensure `github.webhook_secret` matches the secret in GitHub App/repo webhook settings |
| `runner start deferred ... at capacity` | Global or per-spec concurrency limit reached | Wait for running jobs to finish, or increase `max_concurrent_runners` / spec `max_concurrency` |
| Sandbox creation fails | Repository owner has no effective Sandbox service | Open **Repositories**, select the account or organization, and complete **Runner readiness**; admins may also configure an eligible fallback at `/admin/sandbox_service` |
| GitHub reports that a self-hosted runner lost communication | The runner, Sandbox, or its network path stopped reporting heartbeats | Open **Admin โ Runner Requests**, look up the Runner Name shown by GitHub (for example, `e2b-101445685709`), and inspect the request resource's findings and lifecycle timeline |
For a request that is still `running`, an administrator can run a fixed-target probe for the GitHub API, Ubuntu archive, or LLVM APT host from the request detail page. The probe is diagnostic evidence only: it is limited to 10 seconds, a 64 KiB download, and 64 KiB of combined command output; it is rate-limited to once per Sandbox/PID attempt every 30 seconds, retained in Run history, and never changes or extends the Runner lifecycle.
For detailed local debugging steps, see [docs/testing.md](docs/testing.md#8-troubleshooting-order).
## Docker
The container image uses file-config only. Mount `runnerd.yaml` and any referenced secret files into the container:
```bash
docker run --rm -p 25500:25500 \
-v "$PWD/runnerd.yaml:/etc/runnerd/runnerd.yaml:ro" \
-v "$PWD/secrets:/etc/runnerd/secrets:ro" \
ghcr.io/qiniu/ci-runner
```
## Build & Development
```bash
task deps # Install Go dependencies
task ui-deps # Install UI dependencies
task build # Build runnerd with embedded production UI
task ui-production-smoke # Execute the production UI bundle in Chromium
task ui-terminal-fault-test # Timed SSE outages with Chromium and real local PTYs
task dev # Start local dev (runnerd + Vite + smee)
task lint # Run linters
task test # Rebuild UI + run all tests (Go with race detection + Bun UI tests)
task docker-check # Verify the service Dockerfile
task release-check # Verify release build
```
The `staticcheck` step in `task lint` uses the Go version declared in `go.mod`, matching CI.
The `Check` workflow validates application changes. `Template Check` runs source tests, upstream compatibility and Dockerfile checks only for related build/template/script changes, or by manual dispatch. It does not build or publish images; registry failures stay visible in their own jobs. See [CI scope](docs/testing.md#ci-scope).
For focused UI tests, run `cd ui && bun run test`. Use `task ui-production-smoke`
after changing UI dependencies, Vite/Rollup configuration, or production asset
loading.
Run `task ui-terminal-fault-test` after terminal replay, reconnect or idle-cleanup
changes. It requires Go, Bun and Python 3 on Linux/macOS; see
[timed terminal transport faults](docs/testing.md#timed-terminal-transport-faults)
for the scenarios, artifacts and production-verification boundary.
### Sandbox Templates
| Template | Description |
| --- | --- |
| `templates/github-runner-ubuntu-slim` | Ubuntu Slim x64 source with standard 20-GiB and large 80-GiB build configs |
| `templates/github-runner-ubuntu-22.04` | Ubuntu 22.04 x64 source with standard 20-GiB and large 80-GiB build configs |
| `templates/github-runner-ubuntu-24.04` | Ubuntu 24.04 x64 source with standard 20-GiB and large 80-GiB build configs |
| `templates/github-runner-ubuntu-26.04` | Preview Ubuntu 26.04 x64 source with standard 20-GiB and large 80-GiB build configs |
The public `ubuntu-latest-large` Runner Spec is a logical label mapped to the
`github-runner-ubuntu-24-04-large` physical template; it does not add another
source directory or build target.
Run `task template-python-test` when changing Python/pipx installation; it uses
Docker and network access to verify isolation from apt-owned Python packages.
Run `task template-check-all`, then use `task template-build-all` to build all
eight templates sequentially, or use an individual `task template-build-ubuntu-*`
target for one real qshell Sandbox build. See
[Public Runner Templates](docs/default-runner-templates.md) for publication and
cache-resume guidance after a remote build time limit, plus publication and
smoke commands. Shared setup code and the Actions Runner version pin live in
`templates/common/`. Build tasks verify the official Runner archive locally,
upload it as small COPY chunks, and verify it again before installation.
The pin remains the tested preinstalled baseline. Public-name specs use it and rely on regular template rebuilds. Private-ID specs resolve and verify GitHub's official Runner archive before
registration and replaces a stale writable copy before the current Job can be
accepted. Official self-update remains enabled after registration for both
paths.
Fenno checks the official stable Runner daily through `.fennoai.yml` and the
project's `maintain-actions-runner` skill. Its deterministic updater verifies
Linux x64 and Windows x64 archives, synchronizes their pins, regenerates the
compatibility manifest, and runs template checks before maintaining the single
`automation/actions-runner-update` PR. Identical proposals do not create new
commits; both Check and Template Check must pass for the current PR head.
Human review and merge remain required. The manual `Actions Runner Update` workflow only validates a
proposal and cannot push, create PRs, or publish templates.
Confirm Fenno's actual timezone, permissions and single-task execution before
switching on the daily schedule. Source automation receives no production
Sandbox credentials. After merge, an operator uses the existing local build,
publish, catalog and smoke Tasks for two-region promotion and retains real Job
evidence in the versioned release record. See
[Public Runner Templates](docs/default-runner-templates.md#automated-runner-baseline-update-prs)
for activation, asset validation, no-op/failure behavior and acceptance boundaries.
Template source validation requires Node.js 24, and template builds require
qshell 2.19.13 or newer. Standard configs request
`disk_size_mb = 20480`, while large configs request `81920` for new templates.
This setting is the minimum root disk size; the reported total can be larger.
Qshell does not send the setting on same-name
rebuilds. After the provider team's `DiskMb` is adjusted, the build tasks rebuild
the existing names and IDs in place without rejecting their stale pre-rebuild
totals. Publish and catalog checks require the rebuilt totals to meet the
configured lower bounds; a larger total is not a mismatch by itself.
Release smoke reads the corresponding TOML and requires the runtime root disk
size to meet its `disk_size_mb` lower bound.
The 2.337.0 baseline completed the two-region, eight-template rebuild, catalog,
standard workflow, and large workflow gates recorded in
[Issue #93](https://github.com/qiniu/ci-runner/issues/93).
## Documentation
| Document | Description |
| --- | --- |
| [Hosted site guides](https://runner.qiniuinc.com/docs) | Hosted quick start, runnerd deployment, workflow example, organization fork sponsorship, custom template lifecycle, troubleshooting, and managed labels |
| [docs/testing.md](docs/testing.md) | Local testing, GitHub App/OAuth setup, webhook forwarding, troubleshooting |
| [docs/deployment-smoke.md](docs/deployment-smoke.md) | Production-style readiness checklist |
| [docs/deployment.md](docs/deployment.md) | Sandbox Deploy configuration and current control-plane API |
| [docs/default-runner-templates.md](docs/default-runner-templates.md) | Public template labels, qshell release flow, regional smoke, and rollback |
| [docs/runner-architecture-comparison.md](docs/runner-architecture-comparison.md) | Architecture diagrams and comparison with ARC / Fireactions |
| [docs/runner-implementation-review.md](docs/runner-implementation-review.md) | Implementation status and schema migration notes |
## License
Qiniu CI Runner is licensed under the [Apache License 2.0](LICENSE).
## Community & Contributing
Bug reports, feature ideas, documentation improvements, and code contributions are welcome.
- [Report a bug or propose a feature](https://github.com/qiniu/ci-runner/issues).
- [Open a Pull Request](https://github.com/qiniu/ci-runner/pulls) to improve the code or documentation.
- Scan the QR code below to join the community chat.
---
<p align="center">
<img src="./docs/assets/qrcode.png" width="220" alt="Qiniu CI Runner community chat QR code" />
</p>
<p align="center">
<em>Scan the QR code to connect with maintainers and other Qiniu CI Runner users.</em>
</p>