bluex-ops

name	bluex-ops
description	Operate Bluex Next in production: create GitHub issues, inspect Bluex API state, monitor ubuntu-blue over SSH, and query the live production PostgreSQL database. Use when the user asks to operate, monitor, debug, triage, or create work for bluex-next/Bluex.

Bluex Ops

This skill is for operating Bluex Next, especially the production instance on root@ubuntu-blue.

Default posture: observe first, do not mutate production unless the user explicitly asks.

Always establish time context before any operational check or action:

• Run local date '+%Y-%m-%dT%H:%M:%S%z' first.
• When connecting to production, include remote date -Is before status/API/DB queries.
• Compare timestamps against the current time and say how recent/stale observations are.

Production facts

• SSH host: root@ubuntu-blue
• App service: bluex-app-api.service
• App user/group: bluex-app:bluex-app
• Current app symlink: /srv/bluex-app/current
• Config env file: /etc/bluex-app/bluex.env
• Production config: /etc/bluex-app/bluex.config.ts
• API bind: 127.0.0.1:3300
• Public app is proxied by Caddy.
• Database: PostgreSQL, resolved from runtime.databaseUrl after sourcing /etc/bluex-app/bluex.env and loading /etc/bluex-app/bluex.config.ts.
• Env source for the live database: BLUEX_DATABASE_URL.
• Workspace root: /srv/bluex-app/workspaces
• Default repository: awaaate/bluex-next

Production no longer uses the historical SQLite file as the source of truth. If /srv/bluex-app/data/bluex-next.sqlite still exists, treat it as a legacy/stale migration artifact unless the user explicitly asks to inspect old data. Do not use SQLite counts to describe live production state.

Use read-only PostgreSQL access unless explicitly asked to repair or mutate state.

First checks

When asked for status or monitoring, start with:

ssh root@ubuntu-blue 'hostname; date -Is; uptime; systemctl is-active bluex-app-api.service bluex-app-worker.service bluex-app-realtime.service caddy.service actions.runner.awaaate-bluex-next.ubuntu-blue.service; systemctl --failed --no-pager'

Then check the local API:

ssh root@ubuntu-blue 'for p in /runtime/health /runtime/status /openapi.json /issues /works /sessions /pull-requests /events /agents /execution-sites; do printf "%s " "$p"; curl -sS -o /tmp/bluex_probe -w "%{http_code} %{content_type} %{size_download}\n" -m 5 "http://127.0.0.1:3300$p"; done'

/runtime/health is the runtime health signal. Use /openapi.json only as a secondary liveness/documentation probe.

Resolve runtime paths from code/config

If paths are uncertain, derive them from the app config instead of searching blindly:

ssh root@ubuntu-blue 'bash -s' <<'REMOTE'
set -euo pipefail
cd /srv/bluex-app/current
set -a
. /etc/bluex-app/bluex.env
set +a
bun --eval '
import { loadBluexConfig } from "./apps/api/src/bootstrap/config-file";
import { resolveRuntimeConfig } from "./apps/api/src/bootstrap/runtime-config";

function redactUrl(value) {
  try {
    const url = new URL(value);
    if (url.username) url.username = "***";
    if (url.password) url.password = "***";
    return url.toString();
  } catch {
    return "<unparseable>";
  }
}

const config = await loadBluexConfig(process.cwd());
const runtime = resolveRuntimeConfig(config, process.cwd(), process.env.NODE_ENV);
console.log(JSON.stringify({
  databaseUrl: redactUrl(runtime.databaseUrl),
  databaseAdapter: runtime.databaseConfig.adapter,
  databaseSource: runtime.databaseConfig.source,
  workspaceRoot: runtime.workspaceRoot,
  port: runtime.port,
  defaultRepository: runtime.defaultRepository,
}, null, 2));
'
REMOTE

Database inspection

Always derive the live DB URL from runtime config. Do not paste or print database credentials in reports. Keep psql in read-only mode with a statement timeout.

ssh root@ubuntu-blue 'bash -s' <<'REMOTE'
set -euo pipefail
set -a
. /etc/bluex-app/bluex.env
set +a
DATABASE_URL="${BLUEX_DATABASE_URL:-}"
case "$DATABASE_URL" in
  postgres://*|postgresql://*) ;;
  *) echo "Refusing non-PostgreSQL live DB URL: ${DATABASE_URL%%:*}" >&2; exit 1 ;;
esac

PGOPTIONS="-c default_transaction_read_only=on -c statement_timeout=5000" \
  psql "$DATABASE_URL" -v ON_ERROR_STOP=1 <<'SQL'
BEGIN READ ONLY;
SELECT current_database() AS database, current_user AS user_name, now() AS checked_at;
SELECT table_name
FROM information_schema.tables
WHERE table_schema = 'public'
ORDER BY table_name;
ROLLBACK;
SQL
REMOTE

Useful summary query:

ssh root@ubuntu-blue 'bash -s' <<'REMOTE'
set -euo pipefail
set -a
. /etc/bluex-app/bluex.env
set +a
DATABASE_URL="${BLUEX_DATABASE_URL:-}"
case "$DATABASE_URL" in
  postgres://*|postgresql://*) ;;
  *) echo "Refusing non-PostgreSQL live DB URL: ${DATABASE_URL%%:*}" >&2; exit 1 ;;
esac
PGOPTIONS="-c default_transaction_read_only=on -c statement_timeout=5000" \
  psql "$DATABASE_URL" -v ON_ERROR_STOP=1 <<'SQL'
BEGIN READ ONLY;
SELECT 'issues' AS table_name, count(*) AS n FROM issues
UNION ALL SELECT 'works', count(*) FROM works
UNION ALL SELECT 'sessions', count(*) FROM sessions
UNION ALL SELECT 'events', count(*) FROM events
UNION ALL SELECT 'pull_requests', count(*) FROM pull_requests
UNION ALL SELECT 'workspaces', count(*) FROM workspaces
UNION ALL SELECT 'tracked_repositories', count(*) FROM tracked_repositories;
SELECT status, count(*) n FROM issues GROUP BY status ORDER BY n DESC;
SELECT status, count(*) n FROM works GROUP BY status ORDER BY n DESC;
SELECT status, count(*) n FROM sessions GROUP BY status ORDER BY n DESC;
SELECT status, count(*) n FROM pull_requests GROUP BY status ORDER BY n DESC;
ROLLBACK;
SQL
REMOTE

Useful triage query:

ssh root@ubuntu-blue 'bash -s' <<'REMOTE'
set -euo pipefail
set -a
. /etc/bluex-app/bluex.env
set +a
DATABASE_URL="${BLUEX_DATABASE_URL:-}"
case "$DATABASE_URL" in
  postgres://*|postgresql://*) ;;
  *) echo "Refusing non-PostgreSQL live DB URL: ${DATABASE_URL%%:*}" >&2; exit 1 ;;
esac
PGOPTIONS="-c default_transaction_read_only=on -c statement_timeout=5000" \
  psql "$DATABASE_URL" -v ON_ERROR_STOP=1 <<'SQL'
BEGIN READ ONLY;
SELECT id, status, tracker_ref_json->>'project' AS repo, tracker_ref_json->>'displayId' AS num, left(title,70) AS title, updated_at
FROM issues
WHERE status <> 'done'
ORDER BY updated_at DESC
LIMIT 25;
SELECT w.id, w.status, w.agent, w.harness, w.issue_id, left(i.title,55) AS issue_title, w.updated_at
FROM works w LEFT JOIN issues i ON i.id=w.issue_id
WHERE w.status IN ('failed','waiting')
ORDER BY w.updated_at DESC
LIMIT 25;
SELECT s.id, s.status, s.agent, s.harness, s.issue_id, left(s.progress_label,45) AS progress, s.updated_at, s.heartbeat_at
FROM sessions s
WHERE s.status IN ('idle','failed')
ORDER BY s.updated_at DESC
LIMIT 25;
SELECT p.id, p.status, p.review_status, p.checks_status, p.issue_id, p.tracker_ref_json->>'project' AS repo, p.tracker_ref_json->>'displayId' AS num, p.updated_at
FROM pull_requests p
WHERE p.status='open'
ORDER BY p.updated_at DESC;
ROLLBACK;
SQL
REMOTE

For one issue, inspect current pointers and recent events:

ssh root@ubuntu-blue 'ISSUE=issue_awaaate_bluex_next_671 bash -s' <<'REMOTE'
set -euo pipefail
set -a
. /etc/bluex-app/bluex.env
set +a
DATABASE_URL="${BLUEX_DATABASE_URL:-}"
case "$DATABASE_URL" in
  postgres://*|postgresql://*) ;;
  *) echo "Refusing non-PostgreSQL live DB URL: ${DATABASE_URL%%:*}" >&2; exit 1 ;;
esac
PGOPTIONS="-c default_transaction_read_only=on -c statement_timeout=5000" \
  psql "$DATABASE_URL" -v ON_ERROR_STOP=1 -v issue_id="$ISSUE" <<'SQL'
BEGIN READ ONLY;
SELECT id,status,current_work_id,current_session_id,current_pull_request_id,left(meta_json::text,2000) AS meta,left(signals_json::text,1000) AS signals,updated_at,title
FROM issues
WHERE id = :'issue_id';
SELECT sequence,type,source,work_id,session_id,pull_request_id,summary,left(payload_json::text,1200) AS payload,created_at
FROM events
WHERE issue_id = :'issue_id'
ORDER BY sequence DESC
LIMIT 20;
ROLLBACK;
SQL
REMOTE

API inspection

Use API for contract-shaped records:

ssh root@ubuntu-blue 'curl -fsS http://127.0.0.1:3300/issues | jq "{count:(.items|length), items:[.items[:10][] | {id,status,title,trackerRef}]}"'

OpenAPI paths:

ssh root@ubuntu-blue 'curl -fsS http://127.0.0.1:3300/openapi.json | jq -r ".paths | keys[]"'

Known important paths:

• /issues
• /issues/{issueId}
• /issues/{issueId}/execution-context
• /issues/{issueId}/work-runnability
• /issues/{issueId}/workspace/prepare
• /works
• /works/{workId}
• /works/{workId}/runnability
• /sessions
• /events
• /events/timeline
• /pull-requests
• /pull-requests/{pullRequestId}/check-runs
• /pull-requests/{pullRequestId}/validation-runs
• /agents
• /runtime/health
• /runtime/status
• /execution-sites

Logs

App logs:

ssh root@ubuntu-blue 'journalctl -u bluex-app-api.service -n 200 --no-pager'

Caddy logs:

ssh root@ubuntu-blue 'journalctl -u caddy.service -n 100 --no-pager'

GitHub Actions runner logs:

ssh root@ubuntu-blue 'journalctl -u actions.runner.awaaate-bluex-next.ubuntu-blue.service -n 120 --no-pager'

Note: openclaw-gateway may log WhatsApp 401 Unauthorized messages. Treat as separate noise unless the user asks about it.

Creating GitHub issues

Default repo is awaaate/bluex-next.

Bluex issues are work briefs, not implementation tickets. They should describe:

• the problem
• the intention
• the expected observable outcome
• useful context

They should not prescribe the solution. Bluex should investigate, design, and decide how to fix it.

Avoid writing issue bodies that say things like:

• create file X
• add function Y
• use library/endpoint/schema Z
• implement this exact algorithm
• modify this exact module

Only mention concrete files/endpoints/modules when they are part of the observed problem or necessary context.

Preferred issue body template:

## Problema

Qué está pasando, qué falla, qué falta o qué fricción existe.

## Intención

Qué queremos conseguir a nivel de producto, sistema u operación.

## Resultado esperado

Qué debería poder observarse cuando Bluex lo resuelva.

## Contexto

Datos útiles, enlaces, ejemplos, síntomas o restricciones.
No incluir una solución propuesta salvo que sea estrictamente contexto.

Before creating an issue, clarify the problem, intention, expected outcome, and context if they are missing. Prefer small, actionable briefs.

Good example:

## Problema

Al monitorizar Bluex en producción, no existe una forma simple y estable de saber si la API está viva. Actualmente hay que usar `/openapi.json` como proxy de salud, lo que mezcla liveness con documentación.

## Intención

Poder comprobar rápidamente desde operaciones, scripts o Caddy que la API está levantada y responde.

## Resultado esperado

Existe una señal simple de salud de la API que permite distinguir entre servicio caído, servicio vivo y problemas básicos de runtime.

## Contexto

En producción la API corre en `127.0.0.1:3300`. El endpoint `/health` actualmente devuelve 404.

Bad example:

Crear endpoint GET /healthz en apps/api/src/app.ts que devuelva `{ ok: true }`.

Command pattern:

gh issue create --repo awaaate/bluex-next --title "..." --body-file /tmp/issue-body.md

After creating a GitHub issue, if the user wants Bluex to see it immediately, use the Bluex tracked-repo import/sync endpoints from prod after checking OpenAPI/request schema. Do not guess write payloads.

Comments and projection

Do not duplicate the same operator handoff/comment in both GitHub and Bluex.

Bluex has comment projection/sync, so use one canonical channel and let projection propagate it. Prefer GitHub comments when the target is a GitHub issue/PR or when the comment is an operator-facing handoff. Use Bluex comments only when the target exists only inside Bluex or the user explicitly asks for an internal Bluex-only comment.

If projection is stale or broken, report that as a sync/projection issue instead of posting duplicate comments by default.

Production mutation rules

Do not do these without explicit user permission:

• Restart services.
• Edit /etc/bluex-app/*.
• Run INSERT, UPDATE, DELETE, DDL, or maintenance commands against production PostgreSQL.
• Use /srv/bluex-app/data/bluex-next.sqlite as live production data.
• Delete workspaces/releases.
• Trigger new work/runs.
• Merge/close PRs or issues.

Safe by default:

• systemctl status/is-active
• journalctl reads
• curl GETs
• psql BEGIN READ ONLY SELECTs against the runtime-derived PostgreSQL URL
• gh issue/pr list/view

Reporting format

When reporting status, keep it operational:

1. Overall health: API, service, runner, DB reachable.
2. Counts: issues by status, works by status, sessions by status, PRs by status.
3. Current blockers: recent blocked issues, failed works, failed/idle sessions, open PRs with failing checks/reviews.
4. Recommended next action.

Prefer Spanish if the user writes Spanish.