openmaic-hp-deploy

name	openmaic-hp-deploy
description	Sync OpenMAIC project to hp server (192.168.0.111, x86_64) and build/deploy Docker image. Covers rsync, docker build, container lifecycle, and env preservation.
tags	["openmaic","hp","docker","deploy","rsync","x86_64"]

OpenMAIC → hp Deploy

Overview

Sync the OpenMAIC codebase to hp (x86_64, 192.168.0.111) and build/deploy via Docker. hp runs the main OpenMAIC instance on port 8003.

Prerequisites

• SSH access configured in ~/.ssh/config (Host hp)
• Docker Hub login on hp: ssh hp "docker login -u jinguoqian --password-stdin"
• .env.local on hp at /opt/openmaic/.env.local

Step-by-Step

1. Sync code to hp

rsync -avz \
  --exclude='data-backup' \
  --exclude='node_modules' \
  --exclude='.next' \
  --exclude='.git' \
  --exclude='data/classrooms/*/audio' \
  /Users/jinguo/PycharmProjects/OpenMAIC/ \
  hp:/opt/openmaic/

PITFALL: data-backup/ contains ~6GB of audio files. Always exclude it or rsync will take minutes. The data/classrooms/*/audio/ exclusion is also critical — audio files are 21GB+.

2. Build Docker image on hp

ssh hp "cd /opt/openmaic && docker build -t jinguoqian/openmaic:latest ."

Build takes ~4-5 minutes on hp (Alpine packages + pnpm install + Next.js build).

PITFALL — stale Docker cache hides new files: If you rsync new source files (e.g. a new component) after a previous build, Docker's COPY . . layer may be cached and the new files won't enter the image. Always use --no-cache when files changed since the last build:

ssh hp "cd /opt/openmaic && docker build --no-cache -t jinguoqian/openmaic:latest ."

PITFALL — new .ts script files can break the build: The rsync may include new TypeScript script files (e.g. scripts/backfill-actions-tts.ts, scripts/qc-speech-continuity.ts) that have compilation errors. These get picked up by Next.js's Turbopack during pnpm build and cause build failures. Exclude them from rsync:

rsync -avz \
  --exclude='data-backup' \
  --exclude='node_modules' \
  --exclude='.next' \
  --exclude='.git' \
  --exclude='data/classrooms/*/audio' \
  --exclude='scripts/backfill-actions-tts.ts' \
  --exclude='scripts/qc-speech-continuity.ts' \
  /Users/jinguo/PycharmProjects/OpenMAIC/ \
  hp:/opt/openmaic/

If the build already failed, remove the offending files on hp and rebuild:

ssh hp "rm /opt/openmaic/scripts/backfill-actions-tts.ts /opt/openmaic/scripts/qc-speech-continuity.ts"
ssh hp "cd /opt/openmaic && docker build -t jinguoqian/openmaic:latest ."

3. Stop old container and deploy

ssh hp "cd /opt/openmaic && docker compose down"

PITFALL — docker-compose.yml references openmaic-openmaic, not jinguoqian/openmaic: The docker-compose.yml uses build: . which tags the image as openmaic-openmaic:latest (local name). When you build manually with -t jinguoqian/openmaic:latest, docker-compose still references the old local tag. After building, tag the new image to the local name:

ssh hp "docker tag jinguoqian/openmaic:latest openmaic-openmaic:latest"
ssh hp "cd /opt/openmaic && docker compose up -d"

This ensures docker-compose picks up the freshly built image instead of the stale cached one. Without this step, the old image runs and new files (like subtitle-overlay.tsx) won't be present in the container.

Or deploy manually:

ssh hp "docker stop openmaic && docker rm openmaic"
ssh hp "docker run -d \
  --name openmaic \
  -p 8003:3000 \
  -v /opt/openmaic/data:/app/data \
  --restart unless-stopped \
  jinguoqian/openmaic:latest"

4. Verify

ssh hp "docker ps --filter name=openmaic --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}\t{{.Ports}}'"
ssh hp "curl -s -o /dev/null -w 'HTTP %{http_code}' http://localhost:8003/"
ssh hp "curl -s -o /dev/null -w 'HTTP %{http_code}' http://localhost:8003/api/health"

Expected: HTTP 200 on both endpoints.

Docker Compose Config

On hp, /opt/openmaic/docker-compose.yml:

services:
  openmaic:
    build: .
    container_name: openmaic
    ports:
      - "8003:3000"
    env_file:
      - .env.local
    environment:
      - APP_BASE_URL=http://localhost:8003
    volumes:
      - ./data:/app/data
    restart: unless-stopped

Environment Variables

.env.local on hp contains all API keys (TTS, ASR, LLM providers, etc.). These are not synced from the local machine — they already exist on hp at /opt/openmaic/.env.local and are preserved across deploys.

Pitfalls

1. data-backup exclusion: Always exclude data-backup/ from rsync (6GB of audio). Also exclude data/classrooms/*/audio/ (21GB+).

2. Build context size: Without the exclusions above, Docker build context is 6GB+ and COPY . . takes 90+ seconds. With exclusions, it's ~100MB and takes ~2s.

3. .env.local is NOT synced: The .env.local on hp is separate from the local one. Do NOT rsync it or you'll overwrite hp's API keys.

4. docker compose vs docker run: Prefer docker compose on hp since it reads .env.local automatically. docker run --env-file needs the file path.

5. Architecture: hp is x86_64, local dev machine is arm64. Build on hp directly — cross-arch builds via buildx fail because buildkit containers can't reach Alpine repos through the proxy.

6. Permission denied on data dir: rsync from macOS preserves uid 501 and 700 permissions. Container runs as nextjs (uid 1001) and can't read the data dir. Fix after every deploy:

   ssh hp "chmod -R 755 /opt/openmaic/data"
   ssh hp "docker restart openmaic"
   

   This must be done EVERY time the container is recreated (docker compose down + up), because Docker re-mounts the volume with the filesystem's existing permissions.

7. Staging-first workflow: The user requires visual verification on Cloudflare Pages BEFORE deploying to hp. Never auto-deploy to hp after making changes. Workflow: deploy to CF Pages staging → give user URL → wait for explicit approval → sync to hp.

8. Docker image tag mismatch: docker-compose.yml references openmaic-openmaic (the image name from build: .), but docker build -t jinguoqian/openmaic:latest creates a differently-named image. After building, tag it:

   ssh hp "docker tag jinguoqian/openmaic:latest openmaic-openmaic:latest"
   

   Otherwise will use the stale cached image.