run-thor

name	run-thor
description	Deploy and test cyber-inference on the Thor lab server. Use when the user wants to test on Thor, deploy to Thor, run on Thor, verify the server, or mentions thor.lab.

Deploy & Test on Thor

Thor is the GPU lab server used for integration testing of cyber-inference. It is accessible via SSH and hosts the production-like test environment.

Connection Details

Field	Value
Host	thor.lab
User	matt
SSH	ssh [e-mail]
Project path	/home/matt/Local/cyber-inference
Server URL	http://thor.lab:8337

Deploy Workflow

Follow these steps in order. Each depends on the previous.

1. Commit & Push (local machine)

git add -A && git commit -m "<message>" && git push

2. Pull on Thor (remote)

ssh [e-mail] "cd /home/matt/Local/cyber-inference && git pull"

3. Start the Server (remote)

The server runs via start.sh which handles uv sync and auto-restart.

# Interactive (see logs live) - use for debugging
ssh -t [e-mail] "cd /home/matt/Local/cyber-inference && ./start.sh"

# Background (detached) - use for long-running tests
ssh [e-mail] "cd /home/matt/Local/cyber-inference && nohup ./start.sh > /tmp/cyber-inference.log 2>&1 &"

CUDA PyTorch wheels are verified automatically when NVIDIA hardware is detected.

4. Verify the Server

# Health check
curl -s http://thor.lab:8337/health

# List models
curl -s http://thor.lab:8337/v1/models | python3 -m json.tool

# System status
curl -s http://thor.lab:8337/admin/status | python3 -m json.tool

The web GUI is available at: http://thor.lab:8337

5. Test Inference

# Chat completion
curl -s http://thor.lab:8337/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{"model": "<model_name>", "messages": [{"role": "user", "content": "Hello"}]}' \
  | python3 -m json.tool

# Embeddings
curl -s http://thor.lab:8337/v1/embeddings \
  -H "Content-Type: application/json" \
  -d '{"model": "<model_name>", "input": "test text"}' \
  | python3 -m json.tool

Quick One-Liner Deploy

Pull latest and restart in one command:

ssh -t [e-mail] "cd /home/matt/Local/cyber-inference && git pull && ./start.sh"

Troubleshooting

• Server won't start: Check logs with ssh [e-mail] "tail -50 /tmp/cyber-inference.log"
• Port in use: Kill existing process with ssh [e-mail] "pkill -f 'cyber-inference serve'"
• Check running processes: ssh [e-mail] "ps aux | grep cyber-inference"
• GPU/CUDA issues: ssh [e-mail] "nvidia-smi"