north-mini-code

name	north-mini-code
description	Wires Cohere's North Mini Code 1.0 (30B MoE, Apache 2.0) into your coding workflow: detects your hardware tier, selects the right quantization (BF16 / FP8 / W4A16), starts a vLLM or TGI server, configures OpenCode or Claude Code to route through it, and sets token-limit guardrails. Falls back to the Cohere API or OpenRouter if no compatible GPU is found.
version	1.0.0
category	integration
platforms	["CLAUDE_CODE","CODEX_CLI"]

You are a North Mini Code setup agent. Do NOT ask the user questions. Detect the environment, choose the optimal deployment path, and configure end-to-end.

TARGET PROJECT: $ARGUMENTS

============================================================ PHASE 1: DETECT HARDWARE AND ENVIRONMENT

1. Check for NVIDIA GPU availability and VRAM:

   nvidia-smi --query-gpu=name,memory.total --format=csv,noheader 2>/dev/null || echo "no-gpu"
   

2. Determine deployment tier from VRAM:

   • ≥ 80 GB → FP8 quantization on a single H100/A100-80GB (recommended)
   • 40–79 GB → W4A16 quantization on A100-40GB or 2× consumer cards
   • < 40 GB → No local GPU path; use managed API

3. Check for vLLM: python -m vllm.entrypoints.openai.api_server --help 2>/dev/null | head -1

4. Check for TGI: text-generation-launcher --help 2>/dev/null | head -1

5. Check for existing North Mini Code server: curl -s http://localhost:8000/health

6. Check environment variables: COHERE_API_KEY, OPENROUTER_API_KEY, ANTHROPIC_API_KEY

7. Check for OpenCode: which opencode 2>/dev/null

8. Check for Claude Code settings: .claude/settings.json

Report findings as a checklist before proceeding.

============================================================ PHASE 2: INSTALL INFERENCE SERVER (if local GPU found)

If vLLM is not installed and a compatible GPU is present:

pip install vllm --upgrade

If vLLM install fails and TGI is also absent:

# Try TGI as fallback
pip install text-generation

If neither is installable, skip to PHASE 3 (managed API path).

============================================================ PHASE 3: PULL MODEL WEIGHTS (local path only)

Select the Hugging Face model variant based on the VRAM tier determined in Phase 1:

Tier	Model ID
FP8	CohereLabs/north-mini-code-1.0-fp8
W4A16	CohereLabs/north-mini-code-1.0-w4a16
BF16 full	CohereLabs/north-mini-code-1.0

Download via Hugging Face CLI (preferred):

pip install huggingface_hub
huggingface-cli download <MODEL_ID> --local-dir ~/.cache/north-mini-code

Or set up streaming pull (model downloads on first request):

# vLLM will stream weights from HF Hub on first launch if local-dir not provided
# Set HF_HOME to a fast SSD path for best performance
export HF_HOME=~/.cache/huggingface

============================================================ PHASE 4: CONFIGURE AND START THE INFERENCE SERVER

vLLM path (preferred):

Create ~/.config/north-mini-code/start-server.sh:

#!/usr/bin/env bash
# North Mini Code 1.0 — vLLM inference server
# Generated by the north-mini-code integration skill

MODEL_ID="${NORTH_MINI_CODE_MODEL:-CohereLabs/north-mini-code-1.0-fp8}"
PORT="${NORTH_MINI_CODE_PORT:-8000}"
MAX_LEN="${NORTH_MINI_CODE_MAX_LEN:-65536}"

python -m vllm.entrypoints.openai.api_server \
  --model "$MODEL_ID" \
  --dtype auto \
  --max-model-len "$MAX_LEN" \
  --port "$PORT" \
  --served-model-name north-mini-code \
  --enable-chunked-prefill \
  --max-num-seqs 8

Make executable and start:

chmod +x ~/.config/north-mini-code/start-server.sh
~/.config/north-mini-code/start-server.sh &

Wait for the health endpoint to respond (poll up to 120s):

for i in $(seq 1 24); do
  curl -s http://localhost:8000/health && echo "Server ready" && break
  echo "Waiting for server... ($i/24)"
  sleep 5
done

If the server does not respond after 120s, print the last 20 lines of stdout for diagnosis and fall through to the managed API path.

TGI path (fallback):

text-generation-launcher \
  --model-id CohereLabs/north-mini-code-1.0-fp8 \
  --max-total-tokens 65536 \
  --port 8001 &

============================================================ PHASE 5: CONFIGURE MANAGED API FALLBACK

If no local GPU is available (or the server failed to start):

Priority order:

1. Cohere API (if COHERE_API_KEY is set)
2. OpenRouter (if OPENROUTER_API_KEY is set)
3. Print manual setup instructions and exit

Cohere API config:

{
  "endpoint": "https://api.cohere.com/v2/chat",
  "model": "north-mini-code-1.0",
  "apiKey": "$COHERE_API_KEY"
}

OpenRouter config (OpenAI-compatible):

{
  "baseURL": "https://openrouter.ai/api/v1",
  "model": "cohere/north-mini-code-1.0",
  "apiKey": "$OPENROUTER_API_KEY"
}

============================================================ PHASE 6: WIRE INTO OPENCODE (if installed)

If OpenCode is installed, update ~/.config/opencode/config.json:

1. Read the existing config (create {} if absent).
2. Add or update a north-mini-code provider block:

{
  "providers": {
    "north-mini-code": {
      "baseUrl": "http://localhost:8000/v1",
      "apiKey": "local"
    }
  },
  "models": {
    "review": "north-mini-code/north-mini-code",
    "arch": "north-mini-code/north-mini-code"
  }
}

If using the managed API path, set baseUrl to the Cohere or OpenRouter endpoint and apiKey to "$COHERE_API_KEY" or "$OPENROUTER_API_KEY".

============================================================ PHASE 7: WIRE INTO CLAUDE CODE (if .claude/settings.json found)

If .claude/settings.json exists in the target project:

1. Add a modelAliases entry pointing to the North Mini Code endpoint:

{
  "modelAliases": {
    "north-mini-code": {
      "baseURL": "http://localhost:8000/v1",
      "model": "north-mini-code",
      "apiKey": "local"
    }
  }
}

2. Optionally add a hooks.PostToolUse entry to log token usage to a local file:

{
  "hooks": {
    "PostToolUse": [
      {
        "matcher": "*",
        "hooks": [
          {
            "type": "command",
            "command": "echo \"$(date -u) tokens:$CLAUDE_TOOL_OUTPUT_TOKENS\" >> ~/.claude/north-mini-code-usage.log"
          }
        ]
      }
    ]
  }
}

============================================================ PHASE 8: SET TOKEN-LIMIT GUARDRAILS

North Mini Code generates 2–3× more output tokens than comparable models on the same tasks. Set hard limits to prevent cost overruns on managed APIs and runaway sessions on self-hosted infrastructure.

Create ~/.config/north-mini-code/limits.env:

# Token guardrails — source this before starting agent sessions
export NORTH_MINI_CODE_MAX_OUTPUT_TOKENS=16384   # 16K per request
export NORTH_MINI_CODE_MAX_SESSION_TOKENS=131072  # 128K per session
export NORTH_MINI_CODE_WARN_AT_TOKENS=65536       # warn at 64K output

Print a reminder:

TOKEN LIMIT ADVISORY
====================
North Mini Code produces ~3× the output token volume of comparable models.

Default limits set:
  Per-request max  : 16,384 tokens
  Per-session max  : 131,072 tokens
  Warning threshold: 65,536 tokens

Adjust these in ~/.config/north-mini-code/limits.env before first use.
Managed API users: also set a monthly spend cap on the Cohere or OpenRouter dashboard.
  Cohere API:   https://dashboard.cohere.com/billing
  OpenRouter:   https://openrouter.ai/settings/keys

============================================================ PHASE 9: VALIDATE SETUP

Run a smoke test:

curl -s http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer local" \
  -d '{
    "model": "north-mini-code",
    "messages": [{"role": "user", "content": "Write a Python function that reverses a string."}],
    "max_tokens": 256
  }' | python3 -c "import sys,json; r=json.load(sys.stdin); print('✓ North Mini Code responding —', r['usage']['completion_tokens'], 'tokens generated')"

If the test fails with a connection error, diagnose:

• Server not started → re-run start-server.sh
• Wrong port → check NORTH_MINI_CODE_PORT env var
• Auth error on managed API → verify API key env var is set

============================================================ PHASE 10: DELIVERABLE SUMMARY

Print a concise summary:

NORTH MINI CODE SETUP COMPLETE
==============================
Deployment path : <local vLLM | local TGI | Cohere API | OpenRouter>
Model variant   : <BF16 | FP8 | W4A16>
Endpoint URL    : <http://localhost:8000 | managed API URL>

Integrations wired:
  OpenCode        : <yes — ~/.config/opencode/config.json | not installed>
  Claude Code     : <yes — .claude/settings.json | not found>

Token limits:
  Per-request max : 16,384 tokens
  Per-session max : 131,072 tokens

Session start:
  opencode          # interactive with North Mini Code as review/arch model
  /model north-mini-code  # switch to it mid-session in OpenCode

Start server manually (local path):
  ~/.config/north-mini-code/start-server.sh

Spend cap reminder: set monthly limits on your API dashboard before first heavy use.