classifier-lab-subagent

# /classifier-lab-subagent Validate whether subagents can reliably run long-running deterministic tasks. ## The Experiment 3 subagents run concurrently, each training a different text classifier model on the same HuggingFace dataset (`ag_news`). Each subagent runs `/classifier-lab`'s existing self-improvement loop. The orchestrator watches JSON streams, enforces timeouts, and collects results. **The point:** Determine if subagents can reliably perform long-running tasks when 90% of the intelligence is in the code and the subagent is just a process runner. ## Architecture ``` Project agent (interactive session) │ ├── Decides: dataset, 3 models, initial HPs, gate threshold ├── Writes: config YAMLs for each model ├── Tests: preflight sanity (1 epoch each) │ └── orchestrator.py (asyncio) ├── Subagent 1: distilbert-base-uncased ├── Subagent 2: bert-tiny └── Subagent 3: all-MiniLM-L6-v2 │ Each subagent: 1. Runs classifier-lab/run.sh with config 2. Script streams JSON to stdout 3. If crash → debug, write /test-lab test, retry 4. If blocked → report to orchestrator 5. Between rounds: /scillm Gemini for structured HP suggestions ``` ## What the Subagent Gets ``` Run this command: classifier-lab/run.sh e2e --task "ag_news" --dataset ag_news --modality text \ --backbones {model} --gate-f1 0.95 --max-rounds 5 The script streams JSON progress. Let it run. If it crashes: read the error, fix the obvious issue, write a /test-lab test, rerun. If you can't fix it after 3 attempts: report "blocked" with the error. Do not modify training logic. Do not change the self-improvement loop. Do not add features. Just run it and report. ``` ## What the Orchestrator Does - Spawns 3 subagents via /subagent-service - Reads JSON streams from each - Heartbeat: no output for 120s → kill - Wall-clock: 30min max per subagent - Collects final metrics from all 3 - Reports: which succeeded, which failed, which blocked - Computes subagent reliability rate ## HP Adjustment via /scillm (not agent judgment) Between rounds, the self-improvement loop calls /scillm Gemini with structured JSON: ```json POST http://localhost:4001/v1/chat/completions { "model": "text", "messages": [{"role": "user", "content": "...round history..."}], "response_format": {"type": "json_object"} } ``` Returns: ```json { "learning_rate": 3e-5, "batch_size": 16, "epochs": 5, "dropout": 0.2, "reasoning": "Overfitting after epoch 3, reduce LR and increase dropout" } ``` The code applies this directly. No agent interpretation needed. ## Usage ```bash # Run the full experiment (3 concurrent subagents) ./run.sh experiment # Preflight only (test each model for 1 epoch) ./run.sh preflight # Single model test (no subagent, local) ./run.sh single --model distilbert-base-uncased # Check results ./run.sh results ``` ## Success Criteria The experiment succeeds if: 1. All 3 subagents complete (not blocked/crashed) 2. At least 2/3 reach the accuracy gate (95%) 3. JSON streams were continuous (no heartbeat timeouts) 4. Total wall-clock time < 90 minutes This tells us: subagents CAN reliably run deterministic code with monitoring. If it fails: subagents can't even babysit a script, and we know the ceiling.