| name | simforge |
| description | Manage chip verification regression runs using the simforge CLI tool. Use this skill whenever the user wants to submit, monitor, cancel, or analyze regression runs, wants to run simulation testcases in parallel, needs to merge coverage data from multiple simulation runs, asks about testplan YAML, tag-based case filtering, regression status, or failed case analysis — even if they don't explicitly say "simforge". |
simforge — Chip Verification Regression Framework
A lightweight CLI tool for managing simulation regressions asynchronously.
Each project directory runs a background daemon that schedules and tracks
simulation cases. Invoke it via shell commands — no SDK, no Python import.
Workflow
submit → status → results → merge-cov → close
simforge submit — filter testplan by tags, dispatch all cases asyncsimforge status — poll progress (pending / running / passed / failed)simforge results — get per-case detailed resultssimforge merge-cov — merge coverage databases after run completessimforge close — shutdown the daemon when done
The daemon auto-launches on first submit. It auto-shuts down after 3600s idle.
Where to Run simforge
simforge operates relative to a project directory — the root of a DV project.
By default it uses the current working directory (CWD) as the project dir.
Always cd into the project root before running commands, or pass --project-dir <path> explicitly.
cd /proj/dv/my_ip
simforge submit --tags smoke
Or from anywhere:
simforge submit --project-dir /proj/dv/my_ip --tags smoke
The daemon is per-project-dir: each unique directory gets its own daemon process.
Multiple engineers can run separate daemons for different projects on the same machine.
Required Directory Structure
<project_root>/ ← run simforge from here (or --project-dir)
├── cases/
│ └── testplan.yaml ← REQUIRED: defines all simulation cases
├── regress.cfg.yaml ← optional: parallel count, backend, run_dir
└── runs/ ← auto-created by simforge on first submit
└── 20240101_001/ ← one dir per run_id
├── config_snapshot.json
├── final_status.json ← written when run ends (completed/cancelled/error)
├── tc_smoke_1/
│ ├── run.sh
│ └── sim.log
└── tc_smoke_2/
├── run.sh
└── sim.log
Minimum Required Files
| File | Required | Notes |
|---|
cases/testplan.yaml | Yes | Must exist before submit. Use --testplan to point to a different path |
regress.cfg.yaml | No | Place in project root; overrides defaults for backend/parallelism |
testplan.yaml path resolution
- Default:
<project_root>/cases/testplan.yaml
- Override:
simforge submit --testplan path/to/other.yaml (relative to project root)
- Check what's in the testplan without submitting:
simforge list
Run output
- Default run output dir:
<project_root>/runs/ (controlled by run_dir in regress.cfg.yaml)
- Each run gets a sub-directory named by
run_id (YYYYMMDD_NNN)
- Each case gets its own sub-directory inside the run dir:
<run_dir>/<run_id>/<case_name>/
- Simulation log is at
<run_dir>/<run_id>/<case_name>/sim.log
- Coverage database (if produced) at
<run_dir>/<run_id>/<case_name>/coverage.vdb
- Merged coverage after
merge-cov: <run_dir>/<run_id>/merged_cov/
Daemon state
simforge stores daemon registry and auth tokens in ~/.simforge/:
~/.simforge/
├── registry.json ← tracks all running daemons (host, port, pid)
├── remote/
│ ├── s-<user>-<id>.token ← per-daemon auth token (chmod 600)
│ └── s-<user>-<id>.ready ← port/pid info written at startup
└── logs/
└── s-<user>-<id>.log ← daemon log
This directory is managed automatically — no manual setup needed.
Commands
submit
Submit all matching cases from the testplan for async execution.
simforge submit [options]
| Parameter | Default | Description |
|---|
--testplan <path> | cases/testplan.yaml | Testplan file path (relative to project dir) |
--tags <t1,t2> | all | Include only cases with ALL listed tags |
--exclude-tags <t1,t2> | none | Exclude cases with ANY listed tag |
-j <n> | 4 | Max parallel simulation jobs |
--backend <local|lsf> | local | Job scheduler backend |
--seed <n|random> | random | Global simulation seed |
--timeout <sec> | from testplan | Override per-case timeout |
--project-dir <dir> | CWD | Project root directory |
--session <id> | auto | Isolate daemons for parallel workflows |
--json | off | JSON output |
Returns: run_id, total case count, list of case names, run directory path.
Example:
simforge submit --tags smoke -j 8
simforge submit --tags regress --exclude-tags slow -j 16 --seed 42
status
Poll the progress of a regression run.
simforge status [--run-id <id>] [--project-dir <dir>] [--json]
Returns: run status (running/completed/cancelled), counts of total/passed/failed/running/pending cases, elapsed time, list of currently running and failed case names.
Example:
simforge status
simforge status --run-id 20240101_002
simforge status --json
cancel
Cancel a running regression.
simforge cancel [--run-id <id>] [--project-dir <dir>] [--json]
Returns: run status, number of killed running jobs, number of cancelled pending jobs.
results
Get per-case detailed results.
simforge results [--run-id <id>] [--filter all|failed|passed] [--project-dir <dir>] [--json]
| Parameter | Default | Description |
|---|
--filter | all | Show all / only failed / only passed cases |
Returns: summary counts + per-case table with: name, status, seed, elapsed_sec, log_file path, error snippet.
Example:
simforge results --filter failed
simforge results --json
merge-cov
Merge per-case coverage databases into a single merged report.
simforge merge-cov [--run-id <id>] [--tool urg|imc|vcover] [--project-dir <dir>] [--json]
Returns: path to merged coverage directory, merger tool used, number of databases merged.
Auto-detects tool from simulator: VCS → urg, Xrun → imc, QuestaSim → vcover.
list
List testplan cases without starting a daemon. Useful for previewing tag filters.
simforge list [--testplan <path>] [--tags <t1,t2>] [--exclude-tags <t1,t2>]
[--project-dir <dir>] [--json]
Returns: total count + list of case names and their tags.
Example:
simforge list --tags smoke
simforge list --json | python3 -c "import sys,json; d=json.load(sys.stdin); print(d['total'])"
list-runs
List all runs: includes active (in-memory) runs from the current daemon and completed runs persisted to disk from past sessions.
simforge list-runs [--project-dir <dir>] [--session <id>] [--json]
Each entry shows: run_id, status, total/passed/failed, elapsed_sec, finished_at (disk runs), and source (live or disk).
Use this to see the history of all past regressions in a project, even after daemon restarts.
ps
Global view of all running daemons across sessions.
simforge ps [--clean] [--json]
| Parameter | Description |
|---|
--clean | Remove stale (dead process) registry entries |
Returns: list of all active daemons with their run summaries, plus stale entries.
close
Shutdown the daemon for this project.
simforge close [--all] [--project-dir <dir>] [--session <id>] [--json]
| Parameter | Description |
|---|
--all | Close all daemons in this session |
Testplan YAML Format
The testplan file (cases/testplan.yaml by default) defines all simulation cases:
global:
simulator: vcs
top_module: tb_top
compile_opts: "-f rtl/rtl.f"
sim_opts: "+UVM_VERBOSITY=UVM_LOW"
timeout: 3600
plusargs: []
cases:
- name: tc_basic_write
tags: [smoke, regress]
sim_opts: "+TESTNAME=basic_write"
- name: tc_burst_read
tags: [regress]
timeout: 7200
sim_opts: "+TESTNAME=burst_read"
- name: tc_error_inject
tags: [regress, slow]
plusargs: ["+inject_mode=1"]
Tag Filtering Rules
--tags t1,t2 — includes cases that have ALL listed tags (intersection)--exclude-tags t1,t2 — removes cases that have ANY listed tag (union)- Tags are combined: first apply include filter, then apply exclude filter
Config File (regress.cfg.yaml)
Optional project-level config in the project root:
backend: local
run_dir: runs
local:
max_parallel: 8
lsf:
queue: normal
memory: 4G
extra_opts: "-R 'rusage[mem=4096]'"
Key JSON Fields
submit response
{
"status": "ok",
"run_id": "20240101_001",
"total_cases": 42,
"cases": ["tc_smoke_1", "tc_smoke_2", ...],
"run_dir": "/proj/dv/runs/20240101_001",
"backend": "local",
"max_parallel": 8
}
status response
{
"status": "ok",
"run_id": "20240101_001",
"run_status": "running",
"total": 42,
"passed": 30,
"failed": 2,
"running": 4,
"pending": 6,
"elapsed_sec": 147.3,
"failed_cases": [
{"name": "tc_burst", "seed": 1234567, "elapsed_sec": 32.1, "error_msg": "UVM_FATAL: ..."}
]
}
results response (per case)
{
"name": "tc_burst",
"status": "failed",
"seed": 1234567,
"elapsed_sec": 32.1,
"log_file": "/proj/dv/runs/20240101_001/tc_burst/sim.log",
"coverage_dir": "/proj/dv/runs/20240101_001/tc_burst/coverage.vdb",
"error_msg": "UVM_FATAL @ 1500ns: DUT timeout"
}
final_status.json (on-disk persistence)
Written to <run_dir>/<run_id>/final_status.json when a run ends (completed, cancelled, or error). Enables querying past runs across daemon restarts.
{
"run_id": "20240101_001",
"status": "completed",
"project_dir": "/proj/dv",
"run_dir": "/proj/dv/runs/20240101_001",
"testplan": "cases/testplan.yaml",
"tags": ["smoke"],
"backend": "local",
"created_at": "2024-01-01T10:00:00",
"finished_at": "2024-01-01T10:05:00",
"total": 10,
"passed": 9,
"failed": 1,
"cancelled": 0,
"elapsed_sec": 300.0,
"cases": [
{
"name": "tc_smoke_1",
"status": "passed",
"seed": 42,
"elapsed_sec": 25.3,
"exit_code": 0,
"log_file": "/proj/dv/runs/20240101_001/tc_smoke_1/sim.log",
"error_msg": null,
"coverage_dir": null
}
]
}
Common Patterns
Quick smoke regression
cd /proj/dv
simforge submit --tags smoke -j 4
simforge status --json
simforge results --filter failed
Full regression with coverage
simforge submit --tags regress -j 32 --backend lsf
simforge status
simforge merge-cov
simforge results --json
Debug a failed test
simforge results --filter failed --json | python3 -c "
import sys, json
d = json.load(sys.stdin)
for r in d['results']:
print(r['name'], r['seed'], r['error_msg'])
"
simforge submit --tags smoke --testplan cases/testplan.yaml --seed 1234567
Global status check
simforge ps
simforge ps --clean
simforge close --all
Error Codes
| Code | Meaning |
|---|
NO_CASES_MATCHED | Tag filter matched zero cases — check testplan and tag names |
RUN_NOT_FOUND | run_id doesn't exist in current daemon memory and no final_status.json on disk |
INVALID_TESTPLAN | testplan.yaml parse error |
LAUNCH_FAILED | Daemon failed to start (binary not found or permission error) |
NO_DAEMON | No daemon running for this project (use submit first) |
Practical Notes
- run_id format:
YYYYMMDD_NNN (e.g., 20240101_001). Omit --run-id to use the latest run.
- Logs: Each case writes to
<run_dir>/<case_name>/sim.log.
- Seeds: When a case fails, record the seed from
results output to reproduce exactly.
- Daemon persistence: Daemon survives CLI exit. Run
simforge ps to see all active daemons.
- Run persistence: When a run finishes, a
final_status.json file is written to <run_dir>/<run_id>/. Commands status, results, and list-runs automatically fall back to this file when the run is no longer in daemon memory (e.g., after daemon restart). Use list-runs to see all past runs across sessions.
- Multiple projects: Each project dir gets its own daemon. Use
--session to further isolate.
