areno-profile-performance

Measure and diagnose AReno rollout, prefill, decode, training, checkpoint, role-switch, communication, or Python scheduling performance. Use when throughput or step time is slow and evidence from metrics, py-spy, or Nsight is required. Do not optimize before correctness is established.

Jump to install

Repository: inclusionAI/AReno
Last source activity: July 28, 2026 at 03:45
Detected SKILL.md language: English
Stars: 311
Forks: 123

Install options

The review-first prompt is selected by default. You can switch to a direct command or download a local copy.

Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.

A direct command skips the review prompt. Inspect the source before running it.

npx skills add https://github.com/inclusionAI/AReno --skill areno-profile-performance

The command stays on one line. Scroll horizontally to inspect it before copying.

Prefer a local copy? Download the files currently available to SkillsMP.

Review the source files

Read SKILL.md and any companion files shown by SkillsMP before deciding whether to install.

File Explorer

10 files

SKILL.md

Source instructions · Read-only preview

Back to search Back to installation

Profile AReno Performance

Profile a bounded, representative train or serve workload. Collect low-overhead time, GPU, process, and TensorBoard evidence first; use sampling or tracing only after those signals identify the subsystem to inspect.

python .agents/skills/areno-profile-performance/scripts/monitor_gpu.py \ --pid <areno-pid> --duration 60 --output /tmp/areno-gpu.jsonl python .agents/skills/areno-profile-performance/scripts/monitor_process.py \ --pid <areno-pid> --duration 60 --output /tmp/areno-process.jsonl python .agents/skills/areno-profile-performance/scripts/summarize_monitor.py \ /tmp/areno-gpu.jsonl python .agents/skills/areno-profile-performance/scripts/summarize_monitor.py \ /tmp/areno-process.jsonl python .agents/skills/areno-profile-performance/scripts/summarize_events.py \ <metrics-dir> --list python .agents/skills/areno-profile-performance/scripts/summarize_events.py \ <metrics-dir> --pattern 'time/*' --pattern '*throughput*' --drop-first 1 python .agents/skills/areno-profile-performance/scripts/probe_openai_latency.py \ --base-url http://127.0.0.1:8000 --model <model> --requests 16 --concurrency 4 python .agents/skills/areno-profile-performance/scripts/build_nsys_command.py \ --output /tmp/areno-profile -- <bounded-command> [args...]

Workflow

Record workload, commit, model, topology, token lengths, concurrency, GPU, and dependency versions.

Find the parent AReno PID and monitor its process tree. For train jobs, attach during at least two post-warmup steps. For serve jobs, use a fixed request set and concurrency.

Capture per-GPU utilization, memory, power, and target-process memory. Memory capacity, compute utilization, and throughput are separate signals.

For train, list TensorBoard scalar names before selecting series. Summarize stage time, tokens/throughput, communication, optimizer, loss, and memory metrics that actually exist; do not assume fixed tags.

For serve, measure TTFT and total request latency with streaming enabled while the GPU and process monitors run. Record prompt/output lengths and active concurrency.

Exclude initialization, checkpoint load, compilation, and CUDA graph capture from steady-state conclusions, but report them separately when startup is the problem.

Use py-spy record -p <pid> -o /tmp/areno.svg --duration 30 for Python scheduling, data processing, serialization, blocking I/O, or compilation orchestration.

Use a bounded Nsight Systems capture for GPU compute, communication, synchronization, allocator activity, or launch gaps. Read references/profile-policy.md.

Compare baseline and candidate with identical workloads and multiple steady-state observations, then re-run correctness validation after optimization.

Report raw workload metadata, selected window, GPU peak/average memory and utilization, process CPU/RSS, stage breakdown or TTFT/latency, bottleneck evidence, profiler overhead, and before/after metrics. A single warmup step is not a benchmark.