Menu
A bank of parallel expert MLPs that consume routed tokens from TopKRouter and return per-token outputs. The "doer" half of an MoE block. Activate when the user asks "MoE experts", "expert MLPs", "Mixtral", "expert parallel", or builds an MoE block.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
Based on SOC occupation classification
Run a short, reproducible benchmark of one optimizer step (forward + backward + optimizer step over N microbatches) using the project's registered runtime. Activate when the user asks to "benchmark a training step", "measure throughput", "time one optimizer step", or "smoke test the runtime". Wraps run_accumulated_step from curry_train.benchmark.
Diagnose a training failure or stall by inspecting recent logs, loss curves, OOM traces, NaN events, and config. Activate when the user asks "why did my training crash", "loss went to NaN", "OOM during step X", "training is not improving", or "help me debug this run". Delegates to the failure-diagnoser agent.
Lightning Fabric integration recipe — minimal 5-line setup that gives DDP / FSDP / mixed precision / mixed-precision while keeping a raw PyTorch training loop. Activate when the user asks "Lightning Fabric", "torchrun", "DDP setup", "FSDP setup", "mixed precision", or wires up the launch script.
Bootstrap a new curryTrain project by copying the framework Python template into the user's working directory. Use when the user runs /curry-train:init, asks to "start a new training project with curryTrain", "initialize curryTrain", "scaffold a curryTrain project", or wants the framework code copied locally for editing.
Scaffold a new model + dataset + config triple inside an existing curryTrain project. Activate when the user asks to "add a new model to curryTrain", "scaffold a new experiment", "generate model.py / dataset.py / config.yaml for X", or describes an idea they want to start training. Delegates the actual generation to the scaffolder agent.
Compare two training runs and produce a concise markdown diff covering config, key metrics, loss curves, and grad-norm trajectory. Activate when the user asks to "compare run A and run B", "diff two experiments", "did this change actually help", or "is this run better than the previous one". This is both the implementation of the action and the methodology guide for variance-aware comparison.
| name | primitive-experts |
| description | A bank of parallel expert MLPs that consume routed tokens from TopKRouter and return per-token outputs. The "doer" half of an MoE block. Activate when the user asks "MoE experts", "expert MLPs", "Mixtral", "expert parallel", or builds an MoE block. |
The set of expert MLPs in an MoE block, plus the dispatch/combine logic that gathers tokens from the router and returns per-token outputs.
Given:
x: (B, N, D).primitive-topk-router: per-token expert indices and gating weights.Steps:
from curry_train.primitives import Experts
experts = Experts(
d_model=2048,
d_ff=8192, # expert-internal hidden dim
n_experts=8,
activation="silu",
glu=True, # gated-linear-unit (Llama / Mixtral style)
drop_path=0.0,
ep_size=1, # expert-parallel world size; 1 = no EP
)
y = experts(x, route_info) # route_info from router
# y shape: (B, N, D)
When ep_size > 1:
This requires primitive-parallel-state to know rank topology and primitive-distributed-optimizer to handle expert-state sharding.
n_experts = 256 and d_ff = 8192, expert weights dominate. Sharding across EP-ranks is essential at scale.aux_loss_weight in router) significantly affects comms cost.V1: stub at template/curry_train/primitives/experts.py. References: Mixtral (Jiang et al. 2024), DeepSeek-MoE (Dai et al. 2024).
skills/primitive-topk-router — the upstream half.skills/primitive-parallel-state — rank topology for EP.skills/primitive-distributed-optimizer — sharded optimizer state for expert weights.skills/stage4-parallel-primitive-intro — when to add EP.