// Use when working with Paddle's distributed training system: understanding parallelism strategies (DP, ZeRO, TP, PP, SP), semi-automatic parallel with ProcessMesh + shard_tensor, SPMD inference rules, pipeline scheduling, or auto_parallel Engine.
| name | paddle-design-distributed |
| description | Use when working with Paddle's distributed training system: understanding parallelism strategies (DP, ZeRO, TP, PP, SP), semi-automatic parallel with ProcessMesh + shard_tensor, SPMD inference rules, pipeline scheduling, or auto_parallel Engine. |
| 范式 | 核心思想 | 通信原语 |
|---|---|---|
| Data Parallel | 复制模型，切分数据，AllReduce 梯度 | AllReduce |
| Group Sharded (ZeRO) | Stage1 切 optimizer / Stage2 + 切 grad / Stage3 + 切 weight | Broadcast, ReduceScatter, AllGather |
| Model Parallel (Tensor) | Column Parallel 切权重列 / Row Parallel 切权重行 | AllReduce / AllGather |
| Pipeline Parallel | F-then-B / 1F1B 交错前反向 | Send / Recv (P2P) |
| Sequence Parallel | 沿 sequence 维度切分 LayerNorm/Dropout | AllGather / ReduceScatter |
| 范式 | 入口 | 适用场景 |
|---|---|---|
| 手动并行 | fleet.meta_parallel | 灵活但代码量大，适合深度定制 |
| 半自动动态图 | ProcessMesh + shard_tensor | 用户标注切分方式，框架自动推导通信，兼具易用性和灵活性 |
| 半自动静态图 | auto_parallel.Engine | 基于 PIR 做全局优化，追求极致性能 |
import paddle.distributed as dist
mesh = dist.ProcessMesh([0, 1, 2, 3], dim_names=["x"])
x = dist.shard_tensor(x, mesh, [dist.Shard(0)]) # 沿 dim 0 切分
半自动并行的核心：用户只标注部分 Tensor 的切分方式（Placement），框架自动推导每个算子的输入输出应如何切分，必要时插入通信算子。
matmul(X[M,K], Y[K,N]) -> Z[M,N] → mk,kn->mn使用 DimTrans 系统描述维度映射：InputDim / Flatten / Split / Singleton。
| 调度策略 | 特点 |
|---|---|
| F-then-B | 所有前向完毕再反向，简单但显存占用大 |
| 1F1B | warm-up → steady（前反交替）→ cool-down，显存减少约 37.5% |
| 场景 | 参考文档 |
|---|---|
| 分布式策略原理（DP/Sharded/MP/PP/SP） | references/distributed-primer.md |
| SPMD 推导规则与 Pipeline 调度 | references/spmd-rules.md |
| 模块 | 路径 |
|---|---|
| 分布式总目录 | python/paddle/distributed/ |
| ProcessMesh / shard_tensor API | python/paddle/distributed/auto_parallel/ |
| auto_parallel Engine | python/paddle/distributed/auto_parallel/high_level_api.py |
| fully_shard（ZeRO-like） | python/paddle/distributed/auto_parallel/fully_shard.py |
| SPMD 推导规则 C++ | paddle/phi/infermeta/spmd_rules/ |
| SPMD 规则注册 | paddle/phi/infermeta/spmd_rules/rules.h |
| Pipeline 调度 Pass | python/paddle/distributed/passes/pipeline_scheduler_pass/ |
| 分布式策略配置 | python/paddle/distributed/auto_parallel/strategy.py |
| SPMD 规则测试 | test/auto_parallel/spmd_rules/ |
将原生 PyTorch 自定义算子库、Torch extension、生态库（TorchCodec/FlashInfer/DeepEP 等）以及 Kernel DSL 生态（Triton/TileLang/TVM FFI 等）以最小修改方式接入 PaddlePaddle。遇到以下场景务必使用：迁移外部算子库到 Paddle；分析 PFCCLab fork 与上游的兼容差异；处理 paddle.enable_compat、paddle.utils.cpp_extension、TORCH_LIBRARY、torch.ops、at::Tensor/c10 compat 问题；为 compat gap 设计最小 workaround 并准备 Paddle issue 最小复现。
在 Paddle 代码库中定位问题并输出高质量调试报告的专用技能。当遇到以下场景时优先使用：(1) Paddle 框架 bug 调试，(2) 算子实现问题排查，(3) 训练脚本异常诊断，(4) 分布式训练故障定位，(5) CUDA/GPU 相关错误处理，(6) 需要生成结构化调试报告。
PaddlePaddle (飞桨) C++ 算子开发指南。提供从 YAML 配置、InferMeta 函数、Kernel 实现、Python API 封装、单元测试到编译验证的完整算子开发流程指导。在以下场景使用此 skill：(1) 为 Paddle 框架新增 C++ 算子 (2) 修改或调试已有 Paddle 算子 (3) 编写算子的 YAML 配置、InferMeta、Kernel、Python API 或单元测试 (4) 理解 Paddle 算子开发架构和流程 (5) 编译 Paddle 并验证算子正确性
Use when needing to compile, rebuild, or install Paddle from source after code changes. Covers cmake configuration, ninja incremental build, wheel packaging, and common build failure diagnosis.
Use when working with Paddle 3.0 compiler full pipeline: SOT (Symbolic Opcode Translator) for bytecode-level dy2st graph capture, PIR (Paddle IR) for SSA-based intermediate representation, CINN for fused CUDA kernel generation, operator decomposition (Prim), or the end-to-end flow from Python eager code to optimized GPU execution.
Use when navigating Paddle eager-mode (dynamic graph) source code, tracing forward/backward execution, debugging autograd issues, understanding PyLayer, or investigating complex-valued gradient computation. Covers Python API to C++ kernel call chain, backward graph topology sort, and inplace version tracking.