bigmac-design

name	bigmac-design
description	Use when working on BigMac code, tests, docs, or architecture involving multimodal LLM training, nested pipeline scheduling, encoder/LLM/generator disaggregation, 1F1B or interleaved pipeline parallelism, scheduler/executor logic, communication operators, module handoff, microbatch units, activation memory, or compute/memory tradeoffs.

BigMac Design

Overview

Use this skill to keep BigMac code changes aligned with the paper's design: disaggregate encoder, LLM, and generator compute; nest encoder/generator units into an unchanged LLM pipeline; preserve dependency-safe execution; and avoid reintroducing the compute-memory Pareto tradeoff BigMac is designed to break.

Workflow

Before changing BigMac code, identify the subsystem and load the relevant reference:

• Scheduler, schedule construction, warmup, microbatch units, operator ordering, or activation memory: read references/scheduler.md.
• Executor, opcode interpretation, runtime buffers, async communication handles, module handoff, gradient propagation, or DP accumulation semantics: read references/executor.md.
• Communication insertion, gather/scatter, point-to-point matching, or deadlock checks: read references/scheduler.md and references/executor.md.
• Megatron-Core MLLM integration, Qwen3-VL-style vision encoder disaggregation, separate VE DDP/optimizer wiring, or native-vs-BigMac correctness checks: read references/megatron-multimodal-implementation.md.
• Product/design framing, why BigMac exists, or how to explain a code choice in docs/reviews: read references/design-principles.md.

Design Invariants

Preserve these invariants unless the user explicitly asks to change BigMac's design:

• Keep the LLM pipeline schedule as the base truth. BigMac inserts encoder/generator work around the LLM schedule; it must not reorder LLM operators or add LLM pipeline dependencies that change the user-specified 1F1B/interleaved schedule.
• Treat encoder, LLM, and generator compute as disaggregated components. Avoid implementing them as one monolithic pipeline stage sequence unless the goal is explicitly to model a baseline.
• Use pp_size microbatches as the basic encoder/generator scheduling unit.
• Schedule encoder/generator backward as soon as dependency-safe. The point is to release activations promptly: BigMac should hold only O(1) encoder activations and one generator activation, while leaving LLM activation behavior unchanged.
• For MLLMs with generators, use warmup bound W = 4; without generators, W = 3 can be sufficient. Do not make warmup an arbitrary performance knob without preserving LLM pipeline progress.
• Model communication explicitly. Compute order alone is not executable; insert pre/post communication operators that materialize producer-consumer dependencies.
• Run or preserve a deadlock check after communication insertion. Every receive must be matchable with a corresponding send in an executable order.
• Preserve asynchronous overlap only when tensor lifetime and readiness events make it safe. Wait before consuming received tensors or freeing/reusing tensors still referenced by pending sends.
• Keep scheduler logic framework-independent and executor logic runtime-adaptive. The scheduler should reason about operators and dependencies; the executor should dispatch to Megatron/LLM runtime, distributed communication, and encoder/generator module functions.

Implementation Bias

When making code decisions:

• Prefer small, testable operator abstractions with explicit metadata: module type, forward/backward direction, microbatch id, pipeline stage, virtual chunk id when relevant, and dependency edges.
• Encode readiness predicates such as llm_output_ready and encoder_grad_ready directly instead of relying on incidental list positions.
• Keep module handoff metadata explicit: name, shape, dtype, sequence dimension, gradient requirement, source layout, and destination layout.
• Treat generator support as symmetric to encoder support on the LLM output side, but remember the dependency direction is reversed: generator backward produces gradients that trigger LLM backward.
• When adapting to existing code, trust the code's actual APIs and tests, but use this skill to judge whether a change violates BigMac's intended architecture.

Validation Checklist

For scheduler or executor changes, verify the relevant items:

• Existing LLM schedule order is unchanged.
• Encoder forward units are available before their LLM consumers.
• Encoder backward units run after LLM input gradients are available and release saved encoder activations.
• Generator forward units run after last-stage LLM outputs are ready, and generator backward gradients reach the LLM backward path.
• Communication operators are inserted for both pipeline and cross-module edges.
• Deadlock checker or equivalent send/receive pairing validation still runs.
• Runtime buffers are indexed by microbatch id and virtual chunk id when interleaved 1F1B is active.
• DP gradient synchronization is suppressed during non-final accumulation steps and finalized at schedule end.

Source

This skill is distilled from the BigMac paper repository, especially sections/abstract.tex, sections/introduction.tex, sections/background.tex, sections/overview.tex, and sections/design.tex.