agentic-kernel-gen

name	agentic-kernel-gen
description	Optimize a Deep Learning Kernel for specific hardware backend (e.g., Triton, TileLang) using a 3-tier agentic workflow (Supervisor → Orchestrator → Phase Workers). No MCP tools required.

Agentic Kernel Generation Workflow

Architecture Overview

This skill uses a 3-tier agent architecture to optimize a deep learning reference function into a high-performance kernel:

┌─────────────────────────────────────────────────────────┐
│  Supervisor (YOU — this SKILL.md)                       │
│  - Lightweight: only sees structured summaries          │
│  - Makes high-level go/no-go decisions                  │
│  - Audits workspace files for quality control           │
│  - Communicates final results to the user               │
└────────────────────────┬────────────────────────────────┘
                         │ Task(general-purpose)
                         ▼
┌─────────────────────────────────────────────────────────┐
│  Orchestrator (agents/orchestrator.md)                   │
│  - Manages Phase 0→6 sequencing                         │
│  - Dispatches Phase Worker sub-agents                   │
│  - Handles inter-phase data flow via $WORKSPACE files   │
│  - Returns ORCHESTRATOR_REPORT to Supervisor            │
└──┬──────┬──────┬──────┬──────┬──────┬───────────────────┘
   ▼      ▼      ▼      ▼      ▼      ▼
 Phase0 Phase1 Phase2 Phase3 Phase4 Phase5
   │      │      │      │      │      │
   └──────┴──────┴──────┴──────┴──────┘
   Each phase: self-contained generation + validation + retry loop
   Writes artifacts to $WORKSPACE, returns only a structured summary

Why 3 tiers?

• Supervisor stays context-light (no generated code in context) and can independently audit quality.
• Orchestrator focuses purely on flow control, not user interaction.
• Phase Workers each have fresh context dedicated to their specific task (generation, validation, etc.), preventing context bloat.

Prerequisites

• User must provide a reference function (source code or file path).
• Target device (e.g., cuda) and backend (e.g., triton). If backend is missing, YOU (Supervisor) must ask the user via AskUserQuestion.
• framework will be inferred from the code.

Framework-Specific Guides

Framework	Guide File	Key Notes
PyTorch	framework_specific/framework_pytorch.md	Default, no compat layers needed
PaddlePaddle	framework_specific/framework_paddle.md	Use paddle.enable_compat(scope={"triton"}) for Triton or scope={"tilelang"} for TileLang before importing the backend module. No PyTorch dependency — torch imports are proxied to paddle

---

Supervisor Workflow

Step 1: Gather Inputs & Initialize Workspace

1. Read the reference function from the user (file path or inline code).
2. Infer framework (import torch → pytorch, import paddle → paddle).
3. Determine backend: If not specified, ask the user.
4. Determine backward needs: Does the user want backward pass support?
5. Initialize workspace:

   TASK_ID=$(python -c "import uuid; print(uuid.uuid4())")
   mkdir -p ~/.lmcc/workspace/$TASK_ID
   

6. Write reference function to $WORKSPACE/reference_function.py.

Step 2: Launch Orchestrator

Use Task tool with subagent_type=general-purpose to launch the Orchestrator.

Prompt to Orchestrator (include ALL of the following):

Read $SKILL_ROOT/agents/orchestrator.md for your complete instructions.

SKILL_ROOT: <absolute path to this skill directory>
WORKSPACE: ~/.lmcc/workspace/<task-id>
FRAMEWORK: <pytorch|paddle>
BACKEND: <triton|tilelang>
DEVICE: <cuda>
WITH_BACKWARD: <true|false>
REFERENCE_CODE_PATH: $WORKSPACE/reference_function.py
USER_REQUEST: <original user request text>

Execute the full kernel optimization workflow (Phase 0→6).
Return an ORCHESTRATOR_REPORT when complete.

Step 3: Review Orchestrator Report

The Orchestrator returns a structured report:

ORCHESTRATOR_REPORT:
  workspace: ~/.lmcc/workspace/<task-id>
  status: SUCCESS | FAILED
  phases_completed: [0, 1, 2, 4]
  phase_results:
    phase_2:
      verdict: PASSED
      sampler_file: $WORKSPACE/sample_inputs.py
    phase_4:
      verdict: PASSED
      kernel_file: $WORKSPACE/kernel_function.py
      best_kernel_file: $WORKSPACE/kernel_function_best.py
      summary_file: $WORKSPACE/phase_4_summary.md
      iterations_log: $WORKSPACE/iterations.tsv
      purity_check: PASSED (2 allowed, 0 forbidden)
      correctness:
        verdict: PASSED
        stages: {smoke: PASSED, shape_sweep: PASSED, numerical_stability: PASSED, determinism: PASSED, edge_cases: PASSED}
        samples_passed: 10
        samples_failed: 0
      performance: {speedup: 1.50, ref_ms: 0.042, kern_ms: 0.028}
      ncu_summary: {bottleneck: memory_bound, mem_throughput: 85.3%, compute: 12.8%, occupancy: 74.1%}
      iterations: 1
      convergence_reason: target_met_and_recent_gains_small
    phase_5:
      verdict: PASSED
      archived_to: ~/.lmcc/knowledge/triton/implementations/vector_add/
  failure_reason: null

Step 4: Supervisor Audit (Quality Gate)

You MUST independently verify the Orchestrator's claims by spot-checking workspace files AND running validation scripts. Do NOT blindly trust the report.

4.1 Static Code Audit

1. Read $WORKSPACE/kernel_function.py and verify:

   • The file exists and contains valid Python.
   • DSL Usage Check: The kernel MUST use backend DSL primitives:
     • For Triton: Must contain @triton.jit decorator AND tl.load, tl.store, tl.program_id calls
     • For TileLang: Must contain @tilelang.jit or equivalent DSL decorators
   • No Framework Computation: Quick scan for forbidden patterns:
     • NO torch.add, torch.sum, torch.matmul, a + b, a * b (tensor ops) in kernel core
     • NO paddle.add, paddle.sum, etc.
   • If backward was requested: backward kernel actually exists with DSL implementation (not a stub like return grad_output).
   • If $WORKSPACE/kernel_function_best.py exists, verify the final exported kernel_function.py matches it.

2. Read $WORKSPACE/reference_function.py and verify:

   • The original reference function is intact and matches what the user provided.
   • This file was NOT overwritten by Phase 3 (should still be the original).

3. Read $WORKSPACE/phase_4_summary.md and verify:

   • The file exists when Phase 4 passed.
   • It records the accepted best iteration, rejected directions, and convergence reason.
   • Its metrics are consistent with the Orchestrator report.

4. Read $WORKSPACE/iterations.tsv and verify:

   • The file exists when Phase 4 passed.
   • Schema check: First line must be the header iter\tspeedup\tcorrectness\tpurity\tfocus\tstatus\tdescription (7 tab-separated columns).
   • Row count: Number of data rows must match the iterations count in the Orchestrator report. This count includes all rows: accept, reject, and crash.
   • Consistency with phase_4_summary.md: The last accept row's speedup should match the reported final speedup (within 10% tolerance). Rejected directions listed in phase_4_summary.md should appear as reject or crash rows.
   • Progression sanity: Multiple accepted iterations with the same focus are allowed (e.g., progressively tuning memory_coalescing), but each accepted row should have a distinct description showing what changed.

5. Read $WORKSPACE/validate_performance.py and check for lazy implementation:

   • Must have proper warmup: At least 5-10 warmup iterations before timing
   • Must use CUDA events: torch.cuda.Event or paddle.device.cuda.Event for accurate GPU timing
   • Must run multiple iterations: At least 50-100 timed iterations
   • NO fake benchmarks: Reject if it just returns hardcoded values or skips actual timing

6. Read $WORKSPACE/validate_correctness.py and check:

   • Must import actual functions: From kernel_function.py and reference_function.py
   • Must run staged validation: smoke / shape sweep / numerical stability / determinism / edge cases
   • Must emit structured CORRECTNESS_VALIDATION output

4.2 Runtime Verification (CRITICAL)

Actually run the validation scripts to verify the Orchestrator's claims:

# Run correctness validation
cd $WORKSPACE
python validate_correctness.py
# Expected output: structured CORRECTNESS_VALIDATION block

# Run performance validation
python validate_performance.py
# Expected output: structured PERFORMANCE_VALIDATION block

Parse the output and verify:

• Correctness: CORRECTNESS_VALIDATION.verdict must be PASSED
• Correctness: stage-level results must be present and consistent with the Orchestrator report
• Performance: Speedup must match what Orchestrator reported (within 10% tolerance)
• No Python errors or exceptions during execution

4.3 Audit Failure Handling

If static audit fails (missing DSL, lazy validators, framework ops in kernel):

• Record specific issues found
• Re-launch Orchestrator with RETRY_CONTEXT containing:

  AUDIT_FAILURE:
    issue: <e.g., "kernel_function.py uses torch.sum instead of tl.sum">
    file: <path>
    line: <if known>
    fix_required: <specific instruction>
  

If runtime verification fails (validation scripts error or produce wrong results):

• Record the actual error output
• Re-launch Orchestrator with RETRY_CONTEXT containing:

  RUNTIME_FAILURE:
    script: <validate_correctness.py or validate_performance.py>
    error: <actual error message or unexpected output>
    expected: <what should have happened>
  

If the Orchestrator reported FAILED:

• Read the failure_reason.
• Decide whether to retry (re-launch Orchestrator with additional guidance) or report failure to user.

Max Orchestrator retries: 2.

Retry Strategy: Always Fresh Start, Never Resume

When re-launching the Orchestrator (for any reason), always start a new Task invocation instead of using resume. Rationale:

• Workspace files are the persistent state. The Orchestrator detects existing artifacts in $WORKSPACE/ and skips already-completed phases (see "Incremental Execution" in orchestrator.md). No need to carry prior context.
• A failed Orchestrator may have a polluted mental model. If it reported SUCCESS but the audit found issues, its context contains the wrong conclusion. A fresh agent reading the workspace + audit feedback is more reliable.
• Context budget is preserved. A resumed Orchestrator carries all the accumulated sub-agent dispatches and reports from the prior run, leaving less room for new work.

Re-launch prompt template (for retries):

Read $SKILL_ROOT/agents/orchestrator.md for your complete instructions.

SKILL_ROOT: <same as before>
WORKSPACE: <same workspace path>
FRAMEWORK / BACKEND / DEVICE / WITH_BACKWARD: <same as before>
REFERENCE_CODE_PATH: $WORKSPACE/reference_function.py
USER_REQUEST: <original user request>

RETRY_CONTEXT:
  This is retry #N. The previous run produced the following issues:
  <Supervisor's audit findings OR previous failure_reason>

  Existing workspace artifacts that PASSED prior validation:
  <list of files the Supervisor confirmed are good, e.g., sample_inputs.py>

  Focus on fixing: <specific phase/step that needs rework>

Execute the workflow. Skip phases whose artifacts already exist and passed audit.
Return an ORCHESTRATOR_REPORT when complete.

Step 5: Present Results to User

Once the audit passes (both static and runtime verification):

5.1 Summary Report

Present a structured summary to the user:

## Kernel Optimization Complete ✓

### Performance Results
- **Speedup**: X.XXx faster than reference
- **Reference time**: X.XXX ms
- **Kernel time**: X.XXX ms
- **Correctness**: PASSED / FAILED with staged validation
- **Convergence**: <brief convergence reason>

### Profiling Summary (if available)
- **Bottleneck**: compute_bound | memory_bound | latency_bound | balanced
- **Memory throughput**: XX%
- **Compute throughput**: XX%
- **Occupancy**: XX%

### Generated Files
All artifacts saved to: `~/.lmcc/workspace/<task-id>/`

| File | Description |
|------|-------------|
| `kernel_function.py` | Optimized kernel implementation |
| `kernel_function_best.py` | Accepted best kernel implementation |
| `reference_function.py` | Original reference function |
| `sample_inputs.py` | Test input generator |
| `validate_correctness.py` | Correctness validation script |
| `validate_performance.py` | Performance benchmark script |
| `phase_4_summary.md` | Accepted best iteration, rejected directions, and convergence reason |
| `iterations.tsv` | Structured experiment log (one row per optimization iteration) |
| `results.md` | Detailed results summary |

5.2 Show Key Code

1. Show the kernel function code (read from $WORKSPACE/kernel_function.py):

   • Highlight the DSL kernel (the @triton.jit or @tilelang.jit decorated function)
   • Show the wrapper function

2. Show the sample_inputs code (read from $WORKSPACE/sample_inputs.py):

   • So user knows what inputs were used for testing

5.3 Usage Instructions

Provide copy-paste ready usage example:

# Usage example
from kernel_function import kernel_function
from sample_inputs import sample_inputs

# Generate test inputs
inputs = sample_inputs()

# Run optimized kernel
result = kernel_function(*inputs)

5.4 If Optimization Failed

If the workflow ultimately failed after all retries:

## Kernel Optimization Failed

### Failure Reason
<failure_reason from Orchestrator>

### Attempted Phases
- Phase 0 (Knowledge Retrieval): PASSED/SKIPPED
- Phase 1 (Analysis): PASSED
- Phase 2 (Test Generator): PASSED/FAILED
- Phase 3 (Backward): PASSED/SKIPPED/FAILED
- Phase 4 (Kernel Generation): FAILED
  - Issue: <specific issue>
  - Iterations attempted: N

### Partial Artifacts
The following files may still be useful:
- `$WORKSPACE/reference_function.py` - Original function
- `$WORKSPACE/sample_inputs.py` - Test inputs (if generated)

### Suggestions
<If possible, suggest what the user could do differently or manually>

---

Workspace Layout

~/.lmcc/workspace/<task-id>/
├── reference_function.py      # Supervisor writes (Step 1) — ORIGINAL, never overwritten
├── reference_backward.py      # Phase 3 Worker writes (backward-augmented version)
├── sample_inputs.py           # Phase 2 Worker writes
├── kernel_function.py         # Phase 4 Worker writes
├── kernel_function_best.py    # Phase 4 Worker writes (accepted best artifact)
├── validate_correctness.py    # Phase 4 Worker writes
├── validate_performance.py    # Phase 4 Worker writes
├── driver_for_ncu.py          # Phase 4 Worker writes
├── ncu_profile.ncu-rep        # Phase 4 Worker writes
├── phase_4_summary.md         # Phase 4 Worker writes (Accepted best iteration, rejected directions, and convergence reason)
├── iterations.tsv             # Phase 4 Worker writes (Structured experiment log per iteration)
└── results.md                 # Orchestrator writes (Phase 6)

Important: reference_function.py is the source of truth for the user's original code. reference_backward.py is the augmented version used when backward pass is requested.

---

Prompt Files Reference

File	Used By	Purpose
Agents (Prompts)
agents/orchestrator.md	Supervisor → Orchestrator	Full orchestration logic for Phase 0→6
agents/test_case_generator.md	Orchestrator → Phase 2 Worker	Test case sampler generation
agents/backward_generator.md	Orchestrator → Phase 3 Worker	Backward pass generation
agents/kernel_generator.md	Orchestrator → Phase 4 Worker	Kernel generation
Rules (Validation)
rules/framework_ops_allowlist.md	Purity Validator, Supervisor	Allowed/forbidden framework ops
rules/validator_sampler.md	Phase 2 Worker	Test sampler validation
rules/validator_correctness.md	Phase 4 Worker	Kernel correctness validation
rules/validator_purity.md	Phase 4 Worker	Framework op purity check
rules/validator_performance.md	Phase 4 Worker, Supervisor	Performance benchmark validation
rules/validator_ncu.md	Phase 4 Worker	NCU profiling analysis
Knowledge Management
knowledge_management/knowledge_retrieval.md	Orchestrator → Phase 0 Worker	Knowledge base search
knowledge_management/knowledge_archival.md	Orchestrator → Phase 5 Worker	Knowledge archival
Backend Guides
backend_specific/triton_guide.md	Phase 4 Worker	Triton optimization patterns
backend_specific/tilelang_guide.md	Phase 4 Worker	TileLang optimization patterns
Framework Guides
framework_specific/framework_pytorch.md	Phase Workers	PyTorch-specific patterns
framework_specific/framework_paddle.md	Phase Workers	PaddlePaddle-specific patterns

---

Example: Supervisor Trace (Element-wise Addition)

User says:

Optimize this for Triton: def reference_function(a, b): return a + b

Supervisor Step 1: Gather & Init

• Framework: pytorch, Backend: triton, Backward: false
• Workspace: ~/.lmcc/workspace/a1b2c3d4-...
• Write reference to workspace.

Supervisor Step 2: Launch Orchestrator

• Task(general-purpose) with orchestrator.md + params.

Supervisor Step 3: Read Report

ORCHESTRATOR_REPORT:
  status: SUCCESS
  phase_results:
    phase_2: {verdict: PASSED}
    phase_4:
      verdict: PASSED
      best_kernel_file: ~/.lmcc/workspace/a1b2c3d4-.../kernel_function_best.py
      summary_file: ~/.lmcc/workspace/a1b2c3d4-.../phase_4_summary.md
      correctness: {verdict: PASSED, stages: {smoke: PASSED, shape_sweep: PASSED, numerical_stability: PASSED, determinism: PASSED, edge_cases: PASSED}}
      speedup: 1.50x
      ncu: memory_bound 85.3%
      convergence_reason: target_met_and_recent_gains_small

Supervisor Step 4: Audit

4.1 Static Audit:

• Read kernel_function.py:
  • ✓ Contains @triton.jit decorator
  • ✓ Uses tl.load, tl.store, tl.program_id
  • ✓ No torch.add or tensor arithmetic in kernel core
  • ✓ Matches kernel_function_best.py
• Read phase_4_summary.md:
  • ✓ Accepted best iteration and rejected directions are recorded
• Read validate_performance.py:
  • ✓ Has warmup loop (10 iterations)
  • ✓ Uses torch.cuda.Event for timing
  • ✓ Runs 100 timed iterations
• Read validate_correctness.py:
  • ✓ Emits structured CORRECTNESS_VALIDATION

4.2 Runtime Verification:

$ python validate_correctness.py
CORRECTNESS_VALIDATION:
  verdict: PASSED
  stages:
    smoke: PASSED
    shape_sweep: PASSED
    numerical_stability: PASSED
    determinism: PASSED
    edge_cases: PASSED

$ python validate_performance.py
PERFORMANCE_VALIDATION:
  verdict: PASSED
  reference_ms: 0.0420
  kernel_ms: 0.0280
  speedup: 1.50
  speedup_target: 1.20
  delta_vs_target: 0.30
  next_action: accept

• ✓ Correctness matches report (structured verdict + stage results)
• ✓ Speedup matches report (1.50x)

Supervisor Step 5: Present to User

## Kernel Optimization Complete ✓

### Performance Results
- **Speedup**: 1.50x faster than reference
- **Reference time**: 0.042 ms
- **Kernel time**: 0.028 ms
- **Correctness**: PASSED with staged validation

### Profiling Summary
- **Bottleneck**: memory_bound
- **Memory throughput**: 85.3%
- Near roofline for element-wise ops.

### Generated Files
All artifacts saved to: `~/.lmcc/workspace/a1b2c3d4-.../`