k-search-kernel-generation-world-models

name	k-search-kernel-generation-world-models
title	K-Search: LLM Kernel Generation via Co-Evolving Intrinsic World Model
version	0.0.2
engine	skillxiv-v0.0.2-claude-opus-4.6
license	MIT
url	https://arxiv.org/abs/2602.19128
keywords	["program synthesis","GPU optimization","search planning","world models","LLM agents"]
description	Generate optimized GPU kernels by treating LLMs as planning engines that co-evolve with a world model. Decouples high-level algorithmic planning from low-level implementation, enabling structured search through optimization strategies. LLM world model estimates priority scores for pending optimizations while iteratively updating understanding based on execution results. Achieves 2.10× improvement over evolutionary baselines with 14.3× gains on complex MoE kernels.

K-Search: Guided Kernel Optimization via LLM World Modeling

GPU kernel optimization is a high-dimensional search problem combining algorithm selection, memory layout, and platform-specific tuning. Existing approaches treat it as direct code generation—search directly in program space—but this struggles with non-monotonic improvements where some optimizations create temporary performance regressions that unlock larger gains downstream.

The challenge is navigating a complex optimization landscape where local greedy search gets trapped in local optima. Traditional evolutionary approaches apply random mutations uniformly, missing the structure of the optimization space.

Core Concept

K-Search separates high-level algorithmic planning from low-level implementation details. An LLM world model maintains beliefs about which optimization strategies are most promising, estimates their priority scores, and iteratively updates these beliefs based on execution feedback. Rather than random mutation, the system proposes optimizations according to priority estimates, enabling structured exploration of the optimization space.

The approach operates through three iterative phases:

1. Action Selection: World model scores pending optimization strategies and selects most promising
2. Program Instantiation: Generate concrete implementations through repeated sampling
3. World Model Co-Evolution: Analyze results and update world model (insert new hypotheses, update priorities, prune unpromising branches)

Architecture Overview

• Strategy Repository: Maintains list of optimization strategies (loop unrolling, vectorization, shared memory optimization, etc.) with priority scores
• LLM World Model: Estimates which strategies will yield improvement for current kernel; can reason about interactions between strategies
• Implementation Generator: Instantiate concrete kernel code from selected strategy using sampling
• Executor & Profiler: Run kernel and measure actual improvement
• Update Logic: Revise world model beliefs (priorities, confidence) based on real vs. predicted performance

Implementation

Represent optimization strategies and maintain priorities:

class StrategyRepository:
    def __init__(self):
        self.strategies = [
            {: , : , : },
            {: , : , : },
            {: , : , : },
            {: , : , : },
            {: , : , : }
        ]

     ():
        
        available = [s  s  .strategies   s[]]
          available:
             

        sorted_strategies = (available, key= x: x[], reverse=)
         sorted_strategies[]

     ():
        
         s  .strategies:
             s[] == strategy_name:
                
                 improvement_factor > :
                    s[] = (, s[] +  * improvement_factor)
                :
                    s[] = (, s[] - )