ALWAYS use when creating Metal pipeline state objects with Metal 4 descriptors (MTL4RenderPipelineDescriptor, MTL4ComputePipelineDescriptor, MTL4MeshRenderPipelineDescriptor) via MTL4Compiler. Covers function descriptors, function constant specialization, flexible/unspecialized PSOs, color-attachment mapping, async compilation (MTL4CompilerTask), pipeline caching (MTL4Archive, MTL4PipelineDataSetSerializer), static/binary shader linking, and pipeline reflection. Do NOT trigger for HLSL/DXIL → metallib — use compiling-with-metal-shaderconverter.
ALWAYS use when creating Metal pipeline state objects with Metal 4 descriptors (MTL4RenderPipelineDescriptor, MTL4ComputePipelineDescriptor, MTL4MeshRenderPipelineDescriptor) via MTL4Compiler. Covers function descriptors, function constant specialization, flexible/unspecialized PSOs, color-attachment mapping, async compilation (MTL4CompilerTask), pipeline caching (MTL4Archive, MTL4PipelineDataSetSerializer), static/binary shader linking, and pipeline reflection. Do NOT trigger for HLSL/DXIL → metallib — use compiling-with-metal-shaderconverter.
Pipeline Creation
Overview
Covers Metal 4 pipeline state creation when porting games: loading pre-compiled metallibs, building render/compute/mesh PSOs via MTL4Compiler, function constants, flexible PSOs, color-attachment mapping, async compilation, and pipeline caching.
For compiling HLSL/DXIL shaders to metallibs, see the compiling-with-metal-shaderconverter skill.
For compiling Metal shader language (MSL) source, see MSL Source Compilation below.
References
Read the relevant Metal 4 SDK header before writing pipeline code — the headers are the source of truth for property names, types, and method signatures.
Metal 4 SDK headers - $(xcrun --show-sdk-path)/System/Library/Frameworks/Metal.framework/Headers/ — focus on MTL4Compiler.h, MTL4RenderPipeline.h, MTL4ComputePipeline.h, MTL4MeshRenderPipeline.h, MTL4TileRenderPipeline.h, MTL4LinkingDescriptor.h
Metal 3 PSOs work on Metal 4 encoders (and vice versa). This enables incremental porting — pipeline creation can be migrated to independently of encoder migration.
MTL4SpecializedFunctionDescriptor (function constants); flexible PSOs for format/blend
Pipeline cache (disk)
ID3D12PipelineLibrary
VkPipelineCache
MTL4Archive + MTL4PipelineDataSetSerializer
Notes:
-[MTL4Compiler newRenderPipelineStateWithDescriptor:] is polymorphic: render, mesh, and tile pipeline descriptors all derive from MTL4PipelineDescriptor and use the same selector.
Each PSO-creation selector has an async variant taking completionHandler: and returning MTL4CompilerTask.
Producing a metallib from HLSL/DXIL is covered by the compiling-with-metal-shaderconverter skill.
For in-process MSL source compilation, see MSL Source Compilation below.
MTL4Compiler Workflow
Metal 4 uses MTL4Compiler for explicit pipeline compilation, replacing Metal 3's [device newRenderPipelineStateWithDescriptor:error:]. Pipelines use FunctionDescriptor (not MTLFunction), support flexible (unspecialized) states for reduced compilation time, and color-attachment mapping for PSO reuse across render pass configurations.
When debugging binding mismatches, use reflection to inspect pipeline bindings at runtime.
Mesh Shader Pipeline
See MTL4MeshRenderPipeline.h. MTL4MeshRenderPipelineDescriptor adds object/mesh function descriptors and per-threadgroup limits on top of the render pipeline base.
Compute Pipeline
See MTL4ComputePipeline.h. Set threadGroupSizeIsMultipleOfThreadExecutionWidth = YES only when the threadgroup size is guaranteed to be a multiple of execution width — this enables SIMD optimizations.
Specialization
Metal 4 offers three ways to make one PSO serve many cases: function constants (compile-time specialization on values), flexible pipeline states (deferred format/blend selection), and color-attachment mapping (output index remapping).
Not all shaders have function constants — check shader reflection or MSL source for [[function_constant(N)]] declarations
Specialization triggers recompilation — cache the resulting pipelines
In MSL: constant bool &flag [[function_constant(0)]] — branches on this are eliminated at pipeline creation time
Flexible (Unspecialized) Pipeline States
Compile once at launch without committing to pixel format or blend state, then specialize at runtime without recompiling shader code. Use this when the same shader must work with many pixel format or blend state combinations — common in engines with configurable render targets, multiple output formats, or runtime-variable blend modes. If the pipeline configuration is known and fixed, full compilation is simpler and relatively as fast at draw time.
Shared shader code across specializations (saves memory)
Faster runtime specialization than full recompilation
Reduces total pipeline count
Color-Attachment Mapping
Allows remapping a shader's logical [[color(N)]] outputs to different physical render pass attachment indices at draw time. A single PSO works across different render pass output configurations without recompilation:
Compile one pipeline state instead of many for different render pass configurations
Consolidate render commands into fewer passes
Reduce CPU overhead from pipeline state management
Shader Linking
Metal 4 provides two mechanisms for linking pre-compiled shader functions into a pipeline. These are advanced features — most ports don't need them initially, but engines with modular shader architectures (material systems, effect graphs) may benefit.
Static linking links additional shader functions at Metal IR level during PSO creation. Because linking occurs at compile time, the compiler can inline and optimize across function boundaries. Configured per-stage on the pipeline descriptor via vertexStaticLinkingDescriptor / fragmentStaticLinkingDescriptor (render) or staticLinkingDescriptor (compute/tile), using MTL4StaticLinkingDescriptor.
Binary linking links pre-compiled binary functions (MTL4BinaryFunction) to a pipeline. Since the functions are already compiled to machine code, no cross-function inlining or optimization is possible — but compilation is faster because the linked functions don't need recompilation. Configured via MTL4PipelineStageDynamicLinkingDescriptor passed to newRenderPipelineState:dynamicLinkingDescriptor: (or the compute equivalent). To later add binary functions to an existing pipeline, set supportVertexBinaryLinking / supportFragmentBinaryLinking on the pipeline descriptor at creation time.
When to use binary linking: Binary functions save compilation time when the same function is reused across many PSOs — the function is compiled to machine code once and linked without recompilation. However, because the function call cannot be inlined, there is a runtime cost: call frame maintenance and potential stack spilling. Profile to ensure the compilation time savings justify the runtime overhead for your workload. Static linking is preferred when runtime performance matters more than compilation time.
Pipeline Caching
Default cache
Metal maintains a system-wide shader cache - details of which are not specified - that reuses compiled pipelines across runs of the same app. When creating pipeline descriptors, populate input arrays in the same order across runs (for example, the MTL4FunctionDescriptor arrays in MTL4StaticLinkingDescriptor). Reordering changes the cache key which results in a cache miss, forcing recompilation.
Explicit cache
For explicit cache control: on the app's first launch, attach an MTL4PipelineDataSetSerializer to the compiler so it captures pipeline data as PSOs are built, then flush the serializer to an MTL4Archive file. On subsequent app launches, pass the archive via MTL4CompilerTaskOptions.lookupArchives; PSOs whose descriptors match load from disk instead of recompiling.
// First launch — attach serializer so the compiler captures pipeline data as PSOs are built// serializer comes from [device newPipelineDataSetSerializerWithDescriptor:serializerDescriptor error:&error]
compilerDescriptor.pipelineDataSetSerializer = serializer;
id<MTL4Compiler> compiler = [device newCompilerWithDescriptor:compilerDescriptor error:&error];
// ... create PSOs ...
[serializer serializeAsArchiveAndFlushToURL:archiveURL error:&error];
// Subsequent launches — load the archive and let the compiler look up cached PSOsid<MTL4Archive> archive = [device newArchiveWithURL:archiveURL error:&error];
MTL4CompilerTaskOptions* taskOptions = [[MTL4CompilerTaskOptions alloc] init];
taskOptions.lookupArchives = @[archive];
Binary archives are compatible across Metal 3 and Metal 4.
Recommended Practices
Enable shader validation during development — see man MetalValidation for environment variables. Use MTL4PipelineOptions.shaderValidation for fine-grained per-pipeline control.
Compile pipelines async or pre-launch. Synchronous runtime compilation hitches the render thread. MTL4CompilerTask parallelizes compilation off the render thread.
Use Metal 3 pipeline descriptors for incremental porting. Metal 3 PSOs work on Metal 4 encoders — don't block other porting work on pipeline migration. Move to MTL4Compiler when you need its features (flexible PSOs, caching, binary linking).
Use binary archives (MTL4Archive) to cache compiled pipelines across app launches. First-launch compilation can be slow; harvest pipeline states, serialize, and load on subsequent runs.
Use color-attachment mapping when the same shader outputs to different render-pass configurations — one PSO instead of many.
Use flexible PSOs for format/blend variance. Compile once unspecialized, specialize at runtime. Avoids the per-variant PSO explosion; faster than full recompilation and uses less memory.
Pipeline reflection is a debugging aid. Binding slot assignments should be known ahead of time from the shader. Reach for reflection only to resolve binding mismatches.
