Apply systematic performance optimization techniques when writing or reviewing code. Use when optimizing hot paths, reducing latency, improving throughput, fixing performance regressions, or when the user mentions performance, optimization, speed, latency, throughput, profiling, or benchmarking.
Apply systematic performance optimization techniques when writing or reviewing code. Use when optimizing hot paths, reducing latency, improving throughput, fixing performance regressions, or when the user mentions performance, optimization, speed, latency, throughput, profiling, or benchmarking.
Performance Optimization Skill
Apply these principles when optimizing code for performance. Focus on the critical 3% where performance truly matters - a 12% improvement is never marginal in engineering.
Core Philosophy
Measure First: Never optimize without profiling data
Estimate Costs: Back-of-envelope calculations before implementation
Avoid Work: The fastest code is code that doesn't run
Reduce Allocations: Memory allocation is often the hidden bottleneck
Cache Locality: Memory access patterns dominate modern performance
Process in reverse post-order to eliminate per-element checks
Replace interval trees (O(log N)) with hash maps (O(1)) when ranges aren't needed
2. Reduce Memory Allocations
Allocation is expensive (~25-100ns + GC pressure)
# BAD: Allocates on every calldefprocess(items):
result = [] # New allocationfor item in items:
result.append(transform(item))
return result
# GOOD: Pre-allocate or reusedefprocess(items, out=None):
if out isNone:
out = [None] * len(items)
for i, item inenumerate(items):
out[i] = transform(item)
return out
Techniques:
Pre-size containers with reserve() or known capacity
Hoist temporary containers outside loops
Reuse buffers across iterations (clear instead of recreate)
Move instead of copy large structures
Use stack allocation for bounded-lifetime objects
3. Compact Data Structures
Minimize memory footprint and cache lines touched:
Optimize the common case without hurting rare cases:
# BAD: Always takes slow pathdefparse_varint(data):
return generic_varint_parser(data)
# GOOD: Fast path for common 1-byte casedefparse_varint(data):
if data[0] < 128: # Single byte - 90% of casesreturn data[0], 1return generic_varint_parser(data) # Rare multi-byte
Techniques:
Handle common dimensions inline (1-D, 2-D tensors)
Check for empty/trivial inputs early
Specialize for common sizes (small strings, few elements)
Process trailing elements separately to avoid slow generic code
5. Precompute Expensive Information
Trade memory for compute when beneficial:
# BAD: Recomputes on every accessdefis_vowel(char):
return char.lower() in'aeiou'# GOOD: Lookup table
VOWEL_TABLE = [c.lower() in'aeiou'for c in (chr(i) for i inrange(256))]
defis_vowel(char):
return VOWEL_TABLE[ord(char)]
Techniques:
Precompute flags/properties at construction time
Build lookup tables for character classification
Cache expensive computed properties
Validate at boundaries once, not repeatedly inside
6. Bulk/Batch APIs
Amortize fixed costs across multiple operations:
# BAD: N round tripsfor item in items:
result = db.lookup(item)
# GOOD: 1 round trip
results = db.lookup_many(items)
Design APIs that support:
Batch lookups instead of individual fetches
Vectorized operations over loops
Streaming interfaces for large datasets
7. Avoid Unnecessary Work
# BAD: Always computes expensive valuedefprocess(data, config):
expensive = compute_expensive(data) # Always runsif config.needs_expensive:
use(expensive)
# GOOD: Defer until neededdefprocess(data, config):
if config.needs_expensive:
expensive = compute_expensive(data) # Only when needed
use(expensive)
Techniques:
Lazy evaluation for expensive operations
Short-circuit evaluation in conditions
Move loop-invariant code outside loops
Specialize instead of using general-purpose libraries in hot paths
8. Help the Compiler/Runtime
Lower-level optimizations when profiling shows need:
// Avoid function call overhead in hot loops#[inline(always)]fnhot_function(x: i32) ->i32 { x * 2 }
// Copy to local variable for better alias analysisfnprocess(data: &mut [i32], factor: &i32) {
letf = *factor; // Compiler knows this won't changeforxin data {
*x *= f;
}
}
Techniques:
Use raw pointers/indices instead of iterators in critical loops
Hand-unroll very hot loops (4+ iterations)
Move slow-path code to separate non-inlined functions
Avoid abstractions that hide costs in hot paths
9. Reduce Synchronization
Minimize lock contention and atomic operations:
Default to thread-compatible (external sync) not thread-safe
Sample statistics (1-in-32) instead of tracking everything
Batch updates to shared state
Use thread-local storage for per-thread data
Profiling Workflow
Identify hotspots: Use profiler (pprof, perf, py-spy)
Measure baseline: Write benchmark before optimizing
Estimate improvement: Calculate expected gain
Implement change: Focus on one optimization at a time
Verify improvement: Run benchmark, confirm gain
Check for regressions: Ensure no other code paths slowed down
When Profile is Flat (No Clear Hotspots)
Pursue multiple small 1-2% improvements collectively
Look for restructuring opportunities higher in call stack
Optimizing without measurement - changes based on intuition
Micro-optimizing while ignoring algorithmic complexity
Copying when moving would suffice
Growing containers one element at a time (quadratic)
Allocating in loops when reuse is possible
String formatting in hot paths (use pre-built templates)
Regex when simple string matching suffices
Estimation Template
Before optimizing, estimate:
Operation cost: ___ ns/us/ms
Frequency: ___ times per second/request
Total time: cost × frequency = ___
Improvement target: ___% reduction
Expected new time: ___
Is this worth it? [ ] Yes [ ] No
