golang-performance

# Go Performance Optimization ## Overview This skill provides comprehensive guidance for profiling, benchmarking, and optimizing Go applications. Use this skill when working on performance-critical code, investigating bottlenecks, or optimizing production systems. **When to Use This Skill**: - Profiling application performance - Benchmarking code changes - Investigating memory leaks or high allocations - Optimizing hot paths - Tuning garbage collection - Reducing latency in production **Core Tools**: - `pprof` - CPU, memory, and goroutine profiling - `go test -bench` - Benchmarking framework - `go build -gcflags` - Escape analysis - `GOGC` and `GOMEMLIMIT` - GC tuning --- ## 1. Profiling with pprof ### 1.1 CPU Profiling **Enable CPU Profiling in Code**: ```go import ( "os" "runtime/pprof" ) func main() { f, err := os.Create("cpu.prof") if err != nil { log.Fatal("could not create CPU profile: ", err) } defer f.Close() if err := pprof.StartCPUProfile(f); err != nil { log.Fatal("could not start CPU profile: ", err) } defer pprof.StopCPUProfile() // Your application code here runApplication() } ``` **CLI Profiling**: ```bash # Profile a test go test -cpuprofile=cpu.prof -bench=. # Profile a binary go test -c ./myapp.test -test.cpuprofile=cpu.prof -test.bench=. ``` **Analysis Commands**: ```bash # Interactive web UI (recommended) go tool pprof -http=:8080 cpu.prof # Text output - top functions by CPU time go tool pprof -top cpu.prof # Top 20 with cumulative time go tool pprof -top -cum cpu.prof | head -20 # Call graph visualization go tool pprof -svg cpu.prof > cpu.svg # Focus on specific function go tool pprof -focus=processData cpu.prof # Exclude standard library go tool pprof -ignore=runtime cpu.prof ``` **Interpreting CPU Profiles**: - **flat**: Time spent in function itself (excludes callees) - **flat%**: Percentage of total runtime - **sum%**: Cumulative percentage - **cum**: Time spent in function and callees - **cum%**: Cumulative time percentage **Example Output**: ``` Showing nodes accounting for 2.50s, 83.33% of 3.00s total flat flat% sum% cum cum% 0.80s 26.67% 26.67% 1.20s 40.00% processData 0.60s 20.00% 46.67% 0.90s 30.00% parseJSON 0.50s 16.67% 63.34% 0.50s 16.67% validateInput ``` Focus optimization on functions with high `flat` (own time) or `cum` (total time). --- ### 1.2 Memory Profiling **Heap Profiling**: ```go import ( "os" "runtime/pprof" ) func captureHeapProfile() { f, err := os.Create("mem.prof") if err != nil { log.Fatal("could not create memory profile: ", err) } defer f.Close() // Force GC before capturing heap runtime.GC() if err := pprof.WriteHeapProfile(f); err != nil { log.Fatal("could not write memory profile: ", err) } } ``` **Memory Profiling via CLI**: ```bash # Profile memory allocations during test go test -memprofile=mem.prof -bench=. # Run benchmark multiple times for stable results go test -memprofile=mem.prof -bench=. -benchtime=10s ``` **Analysis Commands**: ```bash # Web UI showing allocation sites go tool pprof -http=:8080 mem.prof # Top allocators go tool pprof -top mem.prof # Focus on allocations (inuse_space) go tool pprof -sample_index=inuse_space -top mem.prof # Focus on allocation counts (inuse_objects) go tool pprof -sample_index=inuse_objects -top mem.prof # Show cumulative allocations (alloc_space) go tool pprof -sample_index=alloc_space -top mem.prof # Compare two profiles (before/after) go tool pprof -base=before.prof after.prof ``` **Memory Profile Types**: - `inuse_space`: Memory currently in use (default) - `inuse_objects`: Objects currently in use - `alloc_space`: Total allocations since start - `alloc_objects`: Total object allocations --- ### 1.3 Goroutine Profiling **Detect Goroutine Leaks**: ```go import ( "os" "runtime/pprof" ) func captureGoroutineProfile() { f, err := os.Create("goroutine.prof") if err != nil { log.Fatal("could not create goroutine profile: ", err) } defer f.Close() if err := pprof.Lookup("goroutine").WriteTo(f, 0); err != nil { log.Fatal("could not write goroutine profile: ", err) } } ``` **Analysis**: ```bash go tool pprof -http=:8080 goroutine.prof go tool pprof -top goroutine.prof ``` **Goroutine Leak Indicators**: - Steadily increasing goroutine count - Many goroutines blocked on channel recv/send - Goroutines without termination mechanism --- ### 1.4 HTTP Profiling Endpoint (Production-Safe) **Enable pprof HTTP Server**: ```go import ( _ "net/http/pprof" "net/http" ) func main() { // Start pprof server on separate port (localhost only) go func() { log.Println("pprof server listening on localhost:6060") log.Println(http.ListenAndServe("localhost:6060", nil)) }() // Your application here runServer() } ``` **Access Profiles via HTTP**: ```bash # CPU profile (30 seconds) curl http://localhost:6060/debug/pprof/profile?seconds=30 > cpu.prof # Heap profile curl http://localhost:6060/debug/pprof/heap > heap.prof # Goroutine profile curl http://localhost:6060/debug/pprof/goroutine > goroutine.prof # Analyze immediately go tool pprof http://localhost:6060/debug/pprof/profile # Web UI go tool pprof -http=:8080 http://localhost:6060/debug/pprof/profile ``` **Available Endpoints**: - `/debug/pprof/` - Index of all profiles - `/debug/pprof/profile` - CPU profile - `/debug/pprof/heap` - Heap profile - `/debug/pprof/goroutine` - Goroutine stack traces - `/debug/pprof/threadcreate` - Thread creation profile - `/debug/pprof/block` - Blocking profile - `/debug/pprof/mutex` - Mutex contention profile **Production Security**: ```go // Only expose on localhost http.ListenAndServe("localhost:6060", nil) // Or use SSH port forwarding // ssh -L 6060:localhost:6060 user@production-host // Then access http://localhost:6060/debug/pprof/ ``` --- ## 2. Benchmarking ### 2.1 Basic Benchmarks **Simple Benchmark**: ```go func BenchmarkStringConcat(b *testing.B) { for i := 0; i < b.N; i++ { result := "hello" + " " + "world" _ = result // Prevent compiler optimization } } ``` **Benchmark with Setup**: ```go func BenchmarkProcessData(b *testing.B) { data := generateTestData(1000) b.ResetTimer() // Exclude setup time for i := 0; i < b.N; i++ { processData(data) } } ``` **Running Benchmarks**: ```bash # Run all benchmarks go test -bench=. # Run specific benchmark go test -bench=BenchmarkStringConcat # Benchmark with memory statistics go test -bench=. -benchmem # Run multiple iterations for stability go test -bench=. -count=5 # Longer benchmark time for accurate results go test -bench=. -benchtime=10s # CPU profile during benchmark go test -bench=. -cpuprofile=cpu.prof ``` --- ### 2.2 Sub-Benchmarks **Compare Multiple Implementations**: ```go func BenchmarkStringBuilding(b *testing.B) { items := []string{"hello", "world", "foo", "bar"} b.Run("Concat", func(b *testing.B) { for i := 0; i < b.N; i++ { result := "" for _, item := range items { result += item } _ = result } }) b.Run("StringBuilder", func(b *testing.B) { for i := 0; i < b.N; i++ { var sb strings.Builder for _, item := range items { sb.WriteString(item) } _ = sb.String() } }) b.Run("Join", func(b *testing.B) { for i := 0; i < b.N; i++ { result := strings.Join(items, "") _ = result } }) } ``` **Output**: ``` BenchmarkStringBuilding/Concat-8 500000 3245 ns/op 96 B/op 5 allocs/op BenchmarkStringBuilding/StringBuilder-8 2000000 825 ns/op 64 B/op 1 allocs/op BenchmarkStringBuilding/Join-8 2000000 780 ns/op 48 B/op 1 allocs/op ``` --- ### 2.3 Memory Reporting **Track Allocations**: ```go func BenchmarkWithAllocs(b *testing.B) { b.ReportAllocs() for i := 0; i < b.N; i++ { data := make([]int, 1000) _ = data } } ``` **Output Interpretation**: ``` BenchmarkWithAllocs-8 200000 8234 ns/op 8192 B/op 1 allocs/op ------ ---- ---- ---- iters ns/op bytes/op allocs/op ``` - **ns/op**: Nanoseconds per operation - **B/op**: Bytes allocated per operation - **allocs/op**: Number of allocations per operation **Zero Allocation Goal**: ```go // Bad: 2 allocations func process(data string) string { upper := strings.ToUpper(data) // 1 alloc return strings.TrimSpace(upper) // 1 alloc } // Better: 1 allocation (reuse buffer) func process(data string) string { var sb strings.Builder sb.Grow(len(data)) for _, r := range data { if !unicode.IsSpace(r) { sb.WriteRune(unicode.ToUpper(r)) } } return sb.String() } ``` --- ### 2.4 Benchmark Analysis with benchstat **Compare Before/After**: ```bash # Baseline go test -bench=. -count=10 > old.txt # After optimization go test -bench=. -count=10 > new.txt # Statistical comparison go install golang.org/x/perf/cmd/benchstat@latest benchstat old.txt new.txt ``` **Example Output**: ``` name old time/op new time/op delta StringConcat-8 3.24µs ± 2% 0.82µs ± 1% -74.69% (p=0.000 n=10+10) name old alloc/op new alloc/op delta StringConcat-8 96.0B ± 0% 64.0B ± 0% -33.33% (p=0.000 n=10+10) name old allocs/op new allocs/op delta StringConcat-8 5.00 ± 0% 1.00 ± 0% -80.00% (p=0.000 n=10+10) ``` **Interpretation**: - `±2%` - Variance across runs - `(p=0.000)` - Statistical significance (p < 0.05 = significant) - `n=10+10` - Number of samples used --- ## 3. Memory Optimization ### 3.1 Pre-allocate Slices **Problem: Repeated Reallocation**: ```go // Bad: 14 reallocations for 10,000 items func inefficient() []int { var data []int for i := 0; i < 10000; i++ { data = append(data, i) } return data } ``` **Solution: Pre-allocate Capacity**: ```go // Good: 1 allocation func efficient() []int { data := make([]int, 0, 10000) for i := 0; i < 10000; i++ { data = append(data, i) } return data } ``` **Transformation Pattern**: ```go func transformItems(input []string) []Result { output := make([]Result, 0, len(input)) for _, item := range input { output = append(output, transform(item))