| name | go-perf-contention-and-sharding |
| description | Reduces Go lock contention by sharding state into N lock-striped shards keyed by hash; shard-count and padding guidance; sync.Map's two optimized use cases vs a sharded map[K]V+Mutex for the general case; per-P counters summed on read; batching to amortize a lock; singleflight to collapse duplicate work. Fires on "reduce lock contention", "shard this map", "sync.Map vs mutex map", "hot mutex", "lock striping". Routes sync.Map API to go-sync-primitives, padding to go-perf-false-sharing, measuring to go-perf-block-mutex-profiles. |
| allowed-tools | Read, Glob, Grep |
| compatibility | Claude Code, Codex CLI, Gemini CLI |
Go Contention & Sharding
"The Map type is optimized for two common use cases ... In these two cases, use of a Map may significantly reduce lock contention compared to a Go map paired with a separate Mutex or RWMutex."
— pkg.go.dev/sync
A single mutex guarding shared state is a serialization point: while one goroutine holds it, every other core that wants that state stalls, so throughput stops scaling and can fall as cores are added. This skill owns the performance fix — splitting one lock into many so independent operations rarely collide. The correctness rules of the primitives — sync.Map's two cases, the no-copy Mutex rule, typed atomics — belong to go-sync-primitives; read it first. Always prove the contention exists before sharding: a block/mutex profile (go-perf-block-mutex-profiles) tells you which lock is hot, and benchstat (go-perf-benchmarking-statistics) tells you the shard actually helped.
All Go below builds clean under gofmt, go vet, and go test on Go 1.26.
1. The Core Idea — One Hot Lock Serializes Every Core
When a profile shows goroutines spending wall-clock time waiting on one Mutex, the lock — not the work it guards — is the bottleneck. The CPU profile is blind to this (it only sees on-CPU time); the mutex profile records it, but is off by default and records nothing until you call runtime.SetMutexProfileFraction — "controls the fraction of mutex contention events that are reported in the mutex profile ... To turn off profiling entirely, pass rate 0" (runtime) — and runtime.SetBlockProfileRate for the block profile (runtime). Enablement and reading these are owned by go-perf-block-mutex-profiles.
The fix is lock striping: replace the one lock + one map with N shards, each its own lock guarding its own slice of the keyspace. Hash the key to a shard; two goroutines touching keys in different shards never contend. Contention drops roughly N× when keys spread evenly, because at most ~1/N of the traffic lands on any one lock.
2. A Sharded Map — Lock Striping by Key Hash
The pattern: a fixed array of shards, each a Mutex (or RWMutex when reads dominate — see go-perf-atomics-vs-locks) over its own map, selected by hashing the key. This is exactly how orcaman/concurrent-map works — it shards "the map with minimal time spent waiting for locks," and is pitched for "something more like in-memory db" with frequent writes, unlike sync.Map which it notes "is designed for append-only scenarios" (orcaman/concurrent-map).
const shardCount = 32
type shard struct {
mu sync.RWMutex
m map[string]int
_ [64 - (unsafe.Sizeof(sync.RWMutex{})+unsafe.Sizeof(map[string]int(nil)))%64]byte
}
type ShardedMap struct{ shards [shardCount]shard }
func NewShardedMap() *ShardedMap {
s := &ShardedMap{}
for i := range s.shards {
s.shards[i].m = make(map[string]int)
}
return s
}
func (s *ShardedMap) shard(key string) *shard { return &s.shards[fnv32(key)%shardCount] }
func (s *ShardedMap) Load(key string) (int, bool) {
sh := s.shard(key)
sh.mu.RLock()
defer sh.mu.RUnlock()
v, ok := sh.m[key]
return v, ok
}
func (s *ShardedMap) Store(key string, v int) {
sh := s.shard(key)
sh.mu.Lock()
sh.m[key] = v
sh.mu.Unlock()
}
Two failure modes: a bad hash that clusters keys recreates a single hot shard (use a real hash, not len(key) or a sequential id's low bits); and unpadded shards sharing a cache line, so independent locks still bounce the line between cores — false sharing, owned by go-perf-false-sharing.
3. Shard Count — ≈ GOMAXPROCS or a Power of Two
The shard count sets the contention ceiling. Guidance:
- Floor it at
GOMAXPROCS. With fewer shards than cores, cores still queue on shared locks. A small multiple of GOMAXPROCS (or a fixed power of two comfortably above it, like orcaman's 32) leaves headroom as the schedulable parallelism grows. Note Go 1.25 makes GOMAXPROCS container-aware (it reads the cgroup CPU limit), so the effective core count under a CPU quota may be lower than the host's (Go 1.25).
- Prefer a power of two, so
hash % shardCount lowers to a bitmask instead of a division on the hot path.
- Don't over-shard. Each shard is a live map with its own bucket overhead; thousands of shards waste memory and cache for no contention win past the core count. And pad each shard to a cache line, or adjacent shard locks false-share — route to
go-perf-false-sharing.
4. sync.Map vs a Sharded Map — Match the Access Pattern
sync.Map is not a general "concurrent map" — its own doc is explicit: "The Map type is specialized. Most code should use a plain Go map instead, with separate locking or coordination, for better type safety and to make it easier to maintain other invariants along with the map content" (sync). It is "optimized for two common use cases: (1) when the entry for a given key is only ever written once but read many times, as in caches that only grow, or (2) when multiple goroutines read, write, and overwrite entries for disjoint sets of keys. In these two cases, use of a Map may significantly reduce lock contention" (sync). The decision (its API/correctness depth lives in go-sync-primitives):
| Access pattern | Reach for |
|---|
| Write-once, read-many; cache that only grows (case 1) | sync.Map |
| Goroutines own disjoint key sets (case 2) | sync.Map |
| General read-write churn, overlapping keys, typed values, multi-field invariants | sharded map[K]V + Mutex (§2) |
For the general read-write case a sharded plain map is usually faster and type-safe (no any boxing, no per-read type assertion) — and it lets you hold an invariant across the map under one shard's lock, which sync.Map cannot. Reach for sync.Map only when the pattern is genuinely one of its two documented cases.
5. Counters — Stripe and Sum, Don't Hammer One Atomic
A single atomic.Int64 incremented by every core is itself a contention point: each Add must win exclusive ownership of that one cache line, so the line ping-pongs between cores under load. The fix mirrors map sharding — N per-core counter cells, each on its own cache line, summed on read:
type cell struct {
n atomic.Int64
_ [64 - 8]byte
}
type StripedCounter struct{ cells [shardCount]cell }
func (c *StripedCounter) Add(id uint64, delta int64) { c.cells[id%shardCount].n.Add(delta) }
func (c *StripedCounter) Sum() int64 {
var total int64
for i := range c.cells {
total += c.cells[i].n.Load()
}
return total
}
This trades an exact snapshot read (Sum is not instantaneously consistent) for write throughput that scales with cores — the right trade for high-frequency metrics. The atomic-vs-mutex cost model is go-perf-atomics-vs-locks; padding is go-perf-false-sharing.
6. Batch Under the Lock — Amortize the Acquisition
Lock acquisition has a fixed cost paid per acquisition. Acquiring once per item turns N items into N lock/unlock round-trips; acquiring once for the whole batch pays it once.
for _, item := range items {
mu.Lock()
queue = append(queue, item)
mu.Unlock()
}
mu.Lock()
queue = append(queue, items...)
mu.Unlock()
Hold the lock only for the shared mutation; do per-item computation before Lock so the critical section stays short (the small-critical-section rule is go-sync-primitives). Batching and lock striping compose: shard first, then batch within each shard's lock.
7. singleflight — Collapse Duplicate Concurrent Work
When many goroutines independently request the same expensive thing at once (a cache miss stampede, a thundering herd on one DB row), the contention is duplicated work, not a lock. golang.org/x/sync/singleflight provides "a duplicate function call suppression mechanism": Group.Do "executes and returns the results of the given function, making sure that only one execution is in-flight for a given key at a time. If a duplicate comes in, the duplicate caller waits for the original to complete and receives the same results" (singleflight).
var g singleflight.Group
func (c *Cache) Fetch(ctx context.Context, key string) (*Row, error) {
v, err, _ := g.Do(key, func() (any, error) {
return loadFromDB(ctx, key)
})
if err != nil {
return nil, err
}
return v.(*Row), nil
}
The third return, shared, "indicates whether v was given to multiple callers"; DoChan gives a select-able channel result, and Forget drops a key so the next call re-runs rather than reusing a stale in-flight result (singleflight). One caveat: Do couples callers — a slow or failed shared call delays everyone waiting on that key.
8. Don't
- Don't shard before a profile proves the lock is hot. Enable the mutex profile (
SetMutexProfileFraction, off by default (runtime)) and confirm the wait — depth in go-perf-block-mutex-profiles.
- Don't assume
sync.Map is faster. It is "specialized" and for the general read-write case a sharded map+Mutex usually wins (sync).
- Don't leave shards unpadded. Adjacent shard locks share a cache line and false-share; pad to a cache line (
go-perf-false-sharing).
- Don't shard on a weak hash.
len(key) or sequential-id low bits cluster keys into one hot shard; use a real hash like FNV-1a (orcaman/concurrent-map).
- Don't hammer one global atomic counter from every core, or take the lock per item — stripe the counter (§5) and batch under one acquisition (§6).
- Don't let a stampede duplicate work that
singleflight would collapse (singleflight).
- Don't reach for a channel-as-mutex to "fix" contention where sharding is the real lever — it imports a channel's full failure surface to do a lock's job (
go-sync-primitives).
9. Routing to Related Skills
This plugin (go-perf-* depth):
go-perf-block-mutex-profiles — measure the contention first: enabling and reading block/mutex/goroutine profiles.go-perf-atomics-vs-locks — the atomic-vs-Mutex / RWMutex cost model behind a shard's lock and a striped counter's cell.go-perf-false-sharing — padding shards and counter cells to a cache line (cpu.CacheLinePad); the non-scaling symptom.go-perf-maps — single-map performance (Swiss Tables, prealloc, key cost) that each shard inherits.go-perf-benchmarking-statistics — proving the sharded version is actually faster (n≥10, benchstat, b.RunParallel).
Parent / sibling marketplace (go-* correctness):
go-sync-primitives — sync.Map's two cases, the no-copy Mutex/atomic rules, small critical sections, sync.Pool — the API and correctness this skill's perf depth sits on.go-race-and-memory-model — why unsynchronized shard access is still a data race; happens-before.go-version-feature-map — version floors (container-aware GOMAXPROCS 1.25; typed atomics 1.19).
10. Reference Files
Contention/sharding anti-patterns in LLM-generated Go, each with wrong/right code and citations:
${CLAUDE_SKILL_DIR}/references/common-mistakes.md
Source provenance for every claim in this skill:
${CLAUDE_SKILL_DIR}/references/sources.yaml
