managing-database-sharding

name	managing-database-sharding
description	Process use when you need to work with database sharding. This skill provides horizontal sharding strategies with comprehensive guidance and automation. Trigger with phrases like "implement sharding", "shard database", or "distribute data".
allowed-tools	Read, Write, Edit, Grep, Glob, Bash(psql:*), Bash(mysql:*), Bash(mongosh:*)
version	1.27.0
author	Jeremy Longshore <[e-mail]>
license	MIT
tags	["database","database-sharding"]
compatibility	Designed for Claude Code

Database Sharding Manager

Overview

Implement and manage horizontal database sharding strategies across PostgreSQL, MySQL, and MongoDB. This skill covers shard key selection, data distribution analysis, cross-shard query routing, and rebalancing operations for databases that have outgrown single-node capacity.

Prerequisites

• Database admin credentials with CREATE DATABASE, CREATE TABLE, and replication permissions
• psql, mysql, or mongosh CLI tools installed and configured
• Network connectivity between all shard nodes
• Current table sizes and growth rate data (query pg_total_relation_size or information_schema.TABLES)
• Application query patterns documented or access to slow query logs
• Enough disk and memory on target shard nodes to handle redistributed data

Instructions

1. Analyze the current database size and identify tables exceeding single-node capacity thresholds (typically >500GB or >1B rows). Run SELECT pg_size_pretty(pg_total_relation_size('table_name')) for PostgreSQL or SELECT data_length + index_length FROM information_schema.TABLES for MySQL.

2. Evaluate candidate shard keys by examining query WHERE clauses, JOIN patterns, and data distribution. A good shard key has high cardinality, even distribution, and appears in most queries. Run SELECT shard_key_column, COUNT(*) FROM table GROUP BY shard_key_column ORDER BY COUNT(*) DESC LIMIT 20 to check distribution.

3. Choose a sharding strategy based on workload patterns:

   • Hash-based: Even distribution, best for key-value lookups. Use hash(shard_key) % num_shards.
   • Range-based: Good for time-series or sequential data. Partition by date ranges or ID ranges.
   • Directory-based: Maximum flexibility with a lookup table mapping keys to shards.
   • Geographic: Route by region for data residency or latency requirements.

4. Design the shard topology by determining the number of shards, replication factor, and placement. For PostgreSQL, use Citus extension or manual foreign data wrappers. For MySQL, configure vitess or ProxySQL routing. For MongoDB, enable sharding on the cluster with sh.enableSharding() and sh.shardCollection().

5. Create the shard schema on all target nodes, ensuring identical table definitions, indexes, and constraints across every shard. Generate DDL scripts and verify with checksums.