model-nosql-data

name	model-nosql-data
description	Models data for document, key-value, and wide-column stores access-pattern-first — enumerates queries, then picks partition/sort keys for even distribution, chooses embed-vs-reference per relationship, lays out single-table/aggregate items, denormalizes deliberately with a fan-out path, and avoids hot partitions.
when_to_use	When the datastore is non-relational (DynamoDB, MongoDB, Cassandra/ScyllaDB, Firestore, Bigtable) and you must shape items/documents/rows around queries — picking partition keys, embed vs reference, a single-table model, or a wide-column primary key — before writing the data layer. Distinct from design-relational-schema (normalized tables + joins) and optimize-sql-query (tunes an existing relational query); caching-strategy is a read cache in front of any store, not the store's own model.

When to Use

Reach for this skill when you must shape the store around its queries, before any table/collection exists:

• "Design the DynamoDB table(s) for this service"
• "Should this be embedded or a separate collection in MongoDB?"
• "Pick the partition key and clustering columns for this Cassandra table"
• "We're getting hot partitions / throttling on one key — fix the key design"
• "Model a many-to-many (users↔teams, products↔orders) in a store with no joins"
• "Firestore/Bigtable layout for a feed/timeline read"

NOT this skill:

• A normalized schema with joins in a relational DB (entities, 3NF, FK/CHECK) → design-relational-schema
• A slow query against an existing relational schema → optimize-sql-query
• A read cache (TTL, invalidation, stampede) in front of the store → caching-strategy
• Schema change safety / locks / rollback on a live table → db-migration-safety
• Append-only event streams + projections as the system of record → design-event-sourcing-cqrs
• Background work / queue semantics → message-queue-jobs

Steps

1. Enumerate every access pattern first — this drives 100% of the design. No keys until this table is full. One row per operation, reads and writes. A pattern you forget becomes a full scan in prod.

   Pattern	R/W	Args (known at call time)	Result shape	Freq	Latency target	Selectivity
   Get user by id	R	userId	1 item	very high	<10ms	1
   List orders for user, newest first	R	userId, limit	N items, sorted	high	<20ms	bounded ~100s
   Get order + its line items	R	orderId	1+M items	high	<20ms	bounded
   Create order (+ items, + user counter)	W	order, items	—	med	<30ms	multi-item

   Rule: you can only query by what you have in hand. Every read's Args column must become a key or index prefix in step 3. If an Arg isn't a key, that read is a scan — reject the model.

2. Confirm store-family fit before modeling. Don't model a graph in a KV store.

   Family	Pick when	Avoid when	Examples
   Document	nested aggregate read/written as a unit; flexible fields; secondary indexes needed	heavy cross-doc joins; huge fan-out updates of shared data	MongoDB, Firestore
   Wide-column	massive write volume; time-series/feeds; query = known partition + range scan	ad-hoc queries on non-key columns; multi-key transactions	Cassandra, ScyllaDB, Bigtable
   Key-value / single-table	every access is by a designed key; you want one round trip per pattern	analytics / unpredictable query shapes	DynamoDB single-table, Redis-as-primary

   Default for an app backend with a fixed, known pattern set: document store unless write volume or strict single-digit-ms-at-scale forces wide-column/DynamoDB.

Common Errors

• Designing keys before listing access patterns. Guarantees a missing query path discovered in prod as a scan. Fill the step-1 table first, always.
• Low-cardinality or monotonic partition key (status, date, true, auto-increment id). Concentrates traffic on one shard → hot partition + throttling. Use a high-cardinality, evenly-hit key; write-shard or time-bucket if forced.
• Embedding an unbounded array (comments/events/followers inside the parent). Hits the 400 KB / 16 MB ceiling and makes every append rewrite the whole doc. Reference it as child items.
• Modeling relational then "adding NoSQL on top." Normalized tables + app-side joins = N+1 round trips and scans. Model the aggregate the query needs, even if it duplicates data.
• A Filter/$match on a non-key field mistaken for a query. DynamoDB FilterExpression and Mongo filters on un-indexed fields run after a scan reads everything — billed and slow. Make the filter field a key/index prefix.
• Denormalized copy with no fan-out path. userName cached in 10k orders, never updated on rename → permanent stale data. Define transaction / stream fan-out / async repair per duplicated field.
• One GSI per attribute "just in case." Each index is a full write-amplifying copy. Add a GSI only for an actual read pattern from step 1.
• Synchronous fan-out on the write path (loop updating thousands of copies in the request). Latency spikes and partial failures. Offload to a stream/queue.
• Unbounded partition growth (all rows under one PK, a whale tenant). Wide-column partition > ~100 MB degrades; DynamoDB throttles the key. Bucket by time or write-shard with a suffix.
• Blobs inline in the item. Caps how many items fit per read and wastes throughput. Offload to object storage, keep a pointer.

Verify

1. Coverage: every step-1 access pattern maps to exactly one Get/Query/index path; zero resolve to Scan or post-fetch filter on a non-key field.
2. Distribution: the partition key is high-cardinality and request-even; no sole partition key is a status/boolean/date/sequence. Estimate items & bytes per hottest partition — under the family ceiling (DynamoDB ~10 GB/partition soft, Cassandra < 100 MB, Mongo doc < 16 MB, DynamoDB item < 400 KB).
3. Range reads return already-sorted (sort/clustering key does the ordering); no client-side sort over a fetched set.
4. Embed/reference justified per relationship against the step-4 table; no unbounded array embedded; largest realistic item stays well under the limit.
5. M:N & secondary lookups each have an explicit path (adjacency item pair, composite SK, or GSI) — confirm both directions of every M:N.
6. Each denormalized field names its consistency mechanism (transaction / stream fan-out / async repair); none has two authoritative copies.
7. Throughput sim: project read+write units (or ops/s) per partition under peak from step-1 frequencies; confirm no single key exceeds the per-partition limit; write-shard/time-bucket where it does.
8. TTL set on every ephemeral entity; blobs > ~100 KB offloaded to object storage with only a pointer stored.

Done = every access pattern resolves to a single non-scan key/index path, no partition key is hot or unbounded under projected peak load, every embedded relationship is bounded under the item-size limit, and every denormalized copy has a named write-path keeping it consistent.