using-document-databases

name	using-document-databases
description	Document database implementation for flexible schema applications. Use when building content management, user profiles, catalogs, or event logging. Covers MongoDB (primary), DynamoDB, Firestore, schema design patterns, indexing strategies, and aggregation pipelines.

Document Database Implementation

Guide NoSQL document database selection and implementation for flexible schema applications across Python, TypeScript, Rust, and Go.

When to Use This Skill

Use document databases when applications need:

• Flexible schemas - Data models evolve rapidly without migrations
• Nested structures - JSON-like hierarchical data
• Horizontal scaling - Built-in sharding and replication
• Developer velocity - Object-to-database mapping without ORM complexity

Database Selection

Quick Decision Framework

DEPLOYMENT ENVIRONMENT?
├── AWS-Native Application → DynamoDB
│   ✓ Serverless, auto-scaling, single-digit ms latency
│   ✗ Limited query flexibility
│
├── Firebase/GCP Ecosystem → Firestore
│   ✓ Real-time sync, offline support, mobile-first
│   ✗ More expensive for heavy reads
│
└── General-Purpose/Complex Queries → MongoDB
    ✓ Rich aggregation, full-text search, vector search
    ✓ ACID transactions, self-hosted or managed

Database Comparison

Database	Best For	Latency	Max Item	Query Language
MongoDB	General-purpose, complex queries	1-5ms	16MB	MQL (rich)
DynamoDB	AWS serverless, predictable performance	<10ms	400KB	PartiQL (limited)
Firestore	Real-time apps, mobile-first	50-200ms	1MB	Firebase queries

See references/mongodb.md for MongoDB details See references/dynamodb.md for DynamoDB single-table design See references/firestore.md for Firestore real-time patterns

Schema Design Patterns

Embedding vs Referencing

Use the decision matrix in references/schema-design-patterns.md

Quick guide:

Relationship	Pattern	Example
One-to-Few	Embed	User addresses (2-3 max)
One-to-Many	Hybrid	Blog posts → comments
One-to-Millions	Reference	User → events (logging)
Many-to-Many	Reference	Products ↔ Categories

Embedding Example (MongoDB)

// User with embedded addresses
{
  _id: ObjectId("..."),
  email: "[e-mail]",
  name: "Jane Doe",
  addresses: [
    {
      type: "home",
      street: "123 Main St",
      city: "Boston",
      default: true
    }
  ],
  preferences: {
    theme: "dark",
    notifications: { email: true, sms: false }
  }
}

Referencing Example (E-commerce)

// Orders reference products
{
  _id: ObjectId("..."),
  userId: ObjectId("..."),
  items: [
    {
      productId: ObjectId("..."),      // Reference
      priceAtPurchase: 49.99,          // Denormalize (historical)
      quantity: 2
    }
  ],
  totalAmount: 99.98
}

When to denormalize:

• Frequently read together
• Historical snapshots (prices, names)
• Read-heavy workloads

Indexing Strategies

MongoDB Index Types

// 1. Single field (unique email)
db.users.createIndex({ email: 1 }, { unique: true })

// 2. Compound index (ORDER MATTERS!)
db.orders.createIndex({ status: 1, createdAt: -1 })

// 3. Partial index (index subset)
db.orders.createIndex(
  { userId: 1 },
  { partialFilterExpression: { status: { $eq: "pending" }}}
)

// 4. TTL index (auto-delete after 30 days)
db.sessions.createIndex(
  { createdAt: 1 },
  { expireAfterSeconds: 2592000 }
)

// 5. Text index (full-text search)
db.articles.createIndex({
  title: "text",
  content: "text"
})

Index Best Practices:

• Add indexes for all query filters
• Compound index order: Equality → Range → Sort
• Use covering indexes (query + projection in index)
• Use explain() to verify index usage
• Monitor with Performance Advisor (Atlas)

Validate indexes with the script:

python scripts/validate_indexes.py

See references/indexing-strategies.md for complete guide.

MongoDB Aggregation Pipelines

Key Operators: $match (filter), $group (aggregate), $lookup (join), $unwind (arrays), $project (reshape)

For complete pipeline patterns and examples, see: references/aggregation-patterns.md

DynamoDB Single-Table Design

Design for access patterns using PK/SK patterns. Store multiple entity types in one table with composite keys.

For complete single-table design patterns and GSI strategies, see: references/dynamodb.md

Firestore Real-Time Patterns

Use onSnapshot() for real-time listeners and Firestore security rules for access control.

For complete real-time patterns and security rules, see: references/firestore.md

Multi-Language Examples

Complete implementations available in examples/ directory:

• examples/mongodb-fastapi/ - Python FastAPI + MongoDB
• examples/mongodb-nextjs/ - TypeScript Next.js + MongoDB
• examples/dynamodb-serverless/ - Python Lambda + DynamoDB
• examples/firestore-react/ - React + Firestore real-time

Frontend Skill Integration

• Media Skill - Use MongoDB GridFS for large file storage with metadata
• AI Chat Skill - MongoDB Atlas Vector Search for semantic conversation retrieval
• Feedback Skill - DynamoDB for high-throughput event logging with TTL

For integration examples, see: references/skill-integrations.md

Performance Optimization

Key practices:

• Always use indexes for query filters (verify with .explain())
• Use connection pooling (reuse clients across requests)
• Avoid collection scans in production

For complete optimization guide, see: references/performance.md

Common Patterns

Pagination: Use cursor-based pagination for large datasets (recommended over offset) Soft Deletes: Mark as deleted with timestamp instead of removing Audit Logs: Store version history within documents

For implementation details, see: references/common-patterns.md

Validation and Scripts

Validate Index Coverage

# Run validation script
python scripts/validate_indexes.py --db myapp --collection orders

# Output:
# ✓ Query { status: "pending" } covered by index status_1
# ✗ Query { userId: "..." } missing index - add: { userId: 1 }

Schema Analysis

# Analyze schema patterns
python scripts/analyze_schema.py --db myapp

# Output:
# Collection: users
# - Average document size: 2.4 KB
# - Embedding ratio: 87% (addresses, preferences)
# - Reference ratio: 13% (orderIds)
# Recommendation: Good balance

Anti-Patterns to Avoid

Unbounded Arrays: Limit embedded arrays (use references for large collections) Over-Indexing: Only index queried fields (indexes slow writes) DynamoDB Scans: Always use Query with partition key (avoid Scan)

For detailed anti-patterns, see: references/anti-patterns.md

Dependencies

Python

# MongoDB
pip install motor pymongo

# DynamoDB
pip install boto3

# Firestore
pip install firebase-admin

TypeScript

# MongoDB
npm install mongodb

# DynamoDB
npm install @aws-sdk/client-dynamodb @aws-sdk/util-dynamodb

# Firestore
npm install firebase firebase-admin

Rust

# MongoDB
mongodb = "2.8"

# DynamoDB
aws-sdk-dynamodb = "1.0"

Go

# MongoDB
go get go.mongodb.org/mongo-driver

# DynamoDB
go get github.com/aws/aws-sdk-go-v2/service/dynamodb

Additional Resources

Database-Specific Guides:

• references/mongodb.md - Complete MongoDB documentation
• references/dynamodb.md - DynamoDB single-table patterns
• references/firestore.md - Firestore real-time guide

Pattern Guides:

• references/schema-design-patterns.md - Embedding vs referencing decisions
• references/indexing-strategies.md - Index optimization
• references/aggregation-patterns.md - MongoDB pipeline cookbook
• references/common-patterns.md - Pagination, soft deletes, audit logs
• references/anti-patterns.md - Mistakes to avoid
• references/performance.md - Query optimization
• references/skill-integrations.md - Frontend skill integration

Examples: examples/mongodb-fastapi/, examples/mongodb-nextjs/, examples/dynamodb-serverless/, examples/firestore-react/