content-hash-cache-patterns

name	content-hash-cache-patterns
description	Cache expensive file processing results using SHA-256 content hashes — path-independent, auto-invalidating, with service layer separation.

Content-Hash File Cache Pattern

Cache expensive file processing results (PDF parsing, text extraction, image analysis) using SHA-256 content hashes as cache keys. Unlike path-based caching, this approach survives file moves/renames and auto-invalidates when content changes.

When to Activate

• Building file processing pipelines (PDF, images, text extraction)
• Processing cost is high and same files are processed repeatedly
• Need a --cache/--no-cache CLI option
• Want to add caching to existing pure functions without modifying them
• Adding caching to a batch LLM pipeline that processes the same uploaded documents across multiple runs or user sessions
• Designing a cache layer that must survive file renames and directory reorganizations without invalidating existing entries
• Implementing cache invalidation that is automatic and correct without requiring a manual cache-clear step when file content changes
• Separating caching concerns from a pure extraction or analysis function using a service layer wrapper that respects the single-responsibility principle

Core Pattern

1. Content-Hash Based Cache Key

Use file content (not path) as the cache key:

import hashlib
from pathlib import Path

_HASH_CHUNK_SIZE = 65536  # 64KB chunks for large files

def compute_file_hash(path: Path) -> str:
    """SHA-256 of file contents (chunked for large files)."""
      path.is_file():
         FileNotFoundError()
    sha256 = hashlib.sha256()
     (path, )  f:
         :
            chunk = f.read(_HASH_CHUNK_SIZE)
              chunk:
                
            sha256.update(chunk)
     sha256.hexdigest()