Python JSON parsing best practices covering performance optimization (orjson/msgspec), handling large files (streaming/JSONL), security (injection prevention), and advanced querying (JSONPath/JMESPath). Use when working with JSON data, parsing APIs, handling large JSON files, or optimizing JSON performance.
Install with Codex or Claude Copy this prompt, paste it into Codex, Claude, or another assistant, and let it review the skill page and install it for you.
A direct command skips the review prompt. Inspect the source before running it.
Python JSON parsing best practices covering performance optimization (orjson/msgspec), handling large files (streaming/JSONL), security (injection prevention), and advanced querying (JSONPath/JMESPath). Use when working with JSON data, parsing APIs, handling large JSON files, or optimizing JSON performance.
Python JSON Parsing Best Practices
Comprehensive guide to JSON parsing in Python with focus on performance, security, and scalability.
Quick Start
Basic JSON Parsing
import json
# Parse JSON string
data = json.loads('{"name": "Alice", "age": 30}')
# Parse JSON filewithopen("data.json", "r", encoding="utf-8") as f:
data = json.load(f)
# Write JSON filewithopen("output.json", "w", encoding="utf-8") as f:
json.dump(data, f, indent=2)
Key Rule: Always specify encoding="utf-8" when reading/writing files.
When to Use This Skill
Use this skill when:
Working with JSON APIs or data interchange
Optimizing JSON performance in high-throughput applications
Handling large JSON files (> 100MB)
Securing applications against JSON injection
Extracting data from complex nested JSON structures
Performance: Choose the Right Library
Library Comparison (10,000 records benchmark)
Library
Serialize (s)
Deserialize (s)
Best For
orjson
0.42
1.27
FastAPI, web APIs (3.9x faster)
msgspec
0.49
0.93
Maximum performance (1.7x faster deserialization)
json (stdlib)
1.62
1.62
Universal compatibility
ujson
1.41
1.85
Drop-in replacement (2x faster)
Recommendation:
Use orjson for FastAPI/web APIs (native support, fastest serialization)
Use msgspec for data pipelines (fastest overall, typed validation)
Convert large JSON arrays to line-delimited format:
# Stream process JSONLwithopen("large.jsonl", "r") as infile, open("output.jsonl", "w") as outfile:
for line in infile:
obj = json.loads(line)
obj["processed"] = True
outfile.write(json.dumps(obj) + "\n")
Strategy 2: Streaming with ijson
import ijson
# Process large JSON without loading into memorywithopen("huge.json", "rb") as f:
for item in ijson.items(f, "products.item"):
process(item) # Handle one item at a time
See: patterns/streaming-large-json.md
Security: Prevent JSON Injection
Critical Rules:
Always use json.loads(), never eval()
Validate input with jsonschema
Sanitize user input before serialization
Escape special characters (" and \)
Vulnerable Code:
# NEVER DO THIS
username = request.GET['username'] # User input: admin", "role": "admin
json_string = f'{{"user":"{username}","role":"user"}}'# Result: privilege escalation
Secure Code:
# Use json.dumps for serialization
data = {"user": username, "role": "user"}
json_string = json.dumps(data) # Properly escaped
