python

name	python
description	Python 3.11+ performance optimization guidelines (formerly python-311). This skill should be used when writing, reviewing, or refactoring Python code to ensure optimal performance patterns. Triggers on tasks involving asyncio, data structures, memory management, concurrency, loops, strings, or Python idioms.

Python 3.11 Best Practices

Comprehensive performance optimization guide for Python 3.11+ applications. Contains 42 rules across 8 categories, prioritized by impact to guide automated refactoring and code generation.

When to Apply

Reference these guidelines when:

• Writing new Python async I/O code
• Choosing data structures for collections
• Optimizing memory usage in data-intensive applications
• Implementing concurrent or parallel processing
• Reviewing Python code for performance issues

Rule Categories by Priority

Priority	Category	Impact	Prefix
1	I/O & Async Patterns	CRITICAL	io-
2	Data Structure Selection	CRITICAL	ds-
3	Memory Optimization	HIGH	mem-
4	Concurrency & Parallelism	HIGH	conc-
5	Loop & Iteration	MEDIUM	loop-
6	String Operations	MEDIUM	str-
7	Function & Call Overhead	LOW-MEDIUM	func-
8	Python Idioms & Micro	LOW	py-

Table of Contents

1. I/O & Async Patterns — CRITICAL

   • 1.1 Defer await Until Value Needed — CRITICAL (2-5× faster for dependent operations)
   • 1.2 Use aiofiles for Async File Operations — CRITICAL (prevents event loop blocking)
   • 1.3 Use asyncio.gather() for Concurrent I/O — CRITICAL (2-10× throughput improvement)
   • 1.4 Use Connection Pooling for Database Access — CRITICAL (100-200ms saved per connection)
   • 1.5 Use Semaphores to Limit Concurrent Operations — CRITICAL (prevents resource exhaustion)
   • 1.6 Use uvloop for Faster Event Loop — CRITICAL (2-4× faster async I/O)

2. Data Structure Selection — CRITICAL

   • 2.1 Use bisect for O(log n) Sorted List Operations — CRITICAL (O(n) to O(log n) search)
   • 2.2 Use defaultdict to Avoid Key Existence Checks — CRITICAL (eliminates redundant lookups)
   • 2.3 Use deque for O(1) Queue Operations — CRITICAL (O(n) to O(1) for popleft)
   • 2.4 Use Dict for O(1) Key-Value Lookup — CRITICAL (O(n) to O(1) lookup)
   • 2.5 Use frozenset for Hashable Set Keys — CRITICAL (enables set-of-sets patterns)
   • 2.6 Use Set for O(1) Membership Testing — CRITICAL (O(n) to O(1) lookup)

3. Memory Optimization — HIGH

   • 3.1 Intern Repeated Strings to Save Memory — HIGH (reduces duplicate string storage)
   • 3.2 Use slots for Memory-Efficient Classes — HIGH (20-50% memory reduction per instance)
   • 3.3 Use array.array for Homogeneous Numeric Data — HIGH (4-8× memory reduction for numbers)
   • 3.4 Use Generators for Large Sequences — HIGH (100-1000× memory reduction)
   • 3.5 Use weakref for Caches to Prevent Memory Leaks — HIGH (prevents unbounded cache growth)

References

1. Python 3.11 Release Notes
2. PEP 8 Style Guide
3. Python Wiki - Performance Tips
4. Real Python - Async IO
5. Real Python - LEGB Rule
6. Real Python - String Concatenation
7. Python Tutorial - Data Structures
8. CPython Exception Handling
9. DataCamp - Python Generators
10. JetBrains - Performance Hacks