numpy

# NumPy - Numerical Python The fundamental package for numerical computing in Python, providing multi-dimensional arrays and fast operations. ## When to Use - Working with multi-dimensional arrays and matrices - Performing element-wise operations on arrays - Linear algebra computations (matrix multiplication, eigenvalues, SVD) - Random number generation and statistical distributions - Fourier transforms and signal processing basics - Mathematical operations (trigonometric, exponential, logarithmic) - Broadcasting operations across different array shapes - Vectorizing Python loops for performance - Reading and writing numerical data to files - Building numerical algorithms and simulations - Serving as foundation for pandas, scikit-learn, SciPy ## Reference Documentation **Official docs**: https://numpy.org/doc/ **Search patterns**: `np.array`, `np.zeros`, `np.dot`, `np.linalg`, `np.random`, `np.broadcast` ## Core Principles ### Use NumPy For | Task | Function | Example | |------|----------|---------| | Create arrays | `array`, `zeros`, `ones` | `np.array([1, 2, 3])` | | Mathematical ops | `+`, `*`, `sin`, `exp` | `np.sin(arr)` | | Linear algebra | `dot`, `linalg.inv` | `np.dot(A, B)` | | Statistics | `mean`, `std`, `percentile` | `np.mean(arr)` | | Random numbers | `random.rand`, `random.normal` | `np.random.rand(10)` | | Indexing | `[]`, boolean, fancy | `arr[arr > 0]` | | Broadcasting | Automatic | `arr + scalar` | | Reshaping | `reshape`, `flatten` | `arr.reshape(2, 3)` | ### Do NOT Use For - String manipulation (use built-in str or pandas) - Complex data structures (use pandas DataFrame) - Symbolic mathematics (use SymPy) - Deep learning (use PyTorch, TensorFlow) - Sparse matrices (use scipy.sparse) ## Quick Reference ### Installation ```bash # pip pip install numpy # conda conda install numpy # Specific version pip install numpy==1.26.0 ``` ### Standard Imports ```python import numpy as np # Common submodules from numpy import linalg as la from numpy import random as rand from numpy import fft # Never import * # from numpy import * # DON'T DO THIS! ``` ### Basic Pattern - Array Creation ```python import numpy as np # From list arr = np.array([1, 2, 3, 4, 5]) # Zeros and ones zeros = np.zeros((3, 4)) ones = np.ones((2, 3)) # Range range_arr = np.arange(0, 10, 2) # [0, 2, 4, 6, 8] # Linspace linspace_arr = np.linspace(0, 1, 5) # [0, 0.25, 0.5, 0.75, 1] print(f"Array: {arr}") print(f"Shape: {arr.shape}") print(f"Dtype: {arr.dtype}") ``` ### Basic Pattern - Array Operations ```python import numpy as np a = np.array([1, 2, 3]) b = np.array([4, 5, 6]) # Element-wise operations c = a + b # [5, 7, 9] d = a * b # [4, 10, 18] e = a ** 2 # [1, 4, 9] # Mathematical functions f = np.sin(a) g = np.exp(a) print(f"Sum: {c}") print(f"Product: {d}") ``` ### Basic Pattern - Linear Algebra ```python import numpy as np # Matrix multiplication A = np.array([[1, 2], [3, 4]]) B = np.array([[5, 6], [7, 8]]) # Dot product C = np.dot(A, B) # or A @ B # Matrix inverse A_inv = np.linalg.inv(A) # Eigenvalues eigenvalues, eigenvectors = np.linalg.eig(A) print(f"Matrix product:\n{C}") print(f"Eigenvalues: {eigenvalues}") ``` ## Critical Rules ### ✅ DO - **Use vectorization** - Avoid Python loops, use array operations - **Specify dtype explicitly** - For memory efficiency and precision control - **Use views when possible** - Avoid unnecessary copies - **Broadcast properly** - Understand broadcasting rules - **Check array shapes** - Use `.shape` frequently - **Use axis parameter** - For operations along specific dimensions - **Pre-allocate arrays** - Don't grow arrays in loops - **Use appropriate dtypes** - int32, float64, complex128, etc. - **Copy when needed** - Use `.copy()` for independent arrays - **Use built-in functions** - They're optimized in C ### ❌ DON'T - **Loop over arrays** - Use vectorization instead - **Grow arrays dynamically** - Pre-allocate instead - **Use Python lists for math** - Convert to arrays first - **Ignore memory layout** - C-contiguous vs Fortran-contiguous matters - **Mix dtypes carelessly** - Know implicit type promotion rules - **Modify arrays during iteration** - Can cause undefined behavior - **Use == for array comparison** - Use `np.array_equal()` or `np.allclose()` - **Assume views vs copies** - Check with `.base` attribute - **Ignore NaN handling** - Use `np.nanmean()`, `np.nanstd()`, etc. - **Use outdated APIs** - Check for deprecated functions ## Anti-Patterns (NEVER) ```python import numpy as np # ❌ BAD: Python loops result = [] for i in range(len(arr)): result.append(arr[i] * 2) result = np.array(result) # ✅ GOOD: Vectorization result = arr * 2 # ❌ BAD: Growing arrays result = np.array([]) for i in range(1000): result = np.append(result, i) # Very slow! # ✅ GOOD: Pre-allocate result = np.zeros(1000) for i in range(1000): result[i] = i # Even better: Use arange result = np.arange(1000) # ❌ BAD: Comparing arrays with == if arr1 == arr2: # This is ambiguous! print("Equal") # ✅ GOOD: Use appropriate comparison if np.array_equal(arr1, arr2): print("Equal") # Or for floating point if np.allclose(arr1, arr2, rtol=1e-5): print("Close enough") # ❌ BAD: Ignoring dtypes arr = np.array([1, 2, 3]) arr[0] = 1.5 # Silently truncates to 1! # ✅ GOOD: Explicit dtype arr = np.array([1, 2, 3], dtype=float) arr[0] = 1.5 # Now works correctly # ❌ BAD: Unintentional modification a = np.array([1, 2, 3]) b = a # b is just a reference! b[0] = 999 # Also modifies a! # ✅ GOOD: Explicit copy a = np.array([1, 2, 3]) b = a.copy() # b is independent b[0] = 999 # a is unchanged ``` ## Array Creation ### Basic Array Creation ```python import numpy as np # From Python list arr1 = np.array([1, 2, 3, 4, 5]) # From nested list (2D) arr2 = np.array([[1, 2, 3], [4, 5, 6]]) # Specify dtype arr3 = np.array([1, 2, 3], dtype=np.float64) arr4 = np.array([1, 2, 3], dtype=np.int32) # From tuple arr5 = np.array((1, 2, 3)) # Complex numbers arr6 = np.array([1+2j, 3+4j]) print(f"1D array: {arr1}") print(f"2D array:\n{arr2}") print(f"Float array: {arr3}") ``` ### Special Array Creation ```python import numpy as np # Zeros zeros = np.zeros((3, 4)) # 3x4 array of zeros # Ones ones = np.ones((2, 3, 4)) # 2x3x4 array of ones # Empty (uninitialized) empty = np.empty((2, 2)) # Faster but values are garbage # Full (constant value) full = np.full((3, 3), 7) # 3x3 array filled with 7 # Identity matrix identity = np.eye(4) # 4x4 identity matrix # Diagonal matrix diag = np.diag([1, 2, 3, 4]) print(f"Zeros shape: {zeros.shape}") print(f"Identity:\n{identity}") ``` ### Range-Based Creation ```python import numpy as np # Arange (like Python range) a = np.arange(10) # [0, 1, 2, ..., 9] b = np.arange(2, 10) # [2, 3, 4, ..., 9] c = np.arange(0, 10, 2) # [0, 2, 4, 6, 8] d = np.arange(0, 1, 0.1) # [0, 0.1, 0.2, ..., 0.9] # Linspace (linearly spaced) e = np.linspace(0, 1, 5) # [0, 0.25, 0.5, 0.75, 1] f = np.linspace(0, 10, 100) # 100 points from 0 to 10 # Logspace (logarithmically spaced) g = np.logspace(0, 2, 5) # [1, 10^0.5, 10, 10^1.5, 100] # Geomspace (geometrically spaced) h = np.geomspace(1, 1000, 4) # [1, 10, 100, 1000] print(f"Arange: {a}") print(f"Linspace: {e}") ``` ### Array Copies and Views ```python import numpy as np original = np.array([1, 2, 3, 4, 5]) # View (shares memory) view = original[:] view[0] = 999 # Modifies original! # Copy (independent) copy = original.copy() copy[0] = 777 # Doesn't affect original # Check if array is a view print(f"Is view? {view.base is original}") print(f"Is copy? {copy.base is None}") # Some operations create views, some create copies slice_view = original[1:3] # View boolean_copy = original[original > 2] # Copy! ``` ## Array Indexing and Slicing ### Basic Indexing ```python import numpy as np arr = np.array([10, 20, 30, 40, 50]) # Single element print(arr[0]) # 10 print(arr[-1]) # 50 (last element) # Slicing print(arr[1:4]) # [20, 30, 40] print(arr[:3]) # [10, 20, 30] print(arr[2:]) # [30, 40, 50] print(arr[::2]) # [10, 30, 50] (every 2nd element) # Negative indices print(arr[-3:-1]) # [30, 40] # Reverse print(arr[::-1]) # [50, 40, 30, 20, 10] ``` ### Multi-Dimensional Indexing ```python import numpy as np arr = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]]) # Single element print(arr[0, 0]) # 1 print(arr[1, 2]) # 6 print(arr[-1, -1]) # 9 # Row slicing print(arr[0]) # [1, 2, 3] (first row) print(arr[1, :]) # [4, 5, 6] (second row) # Column slicing print(arr[:, 0]) # [1, 4, 7] (first column) print(arr[:, 1]) # [2, 5, 8] (second column) # Sub-array print(arr[0:2, 1:3]) # [[2, 3], [5, 6]] # Every other element print(arr[::2, ::2]) # [[1, 3], [7, 9]] ``` ### Boolean Indexing ```python import numpy as np arr = np.array([1, 2, 3, 4, 5, 6, 7, 8, 9, 10]) # Boolean condition mask = arr > 5 print(mask) # [False, False, False, False, False, True, True, True, True, True] # Boolean indexing filtered = arr[arr > 5]