opencv

# OpenCV - Computer Vision and Image Processing OpenCV (Open Source Computer Vision Library) is the de facto standard library for computer vision tasks. It provides 2500+ optimized algorithms for real-time image and video processing, from basic operations like reading images to advanced tasks like face recognition and 3D reconstruction. ## When to Use - Reading, writing, and displaying images and videos from files or cameras. - Image preprocessing (resizing, cropping, rotating, color conversion). - Edge detection (Canny, Sobel) and contour finding. - Feature detection and matching (SIFT, ORB, AKAZE). - Object detection (Haar Cascades, HOG, DNN module for YOLO/SSD). - Face detection and recognition. - Image segmentation (thresholding, watershed, GrabCut). - Video analysis (motion detection, object tracking, optical flow). - Camera calibration and 3D reconstruction. - Image stitching and panorama creation. - Real-time applications requiring fast performance. ## Reference Documentation **Official docs**: https://docs.opencv.org/4.x/ **GitHub**: https://github.com/opencv/opencv **Tutorials**: https://docs.opencv.org/4.x/d9/df8/tutorial_root.html **Search patterns**: `cv2.imread`, `cv2.cvtColor`, `cv2.Canny`, `cv2.findContours`, `cv2.VideoCapture` ## Core Principles ### Image as NumPy Array OpenCV represents images as NumPy arrays with shape (height, width, channels). This allows seamless integration with NumPy operations and other scientific Python libraries. ### BGR Color Space (Not RGB!) OpenCV uses BGR (Blue-Green-Red) instead of RGB by default. This is critical to remember when displaying images or integrating with other libraries. ### In-Place vs Copy Operations Many OpenCV functions modify images in-place for performance. Understanding when copies are made is essential for efficient code. ### C++ Performance in Python OpenCV is written in optimized C++, making it extremely fast even when called from Python. Avoid Python loops when OpenCV vectorized operations exist. ## Quick Reference ### Installation ```bash # Basic OpenCV pip install opencv-python # With contrib modules (SIFT, SURF, etc.) pip install opencv-contrib-python # Headless (no GUI, for servers) pip install opencv-python-headless ``` ### Standard Imports ```python import cv2 import numpy as np import matplotlib.pyplot as plt ``` ### Basic Pattern - Read, Process, Display ```python import cv2 # 1. Read image img = cv2.imread('image.jpg') # 2. Process (convert to grayscale) gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # 3. Display cv2.imshow('Grayscale', gray) cv2.waitKey(0) # Wait for key press cv2.destroyAllWindows() ``` ### Basic Pattern - Video Processing ```python import cv2 # 1. Open video capture cap = cv2.VideoCapture(0) # 0 = default camera, or 'video.mp4' while True: # 2. Read frame ret, frame = cap.read() if not ret: break # 3. Process frame gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) # 4. Display cv2.imshow('Video', gray) # 5. Exit on 'q' key if cv2.waitKey(1) & 0xFF == ord('q'): break # 6. Cleanup cap.release() cv2.destroyAllWindows() ``` ## Critical Rules ### ✅ DO - **Check Image Loaded** - Always verify `img is not None` after `cv2.imread()` to catch file errors. - **Use cv2.cvtColor() for Color Conversion** - Don't manually rearrange channels; use the provided conversion codes. - **Release Resources** - Always call `cap.release()` and `cv2.destroyAllWindows()` when done with video/windows. - **Copy Before Modifying** - Use `img.copy()` if you need to preserve the original image. - **Use Appropriate Data Types** - Keep images as uint8 (0-255) for display, convert to float32 (0-1) for mathematical operations. - **Validate VideoCapture** - Check `cap.isOpened()` before reading frames. - **Use BGR2RGB for Matplotlib** - Convert BGR to RGB when displaying with matplotlib. - **Vectorize Operations** - Use OpenCV's built-in functions instead of Python loops over pixels. ### ❌ DON'T - **Don't Assume RGB** - OpenCV uses BGR by default; convert to RGB for matplotlib or PIL. - **Don't Forget waitKey()** - Without `cv2.waitKey()`, windows won't display properly. - **Don't Mix PIL and OpenCV Directly** - Convert between them explicitly (OpenCV uses BGR, PIL uses RGB). - **Don't Process Video in Memory** - Process frame-by-frame to avoid memory issues with large videos. - **Don't Use Python Loops for Pixels** - This is 100x slower than vectorized operations. - **Don't Hardcode Paths** - Use `os.path.join()` or `pathlib` for cross-platform compatibility. ## Anti-Patterns (NEVER) ```python import cv2 import numpy as np # ❌ BAD: Not checking if image loaded img = cv2.imread('image.jpg') gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Crashes if file doesn't exist! # ✅ GOOD: Always validate img = cv2.imread('image.jpg') if img is None: raise FileNotFoundError("Image not found") gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # ❌ BAD: Using Python loops for pixel manipulation for i in range(img.shape[0]): for j in range(img.shape[1]): img[i, j] = img[i, j] * 0.5 # Extremely slow! # ✅ GOOD: Vectorized NumPy operations img = (img * 0.5).astype(np.uint8) # ❌ BAD: Displaying BGR image with matplotlib plt.imshow(img) # Colors will be wrong! # ✅ GOOD: Convert to RGB first img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) plt.imshow(img_rgb) # ❌ BAD: Not releasing video capture cap = cv2.VideoCapture('video.mp4') while cap.read()[0]: pass # Memory leak! Camera still locked! # ✅ GOOD: Always release cap = cv2.VideoCapture('video.mp4') try: while cap.read()[0]: pass finally: cap.release() ``` ## Image I/O and Display ### Reading and Writing Images ```python import cv2 # Read image (returns None if failed) img = cv2.imread('image.jpg') # Read as grayscale gray = cv2.imread('image.jpg', cv2.IMREAD_GRAYSCALE) # Read with alpha channel img_alpha = cv2.imread('image.png', cv2.IMREAD_UNCHANGED) # Write image cv2.imwrite('output.jpg', img) # Write with quality (JPEG: 0-100, PNG: 0-9 compression) cv2.imwrite('output.jpg', img, [cv2.IMWRITE_JPEG_QUALITY, 95]) cv2.imwrite('output.png', img, [cv2.IMWRITE_PNG_COMPRESSION, 9]) # Check if image loaded if img is None: print("Error: Could not load image") else: print(f"Image shape: {img.shape}") # (height, width, channels) ``` ### Display Images ```python import cv2 # Display image in window cv2.imshow('Window Name', img) cv2.waitKey(0) # Wait indefinitely for key press cv2.destroyAllWindows() # Display for specific duration (milliseconds) cv2.imshow('Image', img) cv2.waitKey(3000) # Wait 3 seconds cv2.destroyAllWindows() # Display multiple images cv2.imshow('Original', img) cv2.imshow('Gray', gray) cv2.waitKey(0) cv2.destroyAllWindows() # Display with matplotlib (convert BGR to RGB!) import matplotlib.pyplot as plt img_rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) plt.imshow(img_rgb) plt.axis('off') plt.show() ``` ### Video Capture ```python import cv2 # Open camera (0 = default, 1 = second camera, etc.) cap = cv2.VideoCapture(0) # Open video file cap = cv2.VideoCapture('video.mp4') # Check if opened successfully if not cap.isOpened(): print("Error: Could not open video") exit() # Get video properties fps = cap.get(cv2.CAP_PROP_FPS) width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) total_frames = int(cap.get(cv2.CAP_PROP_FRAME_COUNT)) print(f"Video: {width}x{height} @ {fps} fps, {total_frames} frames") # Read and process frames while True: ret, frame = cap.read() if not ret: print("End of video or error") break # Process frame here cv2.imshow('Frame', frame) if cv2.waitKey(1) & 0xFF == ord('q'): break cap.release() cv2.destroyAllWindows() ``` ### Writing Videos ```python import cv2 cap = cv2.VideoCapture('input.mp4') # Get video properties fps = int(cap.get(cv2.CAP_PROP_FPS)) width = int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)) height = int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)) # Create VideoWriter fourcc = cv2.VideoWriter_fourcc(*'mp4v') # or 'XVID', 'MJPG' out = cv2.VideoWriter('output.mp4', fourcc, fps, (width, height)) while True: ret, frame = cap.read() if not ret: break # Process frame processed = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY) processed = cv2.cvtColor(processed, cv2.COLOR_GRAY2BGR) # Convert back to 3-channel # Write frame out.write(processed) cap.release() out.release() cv2.destroyAllWindows() ``` ## Image Transformations ### Resizing and Cropping ```python import cv2 img = cv2.imread('image.jpg') # Resize to specific dimensions resized = cv2.resize(img, (800, 600)) # (width, height) # Resize by scale factor scaled = cv2.resize(img, None, fx=0.5, fy=0.5) # 50% of original # Resize with interpolation methods resized_linear = cv2.resize(img, (800, 600), interpolation=cv2.INTER_LINEAR) # Default resized_cubic = cv2.resize(img, (800, 600), interpolation=cv2.INTER_CUBIC) # Better quality resized_area = cv2.resize(img, (400, 300), interpolation=cv2.INTER_AREA) # Best for shrinking # Crop (using NumPy slicing) height, width = img.shape[:2] cropped = img[100:400, 200:600] # [y1:y2, x1:x2] # Center crop crop_size = 300 center_x, center_y = width // 2, height // 2 x1 = center_x - crop_size // 2 y1 = center_y - crop_size // 2 center_cropped = img[y1:y1+crop_size, x1:x1+crop_size] ``` ### Rotation and Flipping ```python import cv2 # Flip horizontally flipped_h = cv2.flip(img, 1) # Flip vertically flipped_v = cv2.flip(img, 0) # Flip both flipped_both = cv2.flip(img, -1) # Rotate 90 degrees clockwise rotated_90 = cv2.rotate(img, cv2.ROTATE_90_CLOCKWISE) # Rotate 180 degrees rotated_180 = cv2.rotate(img, cv2.ROTATE_180) # Rotate 90 degrees counter-clockwise rotated_90_ccw = cv2.rotate(img, cv2.ROTATE_90_COUNTERCLOCKWISE) # Rotate by arbitrary angle (around center) height, width = img.shape[:2] center = (width // 2, height // 2) angle = 45 # degrees # Get rotation matrix M = cv2.getRotationMatrix2D(center, angle, scale=1.0) # Apply rotation rotated = cv2.warpAffine(img, M, (width, height)) # Rotate and scale M_scaled = cv2.getRotationMatrix2D(center, 30, scale=0.8) rotated_scaled = cv2.warpAffine(img, M_scaled, (width, height)) ``` ### Color Space Conversions ```python import cv2 img = cv2.imread('image.jpg') # BGR to RGB (for matplotlib) rgb = cv2.cvtColor(img, cv2.COLOR_BGR2RGB) # BGR to Grayscale gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # BGR to HSV (useful for color-based segmentation) hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV) # BGR to LAB lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB) # Grayscale to BGR (add color channels) gray_bgr = cv2.cvtColor(gray, cv2.COLOR_GRAY2BGR) # Extract individual channels b, g, r = cv2.split(img) # Merge channels merged = cv2.merge([b, g, r]) ``` ## Image Filtering and Enhancement ### Blurring and Smoothing ```python import cv2 # Gaussian blur (reduce noise) blurred = cv2.GaussianBlur(img, (5, 5), 0) # (kernel_size, sigma) # Median blur (good for salt-and-pepper noise) median = cv2.medianBlur(img, 5) # kernel_size must be odd # Bilateral filter (edge-preserving smoothing) bilateral = cv2.bilateralFilter(img, 9, 75, 75) # (d, sigmaColor, sigmaSpace) # Average blur avg_blur = cv2.blur(img, (5, 5)) # Box filter box = cv2.boxFilter(img, -1, (5, 5)) ``` ### Edge Detection ```python import cv2 # Convert to grayscale first gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY) # Canny edge detection edges = cv2.Canny(gray, threshold1=50, threshold2=150) # Sobel edge detection (gradient in x and y) sobelx = cv2.Sobel(gray, cv2.CV_64F, 1, 0, ksize=3) # X gradient