bash-master

# Bash Scripting Mastery ## 🚨 CRITICAL GUIDELINES ### Windows File Path Requirements **MANDATORY: Always Use Backslashes on Windows for File Paths** When using Edit or Write tools on Windows, you MUST use backslashes (`\`) in file paths, NOT forward slashes (`/`). **Examples:** - ❌ WRONG: `D:/repos/project/file.tsx` - ✅ CORRECT: `D:\repos\project\file.tsx` This applies to: - Edit tool file_path parameter - Write tool file_path parameter - All file operations on Windows systems ### Documentation Guidelines **NEVER create new documentation files unless explicitly requested by the user.** - **Priority**: Update existing README.md files rather than creating new documentation - **Repository cleanliness**: Keep repository root clean - only README.md unless user requests otherwise - **Style**: Documentation should be concise, direct, and professional - avoid AI-generated tone - **User preference**: Only create additional .md files when user specifically asks for documentation --- Comprehensive guide for writing professional, portable, and maintainable bash scripts across all platforms. --- ## TL;DR QUICK REFERENCE **Essential Checklist for Every Bash Script:** ```bash #!/usr/bin/env bash set -euo pipefail # Exit on error, undefined vars, pipe failures IFS=$'\n\t' # Safe word splitting # Use: shellcheck your_script.sh before deployment # Test on target platform(s) before production ``` **Platform Compatibility Quick Check:** ```bash # Linux/macOS: ✓ Full bash features # Git Bash (Windows): ✓ Most features, ✗ Some system calls # Containers: ✓ Depends on base image # POSIX mode: Use /bin/sh and avoid bashisms ``` --- ## Overview This skill provides expert bash/shell scripting knowledge for ANY scripting task, ensuring professional-grade quality across all platforms. **MUST use this skill for:** - ✅ ANY bash/shell script creation or modification - ✅ System automation and tooling - ✅ DevOps/CI/CD pipeline scripts - ✅ Build and deployment automation - ✅ Script review, debugging, or optimization - ✅ Converting manual commands to automated scripts - ✅ Cross-platform script compatibility **What this skill provides:** - **Google Shell Style Guide compliance** - Industry-standard formatting and patterns - **ShellCheck validation** - Automatic detection of common issues - **Cross-platform compatibility** - Linux, macOS, Windows (Git Bash/WSL), containers - **POSIX compliance** - Portable scripts that work everywhere - **Security hardening** - Input validation, injection prevention, privilege management - **Error handling** - Robust `set -euo pipefail`, trap handlers, exit codes - **Performance optimization** - Efficient patterns, avoiding anti-patterns - **Testing with BATS** - Unit testing, integration testing, CI/CD integration - **Debugging techniques** - Logging, troubleshooting, profiling - **Production-ready patterns** - Templates and best practices for real-world use **This skill activates automatically for:** - Any mention of "bash", "shell", "script" in task - System automation requests - DevOps/CI/CD tasks - Build/deployment automation - Command line tool creation --- ## Core Principles ### 1. Safety First **ALWAYS start scripts with safety settings:** ```bash #!/usr/bin/env bash # Fail fast and loud set -e # Exit on any error set -u # Exit on undefined variable set -o pipefail # Exit on pipe failure set -E # ERR trap inherited by functions # Optionally: # set -x # Debug mode (print commands before execution) # set -C # Prevent file overwrites with redirection # Safe word splitting IFS=$'\n\t' # Script metadata readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" readonly SCRIPT_NAME="$(basename "${BASH_SOURCE[0]}")" ``` **Why this matters:** - `set -e`: Prevents cascading failures - `set -u`: Catches typos in variable names - `set -o pipefail`: Catches failures in the middle of pipes - `IFS=$'\n\t'`: Prevents word splitting on spaces (security issue) ### 2. POSIX Compatibility vs Bash Features **Know when to use which:** ```bash # POSIX-compliant (portable across shells) #!/bin/sh # Use: [ ] tests, no arrays, no [[ ]], no <(process substitution) # Bash-specific (modern features, clearer syntax) #!/usr/bin/env bash # Use: [[ ]], arrays, associative arrays, <(), process substitution ``` **Decision matrix:** - Need to run on any UNIX system → Use `#!/bin/sh` and POSIX only - Control the environment (modern Linux/macOS) → Use `#!/usr/bin/env bash` - Need advanced features (arrays, regex) → Use `#!/usr/bin/env bash` ### 3. Quoting Rules (Critical) ```bash # ALWAYS quote variables to prevent word splitting and globbing bad_cmd=$file_path # ✗ WRONG - word splitting good_cmd="$file_path" # ✓ CORRECT # Arrays: Quote expansion files=("file 1.txt" "file 2.txt") process "${files[@]}" # ✓ CORRECT - each element quoted process "${files[*]}" # ✗ WRONG - all elements as one string # Command substitution: Quote the result result="$(command)" # ✓ CORRECT result=$(command) # ✗ WRONG (unless you want word splitting) # Exception: When you WANT word splitting # shellcheck disable=SC2086 flags="-v -x -z" command $flags # Intentional word splitting ``` ### 4. Use ShellCheck **ALWAYS run ShellCheck before deployment:** ```bash # Install # Ubuntu/Debian: apt-get install shellcheck # macOS: brew install shellcheck # Windows: scoop install shellcheck # Usage shellcheck your_script.sh shellcheck -x your_script.sh # Follow source statements # In CI/CD find . -name "*.sh" -exec shellcheck {} + ``` **ShellCheck catches:** - Quoting issues - Bashisms in POSIX scripts - Common logic errors - Security vulnerabilities - Performance anti-patterns --- ## Platform-Specific Considerations ### Windows (Git Bash) Path Conversion - CRITICAL **ESSENTIAL KNOWLEDGE:** Git Bash/MINGW automatically converts Unix-style paths to Windows paths. This is the most common source of cross-platform scripting errors on Windows. **Complete Guide:** See `references/windows-git-bash-paths.md` for comprehensive documentation. **Quick Reference:** ```bash # Automatic conversion happens for: /foo → C:/Program Files/Git/usr/foo --dir=/tmp → --dir=C:/msys64/tmp # Disable conversion when needed MSYS_NO_PATHCONV=1 command /path/that/should/not/convert # Manual conversion with cygpath unix_path=$(cygpath -u "C:\Windows\System32") # Windows to Unix win_path=$(cygpath -w "/c/Users/username") # Unix to Windows # Shell detection (fastest method) if [[ "$OSTYPE" == "msys" ]] || [[ "$OSTYPE" == "mingw"* ]]; then echo "Git Bash detected" # Use path conversion fi # Or check $MSYSTEM variable (Git Bash/MSYS2 specific) case "${MSYSTEM:-}" in MINGW64|MINGW32|MSYS) echo "MSYS2/Git Bash environment: $MSYSTEM" ;; esac ``` **Common Issues:** ```bash # Problem: Flags converted to paths command /e /s # /e becomes C:/Program Files/Git/e # Solution: Use double slashes or dashes command //e //s # OR: command -e -s # Problem: Spaces in paths cd C:\Program Files\Git # Fails # Solution: Quote paths cd "C:\Program Files\Git" # OR: cd /c/Program\ Files/Git ``` ### Linux **Primary target for most bash scripts:** ```bash # Linux-specific features available /proc filesystem systemd integration Linux-specific commands (apt, yum, systemctl) # Check for Linux if [[ "$OSTYPE" == "linux-gnu"* ]]; then # Linux-specific code fi ``` ### macOS **BSD-based utilities (different from GNU):** ```bash # macOS differences sed -i '' # macOS requires empty string sed -i # Linux doesn't need it # Use ggrep, gsed, etc. for GNU versions if command -v gsed &> /dev/null; then SED=gsed else SED=sed fi # Check for macOS if [[ "$OSTYPE" == "darwin"* ]]; then # macOS-specific code fi ``` ### Windows (Git Bash / WSL) **Git Bash limitations:** ```bash # Available in Git Bash: - Most core utils - File operations - Process management (limited) # NOT available: - systemd - Some signals (SIGHUP behavior differs) - /proc filesystem - Native Windows path handling issues # Path handling # Git Bash uses Unix paths: /c/Users/... # Convert if needed: winpath=$(cygpath -w "$unixpath") # Unix → Windows unixpath=$(cygpath -u "$winpath") # Windows → Unix # Check for Git Bash if [[ "$OSTYPE" == "msys" ]] || [[ "$OSTYPE" == "cygwin" ]]; then # Git Bash / Cygwin code fi ``` **WSL (Windows Subsystem for Linux):** ```bash # WSL is essentially Linux, but: # - Can access Windows filesystem at /mnt/c/ # - Some syscalls behave differently # - Network configuration differs # Check for WSL if grep -qi microsoft /proc/version 2>/dev/null; then # WSL-specific code fi ``` ### Containers (Docker/Kubernetes) **Container-aware scripting:** ```bash # Minimal base images may not have bash # Use #!/bin/sh or install bash explicitly # Container detection if [ -f /.dockerenv ] || grep -q docker /proc/1/cgroup 2>/dev/null; then # Running in Docker fi # Kubernetes detection if [ -n "$KUBERNETES_SERVICE_HOST" ]; then # Running in Kubernetes fi # Best practices: # - Minimize dependencies # - Use absolute paths or PATH # - Don't assume user/group existence # - Handle signals properly (PID 1 issues) ``` ### Cross-Platform Template ```bash #!/usr/bin/env bash set -euo pipefail # Detect platform detect_platform() { case "$OSTYPE" in linux-gnu*) echo "linux" ;; darwin*) echo "macos" ;; msys*|cygwin*) echo "windows" ;; *) echo "unknown" ;; esac } PLATFORM=$(detect_platform) # Platform-specific paths case "$PLATFORM" in linux) SED=sed ;; macos) SED=$(command -v gsed || echo sed) ;; windows) # Git Bash specifics ;; esac ``` --- ## Best Practices ### Function Design ```bash # Good function structure function_name() { # 1. Local variables first local arg1="$1" local arg2="${2:-default_value}" local result="" # 2. Input validation if [[ -z "$arg1" ]]; then echo "Error: arg1 is required" >&2 return 1 fi # 3. Main logic result=$(some_operation "$arg1" "$arg2") # 4. Output/return echo "$result" return 0 } # Use functions, not scripts-in-scripts # Benefits: testability, reusability, namespacing ``` ### Variable Naming ```bash # Constants: UPPER_CASE readonly MAX_RETRIES=3 readonly CONFIG_FILE="/etc/app/config.conf" # Global variables: UPPER_CASE or lower_case (be consistent) GLOBAL_STATE="initialized" # Local variables: lower_case local user_name="john" local file_count=0 # Environment variables: UPPER_CASE (by convention) export DATABASE_URL="postgres://..." # Readonly when possible readonly SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" ``` ### Error Handling ```bash # Method 1: Check exit codes explicitly if ! command_that_might_fail; then echo "Error: Command failed" >&2 return 1 fi # Method 2: Use || for alternative actions command_that_might_fail || { echo "Error: Command failed" >&2 return 1 } # Method 3: Trap for cleanup cleanup() { local exit_code=$? # Cleanup operations rm -f "$TEMP_FILE" exit "$exit_code" } trap cleanup EXIT # Method 4: Custom error handler error_exit() { local message="$1" local code="${2:-1}" echo "Error: $message" >&2 exit "$code" } # Usage [[ -f "$config_file" ]] || error_exit "Config file not found: $config_file" ``` ### Input Validation ```bash validate_input() { local input="$1" # Check if empty if [[ -z "$input" ]]; then echo "Error: Input cannot be empty" >&2 return 1 fi