Parallel execution with xargs, GNU parallel, and batch processing patterns. Use when user mentions "xargs", "parallel", "batch processing", "run in parallel", "parallel execution", "process list of files", "bulk operations", "concurrent commands", "map over files", or running commands on multiple inputs.
Instalar com Codex ou Claude Copie este prompt, cole no Codex, Claude ou outro assistente e deixe que ele revise a página da skill e instale para você.
Um comando direto ignora o prompt de revisão. Verifique a origem antes de executá-lo.
Parallel execution with xargs, GNU parallel, and batch processing patterns. Use when user mentions "xargs", "parallel", "batch processing", "run in parallel", "parallel execution", "process list of files", "bulk operations", "concurrent commands", "map over files", or running commands on multiple inputs.
xargs and Parallel Execution
xargs Basics
Read from stdin and pass as arguments to a command:
# Basic usage: pass stdin lines as argumentsecho"file1.txt file2.txt" | xargs rm# -I {} sets a placeholder for each input linecat urls.txt | xargs -I {} curl -O {}
# -n controls how many arguments per command invocationecho"a b c d e f" | xargs -n 2 echo# Output:# a b# c d# e f# -t prints each command before executing (trace mode)ls *.log | xargs -t rm# Read arguments from a file
xargs -a filelist.txt rm
xargs with find
Always use -print0 / -0 to handle filenames with spaces and special characters:
# Preview what would be deleted
find . -name "*.bak" -print0 | xargs -0 echorm# Use -p to prompt before each execution
find . -name "*.tmp" -print0 | xargs -0 -p rm# With -t to trace commands as they run
find . -name "*.log" -print0 | xargs -0 -t gzip
Error Handling
# xargs exits with 123 if any command fails
find . -name "*.sh" -print0 | xargs -0 -P 4 bash # check $?# GNU parallel: halt on first failurecat jobs.txt | parallel --halt now,fail=1 process_job {}
# GNU parallel: halt when 20% of jobs failcat jobs.txt | parallel --halt soon,fail=20% process_job {}
# Capture per-job exit codes with GNU parallelcat jobs.txt | parallel --joblog joblog.txt process_job {}
# joblog.txt contains exit status for every job
xargs vs for Loops vs while read
Use xargs when:
Processing output from find or another command
You want built-in parallelism (-P)
Batching multiple arguments per invocation (-n)
Use while read when:
You need complex logic per iteration (if/else, multiple commands)
The loop body uses shell variables that must persist across iterations
Use for loops when:
Iterating over a known, small list of items
Glob expansion is sufficient (for f in *.txt)
Readability matters more than performance
# for loop -- simple, readable, no parallelismfor f in *.txt; dowc -l "$f"; done# while read -- complex logic per item
find . -name "*.csv" | whileread -r f; do
count=$(wc -l < "$f")
[ "$count" -gt 1000 ] && echo"Large: $f ($count lines)"done# xargs -- fast, parallel, concise
find . -name "*.csv" -print0 | xargs -0 -P 4 wc -l
Resource-Aware Parallelism
# Use nproc to match available CPU cores
find . -name "*.gz" -print0 | xargs -0 -P "$(nproc)" gunzip
# Use half the cores to leave room for other work
find . -name "*.log" -print0 | xargs -0 -P "$(( $(nproc) / 2 ))" gzip
# GNU parallel: percentage-based, relative, or load-based limits
parallel -j 50% gzip ::: *.log# 50% of cores
parallel -j -2 gzip ::: *.log# cores minus 2
parallel --load 80% process_job ::: * # limit by load average# Limit concurrency for I/O-bound tasks (network, disk)cat urls.txt | xargs -P 5 -I {} curl -sO {}
# Monitor parallel job resource usage
parallel --joblog jobs.log -j 4 heavy_task ::: input_* && column -t jobs.log
