JSON and YAML processing with jq and yq command-line tools. Use when user asks to "parse JSON", "transform YAML", "extract from JSON", "filter JSON array", "convert YAML to JSON", "query JSON", "jq filter", "yq select", "process API response", "parse logs as JSON", "merge YAML files", "convert JSON to CSV", "stream large JSON", "jq reduce", "group_by", "unique_by", or manipulate structured data from the command line.
JSON and YAML processing with jq and yq command-line tools. Use when user asks to "parse JSON", "transform YAML", "extract from JSON", "filter JSON array", "convert YAML to JSON", "query JSON", "jq filter", "yq select", "process API response", "parse logs as JSON", "merge YAML files", "convert JSON to CSV", "stream large JSON", "jq reduce", "group_by", "unique_by", or manipulate structured data from the command line.
jq & yq
Command-line JSON and YAML processing.
jq Fundamentals
Extract Values
# Get field / nested fieldecho'{"name":"John","age":30}' | jq '.name'echo'{"user":{"name":"John"}}' | jq '.user.name'# Array elementecho'[1,2,3]' | jq '.[0]'# Multiple fieldsecho'{"a":1,"b":2,"c":3}' | jq '{a,b}'# Optional field access (no error if missing)echo'{"a":1}' | jq '.b?'
# Sum valuesecho'[1,2,3,4,5]' | jq 'add'# Reduce: accumulate a resultecho'[1,2,3,4,5]' | jq 'reduce .[] as $x (0; . + $x)'# Build object from arrayecho'[{"k":"a","v":1},{"k":"b","v":2}]' | jq 'reduce .[] as $item ({}; . + {($item.k): $item.v})'# Group and aggregateecho'[{"dept":"eng","sal":100},{"dept":"eng","sal":120},{"dept":"sales","sal":90}]' | \
jq 'group_by(.dept) | map({dept: .[0].dept, total: map(.sal) | add, count: length})'
Recursive Descent
# Find all values for a key at any depthecho'{"a":{"name":"x","b":{"name":"y"}}}' | jq '.. | .name? // empty'# Find all strings / numbers in a structureecho'{"a":1,"b":"hello","c":{"d":"world"}}' | jq '[.. | strings]'echo'{"a":1,"b":{"c":2},"d":[3,4]}' | jq '[.. | numbers]'
Output Flags and Format Encoders
# Raw output (no quotes) / compact outputecho'{"name":"John"}' | jq -r '.name'echo'{"a":1}' | jq -c '.'# Slurp multiple values into arraycat lines.json | jq -s '.'# Raw input: treat each line as a stringecho -e "line1\nline2" | jq -R '.'# Null input: generate JSON without input
jq -n '{created: now | todate}'# TSV / CSV / URI encodingecho'[{"n":"a","v":1},{"n":"b","v":2}]' | jq -r '.[] | [.n, .v] | @tsv'echo'[{"n":"a","v":1},{"n":"b","v":2}]' | jq -r '.[] | [.n, .v] | @csv'echo'{"q":"hello world"}' | jq -r '.q | @uri'
CSV/TSV Conversion
# JSON array of objects to CSV with headerecho'[{"name":"a","age":1},{"name":"b","age":2}]' | \
jq -r '(.[0] | keys_unsorted) as $k | $k, (.[] | [.[$k[]]])| @csv'# CSV to JSON (line-by-line)cat data.csv | jq -R 'split(",") | {id: .[0], name: .[1], value: .[2]}'
# Deep merge (second overrides first)
yq ea 'select(fileIndex == 0) * select(fileIndex == 1)' base.yaml override.yaml
# Merge all documents into one
yq ea '. as $item ireduce ({}; . * $item)' *.yaml
Convert Between Formats
yq -o=json file.yaml # YAML to JSON
yq -P file.json # JSON to YAML
yq -o=xml file.yaml # YAML to XML
yq -o=props file.yaml # YAML to properties
curl -s https://api.example.com/data | yq -P # pipe JSON to YAML