json-pipe

name	json-pipe
description	Pipe JSON between shell commands using jq-lite for transformation steps. Use when building shell pipelines that process JSON data through multiple transformation stages. Triggers include "pipe JSON", "shell pipeline JSON", "curl | jql", "transform JSON in pipeline", "chain JSON operations".

json-pipe

Pattern guide for chaining jql in shell pipelines.

Common Pipeline Patterns

curl -> extract -> use

# Get GitHub stars
curl -s https://api.github.com/repos/nodejs/node | jql -r '.stargazers_count'

# Get all repo names
curl -s https://api.github.com/users/octocat/repos | jql -r '.[].name'

# Get environment variable from k8s secret
kubectl get secret myapp-env -o json | jql -r '.data.DATABASE_URL' | base64 -d

filter -> format -> save

cat events.json | jql '.events[] | select(.level == "error")' > errors.json
cat users.json | jql --csv '.users' > users.csv
cat config.json | jql --compact '.' > config.min.json

transform -> pipe to next tool

# Extract emails, send notifications
cat users.json | jql -r '.users[] | select(.active) | .email' | xargs -I{} mail -s "Update" {}

# Get running container IDs
docker ps -q | xargs docker inspect | jql -r '.[].Name' | sed 's/^\///'

NDJSON log processing

# Count errors in log
cat app.log | jql 'select(.level == "error")' | wc -l

# Extract error messages
cat app.log | jql -r 'select(.level == "error") | .msg'

# Top error types
cat app.log | jql 'select(.level == "error") | .type' | sort | uniq -c | sort -rn

Tips

• Use -r (raw) when passing string values to other shell commands.
• Use -c (compact) when storing JSON in shell variables: X=$(cat f.json | jql -c '.field')
• NDJSON (one JSON per line) is processed line by line - safe for large log files.
• Errors go to stderr; safe to pipe stdout to other commands even on parse errors.