deep-debug

name	deep-debug
description	Tactical toolkit for bug-hunting and forensics-style tasks — gdb, strace, hex/base64/shellcode decoding, log triage, targeted regex. Use when the task involves crashes, races, encoded payloads, or tracing execution.

Deep Debug

Reach for this when the task's failure is non-obvious: a segfault, a race, a hanging process, encoded or obfuscated data, a log file hiding a needle. The tools below are the ones that actually move the needle; everything else is a distraction.

Native crashes (segfaults, SIGABRT, unexpected exits)

# Interactive gdb session on a crashing binary
gdb -q ./a.out
(gdb) run <args>
(gdb) bt                    # backtrace after crash
(gdb) frame <n>             # jump to a frame
(gdb) info locals           # inspect variables
(gdb) list                  # source at current line

Non-interactive, script-friendly:

gdb -batch -ex run -ex bt --args ./a.out <arg1> <arg2>

For core dumps: make sure they are enabled (ulimit -c unlimited), run the program, then gdb ./a.out core.

If there's no debuginfo, rebuild with -g -O0 first if the task permits; otherwise inspect with objdump -d, readelf, nm.

Tracing syscalls and library calls

strace -f -o trace.log ./a.out <args>        # all syscalls from the process tree
strace -e openat,read -f ./a.out <args>      # filter to a syscall subset
ltrace ./a.out <args>                        # dynamic library calls

strace is the fastest way to answer "what file / socket / env var is this process actually touching?" Read the last few hundred lines of the trace near the failure — the call that returned -1 is usually your lead.

Reverse-engineering encoded payloads

Terminal-Bench has a lot of CTF-flavored tasks. Quick decoders:

echo "SGVsbG8=" | base64 -d
echo "48656c6c6f" | xxd -r -p
python3 -c "import urllib.parse as u; print(u.unquote('%48%65%6c'))"

# Auto-detect + try common decodings
python3 -c "
import base64, binascii, sys
s = sys.stdin.read().strip()
for name, fn in [('b64', base64.b64decode), ('hex', bytes.fromhex)]:
    try: print(name, '→', fn(s)[:200]); 
    except Exception as e: pass
"

For shellcode / ARM / x86 binaries: objdump -d -M intel, radare2, r2 -AA <bin> for auto-analysis, strings <bin> | head.

For compressed or layered blobs: file <blob> first — it often names the exact format. Then the matching tool (xxd, gunzip, unzip, tar, openssl).

Log triage at scale

Ten-MB+ log file? Don't cat; filter.

grep -iE "error|fail|panic|traceback" app.log | head -50
awk '$9 >= 500' access.log | sort | uniq -c | sort -rn | head   # HTTP 5xx triage
tail -f app.log &                                              # stream during a run

For Apache / nginx access logs, the 9th field is usually status; use awk to histogram status codes, slice by IP, or find spikes. For DDoS-style tasks, look at request rate per IP:

awk '{print $1}' access.log | sort | uniq -c | sort -rn | head

Races and concurrency bugs

Races are about timing, not logic; adding prints often masks them. Tactics:

• Narrow the reproduction to the smallest inputs that still flake.
• Add sleeps on one side to widen the window and confirm the race exists.
• Look for unprotected shared state — counters, file writes, dict mutations.
• Check for missing lock, missing atomic, fire-and-forget goroutines / threads / async tasks without awaits.
• For network server races, suspect the order of connect → accept → write → flush. strace the server.

Process hanging?

ps auxf                                 # who is running
cat /proc/<pid>/status                  # state (R/S/D/Z)
cat /proc/<pid>/stack                   # kernel stack — where it's stuck
ls -l /proc/<pid>/fd/                   # open files and sockets
strace -p <pid>                         # live attach

A D state means uninterruptible (usually I/O). A stuck read on an fd that no one is writing to is a classic hang; ls /proc/<pid>/fd/ tells you which fd.

Regex for targeted extraction

When a task says "find all X in file Y", resist greedy grep -r. Be surgical:

grep -oE '\b[A-Z]{2,}_[A-Z0-9_]+\b' file        # extract tokens matching a shape
grep -cE 'pattern' file                         # count, don't list

In Python for more structure:

import re, pathlib
matches = re.findall(r'(?P<key>\w+)=(?P<val>[^;]+)', pathlib.Path('x').read_text())

Stopping condition

Deep debugging is expensive on tokens. Set a turn budget in your head: "if I don't have a lead in 5 tool calls, I am probably debugging the wrong thing — step back and re-read the brief." Scaffolds with lots of cheap sub-steps burn budget fast; stay deliberate.