file binary # Type, architecture
checksec --file=binary # Security features (for pwn)chmod +x binary # Make executable
Memory Dumping Strategy
Key insight: Let the program compute the answer, then dump it.
gdb ./binary
start
b *main+0x198 # Break at final comparison
run
# Enter any input of correct length
x/s $rsi# Dump computed flag
x/38c $rsi# As characters
Decoy Flag Detection
Pattern: Multiple fake targets before real check.
Identification:
Look for multiple comparison targets in sequence
Check for different success messages
Trace which comparison is checked LAST
Solution: Set breakpoint at FINAL comparison, not earlier ones.
GDB PIE Debugging
PIE binaries randomize base address. Use relative breakpoints:
gdb ./binary
start # Forces PIE base resolution
b *main+0xca # Relative to main
run
Comparison Direction (Critical!)
Two patterns:
transform(flag) == stored_target - Reverse the transform
transform(stored_target) == flag - Flag IS the transformed data!
Pattern 2 solution: Don't reverse - just apply transform to stored target.
Common Encryption Patterns
XOR with single byte - try all 256 values
XOR with known plaintext (flag{, CTF{)
RC4 with hardcoded key
Custom permutation + XOR
XOR with position index (^ i or ^ (i & 0xff)) layered with a repeating key
Quick Tool Reference
# Radare2
r2 -d ./binary # Debug mode
aaa # Analyze
afl # List functions
pdf @ main # Disassemble main# Ghidra (headless)
analyzeHeadless project/ tmp -import binary -postScript script.py
# IDA
ida64 binary # Open in IDA64
Pattern (Carrot): Malware with multiple environment checks before executing payload.
Common checks to patch:
Check
Technique
Patch
ptrace(PTRACE_TRACEME)
Anti-debug
Change cmp -1 to cmp 0
sleep(150)
Anti-sandbox timing
Change sleep value to 1
/proc/cpuinfo "hypervisor"
Anti-VM
Flip JNZ to JZ
"VMware"/"VirtualBox" strings
Anti-VM
Flip JNZ to JZ
getpwuid username check
Environment
Flip comparison
LD_PRELOAD check
Anti-hook
Skip check
Fan count / hardware check
Anti-VM
Flip JLE to JGE
Hostname check
Environment
Flip JNZ to JZ
Ghidra patching workflow:
Find check function, identify the conditional jump
Click on instruction → Ctrl+Shift+G → modify opcode
For JNZ (0x75) → JZ (0x74), or vice versa
For immediate values: change operand bytes directly
Export: press O → choose "Original File" format
chmod +x the patched binary
Server-side validation bypass:
If patched binary sends system info to remote server, patch the data too
Modify string addresses in data-gathering functions
Change format strings to embed correct values directly
Expected Values Tables
Locating:
objdump -s -j .rodata binary | less
# Look near comparison instructions# Size matches flag length
x86-64 Gotchas
Sign extension:0xffffffc7 behaves differently in XOR vs addition
# For XOR: use low byte
esi_xor = esi & 0xff# For addition: use full value with overflow
result = (r13 + esi) & 0xffffffff
Iterative Solver Pattern
for pos inrange(flag_length):
for c inrange(256):
computed = compute_output(c, current_state)
if computed == EXPECTED[pos]:
flag.append(c)
update_state(c, computed)
break
Uniform transform shortcut: if changing one input byte only changes one output byte,
build a 0..255 mapping by repeating a single byte across the whole input, then invert.
Unicorn Emulation (Complex State)
from unicorn import *
from unicorn.x86_const import *
mu = Uc(UC_ARCH_X86, UC_MODE_64)
# Map segments, set up stack# Hook to trace register changes
mu.emu_start(start_addr, end_addr)
Mixed-mode pitfall: if a 64-bit stub jumps into 32-bit code via retf/retfq, you must
switch to a UC_MODE_32 emulator and copy GPRs, EFLAGS, and XMM regs; missing XMM state
will corrupt SSE-based transforms.
Multi-Stage Shellcode Loaders
Pattern (I Heard You Liked Loaders): Nested shellcode with XOR decode loops and anti-debug.
Debugging workflow:
Break at call rax in launcher, step into shellcode
Bypass ptrace anti-debug: step to syscall, set $rax=0
Step through XOR decode loop (or break on int3 if hidden)
Repeat for each stage until final payload
Flag extraction from mov instructions:
# Final stage loads flag 4 bytes at a time via mov ebx, value# Extract little-endian 4-byte chunks
values = [0x6174654d, 0x7b465443, ...] # From disassembly
flag = b''.join(v.to_bytes(4, 'little') for v in values)
Timing Side-Channel Attack
Pattern (Clock Out): Validation time varies per correct character (longer sleep on match).
Exploitation:
import time
from pwn import *
flag = ""for pos inrange(flag_length):
best_char, best_time = '', 0for c in string.printable:
io = remote(host, port)
start = time.time()
io.sendline((flag + c).ljust(total_len, 'X'))
io.recvall()
elapsed = time.time() - start
if elapsed > best_time:
best_time = elapsed
best_char = c
io.close()
flag += best_char
Godot Game Asset Extraction
Pattern (Steal the Xmas): Encrypted Godot .pck packages.