ctf-reverse

# CTF Reverse Engineering Quick reference for RE challenges. For detailed techniques, see supporting files. ## Prerequisites **Python packages (all platforms):** ```bash pip install frida-tools angr qiling uncompyle6 capstone lief z3-solver ``` **Linux (apt):** ```bash apt install gdb radare2 binutils strace ltrace apktool upx ``` **macOS (Homebrew):** ```bash brew install gdb radare2 binutils apktool upx ghidra ``` **Manual install:** - pwndbg — Linux: [GitHub](https://github.com/pwndbg/pwndbg), macOS: `brew install pwndbg/tap/pwndbg-gdb` ## Additional Resources - [tools.md](tools.md) - Static analysis tools (GDB, Ghidra, radare2, IDA, Binary Ninja, dogbolt.org, RISC-V with Capstone, Unicorn emulation, Python bytecode, WASM, Android APK, .NET, packed binaries) - [tools-dynamic.md](tools-dynamic.md) (includes Intel Pin instruction-counting side channel for movfuscated binaries, opcode-only trace reconstruction) - Dynamic analysis tools: Frida (hooking, anti-debug bypass, memory scanning, Android/iOS), angr symbolic execution (path exploration, constraints, CFG), lldb (macOS/LLVM debugger), x64dbg (Windows), Qiling (cross-platform emulation with OS support), Triton (dynamic symbolic execution) - [tools-advanced.md](tools-advanced.md) - Advanced tools: VMProtect/Themida analysis, binary diffing (BinDiff, Diaphora), deobfuscation frameworks (D-810, GOOMBA, Miasm), Rizin/Cutter, RetDec, advanced GDB (Python scripting, conditional breakpoints, watchpoints, reverse debugging with rr, pwndbg/GEF), advanced Ghidra scripting, patching (Binary Ninja API, LIEF) - [anti-analysis.md](anti-analysis.md) - Comprehensive anti-analysis: Linux anti-debug (ptrace, /proc, timing, signals, direct syscalls), Windows anti-debug (PEB, NtQueryInformationProcess, heap flags, TLS callbacks, HW/SW breakpoint detection, exception-based, thread hiding), anti-VM/sandbox (CPUID, MAC, timing, artifacts, resources), anti-DBI (Frida detection/bypass), code integrity/self-hashing, anti-disassembly (opaque predicates, junk bytes), MBA identification/simplification, bypass strategies - [patterns.md](patterns.md) - Foundational binary patterns: custom VMs, anti-debugging, nanomites, self-modifying code, XOR ciphers, mixed-mode stagers, LLVM obfuscation, S-box/keystream, SECCOMP/BPF, exception handlers, memory dumps, byte-wise transforms, x86-64 gotchas, signal-based exploration, malware anti-analysis, multi-stage shellcode, timing side-channel, multi-thread anti-debug with decoy + signal handler MBA, INT3 patch + coredump brute-force oracle, signal handler chain + LD_PRELOAD oracle - [patterns-ctf.md](patterns-ctf.md) - Competition-specific patterns (Part 1): hidden emulator opcodes, LD_PRELOAD key extraction, SPN static extraction, image XOR smoothness, byte-at-a-time cipher, mathematical convergence bitmap, Windows PE XOR bitmap OCR, two-stage RC4+VM loaders, GBA ROM meet-in-the-middle, Sprague-Grundy game theory, kernel module maze solving, multi-threaded VM channels, backdoored shared library detection via string diffing, custom binfmt kernel module with RC4 flat binaries, hash-resolved imports / no-import ransomware, ELF section header corruption for anti-analysis - [patterns-ctf-2.md](patterns-ctf-2.md) - Competition-specific patterns (Part 2): multi-layer self-decrypting brute-force, embedded ZIP+XOR license, stack string deobfuscation, prefix hash brute-force, CVP/LLL lattice for integer validation, decision tree function obfuscation, GF(2^8) Gaussian elimination, ROP chain obfuscation analysis (ROPfuscation) - [patterns-ctf-3.md](patterns-ctf-3.md) - Competition-specific patterns (Part 3): Z3 single-line Python circuit, sliding window popcount, keyboard LED Morse code via ioctl, C++ destructor-hidden validation, syscall side-effect memory corruption, MFC dialog event handlers, VM sequential key-chain brute-force, Burrows-Wheeler transform inversion, OpenType font ligature exploitation, GLSL shader VM with self-modifying code, instruction counter as cryptographic state - [languages.md](languages.md) - Language-specific: Python bytecode & opcode remapping, Python version-specific bytecode, Pyarmor static unpack, DOS stubs, Unity IL2CPP, HarmonyOS HAP/ABC, Brainfuck/esolangs (+ BF character-by-character static analysis, BF side-channel read count oracle, BF comparison idiom detection), UEFI, transpilation to C, code coverage side-channel, OPAL functional reversing, non-bijective substitution, FRACTRAN program inversion - [languages-platforms.md](languages-platforms.md) - Platform/framework-specific: Roblox place file analysis, Godot game asset extraction, Rust serde_json schema recovery, Android JNI RegisterNatives obfuscation, Frida Firebase Cloud Functions bypass, Verilog/hardware RE, prefix-by-prefix hash reversal, Ruby/Perl polyglot constraint satisfaction, Electron ASAR extraction + native binary analysis, Node.js npm runtime introspection - [languages-compiled.md](languages-compiled.md) - Go binary reversing (GoReSym, goroutines, memory layout, channel ops, embed.FS, Go binary UUID patching for C2 enumeration), Rust binary reversing (demangling, Option/Result, Vec, panic strings), Swift binary reversing (demangling, protocol witness tables), Kotlin/JVM (coroutine state machines), C++ (vtable reconstruction, RTTI, STL patterns) - [platforms.md](platforms.md) - Platform-specific RE: macOS/iOS (Mach-O, code signing, Objective-C runtime, Swift, dyld, jailbreak bypass), embedded/IoT firmware (binwalk, UART/JTAG/SPI extraction, ARM/MIPS, RTOS), kernel drivers (Linux .ko, eBPF, Windows .sys), game engines (Unreal Engine, Unity, anti-cheat, Lua), automotive CAN bus - [platforms-hardware.md](platforms-hardware.md) - Hardware and advanced architecture RE: HD44780 LCD controller GPIO reconstruction, RISC-V advanced (custom extensions, privileged modes, debugging), ARM64/AArch64 reversing and exploitation (calling convention, ROP gadgets, qemu-aarch64-static emulation) --- ## Problem-Solving Workflow 1. **Start with strings extraction** - many easy challenges have plaintext flags 2. **Try ltrace/strace** - dynamic analysis often reveals flags without reversing 3. **Try Frida hooking** - hook strcmp/memcmp to capture expected values without reversing 4. **Try angr** - symbolic execution solves many flag-checkers automatically 5. **Try Qiling** - emulate foreign-arch binaries or bypass heavy anti-debug without artifacts 6. **Map control flow** before modifying execution 7. **Automate manual processes** via scripting (r2pipe, Frida, angr, Python) 8. **Validate assumptions** by comparing decompiler outputs (dogbolt.org for side-by-side) ## Quick Wins (Try First!) ```bash # Plaintext flag extraction strings binary | grep -E "flag\{|CTF\{|pico" strings binary | grep -iE "flag|secret|password" rabin2 -z binary | grep -i "flag" # Dynamic analysis - often captures flag directly ltrace ./binary strace -f -s 500 ./binary # Hex dump search xxd binary | grep -i flag # Run with test inputs ./binary AAAA echo "test" | ./binary ``` ## Initial Analysis ```bash file binary # Type, architecture checksec --file=binary # Security features (for pwn) chmod +x binary # Make executable ``` ## Memory Dumping Strategy **Key insight:** Let the program compute the answer, then dump it. Break at final comparison (`b *main+OFFSET`), enter any input of correct length, then `x/s $rsi` to dump computed flag. ## Decoy Flag Detection **Pattern:** Multiple fake targets before real check. **Identification:** 1. Look for multiple comparison targets in sequence 2. Check for different success messages 3. Trace which comparison is checked LAST **Solution:** Set breakpoint at FINAL comparison, not earlier ones. ## GDB PIE Debugging PIE binaries randomize base address. Use relative breakpoints: ```bash gdb ./binary start # Forces PIE base resolution b *main+0xca # Relative to main run ``` ## Comparison Direction (Critical!) **Two patterns:** 1. `transform(flag) == stored_target` - Reverse the transform 2. `transform(stored_target) == flag` - Flag IS the transformed data! **Pattern 2 solution:** Don't reverse - just apply transform to stored target. ## Common Encryption Patterns - XOR with single byte - try all 256 values - XOR with known plaintext (`flag{`, `CTF{`) - RC4 with hardcoded key - Custom permutation + XOR - XOR with position index (`^ i` or `^ (i & 0xff)`) layered with a repeating key ## Quick Tool Reference ```bash # Radare2 r2 -d ./binary # Debug mode aaa # Analyze afl # List functions pdf @ main # Disassemble main # Ghidra (headless) analyzeHeadless project/ tmp -import binary -postScript script.py # IDA ida64 binary # Open in IDA64 ``` ## Binary Types ### Python .pyc Disassemble with `marshal.load()` + `dis.dis()`. Header: 8 bytes (2.x), 12 (3.0-3.6), 16 (3.7+). See [languages.md](languages.md#python-bytecode-reversing-disdis-output). ### WASM ```bash wasm2c checker.wasm -o checker.c gcc -O3 checker.c wasm-rt-impl.c -o checker # WASM patching (game challenges): wasm2wat main.wasm -o main.wat # Binary → text # Edit WAT: flip comparisons, change constants wat2wasm main.wat -o patched.wasm # Text → binary ``` **WASM game patching (Tac Tic Toe, Pragyan 2026):** If proof generation is independent of move quality, patch minimax (flip `i64.lt_s` → `i64.gt_s`, change bestScore sign) to make AI play badly while proofs remain valid. Invoke `/ctf-misc` for full game patching patterns (games-and-vms). ### Android APK `apktool d app.apk -o decoded/` for resources; `jadx app.apk` for Java decompilation. Check `decoded/res/values/strings.xml` for flags. See [tools.md](tools.md#android-apk). ### Flutter APK (Dart AOT) If `lib/arm64-v8a/libapp.so` + `libflutter.so` present, use [Blutter](https://github.com/worawit/blutter): `python3 blutter.py path/to/app/lib/arm64-v8a out_dir`. Outputs reconstructed Dart symbols + Frida script. See [tools.md](tools.md#flutter-apk-blutter). ### .NET - dnSpy - debugging + decompilation - ILSpy - decompiler ### Packed (UPX) ```bash upx -d packed -o unpacked ``` If unpacking fails, inspect UPX metadata first: verify UPX section names, header fields, and version markers are intact. If metadata looks tampered or uncertain, review UPX source on GitHub to identify likely modification points. ### Tauri Packed Desktop Apps Tauri embeds Brotli-compressed frontend assets in the executable. Find `index.html` xrefs to locate asset index table, dump blobs, Brotli decompress. Reference: `tauri-codegen/src/embedded_assets.rs`. ## Anti-Debugging Bypass Common checks: - `IsDebuggerPresent()` / PEB.BeingDebugged / NtQueryInformationProcess (Windows) - `ptrace(PTRACE_TRACEME)` / `/proc/self/status` TracerPid (Linux) - TLS callbacks (run before main — check PE TLS Directory) - Timing checks (`rdtsc`, `clock_gettime`, `GetTickCount`) - Hardware breakpoint detection (DR0-DR3 via GetThreadContext) - INT3 scanning / code self-hashing (CRC over .text section) - Signal-based: SIGTRAP handler, SIGALRM timeout, SIGSEGV for real logic - Frida/DBI detection: `/proc/self/maps` scan, port 27042, inline hook checks Bypass: Set breakpoint at check, modify register to bypass conditional. pwntools patch: `elf.asm(elf.symbols.ptrace, 'ret')` to replace function with immediate return. See [patterns.md](patterns.md#pwntools-binary-patching-crypto-cat). For comprehensive anti-analysis techniques and bypasses (30+ methods with code), see [anti-analysis.md](anti-analysis.md). ## S-Box / Keystream Patterns **Xorshift32:** Shifts 13, 17, 5 **Xorshift64:** Shifts 12, 25, 27 **Magic constants:** `0x2545f4914f6cdd1d`, `0x9e3779b97f4a7c15` ## Custom VM Analysis 1. Identify structure: registers, memory, IP 2. Reverse `executeIns` for opcode meanings 3. Write disassembler mapping opcodes to mnemonics 4. Often easier to bruteforce than fully reverse 5. Look for the bytecode file loaded via command-line arg See [patterns.md](patterns.md#custom-vm-reversing) for VM workflow, opcode tables, and state machine BFS. **Sequential key-chain brute-force:** When a VM validates input in small blocks (e.g., 3 bytes = 2^24 candidates) with each block's output key feeding the next, brute-force each block sequentially with OpenMP parallelization. Compile solver with `gcc -O3 -march=native -fopenmp`. See [patterns-ctf-3.md](patterns-ctf-3.md#vm-sequential-key-chain-brute-force-midnight-flag-2026). ## Python Bytecode Reversing XOR flag checkers with interleaved even/odd tables are common. See [languages.md](languages.md#python-bytecode-reversing-disdis-output) for bytecode analysis tips and reversing patterns. ## Signal-Based Binary Exploration Binary uses UNIX signals as binary tree navigation; hook `sigaction` via `LD_PRELOAD`, DFS by sending signals. See [patterns.md](patterns.md#signal-based-binary-exploration). ## Malware Anti-Analysis Bypass via Patching Flip `JNZ`/`JZ` (0x75/0x74), change sleep values, patch environment checks in Ghidra (`Ctrl+Shift+G`). See [patterns.md](patterns.md#malware-anti-analysis-bypass-via-patching). ## Expected Values Tables **Locating:** ```bash objdump -s -j .rodata binary | less # Look near comparison instructions # Size matches flag length ``` ## x86-64 Gotchas Sign extension and 32-bit truncation pitfalls. See [patterns.md](patterns.md#x86-64-gotchas) for details and code examples. ## Iterative Solver Pattern Try each byte (0-255) per position, match against expected output. **Uniform transform shortcut:** if one input byte only changes one output byte, build 0..255 mapping then invert. See [patterns.md](patterns.md) for full implementation. ## Unicorn Emulation (Complex State) `from unicorn import *` -- map segments, set up stack, hook to trace. **Mixed-mode pitfall:** 64-bit stub jumping to 32-bit via `retf` requires switching to UC_MODE_32 and copying GPRs + EFLAGS + XMM regs. See [tools.md](tools.md#unicorn-emulation). ## Multi-Stage Shellcode Loaders Nested shellcode with XOR decode loops; break at `call rax`, bypass ptrace with `set $rax=0`, extract flag from `mov` instructions. See [patterns.md](patterns.md#multi-stage-shellcode-loaders). ## Timing Side-Channel Attack Validation time varies per correct character; measure elapsed time per candidate to recover flag byte-by-byte. See [patterns.md](patterns.md#timing-side-channel-attack). ## Godot Game Asset Extraction Use KeyDot to extract encryption key from executable, then gdsdecomp to extract .pck package. See [languages-platforms.md](languages-platforms.md#godot-game-asset-extraction). ## Roblox Place File Analysis Query Asset Delivery API for version history; parse `.rbxlbin` chunks (INST/PROP/PRNT) to diff script sources across versions. See [languages-platforms.md](languages-platforms.md#roblox-place-file-analysis). ## Unstripped Binary Information Leaks **Pattern (Bad Opsec):** Debug info and file paths leak author identity. **Quick checks:** ```bash strings binary | grep "/home/" # Home directory paths strings binary | grep "/Users/" # macOS paths file binary # Check if stripped readelf -S binary | grep debug # Debug sections present? ``` ## Custom Mangle Function Reversing Binary mangles input 2 bytes at a time with running state; extract target from `.rodata`, write inverse function. See [patterns.md](patterns.md#custom-mangle-function-reversing). ## Rust serde_json Schema Recovery Disassemble serde `Visitor` implementations to recover expected JSON schema; field names in order reveal flag. See [languages-platforms.md](languages-platforms.md#rust-serde_json-schema-recovery). ## Position-Based Transformation Reversing Binary adds/subtracts position index; reverse by undoing per-index offset. See [patterns.md](patterns.md#position-based-transformation-reversing). ## Hex-Encoded String Comparison Input converted to hex, compared against constant. Decode with `xxd -r -p`. See [patterns.md](patterns.md#hex-encoded-string-comparison). ## Embedded ZIP + XOR License Decryption Binary with named symbols (`EMBEDDED_ZIP`, `ENCRYPTED_MESSAGE`) in `.rodata` → extract ZIP containing license, XOR encrypted message with license bytes to recover flag. No execution needed. See [patterns-ctf-2.md](patterns-ctf-2.md#embedded-zip--xor-license-decryption-metactf-2026).