vmprotect

# VMProtect Deobfuscation & Unpacking Comprehensive reference for identifying, analyzing, and deobfuscating VMProtect-protected binaries. VMProtect (VMP) is a commercial software protection tool that uses code virtualization, mutation, packing, and anti-debug to hinder reverse engineering. > **Versions covered:** VMProtect 2.x and 3.x (including 3.5+). Techniques differ between versions — always identify the version first. > **Parent skill:** [../SKILL.md](../SKILL.md) — generic deobfuscation framework and reusable techniques (mprotect monitoring, API-level string capture, section dumping, deduplication). --- ## 1. Detection Heuristics Before attempting deobfuscation, confirm the target is VMProtect-protected. These heuristics range from trivial to behavioral. ### 1.1 Section Names VMProtect adds characteristic sections to the binary. This is the fastest indicator. ``` .vmp0 — packed/virtualized code (primary VMP section) .vmp1 — additional VMP section (second layer or data) .vmp2 — rare, seen in heavily protected binaries .VMP — older VMProtect 2.x naming UPX0/UPX1 — VMProtect sometimes mimics UPX section names as a decoy ``` **Detection with radare2:** ```bash iS~vmp # list sections matching "vmp" iS~.vmp # stricter match iS~VMP ``` **Detection with rabin2:** ```bash rabin2 -S binary | grep -i vmp ``` ### 1.2 Entry Point Analysis VMProtect-packed binaries have a distinctive entry point pattern: - **VMP 3.x:** Entry point jumps into a `.vmp0` section, not `.text` - **VMP 2.x:** Entry point contains a `push` of all registers followed by a jump to the VM dispatcher - The original entry point (OEP) is virtualized or redirected ```bash # Check if EP lands in a VMP section ie # show entry point iS # compare EP address against section ranges s entry0; pd 20 # disassemble first 20 instructions at EP ``` **Red flags at entry point:** - `pushfd` / `pushad` or `push` of many registers as very first instructions - Immediate jump to a high/unusual address outside `.text` - `call` to a location that computes the next address dynamically ### 1.3 VM Handler Dispatch Pattern The core of VMProtect is a **bytecode interpreter** (virtual machine). Look for this dispatch loop pattern: ``` ; Typical VMP3 dispatch loop (x86-64) mov reg1, [rsi] ; fetch VM opcode from bytecode stream add rsi, 4 ; advance VM instruction pointer lea reg2, [rip + table] ; handler table base mov reg3, [reg2 + reg1*8] ; lookup handler address jmp reg3 ; dispatch to handler ; Alternative pattern (computed jump) movzx eax, byte [rbx] inc rbx jmp qword [rdi + rax*8] ``` **Heuristic signals:** - A single function with hundreds or thousands of basic blocks - Very high cyclomatic complexity (>100) in a single function - Repeated `jmp reg` or `jmp [reg + reg*8]` patterns (indirect jumps) - A large jump table (>50 entries) used as a handler dispatch ```bash # Find functions with abnormally many basic blocks aflj | jq '.[] | select(.nbbs > 200) | {name, addr: .offset, blocks: .nbbs}' # Search for indirect jump patterns (handler dispatch) /c jmp rax /c jmp qword [rdi /c jmp qword [rbx ``` ### 1.4 Import Table Anomalies VMProtect wraps or virtualizes imports: - **No imports or very few imports** visible in the IAT — VMProtect resolves them at runtime via `GetProcAddress`/`dlsym` - Import thunks jump through VMP stubs instead of directly to the API - `GetProcAddress`, `LoadLibraryA/W` are among the few visible imports (used by VMP's own resolver) ```bash ii # list imports — suspiciously few? ii~GetProc # VMP always needs these ii~LoadLib ``` ### 1.5 dlsym Mass-Resolution Pattern (Android/Linux) On Android, VMP-protected `.so` files resolve **all** their imports dynamically via `dlsym` at startup. This produces a distinctive burst of 100-300+ `dlsym` calls immediately after the library loads — covering everything from `malloc` and `memcpy` to OpenGL functions and JNI helpers. This pattern is a strong behavioral indicator: normal libraries use the ELF dynamic linker for most imports and only `dlsym` a handful of optional symbols. Hook `dlsym` and count calls originating from the target module — if you see 100+ distinct symbol resolutions at load time, VMP is almost certainly present: ```javascript var dlsymCount = 0; Interceptor.attach(Module.getGlobalExportByName('dlsym'), { onEnter(args) { if (!isFromTarget(this.returnAddress)) return; var name = args[1].readCString(); if (name) { dlsymCount++; console.log('[dlsym #' + dlsymCount + '] ' + name); } } }); ``` ### 1.6 String Obfuscation Indicators VMProtect encrypts strings referenced by virtualized code: - `iz` (data section strings) returns very few meaningful strings - `izz` (all strings) may reveal encrypted blobs or base64-like data - API name strings are absent (resolved dynamically) - String references (`axt`) point into VMP sections, not `.text` ```bash iz # meaningful strings are missing izz~flag # try to find something izz | wc -l # compare count — VMP binaries have far fewer readable strings ``` ### 1.7 Code Mutation Indicators Mutated (but not virtualized) functions show: - **Junk instructions:** meaningless arithmetic that cancels out (e.g., `add rax, 5; sub rax, 5`) - **Opaque predicates:** conditional jumps where the condition is always true/false - **Register shuffling:** excessive `mov reg, reg` chains - **Constant unfolding:** simple constants split into multi-step computations - **Dead stores:** writes to registers/memory immediately overwritten ```bash # Look for mutation patterns — high instruction count, low semantic density pdf @ fcn.addr | grep -c "nop\|xchg\|stc\|clc\|cmc" ``` ### 1.8 Anti-Debug and Anti-Tamper Signatures VMProtect includes runtime checks: - Calls to `IsDebuggerPresent`, `NtQueryInformationProcess`, `CheckRemoteDebuggerPresent` - `RDTSC` timing checks (measure time between two points) - `INT 2D` / `INT 3` exceptions used as anti-debug - CRC/hash checks over code sections (anti-tamper) - TLS callbacks that run before `main()` - On Android/Linux: reads `/proc/self/status` and checks `TracerPid` field ```bash ii~Debugger ii~NtQuery /c rdtsc /c int 0x2d /c int 3 iS~.tls # TLS section present? ``` ### 1.9 mprotect Storm at Startup (Android/Linux) VMP-protected `.so` files call `mprotect` repeatedly during initialization to: 1. Make packed sections writable (`r-x` → `rwx`) 2. Unpack/decrypt code into those sections 3. Re-protect them (`rwx` → `r-x`) Hook `mprotect` and filter for calls targeting the library's address range. Seeing 5-20+ `mprotect` calls on a single library at load time is a strong VMP indicator (normal libraries make 0-1 calls). See the [parent skill](../SKILL.md#33-mprotect-monitoring) for the hook template. ### 1.10 uncompress / zlib Usage During Unpacking VMProtect uses zlib's `uncompress` function to decompress packed code sections at runtime. If you see `uncompress` calls with output landing inside the target library's memory range right after load, this confirms VMP packing. ```javascript Interceptor.attach(Module.getGlobalExportByName('uncompress'), { onEnter(args) { this.dest = args[0]; this.srcLen = args[3].toUInt32(); }, onLeave(retval) { if (retval.toInt32() !== 0) return; if (this.dest.compare(targetBase) >= 0 && this.dest.compare(targetEnd) < 0) { console.log('[!] uncompress wrote into target module @ ' + this.dest); } } }); ``` ### 1.11 Composite Detection Script (radare2) Run this sequence to get a confidence score: ```bash # 1. Section check iS~vmp # 2. Entry point check s entry0; pd 5 # 3. Import poverty check ii | wc -l # < 10 imports on a complex binary = suspicious # 4. String poverty check iz | wc -l # very few strings for a large binary # 5. Indirect jump frequency /c jmp rax; /c jmp rbx; /c jmp rcx # many hits = VM dispatch # 6. Anti-debug imports ii~Debugger; ii~NtQuery; ii~CheckRemote # 7. Large functions aflj | jq '[.[] | select(.nbbs > 100)] | length' ``` **Scoring:** - `.vmp` sections found → **confirmed VMProtect** - 3+ other heuristics match → **highly likely VMProtect** - 1–2 heuristics match → **possibly VMProtect or other virtualizer** (check for Themida/Code Virtualizer patterns) **Behavioral confirmation (requires running the binary):** - dlsym mass-resolution (100+ calls at startup) → **confirmed VMP import obfuscation** - mprotect storm (5+ calls on the library at load) → **confirmed VMP packing** - uncompress writing into the library → **confirmed VMP compression** - `/proc/self/status` TracerPid check → **confirmed VMP anti-debug** --- ## 2. VMProtect Architecture Understanding the VM is essential for deobfuscation. ### 2.1 VM Components ``` +------------------+ +----------------+ +------------------+ | Bytecode Stream | --> | VM Dispatcher | --> | Handler Table | | (encrypted ops) | | (fetch-decode) | | (native stubs) | +------------------+ +----------------+ +------------------+ | +-----+-----+ | | +---------+ +---------+ | VM Stack| | VM Regs | | (RSP- | | (mapped | | based) | | to mem) | +---------+ +---------+ ``` - **Bytecode stream:** Encrypted VM opcodes stored in `.vmp0`/`.vmp1` - **Dispatcher:** Fetch-decode-execute loop; fetches next opcode, decrypts, jumps to handler - **Handler table:** Array of native code snippets, each implementing one VM opcode - **VM stack:** Separate stack area, often pointed to by a repurposed general register - **VM context/registers:** Virtual registers stored in memory, not mapped 1:1 to CPU registers ### 2.2 VMP Opcode Classes VMProtect VMs implement these broad opcode categories: | Category | Examples | Purpose | |----------|----------|---------| | Stack ops | vPush, vPop | Move data on/off VM stack | | Arithmetic | vAdd, vSub, vMul, vDiv, vNor | ALU operations (NOR is used to build AND/OR/XOR/NOT) | | Memory | vLoad, vStore | Read/write native memory | | Control flow | vJmp, vCall, vRet | Branch, call native, return | | Flags | vPushFlags, vPopFlags | Manage EFLAGS/RFLAGS | | Context | vReadReg, vWriteReg | Access native CPU registers | | Crypto | vDecrypt, vRotate | Decrypt next bytecode chunk | ### 2.3 Version Differences | Feature | VMP 2.x | VMP 3.x | |---------|---------|---------| | Section names | `.VMP0`, `.VMP1` | `.vmp0`, `.vmp1` | | Handler encoding | Direct handler addresses | Delta-encoded / encrypted handler offsets | | Bytecode encryption | Simple XOR/ADD rolling key | Multi-layer, key-dependent decryption per opcode | | VM entry | `pushad; pushfd; call vm_entry` | More varied, often via `call [rip+disp]` | | Handler count | ~30-50 | ~40-80 (more granular) | | Mutation | Light | Heavy mutation + MBA (mixed boolean arithmetic) | | Anti-debug | Basic (`IsDebuggerPresent`) | Layered (timing, exception, TLS, CRC, TracerPid) | --- ## 3. String Deobfuscation VMProtect encrypts strings referenced by protected functions. The most effective approach is to **hook the consumers** of decrypted strings rather than reversing the decryption algorithm. ### 3.1 Strategy: Hook API Consumers (Proven Approach) Decrypted strings must eventually flow through standard APIs — JNI calls, libc string functions, `dlsym`, logging. By hooking these APIs and filtering for calls originating from the target module, you capture all decrypted strings without needing to understand the decryption algorithm. This approach was proven effective on VMProtect-protected Android `.so` files, recovering 2000+ unique strings including class names, method signatures, dynamically resolved API names, and file paths. ### 3.2 JNI Vtable Hooking (Android) The most valuable source of strings on Android. Hook JNI functions by their vtable index to capture all strings crossing the Java-native boundary: ```javascript Java.perform(function () { var env = Java.vm.getEnv(); var vtable = env.handle.readPointer(); // NewStringUTF — native code creating Java strings Interceptor.attach(vtable.add(167 * Process.pointerSize).readPointer(), { onEnter: function (args) { var s = args[1].readUtf8String(); if (s) logString('JNI:NewStringUTF', s); } }); // GetStringUTFChars — native code reading Java strings Interceptor.attach(vtable.add(169 * Process.pointerSize).readPointer(), { onEnter: function (args) { this.jstr = args[1]; }, onLeave: function (retval) { var s = retval.readUtf8String(); if (s) logString('JNI:GetStringUTFChars', s); } }); // FindClass — class names being loaded Interceptor.attach(vtable.add(6 * Process.pointerSize).readPointer(), { onEnter: function (args) { var s = args[1].readCString(); if (s) logString('JNI:FindClass', s); } }); // GetMethodID — method lookups (name + signature) Interceptor.attach(vtable.add(33 * Process.pointerSize).readPointer(), { onEnter: function (args) { var name = args[2].readCString(); var sig = args[3].readCString(); if (name) logString('JNI:GetMethodID', name, sig); } }); // GetFieldID — field lookups Interceptor.attach(vtable.add(36 * Process.pointerSize).readPointer(), { onEnter: function (args) { var name = args[2].readCString();