Instruction tracing with x64dbg.
x64dbg-tracedump.py
github.com/mrexodia/dumpulator/blob/main/tests/x64dbg-tracedump.py
Execution Trace Viewer
github.com/teemu-l/execution-trace-viewer
IDA Lighthouse Plugin
https://github.com/gaasedelen/lighthouse
Dragon Dance Ghidra Trace Plugin
https://github.com/0ffffffffh/dragondance
Lab 3 - Instruction Tracing
In this lab we use instruction tracing to dig deeper than just APIs and identify when direct syscalls are made from user code.
The lab3.exe binary is 64-bit and implements the same NtQueryAttributesFile file check as the previous binaries, however it uses a direct syscall. Though direct syscalls are often avoided due to their easy of detection by EDR/AV (when obfuscated) they are non-trivial to identify using the previous techniques we have discussed. In this lab we will us instruction tracing to identify and intercept the direct syscall.
Open lab3.exe in x64dbg and run until the binary entrypoint.
Observe that there is a call into the obfuscated VMP code from the main function which returns a value that is checked for the pass/fail condition. We want to trace this function.
Place a breakpoint on the instruction after the call, and run until IP is at the call instruction.
Navigate to the Trace tab, right-click in the trace window and select Start recording.
Click on the Tracing menu and select Trace into…
Change the Maximum trace count to a large number (500000000) and click ok.
The trace will stop when it has hit your breakpoint, this may take some time.
When the trace has complete you have two options,
You can close the recoding and convert the trace file into a text file using https://github.com/mrexodia/dumpulator/blob/main/tests/x64dbg-tracedump.py
Or, you can highlight the trace text in the trace window, right-click and select Copy, then Selection to file and save the highlighted text to a file.
Once you have the trace file in a format you can search, open it in your favourite text editor and search for syscall and note the instruction address.
In x64dbg place a breakpoint on the syscall address and re-run the binary.
When the syscall breakpoint is hit, use the x64dbg command syscall.name(rax) to lookup the syscall number in RAX. What is the call?
Use x64dbg to retrieve the arguments for the call, what is the file path that is checked?