strncmp is used. If we load the program into IDA Pro, we see strncmp in the main function at 0x40123A. Let’s load the program into OllyDbg and set a breakpoint at 0x40123A.After we load Lab16-02.exe into OllyDbg, it immediately terminates without pausing the program. We suspect something is amiss, so we check the PE file structure. shows the PE header section names in PEview.
.
Double-click the TLS callback function at 0x401060 to navigate directly to the function and see if there is any anti-debugging functionality. shows the TLS callback code.
Now load the program into OllyDbg, set a breakpoint at the strncmp call at 0x40123A, and add a command-line argument of abcd in OllyDbg before clicking the play button. When you click play, the strncmp function appears to compare abcd to bzqrp@ss; however, strncmp checks only the first 4 bytes of the bzqrp@ss string. We conclude that the password must be bzqr, but if we try that password on the command line outside a debugger, we receive the incorrect password error message. We dig deeper into the code to determine if something else is going on.
We begin by properly labeling the encoded string in the listing. The second parameter passed on the stack to strncmp is byte_408030 (a global variable), which we know to be a byte array of size 4. We change this into a 4-byte array and rename it encoded_password.
Next, we see CreateThread called just before the call to strncmp in the main function. To look at the code in the thread created by this call, double-click the parameter labeled StartAddress. This function appears to be a decoding routine since it contains many logical and shift operations on encoded_password. Examining the decoding routine closely, we see the BeingDebugged flag accessed, as shown in at ❶ and ❷.
sub_401020) is called. We check the cross-reference and see that sub_401020 is called from the TLS callback, as shown in (in bold).arg_4 to the number 2. Recall from our earlier discussion that arg_4 to the TLS callback is used to determine when the TLS callback is made: 1 is used for when the process is starting up, 2 for when a thread is starting up, and 3 when the process is being terminated. Therefore, this TLS callback was called again when the CreateThread executed and caused the OutputDebugStringA to execute.To finally get the password, we start with our OllyDbg PhantOm plug-in installed and set up to protect us from the BeingDebugged flag check and the FindWindow check. We load the program into OllyDbg, NOP-out the add instruction at 0x401051, and set a breakpoint at the strncmp call (0x40123A). This time, the password appears to be byrr. Trying this on the command line, we get the following message:
You entered the correct password!