Reverse Engineering a password protected file + ptrace() Anti-Debugging bypass.

So I have run into a lot of anti-debugging techniques over the years and wanted to show how to bypass one of the simpler ones to start off with. Maybe in another post I will show how to bypass more "difficult" techniques if anyone requests it.

hmmm... Prerequisites:

* Linux OS
* gdb
* objdump
* x86 Assembly knoweledge
* Reverse Engineering basics

gdb and objdump should be installed on your Linux OS by standard, however if they are not you can install them from your favourite package manager.

Yes you will need perseverance if your a beginner to this because things can get tricky,
and basic knowledge of Assembly and reverse engineering binaries.

So lets get to it!

Firstly download the binary file from --> here <--

And lets begin some static analysis:

root@fedvm /pentest/reverseEngineering # file anti-Dbg.bin
anti-Dbg.bin: ELF 32-bit LSB executable, Intel 80386, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.9, not stripped

So its an ELF 32-bit executable and its symbols have not been stripped which is always good for debugging.

lets run the file:
root@fedvm /pentest/reverseEngineering # ./anti-Dbg.bin
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
||        Welcome to an anti debug challenge !!!!!        ||
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Password : password?

Wrong password.

So it gives us a password prompt but we have know way of possibly knowing what it could be. So lets get all static strings from the file, maybe it has been stored in a variable? If you run the command:
root@fedvm /pentest/reverseEngineering # cat anti-Dbg.bin | strings

You can see there is a lot of junk but among those strings you can find the text displayed at run time and password prompt with a simple '| grep "welcome" ' or ' | grep "password" '
However any other strings that look like they could be passwords are not. So next step...debugging the application to get the password.
root@fedvm /pentest/reverseEngineering # gdb -q anti-Dbg.bin
Reading symbols from /pentest/reverseEngineering/anti-Dbg.bin...(no debugging symbols found)...done.
(gdb) r
Starting program: /pentest/reverseEngineering/anti-Dbg.bin
You Lose! Debuggering!....
[Inferior 1 (process 3731) exited with code 01]
(gdb)

So the program has detected us debugging and doesn't even let us start to make any progress... unless we could stop the program at the beginning of execution? Before the anti-debugging has a chance to kick in? If its that type of AD technique... lets try it. Add a breakpoint to main.

(gdb) b *main
Breakpoint 1 at 0x80483f0
(gdb) r
Starting program: /pentest/reverseEngineering/anti-Dbg.bin

Breakpoint 1, 0x080483f0 in main ()

So we have added a breakpoint to main and it has now paused execution. (note. sometimes b *main wouldnt work because of symbols not being included during compilation or if the program utilizes an in-memory encryption-decryption algorithm, so the next step will explain how to set a breakpoint at the start of a program regardless of this issue).

Now we need analyze the x86 ASM code and step through the program avoiding the anti-debugging techniques. We do this using objdump.

root@fedvm /pentest/reverseEngineering # objdump -D anti-Dbg.bin > antiDebugObjDump

Now navigate inside this file to the line which says 080483f0 <main>: 
(your address may be different to mine)

Here is the beginning of the program and the address is what you could use to also set the breakpoint on main if *main doesn't work...: 

(gdb) b *0x080483f0 

Now lets analyze the Assembly code whilst debugging the application, to bypass the anti-debugging techniques in place.

Firstly after the initial stack setup I see these instructions which look interesting:
8048401:       c7 45 f4 88 28 0c 08 movl   $0x80c2888,-0xc(%ebp)

So here a value located at -0xc(%ebp) is being loaded into the address 0x80c2888. I wonder what this is? Lets set a breakpoint here for later.

(gdb) b *0x08048401
Breakpoint 2 at 0x8048401

Next:

8048408:       6a 00                         push   $0x0
804840a:       6a 01                         push   $0x1
804840c:       6a 00                         push   $0x0
804840e:       6a 00                         push   $0x0
8048410:       e8 5b 06 01 00          call   8058a70 <ptrace>

What we have here is arguments being pushed onto the stack for the function call ptrace at 0x08048410. If you already have C knowledge you may have seen this ptrace() function call before. You can use this function to check if any other processes attach to the current process which made the ptrace call. This is most likely how the ptrace is working.

lets continue:
(gdb) c
Continuing.

Breakpoint 3, 0x08048401 in main ()
(gdb) si
0x08048408 in main ()
//We are now at the instruction one after the the movl instruction which moved a value into 0x80c2888. Lets see what that value was:

(gdb) x/8s 0x80c2888
0x80c2888:       "ksuiealohgy"
0x80c2894:       "You Lose! Debuggering!...."
0x80c28af:       ""
0x80c28b0:       '+' <repeats 60 times>
0x80c28ed:       ""
0x80c28ee:       ""
0x80c28ef:       ""
0x80c28f0:       "||        Welcome to an anti debug challenge !!!!!        ||"
So it saved the string "ksuiealohgy". hmmm lets ignore it for now but keep it in mind...
Also below we see other strings that we have encountered.

Before we go any further lets put a breakpoint on the next important instruction:

804841a:       79 1a                   jns    8048436 <main+0x46>
 
 (gdb) b *0x0804841a

and continue using "c"...

(gdb) c
Continuing.
Breakpoint 4, 0x0804841a in main ()

We are now at the JNS instruction. This means Jump Short if Not zero and the value it is interested in is the SF (Sign flag) located in the FLAGS register. lets see what this value is:

(gdb) p $eflags
$1 = [ PF SF IF ]

It seems the SF flag is set so the jump will not be made so lets see what happens now when we continue:

804841c:       83 ec 0c                    sub    $0xc,%esp
804841f:       68 94 28 0c 08          push   $0x80c2894 ; what is this string being pushed onto the stack? it is what is going to be used as an argument in the next function puts().
8048424:       e8 a7 0e 00 00          call   80492d0 <_IO_puts>
8048429:       83 c4 10                    add    $0x10,%esp
804842c:       b8 01 00 00 00          mov    $0x1,%eax
8048431:       e9 c3 00 00 00          jmp    80484f9 <_notng+0x62>
8048436:       83 ec 0c                    sub    $0xc,%esp

(gdb) si //step
0x0804841c in main ()
(gdb) si //step
0x0804841f in main ()
(gdb) si //step
0x08048424 in main ()
(gdb) x/1s 0x80c2894 //show data at this location
0x80c2894:       "You Lose! Debuggering!...."
 The string located at 0x80c2894 as you can see above is telling us that the program is probably about to end and after that instruction further down, one instruction before where we would have jumped if SF = 0, is JMP 0x080484f9. at that location is located:

80484f9:       8b 4d fc                mov    -0x4(%ebp),%ecx
80484fc:       c9                      leave
80484fd:       8d 61 fc                lea    -0x4(%ecx),%esp
8048500:       c3                      ret
8048501:       90                      nop
8048502:       90                      nop

As you can see on your own objdump too, the program is about to exit.

So lets go back to where the JNS instruction was and unset the SF flag so the jump gets executed.
SF is the 7th bit in the register so in order to cancel it we need to xor with 2^7 = 128

(gdb) set $eflags = $eflags ^ 128
(gdb) p $eflags
$4 = [ PF IF ]

Now the SF flag has been unset lets step into the next instruction:

(gdb) si
0x08048436 in main ()

We have successfully bypassed the ptrace() function and continued to execute the program.
Before we continue to execute set a breakpoint on the address: 0x08048495 - This is where the program will continue once you have pressed enter after entering your password.

(gdb) c
Continuing.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
||        Welcome to an anti debug challenge !!!!!        ||
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Password : AAAAAAAA

Breakpoint 5, 0x08048495 in main ()

Now step to the instruction: 0x080484a3 using si...
80484a3:       38 c2                   cmp    %al,%dl

This is comparing %al and %dl the first 8 bits of the $eax and $edx registers. lets see what they are:
(gdb) p $al
$5 = 101
(gdb) p $dl
$6 = 65 

101 in decimal is the value : e
65 in decimal is the value : A

did we not put AAAAAAAA as our password??? moving on...

They were not equal so the next instruction:
80484c1:        75 21                   jne 80484e4 <_notng+0x4d>

Will take the jump to 0x080484e4 and this code looks like so:
 80484e4:       83 ec 0c                sub    $0xc,%esp
 80484e7:       68 8e 29 0c 08          push   $0x80c298e
 80484ec:       e8 df 0d 00 00          call   80492d0 <_IO_puts>
 80484f1:       83 c4 10                add    $0x10,%esp
 80484f4:       b8 00 00 00 00          mov    $0x0,%eax
 80484f9:       8b 4d fc                mov    -0x4(%ebp),%ecx
 80484fc:       c9                      leave
 80484fd:       8d 61 fc                lea    -0x4(%ecx),%esp

As you might be able to gather from last time, this again is the exit routine and you see it calls _IO_puts before it exits with the argument from  0x080c298e... this will probably be the string "wrong password" or something:

(gdb) x/1s 0x080c298e
0x80c298e:       "\nWrong password.\n"

Ahh there you go... so lets, before we continue, make sure the JNE instruction DOES NOT take the jump to the exit routine and continues exiting our code. we must change the Zero flag in order so it represents true. 
The Zero Flag(ZF) is the 6th bit which in decimal is 2^6 = 64.

If any of you are experienced in reverse engineering, looking at the disassembled executable code, you may be able to instantly realize that the program is comparing %al with %dl and then checking if the Zero flag has been set, if it has carry on comparing. The program is comparing our password to something else? The real password?

 80484a3:       38 c2                   cmp    %al,%dl
 80484a5:       75 3d                   jne    80484e4 <_notng+0x4d>
 80484a7:       8a 55 eb                mov    -0x15(%ebp),%dl
 80484aa:       8b 45 f4                mov    -0xc(%ebp),%eax
 80484ad:       83 c0 05                add    $0x5,%eax
 80484b0:       8a 00                   mov    (%eax),%al
 80484b2:       38 c2                   cmp    %al,%dl
 80484b4:       75 2e                   jne    80484e4 <_notng+0x4d>
 80484b6:       8a 55 ec                mov    -0x14(%ebp),%dl
 80484b9:       8b 45 f4                mov    -0xc(%ebp),%eax
 80484bc:       40                      inc    %eax
 80484bd:       8a 00                   mov    (%eax),%al
 80484bf:       38 c2                   cmp    %al,%dl
 80484c1:       75 21                   jne    80484e4 <_notng+0x4d>
 80484c3:       8a 55 ed                mov    -0x13(%ebp),%dl
 80484c6:       8b 45 f4                mov    -0xc(%ebp),%eax
 80484c9:       83 c0 0a                add    $0xa,%eax
 80484cc:       8a 00                   mov    (%eax),%al
 80484ce:       38 c2                   cmp    %al,%dl
 80484d0:       75 12                   jne    80484e4 <_notng+0x4d>
 80484d2:       83 ec 0c                sub    $0xc,%esp
 80484d5:       68 7a 29 0c 08          push   $0x80c297a
 80484da:       e8 f1 0d 00 00          call   80492d0 <_IO_puts> ; finally if we reached this far now print something ?

So lets step through the program now using 
(gdb) si
and at each "cmp" instruction, note down the value of $al, then change the zero flag so we can progress to the next letter.

First set zero flag:

(gdb) si
0x080484a5 in main ()
(gdb) set $eflags = $eflags | 64
(gdb) p $eflags
$11 = [ CF AF ZF SF IF ]
(gdb) b * 0x080484b2
(gdb) c
Continuing.

Breakpoint 6, 0x080484b2 in main ()
(gdb) p $al
$12 = 97
_____________________________________________

0x080484a3 cmp    %al,%dl  ----> %al == 101 ="e"
0x080484b2 cmp    %al,%dl  ----> %al == 97  ="a"
______________________________________________

(gdb) si
0x080484b4 in main ()
(gdb) set $eflags = $eflags | 64
(gdb) b *0x080484bf
Breakpoint 7 at 0x80484bf
(gdb) c
Continuing.

Breakpoint 7, 0x080484bf in main ()
(gdb) p $al
$15 = 115


_____________________________________________

0x080484a3 cmp    %al,%dl  ----> %al == 101 ="e"
0x080484b2 cmp    %al,%dl  ----> %al == 97   ="a"
0x080484bf cmp     %al,%dl  ----> %al == 115 = "s"
_____________________________________________

----- can you tell what it might be yet.......continuing........

(gdb) b *0x080484ce
Breakpoint 8 at 0x80484ce
(gdb) si
0x080484c1 in main ()
(gdb) set $eflags = $eflags | 64
(gdb) c
Continuing.
Breakpoint 8, 0x080484ce in main ()
(gdb) p %al
A syntax error in expression, near `%al'.
(gdb) p $al
$16 = 121

_____________________________________________

0x080484a3 cmp    %al,%dl  ----> %al == 101 ="e"
0x080484b2 cmp    %al,%dl  ----> %al == 97   ="a"
0x080484bf cmp     %al,%dl  ----> %al == 115 = "s"
0x080484ce cmp     %al,%dl  ----> %al == 121 = "y"
_____________________________________________
(gdb) si
0x080484d0 in main ()
(gdb) set $eflags = $eflags | 64
So we have just stepped into the final JNE instruction, set the ZF bit in the $eflags register, and now if we continue, lets see what happens:

(gdb) c
Continuing.

Good password !!!

[Inferior 1 (process 4320) exited normally]

Oh, So we entered AAAAAAAA as the password... the password turned out to be "easy" but we win anyway!

root@fedvm /pentest/reverseEngineering # ./anti-Dbg.bin
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
||        Welcome to an anti debug challenge !!!!!        ||
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

Password : easy

Good password !!!
_________________________________________________________________________

Although the password was "easy", no pun intended, what has been shown is ptrace() is not enough alone for effective anti-Debugging techniques, and also password protections such as comparing each letter to the corresponding password letter, are also very poor and can easily be cracked.

I hope some of you learned something anyway, and there will be some more blog posts on reverse engineering and cracking soon, but not as simple as this :P

~m0x39