Unpacking and Inline Patching FSG v1.0

[TQN]

This method of hacnho can only applied with a small and simple packed exe. OllyDbg will fail when tracing with a large, complex exe. For example, I download FSG 1.0 from this site (ExeTools), pack the Stud_PE and trace with OllyDbg. Failed to find OEP.
We can use PEiD to find OEP. PEiD will find the correct OEP with packed Stud_PE. The plugin "PEiD Generic Unpacker" of PEiD can automatic unpack the FSG 1.0 packed EXE. However, PEiD sometime will fail on a console, packed Exe.
Another way is same as JMI way, use OllyDump to find OEP by "Find OEP by Section Hop (xxx)", but it take a long time.
QUnpack of FEUERRADER can find the correct OEP of Stud_PE packed, but it failed when unpack.
With the OEP found, you can he or bp on it, dump with OllyDump and rebuild IAT with ImpRec.
I am finding the manual way to find the OEP of FSG 1.0 packed exe. If I success, I will post information here.
Regards

[bgrimm]

Originally Posted by TQN

This method of hacnho can only applied with a small and simple packed exe. OllyDbg will fail when tracing with a large, complex exe. For example, I download FSG 1.0 from this site (ExeTools), pack the Stud_PE and trace with OllyDbg. Failed to find OEP.

Your post intrigued me as I had not experimented much beyond "simple" apps with FSG.
I downloaded Stud_PE 1.8.0 (file size 663,552 bytes), I assume that was your target?

Then compressed it with FSG 1.0 resulting in a packed exe 288,864 bytes in size.

I loaded it into Olly (1.10s2) and let it trace bytewise to entry, stopping at OEP.
After a long time, in the order of 10 minutes or so, it arrived on the OEP.

---> OEP 0039F14 <55 PUSH EBP>
(Note: Same as reported by PEiD)

Dumped with OllyDump 2.21.108, no rebuild.
Fixed Imps with ImpRec, all valid.

Ended with an ugly, but fully functional Unpacked Stud_PE.exe (983,040 bytes)

Just for kicks I FSG'd several misc apps (MASM & VC4-6)
Ran them all thru Olly in the way described above. And resolved all OEP's correctly.
I did hit a few snags after OEP on a few of the test apps,
(Note: due to 1-year old daughter clearing off desk rapidly at this moment I must be brief)

One app, PEid did not report the correct OEP with generic OEP finder.
One app, dumped ok, but could not rebuild imports with ImpRec even though all valid.
(haven't had time to look into why)

I to am interested in finding the manual way to OEP and will continue testing
when time allows.

-bg

[TQN]

Hi all !
I think I found the manual way to unpack FSG 1.0.
1. Bp GetProcAddress
2. Go until breakpoint occurred
3. F12 (run until return)
4. If the address of EIP is in the memory range of exe, scroll up 2-3 line. We will see a je OEP. If not, repeat the step 1 (continue run)
Attached file is a script file of OllyScript, will find address of the je and the OEP. It can have bug.
Regards

[bgrimm]

Your process reacts differently on my apps.
After the Break on GetProcAddress, I'll execute-until-return until the EIP is within the exe's memory range.
I get this dialog whenver I hit that point or run your script.

"Suspicious Breakpoint
It looks like you are trying to set a breakpoint on the data. If this is really the case, such a breakpoint will not execute
and may have disastrous influence on the debugged program. Do you really want to set a breakpoint here?"

If I answer yes, the program runs a few lines and breaks with that message again. Repeats a few more times until program
goes off and never breaks again.

If I answer no, the program runs and never breaks again.

I see un-analyzed but unpacked CODE SECTION bytes in the disassembler window.

On the first app I tried, while executing-till-return it broke within 7 bytes of the OEP. (OEP:401000, broke on 401007)
The second app it broke on 401CFC, actual unpacked OEP is 401CD0.
A few more apps, different ranges, but same dialog.

Everything looks unpacked in memory but Olly does not let me do anything except answer the dialog.

The script and the manual process end up at the same dialog.
I thought it might be a simple Olly configuration item but have found nothing relavant.

-bg
Tested on Ollydbg 1.10c & ollydbg1.10b, both with Ollyscript 0.81

[TQN]

I am at work now, so I dont have OllyDbg here to retest it. At home, I remember we will see a call to GetProcAddress as call dword ptr[xxx]. Subtract the value of EIP at the line after F12 with 0xB, you will see a cmp xxx and a JE OEP. FSG 1.0 rebuid the IAT with many calls to GetProcAddress, and until the count of import functions go to 0, it will jump to OEP.
Some VC++ app which uses ComCtlxxx.dll will call GetProcAddress many times, so we need to run until the call GetProcAddress is from code of packed exe (check stack), and we need only press F12 only once.
Hope you will solve it !
Regards

[bgrimm]

Greetings once again,

TQN,

I was now able to manually use your technique to find the OEP. I also made
a couple of modifications to your script to get it to work properly.

Manually I followed it pretty much as you described.
Only extra step I encountered was after executing till return I had to step
out of the call to see the described JE test a few bytes above call.

Your script still did not work until I made the following mods:

Original script contents: (This section of code never resolved correctly.)
-----------------------
@@1:
rtr <----- RTR is buggy and does not break correctly.
mov addr, eip
sub addr, B <----- subtracting only 11 bytes places only offset in addr, therefore cmp for JE bytes below never is true.
mov opcode, [addr]
and opcode, FFFF
cmp opcode, 840F
je @@2
eob @@1
run

Changing the script to this seemed to work perfectly.
---------------------------------------------------
@@1:
rtu
mov addr, eip
sub addr, D
mov opcode, [addr]
and opcode, FFFF
cmp opcode, 840F
je @@2
eob @@1
run

Attached is the revised script that worked on all my test applications.

With one App I tested, PEiD did not find the correct OEP but the script worked.
Turns out it was a problem with it being a console app and an invalid PE header, even though it ran fine.

-bg

[TQN]

Oh, bgrimm, at the end, we are solve problems.
Thank for your revise script. I will retest it.
When I wrote this script, I see a strange behavior of rtr and rtu. I need more time to write test scripts with rtr and rtu, and if it is a bug, I will inform SHaG.
Regards