Denial of Service against below IDA 5.7

[redbull]

Hi Guys,

Some of you have probably seen this already but there is a DOS against a specially crafted QNX file for IDA PRO.

I bid:
http://seclists.org/bugtraq/2010/Jun/237

Could be useful as anti-disasm code I guess.

Denial-of-Service Vulnerability in IDA Pro
------------------------------------------

June 28th, 2010

=======
Summary
=======
Name: Denial-of-Service Vulnerability in IDA Pro
Release Date: June 28th, 2010
Discoverer: Jason Geffner
Version Affected: IDA Pro 3.76 through 5.6
Risk: Low
Status: Published

============
Introduction
============
This paper discusses how a binary file could be crafted to cause IDA Pro to
consume 100% of CPU resources while trying to analyze it, thus preventing
disassembling. While this vulnerability is in the QNX file loader, a functional
COM file could be crafted to masquerade as a QNX file and trigger this issue.
This vulnerability was responsibly disclosed to IDA Pro's support personnel and
this advisory was not released until a fixed build was publicly released.

==========
Background
==========
"The IDA Pro Disassembler and Debugger is an interactive, programmable,
extendible, multi-processor disassembler hosted on Windows, Linux, or Mac OS X.
IDA Pro has become the de-facto standard for the analysis of hostile code,
vulnerability research and COTS validation." [1]

========
Timeline
========
07/28/98 IDA Pro loader for QNX files written
12/31/09 Denial-of-service vulnerability discovered in IDA Pro loader for QNX
files
12/31/09 Detailed vulnerability report responsibly disclosed to IDA Pro's
support personnel
01/04/10 Response received from IDA Pro's support personnel, confirming
vulnerability
06/25/10 IDA Pro 5.7 released, fixing vulnerability
06/28/10 Advisory released

=============
Vulnerability
=============
IDA Pro uses different file loaders to disassemble files of different formats
(PE, ELF, etc.). The loader for QNX files contains a vulnerability that allows
a specially crafted file to cause the loader to go into an infinite loop,
thereby consuming 100% of CPU resources and preventing disassembly.

The for-loop below is designed to iterate through each lmf_data structure in
the input file, advancing the file pointer based on sizeof(lmf_data) +
lmf_data.offset). However, if lmf_data.offset == -sizeof(lmf_data) then at is
never increased and this code will run in an infinite loop.


From \ldr\qnx\qnx.cpp(50):

for(uint32 at = sizeof(ex.lmf_header)+ex.lmf_header.data_nbytes;
lmf_data.segment_index != _LMF_EOF_REC;
at += sizeof(lmf_data) + lmf_data.offset)
{
qlseek( li, at, 0 );
if ( sizeof(_lmf_data) !=
qlread( li, &lmf_data, sizeof(_lmf_data) ) ) return 0;
switch(lmf_data.segment_index)
{
...
case _LMF_COMMENT_REC:
break;
...
}
}

=======
Exploit
=======
While this vulnerability is in the QNX file loader, a functional COM file could
be crafted to masquerade as a QNX file and trigger this issue. As such, it
would be possible for a malware author to create a working malicious COM
program, craft it to appear as a QNX file to IDA Pro, and thus prevent IDA Pro
from being able to disassemble it. Windows 7 will correctly run such a COM
program even if it is named with a .EXE extension.

See below for a proof-of-concept COM file. When run from a command-prompt, this
program will print, "I can't be opened in IDA Pro ". When opened in IDA Pro,
it will cause IDA Pro to spin in an infinite loop.

00000000: 00 00 34 00 00 00 b4 09 eb 02 82 01 ba 13 01 cd ..4.............
00000010: 21 cd 20 49 20 63 61 6e 27 74 20 62 65 20 6f 70 !. I can't be op
00000020: 65 6e 65 64 20 69 6e 20 49 44 41 20 50 72 6f 20 ened in IDA Pro
00000030: 3a 29 24 00 00 00 00 00 00 00 01 00 fa ff ff ff $.............
00000040: 00 00 00 00 00 00 ......

==========
Conclusion
==========
In-depth code reviews and fuzzing should be performed on all software,
especially when the software is designed for analyzing malicious and/or
untrusted data.

===============
Fix Information
===============
This issue has now been resolved. IDA Pro 5.7 can be downloaded from
https://www.hex-rays.com/updida.shtml

==========
References
==========
[1] http://hex-rays.com/idapro/overview.htm

NGSSoftware Insight Security Research
http://www.ngssoftware.com/
http://www.databasesecurity.com/
http://www.nextgenss.com/
+44(0)208 401 0070