IDA SDK C++ tricks and tips

[bongos_man]

Originally Posted by chants

Unfortunately you must compile with QT headers, should link with the exact QT libs that IDA uses so it will reuse the QT DLL included with IDA, in this case for QTPainter - Qt5Gui.dll version 5.6.3.0

this is not entirely true. you can in fact substitute ida's qt dlls with your own (potentially modified) dlls. the primary difference between the stock qt dlls and those bundled with ida is that hex-rays builds theirs with "-qtnamespace QT" and thus all symbols with non-C linkage are not mangled identically to stock qt. here is the original post (from 2011, time flies ;]) from when idaq.exe first shipped:

https://www.hexblog.com/?p=250

not much has changed since then, here is info for ida 7.3 (i can confirm that this works, including complicated cases such as plugins loaded by qt itself--of course these too must be built against an appropriately configured qt installation):

https://www.hexblog.com/?p=1341

if you want to build qt without so much useless garbage, this e.g. will work:

..\qt-everywhere-opensource-src-5.6.3\configure -prefix C:/Qt/5.6.3.0 -debug-and-release -force-debug-info -platform win32-msvc2017 -opensource -confirm-license -shared -no-compile-examples -nomake examples -nomake tests -nomake tools -opengl desktop -no-warnings-are-errors -ltcg -no-rtti -sse2 -sse3 -ssse3 -sse4.1 -sse4.2 -avx -no-ssl -no-openssl -no-dbus -no-audio-backend -no-wmf-backend -no-qml-debug -no-native-gestures -mp -skip qt3d -skip qtactiveqt -skip qtandroidextras -skip qtcanvas3d -skip qtconnectivity -skip qtdeclarative -skip qtdoc -skip qtenginio -skip qtgraphicaleffects -skip qtimageformats -skip qtlocation -skip qtmacextras -skip qtmultimedia -skip qtquickcontrols -skip qtquickcontrols2 -skip qtscript -skip qtsensors -skip qtserialbus -skip qtserialport -skip qtsvg -skip qttools -skip qttranslations -skip qtwayland -skip qtwebchannel -skip qtwebengine -skip qtwebsockets -skip qtwebview -skip qtx11extras -skip qtxmlpatterns -qtnamespace QT

ida 7.2 (can also confirm my own success):

https://www.hexblog.com/?p=1261

see hexblog for more entries for previous versions, but again not much changes from release to release regarding qt.