How to compile on multiple cores using mingw insid

Question

I have a quad-core i7 CPU on my windows desktop. I am trying to get mingw32-make to compile using as many core as possible. I have added -j8 into the "Make Arguments" fields under Build Settings->Build Steps. mingw32-make seems to completely ignore the option since I don't get any speed improvement out of that. When I view the CPU usage in task manager, it stays at 13% and I only see one instance of g++ running.

Right now, rebuilding the whole project using mingw takes 3 full minutes. Rebuilding it using MSVC takes only 15 seconds.

For your information, I have enabled the precompiled header option in the project settings. That makes things a lot quicker with VC as well. But I am still yet to see the benefit of precompiled header with mingw.

Please share your comments if you have ever got mingw to compile multiple source files in parallel from QTCreator. Thanks!

Answer 1

Add -j9 (replace 9 to value of NUMBER_OF_PROCESSORS(Windows)/$(nproc)(Linux) plus one — this is optimal) to all

QString makefilein = " -f " + subtarget->makefile;

lines in qmake\generators\makefile.cpp (find it yourself).

It results as

QString makefilein = " -j9 -f " + subtarget->makefile;

then run configure.exe with proper parametres (!and additional -qmake -dont-process to avoid generation abundance of makefiles!).

The problem is that you get two sets of processes during "debug and release" build. Thereby, total count of processes spawned is 18+.

Answer 2

The issue is that the original make does not have jobserver support. So the mingw32 port didn't have it either. It has since been added however. I believe the oldest version with 'correct' jobserver support is 3.82.90. You can find that at the link below.

http://sourceforge.net/projects/mingw/files/MinGW/Extension/make/make-3.82.90-cvs-20120823/

Answer 3

-j8 probably isn't working because of a limitation in GNU Make on Win32.

Try putting just -j into the make arguments field. That tells Make to spawn as many compile processes as it can--if you have enough RAM and CPU to handle that, it should be faster than a single compile.

Unfortunately these are the only two options (without modifying the Makefiles): either -j1, or unlimited -j

Full details: GNU Make on Win32 doesn't support the job server, so a parent Make process can't track the number of compile processes spawned by any sub-Make's. To be safe, the sub-Make's only run with -j1. I believe qmake/Qt Creator's generated Makefiles use multiple layers of Makefiles. I first figured out this issue with Microchip's MPLAB X IDE, check out this thread for more info

This quote is from README.W32 distributed with GNU Make

Support for parallel builds

Parallel builds (-jN) are supported in this port, with 2 limitations:

• The number of concurrent processes has a hard limit of 64, due to the way this port implements waiting for its subprocesses;

• The job server method (available when Make runs on Posix platforms) is not supported, which means you must pass an explicit -jN switch to sub-Make's in a recursive Makefile. If a sub-Make does not receive an explicit -jN switch, it will default to -j1, i.e. no parallelism in sub-Make's.

Answer 4

Here's what I did.

In the Tools -> Options menu in Qt Creator, under the "Build and Run" section there's an option that says "Use jom instead of nmake". I checked that box, but nothing changed. So instead, I went to the project's build settings and under the build steps category there is a "Make" item. Expand the details on that and you'll find a "Override mingw32-make.exe:" item. I pasted "C:\QtSDK\QtCreator\bin\jom.exe" in there, and suddenly I was building with multiple cores.

Worked for me on Qt Creator 2.4.1. Give it a try.

Answer 5

In Qt Creator go to Projects -> Build & Run -> your specific MinGW build setup -> Build Environment (this is on the screen below General / Build steps / Clean steps..) and then add a variable MAKEFLAGS and set it to -j8. I tested this on my two core machine with -j4 and QtCreator 4.4.1 and with MinGW 5.3.0. When compiling my processor runs at 100 % performance as can be seen in the Task Manager. Without this option it was about 25 % so I assume it works exactly as expected. The build is now much faster.

Answer 6

Use MAKE_COMMAND environment variable:

set MAKE_COMMAND=mingw32-make -j%NUMBER_OF_PROCESSORS%