How to improve Visual C++ compilation times?

Question

I am compiling 2 C++ projects in a buildbot, on each commit. Both are around 1000 files, one is 100 kloc, the other 170 kloc. Compilation times are very different from gcc (4.4) to Visual C++ (2008).

Visual C++ compilations for one project take in the 20 minutes. They cannot take advantage of the multiple cores because a project depend on the other. In the end, a full compilation of both projects in Debug and Release, in 32 and 64 bits takes more than 2 1/2 hours.

gcc compilations for one project take in the 4 minutes. It can be parallelized on the 4 cores and takes around 1 min 10 secs. All 8 builds for 4 versions (Debug/Release, 32/64 bits) of the 2 projects are compiled in less than 10 minutes.

What is happening with Visual C++ compilation times? They are basically 5 times slower.

What is the average time that can be expected to compile a C++ kloc? Mine are 7 s/kloc with vc++ and 1.4 s/kloc with gcc.

Can anything be done to speed-up compilation times on Visual C++?

Answer 1

Are you building on the same machine? Are you using the same OS? I've seen speed differences in the region of 3-10x when comparing GCC in Cygwin and GCC in a VirtualBox machine running inside the Windows hosting Cygwin.

Answer 2

The book "Large-Scale C++ Software Design" by John Lakos has many tips on structuring your code and design for large-scale projects. Including many tips on speeding up compilation. Not directly related to Visual C++, but well worth reading anyway.

Answer 3

Perhaps there is an issue with the dependency checking, unless you are forcing a complete rebuild.

You could make some static libraries. Put code that seldom changes into libraries.

The slowest parts of building a program:

1. Opening and closing files.
2. Parsing and translating source files.

In general, the linking and executable creation phases are the fastest.

Have you determined:

1. which phases are the slowest?
2. Which files are slowest to compile?

Remember, when determining efficiency, always profile (in one manner or another).

Answer 4

Don't know if this is still an issue and how much improvement you got in the menatime, but if it still stands that msbuild doesn't know how to orchestrate concurrently within one project (each cpp should be separately buildable unless you have some codegens - codegens are best moved to a separate project) you hay have to download driver development kit or .NET SSCLI since they both have nmake, build which are known to paralelize things well. SSCLI already had the build build setup, don't remeber if DDK has some build samples of do you have to start from scratch.

Also a bit oldish article on MSBuild parallelization doesn't go into details but mentiones some diff between actual msbuild and msbuild + sln. If the /MP vs. /Gm was the only issue then you may have to make a little script or C# exe to edit .proj files for lab build. Or use explicit cmd line override in projects and take that option from an env var.

Answer 5

I've written two articles on techniques that reduce the compilation time. Among these techniques a post on precompiled header and unity builds that may help you improve compilation times. They ship with CMake scripts that handle the techniques transparently.

Answer 6

The projects depending on each other doesn't imply that no parallelization is possible. The build systems are smart enough to figure out and avoid critical depenedancies, Otherwise gcc wouldn't be able to use 4 cores.

So (in addition to other steps), why not just try enabling multiprocessing in Visual Studio using /MP (See http://msdn.microsoft.com/en-us/library/bb385193.aspx).