What strategies have you used to improve build tim

Question

I once worked on a C++ project that took about an hour and a half for a full rebuild. Small edit, build, test cycles took about 5 to 10 minutes. It was an unproductive nightmare.

What is the worst build times you ever had to handle?

What strategies have you used to improve build times on large projects?

Update:

How much do you think the language used is to blame for the problem? I think C++ is prone to massive dependencies on large projects, which often means even simple changes to the source code can result in a massive rebuild. Which language do you think copes with large project dependency issues best?

Answer 1

One trick that sometimes helps is to include everything into one .cpp file. Since includes are processed once per file, this can save you a lot of time. (The downside to this is that it makes it impossible for the compiler to parallelize compilation)

You should be able to specify that multiple .cpp files should be compiled in parallel (-j with make on linux, /MP on MSVC - MSVC also has an option to compile multiple projects in parallel. These are separate options, and there's no reason why you shouldn't use both)

In the same vein, distributed builds (Incredibuild, for example), may help take the load off a single system.

SSD disks are supposed to be a big win, although I haven't tested this myself (but a C++ build touches a huge number of files, which can quickly become a bottleneck).

Precompiled headers can help too, when used with care. (They can also hurt you, if they have to be recompiled too often).

And finally, trying to minimize dependencies in the code itself is important. Use the pImpl idiom, use forward declarations, keep the code as modular as possible. In some cases, use of templates may help you decouple classes and minimize dependencies. (In other cases, templates can slow down compilation significantly, of course)

But yes, you're right, this is very much a language thing. I don't know of another language which suffers from the problem to this extent. Most languages have a module system that allows them to eliminate header files, which area huge factor. C has header files, but is such a simple language that compile times are still manageable. C++ gets the worst of both worlds. A big complex language, and a terrible primitive build mechanism that requires a huge amount of code to be parsed again and again.

Answer 2

IncrediBuild

Answer 3

This book Large-Scale C++ Software Design has very good advice I've used in past projects.

Answer 4

1. Minimize your public API
2. Minimize inline functions in your API. (Unfortunately this also increases linker requirements).
3. Maximize forward declarations.
4. Reduce coupling between code. For instance pass in two integers to a function, for coordinates, instead of your custom Point class that has it's own header file.
5. Use Incredibuild. But it has some issues sometimes.
6. Do NOT put code that get exported from two different modules in the SAME header file.
7. Use the PImple idiom. Mentioned before, but bears repeating.
8. Use Pre-compiled headers.
9. Avoid C++/CLI (i.e. managed c++). Linker times are impacted too.
10. Avoid using a global header file that includes 'everything else' in your API.
11. Don't put a dependency on a lib file if your code doesn't really need it.
12. Know the difference between including files with quotes and angle brackets.

Answer 5

Create some unit test projects to test individual libraries, so that if you need to edit low level classes that would cause a huge rebuild, you can use TDD to know your new code works before you rebuild the entire app. The John Lakos book as mentioned by Themis has some very practical advice for restructuring your libraries to make this possible.

Answer 6

This is the list of things we did for a development under Linux :

• As Warrior noted, use parallel builds (make -jN)
• We use distributed builds (currently icecream which is very easy to setup), with this we can have tens or processors at a given time. This also has the advantage of giving the builds to the most powerful and less loaded machines.
• We use ccache so that when you do a make clean, you don't have to really recompile your sources that didn't change, it's copied from a cache.
• Note also that debug builds are usually faster to compile since the compiler doesn't have to make optimisations.