What strategies have you used to improve build tim

Question

I once worked on a C++ project that took about an hour and a half for a full rebuild. Small edit, build, test cycles took about 5 to 10 minutes. It was an unproductive nightmare.

What is the worst build times you ever had to handle?

What strategies have you used to improve build times on large projects?

Update:

How much do you think the language used is to blame for the problem? I think C++ is prone to massive dependencies on large projects, which often means even simple changes to the source code can result in a massive rebuild. Which language do you think copes with large project dependency issues best?

Answer 1

The best suggestion is to build makefiles that actually understand dependencies and do not automatically rebuild the world for a small change. But, if a full rebuild takes 90 minutes, and a small rebuild takes 5-10 minutes, odds are good that your build system already does that.

Can the build be done in parallel? Either with multiple cores, or with multiple servers?

Checkin pre-compiled bits for pieces that really are static and do not need to be rebuilt every time. 3rd party tools/libraries that are used, but not altered are a good candidate for this treatment.

Limit the build to a single 'stream' if applicable. The 'full product' might include things like a debug version, or both 32 and 64 bit versions, or may include help files or man pages that are derived/built every time. Removing components that are not necessary for development can dramatically reduce the build time.

Does the build also package the product? Is that really required for development and testing? Does the build incorporate some basic sanity tests that can be skipped?

Finally, you can re-factor the code base to be more modular and to have fewer dependencies. Large Scale C++ Software Design is an excellent reference for learning to decouple large software products into something that is easier to maintain and faster to build.

EDIT: Building on a local filesystem as opposed to a NFS mounted filesystem can also dramatically speed up build times.

Answer 2

Cătălin Pitiș covered a lot of good things. Other ones we do:

• Have a tool that generates reduced Visual Studio .sln files for people working in a specific sub-area of a very large overall project
• Cache DLLs and pdbs from when they are built on CI for distribution on developer machines
• For CI, make sure that the link machine in particular has lots of memory and high-end drives
• Store some expensive-to-regenerate files in source control, even though they could be created as part of the build
• Replace Visual Studio's checking of what needs to be relinked by our own script tailored to our circumstances

Answer 3

Powerful compilation machines and parallel compilers. We also make sure the full build is needed as little as possible. We don't alter the code to make it compile faster.

Efficiency and correctness is more important than compilation speed.

Answer 4

In Visual Studio, you can set number of project to compile at a time. Its default value is 2, increasing that would reduce some time.

This will help if you don't want to mess with the code.

Answer 5

Full build is about 2 hours. I try to avoid making modification to the base classes and since my work is mainly on the implementation of these base classes I only need to build small components (couple of minutes).

Answer 6

1. Fiddle with the compiler optimisation flags,
2. use option -j4 for gmake for parallel compilation (multicore or single core)
3. if you are using clearmake , use winking
4. we can take out the debug flags..in extreme cases.
5. Use some powerful servers.