Very slow compile times on Visual Studio 2005

Question

We are getting very slow compile times, which can take upwards of 20+ minutes on dual core 2GHz, 2G Ram machines.

A lot of this is due to the size of our solution which has grown to 70+ projects, as well as VSS which is a bottle neck in itself when you have a lot of files. (swapping out VSS is not an option unfortunately, so I don't want this to descend into a VSS bash)

We are looking at merging projects. We are also looking at having multiple solutions to achieve greater separation of concerns and quicker compile times for each element of the application. This I can see will become a DLL hell as we try to keep things in synch.

I am interested to know how other teams have dealt with this scaling issue, what do you do when your code base reaches a critical mass that you are wasting half the day watching the status bar deliver compile messages.

UPDATE I neglected to mention this is a C# solution. Thanks for all the C++ suggestions, but it's been a few years since I've had to worry about headers.

EDIT:

Nice suggestions that have helped so far (not saying there aren't other nice suggestions below, just what has helped)

• New 3GHz laptop - the power of lost utilization works wonders when whinging to management
• Disable Anti Virus during compile
• 'Disconnecting' from VSS (actually the network) during compile - I may get us to remove VS-VSS integration altogether and stick to using the VSS UI

Still not rip-snorting through a compile, but every bit helps.

Orion did mention in a comment that generics may have a play also. From my tests there does appear to be a minimal performance hit, but not high enough to sure - compile times can be inconsistent due to disc activity. Due to time limitations, my tests didn't include as many Generics, or as much code, as would appear in live system, so that may accumulate. I wouldn't avoid using generics where they are supposed to be used, just for compile time performance

WORKAROUND

We are testing the practice of building new areas of the application in new solutions, importing in the latest dlls as required, them integrating them into the larger solution when we are happy with them.

We may also do them same to existing code by creating temporary solutions that just encapsulate the areas we need to work on, and throwing them away after reintegrating the code. We need to weigh up the time it will take to reintegrate this code against the time we gain by not having Rip Van Winkle like experiences with rapid recompiling during development.

Answer 1

One cheaper alternative to Xoreax IB is the use of what I call uber-file builds. It's basically a .cpp file that has

#include "file1.cpp"
#include "file2.cpp"
....
#include "fileN.cpp"

Then you compile the uber units instead of the individual modules. We've seen compile times from from 10-15 minutes down to 1-2 minutes. You might have to experiemnt with how many #includes per uber file make sense. Depends on the projects. etc. Maybe you include 10 files, maybe 20.

You pay a cost so beware:

1. You can't right click a file and say "compile..." as you have to exclude the individual cpp files from the build and include only the uber cpp files
2. You have to be careful of static global variable conflicts.
3. When you add new modules, you have to keep the uber files up to date

It's kind of a pain, but for a project that is largely static in terms of new modules, the intial pain might be worth it. I've seen this method beat IB in some cases.

Answer 2

Looking at the machine that you're building on, is it optimally configured?

We just got our build time for our largest C++ enterprise-scale product down from 19 hours to 16 minutes by ensuring the right SATA filter driver was installed.

Subtle.

Answer 3

There are a few things that I have found useful for speading up C# /.NET builds:

• Combine small projects into larger projects as there is a large per project overhead on building a solution. (Use nDepend if needed to control calling across layers)

• Make all our projects build into the some output directory and then set “copy local” to false on all the project references – this can lead to a large speed up due to reduced IO.

• Turn of your virus checker to see if it makes much difference; if so find a faster virus checker, or exclude the "hot" folders from the virus checker scanning

• Use perforce monitor and the sys internal tool to see way your compiles are taking so long.

• Consider a ram disk to put your output directory on.

• Consider using a SSD

• More memory can have a big effect at times – (reduce the ram in the machine if you get a big slow down by removing a little ram, you may get a big speed up by adding more) Remove unneeded project references (you may have to remove unneeded “usings” first)

• Consider using a dependency injection framework and interfaces for your lowest domain layer, so a recompile of everything is only needed when the interface changes – this may not gain much depending on how often the interface is changed.

Answer 4

Use distributed compilation. Xoreax IncrediBuild can cut compilation time down to few minutes.

I've used it on a huge C\C++ solution which usually takes 5-6 hours to compile. IncrediBuild helped to reduce this time to 15 minutes.

Answer 5

Make sure your references are Project references, and not directly to the DLLs in the library output directories.

Also, have these set to not copy locally except where absolutely necessary (The master EXE project).

Answer 6

Turn off VSS integration. You may not have a choice in using it, but DLLs get "accidentally" renamed all the time...

And definitely check your pre-compiled header settings. Bruce Dawson's guide is a bit old, but still very good - check it out: http://www.cygnus-software.com/papers/precompiledheaders.html