Very slow compile times on Visual Studio 2005

Question

We are getting very slow compile times, which can take upwards of 20+ minutes on dual core 2GHz, 2G Ram machines.

A lot of this is due to the size of our solution which has grown to 70+ projects, as well as VSS which is a bottle neck in itself when you have a lot of files. (swapping out VSS is not an option unfortunately, so I don't want this to descend into a VSS bash)

We are looking at merging projects. We are also looking at having multiple solutions to achieve greater separation of concerns and quicker compile times for each element of the application. This I can see will become a DLL hell as we try to keep things in synch.

I am interested to know how other teams have dealt with this scaling issue, what do you do when your code base reaches a critical mass that you are wasting half the day watching the status bar deliver compile messages.

UPDATE I neglected to mention this is a C# solution. Thanks for all the C++ suggestions, but it's been a few years since I've had to worry about headers.

EDIT:

Nice suggestions that have helped so far (not saying there aren't other nice suggestions below, just what has helped)

• New 3GHz laptop - the power of lost utilization works wonders when whinging to management
• Disable Anti Virus during compile
• 'Disconnecting' from VSS (actually the network) during compile - I may get us to remove VS-VSS integration altogether and stick to using the VSS UI

Still not rip-snorting through a compile, but every bit helps.

Orion did mention in a comment that generics may have a play also. From my tests there does appear to be a minimal performance hit, but not high enough to sure - compile times can be inconsistent due to disc activity. Due to time limitations, my tests didn't include as many Generics, or as much code, as would appear in live system, so that may accumulate. I wouldn't avoid using generics where they are supposed to be used, just for compile time performance

WORKAROUND

We are testing the practice of building new areas of the application in new solutions, importing in the latest dlls as required, them integrating them into the larger solution when we are happy with them.

We may also do them same to existing code by creating temporary solutions that just encapsulate the areas we need to work on, and throwing them away after reintegrating the code. We need to weigh up the time it will take to reintegrate this code against the time we gain by not having Rip Van Winkle like experiences with rapid recompiling during development.

Answer 1

I'm also now convinced there is a problem with VS2008. I'm running it on a dual core Intel laptop with 3G Ram, with anti-virus switched off. Compiling the solution is often quite slick, but if I have been debugging a subsequent recompile will often slow down to a crawl. It is clear from the continuous main disk light that there is a disk I/O bottleneck (you can hear it, too). If I cancel the build and shutdown VS the disk activity stops. Restart VS, reload the solution and then rebuild, and it is much faster. Unitl the next time

My thoughts are that this is a memory paging issue - VS just runs out of memory and the O/S starts page swapping to try to make space but VS is demanding more than page swapping can deliver, so it slows down to a crawl. I can't think of any other explanation.

VS definitely is not a RAD tool, is it?

Answer 2

It's sure there's a problem with VS2008. Because the only thing I've done it's to install VS2008 for upgrading my project which has been created with VS2005. I've only got 2 projects in my solution. It isn't big. Compilation with VS2005 : 30 secondes Compilation with VS2008 : 5 minutes

Answer 3

Some analysis tools:

tools->options->VC++ project settings -> Build Timing = Yes will tell you build time for every vcproj.

Add /Bt switch to compiler command line to see how much every CPP file took

Use /showIncludes to catch nested includes (header files that include other header files), and see what files could save a lot of IO by using forward declarations.

This will help you optimize compiler performance by eliminating dependencies and performance hogs.

Answer 4

I have a project which has 120 or more exes, libs and dlls and takes a considerable time to build. I use a tree of batch files that call make files from one master batch file. I have had problems with odd things from incremental (or was it temperamental) headers in the past so I avoid them now. I do a full build infrequently, and usually leave it to the end of the day while I go for a walk for an hour (so I can only guess it takes about half an hour). So I understand why that is unworkable for working and testing.

For working and testing I have another set of batch files for each app (or module or library) which also have all the debugging settings in place -- but these still call the same make files. I may switch DEBUG on of off from time to time and also decide on builds or makes or if I want to also build libs that the module may depend on, and so on.

The batch file also copies the completed result into the (or several) test folders. Depending of the settings this completes in several seconds to a minute (as opposed to say half an hour).

I used a different IDE (Zeus) as I like to have control over things like .rc files, and actually prefer to compile from the command line, even though I am using MS compliers.

Happy to post an example of this batch file if anyone is interested.

Answer 5

I had a similar issue on a solution with 21 projects and 1/2 million LOC. The biggest difference was getting faster hard drives. From the performance monitor the 'Avg. Disk Queue' would jump up significantly on the laptop indicating the hard drive was the bottle neck.

Here's some data for total rebuild times...

1) Laptop, Core 2 Duo 2GHz, 5400 RPM Drive (not sure of cache. Was standard Dell inspiron).

Rebuild Time = 112 seconds.

2) Desktop (standard issue), Core 2 Duo 2.3Ghz, single 7200RPM Drive 8MB Cache.

Rebuild Time = 72 seconds.

3) Desktop Core 2 Duo 3Ghz, single 10000 RPM WD Raptor

Rebuild Time = 39 seconds.

The 10,000 RPM drive can not be understated. Builds where significantly quicker plus everything else like displaying documentation, using file explorer was noticable quicker. It was a big productivity boost by speeding the code-build-run cycle.

Given what companies spend on developer salaries it is insane how much they can waste buy equiping them with the same PCs as the receptionist uses.

Answer 6

If this is a web app, setting batch build to true can help depending on the scenario.

<compilation defaultLanguage="c#" debug="true" batch="true" >  

You can find an overview here: http://weblogs.asp.net/bradleyb/archive/2005/12/06/432441.aspx