Why are xcodebuild and Xcode 4.2 so slow?

Question

I am using Xcode 4.2 on a relatively large project (a few ten thousand lines of code) and it is horribly slow. Editing is ok, but whenever I try to compile the project (in Xcode, or with xcodebuild on the command line), my machine (quad core i7 MacBook Pro, 4 GB RAM) crawls to a halt. I have noticed that directly after starting xcodebuild, it spawns more than 8 clang processes, without the "real" compile processes starting. No xcodebuild output is so far seen on stout. I've tried reducing the number of parallel build processes, but still lots of clang processes are launched at the beginning. The project uses 6 or 7 direct dependent external projects and has maybe 120 source files. Under Xcode 3.2 the project used to be compiled very quickly. What's happening? And how can I make Xcode fast again?

Answer 1

Quick Note Regarding 'Throw more hardware at it' approach..

SUMMARY: I experienced a SMALL speed increase from making a SIGNIFICANT hardware upgrade

Test: Build/Run the exact same project on cloned macbooks (where the only difference should be their hardware)

Old Macbook Air (1.86GHZ Core 2 Duo ONLY 2GB RAM)
vs
Brand New Macbook Pro (2.3GHZ Core i7 8GB RAM)

BUILDING ON IPHONE 3GS
Macbook Air 1:00 - 1:15
Macbook Pro ~1:00

=> 0 to 0:15 of speed increase

BUILDING ON IPHONE 4S
Macbook Pro ~0:35
Macbook Air ~0:50

=> ~15 seconds of speed increase

**Partially tested: There DOES apear to a significant difference between build times for the SIMULATOR between the 2 machines

Answer 2

You can switch on Distributed Building in XCode Preferences, and find some friendly person who will help you build your app by forming compilation machines cluster with you.

The funny thing is that even he is off, your compiler still uses different algorithm/mechanism to build you app in a blazing speed if compared to the problems before ;)

So, that means that they at Apple have forgotten about lonely programmers who don't work in teams and therefore lonely compilation scenario is purely tested in versions 4.0 - 4.2

Answer 3

One more possible solution that in some cases might help speed up Xcode 4: In my case, the main problem seems to have been that accidentally four files from my build/ folder had been checked in with my git repository. During compilation Xcode notices that the build folder changed, and triggers git. Since the build folder contains thousands of files in my case, the performance went down. Removing the build/ folder completely from git (shouldn't have been checked in anyway) reduced the compilation times and system load massively. Performance is still slower than with Xcode 3, but much better than before.

Answer 4

Most of us have three primary options:

• Revert to Xcode 3 for daily development.
• Throw more hardware at it.
• Change your projects' structures and apply large scale development tricks (even though 20-30 KSLOC is not large).

The easiest solution is revert to Xc3. Yes, Xc4 requires a lot more than Xc3; memory, CPU, and disk space and I/O. You will have to determine where your biggest problems are to reduce the amount it affects you.

I recently bought a new MBP with twice the physical cores and twice the physical memory, upgraded to Lion and upgraded Xc4 at the same time. The compilation times did improve, but much of the rest is actually slower, and far more resource hungry. That's not at all what one would expect from an IDE which also disallows multiple open projects and also uses a unified workspace view.

The move to Lion + Xc4 more than doubled my hardware demands in all of the following categories:

Memory

4GB is now too little for most nontrivial projects using Xc4 and Lion. You can still reduce this. I have 8GB and 10GB on my main 2 machines, Xc4 consumes it all quite easily (but my projects are more complex than yours, unless you write reeaeaaaally long lines). Anyways, You can reduce this problem by:

• Buying more memory.
• Disable indexing if you are building out huge projects in Xcode. This can halve Xcode's memory consumption.
• Running Xcode in 32 bit. This is not an option for everyone, because it will exceed 4 GB in larger projects.
• Reduce the number of build processes (again).
• Restarting Xcode often (It doesn't do a very good job cleaning up after itself).
• Use clang as your compiler. Clang instances in general use less memory than Apple's GCC 4.2.
• Offload dependent targets which do not change often. Example: You will not need to rebuild third party libraries daily in most cases.

CPU

Xcode 4 uses a new (more accurate) completion parser.

• Pare down your include dependency graphs, and dependencies. This is quite easy to with Obj-C, since every Obj-C instance is a pointer. Example: remove loosely dependent framework includes from your headers.
• Separate your dependencies and modules properly. Develop libraries, but try to make them fairly small and be aware of the dependencies they will add. Example: I lead a project where a dev added a small feature (less than 1% of the app), but due to the number of dependencies it required (e.g. Three20 and then a few more), the binary size of the final executable doubled (and the build times went up, and the parser had a lot more work to do). Most of this was not needed for the feature - they just could not be stripped because they were objc symbols.
• Reduce translation counts, if possible.
• Optimize how you use prefix headers. You can share them, and you can create them for no good reason. This benefits the compiler more than the IDE.
• Minimize memory usage. GC work (including all NSObject allocs) consumes over 1/3 of the CPU usage in larger projects, but it still spends a lot of time collecting when its heap is huge.
• Use external text editors and VC clients. Pretty obvious because Xc4 displays the SBBOD rather often in large projects.
• Minimize language complexity. In order of complexity: C, ObjC, C++, ObjC++. The more complex the translations, the longer it will take to parse and compile your sources, especially when your dependencies are high. If you can easily set up language barriers in your dependencies, do so.
• You can disable code sense indexing via defaults (also reduces memory demands).

Hard Disk

This can be a speed/size balance.

• Buy a faster one (e.g. SSD).
• Cleanup and minimize your header dependencies.
• Use a RAM Disk, such as Make RAM Disk.
• Buy more memory. With the amount Xc4 consumes, it ends up swapping out to disk often in large projects.
• Optimize your builds to use pch files appropriately. This is not always the obvious direction: I have not used them for several years in large projects.
• Clear out the temp files Xcode and Instruments leave behind, they can be huge. In some cases, you can save them in customized locations. If they do consume tens of GB and your build dir is the same as your boot dir, then you will make your disk work a lot less by cleaning them up regularly.

Builds

• In Xc4, xcodebuild parallel to Xcode now doubles the work (separate build dirs by default). In Xc3, the default was to build to the same destination. Verify your settings - Xcode will do a ton of redundant building, if you allow it (e.g. one target per workspace/config, rather than Xc3's flat build model).

• Or just fill a drawer with quad core MacMinis and use that as a dedicated or distributed builder.

File Bugs

(self explanatory)

Answer 5

Another culprit for slowness is plugins. The Subversions plugin was absolutely killing my Xcode performance. I followed the instructions in this SO post to disable it. WHEW!