What’s the ROI of Continuous Integration?

Question

Currently, our organization does not practice Continuous Integration.

In order for us to get an CI server up and running, I will need to produce a document demonstrating the return on the investment.

Aside from cost savings by finding and fixing bugs early, I'm curious about other benefits/savings that I could stick into this document.

Answer 1

Every successful build is a release candidate - so you can deliver updates and bug fixes much faster.

If part of your build process generates an installer, this allows a fast deployment cycle as well.

Answer 2

There are free CI servers available, and free build tools like NAnt. You can implement it on your dev box to discover the benefits.

If you're using source control, and a bug-tracking system, I imagine that consistently being the first to report bugs (within minutes after every check-in) will be pretty compelling. Add to that the decrease in your own bug-rate, and you'll probably have a sale.

Answer 3

Our biggest gain, is from always having a nightly build for QA. Under our old system each product, at least once a week, would find out at 2AM that someone had checked in bad code. This caused no nightly build for QA to test with, the remedy was to send release engineering an email. They would diagnose the problem and contact a dev. Sometimes it took as long as noon before QA actually had something to work with. Now, in addition to having a good installer every single night, we actually install it on VM's of all the different supported configurations everynight. So now when QA comes in, they can start testing within a few minutes. Now when you think of the old way, QA came in grabbed the installer, fired up all the vms, installed it, then started testing. We save QA probably 15 minutes per configuration to test on, per QA person.

Answer 4

My #1 reason for liking CI is that it helps prevent developers from checking in broken code which can sometimes cripple an entire team. Imagine if I make a significant check-in involving some db schema changes right before I leave for vacation. Sure, everything works fine on my dev box, but I forget to check-in the db schema changescript which may or may not be trivial. Well, now there are complex changes referring to new/changed fields in the database but nobody who is in the office the next day actually has that new schema, so now the entire team is down while somebody looks into reproducing the work you already did and just forgot to check in.

And yes, I used a particularly nasty example with db changes but it could be anything, really. Perhaps a partial check-in with some emailing code that then causes all of your devs to spam your actual end-users? You name it...

So in my opinion, avoiding a single one of these situations will make the ROI of such an endeavor pay off VERY quickly.

Answer 5

From Wikipedia:

• when unit tests fail or a bug emerges, developers might revert the codebase back to a bug-free state, without wasting time debugging
• developers detect and fix integration problems continuously - avoiding last-minute chaos at release dates, (when everyone tries to check in their slightly incompatible versions).
• early warning of broken/incompatible code
• early warning of conflicting changes
• immediate unit testing of all changes
• constant availability of a "current" build for testing, demo, or release purposes
• immediate feedback to developers on the quality, functionality, or system-wide impact of code they are writing
• frequent code check-in pushes developers to create modular, less complex code
• metrics generated from automated testing and CI (such as metrics for code coverage, code complexity, and features complete) focus developers on developing functional, quality code, and help develop momentum in a team
• well-developed test-suite required for best utility

We use CI (Two builds a day) and it saves us a lot of time keeping working code available for test and release.

From a developer point of view CI can be intimidating when Automatic Build Result, sent by email to all the world (developers, project managers, etc. etc.), says: "Error in loading DLL Build of 'XYZ.dll' failed." and you are Mr. XYZ and they know who you are :)!

Answer 6

CI assists with issue discovery. Measure the amount of time currently that it takes to discover broken builds or major bugs in the code. Multiply that by the cost to the company for each developer using that code during that time frame. Multiply that by the number of times breakages occur during the year.

There's your number.