QA to dev ratio [closed]

Question

We have a pretty complex Java system which includes a few back-end tiers including a database and a proprietary Swing front-end. There are back-end APIs to which external parties can attach that mimic our front-end. There are approximately 5 silos within our org that share this system. In total there are around 15 developers maintaining this system.

Is there a rule of thumb on what the size our QA team should be?

Edit: To add a bit of context based on questions raised in the responses thus far:

1. We have about four major releases a year, and a bunch of minor ones in between.
2. Our platform has money changing hands, so there are calcs that mean a lot to our customers.
3. We do use a formal bug tacking system.
4. We don't utilize TDD.
5. We use tools like cruisecontrol for continuous integration.

Answer 1

No, there is no rule of thumb. Every organization is different, as every organization has a completely different set of requirements, needs, projects, etc. You are only going to be able to find what works for your organization.

Answer 2

A lot of it depends on what the system does and what the consequences of defects is? Are you writing financial software? Or social networking? The likely cost of one failing are greater than the other so the QA effort varies. Similarly if it's a product (with multiple installed user bases which might be harder to patch) you need it to be more stable than if it's internal.

You also need to ask are you expecting them to do basic system testing or volume and load testing too. It sounds like you're looking at a combination of UI testing and API testing.

Tgive you a starting point, assuming that you're talking a "normal" type of system and we're talking basic system testing, the metric I'd go with is that 33% of all effort should be test effort. In theory this gives you a 1 to 2 ratio but assume that your developers are unit testing perhaps stretch it to 1 to 3. Certainly I'm currently running with 1 to 4 and that's not enough (I'll be rectifying it as soon as the business allows me to).

But you also want to consider how mature your software development processes are and what your specifications are like. As well as having the right number of testers you need to give them the right information to test against - if they don't have that then they're not going to be able to do a job and it's arguably wasted money.

One other option to throw in there - it sounds like the product has been under development for a while. As well as a core of regular testers, you might want to bulk up for an initial major test using short term contractors. If you're only just bringing test resource in you don't want to start them with a major backlog.

Answer 3

I don't know if his point of view has changed since this article 9 years ago, but Joel would recommend 1 full time QA person per 2 full time devs. I'd say, with a system that's constantly evolving, with regular updates, this ratio is pretty good.

Then again, if you're system's not changing that often, rarely updating, why would you have many devs to begin with? I've mamaged to convince myself that 1:2 is the perfect ratio in most all situations. :)

Answer 4

2 test engineers per 3 developers is a decent start. If many test cases involve stuff that can't be automated (physically plugging/unplugging stuff, ocular verification of UI rendering) add 1 tester per 2 test engineers.

Answer 5

Start off with a 1 QA to 5 or 6 developers. Then start tweaking it depending on how much work you think the QAs are having to do or having nothing to do.

In truth, that's just based on my own experience because there's too many factors involved. How developed or stable is your codebase? How big is it? How comprehensive does the testing need to be? What sort of testing and analysis tools are involved? How often are milestone releases? What sort of development process do you have? How much manual testing and how much automated testing?

There's a lot more questions. So just start with an arbitrary number and go from there.

Answer 6

It likely depends on how involved the code base is (whether there are lots of integrating modules) and how involved the user experience is (if there are lots of visual elements and input controls to test).

If it's a pretty complex system, you may want to consider 2-3 QA engineers for every 5 developers. If however, the features don't really change much or are less involved, I would think you could get away with just 1 QA for every 5 developers (maybe even for every 10 developers).