What mutation-testing frameworks exist? [closed]

Question

In another question I asked if mutation-testing is useful in practice. As I didn't get any answers that satisfy me, I want to check the current tools myself. So I need an overview about current existing mutation-test-frameworks. Which of them are most usable and why?

• I program in Java, so I would prefer Java-tools, but I would risk a look at interesting frameworks for different languages.
• I want to integrate in an automatic build-process, so I would prefer tools that can be executed through command-line.

Answer 1

CREAM is a tool for C#/.Net http://galera.ii.pw.edu.pl/~adr/CREAM/index.php

Answer 2

You can try µJava. I haven't used it, but it looks like mutation testing might be an interesting way to evaluate test suites.

Answer 3

Jester does provide a file for the mutations and they are limited. To some degree, you can add your own mutations to the file.

I've experimented with Jumble and Jester and I found that Jumble provides more mutations and better documentation. Additionally, I've had quick responses from the project owners when I've emailed them. One drawback to Jumble is that it operates on the bytecode using BCEL. That presents something of a learning curve for many developers.

My company, State Farm, wrote an Ant task that we may contribute back to the Jumble project. Based on what I've read in their mailing lists, others are working on an Ant task for Jumble too.

I'm looking at Javalanche as well. I’ll be glad to share what I know when I’m done.

Answer 4

MμClipse only supports JUnit 3 and is no longer maintained. Jester as for it, is laborious and requires a complicated configuration; plus is not maintained anymore. The best tool I could find is Javalanche I had wrote a entire article about this !

Answer 5

Might be of some interest. Microsoft Research's: https://pex4fun.com/

Answer 6

For the .Net community, there is NesTer, but it has some serious limitations. E.g. only supports C# and NUnit.

Does not appear to be actively maintained either, but it might be a starting point.