Making Jenkins build fail when Selenium tests fail

Question

I have setup a Jenkins server for running Selenium tests. The build script is written in Ant (v 1.7.1 on CentOS 6.3) and the tests run on Sauce Labs. I execute the build via shell:

ant -buildfile build.xml

The problem is when a test fails, Jenkins marks it a success. I can avert this by setting haltonfailure to true, see:

<junit printsummary="yes" fork="false" haltonfailure="yes">
    <classpath refid="run.cp" />
    <formatter type="xml" usefile="true"    />
    <batchtest fork="no" todir="test-output/xml">
        <fileset dir="src/">
            <exclude name="**/*LocatorUtils*.java"/>
            <exclude name="**/*TestBase*.java"/>
            <exclude name="**/*TestUtils*.java"/>
            <exclude name="**/*Config*.properties"/>
        </fileset>
    </batchtest>
</junit>

However, this is not ideal because Jenkins will terminate the build on the first failure it encounters. Is it possible to check for failures once a build has completed and then mark it as failed?

Answer 1

When the build fails, testng-failed.xml is created. You can find this xml file inside workspace/Test Report/BuildDirectory/

Here BuildDirectory is created each time. You can write a powershell script to check whether testng-failed.xml is created. If the build is success then this file will not be created.

Answer 2

On the junit task, use the failureproperty attribute. And then call the fail task with an unless attribute.

Further reading:

• http://ant.apache.org/manual/Tasks/junit.html
• http://ant.apache.org/manual/Tasks/fail.html