Is it possible for Ant to lists failed tests at th

Question

The following snippet, from another thread, works to print a message and fail after all unit tests have been run :

<fail if="junit.failed" message="Oh no ! There were some failed unit tests :( "/>

However --- I don't see a how can I also record and print the NAMES of the failed tests in junit/ant, after they have all run. Any thoughts on this ?

I believe others would find such function extremely important, so I'm assuming a simple solution exists : its quite tedious to look through hundreds of failed tests for the offenders.

Answer 1

I'm sure that many of you have no desire to build a custom formatter or whatever. I don't either. I discovered that, with the following configuration, test successes are printed to stdout while failures are printed to stderr:

  <target name="test"
          depends="compile-tests"
          description="runs the unit tests">
    <junit failureproperty="hasFailingTests"
           printsummary="on"
           showoutput="true">
      <formatter type="plain" usefile="false" />
      <batchtest>
        <fileset dir="${test.dir}">
          <include name="**/*Test.java" />
          <exclude name="**/Abstract*Test.java" />
        </fileset>
      </batchtest>
      <classpath refid="tests.classpath"></classpath>
    </junit>
    <fail if="hasFailingTests" />
  </target>

That means that running the following command:

ant test > /dev/null

will only show stderr with the test failures to the console, making it much easier to see what actually failed.

Answer 2

Yes it is. Try using the junitreport task.

e.g.

Try this attribute on your junit task:

printsummary="yes" on junit task

Change your formatter to:

<formatter type="xml" />

and then create the reports with a target that calls this:

<junitreport>
<fileset dir="${testReport.dir}/tmp">
      <include name="*.xml" />
</fileset>
<report format="frames" styledir="${testReportXslt.dir}" todir="${finalReport.dir}/html" />
</junitreport>

For output :

    <concat>
        <fileset dir="${finalReport.dir}/html" includes="*.html"/>
        <filterchain>
            <linecontainsregexp>
                <regexp pattern='some pattern' />
            </linecontainsregexp>
        </filterchain>
    </concat>

Answer 3

I think I finally have a complete answer to this question : Thanks to FailedDev's insights .

First, Make sure you ${reports.dir} variable, specifying the directory for the reports :

    <property name="reports.dir" value="reports" />

Then , when we begin coding the test instructions junit :

<target name="test" depends="compile">

Then, make necessary directories for the reports :

    <mkdir dir="${reports.dir}" />
    <mkdir dir="${reports.dir}/tmp" />
    <mkdir dir="${reports.dir}/style" />
    <mkdir dir="${reports.dir}/final" />

Since we have a report scanner, we can set haltonfailure to no, and fail after (scroll down).

    <junit printsummary="yes" failureproperty="junit.failed" haltonfailure="no" fork="yes" forkmode="once">
        <jvmarg value="-Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl" />
        <classpath refid="common.classpath" />
        <classpath>
            <pathelement location="${build.dir}" />
            <pathelement location="${src.dir}" />
        </classpath>
        <formatter type="xml" />
        <batchtest todir="${reports.dir}">
            <fileset dir="${test.dir}">
                <include name="**/Test*.java" />
                <exclude name="**/AllTests.java" />
                <exclude name="**/*.properties" />
                <exclude name="**/*.xml" />
            </fileset>
        </batchtest>
    </junit> 

Now, here is where the advice of other questions comes in : run the junit report.

    <!-- Capture all failures, simple debugging statements. -->
    <junitreport>
        <fileset dir="${reports.dir}/tmp">
            <include name="*.xml" />
        </fileset>
        <report todir="${reports.dir}/final" />
    </junitreport>

And finally, we can grep the xml files, directly, for errors :

    <!-- This could be its own task, i.e., a java class which directly processed junit test data. -->
    <echo message="Now checking xml test results for errors"    />
        <exec executable="grep" error="/dev/null">
            <arg value="-r" />
            <arg value="-m" />
            <arg value="1" />
            <arg value="-rl" />
            <arg value="errors=\&quot;[1-9]\&quot;" /> 
            <arg value="${reports.dir}" />
        </exec>

Now, since we are'nt failing early (rather we are running the whole build, so we can see which tests failed , if any) we still have to notify the builder that we have failed... This is done via the fail-if syntax :

    <fail if="junit.failed" message="FAILING - unit tests failed." />
    <!-- Now, we check if there were failures, and output the results --> 
</target>

Removing my comments, this code-block should work perfectly if you paste it into your build.