Better way to integrate maven/qunit/phantomjs?

Question

I have been investigating the best way to do JS unit testing in our maven CI environment. What I currently have cobbled together is the following in my maven project:

• qunit resources (JS/CSS files)
• qunit test html files (one for each file under test) with html fixture if required
• index html file which references the test html files as an ordered list of hyperlinks
• PhantomJS runner file, which:
  • opens the index html file and parses out list of test files
  • opens each test file
  • takes a screenshot of the qunit test results for each file
  • If there are any failures, exit with a status of "1"
  • If there are no failures, exit with a status of "0"
• shell file which will exit with "0" if phantomjs isn't installed, will call the phantomjs tests if it is installed

• changes to pom.xml to run phantomjs tests during test phase of build:

  <plugins>
      <plugin>
          <groupId>org.codehaus.mojo</groupId>
          <artifactId>exec-maven-plugin</artifactId>
          <version>1.1</version>
          <executions>
              <execution>
                  <id>PhantomJS Unit Testing</id>
                  <phase>test</phase>
                  <goals>
                      <goal>exec</goal>
                  </goals>
              </execution>
          </executions>
          <configuration>
              <executable>${project.basedir}/src/main/webapp/unittest/phantomcheck</executable>
              <arguments>
                  <argument>${project.basedir}/src/main/webapp/unittest/qunit-runner.js</argument>
                  <argument>${project.basedir}/src/main/webapp/unittest/tests/index.html</argument>
                  <argument>${project.build.directory}/surefire-reports</argument>
              </arguments>
          </configuration>
      </plugin>
  </plugins>
  

So, this works nicely. It runs the qunit tests during builds on our dev and build machines (as long as PhantomJS is installed). The tests run in a headless browser environment with no restrictions on the qunit tests. Other maven/qunit integrations I've seen fall short due to running the tests in Rhino, or other JS environments which place restrictions on the type of tests we can write. Plus phantomjs gives us the ability to have the screenshots of the test runs, which are helpful in troubleshooting any failures.

The drawback to my approach is that a PhantomJS installation is required on the build/dev machine. I don't know how to bundle phantomJS into a dependency such that developers don't need to worry about installing PhantomJS. Can anyone give me a push in this direction? How do I get started?

Answer 1

We just check phantomJS.exe into source control. And then we are certain that the same version of phantomJS is being used on all machines.

Answer 2

The phantomjs-maven-plugin provides an install goal for installing phantomjs so you don't need it pre-installed. After it installs phantomjs it sets a property with the path to the executable that other plugins can then use. It also has an exec goal for executing phantomjs scripts. Full disclosure: I wrote the plugin.

Answer 3

This is an old question, but I thought I would link to a project of mine that uses PhantomJS and QUnit to run with TestNG:

The project is called qunit-testng. I also have a sample project that shows the library in use.

Here's a screenshot of test output:

Answer 4

Using Kyle's answer and another plugin I was able to get a full solution that doesn't require anything but maven preinstalled and sets up phantomjs and qunit to allow the running of tests. I started with a maven-grunt plugin (github.com/eirslett/frontend-maven-plugin) and followed the steps in this guide (http://blog.trifork.com/2014/10/07/setting-up-maven-to-use-gruntnodejs/) to get it set up. Then I tried to use qunit within maven and I ran into phantomjs trouble and came across this post and found out about Kyle's plugin (github.com/klieber/phantomjs-maven-plugin). I had to use a custom qunit source explained in this guide (http://techblog.dorogin.com/2013/08/issues-with-grunt-contrib-qunit.html). This allowed me to use kyles plugin to install phantomjs then link the binary through grunt options to the custom qunit. In the end my pom looked like:

`    <plugin>
        <groupId>com.github.klieber</groupId>
        <artifactId>phantomjs-maven-plugin</artifactId>
        <version>0.4</version>
        <executions>
          <execution>
            <phase>generate-resources</phase>
            <goals>
              <goal>install</goal>
            </goals>
          </execution>
        </executions>
        <configuration>
          <version>1.9.8</version>
        </configuration>
      </plugin>
      <plugin>
        <groupId>com.github.eirslett</groupId>
        <artifactId>frontend-maven-plugin</artifactId>
        <version>0.0.20</version>
        <executions>
          <execution>
            <id>install node and npm</id>
            <phase>generate-resources</phase>
            <goals>
              <goal>install-node-and-npm</goal>
            </goals>
            <configuration>
              <nodeVersion>v0.10.33</nodeVersion>
              <npmVersion>1.3.6</npmVersion>
            </configuration>
          </execution>
          <execution>
            <id>npm install</id>
            <phase>generate-resources</phase>
            <goals>
              <goal>npm</goal>
            </goals>
            <configuration>
              <arguments>install</arguments>
            </configuration>
          </execution>
          <execution>
            <id>grunt build</id>
            <phase>generate-resources</phase>
            <goals>
              <goal>grunt</goal>
            </goals>
            <configuration>
              <arguments>--phantomPath=${phantomjs.binary}</arguments>
            </configuration>
          </execution>
        </executions>
      </plugin>
`  

My Gruntfile.js looked like:

`    module.exports = function(grunt) {
      grunt.loadNpmTasks('grunt-croc-qunit');
      grunt.initConfig({
      pkg: grunt.file.readJSON('package.json'),
      qunit: {
        options: {
          'phantomPath': grunt.option('phantomPath')
        },
        all:['src/test/*.html']
      }
  });
  grunt.registerTask('default',['qunit']);
};`  

And my package.json looked like:

`    {
  "name":"reporting",
  "version":"0.0.1",
  "dependencies": {
    "grunt": "~0.4.5",
    "grunt-cli": "~0.1.13",
    "grunt-croc-qunit":"~0.3.0"
  },
  "devDependencies":{ }
}`  

Answer 5

Building on Kyle's answer I was able to find a solid solution to this issue. Thank you Kyle!

The solution is to use the phantomjs-maven-plugin Maven plugin. I add the plugin to my pom.xml like so (you will need to upgrade Maven to v3.1 or higher to use the plugin):

<plugin>
    <groupId>com.github.klieber</groupId>
    <artifactId>phantomjs-maven-plugin</artifactId>
    <version>0.4</version>
    <executions>
        <execution>
            <goals>
                <goal>install</goal>
                <goal>exec</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <version>1.9.7</version>
        <checkSystemPath>false</checkSystemPath>
        <script>src/test/qunit/run-qunit-testsuite.js</script>
        <arguments>
            <argument>src/test/qunit/testsuite.qunit.html</argument>
        </arguments>
    </configuration>
</plugin>

Important Caveat: in the pom.xml code above, make sure to use relative (not absolute) references to the files, as I've done. I wasted a few hours after using absolute references (starting at ${basedir}) only to find out that it does something strange to PhantomJS's working directory. Using relative references in your pom.xml will enable relative references inside your HTML file (which will maximize code portability).

In the plugin code above, I reference two files: run-qunit-testsuite.js and testsuite.qunit.html. The HTML file is just the QUnit file that executes all of your tests. The JS file is the driver for PhantomJS; it accepts one argument: the HTML QUnit test file to load.

To complete this solution, you can download sample driver and test files from GMarik's GitHub Gist page. You can and should adapt these files to your needs (although be aware that GMarik's page does not include an open source license, you will need to ask for permission for any copyright-infringing use).

When adding this plugin to your Maven code, after executing a Maven build you will see output like the following (adapted from GMarik's page):

[INFO] --- phantomjs-maven-plugin:0.4:exec (default) @ project.name ---
[INFO] Executing phantomjs command
'waitFor()' finished in 200ms.
Tests completed in 21 milliseconds.
5 tests of 5 passed, 0 failed.

If the tests pass then your build will pass. If the tests fail then your build will fail!