How to get Jenkins working with binaries from COMP

Question

I'm working on a project and want to use Jenkins for it. The config should be done by a simple build.xml with a fiew targets:

<?xml version="1.0" encoding="UTF-8"?>
<project name="MyProject" default="full-build">
    <property name="phpcs"   value="phpcs"/>
    <property name="phpdox"  value="phpdox"/>
    <target name="full-build" depends="lint, phpdox" description="Performs static analysis, runs the tests, and generates project documentation"/>
    <target name="lint" description="Perform syntax check of sourcecode files">
        <apply executable="php" taskname="lint" failonerror="true">
            <arg value="-l" />
            <fileset dir="${basedir}/module/">
                <include name="**/*.php" />
            </fileset>
            <fileset dir="${basedir}/phpunit/tests">
                <include name="**/*.php" />
            </fileset>
        </apply>
    </target>
    <target name="phpdox" description="Generate project documentation using phpDox">
        <exec executable="${phpdox}" dir="${basedir}/ci/phpdox" taskname="phpdox"/>
    </target>
</project>

The PHP Lint (php -l) job works perfectly:

Console Output

Started by user anonymous
Building in workspace D:\Data\myproject
 > git.exe rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
 > git.exe config remote.origin.url https://github.com/MyAccount/MyProject.git # timeout=10
Fetching upstream changes from https://github.com/MyAccount/MyProject.git
 > git.exe --version # timeout=10
using .gitcredentials to set credentials
 > git.exe config --local credential.username myusername # timeout=10
 > git.exe config --local credential.helper store --file=\"C:\Windows\TEMP\git8298647783094567497.credentials\" # timeout=10
Setting http proxy: my.proxy.tld:8080
 > git.exe -c core.askpass=true fetch --tags --progress https://github.com/MyAccount/MyProject.git +refs/heads/*:refs/remotes/origin/*
 > git.exe config --local --remove-section credential # timeout=10
 > git.exe rev-parse "refs/remotes/origin/master^{commit}" # timeout=10
 > git.exe rev-parse "refs/remotes/origin/origin/master^{commit}" # timeout=10
Checking out Revision 54af2180160f47d518c42f58f56cba175ca2ee39 (refs/remotes/origin/master)
 > git.exe config core.sparsecheckout # timeout=10
 > git.exe checkout -f 54af2180160f47d518c42f58f56cba175ca2ee39
 > git.exe rev-list 54af2180160f47d518c42f58f56cba175ca2ee39 # timeout=10
[myproject] $ cmd.exe /C '"ant.bat && exit %%ERRORLEVEL%%"'
Buildfile: D:\Data\myproject\build.xml

lint:
     [lint] No syntax errors detected in D:\Data\myproject\module\Application\Module.php
     ...
     [lint] No syntax errors detected in D:\Data\myproject\phpunit\tests\Application\DummyTest.php

But the executing of PHPDox is failing:

phpdox:

BUILD FAILED
D:\Data\myproject\build.xml:18: Execute failed: java.io.IOException: Cannot run program "phpdox" (in directory "D:\Data\myproject\ci\phpdox"): CreateProcess error=2, The system cannot find the file specified
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
    at java.lang.Runtime.exec(Runtime.java:620)
    at org.apache.tools.ant.taskdefs.launcher.Java13CommandLauncher.exec(Java13CommandLauncher.java:58)
    at org.apache.tools.ant.taskdefs.Execute.launch(Execute.java:428)
    at org.apache.tools.ant.taskdefs.Execute.execute(Execute.java:442)
    at org.apache.tools.ant.taskdefs.ExecTask.runExecute(ExecTask.java:629)
    at org.apache.tools.ant.taskdefs.ExecTask.runExec(ExecTask.java:670)
    at org.apache.tools.ant.taskdefs.ExecTask.execute(ExecTask.java:496)
    at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:293)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at org.apache.tools.ant.dispatch.DispatchUtils.execute(DispatchUtils.java:106)
    at org.apache.tools.ant.Task.perform(Task.java:348)
    at org.apache.tools.ant.Target.execute(Target.java:435)
    at org.apache.tools.ant.Target.performTasks(Target.java:456)
    at org.apache.tools.ant.Project.executeSortedTargets(Project.java:1405)
    at org.apache.tools.ant.Project.executeTarget(Project.java:1376)
    at org.apache.tools.ant.helper.DefaultExecutor.executeTargets(DefaultExecutor.java:41)
    at org.apache.tools.ant.Project.executeTargets(Project.java:1260)
    at org.apache.tools.ant.Main.runBuild(Main.java:853)
    at org.apache.tools.ant.Main.startAnt(Main.java:235)
    at org.apache.tools.ant.launch.Launcher.run(Launcher.java:285)
    at org.apache.tools.ant.launch.Launcher.main(Launcher.java:112)
Caused by: java.io.IOException: CreateProcess error=2, The system cannot find the file specified
    at java.lang.ProcessImpl.create(Native Method)
    at java.lang.ProcessImpl.<init>(ProcessImpl.java:386)
    at java.lang.ProcessImpl.start(ProcessImpl.java:137)
    at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
    ... 24 more

Total time: 8 seconds
Build step 'Invoke Ant' marked build as failure
Finished: FAILURE

The same problem I have with other Composer packages, that are installed globally and used from COMPOSER_HOME/bin (e.g. PHP_Codesniffer).

I've tried to define the commands in another way (like <exec executable="cmd"><arg line="/c phpdox ..." /></exec>), but it's also not working.

The direct executing of the build.xml with ant works.

The environment variable for the Composer is set correctly -- I can call the tools via the CLI (CMD or GitBash/MinGW). So the problem must have been caused by something else. What can it be?

How to get Jenkins working with packages installed globally via Composer?

Answer 1

Linux (in my case Ubuntu 14.04 Server)

Setting the environment variables in the Jenkins configuration (Manage Jenkins -> Configure System -> Global properties -> Environment variables: name=PATH, value=$PATH:$COMPOSER_HOME/vendor/bin/) resolved the issue.

$ echo $COMPOSER_HOME
/usr/share/.composer

Note, that the $COMPOSER_HOME should/may not be set to subfolder of a (root) user (e.g. /root/.composer), since it can cause permission issues:

Execute failed: java.io.IOException: Cannot run program "phpcs": error=13, Permission denied

---

Windows (in my case Windows Server 2008 r2)

Additionally to setting of the Composer path some further steps were/are needed:

First of I got the error

Execute failed: java.io.IOException: Cannot run program "phpcs": error=2, The system cannot find file specified

On a forum I read, that the target needs to get the path to the executable, like this:

<target name="phpcs">
    <exec executable="C:\path\to\Composer\vendor\bin\phpcs" taskname="phpcs">
        <arg value="--standard=PSR2" />
        <arg value="--extensions=php" />
        <arg value="--ignore=autoload.php" />
        <arg path="${basedir}/module/" />
    </exec>
</target>

But it didn't work and only causes the next error:

D:\path\to\build.xml:34: Execute failed: java.io.IOException: Cannot run program "C:\path\to\Composer\vendor\bin\phpcs": CreateProcess error=193, %1 is not a valid Win32 application

It's not a Jenkins, but an Ant issue. The cause and the solution are described on the Ant docu page for Exec:

Windows Users

The <exec> task delegates to Runtime.exec which in turn apparently calls ::CreateProcess. It is the latter Win32 function that defines the exact semantics of the call. In particular, if you do not put a file extension on the executable, only ".EXE" files are looked for, not ".COM", ".CMD" or other file types listed in the environment variable PATHEXT. That is only used by the shell.

Note that .bat files cannot in general by executed directly. One normally needs to execute the command shell executable cmd using the /c switch.

...

So in my case the working target specification looks as follows:

<target name="phpcs">
    <exec executable="cmd" taskname="phpcs">
        <arg value="/c" />
        <arg value="C:\path\to\Composer\vendor\bin\phpcs" />
        <arg value="--standard=PSR2" />
        <arg value="--extensions=php" />
        <arg value="--ignore=autoload.php" />
        <arg path="${basedir}/module/" />
    </exec>
</target>

From now the build was passing through. But there was still an issue with phpcs:

phpcs:
    [phpcs] 'php' is not recognized as an internal or external command,
    [phpcs] operable program or batch file.
    [phpcs] Result: 1

This is easily fixed by adding PHP to the Path environment variable in Jenkins (Manage Jenkins -> Configure System -> Global properties -> Environment variables):

That's it. :)