Jenkins Pipeline job can't find script due to

Question

I am writing a pipeline job that will call another script to execute. The Jenkinsfile and script exist in the same directory and yet the job fails to find the script to run.

This is the relevant bit of script;

stage ('Update') {
    try {
        dir('jenkins/pipeline/update-jenkins-plugins-ppln') {
            sh 'ls -l'
            sh 'update-plugins.sh'
        }
}

which returns the following error;

[update-jenkins-plugins-ppln] Running shell script
+ ls -l
total 8
-rw-r--r-- 1 jenkins jenkins 2441 Dec 20 09:34 Jenkinsfile
-rwxr-xr-x 1 jenkins jenkins  506 Dec 19 14:06 update-plugins.sh
[Pipeline] sh
[update-jenkins-plugins-ppln] Running shell script
+ update-plugins.sh
/var/lib/jenkins/workspace/update-jenkins-plugins-ppln/jenkins/pipeline/update-jenkins-plugins-ppln@tmp/durable-11cefdd0/script.sh: 2: /var/lib/jenkins/workspace/update-jenkins-plugins-ppln/jenkins/pipeline/update-jenkins-plugins-ppln@tmp/durable-11cefdd0/script.sh: update-plugins.sh: not found

As you can see, the pathing I'm using is correct because according to the ls the file I need update-plugins.sh is in the directory I've pathed to. For some reason though, when actually searching for the script Jenkins is adding @tmp/durable-8d48734f/script.sh onto the path.

Various troubleshooting:

• I read that you have to checkout the branch again even if you're already checking it out to get the Jenkinsfile, so I am.
• I have ssh'd into the Jenkins box to check and yes, the script is there.

Why is Jenkins adding the @tmp bit, and is there a way to prevent this behavior?

Answer 1

I was having the same issue. I think @Oren technically answered your question about why Jenkins creates this tmp space, but I can share some info about how I solved it.

Basically, my Jenkins host symlinks bin/sh to dash; not bash. So, using a POSIX-compliant shell script solved the issue for me.

For example, I was trying to use shopt -s extglob to do some pattern matching:

stage {
    def shellCommand = $/ "rm -rf ! (_data|_includes|_plugins|Gemfile|_config.yml|page-builder)"/$
    sh(returnStdout: true, script: shellCommand).trim()
}

Since dash doesn't support extglob, replacing that with a POSIX-compliant find command worked:

stage {
    sh('find . -regextype posix-extended -not -regex ".*includes.*|.*data.*|.*plugins.*|.*config.yml|.*Gemfile.*"')
} 

Answer 2

The @tmp folder is there for jenkins job and stages statistics (duration of stages, etc), you can delete it if you want to be sure. I assume that your kind of issue is related for wrong path, double check it.

Answer 3

I guess your pwd is not in PATH so you have to call it like this: sh './update-plugins.sh'

Answer 4

I am guessing this particular case has nothing to do with Jenkins pipelines, since it can be reproduced outside the Jenkins in the console. Once I got this issue due to DOS line-endings in my script which I ran in a pipeline's "sh()". Look at this example:

$ cat -v dos_formatted.sh
#! /bin/sh^M
pwd^M

$ cat script.sh
./dos_formatted.sh

$ sh -xe script.sh
+ ./dos_formatted.sh
script.sh: 1: script.sh: ./dos_formatted.sh: not found

It illustrates well that the "not found" message is misleading. The script is on the right place and the permissions are sufficient, but when you run it from another script it fails with such an error message.

The code of the Durable Task Plugin (https://github.com/jenkinsci/durable-task-plugin/blob/master/src/main/java/org/jenkinsci/plugins/durabletask/BourneShellScript.java) shows that the plugin runs the auto-generated script.sh as "sh -xe ...", therefore it's exactly our case.

The above described situation can happen if you "git clone" some (probably 3rd party) Git project and don't make sure that is was cloned with Unix-style LF.

Answer 5

I was facing a similar issue where though the shell script is present on path /var/lib/jenkins/workspace/New_commit_test/Fibonacci.sh, Jenkins build was getting failed with error message : "/tmp/jenkins6688065235543884785.sh: 3: /tmp/jenkins6688065235543884785.sh: /var/lib/jenkins/workspace/New_commit_test/Fibonacci.sh: not found"

so i changed my earlier command from : ${WORKSPACE}/Fibonacci.sh > New_commit_test.txt

to : (since i was using bash script here ) bash ${WORKSPACE}/Fibonacci.sh > New_commit_test.txt

which sorted issue here for me

Answer 6

Have you tried using the jenkins workspace environment variable WORKSPACE (absolute path of the workspace)? With that your line would look something like this:

sh '${WORKSPACE}/jenkins/pipeline/update-jenkins-plugins-ppln/update-plugins.sh'