Jenkins Slave Issue - invalid stream header: 099EA

问题:

Jenkins 2.7.4 was installed in the RedHat Server and Linux Slaves is configured by Selecting "Launch agent via execution of command on master" option. We created a Shell script and it works fine in the Jenkins version 2.7.4.

Now we upgraded the Jenkins to 2.121.1. Now the same script throws an error

<===[JENKINS REMOTING CAPACITY]===>Exception in thread "main" java.io.StreamCorruptedException: invalid stream header: 099EACED at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:808) at java.io.ObjectInputStream.(ObjectInputStream.java:301) at hudson.remoting.ObjectInputStreamEx.(ObjectInputStreamEx.java:48) at hudson.remoting.ChannelBuilder.makeTransport(ChannelBuilder.java:478) at hudson.remoting.ChannelBuilder.negotiate(ChannelBuilder.java:433) at hudson.remoting.ChannelBuilder.build(ChannelBuilder.java:354) at hudson.remoting.Launcher.main(Launcher.java:743) at hudson.remoting.Launcher.runWithStdinStdout(Launcher.java:691) at hudson.remoting.Launcher.run(Launcher.java:373) at hudson.remoting.Launcher.main(Launcher.java:283) ERROR: Connection terminated ERROR: Unexpected error in launching an agent. This is probably a bug in Jenkins java.io.IOException: Unexpected EOF at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:99) at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63) ERROR: Process terminated with exit code 1 java.io.IOException: Unexpected EOF at hudson.remoting.ChunkedInputStream.readUntilBreak(ChunkedInputStream.java:99) at hudson.remoting.ChunkedCommandTransport.readBlock(ChunkedCommandTransport.java:39) at hudson.remoting.AbstractSynchronousByteArrayCommandTransport.read(AbstractSynchronousByteArrayCommandTransport.java:35) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:63) Also: hudson.remoting.Channel$CallSiteStackTrace: Remote call to rtt-ci-euhrhd0036vdeas at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1741) at hudson.remoting.Request.call(Request.java:202) at hudson.remoting.Channel.call(Channel.java:954) at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:549) at hudson.slaves.SlaveComputer.setChannel(SlaveComputer.java:416) at hudson.slaves.CommandLauncher.launch(CommandLauncher.java:153) at hudson.slaves.SlaveComputer$1.call(SlaveComputer.java:288) at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:46) at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:71) at java.util.concurrent.FutureTask.run(Unknown Source) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.lang.Thread.run(Unknown Source) Caused: hudson.remoting.RequestAbortedException at hudson.remoting.Request.abort(Request.java:340) at hudson.remoting.Channel.terminate(Channel.java:1038) at hudson.remoting.SynchronousCommandTransport$ReaderThread.run(SynchronousCommandTransport.java:96)

If I disable the SSHD port in Manage Jenkins -> Configure Security, Then I can able to launch my slave. But In my script, I will trigger one job using Jenkins-cli.jar to copy the binaries from master to slaves. java -jar jenkins-cli.jar -s http://localhost:8080 --ssh -user username -i ~/.ssh/id_rsa build RTT/RTT-CI-Tools/RTT-CI-Tools-Distribute -s -p SLAVE_REGEX=slave name I'm getting a message as

"WARNING: No header 'X-SSH-Endpoint' returned by Jenkins" 

and build is not getting triggered. I have also tried by replacing -ssh to -http in jenkins-cli.jar command,

java -jar jenkins-cli.jar -s http://localhost:8080 -http -auth username:60b3450a883a2519592af84cdcd0d224 build $CI_JOB -s -p SLAVE_REGEX=$SLAVEHOST

It triggers the job. Again unable to launch the slave machine,

<===[JENKINS REMOTING CAPACITY]===>Exception in thread "main" java.io.StreamCorruptedException: invalid stream header: 099FACED at java.io.ObjectInputStream.readStreamHeader(ObjectInputStream.java:808) at java.io.ObjectInputStream.(ObjectInputStream.java:301) at hudson.remoting.ObjectInputStreamEx.(ObjectInputStreamEx.java:48) at hudson.remoting.ChannelBuilder.makeTransport(ChannelBuilder.java:478) at hudson.remoting.ChannelBuilder.negotiate(ChannelBuilder.java:433) at hudson.remoting.ChannelBuilder.build(ChannelBuilder.java:354) at hudson.remoting.Launcher.main(Launcher.java:743) at hudson.remoting.Launcher.runWithStdinStdout(Launcher.java:691) at hudson.remoting.Launcher.run(Launcher.java:373) at hudson.remoting.Launcher.main(Launcher.java:283) usage: ssh [-1246AaCfgKkMNnqsTtVvXxYy] [-b bind_address] [-c cipher_spec] [-D [bind_address:]port] [-e escape_char] [-F configfile] [-i identity_file] [-L [bind_address:]port:host:hostport] [-l login_name] [-m mac_spec] [-O ctl_cmd] [-o option] [-p port] [-R [bind_address:]port:host:hostport] [-S ctl_path] [-W host:port] [-w local_tun[:remote_tun]] [user@]hostname [command] ERROR: Unexpected error in launching an agent. This is probably a bug in Jenkins ERROR: Connection terminated java.io.IOException: Unexpected EOF

How to fix this issue

回答1:

Unaltered stdin, unaltered stdout

I believe something in your script is tampering with stdin.

Your script should pass the entire, unaltered stdin stream to the Jenkins agent process.

A universal solution

OP's command to establish a Jenkins session is different from mine, but regardless, you should split your launch script into 3 major parts:

Set up: no tampering with stdin or stdout in this part.

Establish Jenkins session: java -jar jenkins-cli.jar ...

Tear down: no tampering with stdin or stdout in this part.

#!/bin/bash

function set_up {
    # your set-up code here
}

function tear_down {
    # your tear-down code here
}

function main {
    # set-up (no stdin, no stdout)
    set_up "$@" < /dev/null > /dev/null || exit $?
    # establish Jenkins session
    java -jar jenkins-cli.jar -blah -blah -blah
    # tear-down (no stdin, no stdout)
    tear_down "$@" < /dev/null > /dev/null || exit $?
}
main "$@"

---

But... why?

The job of your launch script is to establish an untampered communication channel (via stdin and stdout) between the master and the build agent.

              +------------+
"Hello Agent" |            |
   _ _    ----+            +----
    v               Hello Agent  ->
          ----+            +----
              |            |
              |            |  "Hello Master"
          ----+            +----   _ _
       <-  Hello Master             v
          ----+            +----
              |            |
              +------------+
                  launch
                  script

If this communication channel is tampered, Jenkins won't work.

              +------------+
"Hello Agent" |            |
   _ _    ----+            +-----------
    v            Hel PLZ SEND HELP!! t  ->
          ----+            +-----------
              |            |
              |            |  
          ----+            +----   | |
                                    ^
          ----+            +----
              |            |
              +------------+
                  launch
                  script

Some Unix commands may "swallow" the stdin of your launch script if you do not pipe anything into that command, and therefore "corrupt" the communication channel. Consider the following script.

#!/bin/bash

function keep_stdin_intact {
    printf 'I do not consume any stdin, ' >&2
    echo 'and I do not alter the original stdout.' >&2
}

function swallow_stdin {
    echo 'I swallow stdin. Did you see any hexdump below?' >&2
    read yn  # read consumed some stdin
}

echo 'yes' | { keep_stdin_intact; cat -; } | xxd
echo 'yes' | { swallow_stdin; cat -; } | xxd

echo "no you can't now :P" | { swallow_stdin < /dev/null; cat -; } | xxd

• The first yes got piped out and hex-dumped, because keep_stdin_intact did not tamper with stdin, in this case, the "yes" stream.

• The second yes was gone, because swallow_stdin consumed it, so cat has nothing to cat and xdd has nothing to read.

• By piping /dev/null to an stdin-swallowing command, we protected our own stdin.

What's up with ssh?

ssh is one of the evil commands that swallow your stdin.

Let's say you want to remove some files on the build agent before agent.jar is run. Without the boilerplate, you may be tempted to write:

ssh $OPTIONS "$remote" 'sudo rm -rf /var/log/nginx/*'
ssh $OPTIONS "$remote" 'cd $HOME && java -jar agent.jar'

^ But this is wrong! The first ssh command will swallow your stdin, and the Jenkins session will have nothing to read.

The first ssh must be "silenced." Pass /dev/null as its stdin.

ssh $OPTIONS "$remote" 'sudo rm -rf /var/log/nginx/*' < /dev/null
ssh $OPTIONS "$remote" 'cd $HOME && java -jar agent.jar'