starting remote (SSH) Java application with shell

Question

I've seen similar questions, all addressed issues were already solved/not applicable.

I have a bash script in the remote machine that starts a Java application. The relevant lines would be:

#!/usr/bin/env bash
...
java -cp /full/path/to/my.jar com.whatever.hi.wassup.MainClassThing \
    --arg-1 /full/path/to/relevant_dir --arg-2 /full/path/to/another_dir &
...
echo started my app
exit 0

I've done a bunch of tests with small scripts and have no issues. I use the same approach to start another Java application but have no issues (it's a server application with --daemon parameters if it matters).

On the local machine i try calling the remote script inside a local bash script as such:

#!/usr/bin/env bash
...
ssh remote_host myRemoteScript

All my tests and the other Java application return the local prompt. The problem is that this one, however, does indeed start the remote Java application and goes all the way to the last line, outputting "started my app" but the script stalls there, i have no local prompt unless i Ctrl+C it.

Using ssh -v on the working Java app i get this:

debug1: channel 0: free: client-session, nchannels 1
Transferred: sent 2992, received 7060 bytes, in 24.4 seconds
Bytes per second: sent 122.6, received 289.2
debug1: Exit status 0

Yet on the non-working app i get this:

debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype [e-mail] reply 0
"started my app"

But no local prompt. I tried running the same command with nohup and disown, same results. As seen the command has & at the end. Commenting out only that line in the remote script works without an issue (and does nothing useful), so i assume this is something to do with the Java application itself - but isn't process control up to the shell? Running myRemoteScript in the remote machine works as expected as well.

It's a passwordless login. Both machines are RHEL 7.2.

EDIT

Also tried:

• ssh remote_host "myRemoteScript"
• ssh -T remote_host myRemoteScript
• ssh remote_host "bash -c myRemoteScript"
• ssh remote_host "myRemoteScript; exit"

Answer 1

You can redirect the standard input, output, and error for the java process so that its standard input, output, and error are't connected to the remote SSH server:

java ... < /dev/null > /dev/null 2>&1 &

That should permit the remote server to terminate the SSH session immediately.

When you ask the remote SSH server to invoke a command, and you don't request a TTY for that command, the server will create a set of pipes for the standard input, output, and error for the child process. The current behavior of the OpenSSH server seems to be not to close the session when the remote command exits. Instead, the server closes the session when it detects an end-of-file condition on the pipe handling standard output from the remote process.

The java process launched by your remote command is inheriting the pipes to the SSH server as its standard input etc. The SSH server doesn't get an EOF indication on these pipes because there's still a process with access to the pipes.

You can test this by running a command like like:

ssh localhost 'sleep 10 < /dev/null > /dev/null &'

If ssh exits right away without waiting for the sleep to complete, that means you've gotten the remote SSH server not to wait on the sleep.

My own tests suggest that the SSH server is looking at the pipe receiving standard output from the child process, so that's all that you have to redirect. But you should consider redirecting all three pipes--standard input, output, and error--in case the behavior of the SSH server changes in the future.