Command 'exit' doesn't work when some

Question

The 'exit' command doesn't work for my case, and I don't get it why:

ssh user@mysever <<'HEREDOC'
  echo "1. Running PM2 log..."
  pm2 log &
  echo "2. PM2 log is now in background"
  exit
  echo "3. Won't be here"
HEREDOC

echo "4. Out."

I can't terminate the ssh pipeline, even with the 'exit' command above.

I can see the first echo, second echo; no third echo of course; and stuck. I expected the behaviour that I can see the fourth echo. I have to press Ctrl+C, and I see the 4th echo after that.

Answer 1

Short answer: You should redirect the standard input, output, and error for the pm2 process:

pm2 log < /dev/null > /dev/null 2>&1 &

This will prevent the remote ssh server from holding the session open until pm2 exits.

Longer answer:

ssh user@mysever <<'HEREDOC'
    ...
    pm2 log &

When you run ssh in this fashion, the remote ssh server will launch a copy of the remote user's shell to handle the session. To relay input and output from the remote session, the remote ssh server will allocate either a TTY or a set of pipes. Then it sets the TTY or pipes as the standard input, output, and error for the shell process.

So, on the remote system, you have either a TTY or a set of pipes connecting the remote shell process to the SSH server process. Any commands invoked by the shell will inherit the TTY or set of pipes as the command's standard input etc. (unless you use shell features to redirect the standard file handles).

You might think that the ssh server will terminate the session when the remote shell process exits. But that's not what happens. The ssh server terminates the session when it reads an end-of-file condition on the TTY or the pipe connected to the standard output for the session.

In your case, you're invoking this pm2 command on the remote system, and it's inheriting the remote session's standard output. As long as this program is running, the remote ssh server won't get an EOF on the standard output pipe and it won't terminate the session.

The fix is to redirect input for the pm2 process so it doesn't inherit the standard handles that are connected to the ssh server:

pm2 log < /dev/null > /dev/null 2>&1 &

You could redirect to a file instead of /dev/null if you want to capture the output. I think only the standard output redirection is strictly necessary, but you should redirect standard input and error as well just to be sure.

Answer 2

Found the reason why, ssh tunnel wait until termination of stdout (EOF), and no processes reading stdin.

ssh user@myserver <<HEREDOC

  #1. terminate stdout by piping to file
  #2. send to background so it won't read stdin
  pm2 log >log.txt 2>&1 &

  #when going with Bash pipe, end pipe by directing to file
  pm2 log | grep error >log.txt 2>&1 &
HEREDOC